@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

main

feat: implement the min_tokens sampling parameter (#3124)

Travis Johnson committed over 2 years ago

[Core] Adding token ranks along with logprobs (#3516)

Swapnil Parekh committed over 2 years ago

[CI] Try introducing isort. (#3495)

SangBin Cho committed over 2 years ago

[Core] Refactor Attention Take 2 (#3462)

Woosuk Kwon committed over 2 years ago

[Model] Add starcoder2 awq support (#3569)

少年 committed over 2 years ago

[BugFix] tensor.get_device() -> tensor.device (#3604)

Kunshang Ji committed over 2 years ago

[CI/Build] fix flaky test (#3602)

youkaichao committed over 2 years ago

[CI] typo fix: is_hip --> is_hip() (#3595)

youkaichao committed over 2 years ago

[BugFix] 1D query fix for MoE models (#3597)

Nick Hill committed over 2 years ago

[BugFix] Fix Falcon tied embeddings (#3590)

Woosuk Kwon committed over 2 years ago

[Misc] Bump transformers version (#3592)

Roger Wang committed over 2 years ago

[Misc] Fix BLOOM copyright notice (#3591)

Woosuk Kwon committed over 2 years ago

[Bugfix] use SoftLockFile instead of LockFile (#3578)

kota-iizuka committed over 2 years ago

[Hardware][Neuron] Refactor neuron support (#3471)

Zhuohan Li committed over 2 years ago

← Newer Page 1073 of 1120 Older →