A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Bugfix] Fix CustomAllreduce nvlink topology detection (#3974)
Adam Tilghman committed over 2 years ago
[CI/CD] add neuron docker and ci test scripts (#3571)
Liangfu Chen committed over 2 years ago
[Misc] Bump transformers to latest version (#4176)
Nick Hill committed over 2 years ago
[Bugfix] Support logprobs when using guided_json and other constrained decoding fields (#4149)
James Whedbee committed over 2 years ago
[Docs] document that Meta Llama 3 is supported (#4175)
Simon Mo committed over 2 years ago
[Bugfix][Kernel] allow non-power-of-two head sizes in prefix prefill (#4128)
Michał Moskal committed over 2 years ago
[Bugfix] Get available quantization methods from quantization registry (#4098)
Michael Goin committed over 2 years ago
Allow model to be served under multiple names (#2894)
Harry Mellor committed over 2 years ago
[Core] nccl integrity check and test (#4155)
youkaichao committed over 2 years ago
[Typing] Mypy typing part 2 (#4043)
SangBin Cho committed over 2 years ago
[Kernel] Add punica dimension for Swallow-MS-7B LoRA (#4134)
Shoichi Uchinami committed over 2 years ago
[Bugfix] fix output parsing error for trtllm backend (#4137)
Elinx committed over 2 years ago
[Core] RayWorkerVllm --> WorkerWrapper to reduce duplication (#4024)
youkaichao committed over 2 years ago
[CI] Move CPU/AMD tests to after wait (#4123)
Cade Daniel committed over 2 years ago
[Misc] [CI] Fix CI failure caught after merge (#4126)
Cade Daniel committed over 2 years ago
[Speculative decoding 6/9] Integrate speculative decoding with LLMEngine (#3894)
Cade Daniel committed over 2 years ago
[Core] Refactor model loading code (#4097)
Antoni Baum committed over 2 years ago
LM Format Enforcer Guided Decoding Support (#3868)
Noam Gat committed over 2 years ago
[Core] Fix engine-use-ray broken (#4105)
SangBin Cho committed over 2 years ago
[Typing] Fix Sequence type GenericAlias only available after Python 3.9. (#4092)
SangBin Cho committed over 2 years ago