A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 17:00 UTC
main
[CI/Build] Increasing timeout for multiproc worker tests (#8203)
Alexey Kondratiev(AMD) committed about 2 years ago
[Misc] Use ray[adag] dependency instead of cuda (#7938)
Rui Qiao committed about 2 years ago
[Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191)
afeldman-nm committed about 2 years ago
[BugFix] Fix Granite model configuration (#8216)
Nick Hill committed about 2 years ago
[Core] Support load and unload LoRA in api server (#6566)
Jiaxin Shan committed about 2 years ago
sroy745 committed about 2 years ago
Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165)
Michael Goin committed about 2 years ago
[MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029)
Alex Brooks committed about 2 years ago
Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860)
manikandan.tm@zucisystems.com committed about 2 years ago
[Doc] Indicate more information about supported modalities (#8181)
Cyrus Leung committed about 2 years ago
[Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173)
Elfie Guo committed about 2 years ago
[ci] Mark LoRA test as soft-fail (#8160)
Kevin H. Luu committed about 2 years ago
[Misc] Clean up RoPE forward_native (#8076)
Woosuk Kwon committed about 2 years ago
[bugfix] >1.43 constraint for openai (#8169)
William Lin committed about 2 years ago
[Doc] [Misc] Create CODE_OF_CONDUCT.md (#8161)
Maureen McElaney committed about 2 years ago
Bump version to v0.6.0 (#8166)
Simon Mo committed about 2 years ago
[Neuron] Adding support for adding/ overriding neuron configuration a… (#8062)
Harsha vardhan manoj Bikki committed about 2 years ago
[MISC] Replace input token throughput with total token throughput (#8164)
Cody Yu committed about 2 years ago
[Feature] OpenAI-Compatible Tools API + Streaming for Hermes & Mistral models (#5649)
Kyle Mistele committed about 2 years ago
[CI] Change test input in Gemma LoRA test (#8163)
Woosuk Kwon committed about 2 years ago