A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 12:15 UTC
main
[Optim] Compute multimodal hash only once per item (#17314)
Cyrus Leung committed over 1 year ago
[Benchmark] Add single turn MTBench to Serving Bench (#17202)
Ekagra Ranjan committed over 1 year ago
[Model] Add tuned triton fused_moe configs for Qwen3Moe (#17328)
Michael Goin committed over 1 year ago
Support loading transformers models with named parameters (#16868)
Alex Wu committed over 1 year ago
[Model] Qwen3 Dense FP8 Compat Fixes (#17318)
Simon Mo committed over 1 year ago
[Bugfix] Fix moe weight losing all extra attrs after `process_weights_after_loading`. (#16854)
Charlie Fu committed over 1 year ago
Lucas Wilkinson committed over 1 year ago
[Chore] cleanup license indicators in light of SPDX (#17259)
Aaron Pham committed over 1 year ago
[Security] Don't bind tcp zmq socket to all interfaces (#17197)
Russell Bryant committed over 1 year ago
Explicitly explain quant method override ordering and ensure all overrides are ordered (#17256)
Harry Mellor committed over 1 year ago
Make name of `compressed-tensors` quant method consistent across vLLM (#17255)
Harry Mellor committed over 1 year ago
Improve conversion from dataclass configs to argparse arguments (#17303)
Harry Mellor committed over 1 year ago
[Docs] Add a security guide (#17230)
Russell Bryant committed over 1 year ago
[Misc] Clean up Qwen2.5-Omni code (#17301)
Cyrus Leung committed over 1 year ago
[Misc] Minor typo/grammar in `platforms/interface.py` (#17307)
Nicolò Lucchesi committed over 1 year ago
[doc] update wrong model id (#17287)
Reid committed over 1 year ago
[Bugfix] Add missing `get_language_model` to new MLLMs (#17300)
Cyrus Leung committed over 1 year ago
Add missing class docstring for `PromptAdapterConfig` (#17302)
Harry Mellor committed over 1 year ago
Update tpu_worker.py 's typo (#17288)
idouba committed over 1 year ago
[Model] Add Granite Speech Support (#16246)
Alex Brooks committed over 1 year ago