A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 07:15 UTC
main
[BugFix] Fix MultiConnector test after HMA changes (#19291)
Nick Hill committed over 1 year ago
Fixed ppc build when it runs on non-RHEL based linux distros (#18422)
Nishidha committed over 1 year ago
Yu Guo committed over 1 year ago
[Core] Raise when non-multi-instance DP clients target a DP rank (#19227)
jmswen committed over 1 year ago
[Model] Optimize nemotron_h implementation (#19249)
Jee Jee Li committed over 1 year ago
Unit Test for run_dp_sharded_vision_model (#19103)
Siqi Yan committed over 1 year ago
Fix CompilationConfig repr (#19091)
Richard Zou committed over 1 year ago
[Docs] Improve V1 KVConnector interface documentation (#19172)
Nick Hill committed over 1 year ago
[Quantization] Bump compressed-tensors version; update NVFP4A16 test model (#19224)
Dipika Sikka committed over 1 year ago
[TPU] fix kv cache dtype in model runner (#19244)
Chengji Yao committed over 1 year ago
[v1][P/D] Fix a edge case in kv cache schedule (#19182)
Jinghui Zhang committed over 1 year ago
[Chore] update CODEOWNERS (#19247)
Aaron Pham committed over 1 year ago
Support allowed_token_ids in ChatCompletionRequest (#19143)
Xu Song committed over 1 year ago
[TPU] update torch_xla pin (#19231)
Chengji Yao committed over 1 year ago
[v1] Hybrid Memory Allocator (#17996)
Chen Zhang committed over 1 year ago
[Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033)
Benjamin Chislett committed over 1 year ago
Fix AOPerModuleConfig name changes (#18869)
Jerry Zhang committed over 1 year ago
[Model] NemotronH support (#18863)
Luis Vega committed over 1 year ago
[V1] Use FlashInfer by default on Blackwell GPUs (#19118)
Michael Goin committed over 1 year ago
[Quantization] Skip Fp4 Test for `compressed-tensors` (#19217)
Dipika Sikka committed over 1 year ago