A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 08:15 UTC
main
[ROCm][CI] Update GSM8K eval config to use fp8-and-mixed models list (#37619)
Andreas Karatzas committed 8 months ago
[Model] Deprecate the score task (this will not affect users). (#37537)
wang.yuqi committed 8 months ago
[Model Runner V2] Fix draft logits not populated during cudagraph replay (#37639)
Giancarlo Delfin committed 8 months ago
[CI] Removing deprecated rlhf examples reference (#37585)
Andreas Karatzas committed 8 months ago
[Bug] Fix FlashInfer allreduce fusion workspace uninitialized error (#37461)
Wei Zhao committed 8 months ago
[XPU] bump vllm-xpu-kernels to v0.1.4 (#37641)
Kunshang Ji committed 8 months ago
[PluggableLayer][MM] Add PluggableLayer for CustomQwen2Decoder (#37293)
Wangbei25 committed 8 months ago
[Model] Refactor Step3-VL processor to HF style (#37579)
Cyrus Leung committed 8 months ago
[Refactor] Relocate entrypoint tests to match serving code structure (#37593)
Flora Feng committed 8 months ago
[XPU] Automatically detect target platform as XPU in build. (#37634)
Huanxing committed 8 months ago
[Model Runner V2] fix draft attention metadata generation (#37364)
Giancarlo Delfin committed 8 months ago
fix(xpu): Re-compute compile ranges after platform-specific config updates (#37523)
Yuxiang Liang committed 8 months ago
[V0 Deprecation] Deprecate --disable-frontend-multiprocessing (#37612)
Flora Feng committed 8 months ago
[Bugfix][LoRA] Fix Qwen35 LoRA (#36976)
Jee Jee Li committed 8 months ago
[compile][graph_partition]Add tensor size handling (#36038)
Xiao committed 8 months ago
[Feat] Enable CompressedTensorW4A8Int for XPU (#37207)
Tianmu Li committed 8 months ago
fix: disambiguate multimodal prefix cache keys (#36708)
tianshu-Michael-yu committed 8 months ago
[CI] Update mergify tool-calling label paths (#37478)
Flora Feng committed 8 months ago
Fix DP coordinator ZMQ port TOCTOU (#37452)
Itay Alroy committed 8 months ago
[ROCm][Bugfix] fix cache block size mismatch for aiter unified attention (#37606)
Divakar Verma committed 8 months ago