A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 4, 06:15 UTC
main
[CI] Route part of B200 jobs to b200-k8s (#41453)
Kevin H. Luu committed 6 months ago
[ROCm][CI] Use vLLM generation defaults for DeepSeek prefetch-offload eval (#41575)
Andreas Karatzas committed 6 months ago
[CI] Automate Docker Hub release image publishing (#40415)
Kevin H. Luu committed 6 months ago
[Attention] Minor refactor: layer takes ownership of the MLA prefill backend (#41744)
Matthew Bonanni committed 6 months ago
Bump model-hosting-container-standards to >= 0.1.14 (#39755)
Dhruvil Bhatt committed 6 months ago
[Spec Decode] Fix max_model_len logging in speculative config for draft model (#41571)
Lanze Liu committed 6 months ago
Fix DeepGEMM ep_scatter output address overflow (#39213)
Matej Sirovatka committed 6 months ago
[BUGFIX] Support streamed_args_for_tool in MistralToolParser (#41730)
Julien Denize committed 6 months ago
[New Model][ROCm] Add AMD support for DeepSeek V4 (#40871)
Hexiang Wang committed 6 months ago
[Attention] Move FA3→FA4 upgrade into get_flash_attn_version() (#40815)
Canlin Guo committed 6 months ago
Remove unnecessary runtime asserts from linear layers (#41729)
Harry Mellor committed 6 months ago
[Bugfix] Suggest upgrading Transformers for tokenizer class errors (#38099)
Lidang Jiang committed 6 months ago
yzong-rh committed 6 months ago
[BugFix][MyPy]: Module has no attribute "sched_getaffinity" [attr-defined] (#41465)
Martin Hickey committed 6 months ago
[Bugfix][KVConnector] Support DCP/PCP in OffloadingConnector (#41549)
Itay Etelis committed 6 months ago
Fix DeepSeek-OCR for Transformers v4 (#41460)
Harry Mellor committed 6 months ago
[Model] support Qianfan-OCR model (#40136)
bairongz committed 6 months ago
[Feature] Add Triton kernel JIT compilation monitor for inference (#40137)
Artem Perevedentsev committed 6 months ago
[XPU] use xpu topk topp sample kernel (#39285)
Kunshang Ji committed 6 months ago
[P/D][Mooncake] Add KVConnectorStats for transfer observability (#40414)
Zhewen Li committed 6 months ago