A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 09:30 UTC
main
[Bugfix] Fix Deepseekv32 tool parser when stream interval > 1 (#36056)
Flora Feng committed 8 months ago
[Bug] Fix EmbedIOprocessor "classify" <-> "embed" (#37573)
Wentao Ye committed 8 months ago
[Refactor] Remove dead code in pooling model (#37572)
Wentao Ye committed 8 months ago
Fix AttributeError in Qwen3.5 GDN layers with quantized models (#37448)
Jim Smith committed 8 months ago
rasmith committed 8 months ago
[Performance] Enable Triton autotuning disk cache by default (#37188)
Artem Perevedentsev committed 8 months ago
test Qwen/Qwen3-4B-Instruct-2507 for unbacked (#36064)
Laith Sakka committed 8 months ago
[MoE Refactor] Rename "naive" all2all backend (#36294)
bnellnm committed 8 months ago
Comment fix for async rl example (#35244)
Aaron Hao committed 8 months ago
[ROCm][CI] Cleaning and restructuring amd-ci legacy pipeline (#34839)
Andreas Karatzas committed 8 months ago
[MoE Refactor] DefaultMoERunner simplifcation (#33049)
bnellnm committed 8 months ago
[Log] Log once in local node by default (#37568)
Wentao Ye committed 8 months ago
[CI] Add retry with 4x backoff to HTTP fetches for transient failures (#37218)
Andreas Karatzas committed 8 months ago
Fix `SpeculatorsConfig` now that `PreTrainedConfig` is a `dataclass` in Transformers (#37574)
Harry Mellor committed 8 months ago
[Bugfix] Fix AttributeError when serving MXFP8 models with DeepGEMM installed (#37358)
EdalatiAli committed 8 months ago
[torch.compile][BE][Multimodal] Remove requirement to set_model_tag to avoid cache conflict (#37345)
Lucas Kabela committed 8 months ago
[P/D] AnthropicMessages add kv_transfer_params for PD disaggregation (#37535)
Chauncey committed 8 months ago
[CPU][UX] Do not crash when tcmalloc/libiomp are not ldpreloaded (#37561)
Fadi Arafeh committed 8 months ago
[CI] Gate pre-commit on `ready` label or number of contributions (#37544)
Harry Mellor committed 8 months ago
Run MacOS smoke test on daily `cron` job instead of every commit (#37567)
Harry Mellor committed 8 months ago