A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 12:30 UTC
main
[Doc]: fixing typos in various files (#30540)
Didier Durand committed 11 months ago
[main][BugFix] Fixed an accuracy bug of Qwen3-next-MTP when batched inferring (#30632)
drslark committed 11 months ago
[Chore] Remove redundant `RequestPrompt` (#30612)
Cyrus Leung committed 11 months ago
enable unbacked with aot_compile (#30462)
Laith Sakka committed 11 months ago
[ROCm][CI] Add "Qwen3-Next-80B-A3B-Instruct MTP Async EPLB Accuracy Test" Back Into AMD CI (#30590)
Micah Williamson committed 11 months ago
Improve parse_raw_prompt test cases for invalid input .v2 (#30512)
Kayvan Mivehnejad committed 11 months ago
[Bug][KVConnector][Metrics] Remove a vacuous assertion breaking external-launcher (#30577)
Qier Li committed 11 months ago
[Doc] Add instructions for building docker image on GB300 with CUDA13 (#30414)
Qidong Su committed 11 months ago
[Logs] Optimize startup logs 4 (#29903)
Wentao Ye committed 11 months ago
[Doc] Add documents for multi-node distributed serving with MP backend (#30509)
Isotr0py committed 11 months ago
[Docs] Clarify Expert Parallel behavior for attention and MoE layers (#30615)
lif committed 11 months ago
set assume_32bit_indexing and pass unbacked hints (#30459)
Laith Sakka committed 11 months ago
[Refactor] `TokenizerRegistry` only uses lazy imports (#30609)
Cyrus Leung committed 11 months ago
[Bugfix] Qwen3-next with --hf-overrides \{\"num_hidden_layers\":8\} (#30433)
Chen Zhang committed 11 months ago
[CI/Build] Fix broken mm processor test Mistral-3-large (#30597)
Isotr0py committed 11 months ago
[Chore] Adjust tokenizer import to avoid circular imports (#30601)
Cyrus Leung committed 11 months ago
[Scheduer] Simplify stop checking for pooling models (#30591)
Nick Hill committed 11 months ago
[Bugfix] Dictionary MM embeddings for online chat (#30507)
Cyrus Leung committed 11 months ago
[Misc][Quantization] Clarify the intent of GGUF `FusedMoE` weight materialization (#30310)
Tsukasa OI committed 11 months ago
[Feature] Add SM103 (Blackwell Ultra) Support to vLLM (#30484)
Roberto L. Castro committed 11 months ago