A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 3, 03:00 UTC
main
[Bug] Fix ci issue `assert output_size is not None` AssertionError (#43261)
Wentao Ye committed 6 months ago
[Test] Replace zephyr-7b-beta (7B) with SmolLM2-135M in tokenization test (#43085)
Kevin H. Luu committed 6 months ago
Fix FlashInfer TRTLLM NvFP4 monolithic MoE routing (#43223)
zhangxin81 committed 6 months ago
Update KDA chunk prefill decay to use exp2 semantics (#43195)
zexplorerhj committed 6 months ago
[XPU] add setuptools-rust for xpu dependency (#43287)
Kunshang Ji committed 6 months ago
[CI] De-flake test_models for bigscience/bloom-560m (#43197)
haosdent committed 6 months ago
[Bugfix] Fix glm4_moe_tool_parser._is_string_type for /v1/responses FunctionTool format (#39601)
Yiyang "Ian" Liu committed 6 months ago
[Benchmark] Add num-warmup to vllm bench throughput (#43245)
yzong-rh committed 6 months ago
[Bugfix] Warn when renderer_num_workers has no effect on offline LLM (#42905)
Daoyuan Li committed 6 months ago
[Frontend][RFC] Rust front-end integration (#40848)
Nick Hill committed 6 months ago
손세정 committed 6 months ago
[Refactor] Use shared coerce_to_schema_type in Seed-OSS tool parser (#43140)
Flora Feng committed 6 months ago
[Frontend] Normalize reasoning_content to reasoning for client compatibility (#42664)
Ben Browning committed 6 months ago
[CI] Fix "test_vit_cudagraph_[image|video][step3_vl]" failure (#43082)
haosdent committed 6 months ago
[Perf][Gemma4] Batch vision encoder calls for image and video processing (#43169)
Luciano Martins committed 6 months ago
[Bugfix] Fix Qwen3.5 GatedDeltaNet in_proj_ba Marlin failure at TP>=2 (#36329)
sonusflow committed 6 months ago
[Minor] Bigger overlap for FI AR (#43103)
Jee Jee Li committed 6 months ago
update GPU json file based on h200 recipes (#43262)
Louie Tsai committed 6 months ago
[Model Runner V2] Fix lora `Triton Error [CUDA]: device-side assert triggered` (#43139)
Wentao Ye committed 6 months ago
[CI] Add composed-schema regression tests for DeepSeek V3.2/V4 parsers (#43255)
Ace Eldeib committed 6 months ago