A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 8, 10:00 UTC
main
[Bugfix] Fix more multimodal tests for transformers V5 (#34334)
Raushan Turganbay committed 9 months ago
[Bugfix] send None sentinel on final commit so server properly sends transcription.done (#33963)
Junseo Park committed 9 months ago
[Benchmarks] Reduce ready checker log verbosity (#34349)
Tomas Ruiz committed 9 months ago
[GPT-OSS] Remove unnecessary contiguous (#34337)
elvischenv committed 9 months ago
[ROCm] [CI] fix test_unrecognized_env (#34350)
TJian committed 9 months ago
Eldar Kurtić committed 9 months ago
[Bugfix] fix default is_neox_style is True for deepseek (#34353)
Xinyu Dong committed 9 months ago
[Multimodal] Expose `mm_processor_kwargs` for `DummyInputsBuilder` (#34330)
Isotr0py committed 9 months ago
[Model Runner V2] Init cuda graph pool when necessary (#33217)
Xinyu Chen committed 9 months ago
[CI][BugFix] Fix silent failure in shellcheck hook and baseline exist… (#32458)
junuxyz committed 9 months ago
[Docs] Fix typo ("defult") and double spacing (#34348)
SorenDreano committed 9 months ago
[ROCm] [aiter] Split KV cache update for AiterFlashAttention (#33681)
kliuae committed 9 months ago
[Bugfix]: Fix ROCm fusion attn test; use AttentionBackend utils to create kv cache (#33948)
Rohan Potdar committed 9 months ago
Don't try and run GLM-ASR with remote code (#34352)
Harry Mellor committed 9 months ago
Reapply [Attention][FA3] Update FA3 to include new swizzle optimization (#34043)
Lucas Wilkinson committed 9 months ago
Responses harmony system message structured (#34268)
Adam Binford committed 9 months ago
[NVIDIA][test] Tests for flashinfer TRTLLM BF16 MoE (#33715)
Linda committed 9 months ago
Make JAIS compatible with Transformers v5 (#34264)
Harry Mellor committed 9 months ago
Make Qwen3VL compatible with Transformers v5 (#34262)
Harry Mellor committed 9 months ago
[Bugfix][CPU] Fix llama4 inference on CPU (#34321)
Li, Jiang committed 9 months ago