A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 16:30 UTC
main
[Docs] Improve malformed exception caused by backslash line continuations (#31694)
maang committed 10 months ago
Revert "[CI Failure] Disable B200 tests while runner is broken" (#31750)
Michael Goin committed 10 months ago
[Bugfix] vLLM produces invalid UTF-8 tokens and “�” (#28874)
John Calderon committed 10 months ago
[CI/Build] Allow user to configure NVSHMEM version via ENV or command line (#30732)
Seiji Eicher committed 10 months ago
[Bugfix] Properly apply v_scale for mimo_v2_flash (#31175)
Michael Goin committed 10 months ago
[Bugfix] Fix Broken ModelOpt NVFP4 MoE (#31742)
Robert Shaw committed 10 months ago
[MoE Refactor] Aiter Experts for BF16 MoE (#31542)
Yongye Zhu committed 10 months ago
[Bug] Revert torch warning fix (#31585)
Wentao Ye committed 10 months ago
[CI][DeepSeek] Add nightly DeepSeek R1 `lm_eval` tests on H200 (#30356)
Matthew Bonanni committed 10 months ago
[Cleanup] Remove deprecated fields from CachedRequestData class (#31734)
Nick Hill committed 10 months ago
[Model] Nemotron Parse 1.1 Support (#30864)
amitz-nv committed 10 months ago
[docker] install cuda13 version of lmcache and nixl (#30913)
Qidong Su committed 10 months ago
pin lora_b moe weights on cpu (#31317)
gnovack committed 10 months ago
[BugFix] Fix architecture flags to prevent issues on SM103 (#31150)
Roberto L. Castro committed 10 months ago
[Misc][Model][Refactor] Pass the prefix into Linear layers (#31669)
Wang Kunpeng committed 10 months ago
Fix GLM-4.6v flash tool calling in transformers 5.x (#31622)
baonudesifeizhai committed 10 months ago
[Misc] Enable Paligemma's PrefixLM attention mask computation (#31725)
Isotr0py committed 10 months ago
Triton Attention: Support cross-layers blocks (#30687)
Or Ozeri committed 10 months ago
[Bugfix] Add missing extra_tensors arg to DeviceCommunicatorBase.disp… (#31644)
kzwrime committed 10 months ago
[Bugfix][CPU] Fix RotaryEmbedding fallback causing gibberish with --enforce-eager (#31643)
RickyChen / 陳昭儒 committed 10 months ago