A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Update GLM-4.5 Doc transformers version (#25830)
Yuxuan Zhang committed about 1 year ago
Fix random dataset mismatched token length with config. (#24937)
weiliang committed about 1 year ago
[VLM] Update Qwen3-VL max_num_video_tokens calculation for configurable video profiling (#25557)
Isotr0py committed about 1 year ago
[Bugfix] Fix Qwen3-VL regression from #24982 (#25814)
Roger Wang committed about 1 year ago
[MM] Optimize memory profiling for scattered multimodal embeddings (#25810)
Roger Wang committed about 1 year ago
[Bugfix][NIXL] Fix Async Scheduler timeout issue (#25808)
Nicolò Lucchesi committed about 1 year ago
[Core] GC Debug callback (#24829)
Jialin Ouyang committed about 1 year ago
[Bug]: Set LD_LIBRARY_PATH to include the 'standard' CUDA location (#25766)
Clayton Coleman committed about 1 year ago
[torch.compile]: Add VLLM_DEBUG_DUMP_PATH environment variable (#25651)
Jiangyun Zhu committed about 1 year ago
[Core] Refactor self.model() to call a helper for subclassing. (#25084)
Patrick C. Toulme committed about 1 year ago
[env] default nixl side port conflicts with kv-event zmq port (#25056)
Peter Pan committed about 1 year ago
[docs] transcriptions API audio upload (#25446)
yyzxw committed about 1 year ago
[Bugfix][WideEP] Apply TP Attn + EP MoE fix to other models (#24982)
Tyler Michael Smith committed about 1 year ago
[Bugfix] Fix triton import precommit failure (#25803)
Tyler Michael Smith committed about 1 year ago
Fix GPTQ model loading in Transformers backend (#25770)
Harry Mellor committed about 1 year ago
Add filtering for chat template kwargs (#25794)
Russell Bryant committed about 1 year ago
Validate API tokens in constant time (#25781)
Russell Bryant committed about 1 year ago
[Bugfix] Merge MM embeddings by index instead of token IDs (#16229)
Cyrus Leung committed about 1 year ago
[Bugfix] Add missing `image_size` for phi4_multimodal (#25796)
Xiaohan Zou committed about 1 year ago
[Misc] Update openai client example file for multimodal (#25795)
Roger Wang committed about 1 year ago