A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Harry Mellor committed about 1 year ago
[V1] Enable V1 for compute capability < 8.0 + FP32 (#23614)
Cyrus Leung committed about 1 year ago
[Bugfix] Fix scheduling when repeated images in one request (#23544)
Roger Wang committed about 1 year ago
[Bugfix] Fix Qwen25VL packed_modules_mapping (#23604)
Jee Jee Li committed about 1 year ago
Fix CLI parameter documentation inconsistency in pooling_models.md (#23630)
Raghavan committed about 1 year ago
[Docs] Fix titles for multi-file examples that are rendered in the docs (#23573)
Harry Mellor committed about 1 year ago
[Benchmarks] add benchmark for embedding models (#23000)
Jiangyun Zhu committed about 1 year ago
[mypy] Fix incorrect type hint for EAGLE3 support (#23617)
Cyrus Leung committed about 1 year ago
[fix] fix seed-oss-parser (#23560)
Bin Jia committed about 1 year ago
[CI/Build] Fix typo in #23561 (#23616)
Cyrus Leung committed about 1 year ago
Zijing Liu committed about 1 year ago
Remove graph_pool as member of VllmBackend and argument to CUDAGraphWrapper (#23385)
Copilot committed about 1 year ago
[CI/Build] Use vLLM client's user agent to fetch images (#23561)
Cyrus Leung committed about 1 year ago
[Bug] Fix DeepGEMM Env Control (#23591)
Wentao Ye committed about 1 year ago
Update Flashinfer to 0.2.14.post1 (#23537)
weiliang committed about 1 year ago
[CI Fix] Pin deepep and pplx tags in tools/ep_kernels/, gate multigpu tests (#23568)
Michael Goin committed about 1 year ago
[Misc] Add release note draft to PR template (#23598)
Simon Mo committed about 1 year ago
[Misc] Simplify FlashInfer attention metadata (#23585)
Woosuk Kwon committed about 1 year ago
[Docs] Update Documentation of Cohere Command-A Models (#23584)
Terrence Zhao committed about 1 year ago
[TPU][Bugfix] Fixes prompt_token_ids error in tpu tests. (#23574)
Pate Motter committed about 1 year ago