A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 14:45 UTC
main
[Bugfix] fix deepseek fp16 scale bug (#14809)
Jinzhen Lin committed over 1 year ago
[BugFix] Fix Llama4 - Index Error When Single Request Near Max Context (#16209)
Lucas Wilkinson committed over 1 year ago
Add warning that content below line in template will be removed (#16276)
Harry Mellor committed over 1 year ago
[Bugfix] Handle `process_weights_after_loading` for `QKVCrossParallelLinear` (#15328)
Isotr0py committed over 1 year ago
[Bugfix] Proper input validation for multi-modal encoder-decoder models (#16156)
Cyrus Leung committed over 1 year ago
Kero Liang committed over 1 year ago
[New Model]: jinaai/jina-embeddings-v3 (#16120)
wang.yuqi committed over 1 year ago
Update to transformers==4.51.1 (#16257)
Harry Mellor committed over 1 year ago
[Bugfix] Remove triton do_bench fast_flush arg (#16256)
Kebe committed over 1 year ago
[BugFix][ROCm] Fix GGUF MoE Dispatch Block_Dim for ROCm (#16247)
TY-AMD committed over 1 year ago
[Model] use AutoWeightsLoader for phimoe,qwen2_moe,qwen3_moe (#16203)
rongfu.leng committed over 1 year ago
[Misc] Add warning for multimodal data in LLM.beam_search (#16241)
Alex Brooks committed over 1 year ago
[Misc] format and refactor some examples (#16252)
Reid committed over 1 year ago
[Docs] Add Slides from Singapore Meetup (#16213)
Simon Mo committed over 1 year ago
[Misc] Merge the logs of pp layers partitions (#16225)
Kebe committed over 1 year ago
[V1] Add `disable_chunked_mm_input` arg to disable partial mm input prefill (#15837)
Michael Goin committed over 1 year ago
[torch.compile][TPU] Make @support_torch_compile work for XLA backend (#15782)
Siyuan Liu committed over 1 year ago
[Bugfix] Fix and reorganize broken GGUF tests and bump gguf version (#16194)
Isotr0py committed over 1 year ago
[Bugfix] Do not skip "empty" parts of chats that are parsable (#16219)
Michael Goin committed over 1 year ago
Add warning for Attention backends that do not support irope yet (#16212)
Yong Hoon Shin committed over 1 year ago