A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 14:45 UTC
main
[Doc] Document InternVL3 support (#16495)
Isotr0py committed over 1 year ago
Michael Goin committed over 1 year ago
[TPU][V1] Make `--disable_chunked_mm_input` mandatory for serving MM models (#16483)
Nicolò Lucchesi committed over 1 year ago
Fix erroneous "model doesn't support compile" warning (#16486)
Richard Zou committed over 1 year ago
[Hardware][Intel-Gaudi] Multi-step scheduling implementation for HPU (#12779)
Tomasz Zielinski committed over 1 year ago
[Bugfix] Fix bugs of running Quark quantized models (#16236)
chaow-amd committed over 1 year ago
[Kernel] support merge_attn_states CUDA kernel, 3x speedup (#16173)
DefTruth committed over 1 year ago
Don't install triton on `ppc64le` platform (#16470)
Harry Mellor committed over 1 year ago
[Misc] update api_client example (#16459)
Reid committed over 1 year ago
[Misc] Raise error for V1 not supporting Long LoRA. (#16415)
Jee Jee Li committed over 1 year ago
Enforce valid max_num_batched_tokens when disable_chunked_mm_input=True (#16447)
Michael Goin committed over 1 year ago
[Core][LoRA][1/N] Add LoRA for EncoderDecoderModelRunner (#15990)
Jee Jee Li committed over 1 year ago
Revert "[Model] use AutoWeightsLoader for deepseek_v2, internlm2" (#16453)
DefTruth committed over 1 year ago
[Bugfix] Don't set an upper bound on repetition penalty (#16403)
Alex Brooks committed over 1 year ago
[CPU][Bugfix] Fix CPU docker issues (#16454)
Li, Jiang committed over 1 year ago
[Bugfix][VLM] Fix failing Phi-4-MM multi-images tests and add vision-speech test (#16424)
Isotr0py committed over 1 year ago
Update supported_hardware.md for TPU INT8 (#16437)
Michael Goin committed over 1 year ago
[Llama4] Enable attention temperature tuning by default for long context (>32k) (#16439)
Yong Hoon Shin committed over 1 year ago
update benchmark_serving_structured_output to include auto backend (#16438)
Chenyaaang committed over 1 year ago
check input length of sonnet samples (#16423)
Alexey Belyakov committed over 1 year ago