A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 17:15 UTC
main
Separate base model from `TransformersModel` (#15467)
Harry Mellor committed over 1 year ago
[Misc] improve example script output (#15528)
Reid committed over 1 year ago
[Misc] Enhance warning information to user-defined chat template (#15408)
wwl2755 committed over 1 year ago
[FEAT][ROCm] Integrate Fused MoE Kernels from AITER (#14967)
vllmellm committed over 1 year ago
[Feature] Enhance EAGLE Architecture with Proper RMS Norms (#14990)
Bryan Lu committed over 1 year ago
Fix raw_request extraction in load_aware_call decorator (#15382)
daniel-salib committed over 1 year ago
[misc] LoRA - Skip LoRA kernels when not required (#15152)
Varun Sundar Rabindranath committed over 1 year ago
Lucas Wilkinson committed over 1 year ago
[Misc] Warn about v0 in benchmark_paged_attn.py (#15495)
Tyler Michael Smith committed over 1 year ago
[Model] Support multi-image for Molmo (#15438)
Cyrus Leung committed over 1 year ago
Transformers backend already supports V1 (#15463)
Harry Mellor committed over 1 year ago
[CI/Build] LoRA: Delete long context tests (#15503)
Varun Sundar Rabindranath committed over 1 year ago
[Core] LoRA: V1 Scheduler optimization (#15422)
Varun Sundar Rabindranath committed over 1 year ago
[core] add bucket padding to tpu_model_runner (#14995)
Chenyaaang committed over 1 year ago
[V1] Support long_prefill_token_threshold in v1 scheduler (#15419)
Lu Fang committed over 1 year ago
[V1][Minor] Use `SchedulerInterface` type for engine scheduler field (#15499)
Nick Hill committed over 1 year ago
[TPU][V1] Fix Sampler recompilation (#15309)
Nicolò Lucchesi committed over 1 year ago
Add workaround for shared field_names in pydantic model class (#13925)
Maximilien de Bayser committed over 1 year ago
[bugfix] add supports_v1 platform interface (#15417)
Joe Runde committed over 1 year ago
[Bugfix] Support triton==3.3.0+git95326d9f for RTX 5090 (Unsloth + vLLM compatibility) (#15471)
Antonio Gómez committed over 1 year ago