A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Michael Goin committed about 1 year ago
[BUGFIX] Fix crash in Eagle Speculative Decoding models when exceedin… (#24662)
AlonKejzman committed about 1 year ago
[Logging] Remove TORCH_NCCL_AVOID_RECORD_STREAMS to squash a warning (#25532)
Tyler Michael Smith committed about 1 year ago
[Misc] Remove cruft file in repo (#25678)
Nicolò Lucchesi committed about 1 year ago
[Bugfix] Fix InternS1 video processing after Transformers v4.56 (#25644)
Isotr0py committed about 1 year ago
[ux] Switch a warning to debug about a pytorch fallback (#23750)
Russell Bryant committed about 1 year ago
[CPU] update torch 2.8 and fix missing fields in TorchSDPAMetadata (#25652)
Li, Jiang committed about 1 year ago
[torch.compile] Make Query Quantization Fusable (#24914)
Jonas M. Kübler committed about 1 year ago
[misc] log info messages by default for hanging / busy / idle (#25627)
youkaichao committed about 1 year ago
[mypy] Fix wrong type annotations related to tuple (#25660)
Cyrus Leung committed about 1 year ago
[Hardware][RISC-V] Add riscv64 support for vLLM with scalar (#22112)
chenlang committed about 1 year ago
[XPU][Triton]add xpu config in triton_reshape_and_cache_flash (#25643)
Kunshang Ji committed about 1 year ago
[CI/Build] Fix flaky entrypoints test (#25663)
Cyrus Leung committed about 1 year ago
Add backward compatibility for `guided_...` API (#25615)
Harry Mellor committed about 1 year ago
[V0 deprecation] Remove unreachable model_config.supported_tasks (#25642)
wang.yuqi committed about 1 year ago
[Bugfix] Parse SpeculativeConfig Error (#25142)
yyzxw committed about 1 year ago
[mypy] Further improve MM type annotations (#25654)
Cyrus Leung committed about 1 year ago
[Bugfix] Fix Qwen3-VL max_num_video_tokens calculation for video profiling (#25648)
Isotr0py committed about 1 year ago
[Bugfix] Add triton.language.tensor placeholder (#25649)
Agata Dobrzyniewicz committed about 1 year ago
[Misc] Fix Qwen3-VL `video_grid_thw` typing (#25646)
Roger Wang committed about 1 year ago