A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Core] Get num_encoder_tokens from scheduler config (#24989)
Russell Bryant committed about 1 year ago
[misc] fix typo in value error (#24995)
Prashant Gupta committed about 1 year ago
[UX] Remove "quantization is not fully optimized yet" log (#25012)
Michael Goin committed about 1 year ago
[Docs] fix invalid doc link (#25017)
yyzxw committed about 1 year ago
[UX] Enforce valid choices for envs like VLLM_ATTENTION_BACKEND, etc (#24761)
Michael Goin committed about 1 year ago
[Benchmarks] Add MMVU video dataset support and clean up deprecated datasets (#24719)
Isotr0py committed about 1 year ago
[Core] Use `CpuGpuBuffer` for block table tensors (#24795)
Nick Hill committed about 1 year ago
[Doc] Add --force-overwrite option to generate_cmake_presets.py (#24375)
elvischenv committed about 1 year ago
Removes source compilation of nixl dependency (#24874)
Benjamin Bartels committed about 1 year ago
[FP8] Extend per-token-group quantization support to QuantFP8 (#24342)
Tahsin Tunan committed about 1 year ago
[CI] GPT-OSS GPQA eval test for Blackwell (#24920)
Michael Goin committed about 1 year ago
[CI][Bugfix] Fix failing Blackwell test (#24993)
Matthew Bonanni committed about 1 year ago
[Perf] Reuse workspace for FP8+FP4 Marlin MoE (#20500)
Michael Goin committed about 1 year ago
fp8 kv cache support fix for torch.compile (#22758)
Aleksandr Malyshev committed about 1 year ago
Use kwargs for long lists of `EngineCoreRequest` arguments in tests and fix extra kwargs (#24987)
Andrew Sansom committed about 1 year ago
[gpt-oss][1][bugfix] fix streaming final output (#24466)
Andrew Xia committed about 1 year ago
[ROCm] Add dependencies for ROCm (#24900)
Concurrensee committed about 1 year ago
[MISC] Add code owners of vllm/v1 to vllm/v1/core (#24928)
Chen Zhang committed about 1 year ago
[gpt-oss][1b] streaming add item id, content id (#24788)
Andrew Xia committed about 1 year ago
Directly get max encoder len from VLLM config in V1 (#24866)
Sugar committed about 1 year ago