A high-throughput and memory-efficient inference and serving engine for LLMs
main
[Misc][Docs] Fix XPU compute-runtime driver link version mismatch (#49299)
Lena Onyshchenko committed 3 months ago
Revert "[Sampler] Stop upcasting logits to fp32 in apply_sampling_params" (#48641) (#49033)
vllm-agent committed 3 months ago
[Bugfix] Fix Ovis2_5 special tokens for transformers v5 (#47298)
Miłosz Grunwald committed 3 months ago
Propagate Flash Attention cache configuration to Ray workers (#49177)
Stefan Kaestle committed 3 months ago
[Misc] Add @esmeetu to codeowners for rust/src/bench (#49298)
Roy Wang committed 3 months ago
[Bugfix] Propagate quant_config to LFM2 ShortConv projections (#48917)
Alex Yuan committed 3 months ago
[Frontend] Support additional sampling parameters for translation API (#45839)
Guan-Ming Chiu committed 3 months ago
[Rust][Benchmark] Use `tracing` for logs (#48937)
Bugen Zhao committed 3 months ago
[Bugfix][Rust Frontend] Handle zero-column logprobs payloads without panicking (#49113)
Salt Sato committed 3 months ago
Update BGE-M3 token expectations for leading spaces (#49269)
aoshen02 committed 3 months ago
[CI][NIXL] Isolate concurrent engine internal ports (#49129)
Andreas Karatzas committed 3 months ago
[MRV2] Add encoder cache profiling implementation (#47985)
Isotr0py committed 3 months ago
[Bugfix][KV Offloading] Handle queued request aborts without allocated KV blocks (#49146)
Chauncey committed 3 months ago
[Rust Frontend][gRPC] Add engine-aware health reporting (#48992)
Connor Carpenter committed 3 months ago
[CI] Exercise FA3 FP8 attention on SM90 (#49231)
Simon Mo committed 3 months ago
[XPU][Doc] Update XPU docker image documents (#49148)
Kunshang Ji committed 3 months ago
Update qutlass cmake for stable abi (#47879)
Chris Leonard committed 3 months ago
[Bugfix] Fix WSL circular import from pin_memory warning_once (#48444)
Alejandro Paredes La Torre committed 3 months ago
Fixes non-coalesced HBM access in marlin_int4_fp8_preprocess_kernel_awq (#47268)
flutist-alibaba committed 3 months ago
[Misc][Docs] Fix broken protocol link in speech_to_text doc (#47212)
Lena Onyshchenko committed 3 months ago