A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Feature] publisher default set zmq in kv_event config (#26915)
rongfu.leng committed about 1 year ago
[Prefix Cache] Use LoRA name for consistent KV-cache block hashing (#27211)
Sage committed about 1 year ago
[Doc] Fix numbering sequence in prefix caching (#27357)
William Song committed about 1 year ago
[Model] Revert PR #26715: Restore custom PaliGemma and Gemma3-MM impl… (#27309)
Luciano Martins committed about 1 year ago
[Bugfix] Disable FlexAttention direct block mask building for encoder-only models (#27344)
Isotr0py committed about 1 year ago
Support Anthropic API /v1/messages Endpoint (#22627)
RED committed about 1 year ago
[P/D] Dynamic `kv_output_aggregator` collect size (#26734)
Nicolò Lucchesi committed about 1 year ago
[Frontend] Require flag for loading text and image embeds (#27204)
Russell Bryant committed about 1 year ago
[Bugfix] Fix HF format InternVL large variants video processing (#27330)
Isotr0py committed about 1 year ago
[Bugfix] Make `get_mrope_input_positions` instance methods (#27342)
Cyrus Leung committed about 1 year ago
[NIXL] use Host buffer to support TP_ratio > 1 for XPU (#27140)
Chendi.Xue committed about 1 year ago
[Bugfix] Add missing 'is_internal_router' attribute to FusedMoEWithLoRA (#27351)
Jee Jee Li committed about 1 year ago
[bugfix] remove unused parameters to reduce unnecessary vram usage (#26789)
Reinforce-II committed about 1 year ago
[Bug] Fix DeepSeek-V2.5-1210-FP8 issue (#27267)
Wentao Ye committed about 1 year ago
[NIXL] Terminate handshake listener thread in shutdown (#26404)
Mark McLoughlin committed about 1 year ago
[Model] Upstream Deepseek-OCR model (#27247)
Isotr0py committed about 1 year ago
[Chore] Separate out optional dependency checks from vllm.utils (#27207)
dongbo910220 committed about 1 year ago
Mirroring changes in test-pipeline.yaml into test-amd.yaml (#27242)
Alexei-V-Ivanov-AMD committed about 1 year ago
[docs] Update v1 metrics design doc (#27332)
Mark McLoughlin committed about 1 year ago
[Core] Handle MoE LoRA edge cases (#27335)
Jee Jee Li committed about 1 year ago