A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 07:00 UTC
main
[Bugfix]: allow extra fields in requests to openai compatible server (#10463)
Guillaume Calmettes committed almost 2 years ago
[7/N] torch.compile, reduce compilation time (#10460)
youkaichao committed almost 2 years ago
[perf bench] H200 development (#9768)
Simon Mo committed almost 2 years ago
[platforms] restore xpu check for parallel config (#10479)
youkaichao committed almost 2 years ago
[Hardware][CPU] Support chunked-prefill and prefix-caching on CPU (#10355)
Li, Jiang committed almost 2 years ago
[Platforms] Refactor xpu code (#10468)
Mengqing Cao committed almost 2 years ago
[Bugfix] Handle conflicts between modern and legacy fields (#10471)
Cyrus Leung committed almost 2 years ago
[model] Reduce medusa weight (#10454)
Sky Lee committed almost 2 years ago
[ci/build] Combine nightly and optional (#10465)
Kevin H. Luu committed almost 2 years ago
[CI/Build] Support compilation with local cutlass path (#10423) (#10424)
wchen61 committed almost 2 years ago
[CI/Build] Add sphinx/rst linter for docs (#10366)
Rafael Vasquez committed almost 2 years ago
[Bugfix] Enforce no chunked prefill for embedding models (#10470)
Cyrus Leung committed almost 2 years ago
[Bugfix] Fix Mamba model initialization and MLP Speculator weights loading (#10456)
Isotr0py committed almost 2 years ago
[Misc] Add __setitem__ for LazyDict (#10469)
Yanyi Liu committed almost 2 years ago
[Bugfix] Marlin 2:4 temp fix for large M dim (>256) (#10464)
Lucas Wilkinson committed almost 2 years ago
[CI/Build] Update Dockerfile.rocm (#10434)
Alexei-V-Ivanov-AMD committed almost 2 years ago
Change granite chat template to keep json list formatting for tool calls (#10452)
Maximilien de Bayser committed almost 2 years ago
[Model][Quantization] HQQ support through Marlin kernel expansion (#9766)
ElizaWszola committed almost 2 years ago
[Core] Avoid metrics log noise when idle (#8868)
Russell Bryant committed almost 2 years ago
[6/N] torch.compile rollout to users (#10437)
youkaichao committed almost 2 years ago