A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 13, 04:30 UTC
main
[BugFix] Fix misprint introduced by modular_kernel refactoring. (#28728)
Andrey Khalyavin committed 12 months ago
[Bugfix] resolve Qwen3-VL GPTQModel quantized model loading failure (#28663)
GuanH committed 12 months ago
[Chore] Rename `SchedulerConfig.chunked_prefill_enabled` (#28735)
Cyrus Leung committed 12 months ago
[Docs] Enable some more markdown lint rules for the docs (#28731)
Harry Mellor committed 12 months ago
[Bugfix] [ROCm] [AITER]: Fix aiter block quant not compatible with torch compile dynamo (#28716)
TJian committed 12 months ago
LLaMA4 LoRA Adapter Enablement (#28602)
Fardin Hoque committed 12 months ago
docs(lora_resolvers): clarify multi-resolver order and storage path requirement (#28153)
Chen Wang committed 12 months ago
[kernel] Improve FP8 PTPC on Hopper for larger shapes (#28692)
czhu-cohere committed 12 months ago
Remove audio optional dependency for mistral-common (#28722)
Julien Denize committed 12 months ago
Fix typo in comment: existance -> existence (#28737)
Mohammad Othman committed 12 months ago
[Attention] Bump FA for removed method (#28429)
Matthew Bonanni committed 12 months ago
[DisaggEverything] Tokens in<>out `/generate` endpoint (#24261)
Nicolò Lucchesi committed 12 months ago
[CI][CPU] Smoke test for Apple Silicon using GHA MacOS runner (#28688)
Michael Goin committed 12 months ago
[Docs] Update the name of `Transformers backend` -> `Transformers modeling backend` (#28725)
Harry Mellor committed 12 months ago
dongbo910220 committed 12 months ago
[Kernels] Enable FlashInfer FP8 Blockscale on SM90 (for TEP DSR1) (#27134)
Duncan Moss committed 12 months ago
[Config] Clean up SchedulerConfig initialization (#28665)
Cyrus Leung committed 12 months ago
[Bugfix][Nixl] Fix kernel physical<>logical block_size issue (#28677)
Nicolò Lucchesi committed 12 months ago
[Model] Fix bailing_moe accuracy problem (#28277)
zhaozx-cn committed 12 months ago
[Doc] Fix macOS installation dependency resolution issue (#26721)
Fasal Shah committed 12 months ago