A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 06:30 UTC
main
[Docs] Upgrade dynamic LoRA warning to admonition block (#35218)
Russell Bryant committed 8 months ago
[Feature][CI]: compare `func` & `no_func` outputs in test_functionalization.py (#35481)
Bhuminjay Soni committed 8 months ago
[Docs] Update design/multiprocessing.md (#30677)
Michael Yao committed 8 months ago
fix minicpmo4.5: fix attn_mask in vit attn && fix resampler pos_emb i… (#34127)
tc-mb committed 8 months ago
docs(cpu): Clarify pre-built wheels requirement for CPU Python-only build (#35090)
Sage committed 8 months ago
[Doc] Add MTP docs and update speculative decoding guidance (#35197)
Xing Liu committed 8 months ago
fix(docs): use static rdzv backend in multi-node troubleshooting script (#34784)
Manrique Vargas committed 8 months ago
docs: add README for logits_processor examples (#35933)
Dr Alex Mitre committed 8 months ago
Fix phi4-mm and remove cuda binding (#35964)
Yan Ma committed 8 months ago
[Doc] Fix GPU Worker count in Process Count Summary (#36000)
simone-dotolo committed 8 months ago
Use MMEncoderAttention (=use FlashAttention) instead of torch.sdpa in radio.py (#35653)
Netanel Haber committed 8 months ago
Split generic IO Processor plugins tests from Terratorch specific ones (#35756)
Christian Pinto committed 8 months ago
[Misc] Add `--attention-backend auto` option (#35738)
Nicolò Lucchesi committed 8 months ago
[Core] Add All-to-All communication backend for DCP (#34883)
sungsoo ha committed 8 months ago
[Bugfix] Make `kaldi_native_fbank` optional (#35996)
Cyrus Leung committed 8 months ago
[BugFix] Fix implicit and incorrect assumption on ECConnector is_producer (#34783)
Qi Wang committed 8 months ago
[Bugfix][Model] Fix FP8 k_scale/v_scale not loaded for Qwen3-MoE (#35656)
Raghavan committed 8 months ago
Ronen Schaffer committed 8 months ago
[MISC] fixed tool_parser mypy errors (#35640)
Taneem Ibrahim committed 8 months ago
[Feature] Add basic metrics for /realtime endpoint (#35500)
pougetat committed 8 months ago