A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 15:45 UTC
main
[Misc] Support FP8 MoE for compressed-tensors (#8588)
Michael Goin committed about 2 years ago
[Frontend] MQLLMEngine supports profiling. (#8761)
科英 committed about 2 years ago
rename PromptInputs and inputs with backward compatibility (#8760)
Cyrus Leung committed about 2 years ago
[VLM][Bugfix] internvl with num_scheduler_steps > 1 (#8614)
DefTruth committed about 2 years ago
[[Misc]] Add extra deps for openai server image (#8792)
Jee Jee Li committed about 2 years ago
[Kernel] Fullgraph and opcheck tests (#8479)
bnellnm committed about 2 years ago
[CI/Build][Bugfix][Doc][ROCm] CI fix and doc update after ROCm 6.2 upgrade (#8777)
Hongxia Yang committed about 2 years ago
[Misc] Fix minor typo in scheduler (#8765)
Woo-Yeon Lee committed about 2 years ago
[Bugfix] Ray 2.9.x doesn't expose available_resources_per_node (#8767)
David Newman committed about 2 years ago
[Frontend] OpenAI server: propagate usage accounting to FastAPI middleware layer (#8672)
Adam Tilghman committed about 2 years ago
[Bugfix] load fc bias from config for eagle (#8790)
sohamparikh committed about 2 years ago
[Hardware][CPU] Enable mrope and support Qwen2-VL on CPU backend (#8770)
Isotr0py committed about 2 years ago
[BugFix] Propagate 'trust_remote_code' setting in internvl and minicpmv (#8250)
zifeitong committed about 2 years ago
Fix tests in test_chunked_prefill_scheduler which fail with BlockManager V2 (#8752)
sroy745 committed about 2 years ago
[Bugfix][Kernel] Implement acquire/release polyfill for Pascal (#8776)
sasha0552 committed about 2 years ago
Fix test_schedule_swapped_simple in test_scheduler.py (#8780)
sroy745 committed about 2 years ago
[Bugfix] Use heartbeats instead of health checks (#8583)
Joe Runde committed about 2 years ago
[Core] Adding Priority Scheduling (#5958)
Archit Patke committed about 2 years ago
[Core][Bugfix] Support prompt_logprobs returned with speculative decoding (#8047)
Travis Johnson committed about 2 years ago
[[Misc]Upgrade bitsandbytes to the latest version 0.44.0 (#8768)
Jee Jee Li committed about 2 years ago