A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 5, 07:15 UTC
main
update CODEOWNERS file (#39439)
Chendi.Xue committed 7 months ago
[Perf] Reduce H2D pageable memory copies (#38794)
Chaofan Wang committed 7 months ago
[Core] Change max_model_len in EngineCoreReadyResponse to be non-None (#39442)
Nick Hill committed 7 months ago
Add platform manual_seed_all API (#38468)
Yan Ma committed 7 months ago
[BugFix][CPU] Add CPU profiler summary file output (#38366)
Elham committed 7 months ago
[Bugfix] Fix broken explicit unquantized kv cache dtype support (#38922)
Isotr0py committed 7 months ago
[feat]: make DCP error msg clearer (#28443)
Srreyansh Sethi committed 7 months ago
[KV Offload] Implement `shutdown()` in `OffloadingConnector` and related classes (#39182)
Ronen Schaffer committed 7 months ago
[Mergify] Update model vendor auto-label rules (#39312)
Cyrus Leung committed 7 months ago
Kyungmin Lee committed 7 months ago
[BugFix] Fix OOB read in CUTLASS grouped GEMM with epilogue (#38571)
Lucas Wilkinson committed 7 months ago
feat(cpu): add CPU support for draft model speculative decoding (#32662)
Ganesh R committed 7 months ago
[CT][FP8][Marlin] refactor CompressedTensorsW8A16Fp8 to use kernel abstraction (#38244)
Kunshang Ji committed 7 months ago
[Feature] Add auto-detection for reasoning_config when only reasoning_parser is set (#38214)
Chauncey committed 7 months ago
[ROCm] Fix AITER ops fake impl and minor bugs (#36092)
Chuan (Richard) Li committed 7 months ago
fix(gdn): Align prefill warmup with real prefill path (#39169)
Ibrahim Arshad committed 7 months ago
[Model][Perf] Enable checkpoints prefetching for Lustre FS by default (#39422)
Artem Perevedentsev committed 7 months ago
[LMCache] vLLM Block Allocation Event (#38856)
Yuwei An committed 7 months ago
[CI/Build[ Don't auto-rebase PRs with CI failures (#39443)
Cyrus Leung committed 7 months ago
[Spec Decode] fix returning size mismatch on extract hidden states proposer (#38610)
zzaebok committed 7 months ago