A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 11, 15:00 UTC
main
[Bugfix] Fix `task` still being passed in tests/benchmarks (#30476)
Cyrus Leung committed 11 months ago
[Deprecation] Deprecation `--convert reward`, use `--convert embed` instead. (#30463)
wang.yuqi committed 11 months ago
[Fix] Update lazing loading of video loader backend (#30444)
jeremyteboul committed 11 months ago
[Deprecation] Remove missed fallback for `embed_input_ids` (#30469)
Cyrus Leung committed 11 months ago
[Fix]fix import error from lmcache (#30376)
wz1qqx committed 11 months ago
[bugfix] fix MiniMaxM2ReasoningParser streaming output not separating reasoning_content. (#29882)
Rei. committed 11 months ago
[perf] Use direct copy (broadcast) instead of cat for k_nope/k_pe in MLA prefill (#29710)
Ming Yang committed 11 months ago
fix: enhance human_readable_int function (#30337)
Ning Xie committed 11 months ago
[Deprecation] Remove fallbacks for `embed_input_ids` and `embed_multimodal` (#30458)
Cyrus Leung committed 11 months ago
Ensure minimum frames for GLM 4.6V compatibility (#30285)
gh-wf committed 11 months ago
[Doc] Add Baidu Kunlun XPU support (#30455)
xyDong0223 committed 11 months ago
[Chore] Fix torch precision warning (#30428)
Wentao Ye committed 11 months ago
[Deprecation] Remove deprecated task, seed and MM settings (#30397)
Cyrus Leung committed 11 months ago
[Deprecation] Remove deprecated plugin and compilation fields for v0.13 release (#30396)
Cyrus Leung committed 11 months ago
[Bugfix] Fix grouped_topk pytorch impl when num_experts can't be grouped properly (#29439)
Divakar Verma committed 11 months ago
[ROCm] Fix broken import in platform attention backend dispatching (#30432)
Andreas Karatzas committed 11 months ago
Revert "[CI] Add Async Eplb nightly CI tests (#29385)" (#30431)
Sage Moore committed 11 months ago
Add more docs for regex (#30106)
Xu Song committed 11 months ago
[Bugfix] fix confusing OOM errors during v1 init (#28051)
shivampr committed 11 months ago
fix(shm): Add memory barriers for cross-process shared memory visibility (#30407)
Christina Norman committed 11 months ago