A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Document] Add ms-swift library to rlhf.md (#27469)
jinghanhu committed about 1 year ago
[CI/Build] Fix test_torch_utils in AMD CI (#27317)
Zhewen Li committed about 1 year ago
[Bugfix] Fix interns1-vit qk norm code path (#27480)
Isotr0py committed about 1 year ago
[Attention] Add MLA prefill backend: trtllm_ragged_attention_deepseek (#26397)
Ming Yang committed about 1 year ago
[Bugfix] Fix MultiConnector stats reconstruction across process boundaries (#27366)
kourosh hakhamaneshi committed about 1 year ago
[NIXL][BUGFIX] delay done_recving queue cleanup to bottom of get_finished (#27297)
Chendi.Xue committed about 1 year ago
[compile] Turn standalone_compile back on (#27460)
Richard Zou committed about 1 year ago
[cpu][fix] Fix onednn_mm crash on consecutive matmuls with same M,K,N and different dtype (#27472)
Fadi Arafeh committed about 1 year ago
[Doc] Fix minor issues in docs/design/metrics.md (#27436)
Lifans committed about 1 year ago
Fix test named tool use (#27458)
Chauncey committed about 1 year ago
[MISC] `cudagraph_capture_sizes` related improvements (#26016)
fhl2000 committed about 1 year ago
Fix AArch64 CPU Docker pipeline (#27331)
ioana ghiban committed about 1 year ago
[Benchmark] Enable benchmark to run with `encoding_format="bytes"` (#27467)
Cyrus Leung committed about 1 year ago
[Refactor] move tool parsing logic from protocol.py to the tool parser (#27383)
Chauncey committed about 1 year ago
[BugFix] Fix torchrun DP with LLM class (#27395)
22quinn committed about 1 year ago
[MM][Bugfix] Replace `PatchEmbed`'s conv3d to linear layer (#27418)
Isotr0py committed about 1 year ago
[Docs] remove v1 column for embedding models (#27446)
Yu Jiaqi committed about 1 year ago
[Misc] Avoid "PyTorch non-writable tensors" warning in RayPPCommunicator (#27443)
Rui Qiao committed about 1 year ago
[Bugfix] Fix Pydantic union resolution for ResponseFunctionToolCall in Responses API (#26706)
strinczer committed about 1 year ago
[Chore] remove structural tags logging lines (#27451)
Aaron Pham committed about 1 year ago