A high-throughput and memory-efficient inference and serving engine for LLMs
main
[ROCm][BugFix] Triton W4A16 handling for GPTQ/AutoGPTQ qzeros layout (#47770)
Giuseppe Grossi committed 3 months ago
[CI][ROCm] Stabilize ci_base hash calculation and image handoff (#48746)
Andreas Karatzas committed 3 months ago
[Kernel][Helion] Add Helion kernel benchmark script (#48512)
Xiaohong (Sean) Chen committed 3 months ago
[Kernel][Helion] Helion kernel lazy registration (#48264)
Xiaohong (Sean) Chen committed 3 months ago
[Hardware][XPU] Register batch-invariant kernels for XPU (#41934)
Tomasz Zielinski committed 3 months ago
[ROCm] Re-enable cudagraph memory profiling, captured on the current stream (#48526)
peizhang56 committed 3 months ago
[Model] Add RobertaForTokenClassification / XLMRobertaForTokenClassification (#47991)
Gopala-Krishna Char committed 3 months ago
[KVOffload][P2P] Well-known default host/port env vars and per-DP-rank control port (#47636)
liranschour committed 3 months ago
[Test][kv_offload] Fix flaky drain() helper in test_fs_tier.py (#48545)
Chaojun Zhang committed 3 months ago
[Rust Frontend] Fix flaky `tls_handshake_timeout_drops_silent_client` test (#47873)
Tahsin Tunan committed 3 months ago
[Rust Frontend] Tolerate whitespace before the outer brace in JSON tool-call parsers (#48034)
Tahsin Tunan committed 3 months ago
Nicolò Lucchesi committed 3 months ago
[Bugfix] Fix parallel_tool_calls=null crash in Responses API from_request() (#48098)
Mahad Rehman committed 3 months ago
[Bugfix] Fix GLM5 config (#48711)
Jee Jee Li committed 3 months ago
[ROCm][Bugfix] Enable the fp32 head_dtype torch.mm fast path on ROCm (#48688)
Turner Jabbour committed 3 months ago
fix flaky multi example connector consistency (#48206)
Aarushi Jain committed 3 months ago
[ROCm] Add tuned selective_state_update config for AMD MI350 (#48159)
Giuseppe Grossi committed 3 months ago
[CI/Build] Split release artifact annotations by type (#48600)
Kevin H. Luu committed 3 months ago
[Rust Frontend] Integrate MM audio support (#48554)
Bugen Zhao committed 3 months ago
[ROCm][CI] fix test_common.py (#48676)
Charlie Fu committed 3 months ago