A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 2, 20:45 UTC
main
[Misc] Print accuracy value for PD tests even on success (#43583)
Nicolò Lucchesi committed 5 months ago
[KV Connector] Handle Mooncake finish after preemption (#43281)
Zhewen Li committed 5 months ago
Reduce memory usage for granite_speech. (#42933)
Yihuki committed 5 months ago
[Doc] Add section on escalating stalled contributions (#43568)
Roy Wang committed 5 months ago
[Feat][KVConnector] Support DSV4 in SimpleCPUOffloadBackend (#42296)
Yifan Qiao committed 5 months ago
[Docker] Non-root support for vllm-openai; add opt-in vllm-openai-nonroot target (#40275)
Nguyễn Thế Duy committed 5 months ago
[Docs] Reorganize offline inference docs. (#43552)
wang.yuqi committed 5 months ago
fix: MoE model using shared routed experts crashes on AMD GPUs (#42373)
weizhoublue committed 5 months ago
[Kernel] Add mhc_pre_big_fuse_with_norm_tilelang (#43474)
Jee Jee Li committed 5 months ago
File system secondary tier implemented in python (#41735)
Rotem Shavitt committed 5 months ago
Tuning script and configs for Triton Mamba SSU kernel (#43083)
danisereb committed 5 months ago
[ROCm] [DSv4] [Perf] Support DeepSeek v4 MTP (#43385)
TJian committed 5 months ago
[ROCm][CI] Stabilize 400 error return code for invalid schema inputs (#43016)
Andreas Karatzas committed 5 months ago
[kv_offload]: Add DSv4 support (#43142)
Or Ozeri committed 5 months ago
[KV Connector] Keep MooncakeStore full hits block-aligned (#43494)
Dao007forever committed 5 months ago
[Model Runner v2] Force v1 runner for tests (#43233)
Wentao Ye committed 5 months ago
[Bugfix] Fix reasoning dropped on streaming boundary deltas (#42691)
Flora Feng committed 5 months ago
Revert "[Misc] add humming to dependencies" (#43492)
Michael Goin committed 5 months ago
[ROCm][Critical] Fix the GDN import bug (#43486)
TJian committed 5 months ago
[Mooncake] Add metrics for MooncakeStoreConnector operations (#43392)
Dao007forever committed 5 months ago