@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 14, 05:30 UTC

Code Commits 22,390 Issues Pulls Cloud Agent CICD
main
main

[Mamba][KVCacheManager] Simplify kv cache manage logic for mamba + MTP (#25119)

Chen Zhang committed about 1 year ago

3d5f1c8640

EAGLE 3: Fix preamble so that measured speedup over Eagle 1 becomes 32% instead of 5% on MTBench (#25916)

Ekagra Ranjan committed about 1 year ago

1cab2f9cad

[Deepseek v3.2] Support indexer prefill chunking (#25999)

Chen Zhang committed about 1 year ago

1e50f1be70

[Small] Prevent bypassing media domain restriction via HTTP redirects (#26035)

Chenheli Hua committed about 1 year ago

ad87ba927a

[BugFix] Fix FI accuracy issue when used for MLA prefill (#26063)

Lucas Wilkinson committed about 1 year ago

decf7f794b

[CI/Build] Replace `vllm.entrypoints.openai.api_server` entrypoint with `vllm serve` command (#25967)

Cyrus Leung committed about 1 year ago

d00d652998

[CI] Add Blackwell DeepSeek FP8 FlashInfer MoE tests (#26040)

Michael Goin committed about 1 year ago

3b279a84be

[Qwen][ROCm] Flash Attention Rotary Embeddings (#24642)

vllmellm committed about 1 year ago

5e4a8223c6

[Platform][CI] Added OOT platform interface e2e test that running on Ascend NPU (#25470)

leo-pony committed about 1 year ago

e51de388a2

[Model] Use `merge_by_field_config` for MM models (D-F) (#26076)

Cyrus Leung committed about 1 year ago

cc253b73d3

[Model] Use `merge_by_field_config` for MM models (A-C) (#26073)

Cyrus Leung committed about 1 year ago

7d6fb905d9

[FA/Chore] Bump vllm-flash-attention (#25537)

Lucas Wilkinson committed about 1 year ago

418d111f8c

Change size of single CUDA graph for CI to 4 (#26089)

Thomas Parnell committed about 1 year ago

be8921fbba

Update base image to 22.04 (jammy) (#26065)

Huy Do committed about 1 year ago

d4e7a1152d

Run:ai model streamer add GCS package support (#24909)

pwschuurman committed about 1 year ago

be22bb6f3d

[Misc] Make handling of SamplingParams clearer in n>1 case (#26032)

Nick Hill committed about 1 year ago

169313b9f8

[ROCm][Bugfix] Add missing parameter to ROCm backend (#26029)

Gregory Shtrasberg committed about 1 year ago

0b018d8baf

Support RL online quantization with torchao (#23014)

Jerry Zhang committed about 1 year ago

c31246800c

[BugFix] ChunkedLocalAttention is currently not CG compatible (#26034)

Lucas Wilkinson committed about 1 year ago

4134312b35

[Bug] Fix Negative Cuda Memory Usage (#25683)

Wentao Ye committed about 1 year ago

da554f932e
← Newer Page 617 of 1120 Older →