@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 15, 08:30 UTC

Code Commits 22,390 Issues Pulls Cloud Agent CICD
main
main

[Doc] Add community links (#18657)

Cyrus Leung committed over 1 year ago

6d166a8d35

[Doc] Fix indentation problems in V0 Paged Attention docs (#18659)

Cyrus Leung committed over 1 year ago

ef1dd6870f

[MISC][pre-commit] Add pre-commit check for triton import (#17716)

Mengqing Cao committed over 1 year ago

e77dc4bad8

[Doc] Update README links, mark external links (#18635)

Cyrus Leung committed over 1 year ago

07458a51ce

[V1][Spec Decode] Support multi-layer eagle draft model (#18030)

qizixi committed over 1 year ago

c1e4a4052d

[Model] Add support for Qwen2.5-Omni-7B-AWQ (Qwen2_5OmniForConditionalGeneration) (#18647)

Yuanhao WU committed over 1 year ago

a859320575

[Frontend] improve vllm serve --help display (#18643)

Reid committed over 1 year ago

441dc63ac7

[V1][Spec Decode] Small refactors to improve eagle bookkeeping performance (#18424)

qizixi committed over 1 year ago

d55e446d13

FIX MOE issue in AutoRound format (#18586)

Wenhua Cheng committed over 1 year ago

ec82c3e388

config.py: Clarify that only local GGUF checkpoints are supported. (#18623)

Mathieu Borderé committed over 1 year ago

45ab403a1f

[Bugfix][Nixl] Fix Preemption Bug (#18631)

Robert Shaw committed over 1 year ago

2b10ba7491

[Bugfix] Migrate to REGEX Library to prevent catastrophic backtracking (#18454)

Feng XiaoLong committed over 1 year ago

4fc1bf813a

[ModelOpt] Introduce VLLM_MAX_TOKENS_PER_EXPERT_FP4_MOE env var to control blockscale tensor allocation (#18160)

Pavani Majety committed over 1 year ago

f2036734fb

[Doc] Update references to doc files (#18637)

Cyrus Leung committed over 1 year ago

7d9216495c

[CI] Enable test_initialization to run on V1 (#16736)

Michael Goin committed over 1 year ago

0ddf88e16e

Use prebuilt FlashInfer x86_64 PyTorch 2.7 CUDA 12.8 wheel for CI (#18537)

Huy Do committed over 1 year ago

1645b60196

[V1] Support Deepseek MTP (#18435)

Jiayi Yao committed over 1 year ago

2628a69e35

[Doc] Fix broken links and unlinked docs, add shortcuts to home sidebar (#18627)

Cyrus Leung committed over 1 year ago

371f7e4ca2

[Doc] Avoid documenting dynamic / internal modules (#18626)

Cyrus Leung committed over 1 year ago

15b45ffb9a

[Doc] Fix top-level API links/docs (#18621)

Cyrus Leung committed over 1 year ago

273cb3b4d9
← Newer Page 785 of 1120 Older →