@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 8, 21:30 UTC

Code Commits 22,390 Issues Pulls Cloud Agent CICD
main
main

[Core] Don't schedule spec tokens with prefill chunks (#33652)

Nick Hill committed 9 months ago

fa4e0fb028

Change the type signature of MixtureOfExperts.expert_weights to MutableSequence[Sequence[Tensor]] (#33573)

Sage Moore committed 9 months ago

ce498a6d61

Revert "[torch.compile] Significantly speed up cold start times" (#33820)

Richard Zou committed 9 months ago

9f14c9224d

[Model] Add transcription support for Qwen3-Omni (#29828)

Muhammad Hashmi committed 9 months ago

535de06cb1

[Bugfix] Support `RotaryEmbedding` CustomOp for gpt-oss (#33800)

Simon Danielsson committed 9 months ago

4292c90a2a

Implement zero-copy GQA for multimodal and CPU (#33732)

Taeksang Kim committed 9 months ago

6e98f6d8b6

[rocm][ray] Fix: Unify Ray device visibility handling across CUDA and ROCm (#33308)

kourosh hakhamaneshi committed 9 months ago

2f6d17cb2f

[Bugfix] Fix interns1-pro initialization and PP (#33793)

Isotr0py committed 9 months ago

192ad4648b

[Misc] Delay deprecation of CommonAttentionMetadata properties (#33801)

Lucas Wilkinson committed 9 months ago

0e92298622

[Bugfix] Fix ubatch wrapper num_tokens calculate (#33694)

jiangkuaixue123 committed 9 months ago

87d9a26166

[Bugfix] Fix `normalize` still being passed to `PoolerConfig` (#33794)

Cyrus Leung committed 9 months ago

80f921ba4b

[Perf] Optimize spec decoding + async scheduling, 1.5% Throughput improvement (#33612)

Wentao Ye committed 9 months ago

711edaf0d0

[Bugfix][ROCm] Include float8_e4m3fnuz in NCCL Dtype Dispatching (#33713)

Micah Williamson committed 9 months ago

1d367a738e

Apply #33621 to main (#33758)

Cyrus Leung committed 9 months ago

32a02c7ca2

[Perf] Optimize chat completion streaming performance (#33782)

Chauncey committed 9 months ago

f67ee8b859

[Model] Apply #32631 for recent models (#33785)

Cyrus Leung committed 9 months ago

e57ef99b40

[Bugfix][Model] Fix audio-in-video support for Qwen2.5-Omni and Qwen3-Omni (#33605)

Yueqian Lin committed 9 months ago

f8516a1ab9

[PERF] Change GDN Attention State Layout from [N, HV, K, V] to [N, HV, V, K] (#33291)

Vadim Gimpelson committed 9 months ago

824058076c

[KV Connector][BugFix] scheduler: Delay freeing blocks of aborted async loads (#32255)

Or Ozeri committed 9 months ago

8e32690869

[compile] Remove runner type from ignored caching factor list. (#33712)

Zhengxu Chen committed 9 months ago

a208439537
← Newer Page 440 of 1120 Older →