A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[TPU] Deprecate `xm.mark_step` in favor of ``torch_xla.sync` (#25254)
Nicolò Lucchesi committed about 1 year ago
[TPU][Bugfix][CI] Fix broken tests/build dependency (#25255)
Nicolò Lucchesi committed about 1 year ago
Enable Eagle3 speculative decoding for GPT-OSS model (#25246)
Eldar Kurtić committed about 1 year ago
[V0 Deprecation] Remove `MultiModalPlaceholderMap` (#25366)
Cyrus Leung committed about 1 year ago
[V0 Deprecation] Remove V0-only methods in multi-modal registry (#25362)
Cyrus Leung committed about 1 year ago
[Bugfix] Fix hermes tool parser handling of non-string argument types (#22002)
WeiQing Chen committed about 1 year ago
[Docs] GSM8K Accuracy Evaluation doc update (#25360)
WeiQing Chen committed about 1 year ago
[Model] Support Dots OCR (#24645)
Roger Wang committed about 1 year ago
Multimodal - audio tests (#25285)
Deboleina committed about 1 year ago
[Bugfix][V0 Deprecation][CI] use async mock and await for async method (#25325)
Yang Liu committed about 1 year ago
Remove V0 attention backends (#25351)
Woosuk Kwon committed about 1 year ago
[Perf] Further optimization for Qwen3-VL `fast_pos_embed_interpolate` (#25347)
Isotr0py committed about 1 year ago
[V0 Deprecation] Remove V0 sampling metadata (#25345)
Woosuk Kwon committed about 1 year ago
feat: Enable engine-level arguments with speculators models (#25250)
Rahul Tuli committed about 1 year ago
[V0 Deprecation] Remove async_output_proc, preemption mode, delay factor (#25334)
Woosuk Kwon committed about 1 year ago
[V0 Deprecation] Remove V0 Sequence class & Sampler (#25332)
Woosuk Kwon committed about 1 year ago
[Optimization] Cache chat template result when processor fails to be loaded (#25341)
Cyrus Leung committed about 1 year ago
[Bugfix] Typos in error message for missing model config file (#25339)
Simon Danielsson committed about 1 year ago
[MM][Perf] Minor Optimization on Qwen3-VL `fast_pos_embed_interpolate` (#25337)
Roger Wang committed about 1 year ago
[V1] Add sliding window support to Flex Attention backend (#24089)
Isotr0py committed about 1 year ago