A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Model] Add support for ModernBertForTokenClassification (#26340)
antrec committed about 1 year ago
[Feature][OCP MX] Support mxfp6 and mixed mxfp6-mxfp4 (#21166)
fxmarty-amd committed about 1 year ago
[Model] Use `merge_by_field_config` for MM models (Ovis family) (#26308)
Isotr0py committed about 1 year ago
[Docs] add docs for cuda graph v1 (#24374)
fhl2000 committed about 1 year ago
Add TRL example notebook to RLHF docs (#26346)
Sergio Paniego Blanco committed about 1 year ago
Add topk logits torch op for DS3.2. (#25945)
Daniel Cámpora committed about 1 year ago
Optimize KV cache distribution for asymmetric pipeline parallelism (#25164)
Grant Holmes (Ren) committed about 1 year ago
[V0 Deprecation] Remove `VLLM_USE_V1` from docs and scripts (#26336)
Cyrus Leung committed about 1 year ago
Snehlata committed about 1 year ago
[Model] Define merge_by_field_config MM interface (R-T) (#26260)
Ayush Satyam committed about 1 year ago
[responsesAPI][bugfix] serialize harmony messages (#26185)
Andrew Xia committed about 1 year ago
[Deprecation] Deprecate `LLM.set_tokenizer` (#26333)
Cyrus Leung committed about 1 year ago
[Model] Define merge_by_field_config MM interface (U-Z) (#26261)
Ayush Satyam committed about 1 year ago
[BUG] Fix file parsing for load_format runai_streamer_sharded (#26324)
ahao-anyscale committed about 1 year ago
[UX] Support nested dicts in hf_overrides (#25727)
Michael Goin committed about 1 year ago
[Core] Simplify the Dp padding/should ubatch coordination logic (#25768)
Sage Moore committed about 1 year ago
[Docs][DBO] Add initial doc that describes the DBO implementation (#26024)
Sage Moore committed about 1 year ago
[Misc] Define EP kernel arch list in Dockerfile (#25635)
Simon Mo committed about 1 year ago
[Perf] Add decode full-graph support to FlashInfer-MLA backend (#26313)
Benjamin Chislett committed about 1 year ago
[ROCm] Split AITER unified attention into its own backend (#25507)
Gregory Shtrasberg committed about 1 year ago