A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 6, 02:00 UTC
main
[Mypy] Fix mypy for `vllm/model_executor` (except `vllm/model_executor/layers`) (#37904)
Harry Mellor committed 8 months ago
[Core] add option to schedule requests based on full ISL (#37307)
Dan Blanaru committed 8 months ago
[Feature] limit thinking tokens (hard limit) (#20859)
Sungjae Lee committed 8 months ago
Update new contributor message (#37999)
Harry Mellor committed 8 months ago
[Deprecate] Deprecate pooling multi task support. (#37956)
wang.yuqi committed 8 months ago
[Bugfix] Add replacement of _compute_slot_mapping_kernel on CPU (#37987)
Li, Jiang committed 8 months ago
[Bugfix] Suppress spurious CPU KV cache warning in `launch render` (#37911)
Sage committed 8 months ago
[EPLB] Remove main waits in case of slow EPLB (#36271)
Ilya Markov committed 8 months ago
Harry Mellor committed 8 months ago
Fix tool_parser_cls type annotation from Callable to type[ToolParser] (#37957)
Flora Feng committed 8 months ago
Ronen Schaffer committed 8 months ago
[Frontend][Bugfix] Pass default_chat_template_kwargs to AnthropicServingMessages (#37899)
jetxa committed 8 months ago
[release] Move agent queue to Release cluster queues (#37783)
Kevin H. Luu committed 8 months ago
Downsize CPU jobs to use small queue (#37913)
Kevin H. Luu committed 8 months ago
[V0 Deprecation] Refactor kv cache from list to element (#37487)
Wentao Ye committed 8 months ago
[ROCm][CI] Split Entrypoints Integration (API Server 1) into 3 jobs (#37906)
Andreas Karatzas committed 8 months ago
[CI] Add batch invariant test: Block FP8 + small MOE (#37895)
Wentao Ye committed 8 months ago
[Model Runner V2] Gather multimodal embeddings before draft model postprocess (#37932)
Giancarlo Delfin committed 8 months ago
[Test] E2E Nemotron-3-Super tests (#36803)
roikoren755 committed 8 months ago
[CI] Split V1 Others into 3 separate jobs (#37016)
Kevin H. Luu committed 8 months ago