[Spec Decode] Update extract_hidden_states to use deferred kv_connector clear (#37013)

Fynn Schmitt-Ulms committed 8 months ago

Remove unused EVS functions in qwen3_vl.py (#37183)

Tianyu Guo committed 8 months ago

Patch Mistral config (#37104)

Julien Denize committed 8 months ago

[Models][Qwen3 ViT] Keep `max_seqlen` on CPU to prevent D2H sync (#37139)

Lukas Geiger committed 8 months ago

[Build] Fix API rate limit exceeded when using `VLLM_USE_PRECOMPILED=1` (#36229)

elvischenv committed 8 months ago

[Performance] Add prefetch for checkpoints to OS page cache (#36012)

Artem Perevedentsev committed 8 months ago

[Docs] Make the link to hardware plugins clearer (#37174)

Harry Mellor committed 8 months ago

GLM4 tool parser: fix streaming mode (#35208)

Robin Nabel committed 8 months ago

Fix text only inputs for MRoPE models with the Transformers modelling backend (#37055)

Harry Mellor committed 8 months ago

[Hardware] Replace memory related torch.cuda APIs (#37031)

Kunshang Ji committed 8 months ago

Fix pipeline parallel with multimodal models with the Transformers modelling backend (#37057)

Harry Mellor committed 8 months ago

[Misc] Sync pre-commit to 4.5.1 in workflows and docs (#36675)

SoluMilken committed 8 months ago

[Frontend][Misc] Remove unused log in `/is_sleeping` (#37093)

Roy Wang committed 8 months ago

[Bugfix] Fix GDN attention crash with mixed decode/spec-decode batches (#34871)

haosdent committed 8 months ago

[FlashInfer] Revert block_size 16 + head_size 256 workaround on Blackwell (#36987)

Vadim Gimpelson committed 8 months ago

[Bugfix] Fix Qwen2.5-Omni/Qwen3-Omni use_audio_in_video with multi-video inputs (#37147)

Isotr0py committed 8 months ago

use skip_all_guards_unsafe to drop global_state and torch_function_mode_stack guards instead of previous hacks (#36204)

Laith Sakka committed 8 months ago

[Performance][Model Loader] Skip non-local expert weights during EP model loading (#37136)

Roy Wang committed 8 months ago

[ROCm][CI] Retrying in case of batch variance effects and reducing flakiness (#36442)

Andreas Karatzas committed 8 months ago

[ROCm] Fix KV copy methods and auto-select attention backend for ROCm (#36845)

Andreas Karatzas committed 8 months ago