A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 23:30 UTC
main
[misc] Add retries with exponential backoff for HF file existence check (#13008)
Kevin H. Luu committed almost 2 years ago
[Doc] Add link to tool_choice tracking issue in tool_calling.md (#13003)
Yuan Tang committed almost 2 years ago
[core] fix sleep mode and pytorch checkpoint compatibility (#13001)
youkaichao committed almost 2 years ago
Check if selected backend is None in get_attn_backend_cls() (#12975)
Yuan Tang committed almost 2 years ago
[V1] Use msgpack for core request serialization (#12918)
Nick Hill committed almost 2 years ago
[core][rlhf] add colocate example for RLHF (#12984)
youkaichao committed almost 2 years ago
[core] improve error handling when wake up from sleep mode (#12981)
youkaichao committed almost 2 years ago
[MISC] Always import version library first in the vllm package (#12979)
Lu Fang committed almost 2 years ago
[core] port pynvml into vllm codebase (#12963)
youkaichao committed almost 2 years ago
[V1] Cache `uses_mrope` in GPUModelRunner (#12969)
Woosuk Kwon committed almost 2 years ago
[RFC] [Mistral] FP8 format (#10130)
Patrick von Platen committed almost 2 years ago
[V1][Minor] Remove outdated comment (#12968)
Woosuk Kwon committed almost 2 years ago
[VLM] Merged multi-modal processor for GLM4V (#12449)
Jee Jee Li committed almost 2 years ago
[bugfix] fix early import of flash attention (#12959)
youkaichao committed almost 2 years ago
[Bugfix] Remove unused seq_group_metadata_list from ModelInputForGPU (#12935)
shangmingc committed almost 2 years ago
[CI/Build] Auto-fix Markdown files (#12941)
Cyrus Leung committed almost 2 years ago
[Misc] Add qwen2.5-vl BNB support (#12944)
Isotr0py committed almost 2 years ago
[Doc] Correct HF repository for TeleChat2 models (#12949)
Jun Duan committed almost 2 years ago
[CI] Resolve transformers-neuronx version conflict (#12925)
Liangfu Chen committed almost 2 years ago
[Misc] Log time consumption on weight downloading (#12926)
Jun Duan committed almost 2 years ago