A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[Doc] Convert list tables to MyST (#11594)
Cyrus Leung committed almost 2 years ago
[V1] [5/N] API Server: unify `Detokenizer` and `EngineCore` input (#11545)
Robert Shaw committed almost 2 years ago
[bugfix] interleaving sliding window for cohere2 model (#11583)
youkaichao committed almost 2 years ago
[Doc] Minor documentation fixes (#11580)
Cyrus Leung committed almost 2 years ago
[V1][Minor] Set pin_memory=False for token_ids_cpu tensor (#11581)
Woosuk Kwon committed almost 2 years ago
[Hardware][AMD]: Replace HIPCC version with more precise ROCm version (#11515)
hj-wei committed almost 2 years ago
[Model] Remove hardcoded image tokens ids from Pixtral (#11582)
Roger Wang committed almost 2 years ago
[Model] Support InternLM2 Reward models (#11571)
Isotr0py committed almost 2 years ago
[Bugfix] Last token measurement fix (#11376)
Rajveer Bachkaniwala committed almost 2 years ago
[V1] [4/N] API Server: ZMQ/MP Utilities (#11541)
Robert Shaw committed almost 2 years ago
[Doc] Update mllama example based on official doc (#11567)
Chen Zhang committed almost 2 years ago
[Bugfix] Fix for ROCM compressed tensor support (#11561)
Selali committed almost 2 years ago
[Misc] Improve BNB loader to handle mixture of sharded and merged weights with same suffix (#11566)
Isotr0py committed almost 2 years ago
[Misc]Add BNB quantization for MolmoForCausalLM (#11551)
Jee Jee Li committed almost 2 years ago
[MODEL] LoRA support for Jamba model (#11209)
ErezSC42 committed almost 2 years ago
[VLM] Support caching in merged multi-modal processor (#11396)
Cyrus Leung committed almost 2 years ago
[Doc] Add xgrammar in doc (#11549)
Chen1022 committed almost 2 years ago
[Misc] Abstract the logic for reading and writing media content (#11527)
Cyrus Leung committed almost 2 years ago
[Bugfix] Fix TeleChat2ForCausalLM weights mapper (#11546)
Jee Jee Li committed almost 2 years ago
Update deploying_with_k8s.md with AMD ROCm GPU example (#11465)
AlexHe99 committed almost 2 years ago