A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[Platform] Move get_punica_wrapper() function to Platform (#11516)
Shanshan Shen committed almost 2 years ago
[platform] add device_control env var (#12009)
youkaichao committed almost 2 years ago
[Doc] Organise installation documentation into categories and tabs (#11935)
Harry Mellor committed almost 2 years ago
[Doc][V1] Update model implementation guide for V1 support (#11998)
Roger Wang committed almost 2 years ago
[Kernel] unified_attention for Attention.forward (#11967)
Chen Zhang committed almost 2 years ago
Fix Max Token ID for Qwen-VL-Chat (#11980)
Alex Brooks committed almost 2 years ago
[platform] add ray_device_key (#11948)
youkaichao committed almost 2 years ago
[Misc]Minor Changes about Worker (#11555)
Chenguang Li committed almost 2 years ago
[Misc] Fix Deepseek V2 fp8 kv-scale remapping (#11947)
Concurrensee committed almost 2 years ago
[CI][Spec Decode] fix: broken test for EAGLE model (#11972)
Sungjae Lee committed almost 2 years ago
[Bug] Fix usage of `.transpose()` and `.view()` consecutively. (#11979)
Siyuan Li committed almost 2 years ago
[MISC] fix typo in kv transfer send recv test (#11983)
Yangcheng Li committed almost 2 years ago
[V1] [2/n] Logging and Metrics - `OutputProcessor` Abstraction (#11973)
Robert Shaw committed almost 2 years ago
[Model] Support GGUF models newly added in `transformers` 4.46.0 (#9685)
Isotr0py committed almost 2 years ago
[V1][Core][1/n] Logging and Metrics (#11962)
Robert Shaw committed almost 2 years ago
[Hardware][TPU] workaround fix for MoE on TPU (#11764)
Avshalom Manevich committed almost 2 years ago
[Hardware][CPU] Multi-LoRA implementation for the CPU backend (#11100)
Akshat Tripathi committed almost 2 years ago
[Model] Initialize support for Deepseek-VL2 models (#11578)
Isotr0py committed almost 2 years ago
[CI/Build] Add markdown linter (#11857)
Rafael Vasquez committed almost 2 years ago
[V1] Avoid sending text prompt to core engine (#11963)
Roger Wang committed almost 2 years ago