A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 03:15 UTC
main
[distributed] remove pynccl's redundant change_state (#11749)
cennn committed almost 2 years ago
[Frontend] Improve `StreamingResponse` Exception Handling (#11752)
Robert Shaw committed almost 2 years ago
Lancer committed almost 2 years ago
[distributed] remove pynccl's redundant stream (#11744)
cennn committed almost 2 years ago
[MISC] Replace c10::optional with std::optional (#11730)
Lu Fang committed almost 2 years ago
[Bugfix][V1] Fix test_kv_cache_utils.py (#11738)
Jee Jee Li committed almost 2 years ago
[Model] Remove unnecessary weight initialization logic (#11736)
Cyrus Leung committed almost 2 years ago
[Bugfix] Fix precision error in LLaVA-NeXT (#11735)
Cyrus Leung committed almost 2 years ago
[VLM] Merged multi-modal processors for LLaVA-NeXT-Video and LLaVA-OneVision (#11717)
Cyrus Leung committed almost 2 years ago
[Core][Bugfix] Use correct device to initialize GPU data during CUDA-graph-capture (#11233)
Yan Burman committed almost 2 years ago
[V1] Add kv cache utils tests. (#11513)
xcnick committed almost 2 years ago
[V1] Add `RayExecutor` support for `AsyncLLM` (api server) (#11712)
Kunshang Ji committed almost 2 years ago
Update bnb.md with example for OpenAI (#11718)
Alberto Ferrer committed almost 2 years ago
Update tool_calling.md (#11701)
Hust_YangXian committed almost 2 years ago
[V1] log GPU blocks num for MultiprocExecutor (#11656)
WangErXiao committed almost 2 years ago
[V1] Chore: cruft removal (#11724)
Robert Shaw committed almost 2 years ago
Update requirements-tpu.txt to support python 3.9 and 3.11 (#11695)
Michael Goin committed almost 2 years ago
[Misc]Add BNB quantization for Qwen2VL (#11719)
Jee Jee Li committed almost 2 years ago
[V1] Improve TP>1 Error Handling + Stack Trace (#11721)
Robert Shaw committed almost 2 years ago
[Bugfix] Fix ColumnParallelLinearWithLoRA slice (#11708)
ZincCat committed almost 2 years ago