A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 18:30 UTC
main
Add an example for reproducibility (#15262)
Woosuk Kwon committed over 1 year ago
[Misc] Better RayExecutor and multiprocessing compatibility (#14705)
Cody Yu committed over 1 year ago
[Docs] Trim the latest news in README (#15261)
Woosuk Kwon committed over 1 year ago
[Model] RE: Mamba2 Prefill Performance Tweaks: Fixing Flurry of Unnecessary Memory Copies (#14857)
Chih-Chieh Yang committed over 1 year ago
[Bugfix] detect alibi and revert to FA2 (#15231)
Travis Johnson committed over 1 year ago
[V1][TPU] Speed up top-k on TPU by using torch.topk (#15242)
Hyesoo Yang committed over 1 year ago
Mention `extra_body` as a way top pass vLLM only parameters using the OpenAI client (#15240)
Harry Mellor committed over 1 year ago
[Bugfix] Fix incorrect qwen2.5-vl attention mask pre-computation (#15200)
Isotr0py committed over 1 year ago
[ROCM] Upgrade torch to 2.6 (#15244)
Sage Moore committed over 1 year ago
[Misc] Clean up the BitsAndBytes arguments (#15140)
Jee Jee Li committed over 1 year ago
Lu Fang committed over 1 year ago
[CI/Build] LoRA : make add_lora_test safer (#15181)
Varun Sundar Rabindranath committed over 1 year ago
[V1] Scheduler Refactoring [1/N] - Add Scheduler Interface (#15250)
Woosuk Kwon committed over 1 year ago
Enforce that TP > 1 is not supported for Mamba2 if Quantization is Enabled. (#14617)
Yu Chin Fabian Lim committed over 1 year ago
[V1] Add flag to disable cascade attention (#15243)
Woosuk Kwon committed over 1 year ago
[Bugfix] fix V1 Engine crash while handling requests with duplicate request id (#15043)
Jason committed over 1 year ago
[ci] feat: make the test_torchrun_example run with tp=2, external_dp=2 (#15172)
Chi Zhang committed over 1 year ago
Replace `misc` issues with link to forum (#15226)
Harry Mellor committed over 1 year ago
Fix env vars for running Ray distributed backend on GKE (#15166)
Richard Liu committed over 1 year ago
Add user forum to README (#15220)
Harry Mellor committed over 1 year ago