A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[Feature][Frontend]: Continued `stream_options` implementation also in CompletionRequest (#5319)
Itay Etelis committed over 2 years ago
[Model] Initial support for LLaVA-NeXT (#4199)
Cyrus Leung committed over 2 years ago
[Misc] Improve error message when LoRA parsing fails (#5194)
Cyrus Leung committed over 2 years ago
youkaichao committed over 2 years ago
[Frontend][Misc] Enforce Pixel Values as Input Type for VLMs in API Server (#5374)
Roger Wang committed over 2 years ago
[Misc] Update to comply with the new `compressed-tensors` config (#5350)
Dipika Sikka committed over 2 years ago
[Bugfix] Fix KeyError: 1 When Using LoRA adapters (#5164)
Bla_ckB committed over 2 years ago
[Kernel][Misc] Use TORCH_LIBRARY instead of PYBIND11_MODULE for custom ops (#5047)
bnellnm committed over 2 years ago
[mis][ci/test] fix flaky test in test_sharded_state_loader.py (#5361)
youkaichao committed over 2 years ago
[Core][CUDA Graph] add output buffer for cudagraph (#5074)
youkaichao committed over 2 years ago
[Misc][Breaking] Change FP8 checkpoint format from act_scale -> input_scale (#5353)
Michael Goin committed over 2 years ago
[CI/Test] improve robustness of test (vllm_runner) (#5357)
youkaichao committed over 2 years ago
[CI/Test] improve robustness of test (hf_runner) (#5347)
youkaichao committed over 2 years ago
[ROCm][AMD] Use pytorch sdpa math backend to do naive attention (#4965)
Hongxia Yang committed over 2 years ago
[Misc] Add args for selecting distributed executor to benchmarks (#5335)
Benjamin Kitor committed over 2 years ago
[Bug Fix] Fix the support check for FP8 CUTLASS (#5352)
Cheng Li committed over 2 years ago
fix DbrxFusedNormAttention missing cache_config (#5340)
Calvinn Ng committed over 2 years ago
[Misc] Remove unused cuda_utils.h in CPU backend (#5345)
Jie Fu (傅杰) committed over 2 years ago
[Frontend] Add OpenAI Vision API Support (#5237)
Roger Wang committed over 2 years ago
[Kernel] Dynamic Per-Token Activation Quantization (#5037)
Dipika Sikka committed over 2 years ago