A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 06:00 UTC
main
[Hardware][POWER] Add IBM POWER11 Support to CPU Extension Detection (#19082)
Akash kaothalkar committed over 1 year ago
Add H20-3e fused MoE kernel tuning configs for Qwen3-235B-A22B (#19315)
Xu Wenqing committed over 1 year ago
[Misc] Change tests/compile to use VLLM_V1 by default (#19302)
Richard Zou committed over 1 year ago
[Bugfix] Re-enable use_cudagraph in vLLM v1 (#19299)
Richard Zou committed over 1 year ago
[Bugfix]: Fix TypeError: 'float' object cannot be interpreted as an integer (#19283)
Chauncey committed over 1 year ago
[BugFix][V1] Fix memory profiling bug (#18974)
Luka Govedič committed over 1 year ago
[AMD] Update compatible packaging version (#19309)
pramenku committed over 1 year ago
[Nit][Benchmark]Fix example in benchmark_serving_structured_output.py (#19311)
Lifans committed over 1 year ago
[CI/Build] Improve Llama GGUF test robustness (#19287)
Isotr0py committed over 1 year ago
[Misc] refactor context extension (#19246)
Reid committed over 1 year ago
Add FlexAttention to V1 (#16078)
Driss Guessous committed over 1 year ago
Aaruni Aggarwal committed over 1 year ago
[TPU][Test] Add script to run benchmark on TPU for buildkite (#19039)
QiliangCui committed over 1 year ago
[Kernel] Integrate CUTLASS MoE kernel with PPLX (#18762)
ElizaWszola committed over 1 year ago
[Easy][Test] Simplify test_function_tool_use with multiple parametrizes (#19269)
Lu Fang committed over 1 year ago
[Build][ROCm] Update Dockerfile.rocm (#19296)
Alexei-V-Ivanov-AMD committed over 1 year ago
[Misc][Tools][Benchmark] Fix and improve auto tune script (#19163)
Chenyaaang committed over 1 year ago
[BugFix] Fix tpu_model_runner block_id concatenation (#19228)
Nick Hill committed over 1 year ago
[Core] Fix abrupt request abort (#18485)
Nicolò Lucchesi committed over 1 year ago
[Bugfix][Core] Update cancellation logic in `generate()` to handle Generator exits (#19225)
Adolfo Victoria committed over 1 year ago