A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
[Misc] Fix seq_lens for graph capture (#23175)
Woosuk Kwon committed about 1 year ago
Tialo committed about 1 year ago
[Misc] Avoid accessing req_ids inside a loop (#23159)
Woosuk Kwon committed about 1 year ago
[bug fix] Fix llama4 spec decoding (#22691)
qizixi committed about 1 year ago
[Bugfix] Fix broken Minimax-01-VL model (#22116)
Isotr0py committed about 1 year ago
[Misc] Add request_id into benchmark_serve.py (#23065)
hustxiayang committed about 1 year ago
Fix GLM-4.5V-FP8 numerical issue (#22949)
qizixi committed about 1 year ago
[Benchmark] Add flag --served-model-name to benchmark_serving_multi_turn (#22889)
Daniel Serebrenik committed about 1 year ago
[misc] split engine_model into json file for nsys profile tool (#23117)
Grace Ho committed about 1 year ago
[Model] Support Pipeline Parallelism for moonshotai/Kimi-VL-A3B-Thinking-2506 (#23114)
Jiangyun Zhu committed about 1 year ago
[Log] Warning Once for Cutlass MLA (#23137)
Wentao Ye committed about 1 year ago
Migrate InternVLImagePixelInputs (in nemotron_vl.py) to TensorSchema (#22023)
Benji Beck committed about 1 year ago
[Hardware][IBM Z]Enable v1 for s390x and s390x dockerfile fixes (#22725)
Nikhil Suryawanshi committed about 1 year ago
[TPU] make ptxla not imported when using tpu_commons (#23081)
Chengji Yao committed about 1 year ago
chore: disable enable_cpp_symbolic_shape_guards (#23048)
Xiao committed about 1 year ago
[V0 Deprecation] Remove V0 FlashInfer attention backend (#22776)
Woosuk Kwon committed about 1 year ago
[CI][V0 Deprecation] Removed V0 Only Chunked Prefill and Prefix Caching Tests (#22871)
Robert Shaw committed about 1 year ago
[Misc] Add @tdoublep as a maintainer of hybrid model and Triton-attention related code (#23122)
Thomas Parnell committed about 1 year ago
[Spec Decode] Make `propose_draft_token_ids` non-blocking for lower TTFT (#23041)
Woosuk Kwon committed about 1 year ago
[Misc] Minor refactoring for prepare_inputs (#23116)
Woosuk Kwon committed about 1 year ago