A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
main
Chauncey committed about 1 year ago
Remove LoRA bias support (#25807)
Ashwin Phadke committed about 1 year ago
[GPT-OSS] Add support for arrays at tool message content (#25593)
Luis Tomas Bolivar committed about 1 year ago
[Metrics] Log multi-modal cache stats and fix reset (#26285)
Cyrus Leung committed about 1 year ago
[deepseek] kernel block size for UniformTypeKVCacheSpecs (#26559)
Chen Zhang committed about 1 year ago
fix test_simple_inductor_graph_partition (#26522)
Boyuan Feng committed about 1 year ago
[Spec-Decode] Support piecewise cudagraphs for Eagle head (#25109)
Lucas Wilkinson committed about 1 year ago
[Chore]: One pythonic tool parser test uses the wrong parser (#26515)
Ben Browning committed about 1 year ago
[Core] Small simplification in `GPUModelRunner._update_states()` (#26508)
Nick Hill committed about 1 year ago
[DP][ray] Support different VLLM_RAY_DP_PACK_STRATEGY (#23849)
Rui Qiao committed about 1 year ago
Refactor MistralTokenizer (#26358)
Julien Denize committed about 1 year ago
[CI] Fix Pre-commit Issue Cannot determine type of "rank" and "world_size" (#26448)
Wentao Ye committed about 1 year ago
[Bug] Fix modular_kernel: ZeroDivisionError: integer division or modulo by zero (#26528)
Wentao Ye committed about 1 year ago
[Bugfix] Fix CUDA graph selection bug in FlashInfer at high concurrency (#26499)
Benjamin Chislett committed about 1 year ago
[UX] Add FlashInfer as default CUDA dependency (#26443)
Michael Goin committed about 1 year ago
[Core] Remove unused `prev_sampled_token_ids_invalid_indices` input batch field (#26514)
Nick Hill committed about 1 year ago
[Flashinfer][gpt-oss] Support FP8-qkv Flashinfer TRTLLM Sinks Attention (#25674)
elvischenv committed about 1 year ago
[Bugfix] Disable moe inplace for torch >= 2.9 (#26497)
bnellnm committed about 1 year ago
[Bugfix] Enable padded FP4 quantization (#25947)
roikoren755 committed about 1 year ago
[BUGFIX] Add cu_tokens_across_sp to DPMetadata (#26457)
Sage Moore committed about 1 year ago