A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 06:00 UTC
main
[Misc] Unify tokenizer registration (#29767)
Cyrus Leung committed 12 months ago
[crashfix] Eagle + multimodal can crash on mm cache miss (#29750)
Mickaël Seznec committed 12 months ago
[Frontend] Add tool filtering support to ToolServer (#29224)
daniel-salib committed 12 months ago
[Frontend] Resettle pooling entrypoints (#29634)
wang.yuqi committed 12 months ago
[CI] Skip paddleocr_vl for transformer 4.57.3 (#29758)
Huamin Li committed 12 months ago
Make PyTorch profiler gzip and CUDA time dump configurable (#29568)
Yifei Zhang committed 12 months ago
[MoE] CuteDSL MoE with Nvfp4 DeepEP dispatch (#27141)
Shu Wang committed 12 months ago
[Model Runner V2] Use packed mask for prompt bin counts (#29756)
Woosuk Kwon committed 12 months ago
[Misc]Remove redundant hidden_size property in ModelConfig (#29749)
Xingyu Liu committed 12 months ago
[Feat] Support non-gated activations in NVFP4 modelopt path (#29004)
Omer Ullman Argov committed 12 months ago
Fix RoPE failures in Transformers nightly (#29700)
Harry Mellor committed 12 months ago
[ROCm][Attention] Sliding window support for `AiterFlashAttentionBackend` (#29234)
Pleaplusone committed 12 months ago
[Core] Enable `inputs_embeds_size` separate from `hidden_size` (#29741)
Cyrus Leung committed 12 months ago
[Bugfix] Fix mismatched nvfp4 gemm output shape (#29742)
Isotr0py committed 12 months ago
[Misc] Update `TokenizerLike` interface and move `get_cached_tokenizer` (#29730)
Cyrus Leung committed 12 months ago
[Doc]: Fix typo in fused_moe layer (#29731)
朝 committed 12 months ago
[Bugfix][sleepmode][fp8 kv cache]: Fix FP8 KV cache + sleep(level=2) gibberish output (#28783)
Vensen committed 12 months ago
Fix AttributeError about _use_fi_prefill (#29734)
Huamin Li committed 12 months ago
[Quantization] Enable compressed-tensors AWQ for Turing GPU (#29732)
Isotr0py committed 12 months ago
[LoRA] Support FusedMoE LoRA Triton kernel for mxfp4 (#29708)
Xin Yang committed 12 months ago