A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 03:30 UTC
main
test_attention compat with coming xformers change (#20487)
Jeremy Reizenstein committed over 1 year ago
[Misc] remove unused import (#20517)
Reid committed over 1 year ago
[Misc] Add logger.exception for TPU information collection failures (#20510)
Reid committed over 1 year ago
[v1] Re-add fp32 support to v1 engine through FlexAttention (#19754)
Isotr0py committed over 1 year ago
Fix unknown attribute of topk_indices_dtype in CompressedTensorsW8A8Fp8MoECutlassMethod (#20507)
Lucia Fang committed over 1 year ago
[Misc] Remove the unused LoRA test code (#20494)
Jee Jee Li committed over 1 year ago
Reid committed over 1 year ago
[Misc] Add security warning for development mode endpoints (#20508)
Reid committed over 1 year ago
[Doc] fix mutltimodal_inputs.md gh examples link (#20497)
Guy Stone committed over 1 year ago
[CI Bugfix] Fix pre-commit failures on main (#20502)
Michael Goin committed over 1 year ago
[feat]: CUTLASS block scaled group gemm for SM100 (#19757)
Duncan Moss committed over 1 year ago
Enable V1 for Hybrid SSM/Attention Models (#20016)
Thomas Parnell committed over 1 year ago
Chenheli Hua committed over 1 year ago
[CI] add kvcache-connector dependency definition and add into CI build (#18193)
Peter Pan committed over 1 year ago
[Doc] Fix classification table in list of supported models (#20489)
Cyrus Leung committed over 1 year ago
[Model][3/N] Automatic conversion of CrossEncoding model (#20168)
wang.yuqi committed over 1 year ago
[Bug][Frontend] Fix structure of transcription's decoder_prompt (#18809)
sangbumlikeagod committed over 1 year ago
Support Llama 4 for fused_marlin_moe (#20457)
Michael Goin committed over 1 year ago
[Misc] Add SPDX-FileCopyrightText (#20428)
Jee Jee Li committed over 1 year ago
[Perf] Optimize Vectorization Utils for Int 8 Quantization Kernels (#20331)
Wentao Ye committed over 1 year ago