A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 21:00 UTC
main
[Model] New model support for Phi-4-multimodal-instruct (#14119)
Congcong Chen committed over 1 year ago
[V1][Bugfix] Do not reset prefix caching metrics (#14235)
Cody Yu committed over 1 year ago
[Bugfix] Fix gptq_marlin for deepseek-v3 (#13750)
rainkert committed over 1 year ago
Disable GPTQ AllSpark kernels for CUDA Compiler < 12.0 (#14157)
Michael Goin committed over 1 year ago
Moved numba from common requirements to cuda/rocm specific requirements (#14199)
Nishidha committed over 1 year ago
[misc] announce china meetup (#14248)
youkaichao committed over 1 year ago
[V1][TPU] TPU multimodal model support for ragged attention (#14158)
Michael Goin committed over 1 year ago
[ROCm] Disable a few more kernel tests that are broken on ROCm (#14145)
Sage Moore committed over 1 year ago
Clean up unused padding_idx variables across many model definitions (#13240)
Tyler Michael Smith committed over 1 year ago
[Security] Serialize using safetensors instead of pickle in Mooncake Pipe (#14228)
Kuntai Du committed over 1 year ago
[v1][Metrics] Add design doc (#12745)
Mark McLoughlin committed over 1 year ago
[Docs] Update Dockerfile dependency image (#14215)
Michael Goin committed over 1 year ago
[Frontend] Do `prompt_logprobs` clamping for chat as well as completions (#14225)
Harry Mellor committed over 1 year ago
Fix performance when `--generation-config` is not `None` (#14223)
Harry Mellor committed over 1 year ago
[TPU][Profiler] Support start_profile/stop_profile in TPU worker (#13988)
Siyuan Liu committed over 1 year ago
add cutlass support for blackwell fp8 gemm (#13798)
kushanam committed over 1 year ago
[V1][Molmo] Fix get_multimodal_embeddings() in molmo.py (#14161)
lkchen committed over 1 year ago
[V0][Metrics] Deprecate some questionable request time metrics (#14135)
Mark McLoughlin committed over 1 year ago
[V1][BugFix] Fix remaining sync engine client shutdown errors/hangs (#13869)
Nick Hill committed over 1 year ago
[Bugfix] Restrict MacOS CPU detection (#14210)
Michael Goin committed over 1 year ago