A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
main
Ensure that `pid` passed to `kill_process_tree` is `int` for `mypy` (#17051)
Harry Mellor committed over 1 year ago
[Doc] Add top anchor and a note to quantization/bitblas.md (#17042)
Michael Yao committed over 1 year ago
Categorize `tests/kernels/` based on kernel type (#16799)
Michael Goin committed over 1 year ago
Mistral-format support for compressed-tensors (#16803)
Michael Goin committed over 1 year ago
[CI] Run v1/test_serial_utils.py in CI (#16996)
Russell Bryant committed over 1 year ago
Chauncey committed over 1 year ago
[Misc] Improve readability of get_open_port function. (#17024)
huafeng committed over 1 year ago
[BugFix] llama4 fa3 fix - RuntimeError: scheduler_metadata must have shape (metadata_size) (#16998)
Lucas Wilkinson committed over 1 year ago
[V1] Avoid socket errors during shutdown when requests are in in-flight (#16807)
Nick Hill committed over 1 year ago
[Bugfix] Triton FA function takes no keyword arguments (#16902)
vllmellm committed over 1 year ago
[doc] add download path tips (#17013)
Reid committed over 1 year ago
[INTEL-HPU][v0] Port delayed sampling to upstream (#16949)
Chendi.Xue committed over 1 year ago
[misc] tune some env vars for GB200 (#16992)
youkaichao committed over 1 year ago
Revert "[Misc] Add S3 environment variables for better support of MinIO." (#17021)
Chauncey committed over 1 year ago
[BugFix] Revert ROCm Custom Paged Attention Env Flag Check (#17022)
vllmellm committed over 1 year ago
[V1][DP] More robust DP/EP dummy request coordination (#16277)
Nick Hill committed over 1 year ago
[Kernel][ROCM] Upstream prefix prefill speed up for vLLM V1 (#13305)
Aleksandr Malyshev committed over 1 year ago
add Dockerfile build vllm against torch nightly (#16936)
Yang Wang committed over 1 year ago
[Bugfix] validate urls object for multimodal content parts (#16990)
Guillaume Calmettes committed over 1 year ago
[Core][V1][TPU] Enable structured decoding on TPU V1 (#16499)
Chenyaaang committed over 1 year ago