A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[VLM][Model] Add test for InternViT vision encoder (#7409)
Isotr0py committed about 2 years ago
[OpenVINO] Updated documentation (#7687)
Ilya Lavrenov committed about 2 years ago
[Kernel] (1/N) Machete - Hopper Optimized Mixed Precision Linear Kernel (#7174)
Lucas Wilkinson committed about 2 years ago
[Core] Refactor executor classes for easier inheritance (#7673)
Kunshang Ji committed about 2 years ago
[misc][cuda] add warning for pynvml user (#7675)
youkaichao committed about 2 years ago
[XPU] fallback to native implementation for xpu custom op (#7670)
jianyizh committed about 2 years ago
[misc] add nvidia related library in collect env (#7674)
youkaichao committed about 2 years ago
[CI] Organizing performance benchmark files (#7616)
Kuntai Du committed about 2 years ago
[Bugfix] support `tie_word_embeddings` for all models (#5724)
Zijian Hu committed about 2 years ago
[ci] Install Buildkite test suite analysis (#7667)
Kevin H. Luu committed about 2 years ago
[Speculative Decoding] Fixing hidden states handling in batch expansion (#7508)
Abhinav Goyal committed about 2 years ago
[doc] fix doc build error caused by msgspec (#7659)
youkaichao committed about 2 years ago
[Bugfix] use StoreBoolean instead of type=bool for --disable-logprobs-during-spec-decoding (#7665)
Travis Johnson committed about 2 years ago
[TPU] Remove redundant input tensor cloning (#7660)
Woosuk Kwon committed about 2 years ago
[Bugfix] Don't disable existing loggers (#7664)
Andrew Song committed about 2 years ago
[Core] Support tensor parallelism for GGUF quantization (#7520)
Isotr0py committed about 2 years ago
[core] Multi Step Scheduling (#7000)
William Lin committed about 2 years ago
[Bugfix] fix lora_dtype value type in arg_utils.py - part 2 (#5428)
Ali Panahi committed about 2 years ago
[MISC] Add prefix cache hit rate to metrics (#7606)
Cody Yu committed about 2 years ago
[Misc] Remove Gemma RoPE (#7638)
Woosuk Kwon committed about 2 years ago