A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 00:30 UTC
main
Addition of lacked ignored_seq_groups in _schedule_chunked_prefill (#5296)
limingshu committed over 2 years ago
Remove Ray health check (#4693)
Antoni Baum committed over 2 years ago
[Kernel] Switch fp8 layers to use the CUTLASS kernels (#5183)
Tyler Michael Smith committed over 2 years ago
[Misc][Utils] allow get_open_port to be called for multiple times (#5333)
youkaichao committed over 2 years ago
[Feature][Frontend]: Add support for `stream_options` in `ChatCompletionRequest` (#5135)
Itay Etelis committed over 2 years ago
[Misc] Missing error message for custom ops import (#5282)
Jie Fu (傅杰) committed over 2 years ago
[Core] Change LoRA embedding sharding to support loading methods (#5038)
Antoni Baum committed over 2 years ago
[Core] Avoid copying prompt/output tokens if no penalties are used (#5289)
Antoni Baum committed over 2 years ago
[Frontend] enable passing multiple LoRA adapters at once to generate() (#5300)
Matthew Goldey committed over 2 years ago
[Kernel] Retune Mixtral 8x22b configs for FP8 on H100 (#5294)
Philipp Moritz committed over 2 years ago
Bugfix: fix broken of download models from modelscope (#5233)
liuyhwangyh committed over 2 years ago
[CI/Build] Update vision tests (#5307)
Cyrus Leung committed over 2 years ago
[Frontend][Core] Update Outlines Integration from `FSM` to `Guide` (#4109)
Breno Faria committed over 2 years ago
[CI] Disable flash_attn backend for spec decode (#5286)
Simon Mo committed over 2 years ago
[Docs] Add Ray Summit CFP (#5295)
Simon Mo committed over 2 years ago
[Misc] Skip for logits_scale == 1.0 (#5291)
Woosuk Kwon committed over 2 years ago
[Bugfix][Frontend/Core] Don't log exception when AsyncLLMEngine gracefully shuts down. (#5290)
Alex Wu committed over 2 years ago
[Bugfix] Make EngineArgs use named arguments for config construction (#5285)
Michael Goin committed over 2 years ago
[BugFix] Fix log message about default max model length (#5284)
Nick Hill committed over 2 years ago
[Speculative Decoding] Add `ProposerWorkerBase` abstract class (#5252)
Nick Hill committed over 2 years ago