A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
feat: implement the min_tokens sampling parameter (#3124)
Travis Johnson committed over 2 years ago
[Core] Adding token ranks along with logprobs (#3516)
Swapnil Parekh committed over 2 years ago
[CI] Try introducing isort. (#3495)
SangBin Cho committed over 2 years ago
[Bugfix] fix automatic prefix args and add log info (#3608)
TianYu GUO committed over 2 years ago
[Core] Refactor Attention Take 2 (#3462)
Woosuk Kwon committed over 2 years ago
[Model] Add starcoder2 awq support (#3569)
少年 committed over 2 years ago
[Bugfix] store lock file in tmp directory (#3578)" (#3599)
Woosuk Kwon committed over 2 years ago
[BugFix] tensor.get_device() -> tensor.device (#3604)
Kunshang Ji committed over 2 years ago
[CI/Build] fix flaky test (#3602)
youkaichao committed over 2 years ago
[CI/Build] respect the common environment variable MAX_JOBS (#3600)
youkaichao committed over 2 years ago
[CI] typo fix: is_hip --> is_hip() (#3595)
youkaichao committed over 2 years ago
[BugFix] 1D query fix for MoE models (#3597)
Nick Hill committed over 2 years ago
[BugFix] Fix Falcon tied embeddings (#3590)
Woosuk Kwon committed over 2 years ago
[Misc] Bump transformers version (#3592)
Roger Wang committed over 2 years ago
[Misc] Fix BLOOM copyright notice (#3591)
Woosuk Kwon committed over 2 years ago
[Bugfix] use SoftLockFile instead of LockFile (#3578)
kota-iizuka committed over 2 years ago
[Core] Improve detokenization performance for prefill (#3469)
Antoni Baum committed over 2 years ago
Dynamic scheduler delay to improve ITL performance (#3279)
Thomas Parnell committed over 2 years ago
[BugFix] Some fixes for custom allreduce kernels (#2760)
Hanzhi Zhou committed over 2 years ago
[Hardware][Neuron] Refactor neuron support (#3471)
Zhuohan Li committed over 2 years ago