A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
[Bugfix][Model] Fix Qwen2 (#3554)
Roy committed over 2 years ago
[BugFix] gemma loading after quantization or LoRA. (#3553)
Taemin Lee committed over 2 years ago
[Misc] Bump up transformers to v4.39.0 & Remove StarCoder2Config (#3551)
Woosuk Kwon committed over 2 years ago
[Misc][Log] Add log for tokenizer length not equal to vocabulary size (#3500)
Roy committed over 2 years ago
[🚀 Ready to be merged] Added support for Jais models (#3183)
Lalit Pradhan committed over 2 years ago
Fix 1D query issue from `_prune_hidden_states` (#3539)
SangBin Cho committed over 2 years ago
[PREFIX CACHING FOLLOW UP] OrderedDict-based evictor (#3431)
ElizaWszola committed over 2 years ago
[BugFix] Hot fix in setup.py for neuron build (#3537)
Zhuohan Li committed over 2 years ago
Migrate `logits` computation and gather to `model_runner` (#3233)
Roy committed over 2 years ago
[1/n][Chunked Prefill] Refactor input query shapes (#3236)
SangBin Cho committed over 2 years ago
[1/n] Triton sampling kernel (#3186)
Antoni Baum committed over 2 years ago
[Bugfix] Fix ROCm support in CMakeLists.txt (#3534)
James Whedbee committed over 2 years ago
Check for _is_cuda() in compute_num_jobs (#3481)
bnellnm committed over 2 years ago
Abort when nvcc command is not found in the PATH (#3527)
Allen.Dou committed over 2 years ago
[Misc] Remove cache stream and cache events (#3461)
Woosuk Kwon committed over 2 years ago
[Core] Add generic typing to `LRUCache` (#3511)
Nick Hill committed over 2 years ago
ElizaWszola committed over 2 years ago
Use lru_cache for some environment detection utils (#3508)
Simon Mo committed over 2 years ago
[Doc] minor fix of spelling in amd-installation.rst (#3506)
Jim Burtoft committed over 2 years ago
Revert "[Core] Cache some utils" (#3507)
Simon Mo committed over 2 years ago