A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Bugfix] handle hf_config with architectures == None (#3982)
Travis Johnson committed over 2 years ago
[Doc] Fix getting stared to use publicly available model (#3963)
Frαnçois committed over 2 years ago
[Bugfix] Remove key sorting for `guided_json` parameter in OpenAi compatible Server (#3945)
Daniel E Marasco committed over 2 years ago
[Doc] Add doc to state our model support policy (#3948)
youkaichao committed over 2 years ago
[Model][AMD] ROCm support for 256 head dims for Gemma (#3972)
James Whedbee committed over 2 years ago
[Bugfix][ROCm] Add numba to Dockerfile.rocm (#3962)
Woosuk Kwon committed over 2 years ago
[Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty (#3876)
Travis Johnson committed over 2 years ago
[Bugfix] Fix logits processor when prompt_logprobs is not None (#3899)
胡译文 committed over 2 years ago
[Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable (#3955)
zhaotyer committed over 2 years ago
[Benchmark] Add cpu options to bench scripts (#3915)
Zedong Peng committed over 2 years ago
[Misc] Avoid loading incorrect LoRA config (#3777)
Jee Li committed over 2 years ago
[ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm (#3643)
Juan Villamizar committed over 2 years ago
[Bugfix] Fix KeyError on loading GPT-NeoX (#3925)
Junichi Sato committed over 2 years ago
Cade Daniel committed over 2 years ago
[Core] separate distributed_init from worker (#3904)
youkaichao committed over 2 years ago
[BugFix][Model] Fix commandr RoPE max_position_embeddings (#3919)
Roy committed over 2 years ago
[Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations (#3782)
Matt Wong committed over 2 years ago
[Bugfix] Enable Proper `attention_bias` Usage in Llama Model Configuration (#3767)
Kiran R committed over 2 years ago
[Bugfix] Added Command-R GPTQ support (#3849)
egortolmachev committed over 2 years ago
ywfang committed over 2 years ago