A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
main
Revert adding Megablocks (#2030)
Woosuk Kwon committed almost 3 years ago
Update requirements.txt for mixtral (#2029)
Ram committed almost 3 years ago
Ikko Eltociear Ashimine committed almost 3 years ago
Change the load format to pt for Mixtral (#2028)
Woosuk Kwon committed almost 3 years ago
Minor fixes for Mixtral (#2015)
Woosuk Kwon committed almost 3 years ago
Pierre Stock committed almost 3 years ago
Fix completion API echo and logprob combo (#1992)
Simon Mo committed almost 3 years ago
Replace head_mapping params with num_kv_heads to attention kernel. (#1997)
wbn committed almost 3 years ago
[Minor] Add comment on skipping rope caches (#2004)
Woosuk Kwon committed almost 3 years ago
Fix OpenAI server completion_tokens referenced before assignment (#1996)
Jin Shang committed almost 3 years ago
[Minor] Fix code style for baichuan (#2003)
Woosuk Kwon committed almost 3 years ago
Fix for KeyError on Loading LLaMA (#1978)
Jun Gao committed almost 3 years ago
[Docker] Add cuda arch list as build option (#1950)
Simon Mo committed almost 3 years ago
firebook committed almost 3 years ago
Merge EmbeddedLLM/vllm-rocm into vLLM main (#1836)
TJian committed almost 3 years ago
fix typo in getenv call (#1972)
Daya Khudia committed almost 3 years ago
[Docker] Adding number of nvcc_threads during build as envar (#1893)
AguirreNicolas committed almost 3 years ago
Fix quickstart.rst typo jinja (#1964)
gottlike committed almost 3 years ago
Make InternLM follow `rope_scaling` in `config.json` (#1956)
Jie Li committed almost 3 years ago
Fix typo in adding_model.rst (#1947)
Peter Götz committed almost 3 years ago