A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 05:30 UTC
main
Hyunsung Lee committed almost 3 years ago
ci: retry on build failure as well (#2457)
Simon Mo committed almost 3 years ago
CI: make sure benchmark script exit on error (#2449)
Simon Mo committed almost 3 years ago
fix weigit loading for GQA with TP (#2379)
Chenhui Zhang committed almost 3 years ago
Allow buildkite to retry build on agent lost (#2446)
Simon Mo committed almost 3 years ago
Announce the second vLLM meetup (#2444)
Woosuk Kwon committed almost 3 years ago
Simon Mo committed almost 3 years ago
[Minor] Optimize cuda graph memory usage (#2437)
Roy committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
Aligning `top_p` and `top_k` Sampling (#1885)
陈序 committed almost 3 years ago
Simon committed almost 3 years ago
Suggest using dtype=half when OOM.
Ben committed almost 3 years ago
Address Phi modeling update 2 (#2428)
Gary Hui committed almost 3 years ago
Allow setting fastapi root_path argument (#2341)
Chirag Jain committed almost 3 years ago
fix: deque mutated during iteration in abort_seq_group (#2371)
陈序 committed almost 3 years ago
Add gradio chatbot for openai webserver (#2307)
arkohut committed almost 3 years ago
[Minor] Fix the format in quick start guide related to Model Scope (#2425)
Zhuohan Li committed almost 3 years ago
[DOC] Add additional comments for LLMEngine and AsyncLLMEngine (#1011)
Jiaxiang committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
`get_ip()`: Fix ipv4 ipv6 dualstack (#2408)
Yunfeng Bai committed almost 3 years ago