A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Support Longchat and RoPE scaling (#555)
Lily Liu committed about 3 years ago
Allocate more shared memory to attention kernel (#1154)
Antoni Baum committed about 3 years ago
Add comments on RoPE initialization (#1176)
Woosuk Kwon committed about 3 years ago
[Setup] Enable `TORCH_CUDA_ARCH_LIST` for selecting target GPUs (#1074)
Woosuk Kwon committed about 3 years ago
Align `max_tokens` behavior with openai (#852)
Wen Sun committed about 3 years ago
Woosuk Kwon committed about 3 years ago
[FIX] Simplify sampler logic (#1156)
Zhuohan Li committed about 3 years ago
[Sampler] Vectorized sampling (simplified) (#1048)
Zhuohan Li committed about 3 years ago
Announce the First vLLM Meetup (#1148)
Woosuk Kwon committed about 3 years ago
Docs: Fix broken link to openai example (#1145)
Nick Perez committed about 3 years ago
Use `--ipc=host` in docker run for distributed inference (#1125)
Woosuk Kwon committed about 3 years ago
feat: support stop_token_ids parameter. (#1097)
Ricardo Lu committed about 3 years ago
clean api code, remove redundant background task. (#1102)
Roy committed about 3 years ago
Add float16 and float32 (#1115)
Woosuk Kwon committed about 3 years ago
Woosuk Kwon committed about 3 years ago
rope_theta and max_position_embeddings from config (#1096)
Antoni Baum committed about 3 years ago
Add documentation to Triton server tutorial (#983)
Tanmay Verma committed about 3 years ago
Add gpu_memory_utilization and swap_space to LLM (#1090)
Woosuk Kwon committed about 3 years ago
Add pyarrow to dependencies & Print warning on Ray import error (#1094)
Woosuk Kwon committed about 3 years ago
[Community] Add vLLM Discord server (#1086)
Zhuohan Li committed about 3 years ago