A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
main
Save pytorch profiler output for latency benchmark (#1871)
Antoni Baum committed almost 3 years ago
Support ChatGLMForConditionalGeneration (#1932)
dancingpipi committed almost 3 years ago
add custom server params (#1868)
Roy committed almost 3 years ago
Woosuk Kwon committed almost 3 years ago
Fix num_gpus when TP > 1 (#1852)
Woosuk Kwon committed almost 3 years ago
chore(examples-docs): upgrade to OpenAI V1 (#1785)
Massimiliano Pronesti committed almost 3 years ago
Fix broken worker test (#1900)
Woosuk Kwon committed almost 3 years ago
Add PyTorch-native implementation of custom layers (#1898)
Woosuk Kwon committed almost 3 years ago
Add Production Metrics in Prometheus format (#1890)
Simon Mo committed almost 3 years ago
Fix broken sampler tests (#1896)
Woosuk Kwon committed almost 3 years ago
[Docs] Update the AWQ documentation to highlight performance issue (#1883)
Simon Mo committed almost 3 years ago
Fix the typo in SamplingParams' docstring (#1886)
Jerry committed almost 3 years ago
Normalize head weights for Baichuan 2 (#1876)
Woosuk Kwon committed almost 3 years ago
Fix Baichuan tokenizer error (#1874)
Woosuk Kwon committed almost 3 years ago
Support chat template and `echo` for chat API (#1756)
Adam Brusselback committed almost 3 years ago
docs: add instruction for langchain (#1162)
Massimiliano Pronesti committed almost 3 years ago
Fix rope cache key error (#1867)
Roy committed almost 3 years ago
Support max-model-len argument for throughput benchmark (#1858)
aisensiy committed almost 3 years ago
Remove max_num_seqs in latency benchmark script (#1855)
Woosuk Kwon committed almost 3 years ago
Add profile option to latency benchmark script (#1839)
Woosuk Kwon committed almost 3 years ago