@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

main

Implement PagedAttention V2 (#1348)

Woosuk Kwon committed about 3 years ago

Minor fix on AWQ kernel launch (#1356)

Woosuk Kwon committed about 3 years ago

Fix error message on `TORCH_CUDA_ARCH_LIST` (#1239)

Woosuk Kwon committed about 3 years ago

Fix the issue for AquilaChat2-* models (#1339)

Lu Wang committed about 3 years ago

Improve detokenization performance (#1338)

Antoni Baum committed about 3 years ago

Add Aquila2 to README (#1331)

ldwang committed about 3 years ago

Add blacklist in model checkpoint (#1325)

Woosuk Kwon committed about 3 years ago

Add Mistral to quantization model list (#1278)

amaleshvemula committed about 3 years ago

Fix overflow in awq kernel (#1295)

CHU Tianxiang committed about 3 years ago

workaround of AWQ for Turing GPUs (#1252)

twaka committed about 3 years ago

change the timing of sorting logits (#1309)

yhlskt23 committed about 3 years ago

Fix `__repr__` of `SequenceOutputs` (#1311)

Wang Ran (汪然) committed about 3 years ago

lock torch version to 2.0.1 (#1290)

yanxiyue committed about 3 years ago

[Minor] Fix comment in mistral.py (#1303)

Zhuohan Li committed about 3 years ago

Update README.md (#1292)

Zhuohan Li committed about 3 years ago

Move bfloat16 check to worker (#1259)

Antoni Baum committed about 3 years ago

API server support ipv4 / ipv6 dualstack (#1288)

Yunfeng Bai committed about 3 years ago

Use monotonic time where appropriate (#1249)

Antoni Baum committed about 3 years ago

← Newer Page 1100 of 1120 Older →