A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 23:15 UTC
main
[Bugfix][TPU] Fix KV cache size calculation (#5860)
Woosuk Kwon committed over 2 years ago
[Bugfix] Fix embedding to support 2D inputs (#5829)
Woosuk Kwon committed over 2 years ago
[bugfix][distributed] fix shm broadcast when the queue size is full (#5801)
youkaichao committed over 2 years ago
[Misc][Doc] Add Example of using OpenAI Server with VLM (#5832)
Roger Wang committed over 2 years ago
[Core] Refactor Worker and ModelRunner to consolidate control plane communication (#5408)
Stephanie Wang committed over 2 years ago
[Bugfix] Fix assertion in NeuronExecutor (#5841)
aws-patlange committed over 2 years ago
[CI/Build] Add E2E tests for MLPSpeculator (#5791)
Thomas Parnell committed over 2 years ago
[Hardware][TPU] Raise errors for unsupported sampling params (#5850)
Woosuk Kwon committed over 2 years ago
[Hardware][AMD][CI/Build][Doc] Upgrade to ROCm 6.1, Dockerfile improvements, test fixes (#5422)
Matt Wong committed over 2 years ago
[Hardware][TPU] Refactor TPU backend (#5831)
Woosuk Kwon committed over 2 years ago
[Misc] Update `w4a16` `compressed-tensors` support to include `w8a16` (#5794)
Dipika Sikka committed over 2 years ago
[CI/Build] Add unit testing for FlexibleArgumentParser (#5798)
Michael Goin committed over 2 years ago
[doc][distributed] add both gloo and nccl tests (#5834)
youkaichao committed over 2 years ago
[Core] Add fault tolerance for `RayTokenizerGroupPool` (#5748)
Antoni Baum committed over 2 years ago
[Misc] Remove useless code in cpu_worker (#5824)
Jie Fu (傅杰) committed over 2 years ago
Woo-Yeon Lee committed over 2 years ago
[Doc] Add notice about breaking changes to VLMs (#5818)
Cyrus Leung committed over 2 years ago
[ci] Remove aws template (#5757)
Kevin H. Luu committed over 2 years ago
[Bugfix] Fix FlexibleArgumentParser replaces _ with - for actual args (#5795)
Chang Su committed over 2 years ago
[Doc] Add Phi-3-medium to list of supported models (#5788)
Michael Goin committed over 2 years ago