A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 20:45 UTC
main
[misc][distributed] add seed to dummy weights (#6491)
youkaichao committed over 2 years ago
[misc][distributed] improve tests (#6488)
youkaichao committed over 2 years ago
[ci][distributed] add pipeline parallel correctness test (#6410)
youkaichao committed over 2 years ago
[Kernel][Attention] Separate `Attention.kv_scale` into `k_scale` and `v_scale` (#6081)
Michael Goin committed over 2 years ago
[Misc] Log spec decode metrics (#6454)
Cody Yu committed over 2 years ago
[Doc] Fix the lora adapter path in server startup script (#6230)
Jiaxin Shan committed over 2 years ago
[Hardware][TPU] Support MoE with Pallas GMM kernel (#6457)
Woosuk Kwon committed over 2 years ago
[doc][misc] remind to cancel debugging environment variables (#6481)
youkaichao committed over 2 years ago
[CI/Build] Remove "boardwalk" image asset (#6460)
Cyrus Leung committed over 2 years ago
[Core] Use numpy to speed up padded token processing (#6442)
Peng Guanwen committed over 2 years ago
[Misc] Fix typos in spec. decode metrics logging. (#6470)
Thomas Parnell committed over 2 years ago
[Frontend] Support for chat completions input in the tokenize endpoint (#5923)
sasha0552 committed over 2 years ago
[CI/Build] vLLM cache directory for images (#6444)
Cyrus Leung committed over 2 years ago
[Docs] Announce 5th meetup (#6458)
Woosuk Kwon committed over 2 years ago
[Bugfix][CI/Build] Test prompt adapters in openai entrypoint tests (#6419)
Joe committed over 2 years ago
[BugFix][Model] Jamba - Handle aborted requests, Add tests and fix cleanup bug (#6425)
Mor Zusman committed over 2 years ago
Pin sphinx-argparse version (#6453)
Kevin H. Luu committed over 2 years ago
[CI/Build][TPU] Add TPU CI test (#6277)
Woosuk Kwon committed over 2 years ago
[Misc] Add CustomOp Interface to UnquantizedFusedMoEMethod (#6289)
Woosuk Kwon committed over 2 years ago
[Docs] Add Google Cloud to sponsor list (#6450)
Woosuk Kwon committed over 2 years ago