A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
[Doc] minor fix to neuron-installation.rst (#3505)
Jim Burtoft committed over 2 years ago
Update dockerfile with ModelScope support (#3429)
ifsheldon committed over 2 years ago
[BugFix] Avoid initializing CUDA too early (#3487)
Nick Hill committed over 2 years ago
[Doc] Add docs about OpenAI compatible server (#3288)
Simon Mo committed over 2 years ago
[Core] print error before deadlock (#3459)
youkaichao committed over 2 years ago
[Core] Cache some utils (#3474)
Antoni Baum committed over 2 years ago
[Misc] Update README for the Third vLLM Meetup (#3479)
Zhuohan Li committed over 2 years ago
[Core] Zero-copy asdict for InputMetadata (#3475)
Antoni Baum committed over 2 years ago
Cmake based build system (#2830)
bnellnm committed over 2 years ago
[Misc] Fix PR Template (#3478)
Zhuohan Li committed over 2 years ago
[CI/Build] Fix Bad Import In Test (#3473)
Robert Shaw committed over 2 years ago
[Testing] Add test_config.py to CI (#3437)
Cade Daniel committed over 2 years ago
CI: Add ROCm Docker Build (#2886)
Simon Mo committed over 2 years ago
[Bugfix] Make moe_align_block_size AMD-compatible (#3470)
Woosuk Kwon committed over 2 years ago
[CI] Shard tests for LoRA and Kernels to speed up (#3445)
Simon Mo committed over 2 years ago
[Misc] Use dataclass for InputMetadata (#3452)
Woosuk Kwon committed over 2 years ago
Fix setup.py neuron-ls issue (#2671)
Simon Mo committed over 2 years ago
Support arbitrary json_object in OpenAI and Context Free Grammar (#3211)
Simon Mo committed over 2 years ago
[Misc] fix line length for entire codebase (#3444)
Simon Mo committed over 2 years ago
simon-mo committed over 2 years ago