A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 12, 07:15 UTC
main
[Bugfix] Revert test_tokenization.py (#29729)
Jee Jee Li committed 12 months ago
[Kernel][Quantization] add w4a8 support for marlin kernel (#24722)
Jinzhen Lin committed 12 months ago
[Chore] Move `detokenizer_utils` to `vllm/tokenizers` (#29727)
Cyrus Leung committed 12 months ago
[Chore] Enable passing `tokenizer=None` into MM processor (#29724)
Cyrus Leung committed 12 months ago
hfrunner.classify should return list[list[float]] not list[str] (#29671)
Chukwuma Nwaugha committed 12 months ago
[Doc]: fix code block rendering (#29728)
dublc committed 12 months ago
[Misc] Refactor tokenizer interface (#29693)
Cyrus Leung committed 12 months ago
[Model Runner V2] Fuse penalties and temperature into single kernel (#29720)
Woosuk Kwon committed 12 months ago
[Doc]: fixing typos in various files. (#29717)
Didier Durand committed 12 months ago
[Model Runner V2] Add sample/ directory and reorganize files (#29719)
Woosuk Kwon committed 12 months ago
[LoRA] Cleanup LoRA unused code (#29611)
Jee Jee Li committed 12 months ago
[Model Runner V2] Don't use UVA buffer for prefill_len (#29713)
Woosuk Kwon committed 12 months ago
Add gpu memory wait before test_async_tp (#28893)
Angela Yi committed 12 months ago
[BugFix] Fix DBO failing with TypeError: 'NoneType' object is not iterable (#29698)
Lucas Wilkinson committed 12 months ago
[Model Runner V2] Refactor prefill token preparation (#29712)
Woosuk Kwon committed 12 months ago
[Frontend] Perform offline path replacement to `tokenizer` (#29706)
Tsukasa OI committed 12 months ago
[Bugfix] Fix wrong mock attribute (#29704)
Cyrus Leung committed 12 months ago
[Model Runner V2] Support penalties using bin counts (#29703)
Woosuk Kwon committed 12 months ago
[ROCm][Bugfix] Patch for the `Multi-Modal Processor Test` group (#29702)
Andreas Karatzas committed 12 months ago
[Bugfix] Fix O(n²) multimodal string prompt processing (#29667)
Mert Unsal committed 12 months ago