A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 03:00 UTC
main
[Bugfix] Fix type annotations in CPU model runner (#4256)
Woosuk Kwon committed over 2 years ago
[Misc] Add vision language model support to CPU backend (#3968)
Isotr0py committed over 2 years ago
[Core][Distributed] fix _is_full_nvlink detection (#4233)
youkaichao committed over 2 years ago
Hongxia Yang committed over 2 years ago
Make initialization of tokenizer and detokenizer optional (#3748)
GeauxEric committed over 2 years ago
[Doc]: Update the doc of adding new models (#4236)
xiaoji committed over 2 years ago
Don't show default value for flags in `EngineArgs` (#4223)
Harry Mellor committed over 2 years ago
Updating lm-format-enforcer version and adding links to decoding libraries in docs (#4222)
Noam Gat committed over 2 years ago
[Frontend] multiple sampling params support (#3570)
nunjunj committed over 2 years ago
[Kernel][FP8] Initial support with dynamic per-tensor scaling (#4118)
Cody Yu committed over 2 years ago
Fix missing docs and out of sync `EngineArgs` (#4219)
Harry Mellor committed over 2 years ago
[Bugfix] Add fix for JSON whitespace (#4189)
Ayush Rautwar committed over 2 years ago
Pass `tokenizer_revision` when getting tokenizer in openai serving (#4214)
Chirag Jain committed over 2 years ago
[Misc] add nccl in collect env (#4211)
youkaichao committed over 2 years ago
[Bugfix][Core] Restore logging of stats in the async engine (#4150)
Ronen Schaffer committed over 2 years ago
Uranus committed over 2 years ago
Simon Mo committed over 2 years ago
[Bugfix] Fix LoRA loading check (#4138)
Jee Li committed over 2 years ago
Support eos_token_id from generation_config.json (#4182)
Simon Mo committed over 2 years ago
youkaichao committed over 2 years ago