A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 18:15 UTC
main
[MISC] Add .cursor to .gitignore (#32868)
Vadim Gimpelson committed 10 months ago
[Hardware][AMD][CI][Bugfix] Fix regressions from deprecated env vars (#32837)
Matt committed 10 months ago
[Bugfix] ModelScope is supported when downloading LORA models. (#32844)
Xu Jinyang committed 10 months ago
Support bge-m3 sparse embeddings and colbert embeddings (#14526)
Maximilien de Bayser committed 10 months ago
[Misc] Bump opencv-python dependecy version to 4.13 (#32668)
Isotr0py committed 10 months ago
[Cleanup] Move scheduler `get_routed_experts` logic to separate method (#32706)
Nick Hill committed 10 months ago
[torch.compile] Improve Cold Start for MoEs (#32805)
Richard Zou committed 10 months ago
[Misc][BE] Turn on strict type coverage for vllm/compilation (#31756)
Lucas Kabela committed 10 months ago
[Frontend] Introduce Renderer for processing chat messages (using `ModelConfig`) (#30200)
Cyrus Leung committed 10 months ago
OffloadingConnector: Support kernel_block_size != block_size (#30692)
Or Ozeri committed 10 months ago
[Frontend] add prompt_cache_key for openresponses (#32824)
Chauncey committed 10 months ago
[CI] refactor release pipeline config into groups (#32833)
Shengqi Chen committed 10 months ago
[Bugfix] Fix Whisper/encoder-decoder GPU memory leak (#32789)
Nicolò Lucchesi committed 10 months ago
[Frontend][2/n] Make pooling entrypoints request schema consensus | ChatRequest (#32574)
wang.yuqi committed 10 months ago
Enable Cross layers KV cache layout at NIXL Connector (#30207)
liranschour committed 10 months ago
[Benchmark] Don't default to `temperature==0` in `vllm bench serve` (#32723)
Nick Hill committed 10 months ago
[Misc] Replace urllib's `urlparse` with urllib3's `parse_url` (#32746)
Isotr0py committed 10 months ago
[AMD][ROCm] MoRI EP: a high-performance all2all backend (#28664)
Alex Sun committed 10 months ago
[Model] Extend `collect_children` and `no_init_weights` contexts (#32757)
Cyrus Leung committed 10 months ago
[bench] add start_times field to vllm bench serve json result (#32667)
Kebe committed 10 months ago