A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 9, 23:00 UTC
main
support dynamic resolution image encoding for Nemotron Nano VL (#32121)
Netanel Haber committed 10 months ago
[Misc] Remove unused ModelKeys (#32608)
Jee Jee Li committed 10 months ago
Add support for LoRA adapters in Nemotron-H models (#30802)
danisereb committed 10 months ago
[Frontend] Score entrypoint support data_1 & data_2 and queries & documents as inputs (#32577)
wang.yuqi committed 10 months ago
[NIXL][Metrics] Track `nixl_num_kv_expired_reqs` metric in Prometheus (#32340)
Nicolò Lucchesi committed 10 months ago
[CI/Build] Fix dependency conflict between model-hosting-container-standards and starlette (#32560)
Daniel Mescheder committed 10 months ago
[Core] Whisper support `torch.compile` (#30385)
Nicolò Lucchesi committed 10 months ago
[ROCm][CI] Add ROCm attention backend support for EAGLE DP tests (#32363)
Andreas Karatzas committed 10 months ago
[GLM-4.7] GLM Model support for GLM-Lite (#31386)
Yuxuan Zhang committed 10 months ago
[CI][Hardware][AMD] Fix test_rotary_embedding_mla_cache_fused (#32408)
Matt committed 10 months ago
[Frontend] Add render endpoints for prompt preprocessing (#32473)
Hyunkyun Moon committed 10 months ago
[CI/Build] Use Common Event Map Fixture in Harmony / MCP Server Tests (#32531)
Alex Brooks committed 10 months ago
[BugFix] Fix embed_input_ids argument error of QwenVLForConditionalGeneration (#32462)
honglyua committed 10 months ago
[Model Runner V2] Refactor `update_states` (#32562)
Woosuk Kwon committed 10 months ago
[Model Runner V2] Support VLM (#32546)
Woosuk Kwon committed 10 months ago
Vadim Gimpelson committed 10 months ago
[Bugfix] Add OOT backend option (#32471)
Iryna Boiko committed 10 months ago
[Refactor] Remove unused cutlass moe problem size function (#32047)
Wentao Ye committed 10 months ago
[Refactor] Remove unused file `pallas_kv_cache_update.py` (#32433)
Wentao Ye committed 10 months ago
[Doc] Correct comment for _jobs dict in OffloadingConnectorWorker (#32556)
Deming committed 10 months ago