A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 10, 10:15 UTC
main
Adolfo Victoria committed 10 months ago
[ROCm][PD] add moriio kv connector. (#29304)
inkcherry committed 10 months ago
[Misc] Skip hashing kwargs if value is `None` (#32025)
Roger Wang committed 10 months ago
Andreas Karatzas committed 10 months ago
[Feature][Benchmarks] Custom dataset: read output length from dataset (#31881)
Sophie du Couédic committed 10 months ago
fix: remove duplicate engine_id check in nixl_connector (#31948)
Bofeng Xue committed 10 months ago
[Bugfix] Fix Triton FusedMoE LoRA (#30585)
Xin Yang committed 10 months ago
vllmellm committed 10 months ago
[Model] Reorganize pooling layers (#31973)
Cyrus Leung committed 10 months ago
[Bugfix] Fix OpenAPI schema test failures (#31921)
Andreas Karatzas committed 10 months ago
[Bugfix] Fix Var Length Batched Padding in Granite Speech (#31906)
Alex Brooks committed 10 months ago
fix lora moe sharding when rank < max_lora_rank (#31994)
gnovack committed 10 months ago
[Bugfix] Fix FusedMoE LoRA w2_output_size (#31949)
Xin Yang committed 10 months ago
[Cleanup] Remove obsolete spec decoding compatibility logic (#32003)
Nick Hill committed 10 months ago
[CI] [ROCm] Fix `tests/entrypoints/test_grpc_server.py` on ROCm (#31970)
TJian committed 10 months ago
[ROCm][CI] Fix test_token_classification.py::test_bert_models (#31993)
Divakar Verma committed 10 months ago
[Bugfix] missing tokens occur in harmony streaming (#30437)
RioS committed 10 months ago
[Bugfix] Fix typo in FusedMoE LoRA reshape comment (#31992)
Xin Yang committed 10 months ago
[Async][Feat] support apply penalty or bad_words for async + spec (#30495)
zhrrr committed 10 months ago
[Frontend] Add MCP tool streaming support to Responses API (#31761)
daniel-salib committed 10 months ago