A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 7, 12:45 UTC
main
[Misc] Change logging level from info to debug for tool parser import (#35575)
Chauncey committed 8 months ago
[CI] add trainer_send_weights for MockWeightTransferEngine (#35589)
Chauncey committed 8 months ago
[ROCm][CI] Expose tests to AMD production CI and fix amdsmi heap corruption (#35071)
Andreas Karatzas committed 8 months ago
[ROCm] Derive device capability from GCN arch string without CUDA init (#35069)
Andreas Karatzas committed 8 months ago
[ROCm][CI] Adding infiniband mappings for moriio tests (#35170)
Andreas Karatzas committed 8 months ago
[EPLB] Enforce sync eplb for NCCL-based all2all backend (#35212)
Ilya Markov committed 8 months ago
[Bugfix] Move chat completion response_format validation to Pydantic model_validator (#35510)
Umut Polat committed 8 months ago
[Bugfix] Propagate compilation_time from workers to main process for TP>1 (#35503)
Huy Do committed 8 months ago
[1/N] Elastic EP Milestone 2 (#34861)
Itay Alroy committed 8 months ago
[CI/Build] CPU release supports both of AVX2 and AVX512 (#35466)
Ma Jian committed 8 months ago
[MTP] Validate that MTP weights are actually loaded (#35548)
Matthew Bonanni committed 8 months ago
[Bugfix] Fixes for SLA finder (#35537)
Cyrus Leung committed 8 months ago
[ROCm] Add `stablelm` Head Size 80 To Supported Head Sizes For ROCM_ATTN (#35527)
Micah Williamson committed 8 months ago
[ROCm][Quantization] Add Composable Kernel (CK) backend support for M… (#34301)
Douglas Lehr committed 8 months ago
[Model Runner V2] Move MM encoder to Model States [3/N] (#35564)
Woosuk Kwon committed 8 months ago
[Model Runner V2] Support pooling models (#35120)
Woosuk Kwon committed 8 months ago
[Misc] Clean up ResponsesRequest model validators (#35531)
Umut Polat committed 8 months ago
[misc] cleanup one level of error stack when nixl fails to initialize (#35517)
youkaichao committed 8 months ago
[Refactor][Kernel] Add global helper to deduplicate vectorized memory ops (#35105)
Roberto L. Castro committed 8 months ago
[ROCm]: fix aiter rope functionalization (#35533)
Rohan Potdar committed 8 months ago