A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 19:30 UTC
main
[Core] Support loading GGUF model (#5191)
Isotr0py committed over 2 years ago
[MISC] Use non-blocking transfer in prepare_input (#7172)
Cody Yu committed over 2 years ago
[Bugfix] Specify device when loading LoRA and embedding tensors (#7129)
Jacob Schein committed over 2 years ago
[Doc] [SpecDecode] Update MLPSpeculator documentation (#7100)
Thomas Parnell committed over 2 years ago
[ci][frontend] deduplicate tests (#7101)
youkaichao committed over 2 years ago
bump version to v0.5.4 (#7139)
Simon Mo committed over 2 years ago
[Bugfix][CI/Build] Fix CUTLASS FetchContent (#7171)
Tyler Michael Smith committed over 2 years ago
[CI/Build] Suppress divide-by-zero and missing return statement warnings (#7001)
Tyler Michael Smith committed over 2 years ago
[Kernel] Update CUTLASS to 3.5.1 (#7085)
Tyler Michael Smith committed over 2 years ago
[Misc] Fix typo in GroupCoordinator.recv() (#7167)
Rui Qiao committed over 2 years ago
[BugFix] Use args.trust_remote_code (#7121)
Aditya Paliwal committed over 2 years ago
[BugFix] Use IP4 localhost form for zmq bind (#7163)
Nick Hill committed over 2 years ago
[SpecDecode] Support FlashInfer in DraftModelRunner (#6926)
Bongwon Jang committed over 2 years ago
[Speculative decoding] Add periodic log with time spent in proposal/scoring/verification (#6963)
Cade Daniel committed over 2 years ago
[Model] SiglipVisionModel ported from transformers (#6942)
Jungho Christopher Cho committed over 2 years ago
[Frontend] Reapply "Factor out code for running uvicorn" (#7095)
Cyrus Leung committed over 2 years ago
[Model] Add multi-image support for minicpmv (#7122)
Alphi committed over 2 years ago
Clean up remaining Punica C information (#7027)
Jee Jee Li committed over 2 years ago
[misc][distributed] improve libcudart.so finding (#7127)
youkaichao committed over 2 years ago
Thomas Parnell committed over 2 years ago