A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[CI/Build][ROCm] Enabling tensorizer tests for ROCm (#7237)
alexeykondrat committed about 2 years ago
[Hardware][Intel GPU] Add intel GPU pipeline parallel support. (#7810)
Kunshang Ji committed about 2 years ago
[CI/Build][VLM] Cleanup multiple images inputs model test (#7897)
Isotr0py committed about 2 years ago
[Model] Add Mistral Tokenization to improve robustness and chat encoding (#7739)
Patrick von Platen committed about 2 years ago
Cody Yu committed about 2 years ago
[core][torch.compile] discard the compile for profiling (#7796)
youkaichao committed about 2 years ago
[Tests] Disable retries and use context manager for openai client (#7565)
Nick Hill committed about 2 years ago
[Core] Asynchronous Output Processor (#7049)
Megha Agarwal committed about 2 years ago
[Misc] Update compressed tensors lifecycle to remove `prefix` from `create_weights` (#7825)
Dipika Sikka committed about 2 years ago
[Bugfix] neuron: enable tensor parallelism (#7562)
omrishiv committed about 2 years ago
[misc] fix custom allreduce p2p cache file generation (#7853)
youkaichao committed about 2 years ago
[Misc] Update `gptq_marlin_24` to use vLLMParameters (#7762)
Dipika Sikka committed about 2 years ago
[Misc] Update `qqq` to use vLLMParameters (#7805)
Dipika Sikka committed about 2 years ago
[Performance][BlockManagerV2] Mark prefix cache block as computed after schedule (#7822)
Cody Yu committed about 2 years ago
[CI/Build] Avoid downloading all HF files in `RemoteOpenAIServer` (#7836)
Cyrus Leung committed about 2 years ago
[Bugfix]: Use float32 for base64 embedding (#7855)
ℍ𝕠𝕝𝕝𝕠𝕨 𝕄𝕒𝕟 committed about 2 years ago
[Spec Decoding] Streamline batch expansion tensor manipulation (#7851)
Nick Hill committed about 2 years ago
[misc][cuda] improve pynvml warning (#7852)
youkaichao committed about 2 years ago
[Misc] Remove snapshot_download usage in InternVL2 test (#7835)
Isotr0py committed about 2 years ago
[Model][VLM] Support multi-images inputs for Phi-3-vision models (#7783)
Isotr0py committed about 2 years ago