A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 08:30 UTC
main
[Deprecation] Remove fallbacks for Embeddings API (#18795)
Cyrus Leung committed over 1 year ago
[Deprecation] Remove unused sync methods in `async_timeout` (#18792)
Cyrus Leung committed over 1 year ago
[Deprecation] Require overriding `get_dummy_text` and `get_dummy_mm_data` (#18796)
Cyrus Leung committed over 1 year ago
[Bugfix][FailingTest]Fix test_model_load_with_params.py (#18758)
Rabi Mishra committed over 1 year ago
[V1] [Bugfix] eagle bugfix and enable correct lm_head for multimodal (2) (#18781)
RonaldBXu committed over 1 year ago
[V1] fix torch profiling for V1 offline scenarios (#18445)
Divakar Verma committed over 1 year ago
Guillaume Calmettes committed over 1 year ago
[Bugfix] Fix nomic max_model_len (#18755)
wang.yuqi committed over 1 year ago
[rocm] Fix wrong attention log (#18764)
fxmarty-amd committed over 1 year ago
[Core] Improve Tensor serialisation (#18774)
Lukas Geiger committed over 1 year ago
[Build] Fixes for CMake install (#18570)
Luka Govedič committed over 1 year ago
[Bugfix] Disable prefix caching by default for benchmark (#18771)
cascade committed over 1 year ago
Support datasets in `vllm bench serve` and sync with benchmark_[serving,datasets].py (#18566)
Michael Goin committed over 1 year ago
[Neuron] Support quantization on neuron (#18283)
Satyajith Chilappagari committed over 1 year ago
[CI/Build] [TPU] Fix TPU CI exit code (#18282)
Carol Zheng committed over 1 year ago
[Bugfix] Mistral tool calling when content is list (#18729)
Michael Goin committed over 1 year ago
[Core] Automatically cast multi-modal input dtype (#18756)
Cyrus Leung committed over 1 year ago
optimize get_kv_cache_torch_dtype (#18531)
chunxiaozheng committed over 1 year ago
Disable prefix cache by default for benchmark (#18639)
cascade committed over 1 year ago
[V1][Metrics] Add API for accessing in-memory Prometheus metrics (#17010)
Mark McLoughlin committed over 1 year ago