A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 18:15 UTC
main
[ci/test] rearrange tests and make adag test soft fail (#7572)
youkaichao committed over 2 years ago
[Feature]: Add OpenAI server prompt_logprobs support #6508 (#7453)
Grant Pinkert committed over 2 years ago
[Misc] Add quantization config support for speculative model. (#7343)
shangmingc committed over 2 years ago
[Bugfix][Harmless] Fix float16 dtype for model_is_embedding (#7566)
Michael Goin committed over 2 years ago
[Bugfix] Fix default weight loading for scalars (#7534)
Michael Goin committed over 2 years ago
[Misc] Update dockerfile for CPU to cover protobuf installation (#7182)
PHILO-HE committed over 2 years ago
[Bugfix] update neuron for version > 0.5.0 (#7175)
omrishiv committed over 2 years ago
[Bugfix][TPU] Correct env variable for XLA cache path (#7544)
Woosuk Kwon committed over 2 years ago
llama_index serving integration documentation (#6973)
Kameshwara Pavan Kumar Mantha committed over 2 years ago
[Misc] Revert `compressed-tensors` code reuse (#7521)
Kyle Sayers committed over 2 years ago
[TPU] Set per-rank XLA cache (#7533)
Woosuk Kwon committed over 2 years ago
[core] [3/N] multi-step args and sequence.py (#7452)
William Lin committed over 2 years ago
[VLM][Core] Support profiling with multiple multi-modal inputs per prompt (#7126)
Cyrus Leung committed over 2 years ago
[Misc] Deprecation Warning when setting --engine-use-ray (#7424)
Wallas Henrique committed over 2 years ago
[Bugfix][Frontend] Disable embedding API for chat models (#7504)
jack committed over 2 years ago
youkaichao committed over 2 years ago
Fix empty output when temp is too low (#2937)
Chang Su committed over 2 years ago
[doc] update test script to include cudagraph (#7501)
youkaichao committed over 2 years ago
[Bugfix][Docs] Update list of mock imports (#7493)
Cyrus Leung committed over 2 years ago
[misc][ci] fix cpu test with plugins (#7489)
youkaichao committed over 2 years ago