A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 22:15 UTC
main
[misc] Clean up ray compiled graph type hints (#13731)
Rui Qiao committed over 1 year ago
[Bugfix] Modify modelscope api usage in transformer_utils (#13807)
Shanshan Shen committed over 1 year ago
[Core] xgrammar: Expand list of unsupported jsonschema keywords (#13783)
Russell Bryant committed over 1 year ago
[Misc] Check that the model can be inspected upon registration (#13743)
Cyrus Leung committed over 1 year ago
[Core] LoRA V1 - Add add/pin/list/remove_lora functions (#13705)
Varun Sundar Rabindranath committed over 1 year ago
Fix CompressedTensorsWNA16MoE with grouped scales (#13769)
Michael Goin committed over 1 year ago
[Bugfix] Support MLA for CompressedTensorsWNA16 (#13725)
Michael Goin committed over 1 year ago
[Bugfix] Fix deepseek-v2 error: "missing 1 required positional argument: 'residual'" (#13802)
Lucas Wilkinson committed over 1 year ago
[V1][Metrics] Implement vllm:lora_requests_info metric (#13504)
Mark McLoughlin committed over 1 year ago
[Misc][Attention][Quantization] init property earlier (#13733)
wangxiyuan committed over 1 year ago
[Bugfix][Quantization] Fix FP8 + EP (#13784)
Tyler Michael Smith committed over 1 year ago
[Misc] set single whitespace between log sentences (#13771)
cjackal committed over 1 year ago
[Doc] arg_utils.py: fixed a typo (#13785)
Eli Boyarski committed over 1 year ago
Remove unused kwargs from model definitions (#13555)
Harry Mellor committed over 1 year ago
[Misc][Chore] Clean Up `AsyncOutputProcessing` Logs (#13780)
Robert Shaw committed over 1 year ago
[Misc] Clean Up `EngineArgs.create_engine_config` (#13734)
Robert Shaw committed over 1 year ago
Fix precommit fail in fused_moe intermediate_cache2 chunking (#13772)
Michael Goin committed over 1 year ago
Revert "[V1][Core] Fix memory issue with logits & sampling" (#13775)
Roger Wang committed over 1 year ago
[V1] V1 engine implements parallel sampling (AsyncLLM and LLMEngine) (#10980)
afeldman-nm committed over 1 year ago
Nicolò Lucchesi committed over 1 year ago