A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 02:00 UTC
main
[Bugfix] Fix incorrect types in LayerwiseProfileResults (#12196)
Yuan Tang committed almost 2 years ago
[DOC] Add missing docstring in LLMEngine.add_request() (#12195)
Yuan Tang committed almost 2 years ago
[DOC] Fix typo in docstring and assert message (#12194)
Yuan Tang committed almost 2 years ago
Move linting to `pre-commit` (#11975)
Harry Mellor committed almost 2 years ago
[torch.compile] fix sym_tensor_indices (#12191)
youkaichao committed almost 2 years ago
[Bugfix] Fix num_heads value for simple connector when tp enabled (#12074)
shangmingc committed almost 2 years ago
[Model] Support for fairseq2 Llama (#11442)
Martin Gleize committed almost 2 years ago
[V1] Add V1 support of Qwen2-VL (#12128)
Roger Wang committed almost 2 years ago
[Misc] Add BNB support to GLM4-V model (#12184)
Isotr0py committed almost 2 years ago
benchmark_serving support --served-model-name param (#12109)
gujing committed almost 2 years ago
[torch.compile] store inductor compiled Python file (#12182)
youkaichao committed almost 2 years ago
[Bugfix] Fix multi-modal processors for transformers 4.48 (#12187)
Cyrus Leung committed almost 2 years ago
[BUGFIX] Move scores to float32 in case of running xgrammar on cpu (#12152)
Michal Adamczyk committed almost 2 years ago
[V1] Collect env var for usage stats (#12115)
Simon Mo committed almost 2 years ago
[Misc] Support register quantization method out-of-tree (#11969)
yancong committed almost 2 years ago
[core] clean up executor class hierarchy between v1 and v0 (#12171)
youkaichao committed almost 2 years ago
[Model] Port deepseek-vl2 processor, remove dependency (#12169)
Isotr0py committed almost 2 years ago
[Docs] Fix broken link in SECURITY.md (#12175)
Russell Bryant committed almost 2 years ago
[core] further polish memory profiling (#12126)
youkaichao committed almost 2 years ago
[AMD][CI/Build][Bugfix] use pytorch stale wheel (#12172)
Hongxia Yang committed almost 2 years ago