A high-throughput and memory-efficient inference and serving engine for LLMs
main
minicpmv4_6: fix ImageSize (W,H) order for placeholder token calculation (#45244)
tc-mb committed 5 months ago
[KV offload] Parallel-agnostic fs-tier cache for single full-attention group (#44733)
Itay Etelis committed 5 months ago
docs: add fix disclosure policy to SECURITY.md (#45253)
Juan Pérez de Algaba committed 5 months ago
fix: guard flash-attn rotary import (#42679)
Yufeng He committed 5 months ago
[Build] Upgrade CUDA Dockerfiles from GCC 10 to GCC 12 for C++20 compatibility (#44923)
Richard Barnes committed 5 months ago
[Build] Skip spinloop extension on Python < 3.11 (#44783)
jasen committed 5 months ago
[Bugfix] Fix broken profile_modular_kernel.py (#43300)
x41lakazam committed 5 months ago
Only enable PR docs builds manually (#45262)
Harry Mellor committed 5 months ago
[Bugfix][Rust Frontend] Validate out-of-vocab token ids in request params (#44680)
Ting SUN committed 5 months ago
[Security] Apply sanitize_message to Anthropic and STT error paths (#45119)
Juan Pérez de Algaba committed 5 months ago
Georgii Kliukovkin committed 5 months ago
[Rust Frontend] Fix DeepSeek V3.2 continue_final_message rendering (#45155)
Reid committed 5 months ago
Fix/minicpmv46 missing version (#44980)
王金旭 committed 5 months ago
[CPU] Add missing scalar fallback for CPU W4A8 INT4 GEMM (#44523)
wcy committed 5 months ago
[Bugfix] Add fetch_images to MistralCommonImageProcessor (#45180)
Julien Denize committed 5 months ago
[Security] Reject non-finite temperature and repetition_penalty values (#45116)
Juan Pérez de Algaba committed 5 months ago
[Security] Fix info disclosure via int32 truncation in GGUF dequantize kernels (#44971)
Juan Pérez de Algaba committed 5 months ago
[10c/n] Migrate MoE kernels to torch stable ABI (#44565)
Chris Leonard committed 5 months ago
[release] Always block release images to dockerhub (#45236)
Kevin H. Luu committed 5 months ago
[Docker] Fix CUTLASS DSL cu13 install order in Dockerfile (#45204)
Mohammad Miadh Angkad committed 5 months ago