A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 04:15 UTC
main
Fix Baichuan chat template (#3340)
Dinghow Yang committed over 2 years ago
Replace `lstrip()` with `removeprefix()` to fix Ruff linter warning (#2958)
Ronen Schaffer committed over 2 years ago
Fixes the incorrect argument in the prefix-prefill test cases (#3246)
Tao He committed over 2 years ago
youkaichao committed over 2 years ago
Removed Extraneous Print Message From OAI Server (#3440)
Robert Shaw committed over 2 years ago
Asynchronous tokenization (#2879)
Antoni Baum committed over 2 years ago
fix document error for value and v_vec illustration (#3421)
laneeee committed over 2 years ago
Harry Mellor committed over 2 years ago
[Misc] add error message in non linux platform (#3438)
youkaichao committed over 2 years ago
Fixes the misuse/mixuse of time.time()/time.monotonic() (#3220)
Tao He committed over 2 years ago
[Fix] Add args for mTLS support (#3430)
Dan Clark committed over 2 years ago
Fix tie_word_embeddings for Qwen2. (#3344)
Yang Fan committed over 2 years ago
Fix `dist.broadcast` stall without group argument (#3408)
Junda Chen committed over 2 years ago
Add chat templates for ChatGLM (#3418)
Dinghow Yang committed over 2 years ago
Add chat templates for Falcon (#3420)
Dinghow Yang committed over 2 years ago
[Misc] add HOST_IP env var (#3419)
youkaichao committed over 2 years ago
Dynamically configure shared memory size for moe_align_block_size_kernel (#3376)
akhoroshev committed over 2 years ago
fix marlin config repr (#3414)
Enrique Shockwave committed over 2 years ago
Fix assertion failure in Qwen 1.5 with prefix caching enabled (#3373)
陈序 committed over 2 years ago
[issue templates] add some issue templates (#3412)
youkaichao committed over 2 years ago