A high-throughput and memory-efficient inference and serving engine for LLMs
[Bugfix][Frontend] Fix Anthropic count_tokens decorator order driving server load negative (#44725)
Signed-off-by: Ting Sun <suntcrick@gmail.com> Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Ting SUN committed
Jun 18, 2026 at 14:56 UTC
702214146c1f0f2c2120b87e6a460d5a39cef418
1 file changed
+1
-1
vllm/entrypoints/anthropic/api_router.py
+1
-1
@@ -102,8 +102,8 @@ async def create_messages(request: AnthropicMessagesRequest, raw_request: Reques
102
HTTPStatus.INTERNAL_SERVER_ERROR.value: {"model": AnthropicErrorResponse},
103
},
104
)
105
-@load_aware_call
105
@with_cancellation
106
+@load_aware_call
107
async def count_tokens(request: AnthropicCountTokensRequest, raw_request: Request):
108
handler = messages(raw_request)
109
if handler is None: