A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 14, 05:30 UTC
[CI/Build] fix doc build warning: Failed to get 'name: description' pair (#25733)
Signed-off-by: yiting.jiang <yiting.jiang@daocloud.io>
yitingdc committed
Sep 26, 2025 at 16:18 UTC
3edf87d25fd0afcd0798c6cac5917631fde34054
1 file changed
+2
-2
vllm/v1/spec_decode/ngram_proposer.py
+2
-2
@@ -71,8 +71,8 @@ class NgramProposer:
71
Args:
72
valid_ngram_requests:
73
Set of indices of requests that need ngram proposals.
74
- num_tokens_no_spec:
75
- Numpy array of shape (batch_size,) representing the number
74
+ num_tokens_no_spec:
75
+ Numpy array of shape (batch_size,) representing the number
76
of tokens without speculative tokens for each request.
77
token_ids_cpu:
78
Numpy array of shape (batch_size, max_model_len)