@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 23b899a8e6
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 09:30 UTC

[Bugfix] fix detokenizer shallow copy (#5919)

Aurick Qiao committed Oct 22, 2024 at 18:38 UTC 23b899a8e62c7ea07981bf8487b0dc2cb17847b8
1 file changed +1 -1
vllm/transformers_utils/detokenizer.py
+1 -1
@@ -90,7 +90,7 @@ class Detokenizer:
90 prefix_offset = next_iter_prefix_offset
91 read_offset = next_iter_read_offset
92 if prev_tokens is None:
93 - prev_tokens = next_iter_tokens
93 + prev_tokens = next_iter_tokens.copy()
94 else:
95 prev_tokens.extend(next_iter_tokens)
96