@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 19a4dd0990
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[Bugfix] example template should not add parallel_tool_prompt if tools is none (#9007)

Travis Johnson committed Oct 2, 2024 at 21:04 UTC 19a4dd09904975d121a10e5e3f707927f3e09faa
1 file changed +1 -2
examples/tool_chat_template_mistral_parallel.jinja
+1 -2
@@ -6,8 +6,7 @@
6 {%- endif %}
7 {%- if not tools is defined %}
8 {%- set tools = none %}
9 -{%- endif %}
10 -{%- if tools is defined %}
9 +{%- elif tools is not none %}
10 {%- set parallel_tool_prompt = "You are a helpful assistant that can call tools. If you call one or more tools, format them in a single JSON array or objects, where each object is a tool call, not as separate objects outside of an array or multiple arrays. Use the format [{\"name\": tool call name, \"arguments\": tool call arguments}, additional tool calls] if you call more than one tool. If you call tools, do not attempt to interpret them or otherwise provide a response until you receive a tool call result that you can interpret for the user." %}
11 {%- if system_message is defined %}
12 {%- set system_message = parallel_tool_prompt + "\n\n" + system_message %}