@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 21539e6856
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

Add chat templates for Falcon (#3420)

Dinghow Yang committed Mar 15, 2024 at 14:19 UTC 21539e68563ae61d2be311d8b8e656fa039f5a5c
2 files changed +32
examples/template_falcon.jinja new
+15
@@ -0,0 +1,15 @@
1 +{%- for message in messages -%}
2 + {%- if message['role'] == 'user' -%}
3 + {{- 'User: ' + message['content'] -}}
4 + {%- elif message['role'] == 'assistant' -%}
5 + {{- 'Assistant: ' + message['content'] -}}
6 + {%- endif -%}
7 + {%- if (loop.last and add_generation_prompt) or not loop.last -%}
8 + {{- '\n' -}}
9 + {%- endif -%}
10 +{%- endfor -%}
11 +
12 +
13 +{%- if add_generation_prompt and messages[-1]['role'] != 'assistant' -%}
14 + {{- 'Assistant:' -}}
15 +{% endif %}
\ No newline at end of file
examples/template_falcon_180b.jinja new
+17
@@ -0,0 +1,17 @@
1 +{%- for message in messages -%}
2 + {%- if message['role'] == 'system' -%}
3 + {{- 'System: ' + message['content'] -}}
4 + {%- elif message['role'] == 'user' -%}
5 + {{- 'User: ' + message['content'] -}}
6 + {%- elif message['role'] == 'assistant' -%}
7 + {{- 'Falcon: ' + message['content'] -}}
8 + {%- endif -%}
9 + {%- if (loop.last and add_generation_prompt) or not loop.last -%}
10 + {{- '\n' -}}
11 + {%- endif -%}
12 +{%- endfor -%}
13 +
14 +
15 +{%- if add_generation_prompt and messages[-1]['role'] != 'assistant' -%}
16 + {{- 'Falcon:' -}}
17 +{% endif %}
\ No newline at end of file