A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 16, 22:00 UTC
[Bugfix][Doc] Fix Doc Formatting (#6048)
Roger Wang committed
Jul 1, 2024 at 15:09 UTC
8e0817c262da5c104f651a0ce4ac9ee0cd76f4ce
1 file changed
+2
-2
docs/source/serving/faq.rst
+2
-2
@@ -1,5 +1,5 @@
1
Frequently Asked Questions
2
-========================
2
+===========================
3
4
Q: How can I serve multiple models on a single port using the OpenAI API?
5
@@ -9,4 +9,4 @@ A: Assuming that you're referring to using OpenAI compatible server to serve mul
9
10
Q: Which model to use for offline inference embedding?
11
12
-A: If you want to use an embedding model, try: https://huggingface.co/intfloat/e5-mistral-7b-instruct. Instead models, such as Llama-3-8b, Mistral-7B-Instruct-v0.3, are generation models rather than an embedding model
\ No newline at end of file
12
+A: If you want to use an embedding model, try: https://huggingface.co/intfloat/e5-mistral-7b-instruct. Instead models, such as Llama-3-8b, Mistral-7B-Instruct-v0.3, are generation models rather than an embedding model