@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / 8893130b63
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 22:00 UTC

[doc][misc] further lower visibility of simple api server (#6041)

Co-authored-by: Simon Mo <simon.mo@hey.com>

youkaichao committed Jul 1, 2024 at 10:50 UTC 8893130b63e6b2778b5d3db4dbc83ff6206ba44a
1 file changed +6 -1
examples/api_client.py
+6 -1
@@ -1,4 +1,9 @@
1 -"""Example Python client for vllm.entrypoints.api_server"""
1 +"""Example Python client for vllm.entrypoints.api_server
2 +NOTE: The API server is used only for demonstration and simple performance
3 +benchmarks. It is not intended for production use.
4 +For production use, we recommend vllm.entrypoints.openai.api_server
5 +and the OpenAI client API
6 +"""
7
8 import argparse
9 import json