@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 64 lines 1.67 KB
Rendered Raw
1 # Crusoe
2
3 [Crusoe](https://crusoe.ai/) provides Managed Inference, an OpenAI-compatible API for open-weight models, powered by vLLM. Because the service speaks the OpenAI API, code written against a self-hosted vLLM server works against Crusoe endpoints without changes.
4
5 ## Prerequisites
6
7 - A Crusoe account
8 - An Inference API key, created in the [Crusoe Console](https://console.crusoe.ai/) under **Security > Inference API Key**
9
10 Set the key as an environment variable:
11
12 ```bash
13 export CRUSOE_API_KEY="your-api-key"
14 ```
15
16 ## Using the OpenAI SDK
17
18 Point the OpenAI client at the Crusoe endpoint:
19
20 ```python
21 import os
22
23 from openai import OpenAI
24
25 client = OpenAI(
26 base_url="https://api.inference.crusoecloud.com/v1",
27 api_key=os.environ["CRUSOE_API_KEY"],
28 )
29
30 response = client.chat.completions.create(
31 model="zai/GLM-5.2",
32 messages=[{"role": "user", "content": "Hello, how are you?"}],
33 )
34
35 print(response.choices[0].message.content)
36 ```
37
38 ## Verifying with curl
39
40 !!! console "Command"
41
42 ```bash
43 curl https://api.inference.crusoecloud.com/v1/chat/completions \
44 -H "Authorization: Bearer $CRUSOE_API_KEY" \
45 -H "Content-Type: application/json" \
46 -d '{
47 "model": "zai/GLM-5.2",
48 "messages": [
49 {"role": "user", "content": "Hello, how are you?"}
50 ],
51 "max_tokens": 50
52 }'
53 ```
54
55 ## Available models
56
57 List the current model catalog:
58
59 ```bash
60 curl https://api.inference.crusoecloud.com/v1/models \
61 -H "Authorization: Bearer $CRUSOE_API_KEY"
62 ```
63
64 For the full list of available models and API details, see the [Crusoe Managed Inference docs](https://docs.crusoecloud.com/managed-inference/overview).