| 1 | # LangChain |
| 2 | |
| 3 | vLLM is also available via [LangChain](https://github.com/langchain-ai/langchain) . |
| 4 | |
| 5 | To install LangChain, run |
| 6 | |
| 7 | ```bash |
| 8 | pip install langchain langchain_community -q |
| 9 | ``` |
| 10 | |
| 11 | To run inference on a single or multiple GPUs, use `VLLM` class from `langchain`. |
| 12 | |
| 13 | ??? code |
| 14 | |
| 15 | ```python |
| 16 | from langchain_community.llms import VLLM |
| 17 | |
| 18 | llm = VLLM( |
| 19 | model="Qwen/Qwen3-4B", |
| 20 | trust_remote_code=True, # mandatory for hf models |
| 21 | max_new_tokens=128, |
| 22 | top_k=10, |
| 23 | top_p=0.95, |
| 24 | temperature=0.8, |
| 25 | # for distributed inference |
| 26 | # tensor_parallel_size=..., |
| 27 | ) |
| 28 | |
| 29 | print(llm("What is the capital of France ?")) |
| 30 | ``` |
| 31 | |
| 32 | Please refer to this [Tutorial](https://python.langchain.com/docs/integrations/llms/vllm) for more details. |