A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 19:45 UTC
Add training doc signposting to TRL (#14439)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Harry Mellor committed
Mar 8, 2025 at 08:35 UTC
be0b399d7498c00a0d66eb6cee2a0fe3c9b2838f
2 files changed
+21
docs/source/index.md
+8
@@ -100,6 +100,14 @@ features/compatibility_matrix
100
101
% Details about running vLLM
102
103
+:::{toctree}
104
+:caption: Training
105
+:maxdepth: 1
106
+
107
+training/trl.md
108
+
109
+:::
110
+
111
:::{toctree}
112
:caption: Inference and Serving
113
:maxdepth: 1
docs/source/training/trl.md
new
+13
@@ -0,0 +1,13 @@
1
+# Transformers Reinforcement Learning
2
+
3
+Transformers Reinforcement Learning (TRL) is a full stack library that provides a set of tools to train transformer language models with methods like Supervised Fine-Tuning (SFT), Group Relative Policy Optimization (GRPO), Direct Preference Optimization (DPO), Reward Modeling, and more. The library is integrated with 🤗 transformers.
4
+
5
+Online methods such as GRPO or Online DPO require the model to generate completions. vLLM can be used to generate these completions!
6
+
7
+See the guide [vLLM for fast generation in online methods](https://huggingface.co/docs/trl/main/en/speeding_up_training#vllm-for-fast-generation-in-online-methods) in the TRL documentation for more information.
8
+
9
+:::{seealso}
10
+For more information on the `use_vllm` flag you can provide to the configs of these online methods, see:
11
+- [`trl.GRPOConfig.use_vllm`](https://huggingface.co/docs/trl/main/en/grpo_trainer#trl.GRPOConfig.use_vllm)
12
+- [`trl.OnlineDPOConfig.use_vllm`](https://huggingface.co/docs/trl/main/en/online_dpo_trainer#trl.OnlineDPOConfig.use_vllm)
13
+:::