@tduncan / Nvidia-Optimization-Stack-VLLM
Code Commits Issues Pulls Cloud Agent CI/CD
main
main
md 84 lines 2.03 KB
Rendered Raw
1 # Summary
2
3 ## Configuration
4
5 API documentation for vLLM's configuration classes.
6
7 - [vllm.config.ModelConfig][]
8 - [vllm.config.CacheConfig][]
9 - [vllm.config.LoadConfig][]
10 - [vllm.config.ParallelConfig][]
11 - [vllm.config.SchedulerConfig][]
12 - [vllm.config.DeviceConfig][]
13 - [vllm.config.SpeculativeConfig][]
14 - [vllm.config.LoRAConfig][]
15 - [vllm.config.MultiModalConfig][]
16 - [vllm.config.PoolerConfig][]
17 - [vllm.config.StructuredOutputsConfig][]
18 - [vllm.config.ProfilerConfig][]
19 - [vllm.config.ObservabilityConfig][]
20 - [vllm.config.KVTransferConfig][]
21 - [vllm.config.CompilationConfig][]
22 - [vllm.config.VllmConfig][]
23
24 ## Offline Inference
25
26 LLM Class.
27
28 - [vllm.LLM][]
29
30 Prompt schema for LLM APIs.
31
32 - [vllm.inputs.llm][]
33
34 ## vLLM Engines
35
36 Engine classes for offline and online inference.
37
38 - [vllm.LLMEngine][]
39 - [vllm.AsyncLLMEngine][]
40
41 ## Inference Parameters
42
43 Inference parameters for vLLM APIs.
44
45 - [vllm.SamplingParams][]
46 - [vllm.PoolingParams][]
47
48 ## Multi-Modality
49
50 vLLM provides experimental support for multi-modal models through the [vllm.multimodal][] package.
51
52 Multi-modal inputs can be passed alongside text and token prompts to [supported models](../models/supported_models.md#list-of-multimodal-language-models)
53 via the `multi_modal_data` field in [vllm.inputs.PromptType][].
54
55 Looking to add your own multi-modal model? Please follow the instructions listed [here](../contributing/model/multimodal.md).
56
57 - [vllm.multimodal.MULTIMODAL_REGISTRY][]
58
59 ### Internal data structures
60
61 - [vllm.multimodal.inputs.PlaceholderRange][]
62 - [vllm.multimodal.inputs.NestedTensors][]
63 - [vllm.multimodal.inputs.MultiModalFieldElem][]
64 - [vllm.multimodal.inputs.MultiModalFieldConfig][]
65 - [vllm.multimodal.inputs.MultiModalKwargsItem][]
66 - [vllm.multimodal.inputs.MultiModalKwargsItems][]
67
68 ### Data Parsing
69
70 - [vllm.multimodal.parse][]
71
72 ### Data Processing
73
74 - [vllm.multimodal.processing][]
75
76 ### Registry
77
78 - [vllm.multimodal.registry][]
79
80 ## Model Development
81
82 - [vllm.model_executor.models.interfaces_base][]
83 - [vllm.model_executor.models.interfaces][]
84 - [vllm.model_executor.models.adapters][]