| 1 | # Usage Stats Collection |
| 2 | |
| 3 | vLLM collects anonymous usage data by default to help the engineering team better understand which hardware and model configurations are widely used. This data allows them to prioritize their efforts on the most common workloads. The collected data is transparent, does not contain any sensitive information. |
| 4 | |
| 5 | A subset of the data, after cleaning and aggregation, will be publicly released for the community's benefit. For example, you can see the 2024 usage report [here](https://2024.vllm.ai). |
| 6 | |
| 7 | ## What data is collected? |
| 8 | |
| 9 | The list of data collected by the latest version of vLLM can be found here: [vllm/usage/usage_lib.py](../../vllm/usage/usage_lib.py) |
| 10 | |
| 11 | Here is an example as of v0.4.0: |
| 12 | |
| 13 | ??? console "Output" |
| 14 | |
| 15 | ```json |
| 16 | { |
| 17 | "uuid": "fbe880e9-084d-4cab-a395-8984c50f1109", |
| 18 | "provider": "GCP", |
| 19 | "num_cpu": 24, |
| 20 | "cpu_type": "Intel(R) Xeon(R) CPU @ 2.20GHz", |
| 21 | "cpu_family_model_stepping": "6,85,7", |
| 22 | "total_memory": 101261135872, |
| 23 | "architecture": "x86_64", |
| 24 | "platform": "Linux-5.10.0-28-cloud-amd64-x86_64-with-glibc2.31", |
| 25 | "gpu_count": 2, |
| 26 | "gpu_type": "NVIDIA L4", |
| 27 | "gpu_memory_per_device": 23580639232, |
| 28 | "model_architecture": "OPTForCausalLM", |
| 29 | "vllm_version": "0.3.2+cu123", |
| 30 | "context": "LLM_CLASS", |
| 31 | "log_time": 1711663373492490000, |
| 32 | "source": "production", |
| 33 | "dtype": "torch.float16", |
| 34 | "tensor_parallel_size": 1, |
| 35 | "block_size": 16, |
| 36 | "gpu_memory_utilization": 0.9, |
| 37 | "quantization": null, |
| 38 | "kv_cache_dtype": "auto", |
| 39 | "enable_lora": false, |
| 40 | "enable_prefix_caching": false, |
| 41 | "enforce_eager": false, |
| 42 | "disable_custom_all_reduce": true |
| 43 | } |
| 44 | ``` |
| 45 | |
| 46 | You can preview the collected data by running the following command: |
| 47 | |
| 48 | ```bash |
| 49 | tail ~/.config/vllm/usage_stats.json |
| 50 | ``` |
| 51 | |
| 52 | ## Opting out |
| 53 | |
| 54 | You can opt out of usage stats collection by setting the `VLLM_NO_USAGE_STATS` or `DO_NOT_TRACK` environment variable, or by creating a `~/.config/vllm/do_not_track` file: |
| 55 | |
| 56 | ```bash |
| 57 | # Any of the following methods can disable usage stats collection |
| 58 | export VLLM_NO_USAGE_STATS=1 |
| 59 | export DO_NOT_TRACK=1 |
| 60 | mkdir -p ~/.config/vllm && touch ~/.config/vllm/do_not_track |
| 61 | ``` |