A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 15, 13:30 UTC
[Misc] Inline Molmo requirements (#17190)
Signed-off-by: DarkLight1337 <tlleungac@connect.ust.hk>
Cyrus Leung committed
Apr 26, 2025 at 00:41 UTC
9d98ab5ec60ae43453e3bd2e68452483b3ced199
2 files changed
+27
-21
docs/source/models/supported_models.md
+27
-1
@@ -1112,7 +1112,33 @@ To use `TIGER-Lab/Mantis-8B-siglip-llama3`, you have to pass `--hf_overrides '{"
1112
:::
1113
1114
:::{warning}
1115
-For improved output quality of `AllenAI/Molmo-7B-D-0924` (especially in object localization tasks), we recommend using the pinned dependency versions listed in <gh-file:requirements/molmo.txt> (including `vllm==0.7.0`). These versions match the environment that achieved consistent results on both A10 and L40 GPUs.
1115
+The output quality of `AllenAI/Molmo-7B-D-0924` (especially in object localization tasks) has deteriorated in recent updates.
1116
+
1117
+For the best results, we recommend using the following dependency versions (tested on A10 and L40):
1118
+
1119
+```text
1120
+# Core vLLM-compatible dependencies with Molmo accuracy setup (tested on L40)
1121
+torch==2.5.1
1122
+torchvision==0.20.1
1123
+transformers==4.48.1
1124
+tokenizers==0.21.0
1125
+tiktoken==0.7.0
1126
+vllm==0.7.0
1127
+
1128
+# Optional but recommended for improved performance and stability
1129
+triton==3.1.0
1130
+xformers==0.0.28.post3
1131
+uvloop==0.21.0
1132
+protobuf==5.29.3
1133
+openai==1.60.2
1134
+opencv-python-headless==4.11.0.86
1135
+pillow==10.4.0
1136
+
1137
+# Installed FlashAttention (for float16 only)
1138
+flash-attn>=2.5.6 # Not used in float32, but should be documented
1139
+```
1140
+
1141
+**Note:** Make sure you understand the security implications of using outdated packages.
1142
:::
1143
1144
:::{note}
requirements/molmo.txt
deleted
-20
@@ -1,20 +0,0 @@
1
-# Core vLLM-compatible dependencies with Molmo accuracy setup (tested on L40)
2
-torch==2.5.1
3
-torchvision==0.20.1
4
-transformers==4.48.1
5
-tokenizers==0.21.0
6
-tiktoken==0.7.0
7
-vllm==0.7.0
8
-
9
-# Optional but recommended for improved performance and stability
10
-triton==3.1.0
11
-xformers==0.0.28.post3
12
-uvloop==0.21.0
13
-protobuf==5.29.3
14
-openai==1.60.2
15
-opencv-python-headless==4.11.0.86
16
-pillow==10.4.0
17
-
18
-# Installed FlashAttention (for float16 only)
19
-flash-attn>=2.5.6 # Not used in float32, but should be documented
20
-