A high-throughput and memory-efficient inference and serving engine for LLMs
Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git
Synced Nov 17, 06:45 UTC
[Minor] Fix comment in mistral.py (#1303)
Zhuohan Li committed
Oct 9, 2023 at 19:44 UTC
b95ee898fe1ccb77632ed96fba7c517f6e6be931
1 file changed
+1
-1
vllm/model_executor/models/mistral.py
+1
-1
@@ -20,7 +20,7 @@
20
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
21
# See the License for the specific language governing permissions and
22
# limitations under the License.
23
-"""Inference-only LLaMA model compatible with HuggingFace weights.
23
+"""Inference-only Mistral model compatible with HuggingFace weights.
24
25
The input of the model is flattened to a 1D tensor of tokens. The model uses
26
InputMetadata to extract the original 2D shape of the input.