@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / b95ee898fe
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 06:45 UTC

[Minor] Fix comment in mistral.py (#1303)

Zhuohan Li committed Oct 9, 2023 at 19:44 UTC b95ee898fe1ccb77632ed96fba7c517f6e6be931
1 file changed +1 -1
vllm/model_executor/models/mistral.py
+1 -1
@@ -20,7 +20,7 @@
20 # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
21 # See the License for the specific language governing permissions and
22 # limitations under the License.
23 -"""Inference-only LLaMA model compatible with HuggingFace weights.
23 +"""Inference-only Mistral model compatible with HuggingFace weights.
24
25 The input of the model is flattened to a 1D tensor of tokens. The model uses
26 InputMetadata to extract the original 2D shape of the input.