@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / ad50bf4b25
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 17, 04:15 UTC

fix lint

simon-mo committed Mar 15, 2024 at 22:23 UTC ad50bf4b25ba4344a560a7919fdc6ddb57c3d808
2 files changed +3 -3
.github/workflows/ruff.yml
+1 -1
@@ -31,4 +31,4 @@ jobs:
31 ruff vllm tests
32 - name: Spelling check with codespell
33 run: |
34 - codespell --toml pyproject.toml
\ No newline at end of file
34 + codespell --toml pyproject.toml
\ No newline at end of file
tests/kernels/test_prefix_prefill.py
+2 -2
@@ -36,8 +36,8 @@ def test_contexted_kv_attention(
36 torch.cuda.manual_seed(0)
37 torch.set_default_device(device)
38
39 - # Need this, otherwise when we capture the graph the process for GPU 1 would run on both
40 - # GPU0 and GPU1 and things would hang
39 + # Need this, otherwise when we capture the graph the process for GPU 1 would
40 + # run on both GPU0 and GPU1 and things would hang
41 #
42 # see also similar issue: https://github.com/Dao-AILab/flash-attention/issues/523
43 torch.cuda.set_device(device)