@tduncan / Nvidia-Optimization-Stack-VLLM Mirror · read-only / commits / aaccca2b4d
Fork 0

A high-throughput and memory-efficient inference and serving engine for LLMs

Mirrored from https://github.com/tduncan-lum/Nvidia-Optimization-Stack---VLLM.git

Synced Nov 16, 10:45 UTC

[CI/Build] Fix machete generated kernel files ordering (#8976)

Signed-off-by: kevin <kevin@anyscale.com> Co-authored-by: Cody Yu <hao.yu.cody@gmail.com>

Kevin H. Luu committed Sep 30, 2024 at 20:33 UTC aaccca2b4d3895d64d34b123e61731404c8fc2c0
1 file changed +7 -1
csrc/quantization/machete/generate.py
+7 -1
@@ -457,7 +457,13 @@ def generate():
457 )),
458 ]
459
460 - schedules = list(set([x[1] for x in default_heuristic]))
460 + # Do not use schedules = list(set(...)) because we need to make sure
461 + # the output list is deterministic; otherwise the generated kernel file
462 + # will be non-deterministic and causes ccache miss.
463 + schedules = []
464 + for _, schedule_config in default_heuristic:
465 + if schedule_config not in schedules:
466 + schedules.append(schedule_config)
467
468 impl_configs = []
469