@samitouri / QOS-React / commits / c7967b194b

Distribute bundles more evenly into CI shards (#26208)

## Summary Previously, we distributed bundles into shards and then checked if we need to actually build that bundle. This can under-utilize shards heavily (e.g. https://app.circleci.com/pipelines/github/facebook/react/38611/workflows/df9e56e7-d476-49ee-9392-d8b37c81aa66/jobs/630545/parallel-runs/28?filterBy=ALL only building a single bundle). This won't result in an optimal distribution but, if we're lucky, we might end up with shard #26 not taking 7mins anymore. The slowest shard ultimately decicdes when we can start with testing builds. ## How did you test this change? - [x] `CIRCLE_NODE_INDEX=28 CIRCLE_NODE_TOTAL=40 yarn build` building more than 1 bundle - [x] Check timings of `yarn_build` so that we don't up with an over-stuffed shard (e.g. a shard having to build all the expensive bundles). Casually dropping 60min idle time 🎉: - Before: https://app.circleci.com/pipelines/github/facebook/react/38683/workflows/a41533d7-811c-439d-9751-214ba06035c5/jobs/632230/timing - After: https://app.circleci.com/pipelines/github/facebook/react/38686/workflows/8a770df6-5b3e-41ea-b3b5-10abeae703e7/jobs/632247/timing

Sebastian Silbermann committed Feb 20, 2023 at 22:16 UTC c7967b194b41cb16907eed718b78d89120089f6a

1 file changed +4 -4

scripts/rollup/build.js

+4 -4

@@ -527,10 +527,6 @@ function resolveEntryFork(resolvedEntry, isFBBundle) {

527 }

528

529 async function createBundle(bundle, bundleType) {

530 - if (shouldSkipBundle(bundle, bundleType)) {

531 - return;

532 - }

533 -

530 const filename = getFilename(bundle, bundleType);

531 const logKey =

532 chalk.white.bold(filename) + chalk.dim(` (${bundleType.toLowerCase()})`);

@@ -766,6 +762,10 @@ async function buildEverything() {

762 );

763 }

764

765 + bundles = bundles.filter(([bundle, bundleType]) => {

766 + return !shouldSkipBundle(bundle, bundleType);

767 + });

768 +

769 if (process.env.CIRCLE_NODE_TOTAL) {

770 // In CI, parallelize bundles across multiple tasks.

771 const nodeTotal = parseInt(process.env.CIRCLE_NODE_TOTAL, 10);