@samitouri / QOS-React-2 / commits / c0b0b3a9f8

Only restore Yarn caches on exact key hits (#26133)

## Summary [Current Yarn cache size: 555MB](https://app.circleci.com/pipelines/github/facebook/react/38163/workflows/70d0149e-b0bc-44e8-b8c9-e5c744cab89b/jobs/625334?invite=true#step-102-2) [Used Yarn cache size: 344MB](https://app.circleci.com/pipelines/github/facebook/react/38166/workflows/4825d444-1426-4321-b95b-c540e6cdc6d7/jobs/625354?invite=true#step-104-5) When we restore a global Yarn cache that's not specific to a lockfile entry (i.e. a fallback cache), we might restore packages that are no longer used. When we then run yarn install, we potentially add new packages to the cache. For example: 1. we bump a package version 2. lockfile changes 3. cache restore misses for exact key 4. cache restore hits a prefix (fallback) containing the older version, 5. yarn install adds the new version to the cache Yarn is not clearing the unused packages from the global cache. So when we then save the cache we now retain the old and new version of a package in the global cache even though the old version is no longer used. This means that the global cache grows indefinitely. Restoring the cache isn't free so CI install times will degrade over time. Either we 1. periodically prune the cache 2. just not restore anything unless we have an exact hit. The chosen tradeoff depends on the relation of commits changing deps to commits not changing deps. From my experience, we change deps rarely so I opted to only restore the cache on exact hits. ## How did you test this change? - run on `main` has 555MB of Yarn cache: https://app.circleci.com/pipelines/github/facebook/react/38163/workflows/70d0149e-b0bc-44e8-b8c9-e5c744cab89b/jobs/625334?invite=true#step-102-2 - run on this branch only has 334MB of Yarn cache: https://app.circleci.com/pipelines/github/facebook/react/38166/workflows/4825d444-1426-4321-b95b-c540e6cdc6d7/jobs/625354?invite=true#step-104-5

Sebastian Silbermann committed Feb 9, 2023 at 18:00 UTC c0b0b3a9f80fd57e882859afd95c2f08599442ba

1 file changed +4 -9

.circleci/config.yml

+4 -9

@@ -11,10 +11,7 @@ aliases:

11 restore_cache:

12 name: Restore yarn cache for fixtures/dom

13 keys:

14 - - v1-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}-fixtures/dom

15 - - v1-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}

16 - - v1-yarn_cache-{{ arch }}-

17 - - v1-yarn_cache-

14 + - v2-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}-fixtures/dom

16 - &yarn_install_fixtures_dom

17 run:

@@ -32,7 +29,7 @@ aliases:

29 - &save_yarn_cache_fixtures_dom

30 save_cache:

31 name: Save yarn cache for fixtures/dom

35 - key: v1-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}-fixtures/dom

32 + key: v2-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}-fixtures/dom

33 paths:

34 - ~/.cache/yarn

@@ -48,9 +45,7 @@ commands:

45 - restore_cache:

46 name: Restore yarn cache

47 keys:

51 - - v1-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}

52 - - v1-yarn_cache-{{ arch }}-

53 - - v1-yarn_cache-

48 + - v2-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}

49 - run:

50 name: Install dependencies

51 command: |

@@ -60,7 +55,7 @@ commands:

55 fi

56 - save_cache:

57 name: Save yarn cache

63 - key: v1-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}

58 + key: v2-yarn_cache-{{ arch }}-{{ checksum "yarn.lock" }}

59 paths:

60 - ~/.cache/yarn