Skip to content

Boost CI speed #6842

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Jun 30, 2024
Merged

Boost CI speed #6842

merged 26 commits into from
Jun 30, 2024

Conversation

cometkim
Copy link
Member

@cometkim cometkim commented Jun 29, 2024

Strategies

Mostly using actions/cache

build-rewatch

Skip build if there are no changes in rewatch sources and its dependencies.

build

Cache APT dependencies

Pre-install Linux dependencies more efficiently using cache-apt-pkgs-action

Cache OPAM installation

Skip OPAM installation as much as possible.

Once it installed, there are three paths can be cached:

  • ${{ runner.tool_cache }}/opam/...: Shared storage for binaries installed by setup-* actions. It will contain opam executable.
  • ~/.opam: The home directory of OPAM, every modification by OPAM will placed there.
  • _opam: Local switch

After caching these three, we can restore OPAM installation by adding PATH and exporting opam env.

Install JSOO ahead of the job

It can be cached with other deps

Results

build-rewatch on ubuntu-latest:

  • before: 50s~90s
  • after (cached): ~15s

build on ubuntu-latest:

  • before: ~360s
  • after (cached): ~120s

@cometkim cometkim force-pushed the boost-ci branch 8 times, most recently from 8e133a5 to abe75a6 Compare June 30, 2024 00:52
@cometkim cometkim changed the title poc: cache opam actions Boost CI speed Jun 30, 2024
@cometkim
Copy link
Member Author

cometkim commented Jun 30, 2024

The Windows build failure is due to shell compatibility. Fix should be trivial Danm it wasn't

@cometkim
Copy link
Member Author

Note: managing cache

The cache of a public repository is limited to 10GB in total. Once a repository has reached its maximum cache storage, the cache eviction policy will create space by deleting the oldest caches in the repository.

If we want to invalidate the cache at some point, there are several options:

  • Change the cache key in the workflow file.
  • Change the cache key by combining it with repository variables. Then admins can expire multiple caches at once without creating a new commit.
  • Delete directly from the management UI ([repo_url]/actions/caches)
  • Use the GitHub CLI. Need gh extension install actions/gh-actions-cache

@cometkim cometkim force-pushed the boost-ci branch 2 times, most recently from 8449e1d to 02495df Compare June 30, 2024 07:35
@cometkim cometkim force-pushed the boost-ci branch 10 times, most recently from 638473d to 2f7b6b2 Compare June 30, 2024 15:58
@cometkim
Copy link
Member Author

hahahahahaha I won this war

@cometkim cometkim marked this pull request as ready for review June 30, 2024 16:22
@cometkim

This comment was marked as resolved.

This reverts commit 4e4277e.

The custom container feature seems to have some compatibility issues and doesn't provide any additional optimization anyway
@cometkim
Copy link
Member Author

@cknitt I think this will work stably. any remaining feedback?

Copy link
Member

@cknitt cknitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again for the great work!

@cknitt cknitt merged commit 2a0f3ba into rescript-lang:master Jun 30, 2024
19 checks passed
@cometkim cometkim deleted the boost-ci branch July 1, 2024 01:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants