Skip to content

Cache Gradle's dependencies in CI, so that a Maven Central hiccup does not fail a run #61

Description

@abashev

What

The CI run for #59 failed on native (macos-x86-64): https://github.com/openjavaformat/open-java-format/actions/runs/35990674177/job/107605215595. The image had already been built (3 min 7 s) when the job's second Gradle invocation, Test the plugins against the image, went to Maven Central for the Gradle plugin's test-only dependencies and the runner could not resolve the host:

> Could not resolve all files for configuration ':gradle-open-java-format:testCompileClasspath'.
   > Could not download jackson-dataformat-xml-2.22.2.jar (com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.22.2)
      > Could not GET 'https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-xml/2.22.2/jackson-dataformat-xml-2.22.2.jar'.
         > repo.maven.apache.org: nodename nor servname provided, or not known

Three more jars failed the same way (annotations-26.1.0, woodstox-core-7.2.0, stax2-api-4.3.0). The other eight jobs of the run passed, the PR adds two golden files and touches no dependency, and the same code passed on main right after the merge (https://github.com/openjavaformat/open-java-format/actions/runs/35991726913): a transient DNS failure on the runner.

Why

No job in ci.yml keeps Gradle's dependency cache between runs: there is no gradle/actions/setup-gradle, no actions/cache, and setup-java runs without its cache input. Every run downloads everything again, in each of its nine jobs, and each job is only as reliable as repo.maven.apache.org and plugins.gradle.org are at that moment. The native jobs are the most exposed: they run Gradle twice, and the second invocation needs jars the first did not fetch, so a job whose image compile succeeded can still fail on a download. Every job of release.yml runs Gradle the same way, the native image one on all five platforms, and so does codeql-analysis.yml.

With a warm cache Gradle resolves the pinned versions of gradle/libs.versions.toml without touching the network, so an outage costs at most a run that changed a dependency.

How

Add gradle/actions/setup-gradle before the first ./gradlew of every job that runs Gradle, SHA-pinned as sha-pinning-check.yml requires. dependency-submission.yml already pins that repository at v6.3.0, commit 9c971963bec38e04b3d30dcc455b5382be2fdbfb; setup-gradle at the same commit keeps the two in step, and Dependabot bumps both together.

To settle along the way:

  • Who writes. By default the action writes cache entries only from the default branch and reads them everywhere else. ci.yml runs on every push to main, so main keeps the cache warm and PR jobs restore it; a Dependabot PR downloads just the artifact it bumps. Keep the default, or spell it out with cache-read-only: ${{ github.ref != 'refs/heads/main' }}.
  • What is restored. Entries are keyed on OS, job and matrix entry, with prefix fallbacks, so each of the five native platforms gets its own entry and a PR job restores the newest entry of its job. The release jobs have other job ids and another workflow name; check what the fallback keys give them.
  • Size. GitHub keeps a repository's cache under 10 GB and evicts entries unused for 7 days. The build and build (JDK n) entries are the big ones: ./gradlew test resolves the IntelliJ IDE distribution for the IDE plugin's tests through the module cache, several hundred MB as a zip and again unpacked, plus the JetBrains Runtime. Four such entries and five native ones should fit; read the sizes off the job summary, and gradle-home-cache-excludes can leave the IDE out if they do not.
  • The build cache. org.gradle.caching=true in gradle.properties means the action also carries caches/build-cache-1, the task outputs, across runs. That is fine for ci.yml; codeql-analysis.yml already passes --no-build-cache, so its javac still runs.
  • The simpler alternative. actions/setup-java and graalvm/setup-graalvm both take cache: gradle, one line per job, but that keys the whole cache on a hash of the Gradle files and only logs a hit or a miss. setup-gradle writes a job summary with the entries restored and the builds that ran, which is the evidence for "done when".

Done when

  • every job that runs ./gradlew in ci.yml restores Gradle's dependency cache, and release.yml and codeql-analysis.yml do the same or this issue says why not;
  • a PR run's job summary shows the entries restored, and the job log shows no download from repo.maven.apache.org for a dependency the PR did not change;
  • the new steps are SHA-pinned and sha-pinning-check.yml passes.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions