Link checking

How the site’s links are checked, locally and in CI.

The site is link-checked with Lychee, backed by a committed cache of external-link results.

To check links locally, run:

npm run check:links

Common commands

CommandChecking scope
check:linksWhole site, plus the repository’s root Markdown files
check:links:internalSame scope, offline (no external links)
check:links:production-internalSame scope over a production build, offline
fix:link-cacheAlias of check:links; use it to refresh the link cache

The check:links and check:links:internal scripts run over a build of BUILD_KIND. For details, see Build kinds: full and lean.

Links into unpublished pages

Non-production site builds include draft and future-dated pages, and production builds omit them. A link into such a page therefore works in a deploy preview and passes check:links, yet returns 404 on the production site. The check:links:production-internal script catches this: it runs an offline check over a lean production build, and reports each page that links to a target the build didn’t produce. For the contributor’s view of a failure, see CHECK LINKS (production, internal).

The production check complements check:links. Links from pages that the configuration excludes, drifted localized pages in particular, go unchecked in both. External links, and the link cache that records their results, belong to check:links: an offline run fetches nothing, so it has no results to record.

Configuration

The link check reads the generated, git-ignored lychee.toml, which the generate:config:links script derives from lychee.base.toml plus exclusions computed from two front-matter fields:

  • link_check_exclude_path: site-relative path regexes for pages that the link checker must skip, such as blog pagination and old blog posts; for an example, see content/en/blog/_index.md.
    • To cover every locale, start a pattern with ^(../)?: the optional ../ matches a two-letter locale path segment such as ja/.
  • drifted_from_default: marks drifted localized pages, with status true (EN counterpart changed) or file not found (EN counterpart deleted).
    • Links from a drifted page aren’t checked, since they may be stale.
    • Fragment links into a drifted page are skipped when the page is in the build and is its URL’s only content file; otherwise they’re checked in full. Net effect: the page is known to exist and only its fragment goes unchecked. That is what drift calls for: the page can lack sections that its English counterpart has since gained, and English pages shown as locale fallbacks still link to those sections. Implementation: toExcludePaths() in scripts/lychee/config/index.mjs.

Stored drift statuses lag English changes until the next Housekeeping status sync is merged, so the generator also treats drift-pending pages as drifted:

  • Drift-pending page: a locale copy whose English page changed or was deleted since the drift-status baseline, unless the copy itself changed since then (someone is working on it).
  • Drift-status baseline: the main commit recorded in data/l10n-drift.yaml by tree-wide status syncs (npm run fix:i18n). A local sync can rewrite that file; leave the rewrite uncommitted, since the recorded commit might not exist upstream.

Config generation fails when the baseline is missing or can’t be resolved. CI jobs fetch the baseline commit first; locally, either:

  • Fetch the missing history: git fetch upstream main.
  • Override the baseline: DRIFT_BASELINE=HEAD npm run check:links compares against the current commit, so only uncommitted English changes count as drift-pending (stored statuses still apply).

External-link check results are cached in link-cache.jsonc, the committed cache that the link-cache package maintains; Lychee’s own cache file, .lycheecache, is derived from it per run and git-ignored. For the file’s format, including how to seed an entry by hand, see The owned cache; for which URLs a check run fetches and which it serves from the cache, see Operating model. Hand edits are for deliberate seeds only; for a URL that merely blocks link checkers, append ?link-check=no to it instead (see Handling valid external links).

The cache is routinely updated by several scheduled workflows as well as content PRs, so concurrent updates can conflict under Git’s 3-way merge, distinct URLs included: both sides inserting into the same sorted gap, or one side pruning an entry the other refreshed or inserted beside (a hunk can then split an entry). When the branch’s cache changes are routine check results, take main’s whole file and rerun the check, which re-adds whatever the branch needs; otherwise follow the conflict rules in The owned cache, then rerun the check to normalize the file.

If you add or change external links, run npm run check:links before submitting your PR (the site build dominates the run time) and commit the updated link-cache.jsonc along with your content changes. Otherwise the CACHE updates committed? check will fail; for recovery steps, see CACHE updates committed?.

Cache refresh and housekeeping workflows

The following workflows are scheduled daily and run a link checking command:

WorkflowLink-check command
Refcache refreshlog:check:links (full build, after pruning)
Housekeeping (fix-and-test:all)fix:link-cache (full build)
Auto-update registry versionsfix:link-cache

Refcache refresh prunes the oldest cache entries (the count is a workflow input) and re-runs the link check, which refreshes the cache entries for the pruned URLs that are still used in the site.

Double-check of failing links

Some sites serve valid pages to browsers but turn away plain HTTP clients like Lychee (bot walls, crates.io’s unconditional 404s, npmjs.com signin redirects). Since a cached failure is re-fetched on every run, links to such sites would otherwise fail the check each time.

The double-check tooling re-verifies Lychee-reported failures through a browser-grade probe and records the URLs it resolves in link-cache.jsonc with the synthetic status 206 (“OK by analysis”); URLs it can’t resolve keep the failure the check recorded, for triage in the refresh PR. The Refcache refresh workflow runs it after the link check; to run it locally over a captured log:

npm run log:check:links
npm run fix:link-cache:double-check

For options, run npm run fix:link-cache:double-check -- --help. For probe behavior and setup, see the double-check README.

In CI

The check-links.yml workflow builds the site (lean) in its BUILD job and shares that artifact with the CHECK LINKS job, so local runs and CI check the same build. That job fails if any link check fails, and hands the cache it refreshed to the CACHE updates committed? job, which fails if the run left the committed link-cache.jsonc stale.

The CHECK LINKS (production, internal) job runs beside BUILD, with its own build: it runs check:links:production-internal.