Passed or Skipped

While building a tool that detects CI checks which never ran, we scanned some well-known repositories to see what required checks actually look like in the wild. One of them stopped us.

A required status check, on a project used by millions of developers, named:

Read that name again. Someone wrote it deliberately. It is documentation of a compromise, hiding in plain sight in a settings page.

Why this exists, before why it matters

GitHub lets you mark a status check as required: the pull request cannot merge until that check reports success. Sensible. But there is a catch that anyone who has configured it has run into.

A required check that never runs blocks the merge forever. If your integration-test workflow is filtered to skip on documentation-only changes — a completely reasonable thing to want — then a docs PR produces no such check, the requirement is never satisfied, and the PR sits pending indefinitely. GitHub's own documentation warns against using path or branch filters on a workflow that is required to pass before merging.

So teams do the obvious thing. They create one job that always runs, depends on all the real jobs, and reports success. That job becomes the required check. Underneath it, the real work is free to skip when it should.

It solves the problem. It is not incompetence. It is the only available answer to a platform constraint, and the projects doing it are among the most carefully maintained codebases in open source.

What it permits

Here is the part worth sitting with. That aggregator job has to decide what to do when a dependency was skipped. There are two possible rules:

Strict: succeed only if every dependency actually succeeded. A skipped dependency fails the aggregate.
Permissive: succeed unless a dependency actively failed. Skipped counts as fine.

Strict is safe and breaks the very case the aggregator was invented for — the docs PR that legitimately skips integration tests would now be blocked again. So in practice the rule is permissive. We read one such aggregator's source: it runs unconditionally, names eleven dependencies, and its check treats a skipped result as passing.

Which means: the required check reports success whether the work happened or not. Not because anyone was careless — because that is what makes it usable.

The part nobody can see

Now assume something goes wrong. A condition on the security scan is edited slightly. A path filter grows a pattern that matches more than intended. A job's dependency silently skips, so the job skips too.

The aggregator still runs. It still reports success. The required check is still green. The pull request still merges.

And on the branch protection page, everything looks exactly as configured. The gate exists. It is required. It is passing. The work it was supposed to gate did not happen.

There is no alert for this, because nothing failed. There is no red anywhere to investigate. The only signal is a job further down the run summary showing a gray dash instead of a green tick — on a page nobody opens when the merge is already allowed.

Ask what your required check is actually asserting

The mistake would be reading this as a criticism of the projects that do it. It isn't. They identified a real limitation and built the standard workaround. Almost every large repository using required checks has something like it.

The useful move is to go look at yours and ask one question: can this required check report success without the underlying work having happened?

For an aggregator, that means reading the job. Does it assert that its dependencies succeeded, or does it accept anything that is not an outright failure? If it's the latter — and it usually is — then what you have required is not the work. You have required a job that reports on the work, using a rule that tolerates the work being absent.

None of this means the workaround is wrong. It means the green square on the pull request is answering a narrower question than the person looking at it believes. That gap — between the question a check answers and the question people read it as answering — is where most verification failures live.