Lesson 15.815.8 Checks that block

In everyday life. A smoke detector beeps. A circuit breaker cuts the power. Both notice a problem, but only one stops anything. Each is right for its job. The mistake is to believe the beeper is a breaker.

What you will learn. The difference between a check that stops a merge and one that only reports, and how to make the right ones stop it.

Why it matters. A gate is a check that can really stop a merge. A team that mistakes a report for a gate has no gate.

Which checks stop a merge?

Only the ones the repository requires. Every check on a pull request is one of two kinds.

• An advisory check is one that only reports. It may post findings, but the merge button still works.
• A blocking check is one that stops the merge. The pull request cannot join main until it passes.

A check blocks only when the repository says so. Branch protection, or a ruleset, its newer and more flexible form, is a set of rules on a branch such as main. The rules that matter here:¹

What does a workflow do?

It runs the checks.

A check on its own is the smoke detector. Marking it required in branch protection wires it to the breaker: the same check now stops the merge instead of only reporting. Running checks like this on every change is called continuous integration, or CI.

What are Brightline's required checks?

One workflow runs everything a machine can check every time, in three jobs, so a failing test cannot hide a weakened test or a printed key:

Each script fails closed: if it cannot do its job, because it cannot find the base branch, the one the pull request merges into, or a folder it should search, it fails instead of passing. A check that passes when it cannot see is worse than none, because it looks like protection.

What do these checks miss?

A lot, on purpose. Each script catches one common mistake, cheaply, on every change. Neither proves more than that. The test check cannot tell whether a new test checks anything useful, or whether the settings that run the tests were changed to skip some. The secret check misses a logging call split over two lines, or a key in a variable whose name looks harmless. Review covers what they miss. A check is only as strong as the files that run it. So the check scripts, the workflow, package.json and the type checkers' settings all have a named owner (15.9). Here is part of the workflow:

permissions:
  contents: read          # the workflow can read the code, nothing else
jobs:
  checks:                 # the type checks and tests
    steps:
      - run: npm run typecheck
      - run: npm test
  tests-unchanged:        # a separate job, so a failing test cannot hide it
    steps:
      - run: bash scripts/check-tests-unchanged.sh "origin/${{ github.base_ref }}"

The host fills in ${{ github.base_ref }} with the branch the pull request merges into.

Figure 15.13. One job stops at the first failure. One job per check lets every problem report.

Can an AI review block a merge?

Only through the repository. An AI review is advisory by default. It never approves or blocks on its own (15.7). If you want open findings to stop a merge, choose one way to make the repository enforce it. Require conversation resolution, so an unresolved finding's thread blocks the merge. Or add a required job that reads the review's result and fails on an Important finding. Or run a security review inside your own workflow, with your own step that fails when it reports findings. A review in a workflow is the same agent outside the app that you met in 12.7: one task, run headless, with read-only access and nobody watching. The rule is the same in every case: the repository's settings do the blocking, not the reviewer.

What does the pull request template do?

A file, .github/pull_request_template.md, fills every new description with the questions a reviewer needs answered. They ask what and why, how it was tested, with the commands and their output, the level of risk, and how to undo it. It costs the author a minute and saves the reviewer from guessing.

How do I keep a workflow safe?

A workflow runs code from the pull request, so treat that code as untrusted. Three rules:

• Give the workflow's token, the key it uses to act on the repository, read-only permissions.
• Keep keys in repository secrets, which the host stores and gives to workflows.³ Never commit them to a file. The host gives no secrets to workflows started from a fork, an outside copy of the repository.
• Never run a pull request's code with your secrets. pull_request_target is a way of starting a workflow that gives it those secrets, so never use it to build or run that code.⁴

In the case study. No pull request merges while a finding from the bug review or the security review is open. Lint, type checks, tests and builds are required. The template asks how a change was tested and how to undo it.⁵

At work. Ana makes all three jobs required on main, with conversation resolution on. The agent-speedup pull request shows three red marks, one per job. The tests fail on the cutoff case. The test comparison fails on the edited expected value. The secret check fails on the debug line that prints the key. The merge button is gray. Nobody had to read the diff to stop it, though someone still will, to learn why an agent did this.

The operating rule: make every check a machine can run a required check, and never mistake an advisory report for a gate.

Figure 15.14. Advisory feedback and merge requirements. Only enforced requirements control the merge gate, and the repository's settings decide what they are.

Practice it. Practice: checks that block, in your own tool, fifteen minutes.