Skip to content

A flaky test is not a green build.

Detection, quarantine with an owner and an expiry, and separate people to triage, fix, and verify. The workflow on this page is the method, not a live dashboard.

How a suite learns to lie.

Retry until green.

The failed attempt disappears, so the flake has no history.

Quarantine without an expiry.

A skipped test becomes permanent, and the risk moves to production.

One person owns everything.

The same engineer triages, patches, and declares the fix real.

What the method keeps visible.

Figures in an example are illustrative. We do not publish a client flake rate.

Classification.

Test, product, data, environment, infrastructure, or unknown. A flake and a real defect are not the same queue.

A quarantine with terms.

Reason, owner, expiry, and the failure still reported. Silence is not a fix.

Three roles.

Someone triages, someone fixes, someone else verifies. Severity and priority stay distinct.

States, not a product screenshot.

Illustrative sequence. Not a claim that this screen exists in a shipped tool.

  1. Detected.

    The raw failure is kept, including retries.

  2. Classified and assigned.

    Hypothesis, owner, and the evidence link.

  3. Fixed, then verified.

    A second person reruns and either closes or reopens. Quarantine ends or it is escalated.

What is live, and what is not.

Tuned Pipelines is the live product.

It diagnoses pipeline failures. It is not this whole triage workflow, and it is not a claim of every integration.

No client metric.

We will not invent a mean time to triage, a flake percentage, or a team name.

Questions we expect.

Should retries be deleted?

No. A retry can unblock people. The first failure still has to remain visible.

Can AI close the defect?

It can suggest a class or a likely change. A person still verifies. A model is not the evidence.

Bring the constraint, not a tool list.

We will say what is in scope, what is not, and what would count as proof.

Flaky tests and engineering intelligence · Solid Solutions