Practice note
A green pipeline is not a release decision.
A passing job answers a narrow question: the checks that ran, in the environment they ran in, did not fail. A release decision is wider. It includes the checks that were skipped, the data that was synthetic, the flag that is off in CI, and the production behaviour nobody asserted.
Treat reruns as information. If the second attempt is how the team gets to green, the first attempt found something the process is choosing to ignore. Classify it. A flake with no owner becomes the culture.
The useful artefact is a sentence a leader can say: we blocked on these causes, we accepted these risks, and production still has to show us this. Anything softer is a status report.
Practice note
Do not point an agent at a workflow you cannot evaluate.
An agent demo is easy because the happy path is chosen by the person presenting it. A workflow is different. It meets documents you did not write, permissions you did not intend, and a user who pastes an instruction into the material the model will read.
Before tools can write, delete, pay, or send, write the cases that must fail. Include a question the corpus cannot answer, a document the user must not see, and a tool call that should have stopped. If those cases cannot go red, you are watching a model be fluent.
RAG, a stronger prompt, and fine-tuning are not interchangeable. Use retrieval when the facts change. Use a prompt for a stable instruction. Fine-tune later, for a narrow behaviour you can already measure. MCP can expose the tools. It does not choose the permission. A person does, until the evaluation says the stop is no longer doing work.
