Grounding
The model should be able to point at the passage it used. A fluent answer without a source is not evidence.
AI inside a system.
Agents are useful when the tools, the data, and the approval are designed. They are a liability when a prompt is treated as a product.
- Retrieval before fine-tuning
- Tools with permissions
- Evals before scale
An agent is a workflow with a model in it.
Name the steps a person could audit. If you cannot, do not ship it.
Intent
What decision or draft is the model allowed to produce? What is it forbidden to send?
Context
Which documents, tickets, or transcripts are in scope, and under whose permission?
Tools
Each tool has a name, a schema, and a blast radius. MCP is one way to expose them, not a strategy.
Approval
Irreversible actions stop for a person. The log says who approved what.
Evaluation
A set of cases you can rerun when the model, the index, or the prompt changes.
Retrieval is a product surface.
If the answer depends on your documents, the index is part of the system under test.
Freshness and permission
Stale chunks and over-broad access are how a helpful bot leaks or lies.
When not to fine-tune
Fine-tuning changes style or format. It does not replace a source of truth you can update tomorrow.
MCP is a port, not a promise.
Model Context Protocol lets a client call tools and read resources through a standard. The risk is still yours.
Least privilege
A server that can read everything will eventually be asked to. Scope the tools to the job.
Prompt injection
Untrusted text, including retrieved documents and tickets, can instruct the model to misuse a tool. Treat tool output and document text as data.
Human gates
Write, delete, pay, and send stay behind an approval until the evaluation says otherwise.
Models are replaceable. The harness is not.
Choose a model for the task. Keep a way to notice when a cheaper one is lying.
A hierarchy, not a mascot
A small model for classification, a stronger one for synthesis, a person for the irreversible step.
Cases you can rerun
Questions with known sources, refusal cases, and tool-use traces. Not a single demo transcript.
Change control
Prompt, index, and model version are releases. They get the same suspicion as code.
The risks we will not wave away.
We do not claim a certified security programme. We design for these failures anyway.
Data leaving the boundary
Customer content, recordings, and transcripts need a purpose and a permission. Convenience is not a purpose.
Silent actions
An agent that can open a pull request or send a message needs a log and a stop.
Evaluation theatre
A leaderboard score is not a test of your documents. If you cannot fail a case, you do not have an eval.
Questions we expect.
Can the agent approve its own change?
Not for a consequential action. A person approves. The model does not grade itself.
Do you promise a ranking in AI answers?
No. Clear pages help people and machines read the offer. That is not a promise of citation.
Bring the workflow, not the model name.
We will say whether retrieval, a tool boundary, or a decision to wait is the honest next step.