← Insights

Your evals are your touchstone.

AI agents will soon run whole workflows end to end. When every competitor can buy the same agents, the advantage moves to whoever owns the test: their data, their definition of a good outcome, and an environment every agent has to pass.

Pieter-Jan Van Haute · 9 October 2026 · 4 min read

Every enterprise now knows it has to spread AI through the organisation as fast as it can. Most have also understood that this needs a foundation, above all in data. Far fewer have noticed the trap in how they are going about it: buying agents one workflow at a time, and letting each vendor decide what “good” looks like.

01The tools will become table stakes

Vertical AI agents are getting very good at end-to-end work: customer service, procurement, contract review, finance operations. That is good news, and it is also the problem. Once they work, every competitor can buy the same ones. A firm that runs its operations on them serves its customers exactly the way its competitors do.

Take law. If every firm builds on the same legal agent, with the same workflows and the same evaluations, what is any one firm’s long-term differentiation? The same question applies to every function that is about to be automated.

Buying the best tool is necessary. It is not a strategy.

What stays different between firms is what only each firm has: its data, its standards for a good outcome, and its judgement in the cases that matter. Advantage depends on capturing that expertise and using it to choose, test and shape the agents you run.

02What this means in practice

  • Data readiness comes first. Expertise can only be captured if the business is recorded: every case, decision, correction and outcome, joined up across systems and recoverable as it stood on any given day. A firm whose knowledge sits in an old ERP, in inboxes and in people’s heads cannot check a vendor’s claims, however good the agents on the market become.
  • Piecemeal adoption creates dependence. Each vendor holds part of your process knowledge, measures quality by its own yardstick and improves on its own roadmap. You can neither check whether its agent is right on your cases nor swap it without starting over.
  • Control comes from owning the record and the referee. The record is every input, action, approval, correction and outcome, in your own storage. The referee is an eval environment: realistic tasks drawn from your history, run in a faithful copy of your systems, scored against your own definition of a good outcome.

03The touchstone, rebuilt for AI

Assayers never took a seller’s word for the purity of gold. They kept their own stone and a set of needles of known purity. Rub the metal, compare the streak, add acid: the test belonged to the buyer, not the seller.

An eval environment is the same idea for AI. It is your standard, made testable: the place where any agent, bought or built, proves it handles your cases the way you want before it touches a customer. It has five parts.

  1. Dataset: your real cases and how they ended. Your needles of known purity. For example, 300 past quality complaints, each with the customer’s emails, the order, and whether the case was closed without being reopened.
  2. Harness: a safe copy of your business. Your systems as they were on the day of each case, with only the actions you allow, such as looking up an order or issuing a credit up to an agreed limit. Customers and suppliers are played by simulations briefed from real conversations.
  3. Rubric: your strategy as a scorecard. What a good outcome means for you: margin kept, the right answer, no promise operations cannot keep, the customer retained. Some rules are absolute; others are weighed against each other.
  4. Calibration: your experts keep the scoring honest. Specialists review a sample of scored cases, so the automated scoring judges the way they would, and it is rechecked when prices or policies change.
  5. Policy: how you want each case handled. Drafted with AI from your own history and its outcomes, then improved and signed off by the people who own the process.

The less automated your business is today, the less this seems to matter. The more automated it becomes, the more the eval environment is the business.

04Rent, build, and keep the right to swap

Own the test, and agents become swappable. Rent them where the work is a commodity. Build where the agent embodies your judgement. Let the eval environment decide when a cheaper tool, a new vendor or an in-house agent is good enough to take over.

The loop: record, test, gate, promote Whatever does the work leaves a record. The record feeds tests. Only changes that fix their target and break nothing pass the gate to the process owner. Promoted changes go live and are recorded in turn. I · RECORDII · TESTIII · GATEIV · PROMOTE Every case, actionand outcome, inyour own storage Candidate tool orchange runs on yourpast cases Fixes its target,breaks nothing →process owner signs Goes live, and itswork is recordedlike everything else THE RECORD FEEDS THE NEXT TEST I · RECORDEvery case, action and outcome,in your own storageII · TESTCandidate tool or change runson your past casesIII · GATEFixes its target, breaks nothing→ process owner signs offIV · PROMOTEGoes live, and its work isrecorded like everything else
The loop: whatever does the work (an agent, a vendor or a person) leaves a record; the record feeds the tests; only changes that pass reach the process owner.

05Where to start

None of this needs a multi-year programme before the first tool goes live. It does change the order in which you do things.

  1. Start recording now. History only accumulates from the day you capture it. There is no need to replace your ERP first; the record is built beside the systems you already run.
  2. Define a good outcome before you look at vendors. One problem statement per use case, with how success will be scored.
  3. Build the test set from your own history. A few hundred real cases with known outcomes beat any vendor benchmark.
  4. Make vendors compete on your test, not their demo. Shortlist incumbents and start-ups alike, and run them on the same cases.
  5. Keep the record and the referee in-house. Insist on full traces, data export and the right to switch. Whoever owns the test owns the decision.

This is the work we do at Touchstone Router: we help enterprises define the standard, test the market against it, and adopt what holds up.

Start a conversation → More insights →