Cross-Examining Agentic Commercial Agents

The potential relevance of agentic chain-of-thought in contract adjudication

Back in May, I looked at James Toomey’s paper about Zombie contracts. Today’s substack returns to the problem from a different angle.

The canonical agentic commerce problem asks us to imagine that two agents make a deal — a buyer’s procurement bot and a supplier’s sales bot. The bots settle terms and clear payment while everyone sleeps. Months later something goes wrong, and the parties find themselves before a decisionmaker seeking to show that the contract’s language allocates the relevant risk to the other side. Let’s stipulate that the relevant language is ambiguous.¹

In that scenario, it’s easy to see why a lawyer might want to engage in fact discovery about the assumptions and “state of mind” of the other side’s AI agents. For many commercial law questions, the why matters: determining if performance was in good faith asks about honesty-in-fact, some kinds of willful breach questions ask about the breaching party’s state of mind; frustration/impracticability/mistake doctrine might turn on contemporaneous-to-formation knowledge about the world.

Indeed, the very reason we get contests about what the parties really meant at contracting is because we think it’s relevant to a fact in dispute! And agentic agents, being digitally created, would appear to be creating a bread-crumb trail of what they “thought” which could be preserved and captured, and then used to buttress or undermine parties’ positions. So, at least notionally, the lawyer would want to put the agent on the stand, asking it what it “knew,” “thought,” and “assumed” when it translated a diffuse delegated judgment (“buy at reasonable terms”) to a specific set of instructions.

To wit.

But the more I think about it, the less sure I am about any of this.

Let’s start with the technical problems. (And here, I hope that those of you who are more tech’d up than me will forgive me. Believe me when I say that I at least tried to get this right.)

The thing to get straight first is that “a log of the agent’s reasoning” is really two different artifacts.

One is a provenance record: what the agent was prompted, what it was authorized to do, which tools it called, what it committed to, timestamped and signed. Securing this evidence is what the infrastructure providers have spent the last year doing, and the problem is well on its way to being solved. The intent mandate contains this exact information.

The other is a fidelity record: a trustworthy account of why the agent did what it did — the assumptions actually driving the decision. I’m not sure that these data in fact exist for most of today’s models. And even if they did, saving an agent's stated chain-of-thought does not make that chain faithful to the computation that produced the output: Anthropic's own 2025 work found that planted cues which demonstrably changed a model's answer were acknowledged in its reasoning only about a quarter of the time for Claude 3.7 Sonnet, and the introspection research shows that a model's ability to report its own internal states is real but limited, unreliable, and context-dependent.

Will this improve? Maybe? Some people appear to be sort of optimistic, to the extent that I can understand them. But incentives can also push reasoning out of the visible trace, so faithfulness is not on a guaranteed upward curve the way provenance tooling is. It’s absolutely the case that any near-term-horizon for agentic commerce can’t possibly rest on accurate reconstruction of how AI agents got to the judgments they did.

But the problems with cross-examined agents are more than merely technical.

The more I’ve thought about it, the less I’m sure that the law has the vocabulary or right set of tools to grapple with a more foundational issue. The basic question is this: when you delegate to a non-deterministic agentic AI agent, are that agent’s logs (to the extent they exist) relevant to a live legal issue?

Some people think not. Just last month, Dan Gervais and John Nay argued in The Phantom Agent that contract law never actually cared what was in anyone’s head. We don’t enforce promises because we’ve peered into a promisor’s soul, but rather manifestations, what your words and conduct would convey to a reasonable person across the table. Lucy v. Zehmer binds the man who swore he was only joking. The whole objective theory was built so we’d never have to cross-examine the mind.

My sense of existing law is that this is right some but not all the time. Even in Lucy, the test that falls out of the case attends the plaintiff’s subjective state of mind: did the person seeking to prevail on the claim actually believe that the counterparty was making a legal promise. Thus, the so-called objective part of contract formation always had a subjective limiting principle, since we don’t want to permit plaintiffs to bring cases on contracts they subjectively discounted at formation.

Worse for objectivists, the authority/agency question of whether you are bound is pretty different from the interpretation question of bound to what? When a deal hits a contingency nobody provided for, and the words genuinely cut both ways, the objective theory doesn’t finish the job. Courts at least some of the time really do reach for what these parties understood, and failing that, for the tie-breakers they keep in the drawer for irreducible ambiguity — burden of proof, contra proferentem, and at the bottom, voiding the thing for want of a meeting of the minds. As Bruce v. Blalock, 127 S.E.2d 439, 442 (S.C. 1962) said, the goal of interpretation is to figure out the following:

What would the parties have said about the meaning of a disputed phrase at the time they entered the contract?

That’s why, at least some of the time, interpretation disputes require you to get information about the parties’ contemporaneous understanding and even depose the drafter, asking what she meant, and watch her squirm. And if the drafter delegates judgment to human agents, then both agency and contract law require questions of those agents about what they were up to. In short, interpretation doctrine appears to seek to capture a mind’s thoughts, though it often settles for the mind’s textual residue.

A related paper, from the pre-ChatGPT world, comes from Anthony Casey and Anthony Niblett, whose Self-Driving Contracts imagined deals that fill their own gaps. They were enthused by the prospect of the parties setting a broad objective and letting the machine translate it into specific terms at the moment of performance. They argued that the ambiguity gets resolved by the contract itself, in real time, so there’s nothing left over for a court to construe — and nothing to ask the agent, because the agent already answered in the only way that counts, by performing.

But the machine that interprets itself, in Casey and Niblett’s vision, was deterministic. A deterministic program has an answer you can reconstruct — same inputs, same output, reproducible on the stand. A deterministic algorithm is what made B2C2 v. Quoine a relatively easy case about mistake: it was the programmer’s knowledge that ultimately mattered, a fact you could validate by looking at the software’s terms.

The agents actually closing deals today are nothing like that. Ask one why it agreed and you don’t get a recollection; you get a fresh sentence, composed in the asking. Ask again tomorrow and you may get a different one. I think the real question is whether the “reasoning trace” everyone is so eager to discover really is more like testimony or a party admission or something weirder and less illuminating still. Testimony and admissions both assume there's a fixed thing back there — something the witness saw, something the party said — and the job is to retrieve it. But the trace, at least now, retrieves nothing obviously probative whatsoever.

So, what happens when there is a genuinely ambiguous deal, where a court would ordinarily look behind the words to what was meant. In those cases— if there’s some residue of commercial dispute that can only be settled by asking the agent what it understood — what do we make of the fact that the witness has no past self to be faithful to, and will cheerfully invent a new answer every time we ask?

Maybe this kind of case is rare, and the comforting picture holds for ninety-nine deals out of a hundred. Certainly, the number of true cases where courts resort to parol evidence or focus on the negotiations in deciding gap-filling-adjacent construction questions is relatively small as a percentage of the whole. And maybe markets route around even these rarities, structuring terms so machine-readably that nothing is ever ambiguous enough to need a witness. Or we could come up with a new kind of risk-allocation technology— e.g., a merger clause for agentic agents.² But there’s simply a ton of talk in the cases about the intent that seems headed for a collision course with agentic reality.

There’s an older answer to all of this. As Dworkin explained in Law’s Empire, we don’t so much recover meaning as build it — you take the reading that shows the text in its best light and hangs together with everything around it. His image was a chain novel: each writer reads the chapters before and writes the next one to make the whole thing as good as it can be.

Now obviously, Dworkin was talking about judges reading statutes and common law, not two bots and a purchase order. But if he’s even roughly right, the missing agent’s mind is no great loss. The court reads our ambiguous overnight deal the way it reads any deal — off its structure, the parties’ dealings, the customs of the trade, whatever the thing was plainly for. On this view, the doctrinal talk about intent was always a bit of stage-dressing. That view has a long pedigree in commercial law.

The catch is that Dworkin’s kind of reading runs on fit, and fit needs something to fit to — past conduct, shared practice, the built-up context that lets a judge say this reading works and that one doesn’t. The chain novel works because there are actually (in fact!) some earlier chapters. Two bots closing one deal overnight, on a loose mandate, with no history and no course of dealing yet, are potentially more like mad libs than the Aubrey/Maturin opus. And, over time, the models will draft recursively on themselves, getting ever further away from the lee shore.³

The real case to think about then isn’t Lucy but rather Raffles v. Wichelhaus. Now, like many, I think, I teach Raffles as a grotesque — it’s insane that there were two ships named Peerless of all things, making the same trip, leaving no objective truth to recover.

But there’s a version of agentic commerce where this kind of problem stops being a curiosity and becomes the ordinary case: no shared usage to pick one Peerless over the other, nothing built up around the words to break the tie. That problem wouldn’t be insoluble, but it would require courts to talk about the goals of contractual interpretation in quite different ways than they have to date. Or, as I’ll be exploring in this space a bit later this summer, use AI tools to help get us out of the pickle that they’ve gotten us into.

1. The deal is canonical but not commercially important, since the real use cases are scaled or multi-party.

2. Maybe something like:

§ ___. Integration; Agent Reasoning Not Relevant or Admissible.

(a) Entire agreement. This Agreement, together with the [Order/Cart Mandate] and the schedules it references, is the complete and exclusive statement of the parties’ agreement and supersedes all prior or contemporaneous understandings, communications, and negotiations, whether between the parties or between any agents, models, or automated systems acting on their behalf.

(b) Agent artifacts are not terms. No prompt, instruction, configuration, system message, intermediate output, log, reasoning trace, chain-of-thought, or other internal or interstitial artifact generated by any automated agent in forming this Agreement is part of this Agreement or shall be treated as a term of it.

(c) No reliance on agent deliberation. Each party acknowledges that it has not relied and will not rely on any account—however and whenever produced, including one generated in response to a later inquiry—of why its own or the other party’s agent reached, proposed, or accepted any term. The parties agree that any such account is a generated artifact that may not reflect any actual basis for the agent’s conduct, and each party waives any claim premised on the content of such an account.

(d) Conclusive record. As between the parties, the [executed Order/Cart Mandate] is conclusive evidence of the terms agreed. To the fullest extent permitted by law and by the rules of any tribunal, each party waives any right to introduce, demand, or compel production of agent prompts, configurations, reasoning traces, model internals, or deliberative logs for the purpose of varying, supplementing, or interpreting the terms of this Agreement, and agrees that no such material is admissible to prove the meaning of any term.

(e) Exceptions. Subsections (c) and (d) do not apply, and agent prompts, configurations, traces, logs, and model internals remain discoverable and admissible, to the extent such material is offered to:

(i) prove fraud, intentional misrepresentation, or that a party deliberately caused its agent to produce or conceal a record;

(ii) authenticate the [Order/Cart Mandate] or establish whether it is the genuine and complete record of the transaction;

(iii) determine whether an agent acted within the authority its principal conferred, where that authority is disputed;

(iv) resolve a claim that the Agreement is void or voidable for reasons the law does not permit parties to waive, including illegality, incapacity, or unconscionability; or

(v) comply with a discovery obligation, subpoena, or order that a court or tribunal declines to limit under this Section.

(f) Severability of this Section. If any part of subsection (d) is held unenforceable, subsections (a) through (c) remain in full effect. The parties intend that the agent artifacts described above be given no weight in interpreting this Agreement even where a tribunal declines to exclude them from evidence.

3. I hope you saw what I did there.