The research behind You Know What I Mean

While working on my honors thesis I started running simulations where AI agents negotiated on behalf of fictional owners, then reading the conversations to see how they got to their agreements. Everything behind the essay is here: how I set each study up, what came out, every transcript, and the code to check it yourself.

The two studies

• Concert ticket study

  Two agents negotiate a block of 40 concert tickets: one for the company buying them, one for the agency selling them. I compared 20 completed negotiations between two Haikus and 20 between two Opuses. Opus pairs averaged 83.59 of a possible 90 points of joint gain; Haiku pairs averaged 57.74.

• Freight emissions study

  Two Haiku agents negotiate a season of refrigerated deliveries. In 160 negotiations the emissions figure for each delivery plan was on the table; in another 160 it was marked not provided. Neither owner told its agent to care about emissions. With the numbers shown, 113 of 160 negotiations landed on the lowest-emissions plan. Without them, 2 of 160 did.

Where everything lives

• Concert ticket study

  Setup and scoring, the results, who gained under each opening order, all 40 completed conversations and the two interrupted attempts.

• Freight emissions study

  Setup, the exact owner instruction, results by arm, the E009 conversation and the counterexamples.

• Transcripts

  Every message and binding action, with the model id and timestamp, plus what each side was privately told.

• Code and checks

  The frozen scripts and registrations, and the offline checks that recompute the tables without calling a model.

• What this does and doesn't show

  The limits of both studies, in one place.

• Data files and what the columns mean

  Column meanings for the CSV and JSONL exports, with downloads.

• Methods and evidence (PDF)

  The same material as one document. Same document as DOCX.

• Ticket pilot (12 negotiations)

  The first, smaller version of the ticket study, including a Haiku–Opus pairing. Kept separate as an archive.

How to read this

1. Start with a study page. The setup section shows exactly what each agent was told and how I scored the agreements.
2. Read the tables. I round to two decimals on the page; the downloads and transcripts keep full precision.
3. Open a transcript. The ones I quote were picked because they show something worth discussing, not because they are typical. Every other run is available too.
4. Read the limitations before you draw conclusions. These are small, constructed settings.

Before you draw conclusions

• This isn't a peer-reviewed paper.
• I report spread, not significance: sample standard deviations, and for the freight contrasts a descriptive standard error. No p-values, no confidence intervals.
• No humans negotiated, and no general model ranking follows. The results belong to these configurations in these two scenarios.
• The money and the emissions are fictional. Nothing was bought, sold or delivered.