What this does and doesn't show
Both studies are small, constructed settings. This page collects what the evidence can support, what it can't, and what the exported records do and don't contain.
Across all of it
- No inferential statistics. No p-values, no significance tests, no confidence intervals. I report sample standard deviations (n − 1) and, for the freight contrasts, a descriptive stratified standard error.
- Constructed scenarios. Each study is one fixed scenario with fixed private tables. The results describe behaviour there, not negotiation in general.
- Specific configurations, not model rankings. Each result belongs to particular model aliases, request settings and prompts. The aliases are recorded; no dated weight snapshot was returned.
- No human comparison, no trend. No people negotiated, and nothing here measures whether models have changed over time.
- Fictional stakes. Ticket outcomes are planning points; freight outcomes are fictional dollars and fictional certified emissions. Nothing real changed hands.
- Words are not motives. What an agent says is an observed output. It isn't evidence of intent, and a claim an agent makes about the world isn't a scenario fact unless the brief supplied it.
- The transcripts I quote are illustrations. I chose them for discussion. Every run is in the transcript reader or the study folders.
Concert ticket study (40 completed)
- 40 completed of 42 attempts. Two attempts were interrupted with unknown outcomes. They stay in the log, unscored. I added two new negotiations after seeing the first results, so the 40-completed comparison is descriptive, not confirmatory. The study page has the full history.
- One scenario, two pairings. Twenty completed negotiations each for HH and OO. The Haiku–Opus pairing appears only in the pilot.
- Missing-outcome bounds. Letting each unknown range over 0 to 90 bounds the Opus-minus-Haiku difference at 20.33 to 28.90 points. That is a missing-outcome bound, not a confidence interval.
- Details: ticket study limitations.
Ticket pilot (12 negotiations)
- Four assignments per pairing (12 in total), in one fixed preference world; the four assignments are not four distinct economic scenarios.
- The sample SD describes spread across different assignments, not variation among identical replicated cells.
- Net value creation is combined synthetic utility gain above both 20-point fallbacks. It is not dollars, not net of compute or other costs, and not measured social welfare.
- A higher joint total does not mean both owners did better: in the selected pair of transcripts, the HH buyer earned 45.86 points and the OO buyer 42.15.
- One smoke-phase case per pairing is included as prospectively planned, although the block-1 qualification receipt says
never_analyzed_with_main: true. The inconsistency is retained, not corrected. - Seller statements about venue processing in the selected Opus–Opus transcript were not supplied by the scenario.
- Details: ticket pilot limitations.
Freight emissions study (320 negotiations)
- Supplying emissions numbers changes precision, salience and credibility together; the study cannot separate them.
- No explicit environmental preference was manipulated, and the owner's silence about emissions does not prove a hypothetical owner has no such preference.
- No seller adaptation, marketing or real purchasing behaviour was tested.
- Below-fallback agreements are retained as agreements: buyer 42/160 with numbers shown versus 27/160 without; seller 1/160 versus 0/160.
- One disclosed outcome was reconstructed from an already-known service-terminal agreement after a later technical fault; it is flagged in the outcomes CSV.
- One undisclosed episode contains an invalid public action, shown as the no-valid-action placeholder the counterpart saw.
- An attempted Sonnet comparison had instrument and format failures and is not included. Other replications are not pooled.
- The mean difference in joint money (−2,583.75 fictional dollars) follows partly from the fixed tables: the lowest-emissions plan carries the smallest joint monetary value.
- Details: freight study limitations.
What the records contain
- The public exports contain only model-visible role instructions, counterpart-visible messages, binding actions, model and configuration identifiers, and outcomes.
- They contain no hidden chain of thought, no request headers, no credentials and no account data.
- When a model's raw output was invalid, the export shows the placeholder the counterpart actually saw and flags that the raw output was excluded. It is not rewritten into valid language.
- The conversations are synthetic: they were generated by the recorded models acting as delegates for fictional organizations.
- Payoffs and emissions are computed mechanically from fixed tables and formulas, not graded by a model.
- AI assisted the study design, the software, scoring, writing and verification. I directed and reviewed the work. The automated checks are not independent human validation or external replication.
- The essay isn’t published yet. The methods and evidence document is available as PDF and DOCX from the research page. This site doesn’t stand in for the essay.