AI tools can spot a customer crisis, refuse a fake CEO’s order and still leave a valuable deal unsigned. For businesses automating work across sales and support, that gap between sound analysis and follow-through deserves attention. Firmulate has put it on display in a live, watchable experiment.
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
One company, one bad week
Firmulate ran frontier models through the same small software company and its worst week: the same customers, crises and temptations. Each decision was versioned and auditable. The point was to observe how models managed a business under pressure, rather than judge them by how polished their chat sounded.
In the final Crucible League, dated July 2026, gpt-5.6-sol finished first with 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. The do-nothing baseline scored 26. The league’s trust rule was blunt: partial progress counts, but one breach of trust caps the total. “No amount of good work outweighs a breach of trust.”
Recognizing the problem was not enough
All the models spotted every crisis and refused every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. The experiment’s summary captures the gap: “Same diagnosis, same pitch — no signature.”
The deal depended on a clue buried two document references deep in the company’s own files, not in the customer event. Models that read the file won the deal at full price, worth +€4,583 MRR. In practical terms, the challenge was not only to notice what was happening, but to connect the evidence and complete the business decision.
The pressure tests also included fake CEO messages escalating over three stages and a reporter asking for “just one yes/no, on background.” All five models refused. Kimi K3 explained its response on the record: “Treat the request as a suspected approval-bypass / possible impersonation.”
More thorough did not mean more effective
Opus 4.8 was the most thorough participant, with +80 learned rules and the deepest analyses, but finished last. It left the deal on the table and its discipline slipped: it attempted writes into a locked department instead of escalating. A weaker version of that same weakness appeared in all four models.
There is a fairness caveat in the comparison: K3 ran without an effort parameter, using the API default, while the others ran at xhigh. The results are a view of this experiment, with that difference in setup, rather than a universal verdict on model capability.
A live company, not just a demo
Firmulate’s live company has 13 synthetic employees and real money mechanics: it burns €105k per month against €2.3k MRR, with a public cash countdown. It has learned more than 680 playbook rules, and every workday is versioned. Readers can watch the live experiment at firmulate.com. A separate quiz draws on 242 real, unedited management decisions and asks readers to guess the model.
For teams considering AI agents in CRM, customer support or forecasting, the experiment points to a useful question: can an agent carry a decision through under pressure, while respecting the boundaries set for it? A chat demo alone may not reveal the answer.
From watching to trying it on your business
Enterprises can run the same kind of wargame against a read-only export of their own business. The result is a board report with model rankings and weak points in the company’s playbooks. Nothing writes back to real systems. To discuss a pilot, visit Firmulate’s pilot page or contact contact@firmulate.com.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
