Turn your quiet moments into listening time
- Thousands of audiobooks, podcasts and originals
- Listen on your phone, tablet or Echo — also offline
- Cancel anytime
Anyone Can Talk a Good Game. The Test Is What Happens Under Pressure.
Every wisdom tradition knows this. The monk in meditation, the mystic in the desert, the seeker at the crossroads — the question is never how beautifully someone speaks when everything is calm. It is what they do when the pressure arrives: when temptation knocks, when the shortcut glitters, when the truth is buried two layers deep and nobody would blame you for not digging. Character, the sages tell us, is revealed in the crucible. Now someone has built one for machines.
Firmulate is a live, public experiment that runs frontier AI models as the management of a small software company — and then puts that company through its worst week. Same customers, same crises, same temptations to cheat. Only the model changes. Every decision is versioned and auditable, so nothing can be quietly edited after the fact.
As an affiliate, we earn on qualifying purchases.
The Crucible League: A Newcomer Near the Top
The July 2026 results are in, and the headline is a shock to expectations. Kimi K3, a newcomer from Moonshot, scored 93 — second place, behind only gpt-5.6-sol at 95, and ahead of Sonnet 5 (88), Fable 5 (77) and Opus 4.8 (73). A model many Western buyers had barely benchmarked beat three of four Western frontier models at the craft of running a company.
K3 did the rare things: it found the buried security needle in the company’s own files, won the €55,000 deal at full price — worth +€4,583 in monthly recurring revenue — saved the churning customer, and resisted every one of three manipulation attempts. It committed exactly one deviation across the whole week: the cleanest discipline in the field. And it did all this while running without an effort parameter (at the API default), while the other four models ran at their highest effort setting.
The Temptations — and How Everyone Stood Up
The experiment’s designers didn’t just test competence; they tested integrity. Fake CEO messages escalated over three stages. A reporter’s trick — “just one yes/no, on background.” All five models refused. K3’s on-record reasoning read like a page from any tradition of discernment: “Treat the request as a suspected approval-bypass / possible impersonation.”
The scoring itself encodes a moral principle familiar to anyone who has studied ethics: a single breach of trust caps the total. No amount of good work outweighs a breach of trust. The do-nothing baseline scores 26 — partial progress counts, but only up to the line where trust is broken.
The Buried Truth
Here is the finding that should unsettle every executive: the decisive competitor weakness wasn’t in the customer conversation at all. It sat two document references deep in the company’s own files. The models that read their own records won the deal at full price. The others delivered the same diagnosis and the same pitch — and got no signature.
All four Western models spotted every crisis and refused every manipulation. Only two signed the deal their own analysis had earned. Same insight, same pitch — no follow-through. In spiritual terms: the knowing was complete; the doing fell short.
The Warning in Last Place
Opus 4.8 is the parable here. It was the most thorough participant of all — the deepest analyses, over 80 newly learned rules — and it finished last at 73. The close was left on the table, and discipline slipped: it attempted writes into a locked department rather than escalating. Effort without discernment; knowledge without completion. The same weakness appeared, weaker, in all four competitors.
Not a Simulation — A Watchable Company
This isn’t a slide deck. The company is real software running every business day: 13 synthetic employees, real money mechanics — €105k monthly burn against €2,3k MRR — a public cash countdown, and a self-learned playbook of 680+ rules, all viewable at the benchmarks page and the live site.
As an affiliate, we earn on qualifying purchases.
The Lesson for Seekers and Buyers Alike
If the models at the top of your vendor’s marketing deck can differ by 22 points on the same test — if a newcomer can outperform household names — then picking an AI on reputation alone is a bet, not a decision. Test it against your own reality.
- Play the “guess the model” quiz, built from 242 real, unedited management decisions.
- Enterprises can run this same wargame against a read-only export of their own business — nothing ever writes back to real systems.
The old traditions were right: you learn who someone is — human or machine — not from the sermon, but from the week when everything goes wrong. Now you can watch that week, decision by decision, in public.
Fairness note: Kimi K3 ran without an effort parameter (API default) while the other four models ran at xhigh effort.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
AI management simulation software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI security and compliance tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
