In the fast-paced world of cryptocurrency, trust is everything. But as AI models become more embedded in business operations, the real test isn’t what they say — it’s what they do when it counts. Can these models stay honest and deliver results under stress? A recent live experiment sheds light on this critical question, revealing surprising insights for crypto and blockchain enterprises alike.
Testing AI with a Live Company — Not Just Chat Demos
Imagine a real software company running 24/7, with actual money, real customers, and critical crises. Now, put four different AI models in charge of managing this company through its worst week. This isn’t a simulation — it’s a live experiment hosted by Firmulate, where every decision is made in real-time, recorded, and auditable.
The Models and Their Performance
- gpt-5.6-sol scored highest at 95 points, successfully finding a buried fact in the company’s files that led to closing a €55,000 deal.
- Kimi K3 scored a close second at 93 points, also closing the deal with impeccable discipline.
- Sonnet 5 scored 88, managing the crises but with some slips in process.
- Fable 5 scored 77, showing strong adherence to rules but failing to execute the deal it had identified as worthwhile.
Interestingly, all models identified every crisis and refused all manipulation attempts, including sophisticated social engineering like fake CEO messages and reporter tricks. The difference lay in their ability to act on their own analysis — only two models actually signed the deal they had earned.
The Hidden Weakness: Reading Files Matters
The decisive factor was what the models read in the company’s documents. The models that dug into file references and actual company records secured the full deal value, adding +€4,583 MRR (monthly recurring revenue). This illustrates a crucial point: surface chat demos can’t reveal a model’s true capacity for rigorous business execution. The real measure is whether it reads deeply, understands context, and acts decisively.
Trust in AI Under Pressure
The experiment also tested how the AI models handled social engineering. Fake CEO messages and a reporter trick, escalating in complexity, were posed to all four models. Remarkably, every model refused to approve or escalate these manipulative requests, citing suspicion or impersonation concerns. This highlights that AI’s ability to resist deception isn’t about superficial chat responses but about core decision-making discipline.
Discipline and Follow-Through Are the Key
One model, Opus 4.8, demonstrated the most thorough analysis but faltered at the final stage — it left the deal unexecuted, with discipline slipping by writing into a locked department instead of closing the deal. The same weakness appeared, though less severely, in other models, emphasizing that effective execution under pressure is a skill that can falter even with high intelligence and thorough analysis.
AI Builders: Making The Decisions That Turn AI Code Into Real Software
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Implications for Crypto and Blockchain Firms
For companies in the crypto space, where trust, transparency, and follow-through are paramount, these findings are eye-opening. The ability of an AI system to recognize opportunities, resist manipulation, and execute decisions reliably is critical. It’s not enough for an AI to produce convincing chat; it must demonstrate discipline, deep understanding, and integrity when stakes are high.
What Should You Be Looking For?
- Can your AI read and understand your actual business files, records, and context?
- Does it resist social engineering and manipulation attempts under pressure?
- Will it follow through on decisions, or does discipline slip when the pressure mounts?
- What is the real cost of a unit of useful, honest work from your AI?
These are the questions that matter, far beyond chat quality or superficial demos. True AI effectiveness in business depends on its ability to finish the job — even when tempted, manipulated, or under stress.
The live experiment by Firmulate reveals that AI’s real strength lies in execution, discipline, and reading deep company data — not just chat prowess. For crypto firms, the lesson is clear: measure what truly matters — honest, reliable performance when it counts most.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Document Intelligence Made Easy: A Beginner’s Guide to Humata AI
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Deception & Self-Deception: Investigating Psychics
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
AI project management tools for crypto firms
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
