🔍 Read the full analysis: Inside The Industry Benchmark That Resists Zero Scores For AI Managers on ThorstenMeyerAI.com
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
TL;DR
A recent industry benchmark tests AI managers in real-world scenarios, showing they rarely receive zero scores and highlighting the importance of trust and partial progress. The results challenge traditional notions of AI performance and management standards.
A new industry benchmark conducted by Firmulate reveals that AI managers rarely score zero, even in worst-case scenarios, emphasizing the value of partial progress and trust in AI management. The final standings, published in July 2026, show scores topping out at 95, with the lowest at 26, illustrating a nuanced view of AI performance that challenges traditional grading expectations.
The benchmark involved four frontier AI models managing a small software company’s operations over seven days of crises, customer manipulations, and trust tests. Each model was tasked with handling real business challenges, with decisions fully auditable and scored based on effectiveness and integrity.
The top performer, gpt-5.6-sol, scored 95 out of a possible 100, while the lowest, Opus 4.8, scored 73. The surprising aspect is that the baseline, representing minimal effort or do-nothing management, scored 26, not zero. This reflects the benchmark’s principle that partial, useful work is valued, and that even minimal management efforts have tangible worth.
Another key finding is the emphasis on trust. The benchmark explicitly states that “no amount of good work outweighs a breach of trust,” meaning that a single trust violation can negate otherwise excellent performance. This principle underscores the importance of integrity in AI management, especially when AI agents are given access to sensitive company systems.
Performance was also measured under social engineering attacks, such as fake CEO messages and impersonation attempts. All models refused these manipulative requests, with Kimi K3 providing the most detailed reasoning for its refusal. Interestingly, models that read and referenced their own documentation were more successful in closing sales deals, highlighting the importance of thoroughness and proper information access.
Inside the Industry Benchmark That Resists Zero Scores for AI Managers
Four frontier AI models ran a small software company through its worst week — seven days of crises, customer manipulation and trust tests. The result: no zero scores, a hard cap on perfection, and proof that integrity outweighs brilliance.
Performance on the Scoreboard
Scores topped out at 95 while the lowest model scored 73. The true surprise sits at the bottom: the baseline representing minimal, do-nothing management still earned 26 points — because the benchmark values partial, useful work.
Three Principles That Redefine AI Evaluation
The benchmark shifts attention from flawless output to consistent, trustworthy management — measuring whether AI can finish tasks, read documentation, and keep integrity under pressure.
Partial Work Has Worth
Zero scores are effectively banned. Even minimal management effort earns 26 points, recognising tangible value in crisis situations rather than punishing imperfection with nothing.
One Breach Caps Everything
The scoring rules state that no amount of good work outweighs a breach of trust. A single trust violation can negate otherwise excellent performance across the entire week.
Documentation Drives Deals
Models that read and referenced their own documentation closed more sales deals — thoroughness and proper information access outperformed raw conversational skill.
No amount of good work outweighs a breach of trust.
Firmulate benchmark — explicit scoring ruleUnder Attack: Social Engineering Results
Every model was probed with fake CEO messages and impersonation attempts. All refused — but the quality of reasoning varied.
| Model | Fake CEO Message | Impersonation Attempt | Refusal Reasoning |
|---|---|---|---|
| gpt-5.6-sol | ✓ Refused | ✓ Refused | Concise, correct |
| Kimi K3 | ✓ Refused | ✓ Refused | Most detailed reasoning |
| Opus 4.8 | ✓ Refused | ✓ Refused | Adequate |
| All models | ✓ 100% refusal | ✓ 100% refusal | Integrity held firm |
From Crisis to Scored Verdict
Every decision in the benchmark follows an auditable chain — a model for accountability in AI governance.
🚨 Crisis Injected
A company’s worst week: outages, angry customers, and manipulative requests hit the models.
🧠 AI Manager Acts
The model reads documentation, makes calls, and manages operations in real time.
🔍 Decisions Audited
A fully auditable decision trail records every action for integrity review.
⚖️ Trust Check
Any breach of trust caps the final score, regardless of technical brilliance.
🏆 Score Awarded
Partial progress counts; the floor is 26, the ceiling is earned, never assumed.
Why This Benchmark Breaks the Mold
Traditional AI benchmarks measure conversational ability or task performance — rarely follow-through, integrity, or handling manipulation. Firmulate is among the first to simulate a company’s worst week end-to-end. But questions remain: results may not generalise beyond the tested scenario, and trust-weighted scoring may not capture every dimension of effectiveness.
Static Skill Tests
Prior benchmarks measured language understanding and isolated tasks, ignoring real-world process management and ethical behavior under pressure.
Live Worst-Week Sim
AI managers ran actual business operations for seven days with fully auditable decisions, scored on effectiveness and integrity together.
Broader Scenarios
Future rounds are expected to span more industries, weigh trust metrics further, and shape deployment standards for critical business functions.
At a Glance: FAQ
Why do AI managers rarely score zero?
The benchmark values partial, useful work and assigns 26 points to minimal effort, recognising that even basic management has tangible value in a crisis.
What does the score cap mean?
Perfect scores are treated as suspicious, and trust breaches are judged more damaging than minor lapses — ethics over technical prowess.
How does trust influence scores?
A single breach of trust can negate high performance, making integrity the decisive factor in the final evaluation.
Do the results apply to real deployments?
They’re specific to the tested scenario, but they signal that reliability and trustworthiness are the essential metrics for operational AI.
What comes next for benchmarking?
Expect more diverse scenarios and trust-focused metrics, shaping standards for AI in CRM, support, sales, and other critical functions.
Why did documentation matter so much?
Models that read and referenced their own docs closed more sales deals — evidence that thoroughness beats superficial fluency.
Implications of Partial Progress and Trust in AI Management
This benchmark shifts the focus from perfect scores to the importance of consistent, trustworthy management. It demonstrates that AI systems are better evaluated on their ability to finish tasks, read relevant documentation, and maintain integrity under pressure rather than on their ability to produce flawless results.
For organizations deploying AI managers, the results suggest that the key metrics are whether the AI can complete its tasks reliably and ethically, especially in high-stakes environments. The explicit rejection of zero scores and the cap on perfect scores reinforce that AI effectiveness is a spectrum, and trustworthiness is paramount.
Furthermore, the scoring system’s transparency and auditable decision trail provide a model for accountability in AI governance, which is increasingly critical as AI systems become more integrated into business operations.
As an affiliate, we earn on qualifying purchases.
Background on AI Benchmarks and Management Challenges
Traditional AI benchmarks tend to measure conversational ability, language understanding, or specific task performance, often ignoring how AI manages real-world processes or maintains trust. Recent developments have seen a push for more realistic testing, especially as AI tools are integrated into critical business functions like CRM, customer support, and sales.
Previous efforts have highlighted AI’s technical capabilities but rarely addressed issues like follow-through, integrity, or handling manipulative tactics. The Firmulate benchmark is among the first to simulate a company’s worst week, testing AI managers’ ability to handle crises, trust attacks, and decision-making under pressure.
The results challenge the assumption that AI’s value is solely in its intelligence or language skills, emphasizing instead the importance of reliability, ethical behavior, and thoroughness in real-world applications.
As an affiliate, we earn on qualifying purchases.
Unresolved Questions About Benchmark Limitations
It remains unclear how these results generalize beyond the specific scenario tested, or how different types of AI models might perform in other operational contexts. The scoring system’s emphasis on partial work and trust may not capture all dimensions of AI effectiveness in complex environments.
Additionally, the long-term implications of relying on such benchmarks for AI deployment standards are still being debated, and whether trust-focused metrics will dominate future evaluations is uncertain.
As an affiliate, we earn on qualifying purchases.
Next Steps in AI Management Benchmarking
Following the July 2026 results, industry leaders and AI developers are expected to refine evaluation standards further, emphasizing trustworthiness, follow-through, and ethical behavior. Future benchmarks may incorporate more diverse scenarios, including different industries and operational challenges.
Organizations planning to deploy AI managers should monitor these developments, consider participating in similar tests, and focus on building systems that prioritize integrity and reliability over superficial performance metrics.
Further research is also anticipated to explore how trust breaches impact long-term AI adoption and how to mitigate risks associated with AI decision-making in critical business processes.
As an affiliate, we earn on qualifying purchases.
Key Questions
Why do AI managers rarely score zero in this benchmark?
The benchmark values partial, useful work and assigns 26 points to minimal effort, recognizing that even basic management efforts have tangible value in crisis situations.
What does the score cap mean for AI performance standards?
The cap indicates that perfect scores are suspicious, and that trust breaches are considered more damaging than minor lapses, emphasizing ethical behavior over technical prowess.
How does trust influence AI management scores?
The scoring system explicitly states that a single breach of trust can negate high performance, making integrity a critical factor in evaluation.
Can these benchmark results be applied to real-world AI deployment?
While designed to simulate real business crises, the results are specific to the tested scenario. They suggest that reliability and trustworthiness are essential metrics for operational AI systems.
What are the future implications of this benchmarking approach?
Future benchmarks are likely to incorporate more diverse scenarios, emphasizing trust, follow-through, and ethical decision-making, shaping standards for AI in critical business functions.
Source: ThorstenMeyerAI.com
Fall Picks
fall essentials
As an affiliate, we earn on qualifying purchases.
