MH-6272MarTech & AI
Epoch: No AI Model Met Employee Standards on Any Task
Epoch found no tested AI model met its employee standards on any task. Claude Fable 5.2 led an AI data-center assignment, but models favored weak or repetitive research.
Wire notes
- Epoch found no tested model met its employee standards on any individual task.
- GPT-6.2 Astra averaged 65%, 17 percentage points below Claude Fable 5.2 at 82%.
- Fable 5.2 and Opus 5.5 scored about 98% and 88% on the AI data center assignment.
- Opus 5.5 finished in just under three hours; Fable 5.2 took around 6.5 hours and delivered higher-quality work.
- Models preferred their own output over human work except in one comparison, where Gemini 4 Argon favored humans.

By Priya Raman
No tested AI model met the standards Epoch expects from its employees on any individual workplace task, according to the AI research organization’s new evaluation.
The result contrasts with strong general benchmark scores. GPT-6.2 Astra averaged 65% on Epoch’s workplace evaluation, 17 percentage points below Claude Fable 5.2 at 82%. Epoch says the two models roughly tied on its Epoch Capabilities Index.
Kelly Hong explained in the report: "GPT-6.2 Astra improved over GPT-6 Astra in several tasks, reaching performance comparable to Gemini 4 Argon." She added: "We saw more concise writing, improved graphic design, and a 25-point jump in AI data center research."
What does this change for AI buyers and marketing teams?
It changes what buyers should seek beyond a benchmark score. Epoch’s findings support task-specific trials that measure experimental design, the quality of findings, clarity of presentation and whether a model improves its work after the first attempt.
One assignment provided a clearer comparison. Epoch gave the models an AI data center research assignment:
- Claude Fable 5.2 scored about 98%.
- Opus 5.5 scored about 88%.
- Epoch judged both outputs comparable to work from its contractors, who typically spend about 16 hours researching each data center.
Speed and quality moved in different directions. Opus 5.5 completed the assignment in just under three hours at a lower cost. Fable 5.2 took around 6.5 hours and produced the higher-quality result, according to Epoch.
Epoch’s findings were less favorable across other research tasks. The organization said all tested models struggled with:
- Designing experiments.
- Identifying meaningful findings.
- Presenting research clearly.
On a chess task, all tested models used existing variants rather than creating new ones. Epoch said that choice made it difficult to determine how well the models could handle unfamiliar work.
The report highlighted a related presentation problem: "Claude models used variants that already exist, but presented them as novel." Epoch also found that all tested models struggled to improve their research and often relied on their first attempt throughout the project.
Models also showed a consistent preference for their own work. They chose their own outputs over human work in every comparison except one. Gemini 4 Argon favored the human outputs in that exception, Epoch said.
The model evaluations often focused on superficial measures, including the number of experiments conducted, instead of the quality of the research. Epoch said this approach could reward visible activity while missing weak experimental design or unclear findings.
The organization cautioned that the study has three material limits. Each model was tested only once per task, the task selection was limited, and several grading criteria involved subjective human judgments.
For marketing teams evaluating AI research tools, Epoch’s results favor side-by-side workflow trials over benchmark rankings alone. The practical test is whether a system can improve a task, explain its findings and win human preference; the report’s one-run design also makes repeat testing prudent before purchasing decisions.
via digitalinformationworld.com (Source)
More from Priya Raman
Show full bio
Correspondent covering people, launches and day-to-day practice in marketing, advertising, marketing technology, media, SEO and search at Marketing Herald.
348 articles
More on the wire
- OpenAI Launches GPT-6.2 Astra After Safety Scrapped Its Predecessor
- OpenAI fires back at Claude with GPT-6.2 Astra at $8 per million tokens
- Gemini 4 Carbon trails Claude Opus 5.5 in new benchmark but wins on cost
- Google Fuses AI Mode With Search as Gemini Hits 1 Billion Users
- Only 10.8% of AI Agent Projects Reach Scale as Martech Spend Climbs