Artificial intelligence just got a lot smarter — and a lot less reliable at the same time.
New findings from OpenAI highlight a concerning trend in some of their most capable models. Their latest reasoning systems, code named o3 and o4 mini, produced inaccurate responses up to 48 percent of the time when put through their paces with a benchmark called PersonQA. That’s not just a marginal problem, either — those error rates have soared in comparison to previous models like o1.
This unintended effect puts a spotlight on the trustworthiness of these large language models. Eleanor Watson, who specializes in AI ethics for Singularity University and is a member of the Institute of Electrical and Electronics Engineers, pointed out exactly why this matters. “When a system outputs fabricated information — such as invented facts, citations or events — with the same fluency and coherence it uses for accurate content, it risks misleading users in subtle and consequential ways,” she explained.
The dilemma is striking: as AI gets more creative, the more likely it is to produce convincing but false responses. In other words, these systems don’t just make mistakes — they often do it with such polish that users could easily be fooled. This has real world implications, especially in sensitive sectors like healthcare, finance, and law where accuracy is critical.
Hallucination: The Strange Side Effect of AI Creativity
Surprisingly, hallucination is not a simple flaw. Sohrob Kazerounian, a researcher at Vectra AI, sees it differently. “It’s important to note that hallucination is a feature, not a bug, of AI,” he said. The basic premise is this: for a model to deliver fresh ideas — to compose original music or answer questions no one’s ever asked — it has to go beyond just repeating information it’s been fed. Sometimes, that leap into creativity means making up details.
Still, the deeper these systems reason, the harder it gets to tell when they’re right and when they’re confidently selling fiction. The danger sneaks up when AIs embed their mistakes within logical sounding explanations, making errors harder and harder for humans to spot. Users may not even realize when they’ve been misled until it’s too late.
To make matters more complicated, deciphering why an AI chooses one solution over another is nearly impossible. Even the experts at the top of the field are left puzzling over the opaque thinking process behind algorithmic decisions. Dario Amodei, who leads the AI company Anthropic, recently wrote about this mystery, pointing out that no one really knows why models like these make the choices they do — right or wrong.
AI’s persistent tendency to invent information has already resulted in chatbots creating fake sources and making up policies for companies, often without any simple fix in sight. Experts like Watson believe some progress can be made by grounding AI’s answers in real world, verifiable data, and by training them to double check their own logic for consistency.
But until such measures are perfected and widespread, caution will remain the best tool for anyone relying on AI for facts instead of fantasy, especially as research into large language model accuracy (target=blank href=”https://openai.com/index/gpt-4-research/”) and recent growing concerns over chatbot hallucinations (href=”https://critiqs.ai/ai-news/concise-prompts-raise-risk-of-ai-chatbot-hallucinations/”) continue to highlight these risks.
