Generative artificial intelligence has swept into workplaces from classrooms to courtrooms, but as adoption rockets across the economy, some are pressing pause to ask just how valuable this technology really is for growth.
Much of the skepticism circles around a persistent and sometimes embarrassing problem: these large language models can sound convincing even when inventing facts entirely. The phenomenon, known as “hallucination,” has sparked a few disasters already in the legal world.
Tahir Khan, a barrister, recently recounted cases where lawyers relied on AI-written filings and discovered the AI had slipped in imaginary laws and court cases with plausible details. Khan cautioned, “Hallucinated legal texts often appear stylistically legitimate, formatted with citations, statutes, and judicial opinions, creating an illusion of credibility that can mislead even experienced legal professionals.”
The technology’s tendency to authoritatively present fiction as reality raises bigger questions. Critics argue that, by design, these systems are not reasoning engines but are built to predict the next likely word or phrase based on their vast training data. Ed Zitron, a tech journalist, dismisses the current crop of chatbots for business, explaining that their readiness to state untrue “facts” means, “It’s a non starter for most business customers, where (obviously) what you write has to be true.”
Why It Matters for Work and Policy
Even as newer models promise advances, an latest advancements in large language models study highlighted that even the sharpest “large reasoning models” collapse over complicated tasks. Some researchers describe these blunders as something closer to bluffing or “bullshit,” with Michael Townsen Hicks writing in a recent paper, “Large language models simply aim to replicate human speech or writing. This means that their primary goal, insofar as they have one, is to produce human-like text.”
Officials are now grappling with the limits of AI’s reliability. Instead of these models replacing human staff wholesale, many experts suggest limiting them to support roles rather than handing over the reins.
Daron Acemoglu, last year’s co-recipient of the Nobel in economics, predicts significant effects only in narrow industries for now. Data summarization, visual pattern spotting, and similar office roles make up, in his estimation, about five percent of the economy.
That has real repercussions for policymakers. Hopes that AI could rescue stagnant productivity in places like Britain are starting to look overblown. If benefits remain this mixed, the social and economic risks—energy drain, information pollution, and public confusion—take on more weight and urgency.
Ethics scholars warn of broader consequences, too. Sandra Wachter of the Oxford Internet Institute explained, “Everybody’s just throwing their empty cans into the forest. So it’s going to be much harder to have a nice walk out there because it’s just being polluted, and because those systems can pollute so much quicker than humans could.”
The message for governments and business leaders is coming into focus now: while AI tools are breathtakingly good at sifting and spinning information, their tendency to invent, embellish, or completely fabricate means we should use them with caution. The allure of an always charming, ever helpful assistant may never fully eclipse the irreplaceable value of a sharp, discerning human mind according to performance of the newest generative models.
