A growing chorus of researchers and commentators is making a seductive argument: that AI chatbots, unlike social media, push users toward expert consensus and moderate views.
For example, a much-shared article from The Financial Times presented evidence that large language models elevate mainstream science and scholarship — in contrast to social platforms, which reward outrage and fringe voices. If social media is “populist and polarizing,” the argument goes, AI may be the antidote.
We want this to be true. But we’ve spent years studying the forces that polarize Americans, including the role of social media, and the pattern we see emerging in the AI industry is unsettlingly familiar.
Facebook introduced its first engagement-based algorithm in 2009, replacing a chronological feed with one sorted by popularity. Twitter and YouTube followed similar paths. In doing so, they sacrificed the potential of these platforms to connect people across divides in favor of a business model — advertising — that required maximizing the time users spent on their apps. That decision was fateful.
The platforms didn’t intend to polarize America. They intended to maximize engagement, and polarization was the byproduct.
New research has shown that content associated with intergroup conflict and moral outrage is systematically amplified by engagement-based algorithms because divisive content captures attention and drives the engagement metrics that generate advertising revenue.
These algorithms don’t just show users polarizing content; they teach users to produce more of it. When a user posts moral outrage and gets rewarded with likes and shares, they learn to post more outrage next time.
The consequences may include political sectarianism, and social media’s engagement algorithms turbocharged this by overrepresenting extreme voices and making Americans believe their fellow citizens were more hostile than they actually are.
There is, to be sure, exciting evidence that chatbots can push toward truth and away from toxicity. For example, they can reduce beliefs in conspiracy theories and shift partisan attitudes through active listening.
But the excitement wilts once we recognize a crucial detail: Those findings describe chatbots that were deliberately designed to challenge users’ views with evidence. If past is prologue, that is the opposite of what an engagement-optimized system would do.
Even in their current form, AI chatbots possess a troubling design feature called sycophancy — the tendency to tell users what they want to hear rather than what is true. Multiple studies have documented this pattern across leading AI assistants, including in politically charged contexts. So the vision of AI as a neutral arbiter of expert consensus is already at odds with how these systems actually behave.
But it is about to get much worse. In January, OpenAI announced it would begin testing ads in ChatGPT. By February, ads were live for hundreds of millions of users on the free tier. Within six weeks, the company’s ad revenue surpassed $100 million on an annualized basis, with more than 600 advertisers signed up. Google has told advertisers it plans to bring ads to Gemini this year. Meta already uses data from interactions with its AI assistant to sharpen ad targeting across its platforms.
OpenAI has been emphatic in its reassurances. “Ads do not influence the answers ChatGPT gives you,” the company wrote when it launched the program. Facebook said the same kind of thing about its News Feed — that it simply showed users what was most “relevant.” The reassurance, both then and now, misses the point.
The issue is not whether an advertiser can directly alter a chatbot’s response, but rather how a business model dependent on user engagement subtly bends design decisions toward maximizing time on the platform. Nobody decides to make the product more polarizing. The incentive structure rewards whatever keeps people coming back, and what keeps people coming back is rarely what challenges them.
Layering advertising incentives on top of sycophancy produces a system primed to flatter users. Such flattery produces prolonged sessions that allow users to luxuriate in the validation of their preexisting beliefs, even as they become further convinced of the idiocy and turpitude of those who believe otherwise.
Indeed, a chatbot trained to please and optimized for engagement has every incentive to validate a user’s political grievances rather than offer the kind of firm, evidence-based pushback that the anti-conspiracy chatbot studies showed was effective.
Some will argue that our analogy between chatbots and social media platforms is overdrawn — that chatbots are conversational tools, not social networks. But the issue is not the technology. It is the business model. When the product is free and the revenue comes from advertisers, the money comes from capturing users’ attention. This was true of broadcast television. It was true of social media. And it will be true of AI.
We are not opposed to AI — far from it. The evidence we’ve cited suggests it can be a powerful tool for improving reasoning and reducing prejudice. But those benefits depend on what the chatbots are optimized for.
The argument that AI is fundamentally different from social media, that it will elevate expertise rather than amplify outrage and moderate views rather than entrench them, is seductive precisely because we want it to be true. But that argument deserves scrutiny, not credulity. If anything, the case for skepticism is stronger here than it was for social media.
Chatbots speak to us in the first person, adapt to our reactions in real time, and are trusted by users in ways that a Facebook feed never was. A technology that persuasive, paired with an incentive structure that rewards flattery, is not a corrective to the problems of social media. It is a more potent version of them.
Social media companies saw mounting evidence that optimizing for engagement fueled misinformation and eroded democratic health. They ignored it and pressed forward. We watched the consequences unfold in rising sectarianism and a fraying social fabric.
AI is giving us a rare second chance to scrutinize that decision before it hardens. The time horizon for that scrutiny is running short.
William J. Brady is an assistant professor of management and organizations at Northwestern University’s Kellogg School of Management. Eli J. Finkel is a professor of psychology and of management and organizations at Northwestern University.
Submit a letter, of no more than 400 words, to the editor here or email letters@chicagotribune.com.
