If you publish anything on the web, the question of whether AI answers in search send fewer visitors to your pages has probably crossed your mind. A new academic preprint offers one of the more careful attempts to measure it, and the headline figure is hard to ignore. In this article we will discuss how the study was run, what it found about clicks and user satisfaction, which limits its authors and commentators point out, and what site owners can sensibly do with the results.
Researchers, Design and Method
Authors and Paper
Researchers at the University of Pennsylvania and Northeastern University posted a preregistered field experiment on the arXiv preprint server on 18 August 2026. Its title is “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence.” The detailed account here comes from a TechWyse write-up published on 22 September 2026, so the specifics below are as reported there and are worth checking against the paper itself.
Two terms deserve a quick explanation. A preprint is an early version of research shared publicly before it has gone through peer review, so its conclusions have not yet been checked by independent experts chosen by a journal. Preregistration means the researchers stated their hypotheses and methods in advance, which makes it harder to go hunting for flattering results afterward.
Design of the Experiment
The team recruited 1,100 adults in the United States between 17 and 19 March 2026. Participants had to use Chrome and Google Search as their main tools. A browser extension then randomly assigned each person to one of three conditions, which is what makes this an experiment rather than a survey or a log of observed traffic.
| Condition | What participants experienced |
| Standard Google, AI suppressed | AI Overviews and AI Mode were hidden |
| Default Google | The normal results page, with AI features as Google shows them |
| Forced AI Mode | Searches were routed into AI Mode |
Random assignment matters because people who choose AI features may differ from people who avoid them. By letting chance decide, the researchers can compare groups that are alike on average and attribute differences in behavior to the condition itself. The sample is American and limited to Chrome users, so the results do not automatically transfer to other countries, other browsers or phones.
Unusual Method
Most public numbers about AI and website traffic are drawn from publishers’ own analytics or from panels that merely observe what people do. This study intervened instead. The extension changed what participants saw on the page, and the researchers then watched how searching behavior diverged. That is closer to how medicine tests a treatment than to how marketers read a dashboard, and it is why the paper is getting attention beyond the usual search-industry audience.
It is also worth knowing what such a design cannot do. Participants agreed to install software that records their search activity, and people who accept that are not a random slice of the population. The recruitment window was also very short, three days in March 2026, so the results show a snapshot of products as they stood then. Google changes its search features frequently, and behavior could look different by now.
Headline Numbers on Clicks and Searching
According to the TechWyse report, being assigned to AI Mode reduced external click-through rates by 18.8 percentage points. In the other direction, removing the AI features increased click-through by 8.8 percentage points. Those two figures describe the gap between the experience on a results page with AI answers and the experience without them.
The two numbers are not mirror images, and it would be a mistake to expect them to be. The default group already saw AI features, so removing them restored some clicks, while forcing everyone into AI Mode pushed behavior much further in the other direction. In other words, ordinary Google with its AI Overviews sits somewhere between the two extremes, and the study treats that middle experience as its reference point. The sheer size of the AI Mode effect is partly a result of how total the forced condition was, since few real users spend every search inside it.
Percentage Points Versus Percent
A percentage point is the arithmetic difference between two rates. The coverage reviewed here does not give the baseline click rate, so we cannot turn 18.8 points into a share of lost traffic, and you should be suspicious of anyone who does. As an illustration with invented round numbers, imagine that 50 out of every 100 searches ended in a click to an outside website. A fall of 18.8 points would leave about 31 of every 100, a drop of more than a third in relative terms, but if the real starting rate were 20, the same fall would be a much larger relative loss. The same drop in points can mean very different things depending on where you start.
Sites That Lost Clicks
The paper reports that AI Mode reduced clicks to news sites, Reddit and Wikipedia. That list is informative, because it spans professional journalism, community discussion, and a general reference source, which are very different kinds of publishers. It suggests the effect was not limited to one type of site, although the coverage does not break the numbers out by site.
Time, Sessions and Rival Engines
Users spent more time per session in AI Mode, but they ran fewer sessions. A longer visit in one place and fewer visits overall is a pattern that fits people getting what they want from a single conversation instead of bouncing between searches and pages. Heavy Google users showed a larger drop in sessions than lighter users.
For a publisher, this combination is a double-edged signal. Fewer sessions mean fewer chances for your pages to appear in front of someone, and longer sessions inside the AI product mean the engagement is happening somewhere you cannot see. It is the kind of pattern that makes ordinary analytics look flat even as the underlying behavior changes, so it helps to track more than one measure.
There was also a shift toward rivals. Assignment to AI Mode raised the share of users searching on Bing, DuckDuckGo, or Yahoo by 11.2 percentage points. For site owners, that is a reminder that audience behavior is not fixed, and that some searchers will move their habits when a product changes in ways they dislike.
User Ratings of the Experience
The study did not only count clicks. It also asked people how they felt about their searches, and the picture there is less flattering to AI Mode. The reported results show that AI Mode reduced trust, satisfaction, perceived usefulness, sense of agency and perceived relevance. Sense of agency is the feeling of being in control of the search, which can drop when an answer is handed over instead of discovered.
Removing the AI features, by contrast, produced no detectable change in how people rated their experience. In plain terms, taking AI away did not make participants unhappy, while pushing them into AI Mode did seem to. For news queries, the difference in perceptions was not statistically significant, which means the data cannot rule out that it was due to chance within that smaller slice.
The paper’s title makes a claim about user experience, and that framing deserves some caution. These are self-reported perceptions from a particular group at a particular time, and they may change as the products change. Still, they matter to publishers for a practical reason. A common argument in favor of AI answers is that readers prefer them, even if publishers lose some visits. This experiment found no improvement in the participants’ own ratings, at least in this setup.
That does not mean AI answers are poor for every task. Plenty of people find them useful for summarizing, comparing options or getting oriented in a new subject, and the study reports averages across all participants and queries. What it does show is that the average searcher in this sample did not report being better off, so the claim that publishers’ losses are readers’ gains has less support here than its supporters might hope.
Limits of the Study That Readers Should Weigh
Hidden Features That Stayed Visible
TechWyse highlights a technical complication. Google changed the HTML of AI Overviews while the study was running, which meant the browser extension could no longer reliably hide them. As a result, the extension hid them in only 51.1 percent of the no-AI condition. That means the group meant to see no AI features still saw them roughly half the time, which blurs the comparison.
The direction of this error is worth thinking through. If the supposedly AI-free group was partly exposed to AI, then the true gap between a clean AI-free results page and an AI Mode page could be larger than what the experiment measured, not smaller. But that is reasoning about a measurement flaw, not a finding, and the researchers themselves are better placed to say how much it matters.
Readers who want the detail should look for how the paper describes this problem, including whether the analysis was adjusted for it. Good research reports such limits openly, and the fact that this one is public in a preprint means other specialists can examine it and say whether the corrections hold.
Missing Outside Verification
TechWyse also notes that Google has not published independently verifiable click or referral data for AI Mode. That means the results cannot be checked against Google’s own figures, and there is no neutral dataset to settle disagreements. The coverage reviewed for this article includes no response from Google, so we do not know how the company views the findings, and we will not guess.
Preprint Status
The paper has not been peer reviewed. Peer review is imperfect, but it is the process through which outside specialists test methods and challenge claims, and a preprint can change substantially or be corrected before publication. Treat the numbers as strong evidence worth taking seriously, not as settled fact. The sample of 1,100 is respectable for a field experiment, but small relative to the total population of searchers, and people recruited for studies are not a perfect mirror of everyone online.
Earlier Experiment Points in the Same Direction
The new study is not the first randomized test of AI in search. Search Engine Journal reported on 27 April 2026 on an earlier experiment, whose researchers were not named in the article. That experiment used a Chrome extension to randomly hide or force AI Overviews for more than 1,000 desktop Chrome users recruited through Prolific, a platform that pays people to take part in research.
| Feature | Earlier experiment (reported by Search Engine Journal) | Penn and Northeastern preprint (reported by TechWyse) |
| Participants | More than 1,000 desktop Chrome users via Prolific | 1,100 US adults on Chrome |
| AI feature tested | AI Overviews | AI Overviews and AI Mode |
| Reported click effect | Outbound clicks down 38 percent on queries that triggered them | External click-through down 18.8 points in AI Mode |
| Other findings | Zero-click searches rose from 54 percent to 72 percent | Fewer sessions, more searching on other engines |
The earlier experiment found that AI Overviews cut outbound clicks by 38 percent on queries that triggered them, and that zero-click searches rose from 54 percent to 72 percent. A zero-click search is one where the person finishes without visiting any outside page. Satisfaction, information quality and ease of finding information stayed flat. AI Overviews placed in the top position roughly halved click rates.
Do not add the two sets of numbers together or compare them directly. They measure different things, in different samples, using different baselines: one is a relative reduction on triggering queries, the other a percentage-point change across a broader set of searches. What they share is direction, a drop in outside visits without an offsetting gain in satisfaction. The Search Engine Journal commentary on the earlier study also noted that it covered desktop Chrome only and called for peer-reviewed replication, a call that applies equally to the new preprint.
Two experiments, different teams and different months, pointing the same way is more persuasive than either alone, though still short of proof. Independent replication is what turns a promising result into an accepted one, because it shows the pattern does not depend on one sample, one recruitment method, or one software setup. Until that happens, a careful reader holds the evidence firmly enough to act on and loosely enough to revise.
Reading a Randomized Experiment Sensibly
Strengths of the Design
Most claims about AI and traffic come from before-and-after comparisons of a site’s own analytics, which are hard to interpret because many things change at once. Seasonality, algorithm updates, competitor moves and shifts in content all muddy the picture. A randomized experiment sidesteps much of that by creating comparable groups, so it carries more causal weight than a trend line from a single site.
Points Where It Can Mislead
A study of volunteers using an extension is still not the same as the real world. Participants know they are being observed they may behave differently, and the experiment forces conditions, such as locking people into AI Mode, that most users would never face. Real behavior also differs between people searching for a recipe, a medical question, a product, or breaking news, and a pooled average hides those differences.
One more habit is useful when you read any study like this: look for what the authors did not measure. This one reports clicks, sessions, search engine choice, and perceptions. It does not report revenue, subscriptions, ad income, or whether visitors who did click were more valuable. A publisher whose lost clicks were casual one-time visitors is in a different position from one whose lost clicks were its most loyal readers, and the study cannot tell those cases apart.
The sensible reading is that the study gives credible support to a concern many publishers already had, while leaving open how large the effect is on any particular site. Resist both extremes: the idea that this proves search traffic is finished, and the idea that it proves nothing. Neither follows from the evidence.
If you want a quick test of any headline about AI and traffic, ask four questions. Who was measured, and are they like your audience? What exactly was compared with what? Is the size of the effect given in points or percent, and from what baseline? And has anyone outside the author team checked it? This study answers the first two well, the third only partly in the coverage reviewed here, and the fourth not yet.
Consequences for Content Sites and How to Measure Them
Your Own Data First
Begin in Search Console. Compare clicks with impressions over time, because a site whose impressions hold steady while clicks slip is showing the signature of results that display your content without sending visitors. Segment by query type, separating informational questions from transactional or local searches, and by country, since AI features roll out at different times in different markets. Annotate dates when new search features reach your audience, and keep those notes next to your charts.
A simple way to split queries is by intent. Branded searches, where people type your name, behave differently from broad informational ones, and a dip confined to the second group looks more like answer-style competition than a loss of reputation. Search Console lets you filter queries by text and export them, so even a small site can build a rough grouping in an afternoon. Revisit it monthly and compare against the same weeks the year before when you can.
Be careful with causation. A fall in clicks can come from AI answers, but also from lost rankings, seasonality, a technical fault or a competitor publishing something better. The experiment shows an average effect across many participants and does not tell you which of these explains your own dip.
Content That Still Earns the Click
From general experience, pages that offer something a summary cannot easily replace tend to hold up better: original data or research, tools and calculators, first-hand testing and photographs, detailed comparisons, and local or transactional pages where the visitor needs to act, not just read. If your page only restates widely known facts, an AI answer can replace it. If it adds evidence, judgment, or an experience that others lack, there is a stronger reason for a reader to click through.
Widen the Doors Into Your Site
Dependence on a single source of visitors is a risk, whatever the technology. Build an email list so you can reach readers directly, encourage bookmarks and direct visits, and take part in communities where your audience already gathers. Video can reach people who prefer to watch, and other search engines still send visitors, as the shift of some participants toward Bing, DuckDuckGo, and Yahoo hints. None of this means abandoning search engine optimization, which still matters, but it does mean treating Google as one channel rather than the whole business.
Smaller sites often do best by choosing channels that match what they already do well. A cooking blog may find video natural, while a local trade business may gain more from reviews, directory listings, and word of mouth. A niche research site may build loyalty through a weekly email. The point is to make the audience yours in some form, so a change to one results page does not sever your connection to it.
Brand strength also helps. People who recognize your name are more likely to search for it, type your address, or choose your result when they see it. That kind of recognition is built slowly through consistent quality, and it is hard for any interface to erase.
A Sensible Order of Choices for the Coming Months
Start with measurement, because decisions without data tend to be expensive. Pull a year of Search Console data, split clicks and impressions by query type and country, and note where the gap between them widens. If you see no such pattern, there is nothing to fix yet, and the study is simply background reading.
Next, identify the pages that drive most of your revenue or leads and ask how exposed each is. A page that answers a simple factual question is more exposed than one that sells a product, books an appointment, or hosts a tool. Put your improvement effort into the pages that matter most and offer something distinctive, and be honest about those that never will.
Then add routes that do not depend on a results page. Pick one or two, for example, a newsletter and a community, and run them consistently for several months before judging them. Spreading across five channels badly is worse than doing two well.
Be practical about time as well. A one-person site cannot do everything, so decide what you would stop doing to make room for something new. Many owners find that a few hours a week moved from chasing minor ranking tweaks to writing an email or recording a short video pays off more over a year, though that is a judgment call, not a result from the study.
Finally, keep watching the evidence. A peer-reviewed version of this paper, or independent data from Google, would change how confident anyone can be. Until then, treat the 18.8-point figure as a serious signal from one careful US experiment, not as a forecast for your site. This is general information and not professional advice, and anyone running a business that depends on search may want an experienced analyst to review their own figures.
Frequently Asked Questions
1. How Much Did AI Mode Reduce Clicks in the New Study?
According to the TechWyse report on the Penn and Northeastern preprint, AI Mode reduced external click-through rates by 18.8 percentage points. That is a difference in rates, not a share of all traffic lost.
2. Who Ran the AI Mode Experiment?
Researchers at the University of Pennsylvania and Northeastern University posted it on arXiv on 18 August 2026. It involved 1,100 US adults who used Chrome and Google Search as their main tools.
3. Can I Trust a Preprint Like This?
Treat it as credible but early. A preprint has not been through peer review, and the report notes that the AI-hiding extension worked in only 51.1 percent of the no-AI condition.
4. Does the Study Mean I Should Stop Doing SEO?
No. The study measures clicks under different search setups and does not show that search stopped sending traffic. Keep improving your pages, and add other channels so you rely less on one source.
5. Is the Earlier 38 Percent Finding the Same Thing?
No. Search Engine Journal reported on a separate experiment in which AI Overviews cut outbound clicks by 38 percent on queries that triggered them. It measured a different feature, sample, and baseline, so the numbers should not be compared directly.
6. Should I Expect the Same Drop on My Own Site?
Not necessarily. Averages across a study sample say little about a specific site, so check your own Search Console clicks and impressions by query type before making changes.
