Google’s AI Overviews: 5 things publishers should know

After the U.S.-wide rollout and last week’s hullabaloo about odd and incorrect answers, we take a look at some things publishers need to know

Jun 03, 2024

#SPONSORED

NewzDash: Best Real-Time News SEO Tool & Dashboard

Traditional SEO tools lag with old data, NewzDash delivers real-time search volumes and rankings. Get hourly search volume estimates and rankings updated every 15 minutes with NewzDash — the only tool designed for news SEO.

Try a demo today & Get your FREE Site Analysis!

Hello, and welcome back. Shelby here, back from a beautiful weekend celebrating my best friend’s engagement. Love is great. The feeling I had after the many, many seltzers I consumed? Not so much. Meanwhile, did you know it rains in Vancouver? Thankfully, both a drizzle and a downpour are easier to handle when the cabin views include mountains (says Jessie, live from the mountains).

This week: Five things to know about Google’s AI overviews, the SEO community’s commentary and the potential threat this change poses for the news business. It’s been a crazy couple of weeks in the industry, so let’s break it down.

Join our masterclass: In just EIGHT days, we’re teaming up with
Barry Adams
for another masterclass. If you’ve ever looked at the crawl stats report in Google Search Console and thought, “What is this telling me?” Or do you see the message, “Duplicate, Google chose different canonical” for a URL, but you don’t know what to make of it? If so, this class is for you! Spots are limited, so sign up today.

Let’s get it.

1. Google launched AI Overviews in the U.S.

For almost a year, Google has tested and experimented with Search Generative Experience. Last month, on May 14, Google announced the launch of AI Overviews — the new name for SGE — in the U.S., with a few tweaks to the original prototype.

An AI Overview is an AI-generated answer to a query on Google that shows at the top of the search results. Depending on the search term, it will provide an answer that combines multiple sources from the internet.

Clicking “show more” on a result returns more detailed information and up to 10 links with citations.

Who does this show up for and how?

Most of the data around AI Overviews is very limited, as currently you must be in the United States, be logged into your Google account, using a Chrome browser and not be incognito mode (y’know just a few conditions). Therefore, it’s harder for search tools and APIs to get clear data.

Pulling from Barry Adams’ incredible newsletter on AI Overviews (based on a talk he gave at WAN-IFRA World News Media Congress — go look at the slides, too), the data we do have is anecdotal and based on small query sets. According to the newsletter issue, estimates of how often AI Overviews are shown range between 1.5 per cent and 42 per cent of Google queries.

On average 14 per cent of queries show AIO by default;
28 per cent of queries have a Generate option to create an AIO;
58 per cent of queries have no AIO feature.

A small study done by Kevin Indig, who runs Growth Memo, went deeper into when AIOs show up using ZipTie, Ahrefs and Google Search Console. Informational queries that begin with “how” and “best” are more likely to show an AIO feature than terms that start with “buy,” according to Kevin. AIOs are also more likely to show up alongside People Also Ask (PAA), Featured Snippets and Discussions and Forums modules, which are common for informational searches.

Query syntax and AIO — A screenshot from Kevin Indig's study on the correlation between words included in a query and if an AIO is triggered.

SEMRush came out a few days after the AIO launch with an update to their that enables tracking these overviews. They offer two tools, Position Tracking and Sensor (how much AI Overviews appear across various industries).

Interesting note: In some early iterations of Google’s AIO, it seems they are also serving ads. This shouldn’t be surprising, but unfortunately, it’s another indication of how Google is looking at this new AI feature.

2. Reports of issues ignited chatter beyond the SEO industry

Despite Google’s claim that people want AI Overviews, there was an influx of concerns and user contempt of the product.

Many Google users went searching for how to turn off the feature. Using Google Trends, you can see the increase in people searching “ai overview” after the product’s release in the U.S.

Interest spiked during the week people were receiving faulty, incorrect and concerning responses to some questions, which were widely reported by news organizations like the New York Times and The Verge.

From the “ai overview” search screenshotted above, there were a number of breakout queries around “how to turn off google ai.”

Google turned on AI overviews for millions of users — without a clear option to opt out. Going incognito or logging out of your Chrome does seem to disable the feature, but it is not clear exactly how to turn it off.

Additionally, these queries were returning sites that, in normal circumstances, would not rank at the top of search results. Lily Ray reported finding URLs ranking in AIO features that:

Include parked domains, dead sites, 404, 410 and 405 response codes linked in AIO, including for health queries;
Include examples of site reputation abuse (aka “parasite SEO” appearing in the links);
Include tons of results with random, outdated posts on Reddit and Quora;
Many non-U.S. sites ranking for U.S.-specific queries;
Include links to extremely low quality blogs and e-commerce websites with little to no organic traffic;
Include extremely biased content, like a blog on a plastic surgeon's website or K-beauty brand suggesting their products and services as the best treatments.

Chatter around the inaccuracies and misleading information took over the internet last week — even beyond the SEO industry.

The New York Times, The Verge, users on Twitter and Reddit have all reported on inaccurate information being highlighted in Google’s AIO.

The New York Times reported that some users were told to ingest rocks for vitamins and minerals (that information is pulled from The Onion, a satire website.);
The Verge reported on Google’s now-viral recommendation to put glue on pizza, and Google’s frantic push to try to “fix” the feature.

In a comment to the New York Times, Google spokeswoman Lara Levin said that many of the examples of inaccurate information have been “uncommon queries” and that the company has “also seen examples that were doctored or that we couldn’t reproduce.”

(Some screenshots of wildly incorrect overviews could not be replicated. And because it’s the internet, many screenshots floated were the work of Photoshop.)

So while, as Lara suggests, the searches might be “uncommon queries,” inaccurate information is inaccurate information. As access to AIO increases, the volume of “uncommon” or entirely new queries is only set to grow, too. Accuracy at scale remains a potentially huge concern for Google’s AI-written results.

3. Google responded to the concerns, clarifying WHAT?

Google responded last week to the concerns from the public in a blog post. Google’s Liz Reid, vice-president and head of Google Search, admitted there were errors in AI Overviews. She said that “when AI Overviews get it wrong, it’s usually for other reasons: misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.” Liz added that this happens with other features, too.

As a response, Google says it “made more than a dozen technical improvements to our systems” following the initial release to cut back on answers for “nonsensical queries,” limited user-generated content and “triggering restrictions” for searches where AIO is less useful.

The blog post also says that some genuinely incorrect answers were corrected with “improvements to our algorithms” or “established processes to remove responses that don't comply with our policies.”

Google also wrote that “because accuracy is paramount in Search, AI Overviews are built to only show information that is backed up by top web results.”

However, as Lily also pointed out, not all the results are from top organic results. “The cited links often… don’t rank anywhere in the top 100 results, which I (and others) have shown many examples of in the past week,” she wrote on LinkedIn.

4. News seems to be omitted

Barry tried to trigger an AIO in combination with a Top Stories box for a news query. Despite his best efforts — including searching hundreds of news-related queries — an AI Overview did not show up with any news terms.

And in Google’s response to concerns raised last week, it said it already has “strong guardrails in place” for topics like news and health. The company said it aims “to not show AI Overviews for hard news topics, where freshness and factuality are important.”

A line in Google's response last week says they will not show AI Overviews for hard news topics.

This is good news for the journalism industry, and those who use search to find news and information. The major concern for news publishers came from how AIO would change the Top Stories carousel and therefore news search behaviour as a whole.

However, if Google continues to limit how often this feature is surfaced when a query is related to news and YMYL topics (hopefully), this should not affect traffic levels for publishers too much.

There are still a lot of unknown factors — especially for niche publishers that focus on topics outside of hard news. Since AIO is only served to logged in users on Chrome (unlike some search data tools) and reporting is not available in GSC, there are limited options to see exactly where your content is being affected.

5. AI Overviews seem to be much more sparing than anticipated, but are still in the environment

It's still early, but it does appear AI Overviews are triggered less often than expected, with daily changes in the how and when they appear. In Kevin’s small study of 1,675 queries in the health vertical, 42 per cent of queries returned an AIO. However, this may have gone down since publication. Lily had an instance where she had an AI Overview come up for the healthy query “giardia treatment” — which provided a wildly misleading answer — but the next time she ran the query, the feature wouldn’t generate a response.

In fact, several example queries from Google’s blog post announcing Search Generative Experience in 2023 do not return an AI Overview. The primary example Google used to demonstrate SGE’s utility was, “what's better for a family with kids under 3 and a dog, bryce canyon or arches?” Currently, it does not return an AIO.

Screenshot of Google's results page as of June 1.

The suggested follow up question Google used, “how long to spend at Bryce Canyon with kids,” also only displays a featured snippet.

It’s interesting to see that even Google’s prime use cases for AI as a value-add on the SERPs does not make use of AI-generated overviews.

The bottom line and next question: Where is Google’s public editor? Most news organizations have a readers or public editor — a person who owns the errors made by the organizations, issues corrections and clarifications and is responsible for setting editorial standards. Google, despite now creating and publishing content (like publishers do), has no such role. As we move forward in the next phase of AIO, it will be interesting to see what the feature develops into.

#SPONSORED - The Classifieds