Google’s new head of search, Liz Reid, says the recent AI Overview’s inaccurate answers affected less than one in every 7 million search queries.
The recent Google Search data leak diverted attention from another embarrassing episode for Google earlier this week, the crazy and potentially dangerous answers its AI Search Overviews gave upon its launch.
Google responds to criticism
Google has been addressing and resolving the AI Overviews problems.
In a recent blog post, Liz Reid explains where the inappropriate answers originated from, how AI Overviews works, and the improvements Google has made to ensure it doesn’t advise people to relax and unwind by bathing with a toaster.
Liz Reid kicked off her post by accusing people on social media of spreading false screenshots of AI Overviews providing inappropriate responses.
- “In the last week, people on social media have shared some odd and erroneous overviews (along with a very large number of faked screenshots). We know people trust Google Search to provide accurate information, and they’ve never been shy about pointing out oddities or errors when they come across them — in our rankings or in other Search features. We hold ourselves to a high standard, as do our users, so we expect and appreciate the feedback and take it seriously.”
Liz also said:
- “Users have a higher satisfaction with their search results, and they’re asking longer, more complex questions that they know Google can now help with.”
AI Overviews doesn’t hallucinate
AI Overviews erratic answers weren’t because of a sip of Psilocybin tea, as Google’s Liz Reid explained:
- “AI Overviews generally don’t hallucinate or make things up in the ways that other LLM products might.”
Adding:
- “When AI Overviews get it wrong, it’s usually for other reasons: misinterpreting queries, misinterpreting a nuance of language on the web, or not having a lot of great information available.”
Google addresses those “odd results”
Google’s head of search explained they extensively tested AI Overviews before launching. Admitting, however, that there’s nothing like the real thing.
- “There’s nothing quite like having millions of people using the feature with many novel searches.”
Liz also pointed out that people intentionally tried to make AI Overviews give weird results by asking inappropriate questions.
- “We’ve also seen nonsensical new searches, seemingly aimed at producing erroneous results.”
Google also said numerous screenshots were manipulated to present false AI Overview responses, asserting that “Those AI Overviews never appeared.”
Eating rocks and data voids
The AI Overview answer to “How many rocks should I eat” made global headlines; Liz approached this hot rock in her post, explaining that data voids were to blame.
A data void means what you’re probably thinking: a lack of data on a particular subject. It’s no great surprise to hear that “How many rocks should I eat?” was a query with few historical searches and even less quality content.
Google explained how AI Overviews gave its answer:
- “However, in this case, there is satirical content on this topic that also happened to be republished on a geological software provider’s website. So when someone put that question into Search, an AI Overview appeared that faithfully linked to one of the only websites that tackled the question.”
Google shares improvements
Google has taken recent search result examples to identify patterns where it went wrong, making numerous technical improvements to its systems.
Liz shared some improvements Google has made to AI Overviews, explaining, “We don’t simply fix queries one by one, but we work on updates that can help broad sets of queries, including new ones that we haven’t seen yet.”
Here are some of those improvements:
- We built better detection mechanisms for nonsensical queries that shouldn’t show an AI Overview and limited the inclusion of satire and humor content.
- We updated our systems to limit the use of user-generated content in responses that could offer misleading advice.
- We added triggering restrictions for queries where AI Overviews were not proving to be as helpful.
- For topics like news and health, we already have strong guardrails in place. For example, we aim to not show AI Overviews for hard news topics, where freshness and factuality are important. In the case of health, we launched additional triggering refinements to enhance our quality protections.
Last but not least, Liz Reid ended with:
- “We’ll keep improving when and how we show AI Overviews and strengthening our protections, including for edge cases, and we’re very grateful for the ongoing feedback,”
Are AI Overviews here to stay?
AI Overviews isn’t going anywhere, and the global rollout will follow as planned. Google emphasized that we can expect search results to improve as it continues strengthening its system based on people’s search queries.
When he is not writing content, Terry can be found on his boat in Italy or chilling in his villa in Spain.
- Google Goes Nuclear To Power AI Data Centers - October 15, 2024
- SEO Weekly News Roundup [October 7 to 11, 2024] - October 14, 2024
- Google Local Service Ads Update On The Way - October 11, 2024
