A recent study by Dragon Metrics has discovered that Google’s Featured Snippets show opposing information taken from the same source depending on how you phrase your questions.
The study’s findings could be helpful to SEO professionals and content creators, as it shows Google considers user intent a priority when displaying search results.
The study
Director at Dragon Metrics, Sarah Presch, began researching Google’s Featured Snippets a few years ago, testing whether it gave contradictory information from the same content source when framing a question differently.
Presch told the BBC she soon noticed a problem:
- “I started looking at how Google handles topics where there’s heated debate,” she says. “In a lot of cases, the results were shocking.”
Sarah said her findings also contradict what Google says its mission is regarding sourcing and providing the most useful information relative to the user’s intent, AKA the Helpful Content Update, which devastated many websites’ rankings and traffic since its launch in September 2023:
- “Google’s whole mission is to give people the information that they want, but sometimes the information that people think they want isn’t actually the most useful.”
Google’s answer depends on your question
Presch provided examples of how Google’s Featured Snippet provides conflicting answers depending on how users phrase their questions.
She found some of the most opposing results when asking about health, politics, and whether you phrased your query positively or negatively.
For instance, when Sarah searched using “link between coffee and hypertension,” Featured Snippets highlighted an article from the Mayo Clinic, saying:
- “Caffeine may cause a short but dramatic increase in your blood pressure.”
When Persch used the search term “no link between coffee and hypertension,” the Featured Snippet provided a contradictory result from the same article:
- “Caffeine doesn’t have a long-term effect on blood pressure and is not linked with a higher risk of high blood pressure.”
Presch found similar findings when asking if a politician is “good” versus “bad” and asking if a country’s tax system is “fair” versus “unfair,” with all returning opposing results despite the question being the same.
Sarah told the BBC during the interview that:
- “What Google has done is they’ve pulled bits out of the text based on what people are searching for and fed them what they want to read,” It’s one big bias machine.”
Google is bias
Presch’s statement, “It’s one big bias machine,” refers to her findings that show Google’s Featured Snippets prioritize content matching the user’s intent (how they ask the same question) instead of providing proven and comprehensive information.
Google denies this, saying it provides unbiased results that match users with the information they are searching for:
- “As a search engine, Google aims to surface high-quality results that are relevant to the query you entered. We provide open access to a range of viewpoints from across the web, and we give people helpful tools to evaluate the information and sources they find.”
However, a leaked internal Google document where an engineer said, “We do not understand documents—we fake it,” contradicts that statement.
The engineer wrote in his slideshow in 2016:
- “A billion times a day, people ask us to find documents relevant to a query… Beyond some basic stuff, we hardly look at documents. We look at people. If a document gets a positive reaction, we figure it is good. If the reaction is negative, it is probably bad. Grossly simplified, this is the source of Google’s magic.”
- “That is how we serve the next person, keep the induction rolling, and sustain the illusion that we understand.”
Google says the information is outdated
Google said the leaked internal document is outdated, and its systems for deciphering users’ queries to match them with the relevant content pages are now more sophisticated.
However, some SEO experts believe the Search limitations of 2016 persist.
Professor of information systems at the University of South Florida Varol Kayhan said about Presch’s findings:
- “We’re at the mercy of Google when it comes to what information we’re able to find.”
Founder of AlsoAsked Mark Williams-Cook said:
- “Google builds models to try and predict what people like, but the problem is this creates a kind of feedback loop. If confirmation bias pushes people to click on links that reinforce their beliefs, it teaches Google to show people links that lead to confirmation bias.”
What it might mean to content and SEO
Google says it provides unbiased results that match people with the content they’re looking for.
However, Sarah Presch’s findings suggest that as Google moves closer to using AI-generated responses determined by how people phrase their queries, publishers should create content that remains accurate regardless of the search term.
