Forum Topics Interesting take on Google search
Solvetheriddle
Added 5 months ago

This comes from Substack MBI--which is about to go behind a paywall :(, but I found this very interesting, obviously a dynamic situation, but GOOG far from dead, maybe i liked it because of confirmation bias (guilty as charged), lol


“Search Engine Land” published some interesting data about the evolution of Google Search post-AI Mode launch. Some key excerpts from the piece:

The biggest shift has been from 1-2 word keywords to 3-4 word search terms.
Short queries dropped from 42% in January to just 31% in June – suggesting users are increasingly communicating in more natural language.
Shorter keywords have a CTR drop of 50%. Keywords with more than eight words are down by 26%.
Shorter keywords accounted for 76% of conversions in January, but dropped to 50% by June.
Three- and four-word queries now make up 40% of conversions, up from 20% in January.

Clearly, search behavior is changing, but instead of clinging onto to the old search model, Google seems to be evolving just as fast. In fact, Eric Seufert recently quoted a survey by Oppenheimer to point out an intriguing data point:

…of users familiar with AI Mode and who pay for ChatGPT, 82% find AI Mode more helpful than Google Search and 75% find AI Mode more helpful than ChatGPT.

Admittedly, this surprised me because of two things: a) these ChatGPT users in this survey are paying subscribers (my initial reaction was they must be comparing free version of ChatGPT vs Google’s AI mode), and b) I personally would respond differently in this survey. Having said that, sample size in this survey was only 263, so not sure how representative this is for more broader population. In any case, I do think the real fight between ChatGPT vs Google is on the free users. If Google AI mode is deemed decisively better than ChatGPT’s free version, users will have no compelling reason to shift their decade long search habits, and Google search can survive and thrive just fine.

Speaking of Google thriving, Ben Thompson wrote a piece yesterday that has profound positive implications for Google. Thompson highlighted how Cloudflare is trying to reshape the open web by forcing AI crawlers pay for content. This is potentially a bad news for model developers’ margins unless you are Google. From Ben Thompson’s piece yesterday:

Google, which has two crawlers. Googlebot crawls the web for Google search, while Google-Extended crawls the web to capture data for Gemini. What is critical to understand, however, is that data for Google Search AI products — including AI Overviews and AI Mode (i.e. the search funnel) — is gathered by Googlebot; that means that if you want your website to show up in Google Search you have no choice but to have that data also be used by any AI products that are under the Search umbrella.
Just to be clear, what Cloudflare is doing is not simply amending robots.txt; rather, they are straight-up denying access to AI crawlers — again, except for Googlebot, which a Google executive confirmed in court captured data for use in Search AI products
In short, Cloudflare is actively helping Google’s competitive position, insomuch as Google’s most important AI product is in fact Search

I am not super confident that Google can maintain these terms as I imagine Google’s SOTA model competitors will highlight these terms to the regulators and force a level playing field. If such attempts to level playing the field are not successful, this would be a decisive competitive advantage for Google. Of course, if the chatbots need to pay for content, they may try to be much more careful in crawling the number of pages before answering a query or running deep research which can hurt answer quality.

Frankly speaking, these terms do seem unfair. I can digest the argument “AI overviews” as part of the core traditional search experience, but including “AI mode” in “Googlebot” terms feels a bridge too far. AI Mode has a chat like interface too which makes it functionally no different than something like ChatGPT. If ChatGPT needs to pay for content, so should “AI Mode” in Google.


18

Chagsy
Added 5 months ago

It’s difficult to see how the web is going to look in the coming years. The basic premise of how it works is now broken: monetise visitors to your website. If search increasingly is done through AI, and the Cloudflare move isn’t universal, then there will be a lot of losers.

Large companies can probably afford to demand that AI crawlers pay or don’t have access. Smaller ones, less. Likewise for information sites. Already this year traffic to information sites has reduced by 15-25%.

Does this fundamentally challenge the thesis for many companies? Quite possibly but I’m not smart enough to work out how or which ones.

18

Chagsy
Added 5 months ago

AI is killing the web. Can anything save it? 

https://economist.com/business/2025/07/14/ai-is-killing-the-web-can-anything-save-it

from The Economist

8

Solvetheriddle
Added 5 months ago

@Chagsy , I think you are right; chatbots are freeloaders, and their behaviour will reduce the useful sources they rely upon. As you say, maybe there is a quid pro quo; I'm like you, not sure how it works out. However, it is worth following, in my opinion, because I want to have an idea of who the winners will be here; the prize could be big.

8