Is it time to block google bots?

Mr Positive

Mr Positive

Elite Member
Executive VIP
Jr. VIP
Joined
Mar 30, 2018
Messages
12,091
Reaction score
14,356
Never thought I would ask this but AI overviews, and now AI mode in google search engine makes me wonder if we’re not going to get traffic and our content is being used as free fodder ain’t it time to block their bots? Not a advice just a discussion.
 
Advertise on BHW
The bigger question is what is the future of search and content?

If AI becomes better and eventually takes over search, I assume? What will the future model for content ingestion be? And will there be scope to profit from it directly and if so, how?
 
Mr Positive said:
Never thought I would ask this but AI overviews, and now AI mode in google search engine makes me wonder if we’re not going to get traffic and our content is being used as free fodder ain’t it time to block their bots? Not a advice just a discussion.
Click to expand...
During the early days when Google was testing the AI overview thingy, I predicted exactly this. That publishers would resolve to block these bots since their sites are source of data/content.

Google and Bing would have to choose. Right now, one of my sites ranks 1 on a sizable keyword on Bing but AI overview is above it taking all the traffic only trickling a little down to my site.
 
looks like ai overviews uses the same google-bot… block google?
 
davids355 said:
The bigger question is what is the future of search and content?

If AI becomes better and eventually takes over search, I assume? What will the future model for content ingestion be? And will there be scope to profit from it directly and if so, how?
Click to expand...

AI can produce content but without human written content it’ll fabricate a lot, OpenAI is paying publisher, and I’m hoping if AI uses your content you get paid some kind of payment has to come internet without human content is really scary cause we will then be applying glue on our pizza for sure :p

HenryObi said:
During the early days when Google was testing the AI overview thingy, I predicted exactly this. That publishers would resolve to block these bots since their sites are source of data/content.

Google and Bing would have to choose. Right now, one of my sites ranks 1 on a sizable keyword on Bing but AI overview is above it taking all the traffic only trickling a little down to my site.
Click to expand...
Dam blocking is risky but if we don’t they won’t care I feel publishers will have to make a choice soon otherwise it maybe too late
 
It's really heartbreaking to see the top filled with the AI overview. For informational keywords, most people aren't even opening the results now because they already get the answer from the AI overview. I find myself doing the same. I think building an eCommerce brand is the only thing that sounds profitable for the long term. I say this because people would need to visit the eCommerce sites to buy stuff, no matter what.
 
Human interactions and meaningful conversations are only part of online forums now.
Almost anything we search is available as a generated summary. We don't have to click links and search for knowledge anymore.

WhatsApp makes conversations real-time, and usually too much information makes it difficult to find anything in WhatsApp groups.

Online forums are the only way now that manage information as threads to have meaningful conversations with humans.1742282871115.png
 
Mr Positive said:
Dam blocking is risky but if we don’t they won’t care I feel publishers will have to make a choice soon otherwise it maybe too late
Click to expand...
Both parties have to make that choice. Punishers who have successfully diversified their traffic, particularly those with majority coming from sources that are not Google o r search can block them without fear of anything.
 
I think we need more time to come up with a conclution. But it is certain that the future of search is not only Searching in Search engine alone. We need to move to adopt more dynamic channels to meet the tends and build stronger connections with the audiences like social media, videos etc.
 
I think it’s essential to remember that everything published openly on the internet becomes accessible to everyone: both people and robots. If your content has been used as training material for LLMs, it’s not surprising. This is entirely expected in our time. Therefore, I believe that blocking Google bots is a rather questionable idea because blocking them would prevent indexing, and how would we get traffic then? Perhaps it’s better to block SEMRush and Ahrefs crawlers to prevent competitors from accessing the metrics of your commercial website.
 
Right, blocking bots might feel like taking a stand, but what we achieve with that?

Google won't care if small publishers fight back, it's their game, their rules. The real risk is that while AI overviews might eat into clicks, Google is still the biggest source of traffic. Blocking the crawlers could mean cutting off your own reach.

So the smarter move is to adapt, optimize for AI summaries, find ways to get visibility within them, and diversify traffic sources instead of relying too much on search.
 
They'll throw a carrot to the publishers to get more data. The whole AI works on user fed data. No new data, AI dies.
 
Do you realise that Google crawls your website without your consent( maybe I made that website for family and friends not for the whole world) puts it in a database and makes billions by displaying it in a subjective top 10?.....Why do we need to take extra steps to prevent money hungy whore Google to index our property.....it should be the other way around....G must ask for permission to index a private property....just think about it.
 
Mr Positive said:
Never thought I would ask this but AI overviews, and now AI mode in google search engine makes me wonder if we’re not going to get traffic and our content is being used as free fodder ain’t it time to block their bots? Not a advice just a discussion.
Click to expand...
Most of the think I search this days get answered by the google AI answer at the top , So the answer would be yes the traffic will drop significantly for websites
 
Blocking bots could be an option, but then there’s the risk of losing visibility too
 
This is why they ruined the quality of their search results.
To make it more natural to accept new things, I whipped it first and put out a carrot called ai search results.
 
The future doesn't look rosy. AI uses our data to train and then those AIs are being used to create content.
I think the Internet will be full of AI in the next years, which will cause the masses to either stop using it or stay in big social media.

This is a logical conclusion. People are already flooding the web with AI slop, which can't be identified nor regulated.

I've stopped visiting unknown sites while searching for information. I only go to Reddit, because other websites are pure clickbait. They don't answer my questions, they are just shallow word padding.

I sometimes ask myself if the web would be better, if search engines would treat websites as apps. So instead of on website flooding the search results with pages, we would have just the website listed.
 
AI Overview has killed a lot of good informational sites. Only e-commerce sites are safe as of now, but later on those sites may also be targeted. This seems to end of the blogging era.

Wait and watch to see where we are heading....
 
I think this is a question we really need to start taking seriously our content is basically free fuel for Google’s AI systems, and we’re getting little to nothing in return. For sites that are already seeing traffic drops, blocking bots is starting to look like a strategy worth testing though the long-term impact is still a big unknown.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top