The "Infinite Crawl Space" Concept - Why You Should Block Internal Search Results

Starblazer

Elite Member
Jr. VIP
Joined
Feb 28, 2019
Messages
7,445
Reaction score
7,898
John Mueller from Google said in the latest Search Off the Record podcast that it is no longer in the Google Search Essentials to block internal search results pages. He added that you should still block them from indexing because of intense server load and quality issues if abused, which can cause Google search ranking issues as a result.

In this episode, he discussed the "Infinite Crawl Space" concept - how Googlebot treats internal search functions as infinite crawl spaces that can generate an endless loop of new URLs. He explained that hackers could search for irrelevant terms like adult, casino, pharma keywords and your website generates pages for them which will be indexed by Google. It might result in your site getting flagged as hacked in Google Search Console.

He advised webmasters to block internal search results from indexing as a precaution despite it not being a mandatory guideline from Google.
 
Back
Top