anyone who can give me block everything except google bot robots.txt file ?

Hi, anyone who can give me block everything except google bot robots.txt file ?

Example

block
-ahrefs
-semrush

etc

Yikes. While possible this is a bad idea. Robots.txt is ignored by bad bots so you will end up with only bad bots. The good bots that might be helpful, like those from other search engines, will obey and ignore your site.

ANSWER: To do it you simply dissalow all and ad a single exception for googlebot. Just Google "robots.txt" and Google has a developer guide to show you how you could do that.

A BETTER WAY: Use your htaccess to control bots. Robots.txt is like asking bots to obey, with htaccess you're making it impossible not to obey.

jakeriley said:

Yikes. While possible this is a bad idea. Robots.txt is ignored by bad bots so you will end up with only bad bots. The good bots that might be helpful, like those from other search engines, will obey and ignore your site.

ANSWER: To do it you simply dissalow all and ad a single exception for googlebot. Just Google "robots.txt" and Google has a developer guide to show you how you could do that.

A BETTER WAY: Use your htaccess to control bots. Robots.txt is like asking bots to obey, with htaccess you're making it impossible not to obey.

Click to expand...

Yes I know that is bad idea. however can you give me that ?

Anis1997 said:

Yikes. While possible this is a bad idea. Robots.txt is ignored by bad bots so you will end up with only bad bots. The good bots that might be helpful, like those from other search engines, will obey and ignore your site.

ANSWER: To do it you simply dissalow all and ad a single exception for googlebot. Just Google "robots.txt" and Google has a developer guide to show you how you could do that.

A BETTER WAY: Use your htaccess to control bots. Robots.txt is like asking bots to obey, with htaccess you're making it impossible not to obey.

Click to expand...

for real,.bro? Did you just copy-paste that guy's reply, ad literam, just so you can inflate your post count? Are you guys this low on the evolutionary scale?

Disallowing search engine bots with robots.txt can potentially affect your ranking if you disallow the wrong bots or pages. When search engine bots crawl your site, they use information from your pages to determine how to rank them. If you disallow certain bots or pages, the bots won't be able to crawl and index them, which can negatively impact your ranking. Instead, you may want to consider diversifying your SEO tactics by focusing on creating high-quality content that naturally attracts links and engages your target audience. Additionally, you can also explore other channels to drive traffic to your site.

Well, this would be the correct syntax for a robots.txt file that blocks everything but Googlebot

User-agent: Googlebot
Allow: /

User-agent: *
Disallow: /

Click to expand...

But robots.txt is ignored by pretty much any scraper. To block Ahrefs/SEMRush et al you're best off blocking at server level, by both user-agent and more preferably IP level.