Robots.txt problem...

[Deleted member 834202]

Hi,

I downloaded a plugin "Virtual robots.txt" for Wordpress. The plugin change my robots.txt but now I can't edit it. So I remove it. But my robot is always the same. I tried to download some others plugins but it override everything. When I upload a physical robot.txt on my root it override too... I'm on the problem since 5 hours, I'm tired. If my problem is resolve, I don't download plugins anymore... Everytime I've got problems.

Someone can help me please ?

# This virtual robots.txt file was created by the Virtual Robots.txt WordPress plugin: https://www.wordpress.org/plugins/pc-robotstxt/
User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /wp-includes/
Allow: /wp-includes/js/
Allow: /wp-includes/images/
Disallow: /trackback/
Disallow: /wp-login.php
Disallow: /wp-register.php

 

[vistatech]

Do you have any other plugin that might be caching the robots.txt file?

 

[Deleted member 834202]

Do you have any other plugin that might be caching the robots.txt file?

I don't think... I disabled all my plugins for a test and nothing change.

 

[vistatech]

I don't think... I disabled all my plugins for a test and nothing change.

Where is the website hosted?

 

[Deleted member 834202]

Where is the website hosted?

Easyname.com, it's an issue with my host ? I use Cloudflare too.

 

[nahid133]

Try to edit your robot.txt file. You can do it through Yoast seo plugins. Google for more help.

 

[vistatech]

Easyname.com, it's an issue with my host ? I use Cloudflare too.

You'd try contacting your host and tell them about the problem. Maybe they're automatically caching your content. Or, as @nahid133 suggested try using Yoast's SEO Plugin, if you have it installed to edit the file before you contact your host.

 

[Deleted member 834202]

Try to edit your robot.txt file. You can do it through Yoast seo plugins. Google for more help.

I tried... A lot... There's no robots.txt from "Virtual robots.txt" plugin...

 

[Deleted member 834202]

You'd try contacting your host and tell them about the problem. Maybe they're automatically caching your content. Or, as @nahid133 suggested try using Yoast's SEO Plugin, if you have it installed to edit the file before you contact your host.

I had just created my robots.txt. Do you think it's possible to caching instantly ?

 

[nahid133]

Forget "Virtual robots.txt" plugin... as u already uninstalled it . try to find your robot.txt file

 

[Deleted member 834202]

Guys thank you... That was cache... I'm going on Cloudflare, I make a purge and my robots.txt have changed !!! Thank you very much. I was desperate.

 

[vistatech]

Guys thank you... That was cache... I'm going on Cloudflare, I make a purge and my robots.txt have changed !!! Thank you very much. I was desperate.

Thought so.

 

[AstTrailerParkSupervisor]

Just curious, wouldn't it be easier to just manually edit the robots.txt and upload it without using a plugin to do so?

 

[Deleted member 834202]

Just curious, wouldn't it be easier to just manually edit the robots.txt and upload it without using a plugin to do so?

Yes totally easier. But I couldn't remember that was so easy.... It's been a long time since I've made a website.

 

[AstTrailerParkSupervisor]

Yes totally easier. But I couldn't remember that was so easy.... It's been a long time since I've made a website.

Ok cool, I'm no genius when it comes to webmastery either, just wanted to see if I was missing out on something lol

 

[umerjutt00]

Guys thank you... That was cache... I'm going on Cloudflare, I make a purge and my robots.txt have changed !!! Thank you very much. I was desperate.

Exactly. Next time when you make a change in your robot.txt or even css file, just remember to purge the cache in CloudFlare. Once I made some changes to my CSS file and it was still showing the old one. Took me few hours to realize that it was a cache problem at CF.

 

[Deleted member 834202]

Ok cool, I'm no genius when it comes to webmastery either, just wanted to see if I was missing out on something lol

Haha no, you were right! But when you start to download a plugin, you think you need an other plugin for everything lol

Exactly. Next time when you make a change in your robot.txt or even css file, just remember to purge the cache in CloudFlare. Once I made some changes to my CSS file and it was still showing the old one. Took me few hours to realize that it was a cache problem at CF.

Next time it will be the first thing I will think... I searched everywhere and I was so tired. That was exhausting! But it's ok, all my mistakes learn me to search and ask for help. I'm not familiar with that.