Should I edit robots.txt?

[Rollfic]

So there is no robots.txt in my file manager. Searched it more than one time lol.

This is what my robots.txt look like from the URL -

Should I remove the Disallow and Allow and add Crawl-Delay of 5 or something?

 

[hideath]

you don't want crawlers to actually check this folder,
you can add rules to disallow tags and categories if you want,
apart from that, nothing to worry about

 

[Rollfic]

you don't want crawlers to actually check this folder,
you can add rules to disallow tags and categories if you want,
apart from that, nothing to worry about

Still wondering why there is no robots.txt file in my file manager? Do web hosters hide that nowadays?

 

[fb3003]

Still wondering why there is no robots.txt file in my file manager? Do web hosters hide that nowadays?

not that I know of.

please see this
https://yoa.st/robots-txt

yoast point of view on best practice on how your robots.txt should look like

 

[Mymusic]

If you want to edit your robot.txt file you can install the rank math plugin and there you will find the option to edit it within the plugin options