website help.

  1. P

    Robots.txt help

    Hi I need help with this, currently my Robots.txt file is setup in this order as i dont want my website to be crawled in other directories other than the important ones? User-agent: * Disallow: Disallow: /cgi-bin/ Disallow: /Admin/ Disallow: /images/ Disallow: /Account/ Disallow: /bin/...
Back
Top