1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

eCommerce Robots.txt File

Discussion in 'White Hat SEO' started by cpdudeson, Jun 18, 2012.

  1. cpdudeson

    cpdudeson Registered Member

    Joined:
    Mar 16, 2012
    Messages:
    51
    Likes Received:
    11
    I have put together a robots.txt file for eCommerce. This file has been specifically designed for Magento, but the principles remain the same across eCommerce sites. Before you just upload this to your site make sure you do you research.

    User-agent: *
    Disallow: /admin/
    Disallow: /app/
    Disallow: /catalog/category/view/
    Disallow: /catalog/product/view/
    Disallow: /catalogsearch/
    Disallow: /checkout/
    Disallow: /customer/
    Disallow: /downloader/
    Disallow: /errors/
    Disallow: /lib/
    Disallow: /review/
    Disallow: /sendfriend/
    Disallow: /skin/
    Disallow: /report/
    Disallow: /tag/
    Disallow: /var/
    Disallow: /wishlist/




    Disallow: /*.js$
    Disallow: /*.css$
    Disallow: /*.php$
    Disallow: /*?p=*&
    Disallow: /*?SID=
    Disallow: /pkginfo/