1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Wildcard in robots.txt

Discussion in 'White Hat SEO' started by dizen, Aug 15, 2012.

  1. dizen

    dizen Regular Member

    Joined:
    Jun 3, 2009
    Messages:
    483
    Likes Received:
    35
    I have a lot of pages I want to exclude from indexing ....

    A lot of my pages has "_c" on the URLs

    for example

    /electronic/computer_c127
    /hobby_c139
    car/suv/audi_c128
    mykoolcar_c123.html

    As you can see the string "_c" can be found on many folders and sub folders and even html files

    Will this code exclude them all from indexing?

    Disallow: *_c*


    Thanks !
     
    • Thanks Thanks x 1
  2. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    111
    yes you can use wildcards.
     
    • Thanks Thanks x 1
  3. dizen

    dizen Regular Member

    Joined:
    Jun 3, 2009
    Messages:
    483
    Likes Received:
    35
    Is my code correct for my situation?remember "_c" can be found on many folders and sub folders and even html files
     
  4. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    111
    that would do it. it would also weed out /help_contact_us.html -- just be sure you really want all _c filenames/parameters weeded out.
     
  5. SEOnetic

    SEOnetic Senior Member

    Joined:
    May 21, 2012
    Messages:
    995
    Likes Received:
    120
    Hey utuxpia, can you explain it a bit more? I have been trying to get it to work but it doesnt.
     
  6. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    111
    Just put this in robots.txt

    Code:
    User-agent: *
    Disallow: *_c*