Wildcard in robots.txt

Discussion in 'White Hat SEO' started by dizen, Aug 15, 2012.

  1. dizen

    dizen Regular Member

    Joined:
    Jun 3, 2009
    Messages:
    483
    Likes Received:
    37
    I have a lot of pages I want to exclude from indexing ....

    A lot of my pages has "_c" on the URLs

    for example

    /electronic/computer_c127
    /hobby_c139
    car/suv/audi_c128
    mykoolcar_c123.html

    As you can see the string "_c" can be found on many folders and sub folders and even html files

    Will this code exclude them all from indexing?

    Disallow: *_c*


    Thanks !
     
    • Thanks Thanks x 1
  2. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    112
    yes you can use wildcards.
     
    • Thanks Thanks x 1
  3. dizen

    dizen Regular Member

    Joined:
    Jun 3, 2009
    Messages:
    483
    Likes Received:
    37
    Is my code correct for my situation?remember "_c" can be found on many folders and sub folders and even html files
     
  4. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    112
    that would do it. it would also weed out /help_contact_us.html -- just be sure you really want all _c filenames/parameters weeded out.
     
  5. SEOnetic

    SEOnetic Senior Member

    Joined:
    May 21, 2012
    Messages:
    994
    Likes Received:
    120
    Hey utuxpia, can you explain it a bit more? I have been trying to get it to work but it doesnt.
     
  6. utuxia

    utuxia BANNED BANNED

    Joined:
    Feb 14, 2011
    Messages:
    673
    Likes Received:
    112
    Just put this in robots.txt

    Code:
    User-agent: *
    Disallow: *_c*