1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to write my Robot.txt

Discussion in 'White Hat SEO' started by yufei1990, Jul 23, 2016.

Tags:
  1. yufei1990

    yufei1990 Newbie

    Joined:
    Jul 23, 2016
    Messages:
    13
    Likes Received:
    1
    Gender:
    Female
    Somebody tell me how to write my robots.txt file?

    I saw google had indexed those kind urls:

    ?cat=34&dir=asc&limit=20&mode=list
    ?cat=51&dir=asc&limit=15&mode=list
    ?cat=54&dir=desc&limit=15&mode=list
    ?cat=73&dir=desc&limit=20&mode=list
    ?cat=64&dir=desc&limit=25&mode=grid

    This is sub catagory, the full url is like this kind: http://www.xxx.com/ddd?cat=64&dir=desc&limit=25&mode=grid

    How to write the robots file to forbid google to index such urls?
     
  2. coltreign

    coltreign Junior Member

    Joined:
    May 13, 2016
    Messages:
    148
    Likes Received:
    29
    Occupation:
    I Do AdWords
    Location:
    The Internet
    You can do it directly through search console. Do you have search console set up through Google?

    If you want to do it quick and dirty, you can do it through robots.txt, but it would be best to do it via meta tagging. Know what I'm saying?
     
  3. yufei1990

    yufei1990 Newbie

    Joined:
    Jul 23, 2016
    Messages:
    13
    Likes Received:
    1
    Gender:
    Female
    I just fix this in google webmaster.
     
    • Thanks Thanks x 1
  4. coltreign

    coltreign Junior Member

    Joined:
    May 13, 2016
    Messages:
    148
    Likes Received:
    29
    Occupation:
    I Do AdWords
    Location:
    The Internet
    Cool! Be sure to give me a thumbs up. ;)
     
    • Thanks Thanks x 1
  5. skylang

    skylang Newbie

    Joined:
    Jul 2, 2015
    Messages:
    35
    Likes Received:
    1
    Gender:
    Male
    Occupation:
    Business Man
    Location:
    Japan
    Home Page:
    Check your competitor robot txt file. it will help you to write robot txt file.