1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

My site's robots.txt contains one extra Disallow: /forums/-/

Discussion in 'Black Hat SEO' started by Xfactormaster, Dec 15, 2015.

  1. Xfactormaster

    Xfactormaster Registered Member

    Joined:
    Mar 10, 2013
    Messages:
    50
    Likes Received:
    0
    My site's robots.txt contains one extra Disallow: /forums/-/
    Remaining Disallows are same as xenforo forum.
    I want to know what is this meaning. Does it hurt google indexing and ranking?

    Thank you
     
  2. Nonilol

    Nonilol Elite Member

    Joined:
    Mar 1, 2015
    Messages:
    1,645
    Likes Received:
    559
    Gender:
    Male
    Occupation:
    Media Informatics & Webdev Student
    Location:
    Hamburg, GER / Southampton, UK
    Home Page:
    I am not 100% sure what this /-/ is doing there. Or do you have a subdirectory called "-" within "forums"?

    If the latter, this line in your robots.txt basically means that bots are not allowed to visit everything that is within the "-" directory.
    You should check that folders content via ftp. If there is content you want to be indexed, remove this line from your robots.txt, otherwise keep it.
     
  3. Jumbuck1976

    Jumbuck1976 Regular Member

    Joined:
    Jun 13, 2015
    Messages:
    205
    Likes Received:
    37
    Occupation:
    Social media writer for a marketing company in Dub
    Location:
    Naas, Co Kildare, Ireland
    Do you actually want Google to crawl the forum directory? If you remove the /-/ you'll be moving the problem onto the next directory up the tree.
     
  4. Xfactormaster

    Xfactormaster Registered Member

    Joined:
    Mar 10, 2013
    Messages:
    50
    Likes Received:
    0
    Thanks for the information, i removed it, let me see what happens