1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Robots.txt File Optimization... Is this OK.?

Discussion in 'White Hat SEO' started by K.H.R, Sep 25, 2015.

  1. K.H.R

    K.H.R Jr. VIP Jr. VIP

    Joined:
    Dec 9, 2013
    Messages:
    3,154
    Likes Received:
    1,006
    Gender:
    Male
    Location:
    BHW
    Hello Black Hatters,

    Hope all are doing well. I see, there are many Block Resources in Webmaster Tools. So I edited Robots.txt file.

    Code:
    User-agent: Googlebot
    Disallow:
    Disallow: /out/
    Disallow: /wp-admin/
    
    User-agent: *
    Allow: /wp-content/uploads/
    Disallow: /
    And test with Fetch as Google tool & everything is OK. So Google can Crawl my full site & Its Good for SEO.

    My site is under attack, some one trying to hack my site. I getting too much Login Attempt Failed & Ip Blocking Email, Sometimes they also deleted Robots.txt file. So I am really concerned about the Security Issue.

    Now the questions is:
    1. What about the security issue?
    2. Is this can be a Weak Point of my site?
    3. Can Hacker use this for hacking my site?

    This is Off Topic but Related to SEO Practice, that's why I posted it here. If this is not appropriate, MOD Please Move it to the related section.

    Thanks.
     
  2. nikiobicata

    nikiobicata Regular Member

    Joined:
    Mar 4, 2011
    Messages:
    451
    Likes Received:
    376
    Occupation:
    IT Director
    Location:
    New York
    Home Page:
  3. rajanant

    rajanant Registered Member

    Joined:
    Sep 16, 2015
    Messages:
    62
    Likes Received:
    10
    If someone deleted yours robots.txt than this someone has access to your site as owner. You should change your passwords.
    I'm not sure that hiding robots can help in this situation, because this file shouldn't be editable from outside, and must be readable by everyone from outside.
     
  4. AdludumAndre

    AdludumAndre Registered Member

    Joined:
    Mar 21, 2015
    Messages:
    51
    Likes Received:
    8
    Occupation:
    CTO & Co-Founder of Adludum.com
    Location:
    Germany
    Home Page:
    This. Make sure to change all passwords and make sure your wordpress is secure! To hide your robots.txt will not safe you from getting hacked.
     
  5. K.H.R

    K.H.R Jr. VIP Jr. VIP

    Joined:
    Dec 9, 2013
    Messages:
    3,154
    Likes Received:
    1,006
    Gender:
    Male
    Location:
    BHW
    I understand & I am using SiteLock Security.
     
  6. victorrex

    victorrex Junior Member

    Joined:
    Dec 6, 2014
    Messages:
    174
    Likes Received:
    37
    From the robots code you posted it seems you are using wordpress, As rightly pointed out above the robots. txt should be readable by everyone as it is a txt file including google and all other bots. You can use commands to block all other bots from accessing your site other than google's. However please remember that robots.txt is not really binding on all bots as they are just a set of instructions and generally a malware will surely not listen.

    http://www.blackhatworld.com/blackhat-seo/blogs/sunny_clicks/
    refer this blog and it will be quite beneficial for you.

    Thanks
    Victor