1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

robots.txt not showing in cpanal ?? pls help

Discussion in 'Blogging' started by shaun_shaun, Oct 7, 2011.

  1. shaun_shaun

    shaun_shaun Junior Member

    Joined:
    Mar 22, 2011
    Messages:
    157
    Likes Received:
    82
    hi, I just added my wordpress site in webmasters tools.but my robots.txt is blocking search engine robots.

    I already enabled bot access in wordpress settings .. the main issue is I cant see my robots.txt in cpanal or Filezilla..

    but after I visit, www.mydomain.com/robots.txt , its showing as

    User-agent:
    * Disallow:

    how can I delete current robots.txt.. it is not visible but only can access.. pls help????
     
    Last edited: Oct 7, 2011
  2. ronywilliam

    ronywilliam Senior Member

    Joined:
    Jan 20, 2011
    Messages:
    1,150
    Likes Received:
    431
    When opening files with file manager select "show hidden files"

    Edit: NVM thought its .htaccess .. Not sure about robots.txt
     
    Last edited: Oct 7, 2011
  3. wannabie

    wannabie Elite Member

    Joined:
    Mar 11, 2009
    Messages:
    3,807
    Likes Received:
    2,954
    Occupation:
    Seo and Marketing Suprisingly
    Location:
    Your bedroom window
    Home Page:
    It will be in your public_html folder

    Its a simple .txt file that tells search engines what to do, why would you want to delete it?
     
  4. shaun_shaun

    shaun_shaun Junior Member

    Joined:
    Mar 22, 2011
    Messages:
    157
    Likes Received:
    82
    it is currently blocking search engines
    User-agent: * Disallow:


    I cant edit it because it wont show in cpanal .. but I can access it.
     
  5. shaun_shaun

    shaun_shaun Junior Member

    Joined:
    Mar 22, 2011
    Messages:
    157
    Likes Received:
    82
    I enabled hidden files..but still it wont show :( ??? but I can access it
     
  6. liviug

    liviug Junior Member

    Joined:
    Feb 4, 2011
    Messages:
    172
    Likes Received:
    79
    Just install irobots plugin in WP.
    It will overwrite the normal robots, will also block g from seeing your dirs
     
  7. shaun_shaun

    shaun_shaun Junior Member

    Joined:
    Mar 22, 2011
    Messages:
    157
    Likes Received:
    82
    the problem is wordpress using a stupid F***** virtual robots.txt . This issue started after I set my new site to private .. now I created a static robots.txt and allowed all the bots.. but still google bot wont fetch my site..

    I looked for an option for this in WP forums found few useful tips.
    Code:
    [URL]http://wordpress.org/support/topic/robotstxt-set-to-disallow-cant-change/page/2[/URL]
    as one of their member said, I ve to wait for few more days till google recrawl my site again...

    this is soo annoying !!!


    ------------------------------------------------------------------------------------------------

    P.S = after nearly 12 hrs google finally fetched my site again...

    anyway this is a good lesson to all of us.because lots of people dont know about this wordpress virtual robot.txt. If u want to make ur blog private, just create a static robot.txt and disallow the robots. DONT MAKE UR BLOG PRIVATE with Wordpress privacy settings .
     
    Last edited: Oct 8, 2011
  8. extremephp

    extremephp BANNED BANNED

    Joined:
    Oct 19, 2010
    Messages:
    1,293
    Likes Received:
    1,272
    Go to your webmasters and set the crawl rate to maximum. That will make it quick and will get your website indexed quick. And maximum crawl rate always helps getting your new content indexed. but if your site goes down, make sure you set the rate to minimum or you are fucked. :rolleyes: