1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Creating Web2.0? Make sure to check robots.tx before posting

Discussion in 'Black Hat SEO' started by xsuni, Jan 29, 2013.

  1. xsuni

    xsuni Junior Member

    Joined:
    Jan 17, 2013
    Messages:
    132
    Likes Received:
    42
    This is my first thread anyway and I thought to inform you a small tip. Before creating any Web2.0, make sure you check the robots.txt. There are many sites that disable Google crawling to avoid spam. For example - Jimdo and Onsugar. They allow Google crawling only when you upgrade your account to pro.

    Just thought to let you know to save your 5 mints (in case you are doing it manually). :)
     
    • Thanks Thanks x 4
  2. 9to5destroyer

    9to5destroyer Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 14, 2011
    Messages:
    355
    Likes Received:
    205
    if you don't spam and build a proper account they let google index it. best to drop the link in after
     
  3. 312k1t

    312k1t Registered Member

    Joined:
    Oct 26, 2012
    Messages:
    56
    Likes Received:
    5
    I have the same experience with livejournal.
     
  4. xsuni

    xsuni Junior Member

    Joined:
    Jan 17, 2013
    Messages:
    132
    Likes Received:
    42
    Try it, you will come to know :p. Even though you don't spam and use 100% unque and good content, Jimdo won't eve make it crawlable unless you upgrade your account.
     
  5. Smart SEO

    Smart SEO Senior Member

    Joined:
    Apr 17, 2011
    Messages:
    857
    Likes Received:
    759
    Location:
    Google HQ
    Thanks xsuni, as a new member i see you developing into a great member! I was wondering why some of my web 2.0s are almost impossible to index. Yes livejoural being one of them, i think they activate google bots when you account is activated via sms or phone
     
    • Thanks Thanks x 1
  6. xsuni

    xsuni Junior Member

    Joined:
    Jan 17, 2013
    Messages:
    132
    Likes Received:
    42
    Thanks abdullah1994uk too for your words. Actually, I joined here lately..mistake :-( Anyway, finally joined. Link building is getting difficult day by day :-( Even many platform closed registration.
     
  7. Hogare

    Hogare Registered Member

    Joined:
    Apr 17, 2011
    Messages:
    77
    Likes Received:
    93
    Holy shit! Those motherfuckers!Thanks for the tip dude,you've saved me from a huge headache.I've actually got three jimdo blogs indexed out of 13.I wonder why?
     
    • Thanks Thanks x 1
  8. xsuni

    xsuni Junior Member

    Joined:
    Jan 17, 2013
    Messages:
    132
    Likes Received:
    42
    Welcome and glad to help you. Sometime jimdo unhold this if they see some activity on the site. Now, they are not clear how much activity they want and what they need actually. I don't have time to show activity on Tier1 which I am building just to support my money site. So, not using them anymore.
     
  9. 9to5destroyer

    9to5destroyer Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 14, 2011
    Messages:
    355
    Likes Received:
    205
    if you update a couple of times without outbound links they will index it on jimdo. now if you building crappy one page first tier avoid these sites.
     
    • Thanks Thanks x 1
  10. xsuni

    xsuni Junior Member

    Joined:
    Jan 17, 2013
    Messages:
    132
    Likes Received:
    42
    Well, it was not crappy actually. I experienced that before though, not now. I used unique and good content with links. Just to note - I am not pressuring someone to stop doing site on Jimdo. You can do whatever you like :) I just thought to let other member know so that they can save some time if they are doing it manually. Even after indexing, it won't take much time for them to change the robots.txt again. (once you insert your links)
     
  11. livecrunch

    livecrunch Registered Member

    Joined:
    Oct 1, 2009
    Messages:
    74
    Likes Received:
    26
    htaccess is another example to look into and edit as well. That is if you have access to it.
     
  12. mikpel

    mikpel Regular Member

    Joined:
    Dec 1, 2012
    Messages:
    219
    Likes Received:
    115
    Now you tell me! I was wondering what the heck is wrong with jimbo as all my other web2.0 properties indexed quite nicely. I thought that I just have to blast it with more bookmarks, lol. :D


    Thanks for the tip, I'll try and see if can save my jimbocrap. I afraid they will block indexing again if they find any links though.


    EDIT: I removed my external links on jimbo page and robots.txt was cleared instantly. After that I put only one link into my article. Robots.txt is still clear, lets see if they find out... :)
     
    Last edited: Jan 29, 2013
  13. Maja92

    Maja92 Regular Member

    Joined:
    Jul 10, 2011
    Messages:
    319
    Likes Received:
    66
    Yeah I was aware... Jimdo has become sneaky :D
     
  14. Takoona

    Takoona Junior Member

    Joined:
    Aug 11, 2015
    Messages:
    147
    Likes Received:
    35
    Occupation:
    Judo-ing peoples lips off
    Yeah, thanks for this. I only wish I would of read this thread before I uploaded my nice, unique article to livejournal. Deleting my account right now.