Creating Web2.0? Make sure to check robots.tx before posting

xsuni

xsuni

Junior Member
Joined
Jan 17, 2013
Messages
174
Reaction score
62
This is my first thread anyway and I thought to inform you a small tip. Before creating any Web2.0, make sure you check the robots.txt. There are many sites that disable Google crawling to avoid spam. For example - Jimdo and Onsugar. They allow Google crawling only when you upgrade your account to pro.

Just thought to let you know to save your 5 mints (in case you are doing it manually). :)
 
Advertise on BHW
if you don't spam and build a proper account they let google index it. best to drop the link in after
 
9to5destroyer said:
if you don't spam and build a proper account they let google index it. best to drop the link in after
Click to expand...

Try it, you will come to know :p. Even though you don't spam and use 100% unque and good content, Jimdo won't eve make it crawlable unless you upgrade your account.
 
Thanks xsuni, as a new member i see you developing into a great member! I was wondering why some of my web 2.0s are almost impossible to index. Yes livejoural being one of them, i think they activate google bots when you account is activated via sms or phone
 
Thanks abdullah1994uk too for your words. Actually, I joined here lately..mistake :-( Anyway, finally joined. Link building is getting difficult day by day :-( Even many platform closed registration.
 
Holy shit! Those motherfuckers!Thanks for the tip dude,you've saved me from a huge headache.I've actually got three jimdo blogs indexed out of 13.I wonder why?
 
Welcome and glad to help you. Sometime jimdo unhold this if they see some activity on the site. Now, they are not clear how much activity they want and what they need actually. I don't have time to show activity on Tier1 which I am building just to support my money site. So, not using them anymore.
 
xsuni said:
Welcome and glad to help you. Sometime jimdo unhold this if they see some activity on the site. Now, they are not clear how much activity they want and what they need actually. I don't have time to show activity on Tier1 which I am building just to support my money site. So, not using them anymore.
Click to expand...

if you update a couple of times without outbound links they will index it on jimdo. now if you building crappy one page first tier avoid these sites.
 
Well, it was not crappy actually. I experienced that before though, not now. I used unique and good content with links. Just to note - I am not pressuring someone to stop doing site on Jimdo. You can do whatever you like :-) I just thought to let other member know so that they can save some time if they are doing it manually. Even after indexing, it won't take much time for them to change the robots.txt again. (once you insert your links)
 
htaccess is another example to look into and edit as well. That is if you have access to it.
 
xsuni said:
This is my first thread anyway and I thought to inform you a small tip. Before creating any Web2.0, make sure you check the robots.txt. There are many sites that disable Google crawling to avoid spam. For example - Jimdo and Onsugar. They allow Google crawling only when you upgrade your account to pro.

Just thought to let you know to save your 5 mints (in case you are doing it manually). :)
Click to expand...

Now you tell me! I was wondering what the heck is wrong with jimbo as all my other web2.0 properties indexed quite nicely. I thought that I just have to blast it with more bookmarks, lol. :D


9to5destroyer said:
if you update a couple of times without outbound links they will index it on jimdo. now if you building crappy one page first tier avoid these sites.
Click to expand...
Thanks for the tip, I'll try and see if can save my jimbocrap. I afraid they will block indexing again if they find any links though.


EDIT: I removed my external links on jimbo page and robots.txt was cleared instantly. After that I put only one link into my article. Robots.txt is still clear, lets see if they find out... :)
 
Last edited:
Yeah, thanks for this. I only wish I would of read this thread before I uploaded my nice, unique article to livejournal. Deleting my account right now.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top