Ideas for scraping

[banel]

So, how you guys scrape those big list of blogs? I always read here on BHW things like, "just scrape 10 000, 50 000 blogs and blast them all".
I can't scrape more than 2k, 3k , 5k because, after the wordpress footprint I don't know what popular keywords to add.
How do you solve this problem? Where do you get this kind of keywords?

 

[banel]

No one ?

 

[lrobbins]

I sometimes after all my logical keywords related to my site is finished. I just put random words or items, whatever pops in my head and scrape it.

It makes me scrape areas I wouldn't usually but a backlink is a backlink

 

[JamesHenry]

when you have the list you retrieved do you take it to the root domain then find all the internal pages to post to.

If I find 2000 urls I take them to the root, usually come up with 80k to 120k url list I have put a link on.

 

[fairlyliquid]

when you have the list you retrieved do you take it to the root domain then find all the internal pages to post to.

If I find 2000 urls I take them to the root, usually come up with 80k to 120k url list I have put a link on.

ive been avoiding doing this as a while back i posted one comment to a wordpress site that had a widget on every page that displayed 'recent comments'. within a few days google webmaster tools was showing 2000+ backlinks from that site and it whacked my search position on that site from page 1 to down to 5 at the best but dancing all over the place. Now even though if i visit that site it no longer shows my comment in the recent comments GWT is still showing over 2000 backlinks from there, im hoping this will fall as google recrawls the pages.

 

[Mokodoki]

So, how you guys scrape those big list of blogs? I always read here on BHW things like, "just scrape 10 000, 50 000 blogs and blast them all".
I can't scrape more than 2k, 3k , 5k because, after the wordpress footprint I don't know what popular keywords to add.
How do you solve this problem? Where do you get this kind of keywords?

People who post in those numbers generally aren't going after targeted blogs. They are basically either buying lists or creating them using a massive list of random keywords. There are many different ways to amass these untargeted lists and everyone has a different strategy.

It's important that you get the most out of your keyword list. There's a lot of ways to get more results from the same keywords. One of my favorite loopline videos can get you started thinking on how to accomplish this:

http://www.youtube.com/watch?v=hKC69k_cRsc&feature=channel_video_title

Think outside the box when it comes to finding new keyword lists. You can use dictionary files to start out with and scrape more keywords using the keyword harvester. You could also use a program like "WordList Expert" to strip text down into individual words. I used to do this with all sorts of things from books to RSS feeds.

I personally use the paid version of Crazyflx's Google Suggest scraper to make most of my keyword lists.