Hey guys, I have formed a list of 500 footprints for the platforms I desire, but I'm unsure how I should go off filtering them to keep only the best footprints. I can find out how many results each footprint produces in google, but results may not be a factor when it comes to scraped URL quality. Also, I have 1M+ keywords, how would you guys suggest I pick keywords out and use them along with the footprints? Relevance is not necessary. Thanks.
Manual checking of footprints is always important i find before loading them into scrapebox. Also, 1M scraping keywords? I bet they all look like this.. cheap airline tickets to california from chicago cheap airline tickets to california from florida cheap airline tickets to california from hawaii cheap airline tickets to california from houston cheap airline tickets to california from minnesota cheap airline tickets to california from new york cheap airline tickets to california from ohio cheap airline tickets to california from texas cheap airline tickets to california san diego cheap airline tickets to europe cheap airline tickets to europe for students cheap airline tickets to europe from canada cheap airline tickets to europe tips cheap airline tickets to florida cheap airline tickets to florida from boston cheap airline tickets to florida from california cheap airline tickets to florida from cleveland cheap airline tickets to florida from minnesota cheap airline tickets to florida from new york cheap airline tickets to florida from newark cheap airline tickets to florida from ohio cheap airline tickets to florida from philadelphia cheap airline tickets to florida from wisconsin That's a lot of wasted resources you'll be using if they look like that since you're gonna be rescraping pretty much the same urls over and over with almost identical scraping keywords. You should take your time to make a decent scraping keyword list by using say the keyword scraper in scrapebox with the word cheap and appending a - z to it then and setting it to scrape only 1 level deep. When it has finished scraping use find and replace in notepad and remove the word cheap and the space after it and you'll be left with a list like this... maxi dresses maternity clothes mattresses motels motorcycle insurance meals military flights moving boxes magazine subscriptions motorcycles nike shoes nfl jerseys north face jackets nikes name brand clothes nursing uniforms new cars nursing scrubs notebook computers nfl tickets cheapoair.com cheapoair oakley sunglasses outdoor furniture one way flights old cars for sale oil changes one way car rental oakleys office chairs plane tickets prom dresses So in other words your scraping keywords will be a lot more diverse. Then just repeat and repeat again by changing the word cheap to anything you want... blue, brown, tall, whatever.. Delete any duplicate keywords and you're good to go. Depending on what language you want to be scraping then you could even load all the keywords into Google translate and multiply your scraping keywords list by any amount you want
@mindlesswizard, how do I identify which footprints should be used and which shouldn't? @Rua999, thanks for the tip. So you're suggesting that I refine the keywords so they're diverse. Lets say I end up with 50K-100K diverse keywords, and 100 footprints, that's 5M-10M total footprinted keywords. If chose 100 urls per search for my harvesting, that'd bring me to 500,000,000 total links. This appears way too much. Can you suggest anything to refine the scraping even more?
Ye i was facing the same problem as well with footprints created by the footprint factory. I manually checked most of the footprints and only used the very best one(s) to cut down on the amount of scraping. The results are all good so far
Manually.. go to google with them and put them in manually and see which ones are giving you the most accurate results. Or do a small scrape with one of them at a time and see how accurate the results are.
