Discussion in 'Black Hat SEO' started by atongalee, Nov 12, 2010.
Pls do you know any tolls that can do this ? what tool . Pls help
Try httrack. Google it.
scrapebox. Does WONDERS.
kind of hard to find all the 10 000 pages , however it's possible.
1. use scrapebox to find all the indexed pages of the website using site: operator (however you will get just 1000 urls in this way)
2. use scrapebox internal link adon and check every internal link on the urls you already have scraped on the first step
3. put the lists together and then remove duplicate urls
Hope this helped
SB can only handle 1m urls
you can run almost infinite instances of SB at the same time.
are you scraping for emails? Other Phone, etc. Try GSA spider or scrapebox.
with 10,000 pages they probably have a database call for pages find the number ranges of the pages and have gsa crawl through them
Try using Visual Web Spider. I have been using it for years to create URL lists for my clients. Google it for the direct site!
Scrapebox is the most useful tool.
Separate names with a comma.