How to find all the pages of lots sites with scrapebox

MailBoxSeller

Regular Member
Joined
Oct 23, 2009
Messages
236
Reaction score
5
It should be easy if there is just one site,but when you find a list of ******** blogs,you want to find all the high pr pages of those sites.Scrapebox do have some tools for this -sitemap scraper and link extractor.But a lot of problems there.I'm sure link extractor don't scrape all the pages,it just lost something,it seems that is can't scrape deeply,it scrape lots of urls with category or tag,but not post.Sitemap scraper needs you to get the sitemap of a site.I have to add sitemap.xml manually.It's not a big problem.The problem is sometimes the sitemap is not like that.What's your idea about this?
 
did you find a solution ? got the same problem
 
Let's say you found some AA blogs and you want to find all pages from those blogs, then do the following:

Trim them to root, put the trimmed domains to SB text editor and there use Replace function (replace "http://" with "site:" ). Now your domains will have such format: site:yourdomain.com

Now put all domains to SB harvester and scrape them. You will get all pages from those domains.

Enjoy:)
 
site: is not working for me, or atleast with public proxies it takes just too damn long for everything to scrape. Any sollution to this?

I know you can't use yahoo for searching an engine with site:
 
site: is not working for me, or atleast with public proxies it takes just too damn long for everything to scrape. Any sollution to this?

I know you can't use yahoo for searching an engine with site:

So why are you complaining in every SB thread? You have created a thread about the issue, and keep it to that only, please.
 
if u do a site: search you will get many sites where u can not post to. /tag/ or /category/ sites....

is there s spider out there, that only returns article pages?
 
Back
Top