1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to find all the pages of lots sites with scrapebox

Discussion in 'Black Hat SEO Tools' started by MailBoxSeller, Apr 26, 2011.

  1. MailBoxSeller

    MailBoxSeller Regular Member

    Joined:
    Oct 23, 2009
    Messages:
    236
    Likes Received:
    5
    It should be easy if there is just one site,but when you find a list of ******** blogs,you want to find all the high pr pages of those sites.Scrapebox do have some tools for this -sitemap scraper and link extractor.But a lot of problems there.I'm sure link extractor don't scrape all the pages,it just lost something,it seems that is can't scrape deeply,it scrape lots of urls with category or tag,but not post.Sitemap scraper needs you to get the sitemap of a site.I have to add sitemap.xml manually.It's not a big problem.The problem is sometimes the sitemap is not like that.What's your idea about this?
     
  2. memme

    memme Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 19, 2009
    Messages:
    1,168
    Likes Received:
    99
    Location:
    Blackhatlinks.com
    Home Page:
    did you find a solution ? got the same problem
     
  3. mikey3412

    mikey3412 Junior Member

    Joined:
    Feb 3, 2010
    Messages:
    127
    Likes Received:
    2
    Let's say you found some AA blogs and you want to find all pages from those blogs, then do the following:

    Trim them to root, put the trimmed domains to SB text editor and there use Replace function (replace "http://" with "site:" ). Now your domains will have such format: site:yourdomain.com

    Now put all domains to SB harvester and scrape them. You will get all pages from those domains.

    Enjoy:)
     
  4. spankythepyro

    spankythepyro Power Member

    Joined:
    Mar 7, 2009
    Messages:
    572
    Likes Received:
    94
    site: is not working for me, or atleast with public proxies it takes just too damn long for everything to scrape. Any sollution to this?

    I know you can't use yahoo for searching an engine with site:
     
  5. TheMatrix

    TheMatrix BANNED BANNED

    Joined:
    Dec 20, 2008
    Messages:
    3,444
    Likes Received:
    7,279
    So why are you complaining in every SB thread? You have created a thread about the issue, and keep it to that only, please.
     
  6. memme

    memme Jr. VIP Jr. VIP Premium Member

    Joined:
    Sep 19, 2009
    Messages:
    1,168
    Likes Received:
    99
    Location:
    Blackhatlinks.com
    Home Page:
    if u do a site: search you will get many sites where u can not post to. /tag/ or /category/ sites....

    is there s spider out there, that only returns article pages?