Mass scraping of inner pages from domain?

Discussion in 'Black Hat SEO Tools' started by lmaestro, Jun 16, 2011.

  1. lmaestro

    lmaestro Power Member

    Joined:
    Jan 28, 2011
    Messages:
    667
    Likes Received:
    227
    How do i do this with scrapebox? I know you can do it by putting a domain as custom footprint but how do i scrape tons of domains?
     
  2. TheMatrix

    TheMatrix BANNED BANNED

    Joined:
    Dec 20, 2008
    Messages:
    3,440
    Likes Received:
    7,283
    Scrape a few keywords. Scrape a lot of URL's using those. Remove dupe, trip to root. Export to clipboard and paste into notepad.

    Next, find "http://", and replace with "site:http://"

    Now put these as keywords, and scrape.

    You will get all indexed pages of these domains.
     
    • Thanks Thanks x 1
  3. super nuver

    super nuver Newbie

    Joined:
    Aug 30, 2007
    Messages:
    42
    Likes Received:
    3
    wow i did this with public proxies and scraped about 57,000 urls. The most ive been getting is 4,000. Thanks again.