1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrape All Pages of a Website??

Discussion in 'Black Hat SEO Tools' started by neo3187, Jan 23, 2011.

  1. neo3187

    neo3187 Senior Member

    Joined:
    Sep 19, 2010
    Messages:
    868
    Likes Received:
    360
    Is it possible to scrape all pages of a website which are indexed?? I tried doing it with the custom footprint like site:xyz.com and putting letters a-z in the keywords column, but everytime it just fetches 200 and stops... any suggestions??

    EDIT: Iam using scrapebox already, need advise for it
     
    Last edited: Jan 23, 2011
  2. peter73

    peter73 Regular Member

    Joined:
    Jun 27, 2010
    Messages:
    341
    Likes Received:
    90
    Location:
    X marks the spot----------------------------X
    Your best bet is to use ScrapeBox. Works like a charm.
     
  3. neo3187

    neo3187 Senior Member

    Joined:
    Sep 19, 2010
    Messages:
    868
    Likes Received:
    360
    Iam using scrapebox, sorry for not mentioning in the OP
     
  4. movieman32

    movieman32 Regular Member

    Joined:
    Aug 6, 2008
    Messages:
    371
    Likes Received:
    346
    It may be stopping because G may say a site has x number of results, and shows 10 pages of results, but if you were to manually click on the search results page numbers, G may only list part of the site pages it claims to have indexed.