Scrapebox assistance

Discussion in 'Black Hat SEO' started by schmidcpc, Oct 27, 2010.

  1. schmidcpc

    schmidcpc Newbie

    Joined:
    Feb 15, 2010
    Messages:
    6
    Likes Received:
    0
    I understand the 1000 URL harvest limitation for scrapebox per keyword/per custom footprint, however does anyone have any recommendation on how to work around this?

    Ultimately I need to scrape one particular site for approx 30k urls.

    Any suggestions? and thank you for your feedback
     
  2. thedorf

    thedorf Supreme Member

    Joined:
    Oct 1, 2008
    Messages:
    1,223
    Likes Received:
    731
    Occupation:
    what? I gotta have a job?
    Location:
    BHW - Where else?
    From what I understand the 1000 limitation only applies to the search engines. You can scrape from websites all day if you want.

    Anyone verify this?
     
  3. darshan1994

    darshan1994 BANNED BANNED

    Joined:
    Oct 9, 2009
    Messages:
    654
    Likes Received:
    319
    if you are scraping backlinks than get all the urls of the domain and use backlink analyzer.
     
  4. schmidcpc

    schmidcpc Newbie

    Joined:
    Feb 15, 2010
    Messages:
    6
    Likes Received:
    0
    I just ran a custom footprint site:whatever with no keywords and returned exactly 1000 results. Apparently the limitation does apply
     
  5. Spaceman

    Spaceman Regular Member

    Joined:
    Aug 8, 2009
    Messages:
    435
    Likes Received:
    53
    Hi - depends on the site - but I sometimes put a date in the footprint.
    or a day. works well for forums once you know the date format and scrape multiple times changing the date.
    Im not sure you will get the lot with that though - you really need to write a small crawler - or find one which returns all URLS for a site - im sure theres one somewhere.....

    Spaceman