1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrapebox assistance

Discussion in 'Black Hat SEO' started by schmidcpc, Oct 27, 2010.

  1. schmidcpc

    schmidcpc Newbie

    Joined:
    Feb 15, 2010
    Messages:
    6
    Likes Received:
    0
    I understand the 1000 URL harvest limitation for scrapebox per keyword/per custom footprint, however does anyone have any recommendation on how to work around this?

    Ultimately I need to scrape one particular site for approx 30k urls.

    Any suggestions? and thank you for your feedback
     
  2. thedorf

    thedorf Senior Member

    Joined:
    Oct 1, 2008
    Messages:
    1,193
    Likes Received:
    700
    Occupation:
    what? I gotta have a job?
    Location:
    BHW - Where else?
    From what I understand the 1000 limitation only applies to the search engines. You can scrape from websites all day if you want.

    Anyone verify this?
     
  3. darshan1994

    darshan1994 BANNED BANNED

    Joined:
    Oct 9, 2009
    Messages:
    654
    Likes Received:
    318
    if you are scraping backlinks than get all the urls of the domain and use backlink analyzer.
     
  4. schmidcpc

    schmidcpc Newbie

    Joined:
    Feb 15, 2010
    Messages:
    6
    Likes Received:
    0
    I just ran a custom footprint site:whatever with no keywords and returned exactly 1000 results. Apparently the limitation does apply
     
  5. Spaceman

    Spaceman Regular Member

    Joined:
    Aug 8, 2009
    Messages:
    435
    Likes Received:
    53
    Hi - depends on the site - but I sometimes put a date in the footprint.
    or a day. works well for forums once you know the date format and scrape multiple times changing the date.
    Im not sure you will get the lot with that though - you really need to write a small crawler - or find one which returns all URLS for a site - im sure theres one somewhere.....

    Spaceman