1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Scrape all the pages from a site?

Discussion in 'Black Hat SEO' started by hist0ry, Aug 4, 2011.

  1. hist0ry

    hist0ry Regular Member

    Joined:
    May 1, 2011
    Messages:
    204
    Likes Received:
    10
    Hello all,
    Basically Im asking how I can scrape all the pages from a blog. Basically if I have blognamehere.com how can i then scrape all the posts on that blog? sorry if its a nooby question and yes ive searched im just stuck on trying to figure this out.

    And if it is possible is it possible to do it in bulk with multiple sites?

    Note that im wanting them for scrapebox purposes
     
    Last edited: Aug 4, 2011
  2. danny_boy

    danny_boy Junior Member

    Joined:
    Apr 3, 2009
    Messages:
    181
    Likes Received:
    28
    Occupation:
    SEO
    Location:
    London
    Have you tried Httrack..
    It does most of the job for me when comes to ripping pages of a website..
     
  3. hist0ry

    hist0ry Regular Member

    Joined:
    May 1, 2011
    Messages:
    204
    Likes Received:
    10
    When you say ripping do you mean actually downloading them?

    I want the page urls so I can use it in scrapebox
     
  4. HostStage

    HostStage Jr. VIP Jr. VIP Premium Member UnGagged Attendee

    Joined:
    May 20, 2010
    Messages:
    1,770
    Likes Received:
    1,729
    Occupation:
    BHW - CEO of Webhosting Company
    Location:
    BWH from France
    Home Page:
    In srapebox, you can do this :

    HTML:
    site:http://www.thesiteyouwant.com

    Then harvest it.
     
  5. hist0ry

    hist0ry Regular Member

    Joined:
    May 1, 2011
    Messages:
    204
    Likes Received:
    10
    I actually tried that and it only found like 5 pages so i assumed that it wasnt right.

    I select Custom Footprint and type that in the custom footprint field right? thats what i tried before. Ill try it on some other sites. Also Can you do it with more then 1x site at a time with that?
     
  6. Seo Lover

    Seo Lover Jr. Executive VIP Jr. VIP Premium Member

    Joined:
    Jan 30, 2011
    Messages:
    5,693
    Likes Received:
    4,117
    Gender:
    Male
    Occupation:
    Hanging Around Interwebs !
    Location:
    <-----------------Sin City
    I also want to harvest all pages of the URL through SB , can anybody tell me in details .
    I am a totally new to Scrapebox .
     
  7. madoctopus

    madoctopus Supreme Member

    Joined:
    Apr 4, 2010
    Messages:
    1,249
    Likes Received:
    3,498
    Occupation:
    Full time IM
    if you can't code hire a freelance coder to do it for you. would cost you $50-$150. he can use an existing open source scraper.
     
  8. Panique

    Panique Power Member

    Joined:
    Sep 21, 2008
    Messages:
    589
    Likes Received:
    412
    Location:
    Caribbean Islands
    Home Page:
    There is a program called BlackWindow or BlackWiddow or something it can extract the whole site.
     
  9. maleguru

    maleguru Junior Member

    Joined:
    May 24, 2009
    Messages:
    105
    Likes Received:
    13