1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to best scrape when the info needed is one page deep?

Discussion in 'Black Hat SEO' started by Ampix0, Nov 17, 2015.

  1. Ampix0

    Ampix0 Power Member

    Joined:
    Jan 10, 2012
    Messages:
    528
    Likes Received:
    60
    Home Page:
    I have Scrapebox and Web Harvy, web harvy is definitely more of what I need for this but I don't believe it can do what I need either. Web harvy is very good at scraping information off a page and then proceeding to the next page. Very good for tabular data that might run multiple pages, like yellow book

    What I often run in to is a list of links, and the information I need is all still formatted the same so Web Harvy COULD pick it up, but you have to follow the link, grab the info, come back, go to the next link and so on. Web Harvy can not go another level deep is what I mean (as far as I know).

    ex:

    Code:
    [B]List of Tennis Players[/B]
    Person 1
    Person 2
    ...
    NEXT PAGE
    Now imagine each one of those is a link to a profile page like so:
    Code:
    [B]Person 1[/B]
    score: 86
    height: 5'3
    ...
    

    I need the information from each page like so scraped.
     
  2. trance92071

    trance92071 Senior Member

    Joined:
    Nov 1, 2009
    Messages:
    950
    Likes Received:
    849
    Occupation:
    Internet Marketing
    Location:
    BoosterBots.com
    Home Page:
    Humm.. I can build you a custom bot for this. Let me know if you are looking for someone.
     
  3. Ampix0

    Ampix0 Power Member

    Joined:
    Jan 10, 2012
    Messages:
    528
    Likes Received:
    60
    Home Page:
    I appreciate the offer, but I wasnt looking to spend anything currently. I am also a programmer, so if I REALLY need to I will come up with some simple script, probably opt to go imacros for simplicity. I am just shocked nothing exists.