Screen Scraper Tricks by Michael Schrenk

Discussion in 'Black Hat SEO' started by MrBlue, Jan 27, 2010.

  1. MrBlue

    MrBlue Senior Member

    Joined:
    Dec 18, 2009
    Messages:
    975
    Likes Received:
    682
    Occupation:
    Web/Bot Developer
    This is one of the more interesting talks from the last DEFCON.
    Screen scraping and automation with iMacros and PHP.

    Screen Scraper Tricks: Extracting Data from Difficult Websites

    by Michael Schrenk

    PDF
    Code:
    http://www.defcon.org/images/defcon-17/dc-17-presentations/defcon-17-michael_shrenk-screen_scraper_tricks.pdf
    Video
    Code:
    https://media.defcon.org/dc-17/video/DEFCON%2017%20Hacking%20Conference%20Presentation%20By%20Michael%20Schrenk%20-%20Screen%20Scraper%20Tricks%20Extracting%20Data%20from%20Difficult%20Websites%20-%20Video%20and%20Slides.m4v
     
    • Thanks Thanks x 4
  2. haridada

    haridada Senior Member

    Joined:
    Oct 9, 2008
    Messages:
    1,000
    Likes Received:
    383
    Location:
    Chennai
    Thanks. Was looking for some thing like this.
     
  3. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    1,001
    Nice find. Schrenk is a smart guy and a great author.

    I highly recommend his book Webbots, Spiders, and Screen Scrapers if your interested in beginner-friendly guide to php/cURL automation.
     
  4. haridada

    haridada Senior Member

    Joined:
    Oct 9, 2008
    Messages:
    1,000
    Likes Received:
    383
    Location:
    Chennai
    @Grizzy, do you happen to have that book? If so, can you upload it?
     
  5. CoyoteAssassin

    CoyoteAssassin Elite Member

    Joined:
    Jan 3, 2010
    Messages:
    1,870
    Likes Received:
    3,943
    Occupation:
    Full Time IMer
    Location:
    USA
    This is a great addition - thank you for adding it. I run into the complex pages at time and the tools here should work. I could not get the video loading. Any luck for anyone else?
     
  6. nixnash

    nixnash Power Member

    Joined:
    Oct 26, 2009
    Messages:
    581
    Likes Received:
    205
    Occupation:
    Student
    Location:
    BHW
    mind blowing...Been trying enhance a scrapper, which i use ofr myself...Was finding it difficult to find content and references...would help me a lot...
     
  7. Grizzy

    Grizzy Senior Member

    Joined:
    Nov 11, 2008
    Messages:
    919
    Likes Received:
    1,001
    I actually bought the book a while back, it was on sale (at a grocery store of all places) and it caught my eye.

    But I'll just leave this here..
    Code:
    http://hotfile.com/dl/26523793/be45312/No-Starch-Press-Webbots-Spiders-and-Screen-Scrapers-Mar-2007.rar.html