1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

looking for a good scraping tool any recommendations?

Discussion in 'Black Hat SEO' started by iTheExpert, May 18, 2014.

Tags:
  1. iTheExpert

    iTheExpert BANNED BANNED

    Joined:
    Oct 25, 2013
    Messages:
    116
    Likes Received:
    38
    I use gscraper but what I do not like about it is I am not getting platform identification. It does not tell me that particular url http://www.abcd.com is for example: articledirectory-articlebeach or PHPLD or it's bookmarking - pligg platform or Oxwall or forum profile - xoops. I am looking for a way to identify all the URLs or another scraping tool where I can get a list with platform information. Any suggestions? Thank you!
     
  2. Scritty

    Scritty Elite Member Premium Member

    Joined:
    May 1, 2010
    Messages:
    2,807
    Likes Received:
    4,496
    Occupation:
    Affiliate Marketer
    Location:
    UK
    Home Page:

    The real choices are Gscraper, Scrapebox and Hrefer.

    However, mis-identified platforms goes with the territory. It's a function of the search term and NOT a function of the tool which transmits the search term.
    In other words - the footprint you use to scrape with.

    95% none conformity is pretty normal.
    I scrape on several severs 24/7 - about 2.2 million URLS a day the vast majority of which are not the platforms they were supposed to be, and I use some really good footprints that I keep up to date all the time (and sorry - no - I don't give them away - to anyone - ever - they are FAR too valuable to me)

    But I suspect your answer is better footprints and a more thorough post scrape testing regime rather than a change of tool.

    Good luck. Once you develop a system then it gets easier.
    Hint A spreadsheet program like Excel or Openoffice Calc (free) is VERY useful.

    Scritty
     
  3. WebsiteLiving

    WebsiteLiving Registered Member

    Joined:
    Feb 5, 2014
    Messages:
    99
    Likes Received:
    18
    Home Page:
    Have you written your own scraper scripts or do you use scrapebox?
     
  4. notahipster

    notahipster Newbie

    Joined:
    May 8, 2014
    Messages:
    8
    Likes Received:
    1
    What are you looking to scrape? I use the Scrape Similar chrome extension.