1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tool for email scraping ? - Must be able to get past login

Discussion in 'Black Hat SEO Tools' started by MinistryOfThought, May 1, 2016.

  1. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    Hi BHW,

    I am looking for a tool that will allow me to scrape emails from sites like forums, linkedin, twitter, ect..
    I found a free tool call EmailscaperWizard which is free and good but it cannot get pass logins

    I want to scrape a forum that requires you to login to see posts and member info (I have a login)

    Can anyone recommend a good software ?
    I am not sure if scrapebox can do this. Paid is fine.

    Thanks !
     
    • Thanks Thanks x 1
  2. goodperson

    goodperson Elite Member

    Joined:
    Jan 3, 2015
    Messages:
    1,649
    Likes Received:
    227
    Location:
    www.niceaccount.com
    Home Page:
    Try Email Extractor, 1 addon of firefox. It's ok
     
  3. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    I am looking for something that will also be able to crawl the site. There are thousands of users and posts.
     
  4. ads9oa7

    ads9oa7 Newbie

    Joined:
    May 1, 2016
    Messages:
    16
    Likes Received:
    0
    I'd also be interested in such a tool. Especially the full website scraping part...

    There is a python framework optimised for scraping, it's called scrapy and there are some nice tutorials on YouTube. Looks like a bit more work than I was willing to invest...
     
  5. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    perhaps hiring a VA do the leg work ?
    or pooling resources together ? what use are you trying to get out of yours ?
     
  6. skytek666

    skytek666 Newbie

    Joined:
    Feb 4, 2016
    Messages:
    28
    Likes Received:
    6
    Im also interested in what your after as i could be interested?
     
    • Thanks Thanks x 1
  7. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    I am trying to get a bot that will scrape the emails from users profiles. The site that I want to run it on has thousands of users however, it is brick-walled behind a login.
    I have a pro account and thus access to the users via member search and forum posts, I want a bot to crawl the forums and go to member's pages to get emails.
    Should be simple, I tried a few bots but they dont work since they have no login functionality.
     
  8. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    Do you know of anyway to integrate this with a crawler ?
     
    • Thanks Thanks x 1
  9. apexbeat

    apexbeat Junior Member

    Joined:
    Sep 24, 2012
    Messages:
    125
    Likes Received:
    40
    You want to scrap a website after your account logs in right? You can do this with Linux wget command for free (if website has cloud flair that might be an issue)
    wget --recursive --post-data "email=$EMAIL&password=$PASSWRD" read the wget manual - its a long read but should will work (maybe not if they have cloud flair)

    https://www.gnu.org/software/wget/manual/wget.html

    If u need help and i got the time PM me. because "email=$EMAIL&password=$PASSWRD" might not be the proper format expected by the site :)
     
    • Thanks Thanks x 1
  10. SEO Sicko

    SEO Sicko Power Member

    Joined:
    Oct 24, 2014
    Messages:
    778
    Likes Received:
    132
    Occupation:
    Digital marketer
    Home Page:
    You can try The Best Email Scraper.
    It is easy tool you can just place the URL's you want to scrap.
    After the process the results will be downloaded automatically.
     
  11. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    Thanks I will take a look into this. I have entry level coding skills but hopefully I can figure it out. I can't PM you now due to post count but I will give it a try myself and let you know if I need a little help. Big thanks.
     
  12. MinistryOfThought

    MinistryOfThought Newbie

    Joined:
    Apr 28, 2016
    Messages:
    12
    Likes Received:
    3
    I am not sure if I can find the tool you are referring to since a google search for that term brings many results and I didnt see any software with that specific link. Maybe you could link directly ?

    Are you suggesting that I scrape URLs with another tool such as scrapebox and then run the URLs through "the best email scrapper?" I am looking to crawl thousands of pages so manual is not really an option.
    The solution above could work though since although user profile data is blocked out I can browse the forum and get to member's pages.
    I could then log in and scrape using the best email scraper on my browser I assume
     
  13. kimberlarck

    kimberlarck Newbie

    Joined:
    Apr 19, 2016
    Messages:
    5
    Likes Received:
    0
    maybe u can try scrapebox....
     
  14. fastlinks

    fastlinks Power Member

    Joined:
    Feb 4, 2015
    Messages:
    617
    Likes Received:
    75
    I can make custom bot for this task, add me

    skype: ipowerhost2