I'm looking for a an email scraper for facebook and web, suggestions?

[ill13]

As the title says, I'm looking for a an email scraper for facebook and web forums. It seems things have changed over the past few years and it is getting harder to scrape emails from bigger sites.

Atomic email hunter versus scrapebox

A PHP or Python solution?

Thank you

 

[loopline]

Scrapebox can scrape business emails from facebook as those tend to generally be publicly available. However personal mails are hard to get and typically are done via the API, which scrapebox does not support.

As for scraping forms it can do that just fine, assuming 2 criteria are met (for any site actually):

1 - The email is publicly viewable with javascript turned off. Meaning the email is not rendered by javascript but can be found when viewing the page source with javascript off

2 - The email is publicly viewable, meaning you do not have to be logged in to see the email.

I can't speak for the other programs or solutions Im just sharing how scrapebox works so you can evaluate and make the best choice.

 

[ill13]

Unfortunately that confirms most of my concerns.

Most forums hide the user's email addresses regardless of javascript usage or not. And many websites are following suit

At this point it looks like I'll need to roll my own or find something on github that uses requests-html, puppetter / pypetter and chromium / selenium since they can at least do javascript rendering and parsing.

Thank you!

 

[loopline]

Unfortunately that confirms most of my concerns.

Most forums hide the user's email addresses regardless of javascript usage or not. And many websites are following suit

At this point it looks like I'll need to roll my own or find something on github that uses requests-html, puppetter / pypetter and chromium / selenium since they can at least do javascript rendering and parsing.

Thank you!

Your welcome, good luck!

 

[MiLe31]

As the title says, I'm looking for a an email scraper for facebook and web forums. It seems things have changed over the past few years and it is getting harder to scrape emails from bigger sites.

Atomic email hunter versus scrapebox

A PHP or Python solution?

Thank you

I've been happy with a product by Redscraper. https://www.redscraper.com/lead-prospector/
You can scrape YT, FB, LinkedIn, Quora, etc. Really solid.


Strengths I've found:
1.) really reasonable, low-key customer service. They are India-based, so they'll communicate over Whatsapp. They work late all the time on my behalf.
2.) good data points exportable into .CSV
3.) seemed cheap to me...

Demerits:
1.) kind of slow (2-5k emails scraped per hour... unless you buy your personal proxy server with them, which they'll hook you up with for $100 a month for a few rotating proxies)
2.) small team... not sure how long they'll be around

 

[MiLe31]

I've been happy with a product by Redscraper. https://www.redscraper.com/lead-prospector/
You can scrape YT, FB, LinkedIn, Quora, etc. Really solid.


Strengths I've found:
1.) really reasonable, low-key customer service. They are India-based, so they'll communicate over Whatsapp. They work late all the time on my behalf.
2.) good data points exportable into .CSV
3.) seemed cheap to me...

Demerits:
1.) kind of slow (2-5k emails scraped per hour... unless you buy your personal proxy server with them, which they'll hook you up with for $100 a month for a few rotating proxies)
2.) small team... not sure how long they'll be around

 

[ill13]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a https://github.com/search?l=Python&o=desc&q=email+scraper&s=updated&type=Repositories.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

 

[MiLe31]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a python script for free.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

Sure, I knew the proxy cost was high... but... the value I get from that and the ease of use... and the SPEED... man... I'm scraping 100k emails a day. It was worth. And these guys are so amiable and hardworking. I wasn't about to haggle with them.

 

[MiLe31]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a python script for free.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

The emails I'm getting are gmail, outlook, yahoo, icloud, etc... no business domains. I use EverString for that.

 

[MiLe31]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a python script for free.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

They have a single search free month trail you can check out. You can scrape Facebook, Quoara, Pintrest, etc... this will better ensure that you're not getting business emails, regardless of domain

 

[ill13]

I have to admit, I had to check out your other posts to see if you were selling this software yourself. It seems like you are working on a completely different project.

With that in mind, I will actually check out your recommendation!

Thank you!

 

[MiLe31]

I have to admit, I had to check out your other posts to see if you were selling this software yourself. It seems like you are working on a completely different project.

With that in mind, I will actually check out your recommendation!

Thank you!

Haha. Yeah, it's all an elaborate scheme to sell my product.

I'm just satisfied with their product. I'm using it for a huge project I'm working on.

Since we are on the topic of emails... what is the best email verification service y'all use? The one I"m using says they are clean but I'm still getting huge bounce rates even after paying fees.

 

[ill13]

LOL...I think 87% of the posts here are an elaborate scheme to sell their products!

Bounce rates might be an issue with the way you are sending mail. If you are sending from your own IP addresses (and domains), I believe there's a fair amount of prep work you need to do with warming up IPs, DMARC, SPF and DKIM settings.

With that said, you may want to scale way back on sending emails until you 'warm up' your IP addresses. I think that you may screw your up domain's ability to send email without being considered spam, if you are not extra careful.


You may already know all of this...

I looked at a few more of your posts. When you get some time highly suggest you look into sendy (https://sendy.co/). There's a lot of technical stuff here, but you can probably manage it.

You can use a service at outrageous prices...Or roll your own with sendy. From what I understand 'cold email marketing' has many rules that didn't exist 2-5 years ago. I too, am gearing up for some serious cold email marketing campaigns.

If you've got the money to spend I'd certainly look into getting some paid help from here or fivver or whatever.

 

[MiLe31]

LOL...I think 87% of the posts here are an elaborate scheme to sell their products!

Bounce rates might be an issue with the way you are sending mail. If you are sending from your own IP addresses (and domains), I believe there's a fair amount of prep work you need to do with warming up IPs, DMARC, SPF and DKIM settings.

With that said, you may want to scale way back on sending emails until you 'warm up' your IP addresses. I think that you may screw your up domain's ability to send email without being considered spam, if you are not extra careful.


You may already know all of this...

I looked at a few more of your posts. When you get some time highly suggest you look into sendy (https://sendy.co/). There's a lot of technical stuff here, but you can probably manage it.

You can use a service at outrageous prices...Or roll your own with sendy. From what I understand 'cold email marketing' has many rules that didn't exist 2-5 years ago. I too, am gearing up for some serious cold email marketing campaigns.

If you've got the money to spend I'd certainly look into getting some paid help from here or fivver or whatever.

As of yet, I am not sending mass emails. I am merely testing different verification software and offerings online by sending 10 or so emails... mostly have 10% bounce rates even after spending $10 per 1000 emails verfied. It's frustrating and worrying for when I'm ready for my big email campaign.

 

[MiLe31]

If you only need to scrape a few dozen emails, you could always do it manually by creating a custom Google search engine.

Here is a TY video tutorial I've used and it works.

 

[velour]

try mail.biz i got it

 

[digital_dan]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a https://github.com/search?l=Python&o=desc&q=email+scraper&s=updated&type=Repositories.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

You seem quite knowledgeable in this field.
I got lost in the VPS, VPN bit. You're gonna use OpenVPN to create your proxies? how does that work?

Currently I just have IPs rotating on each server connected to mailwizz. Although I won't give Mailwizz a great score(6) for sending campaigns. breaks so easily for one.. What other mailer have you tried? (sendy 6.5 i'll say)

And what's your view on socks5?

 

[ill13]

First off, I misspoke about OpenVPN, it won't work in this way.

So, ,moving on...Currently I use sitegrounds for all of my business wordpress installs, none of the following will work with sitegrounds, hostgator, or whatever. I only bring this up because I feel it is much simpler to have a wordpress woocommerce install on a commercial server as the cost is so low and they take care of the maintenance.


However, I use VPSs to host some of my cloud backup servers and programming projects. I pay about $40 a year for a server in NYC and under $5 for Romanian, I'm sure other EU countries are around the same price.

They all run debian.

I can install squid proxy on these Debian servers with this:

https://www.liquidweb.com/kb/install-squid-proxy-server-ubuntu-16-04/
Following that tutorial will turn a cheap VPS server into a proxy. First one might take you up to a week to figure out. After that learning process you can probably set one up in 5 minutes on any VPS anywhere.

To be honest, I have not implemented any of this, but I understand enough to make this work...Github and StackOverflow will fill in any blank spots as none of this is any more advanced as setting up apache or nginx.

 

[pennya08]

Thanks, for the info.

Unfortunately, as you can see almost all of those emails are business email addresses and as such have, what I feel to be very low value.

Plus, those emails can be scraped pretty easily with a python script for free.


In regards to proxies....$100 a month is very steep. I could buy/rent my own VPSs and VPNs all over the world for $5 a month or less.


Granted I have may have a little more technical knowledge than some people, but installing OpenVPN is pretty painless.

10 personal proxies for $50 a month and I use the other $50 for...anything else. Plus no subscription fees adding to $1200 a year. Heck I could pay someone on upwork or fivver $20USD once to setup all my servers for me.

With that said, if that app fulfills your needs, there is nothing wrong with it!

Thank you!

Where do you get your VPNs and VSPs from?

 

[ill13]

vultr,hostinger,ramnode