How to extract emails from 1000s URLs?

Jennifer lopez · May 23, 2018

I have a huge list (thousands) of URLs and I need to extract emails from them - all legit, live sites.
I was thinking to get the WHOIS email contacts - but this contacts might not be the best as well as many have hidden contact details.

I am not looking to hire a freelancer, but rather looking for a suggestion, tool or a script (github etc.)

Thanks!

KidRockLV · May 23, 2018

It's quite simple.. A bot that just enters the url, gets source code, filters with regex and goes to the next url.

MatthewGraham · May 23, 2018

KidRockLV said:
It's quite simple.. A bot that just enters the url, gets source code, filters with regex and goes to the next url.

You can even do this with Google Sheets. Go to Tools --> Script Editor and you can write custom code, including code that retrieves other URLs.

Write a script that takes in a URL, grabs the page source, and runs a REGEX that extracts and returns the found email(s). Put URLs you want to scrape in column A and run the function in column B.

proxified · May 23, 2018

The best and easiest way would be using scrapebox if you have a license for it or you can get one now.

710fla · May 23, 2018

I'm pretty sure you can do that with Scrapebox.

thetrustedzone · May 23, 2018

I shared simple tool for that before https://www.blackhatworld.com/seo/get-deep-extractor.731801/

Jennifer lopez · May 27, 2018

KidRockLV said:
It's quite simple.. A bot that just enters the url, gets source code, filters with regex and goes to the next url.

MatthewGraham said:
You can even do this with Google Sheets. Go to Tools --> Script Editor and you can write custom code, including code that retrieves other URLs.

Write a script that takes in a URL, grabs the page source, and runs a REGEX that extracts and returns the found email(s). Put URLs you want to scrape in column A and run the function in column B.

proxified said:
The best and easiest way would be using scrapebox if you have a license for it or you can get one now.

710fla said:
I'm pretty sure you can do that with Scrapebox.

I need to clarify that I am looking to extract not just any email but the @domain.com email so if the site is example.com the emails I am looking are @example.com emails.

@MatthewGraham / @KidRockLV can method you described pick specific emails, not just any?

@proxified I was looking into scrapebox - can it get the @domain.com emails only? can it grab names along with the emails? and/or look @domain.com emails not only at domain.com but on in search or social media?

MatthewGraham · May 27, 2018

Jennifer lopez said:
@MatthewGraham / @KidRockLV can method you described pick specific emails, not just any?

Yes, a REGEX can be customized to target any type of pattern, including only emails from a particular domain or so on.

Jennifer lopez · May 27, 2018

MatthewGraham said:
Yes, a REGEX can be customized to target any type of pattern, including only emails from a particular domain or so on.

thanks for replying so fast
when the script gets a source code - does it get the code of the whole website or just URL provided on?
My list format is domain.com and if email is not there the script won't pick it if it only sees code of a single URL

thetrustedzone · May 27, 2018

Jennifer lopez said:
thanks for replying so fast
when the script gets a source code - does it get the code of the whole website or just URL provided on?
My list format is domain.com and if email is not there the script won't pick it if it only sees code of a single URL

the tool i shared link above supports custom Regex .

Alchemist-Programmer · May 28, 2018

Use this email regex :

Code:

((?:[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*|"(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21\x23-\x5b\x5d-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])*")@(?:(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?|\[(?:(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.){3}(?:25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?|[a-z0-9-]*[a-z0-9]:(?:[\x01-\x08\x0b\x0c\x0e-\x1f\x21-\x5a\x53-\x7f]|\\[\x01-\x09\x0b\x0c\x0e-\x7f])+)\])

to extract emails. It works 99.99% time

proxified · May 28, 2018

Jennifer lopez said:
@proxified I was looking into scrapebox - can it get the @domain.com emails only? can it grab names along with the emails? and/or look @domain.com emails not only at domain.com but on in search or social media?

Yes, you can certainly do this with scrapebox

How to extract emails from 1000s URLs?

Jennifer lopez

Regular Member

KidRockLV

Registered Member

MatthewGraham

BANNED

proxified

Elite Member

710fla

Elite Member

thetrustedzone

Elite Member

Jennifer lopez

Regular Member

MatthewGraham

BANNED

Jennifer lopez

Regular Member

thetrustedzone

Elite Member

Alchemist-Programmer

Newbie

proxified

Elite Member

Main Menu

Marketplace

Making Money

BlackHat World