[HELP] Configure Scrapebox with regex for scrape mails

[Milor123]

Hi Again!!!! @loopline, I've bought the scrapebox (really was bought by a friend hahah but i work with his proyect). now I want extract mails from some urls.

I've used this expression

[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}

All work nice, but the problem is that captured things like this: "le me at</p>"

Note: when I've tested the regular expression in regex101 (online tester), all work nice, i dont understand about why scrapebox save tags and other things in the results

See this example:

[email protected]
[email protected]
[email protected]
le me at</p>
<p>
user = <a h
:[email protected]
e-field-blog_subs
[email protected]

I have question: in the CONFIGURE MASK >> The extracted data MUST NOT contain...
what is it? how should use it? with regex? one word? word separate by comma?
I would like exclude thing like this: ">, <, : "
Thank you very much

 

[sagarmadan]

1. Try this regex
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$

2. Scrape using your regex; Goto http://emailx.discoveryvip.com/ ;paste everything; click extract

 

[Milor123]

1. Try this regex
^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$

2. Scrape using your regex; Goto ***** paste everything; click extract

Nope it not work, because you use ^ the which assert to must search at beginning of line, and the $ it's the opposite.... I need scrape emails in anywhere

Also don't want use other tools, because scrapebox should be able to extract all correctly.
theoretically, the regular expression that I've used in my example, should work correctly, but I do not understand why the tool captures some things that it should not.

 

[Gogol]

Try this one too:

[a-zA-Z0-9-]{1,30}@[a-zA-Z0-9-]{1,65}.[A-Za-z]{1,}

Basically, you are going to get some malformed strings no matter what regex you use. What you can do is run that output through some kind of script for making sure that it is an email. E.g. if you use php, try this one ( it isn't flawless either but should work just fine).

 

[HoNeYBiRD]

Try this: (\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b)

If the data still doesn't seem right, you can clean it up with Notepad++ easily: http://kniko.net/how-to-extract-ema...ext-file-using-notepad-with-no-coding-at-all/

 

[Milor123]

Try this one too:

[a-zA-Z0-9-]{1,30}@[a-zA-Z0-9-]{1,65}.[A-Za-z]{1,}

Basically, you are going to get some malformed strings no matter what regex you use. What you can do is run that output through some kind of script for making sure that it is an email. E.g. if you use php, try ***** ( it isn't flawless either but should work just fine).

Yes you have reason, my regular expression have a bugs, but it is not the problem.

Try this: (\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}\b)

If the data still doesn't seem right, you can clean it up with Notepad++ easily: ****

Now I've trying with you regular expresion, work nice but when is used throght of scrapebox, still gol capture bad words, like this

NOTE: I've captured so many mails but also this bad words

➜ [email protected]
to → conntactmda@gma
O / NEGOCIOS ➜ connta
Contacto → co
/>
CONTACTO / NEGO
ociales <br />
C

is by this reason that I want know about how work this feature> CONFIGURE MASK >> The extracted data MUST NOT contain

 

[HoNeYBiRD]

Now I've trying with you regular expresion, work nice but when is used throght of scrapebox, still gol capture bad words, like this

NOTE: I've captured so many mails but also this bad words


is by this reason that I want know about how work this feature> CONFIGURE MASK >> The extracted data MUST NOT contain

Maybe the email scraping function is a bit broken in SB, because if you use that regex, it's not supposed to extract anything else but email addresses. Adding anything to "The extracted data MUST NOT contain" might not make any difference, nevertheless you can try.
I don't use SB, but even if you get bad data among the good one, you can remove it with Notepad++ easily the way how it's described on the link i posted in my previous reply. Or you can use a php script, which was already posted above or this one: https://clients.surf7.net/downloads.php?action=displaycat&catid=2 or the hosted version: https://eel.surf7.net.my/. Any of these will clean your data. It really is just one extra step, takes close to no time.

 

[Milor123]

Maybe the email scraping function is a bit broken in SB, because if you use that regex, it's not supposed to extract anything else but email addresses. Adding anything to "The extracted data MUST NOT contain" might not make any difference, nevertheless you can try.
I don't use SB, but even if you get bad data among the good one, you can remove it with Notepad++ easily the way how it's described on the link i posted in my previous reply. Or you can use a php script, which was already posted above or this one: *** or the hosted version: ****. Any of these will clean your data. It really is just one extra step, takes close to no time.

Thank you very much, I understand, probably is a SB bug or bad configuration, I would like that @loopline talking with us about of this problem.

 

[sagarmadan]

Nope it not work, because you use ^ the which assert to must search at beginning of line, and the $ it's the opposite.... I need scrape emails in anywhere

Also don't want use other tools, because scrapebox should be able to extract all correctly.
theoretically, the regular expression that I've used in my example, should work correctly, but I do not understand why the tool captures some things that it should not.

http://www.scrapebox.com/email-scraper
If you didn't know about it...

 

[Milor123]

I've found the how use this, CONFIGURE MASK >> The extracted data MUST NOT contain...

Like a wildcard, using "|" as "OR", for example:

"|:|→|>|<|/|.jpg|.png|►|&|;|➜|➜|.gov

even so, I need know about that is doing scrapebox, because these data should not be captured. Can you help me? @loopline

 

[loopline]

I've found the how use this, CONFIGURE MASK >> The extracted data MUST NOT contain...

Like a wildcard, using "|" as "OR", for example:

"|:|→|>|<|/|.jpg|.png|►|&|;|➜|➜|.gov

even so, I need know about that is doing scrapebox, because these data should not be captured. Can you help me? @loopline

So lets first address the elephant in the room. Scrapebox has a excellent built in email scraper and then also an email scraper plugin that has tons more filters and extra features like email verifier and advanced custom data scraper. So if you just want to scrape emails and/or emails and the url that the mail came from, then there is zero point in reinventing the wheel.

So just use whats built in

AND

Now, moving on and assuming you are using the custom data scraper to try and get the email and other data you can not get with the email scraper that is already built in:

Scrapebox uses the Delphi regex flavor, so that may help you some.

Next is me stepping up and saying what I am good at and what I am not. I can do a lot of things, but I hate to code, I even don't like regex. It is not something that flows naturally for me.

I taught myself python and it took months and I can only do rudimentary things and its verbose and looks terrible, but it works, lol.

I learned python so I could better direct the person who now develops everything I build, from a pure code standpoint. I don't know regex well enough to sort your issue, and if I wanted to try it would take me weeks probably. Why?

Because I don't like it and I have a billion things on my plate and its just not my thing. Im a firm believer in doing what you do well and do that to the max and surround yourself with people who are good at what you are not.



As to your question - what the extracted data must not contain part - here is an ultra simple example.

Lets say you want to extract links external links from domain.com but not internal links,

domain.com

in the must not contain. That way if scrapebox finds a string that matches your must contain it then checks it against the must not contain. If it finds the must not contain, it doesn't save the data.

I will double check, but I do Not believe that the must and must not contain fields support regex. So I think only "regex to extract data, or before /after" box is the only one that supports regex.

then you would put like

=|<|>

in the must not contain box and it would filter out any matching string that has that. However if that happens to be in the same string as the email its self then it would also filter out the email. So use with caution.

 

[Milor123]

So lets first address the elephant in the room. Scrapebox has a excellent built in email scraper and then also an email scraper plugin that has tons more filters and extra features like email verifier and advanced custom data scraper. So if you just want to scrape emails and/or emails and the url that the mail came from, then there is zero point in reinventing the wheel.

So just use whats built in

AND

Now, moving on and assuming you are using the custom data scraper to try and get the email and other data you can not get with the email scraper that is already built in:

Scrapebox uses the Delphi regex flavor, so that may help you some.

Next is me stepping up and saying what I am good at and what I am not. I can do a lot of things, but I hate to code, I even don't like regex. It is not something that flows naturally for me.

I taught myself python and it took months and I can only do rudimentary things and its verbose and looks terrible, but it works, lol.

I learned python so I could better direct the person who now develops everything I build, from a pure code standpoint. I don't know regex well enough to sort your issue, and if I wanted to try it would take me weeks probably. Why?

Because I don't like it and I have a billion things on my plate and its just not my thing. Im a firm believer in doing what you do well and do that to the max and surround yourself with people who are good at what you are not.



As to your question - what the extracted data must not contain part - here is an ultra simple example.

Lets say you want to extract links external links from domain.com but not internal links,

domain.com

in the must not contain. That way if scrapebox finds a string that matches your must contain it then checks it against the must not contain. If it finds the must not contain, it doesn't save the data.

I will double check, but I do Not believe that the must and must not contain fields support regex. So I think only "regex to extract data, or before /after" box is the only one that supports regex.

then you would put like

=|<|>

in the must not contain box and it would filter out any matching string that has that. However if that happens to be in the same string as the email its self then it would also filter out the email. So use with caution.

Hey loopline, thank you very much, I understand

Why I've said that SB could have a bug?... I've develop a python script for capture mails and numbers, in a list de urls with proxys (single thread), and when I get not the mails have nothing these semantic malformations. I get all like should work.

if you want I could upload the python code to github for that you test it.

Thank you very much for you reply
PD: I dont speak english xD

 

[Gogol]

So lets first address the elephant in the room. Scrapebox has a excellent built in email scraper and then also an email scraper plugin that has tons more filters and extra features like email verifier and advanced custom data scraper. So if you just want to scrape emails and/or emails and the url that the mail came from, then there is zero point in reinventing the wheel.

So just use whats built in

AND

Now, moving on and assuming you are using the custom data scraper to try and get the email and other data you can not get with the email scraper that is already built in:

Scrapebox uses the Delphi regex flavor, so that may help you some.

Next is me stepping up and saying what I am good at and what I am not. I can do a lot of things, but I hate to code, I even don't like regex. It is not something that flows naturally for me.

I taught myself python and it took months and I can only do rudimentary things and its verbose and looks terrible, but it works, lol.

I learned python so I could better direct the person who now develops everything I build, from a pure code standpoint. I don't know regex well enough to sort your issue, and if I wanted to try it would take me weeks probably. Why?

Because I don't like it and I have a billion things on my plate and its just not my thing. Im a firm believer in doing what you do well and do that to the max and surround yourself with people who are good at what you are not.



As to your question - what the extracted data must not contain part - here is an ultra simple example.

Lets say you want to extract links external links from domain.com but not internal links,

domain.com

in the must not contain. That way if scrapebox finds a string that matches your must contain it then checks it against the must not contain. If it finds the must not contain, it doesn't save the data.

I will double check, but I do Not believe that the must and must not contain fields support regex. So I think only "regex to extract data, or before /after" box is the only one that supports regex.

then you would put like

=|<|>

in the must not contain box and it would filter out any matching string that has that. However if that happens to be in the same string as the email its self then it would also filter out the email. So use with caution.

Dude, an awesome reply! but your reply doesn't answer why

[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}

still produces some invalid data, doesn't it @Milor123 ?
Or does it? Why don't you reply with the regex lol? Okay may be with the flavor you like ? Why complicate stuffs fr your future customers?



oh btw let me add to this... is this how you usually reply your customers? LOL I am proud that i write my own code lmao.

 

[Milor123]

Dude, an awesome reply! but your reply doesn't answer why

[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}

still produces some invalid data, doesn't it @Milor123 ?
Or does it? Why don't you reply with the regex lol? Okay may be with the flavor you like ? Why complicate stuffs fr your future customers?



oh btw let me add to this... is this how you usually reply your customers? LOL I am proud that i write my own code lmao.

Really the these regular expression is not fully optimized but even so, it should don't capture malformed data, you could try use this same expression and scrape in python and it haven't problems.

I think that ScrapeBox could have some BUG or bad configuration in the code, example in the regex flags ... I dont know... We only can wait for @loopline do debugging

 

[loopline]

Hey loopline, thank you very much, I understand

Why I've said that SB could have a bug?... I've develop a python script for capture mails and numbers, in a list de urls with proxys (single thread), and when I get not the mails have nothing these semantic malformations. I get all like should work.

if you want I could upload the python code to github for that you test it.

Thank you very much for you reply
PD: I dont speak english xD

I still don't know what your doing. I mean the scrapebox email scraper doesn't return any of this.

Dude, an awesome reply! but your reply doesn't answer why

[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,4}

still produces some invalid data, doesn't it @Milor123 ?
Or does it? Why don't you reply with the regex lol? Okay may be with the flavor you like ? Why complicate stuffs fr your future customers?



oh btw let me add to this... is this how you usually reply your customers? LOL I am proud that i write my own code lmao.

I don't own scrapebox, I don't develop it, and I don't work for them.

I mean I own licensed copies, but I don't own the company, not even 1 share of it. I don't develop it, I don't work for them. Im just an expert user and I like to help people.

so Im not really sure what the point of your reply is, Im just donating my time to try and help people and I don't know why the OP isn't using the email scraper in the first place. I also clearly stated I don't like coding and when I said I hired a coder its for my own personal projects, not something I sell.

The people that own scrapebox, SweetFunny (here on BHW) is the person that codes it also softtouch (also here on BHW) codes scrapebox.

Anyway moving on.

 

[Milor123]

I still don't know what your doing. I mean the scrapebox email scraper doesn't return any of this.



I don't own scrapebox, I don't develop it, and I don't work for them.

I mean I own licensed copies, but I don't own the company, not even 1 share of it. I don't develop it, I don't work for them. Im just an expert user and I like to help people.

so Im not really sure what the point of your reply is, Im just donating my time to try and help people and I don't know why the OP isn't using the email scraper in the first place. I also clearly stated I don't like coding and when I said I hired a coder its for my own personal projects, not something I sell.

The people that own scrapebox, SweetFunny (here on BHW) is the person that codes it also softtouch (also here on BHW) codes scrapebox.

Anyway moving on.

Ahh sorry is for my english, I thought that you was the ScrapeBox Author hahaha. do you know how can report it to ScrapeBox developers?

 

[loopline]

Ahh sorry is for my english, I thought that you was the ScrapeBox Author hahaha. do you know how can report it to ScrapeBox developers?

There is nothing to report, Ive known of many people that have used regex to scrape mails with scrapebox.


http://www.regexlib.com/Search.aspx?k=email&c=-1&m=-1&ps=20


There’s a number of email extraction regex compatible with SB here you can try. When you add regex, if it's listed with a ^ at the start and $ at the end you just need to remove those 2 symbols because they mean to match the start and end of a line which is not required when dealing with content randomly located within HTML and not exactly at the start and end of a line.

There are tons of regex flavors and variations so what you use in other languages may not work in Scrapebox. The issue here isn't a bug with scrapebox, its just a matter of building a regex that works with the way scrapebox is setup, not trying to force scrapebox to work with the regex you built that works elsewhere.

 

[Gogol]

I don't own scrapebox, I don't develop it, and I don't work for them.

I mean I own licensed copies, but I don't own the company, not even 1 share of it. I don't develop it, I don't work for them. Im just an expert user and I like to help people.

so Im not really sure what the point of your reply is, Im just donating my time to try and help people and I don't know why the OP isn't using the email scraper in the first place. I also clearly stated I don't like coding and when I said I hired a coder its for my own personal projects, not something I sell.

The people that own scrapebox, SweetFunny (here on BHW) is the person that codes it also softtouch (also here on BHW) codes scrapebox.

Anyway moving on.

Oops OP sounded like you are responsible for the software. I am really sorry if I have hurt you by any means

 

[loopline]

Oops OP sounded like you are responsible for the software. I am really sorry if I have hurt you by any means

Its all good, just trying to help. Hope you can sort it.

Ive seen this 20+ times on regex stuff, where it requires the delphi flavor and someone is having an issue with regex that works in a different script/language/flavor/variation just fine. But generally people eventually sort it and get what they want from scrapebox.

Regex is just one of those things where there is no set "standard" that works universally. So you have to sort out what it takes to work with scrapebox. Hopefully the above examples in my above post will help.

 

[Yzetien]

/[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?/g

Try this for email extracting, and anyone know where is the original QUOTE button in the forum post...?