anybody knows how to extract data with Regex?

[tazarbm]

Hi,

I am trying to extract data from a vendor's profile on Warriorplus using Scrapebox, and there are 2 options in the Custom Data Grabber function of Scrapebox for that:

- extract by using HTML (before|after) code
- extract using Regex

I can't use HTML option because the data that I need doesn't show up in the source code of the profile's page, and I can't use Regex because I don't know it...

I was wondering, can one of you, code wizards (who's good with Regex) provide the correct formula for extracting the data for the "Last Active: " field (see the red circle below):

 

[satyr85]

Scrapebox is not really meant for this.
Warrior+ has cloudflare protection, scrapebox wont bypass it.

Custom python code + good proxies.

 

[tazarbm]

Warrior+ has cloudflare protection, scrapebox wont bypass it.

yeah, I noticed...

But in the meantime I solved the problem (manually) as I only had 135 URLs to go through, and it took me like 40 minutes. But I was hoping I would get the formula to use in the future. Oh well, we can't have everything in life, and for free too

Thanks for the tip!

 

[IM Dude]

Ask chatgpt it's good with regex(regular expressions) last time I used. You can upload the webpage too.

 

[tazarbm]

Ask chatgpt it's good with regex(regular expressions) last time I used. You can upload the webpage too.

I'm not touching that thing...

And it wouldn't work anyway, because SB can't bypass Cloudflare as both @satyr85 and I noticed.

But thanks for the tip

 

[satyr85]

But I was hoping I would get the formula to use it for the future...

Code may change in the future and formula wont work.
Quick&dirty regex right now is:

(?<="last_active":").*?(?=",")

Feel free to tag me if you need programming related help in the future

 

[tazarbm]

Feel free to tag me if you need programming related help in the future

cool, thanks

Code may change in the future and formula wont work.
Quick&dirty regex right now is:

(?<="last_active":").*?(?=",")

see? That's why I asked for help, just look at this alien crap OMG, LOOK AT IT! Who in their right mind would be able to remember these hieroglyphs??

I mean, people with brains would remember them, of course! But me? I see this thing and I instantly feel like going out to play ping-pong or something...

Anyway, saved the code for the future because I sure as hell won't remember this abomination, much less be able to come up with it on my own...

 

[satyr85]

cool, thanks


see? That's why I asked for help, just look at this alien crap OMG, LOOK AT IT! Who in their right mind would be able to remember these hieroglyphs??

I mean, people with brains would remember them, of course! But me? I see this thing and I instantly feel like going out to play ping-pong or something...

Anyway, saved the code for the future because I sure as hell won't remember this abomination, much less be able to come up with it on my own...

Its actually quite simple, let me explain

(?<=exact_phrase_right_before_content_you_are_looking_for).*?(?=exact_phrase_right_after_content_you_are_looking_for)

For reference - part of content we are parsing:

,"last_active":"1 hour ago","

Play with this yourself and you will be able to do basic regex on your own

 

[tazarbm]

Play with this yourself and you will be able to do basic regex on your own

I mean, I already started a basic Regex tuturial on this site a few hours ago, but I only got to step 7 because other stuff came in the way so I had to stop:

https://www.regexone.com/lesson/kleene_operators?

It's fascinating, not gonna lie. But only when you learn it from the beginning, if you slam me right in the face with that thing I feel like finding something better to do with my life

Anyway, slow and steady wins the race, so I will probably get to learn Regex at some point. But now I have other stuff to tend to that's more important than these hieroglyphs.

But thanks for trying to explain, today I got forcefully learn'd

 

[xcodes]

However, in the mass scrap process, this works better with minimal load on the scrap process.

(\d+ hours ago)

Good luck

 

[tazarbm]

However, in the mass scrap process, this works better with minimal load on the scrap process.

(\d+ hours ago)

Good luck

thanks! I'll try both of your code by today's end and will let you guys know how it turned out

 

[Uepsa]

If you ever need to test regex on a string while you're building it. This is what I always used to use, just helps visualise the process
regex101.com

 

[Thomps Martin]

Hi,

I am trying to extract data from a vendor's profile on Warriorplus using Scrapebox, and there are 2 options in the Custom Data Grabber function of Scrapebox for that:

- extract by using HTML (before|after) code
- extract using Regex

I can't use HTML option because the data that I need doesn't show up in the source code of the profile's page, and I can't use Regex because I don't know it...

I was wondering, can one of you, code wizards (who's good with Regex) provide the correct formula for extracting the data for the "Last Active: " field (see the red circle below):

View attachment 476197

Try this in Scrapebox Custom Data Grabber → Regex:
(?i)Last\s*Active:\s*([^<\r\n]+)
If the profile is loaded via XHR/JSON, use:
(?i)"last[_\- ]?active"\s*:\s*"([^"]+)"
Works for most cases — post a snippet if you want me to tweak it.

 

[loopline]

If its a non cloud flare protected site its really easy. Just feed the desired text, along ideally with a screenshot of the page, and the html of the page, or at least the section involved to chat gpt and it will give you the regex.

also tell chat gpt (or your preferred AI) that scraepbox uses PCRE, which is Perl Compatible Regular Expressions. Any regex should have the leading ^ and ending $ removed.

It will give you a spot on regex 99% of the time and if it doesnt work go for round 2 and you will likely get it. I don't write them by hand any more, AI does it in seconds and with better accuracy.

 

[tazarbm]

yeah scrapebox wont really grab it clean, warriorplus hides a lot behind scripts. but if you wanna try anyway, a quick & kinda dirty regex for last active is:

Last Active:\s*([^<]+)

it might break if they change the page tho, so don’t rely on it too much.

at the moment I'm not interested in this, but I will keep the formula in mind as I will get to it eventually

Thanks!