To Scrape A Forum

[hkhkhkhkhk123]

Hi BHW People.

I have a forum to crawl and an available VPS to use for that purpose. Any suggestion on that? I know little python, crawler, scrappy, and stuff like that.

Thanks.

 

[stallion45d]

Hi BHW People.

I have a forum to crawl and an available VPS to use for that purpose. Any suggestion on that? I know little python, crawler, scrappy, and stuff like that.

Thanks.

Scrapebox can do that for you. If you have a licence , It works like a charm

 

[hkhkhkhkhk123]

Scrapebox can do that for you. If you have a licence , It works like a charm

I am not after those links on the forum. Instead, I'm interested in the content on the forum. Can Scrapebox do that?

 

[Rollfic]

While we are on the subject, how do you scrape member's email from a forum?

 

[uncutu]

httrack
webcopy
wget if youre confident on command line.
the other two are freeware/gui tools very easy to use. i've used them both to scrape forums and misc websites in the past successfully.

 

[uncutu]

While we are on the subject, how do you scrape member's email from a forum?

are emails public? scrapebox has functionality to do that. alternatively learn some python scraper or these days i've seen freemium SaaS scrapers that are really cool.

i would first identify a URL that has all the members https://www.blackhatworld.com/online/?type=member
and tell scrapebox to find every url containing blackhatworld.com/members
once u get hefty list, there's a feature to extract all emails found in those links in scrapebox.

 

[macdonjo3]

You ever tried Fiverr? Could get this done for super cheap.

 

[jmarc69]

zennoposter

 

[hkhkhkhkhk123]

You ever tried Fiverr? Could get this done for super cheap.i

You ever tried Fiverr? Could get this done for super cheap.

nah.. not a big fan of Fiverr though, a waste of time and money..

 

[macdonjo3]

nah.. not a big fan of Fiverr though, a waste of time and money..

"A waste of time" you say meanwhile you're going to code this yourself. The irony.

You should brush up on your outsourcing skills.

 

[jmarc69]

I repeat it but I think zennoposter is perfect to do the job (and it is true that sometimes it is better to delegate)

 

[MasonF]

A little from everything won't help you, especially if that forum has a strong antibot system. If not, scrappy will do