What tool to use to scrape multiple forum threads?

[Billy Batts]

Marketing, SEO, eCom and design I have become to know quite good however coding and such tools I'm a noob.

My sub on a paid forum will expire in a week so I want to rip a bunch of threads for reading at later times. Now I used to know how I did it but I forgot and can't find the tool either anymore.

I tried HTTrack but when I ripped a thread it still asks for the forum login.

So I need to rip multiple forum threads with the complete CSS and HTML.

Suggestions?

 

[teekayo2]

I didn't know a tool.
I would write my own screenscraper with php for that.
If you need a fast soultion, just save the whole website as html.
This should work. A while I'm writing this, I do know a tool which exacly does this, savong webpages, but I don't remember its name. You can manually do this.
Or screenshot it

 

[Deleted member 657006]

not tried it but here
https://github.com/ZiCog/parallax-scrape

 

[Billy Batts]

I didn't know a tool.
I would write my own screenscraper with php for that.
If you need a fast soultion, just save the whole website as html.
This should work. A while I'm writing this, I do know a tool which exacly does this, savong webpages, but I don't remember its name. You can manually do this.
Or screenshot it

I guess I will be doing it manually again

not tried it but here
https://github.com/ZiCog/parallax-scrape

Thanks for thinking with but that tool is written for a specific forum and transfers the page to a text file.

NP, I'll do it manually.

 

[Deleted member 657006]

might help aswell
https://stackoverflow.com/questions...at-are-hundreds-of-pages-deep-w-beautifulsoup

 

[Topiano]

You could really set ScrapeBox to work ...

 

[mhtro]

Do you mean something like "supagrowth Forum Traffic Generator"?

 

[teekayo2]

You may want to have a look in the book: Webbots, Spiders and Screenscrapers.
You could be able to program useable stuff in a few days with 0 knowledge. Its worse the time.

 

[mnunes532]

You would need python (or other programming language) along with selenium.

Go to bhw, login, then go to the specific sections and save them into a database or something like that.

 

[JustChillin]

Why not just save the page(s) with Chrome's built-in feature?

 

[SEO Supreme]

Marketing, SEO, eCom and design I have become to know quite good however coding and such tools I'm a noob.

My sub on a paid forum will expire in a week so I want to rip a bunch of threads for reading at later times. Now I used to know how I did it but I forgot and can't find the tool either anymore.

I tried HTTrack but when I ripped a thread it still asks for the forum login.

So I need to rip multiple forum threads with the complete CSS and HTML.

Suggestions?

You can use a custom PHP script that sends cookie data when fetching the data.

 

[teekayo2]

You can use a custom PHP script that sends cookie data when fetching the data.

For what exacly is this good?

 

[amoon]

i think you reviewed outscrape, can this tool help you with that...

 

[HoNeYBiRD]

iMacros and save as html? If you know the thread titles/urls and the page urls of the multipage threads have some kind of pattern (e.g incrementing numbers), you can loop through them with the macro, so it saves them one-by-one.

 

[seoxz]

Can you provide the name to the forum may be I can help

 

[thetrustedzone]

i code my own ....

 

[SpoonFeeder]

@Billy Batts Mate, PM me the forum url. I'll build you a custom scraper for FREE. Hope you have a couple of days on the subscription so we can rip almost every thread!

 

[SEO Supreme]

For what exacly is this good?

Basically for anything that doesn't require client-side content scraping. There are ways for that, but it's a more complex process...