Is there any way (or tool) to scrape threads from forum and post in another?

[gorgonite]

Hi guys..

my title says pretty much what I'm trying to find out..

I saw a tool once that scraped a forum, but everythin in a .sql file and then you just had to dump it in yoir db, getting all the posts, usernames.. everything...

is there a "free" tool out there?

 

[scriptkiddies]

mysql
phpmyadmin !

 

[gorgonite]

mysql
phpmyadmin !

yes, but how do I get the dump in the first place in order to import it in my db with phpmyadmin?

 

[195471]

is there a "free" tool out there?

Yeah, the "free" tool is one that you code yourself.

To my knowledge, there's no tool that can scrape any forum. Scraping a web site requires that you have access to the site's HTML code (which you can get by viewing the source). The scraper has to be coded to look for certain parts of the target site's HTML code, or else it won't work. If and when the target site updates its HTML, the scraper will also need to be updated.

If you want to get into scraping, have a look at this book:

http://nostarch.com/webbots.htm

You might also want to check out the services of this company:

http://www.screen-scraper.com/

 

[bbyte]

Here's the eBook that's mentioned in the previous post, a great read if you're into stuff like this:

http://www.mediafire.com/download.php?ghdxmbmmzym

 

[gorgonite]

Here's the eBook that's mentioned in the previous post, a great read if you're into stuff like this:

http://www.mediafire.com/download.php?ghdxmbmmzym

thanks buddy.. downloading now...

 

[wingchun3]

i have made a tool that scrapes forum posts and then adds to my forum

i used many of the libs form that book, check out the website for the book and all the tools you need to build you scrper from are there

also you will need to google a few examples of loggin into a site, extracting certian post keys and then using these in your curl script

the basic process is

first go around registering accounts on various forums (most use capthca)

make a little function to

1: login into the forum with your username - password
2: extract any post keys or session ids
3: then post up your new topic ( or comment on existing threads)


take alook at the auto form commenter i posted up in the tools sction, this just coments to random threads in vb forums

you can very easily modify this to post new threads in stead of commenting on existing threads

the problem is tho if your spamming rndom crap, its easy to hide in exisitng posts, instead of starting your own new threads

i'm just working on my forum poster now, making more funtcion s to post to other forums including vb, phpbb3 and smf

if you make any more scipts or addons post them up here for sharing or hit me up to share some of my tools

 

[oqsales]

I'd like to pay someone to scrape and populate my SMF
forum with posts related to my websites topic, scraped from similar forums. Posts should be auto-rewritten.

If this works out well I have many other sties I'd like to do this for.

 

[Tirade]

If you want to get into scraping, have a look at this book:

hxxp://nostarch.c0m/webbots.htm

I realize that this is a very old thread, but I'd really appreciate if someone could please upload this ebook (the mediafire link is obviously down after all of this time). I've already tried using the search function and haven't been able to find this anywhere.

 

[195471]

Here you go:

http://www.mediafire.com/?6mece08nshn6e6z

 

[195471]

double post