ghettogong
Regular Member
- Oct 7, 2009
- 299
- 28
Your members will answer differently as the orginal stolen thread stops where is stopped.
Posted via Mobile Device
You are using an out of date browser. It may not display this or other websites correctly.
You should upgrade or use an alternative browser.
You should upgrade or use an alternative browser.
Stealing a forum
- Thread starter ghettogong
- Start date
Your members will answer differently as the orginal stolen thread stops where is stopped.
Posted via Mobile Device
I did this quite a few years ago in an attempt to jump start an adult forum.
Maaaan it was one helluva challenge, but I still have the full design of the software. Of course it doesn't work anymore and the coder is AWOL. But it was a pain in the ass to get running.
Every forum is formatted differently... plus you had to register new fake nicks every time, get the posts counts to match, etc etc.
Its alot more hassle than you guys think
Maaaan it was one helluva challenge, but I still have the full design of the software. Of course it doesn't work anymore and the coder is AWOL. But it was a pain in the ass to get running.
Every forum is formatted differently... plus you had to register new fake nicks every time, get the posts counts to match, etc etc.
Its alot more hassle than you guys think
BreakAllTheClocks
Regular Member
- Apr 23, 2010
- 421
- 245
There are two parts to this.
1. Scraping the forum is extremely easy. Start with the member list, then section, thread, posts. Store as csv or in sql. Very easy to pipe data into new forum db while maintaining referential integrity.
2. Keeping forum updated/synced. Slightly more difficult, but basically the same scraping script, cron jobbed every 5 minutes on the forum's 'new posts' search results.
There is almost no logic involved here, only when posts by a new user are discovered is there any branching involved.
Also set user agent of scraper to googlebot to stay under the radar.
edit: as far as being ethical/legal etc, I don't see any difference between creating an autoforum and creating an autoblog.
1. Scraping the forum is extremely easy. Start with the member list, then section, thread, posts. Store as csv or in sql. Very easy to pipe data into new forum db while maintaining referential integrity.
2. Keeping forum updated/synced. Slightly more difficult, but basically the same scraping script, cron jobbed every 5 minutes on the forum's 'new posts' search results.
There is almost no logic involved here, only when posts by a new user are discovered is there any branching involved.
Also set user agent of scraper to googlebot to stay under the radar.
edit: as far as being ethical/legal etc, I don't see any difference between creating an autoforum and creating an autoblog.
Last edited: