Any tips on my first article scraper?

My first time writing one. I was just wondering if there were any, I wish I'd have done that first like ideas which may save me time.

To give back a little here's what I'm doing

Grabbing article rss
Creating my own copy
Grabbing and editing articles
Putting my ads in them
creating my own rss feed
dripping it to my forum


That's about it. Any tips?

Rick

Hi ComputerHelp808,

If you're using PHP, take a look at PHP's DOM. This will let you use the scraped as an object, and you can extract whatever info you want more easily.

Depending on your comfort level with SQL - you can dump your edited/spun articles right into your forum's database, automating the whole process.

If you have more details I can provide more help.

Best of luck.

instead phpdom i reccommned use symfony dom crawler compoonent, it more usefull

Make sure you have at least a few proxies to avoid errors and IP bans later from scraping.

When they index all get ban from dup info.

There already a app designed for this and it works perfect .

But I'll like to see your app in action .

I am a ex PHP programmer like to see what you come up with .