Hi, Currently, I work on a webcrawler tool made in C#(a part of my big project of SEO tools), yes I prefer make my tool, it's funnier. Properties: - crawl on 1 or more web sites. - save datas in SQLite database, for nice SQL requests. - statistics and datas panel - user can save urls, mails, images, forms, a specific markup or a custom harshvest with regex. - analyze forms for automatic submit by user - save metadata - proxy switch - timeout - threads - export datas in XML, CSV, plain text, HTML. Do you have any suggestions? I want to improve it. Web crawlers has been used always by black hat SEO guys now ?