I guess I would need to know what discovery method your tool plans to use. Does it crawl looking for 404s, or use some other method for discovery? The biggest issue with Scrapebox is that because it wasn't designed for this specific purpose, you're constantly having to export URLs, and import them into various addons. If I were to buy a tool as you are making, these are some of the things I would expect it to have/support:
- A consistent interface. I don't want buttons and options jumping all over the place etc. Just place things consistently.
- Proxy support (including importing proxies from file and clipboard)
- The option of single thread and multi-thread.
- Project support (so you can switch between projects)
- The ability to handle a lot of data without freezing like crazy.
- Multiple discovery options would be cool (Scraping for URLs, and crawling seed URLs)
- If scraping, then multiple engine selection (although you do have to consider how much work supporting scraping search engines would be for you)
- If crawling is an option, then also the crawl depth.
- If crawling is an option, then control over which HTTP responses are considered "dead"
- Metrics checking of some variety (this has gotten hard since all of the free metrics are basically disappearing)
- Filtering data based on keywords (including providing individual keywords and filtering against a file loaded with keywords)
That's all I can think of off the top of my head.
