Proxy Crawler Ideas

Discussion in 'Proxies' started by civilwar, Apr 12, 2012.

  1. civilwar

    civilwar Newbie

    Joined:
    Apr 12, 2012
    Messages:
    1
    Likes Received:
    0
    What kind of features would you like to see in a proxy crawler? I am writing a small one for a project. So far I have settings for websites to crawl, keywords to search google,yahoo,bing regex to find ip port. What other kind of things should I try? Also anyone have a good website/keyword list?