1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Proxy Crawler Ideas

Discussion in 'Proxies' started by civilwar, Apr 12, 2012.

  1. civilwar

    civilwar Newbie

    Joined:
    Apr 12, 2012
    Messages:
    1
    Likes Received:
    0
    What kind of features would you like to see in a proxy crawler? I am writing a small one for a project. So far I have settings for websites to crawl, keywords to search google,yahoo,bing regex to find ip port. What other kind of things should I try? Also anyone have a good website/keyword list?