1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Download images from URL list with proxy?

Discussion in 'BlackHat Lounge' started by Pwnageboy, Dec 28, 2014.

  1. Pwnageboy

    Pwnageboy Regular Member

    Joined:
    Nov 9, 2014
    Messages:
    206
    Likes Received:
    54
    Location:
    The Netherlands
    Hi yall,

    I am scraping a website atm for image links and put them into a txt file.

    The images are direct links to the images so no steps in between.

    Now, even when i'm scraping the images i get blocked by my ISP for a while after scraping about 1500 pages ( complete internet disconnect until i reset my router ).

    Fun fact.. every page has 8 images, am scraping 28K+ pages.

    Conclusion: I need to download approx 174.000 images from that website.

    As far as i know i can not do this without any proxy since my ISP cut me down with the scraping itself already.

    Does any one have a idea how to download those images with proxy support? Or is there any software that can do this task?

    Again i really need to use proxies. Software to download the images is a easy task to create but i have no experience in coding proxies to use.

    Thanks in advance!