Download images from URL list with proxy?

Discussion in 'BlackHat Lounge' started by Pwnageboy, Dec 28, 2014.

  1. Pwnageboy

    Pwnageboy Regular Member

    Joined:
    Nov 9, 2014
    Messages:
    258
    Likes Received:
    78
    Gender:
    Male
    Location:
    The Netherlands
    Hi yall,

    I am scraping a website atm for image links and put them into a txt file.

    The images are direct links to the images so no steps in between.

    Now, even when i'm scraping the images i get blocked by my ISP for a while after scraping about 1500 pages ( complete internet disconnect until i reset my router ).

    Fun fact.. every page has 8 images, am scraping 28K+ pages.

    Conclusion: I need to download approx 174.000 images from that website.

    As far as i know i can not do this without any proxy since my ISP cut me down with the scraping itself already.

    Does any one have a idea how to download those images with proxy support? Or is there any software that can do this task?

    Again i really need to use proxies. Software to download the images is a easy task to create but i have no experience in coding proxies to use.

    Thanks in advance!