1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Need to get all files served by a site, served under various PHP scripts.

Discussion in 'Web Design' started by mark.avenger, Jun 30, 2013.

  1. mark.avenger

    mark.avenger Newbie

    Joined:
    Oct 29, 2012
    Messages:
    2
    Likes Received:
    0
    Hi guys.

    I need to download all files from a website. The problem is that they use several GET request to different PHP scripts to get them.

    First, it sends something like this to get to the file's download page: GET /download/315532/0 HTTP/1.1

    Then, on the send page, you need to click on download again and you send this: GET /download.php?file=FILE_NAME.ZIP&646965504532567d5954262e5a=1 HTTP/1.1

    Lastly, on the third page, you send this via GET: DOMAIN-NAME/download.php?file=FILE_NAME&646965504532567d5954262e5a=1

    The question is which software can crawl to the whole site and download all .zip/.rar files?
    I've tried several web spiders, none seem to work.