lovinboi
Junior Member
- Jun 2, 2011
- 156
- 114
Inspired by a thread posted by majdi107
Sadly he couldn't get the domain content copied, so I did a "lil googlin" and came up with this
https://github.com/hartator/wayback-machine-downloader
To make this work:
Step 1: Download latest ruby installer with devkit
Check this from Ruby website, google it.
Step 2: Install Wayback Machine Downloader
Run CMD with Ruby and paste this "gem install wayback_machine_downloader"
Done.
Basic Command to download snapshots
wayback_machine_downloader http://example.com
More optional commands:
Directory to save the downloaded files into Default is ./websites/ plus the domain name
-d, --directory PATH
Download all snapshots/timestamps for a given website
-s, --all-timestamps
Only files on or after timestamp supplied (ie. 20060716231334)
-f, --from TIMESTAMP
Only files on or before timestamp supplied (ie. 20100916231334)
-t, --to TIMESTAMP
Download only the url provied and not the full site
-e, --exact-url
Restrict downloading to urls that match this filter (use // notation for the filter to be treated as a regex)
-o, --only ONLY_FILTER
Skip downloading of urls that match this filter (use // notation for the filter to be treated as a regex)
-x, --exclude EXCLUDE_FILTER
Expand downloading to error files (40x and 50x) and redirections (30x)
-a, --all
Number of multiple files to dowload at a time. Default is one file at a time (ie. 20)
-c, --concurrency NUMBER
Maximum snapshot pages to consider (Default is 100). Count an average of 150,000 snapshots per page
-p, --maximum-snapshot NUMBER
Only list file urls in a JSON format with the archived timestamps, won't download anything
-l, --list
I used "wayback_machine_downloader http://example.com --all-timestamps" to download all timestamps/snapshots for the website I wanted to copy
Sadly he couldn't get the domain content copied, so I did a "lil googlin" and came up with this
https://github.com/hartator/wayback-machine-downloader
To make this work:
Step 1: Download latest ruby installer with devkit
Check this from Ruby website, google it.
Step 2: Install Wayback Machine Downloader
Run CMD with Ruby and paste this "gem install wayback_machine_downloader"
Done.
Basic Command to download snapshots
wayback_machine_downloader http://example.com
More optional commands:
Directory to save the downloaded files into Default is ./websites/ plus the domain name
-d, --directory PATH
Download all snapshots/timestamps for a given website
-s, --all-timestamps
Only files on or after timestamp supplied (ie. 20060716231334)
-f, --from TIMESTAMP
Only files on or before timestamp supplied (ie. 20100916231334)
-t, --to TIMESTAMP
Download only the url provied and not the full site
-e, --exact-url
Restrict downloading to urls that match this filter (use // notation for the filter to be treated as a regex)
-o, --only ONLY_FILTER
Skip downloading of urls that match this filter (use // notation for the filter to be treated as a regex)
-x, --exclude EXCLUDE_FILTER
Expand downloading to error files (40x and 50x) and redirections (30x)
-a, --all
Number of multiple files to dowload at a time. Default is one file at a time (ie. 20)
-c, --concurrency NUMBER
Maximum snapshot pages to consider (Default is 100). Count an average of 150,000 snapshots per page
-p, --maximum-snapshot NUMBER
Only list file urls in a JSON format with the archived timestamps, won't download anything
-l, --list
I used "wayback_machine_downloader http://example.com --all-timestamps" to download all timestamps/snapshots for the website I wanted to copy
