There is a site that I am looking to scrape +100K files from. Accessing the site is free but requires registration.
In order to prevent bots, they put up captchas every 15 or so files.
They also ban IP addresses if the usage looks patterned.
I want the files - not a script to get the files, just the files. Any recommendations.
I was thinking GetAFreelancer would be an option, but how should that be implemented? Have GAF users do them in 1000 file blocks or something?
How much should I expect to pay?
In order to prevent bots, they put up captchas every 15 or so files.
They also ban IP addresses if the usage looks patterned.
I want the files - not a script to get the files, just the files. Any recommendations.
I was thinking GetAFreelancer would be an option, but how should that be implemented? Have GAF users do them in 1000 file blocks or something?
How much should I expect to pay?