Is it possible to use scrapebox to copy a large website?

Site has billions of pages and captcha protection so I'd like to use a bundle of proxies and 3rd party captcha integration like that with scrapebox. Hoever, I've yet to locate an option in the program for this.

seochris96 said:

Site has billions of pages and captcha protection so I'd like to use a bundle of proxies and 3rd party captcha integration like that with scrapebox. Hoever, I've yet to locate an option in the program for this.

Click to expand...

Can you be more specific about "copy" a website. If you want to crawl it for data or links, then possibly, depending on how the captchas work.

But if you mean just download the entire pages, then htttrack is a free program that does this, although again the captchas may pose an issue depending on how they work. you can google htttrack

I've been using httrack for years, shit, decades actually tbh. However, as httrack has no captcha solving integration, it won't work for this project. From each url on the site, I need the name of the url, and all of the text and phone numbers on each url, which is only around 400 characters or so per page.

seochris96 said:

I've been using httrack for years, shit, decades actually tbh. However, as httrack has no captcha solving integration, it won't work for this project. From each url on the site, I need the name of the url, and all of the text and phone numbers on each url, which is only around 400 characters or so per page.

Click to expand...

Scrapebox has no captcha solving for anything except posting.

So if you have to solve a captcha just to view the page then it wont work. If you can use a bucketload of proxies and then go slow enough you avoid the captcha, then scrapebox can probably work. You could try the custom data scraper in the email scraper plugin, however you have to be able to publicly view the data you want without having to be logged in and with javascript off. So you can test that in a browser and see if you can still see what you want.