Instagram scraper with proxies

[Thibault]

I've been scraping IG biographies for a couple of years, looking for emails when projects I was working on needed some emails lists.
Recent IG update makes it much slower to get a decent amount of emails.

I'm looking for a tool that would work with proxies so I can get back to normal speed, and scrape 2-5 millions profiles per day (vs. 150k per day with one account/IP)?

I've used risto tool, his instagram scraper, it's great but the proxy part doesn't work.

 

[FatBee]

Not gonna happen. iG being strict this days.

 

[perrymason]

you're gonna need to find excellent proxies and that ain't easy with IG nowadays.

 

[Thibault]

I have 4G proxies of high quality, but which solution/tool would work with it?

 

[arcangel]

why do you need so many, like millions per day? Maybe scrape box has a IG scrape module

 

[Thibault]

why do you need so many, like millions per day? Maybe scrape box has a IG scrape module

Because with current limits, once you've scrapped and cleaned emails, you don't have enough to create impactful marketing campaign.
I'm going to check Scrapebox, thanks

 

[webguy2000]

millions accounts per day per 1 IP? We have been struggling to do more than 1 request per minute without getting blocked after 150 requests. They are SUPER strict now.

 

[arcangel]

Because with current limits, once you've scrapped and cleaned emails, you don't have enough to create impactful marketing campaign.
I'm going to check Scrapebox, thanks

What metrics are you looking for when you scrape?

 

[King_zyzz]

I don't think there isnt anything better on the public market then what your using atm

 

[Thibault]

millions accounts per day per 1 IP? We have been struggling to do more than 1 request per minute without getting blocked after 150 requests. They are SUPER strict now.

I'm currently scraping ~16k accounts per hour per IP. I would like to use proxies to duplicate this and scrape 10x more.

What metrics are you looking for when you scrape?

I would like to scrape 200k+ accounts per hour using proxies.

 

[Deleted member 15724]

I'm currently scraping ~16k accounts per hour per IP. I would like to use proxies to duplicate this and scrape 10x more.

I would like to scrape 200k+ accounts per hour using proxies.

Are you using rotating proxies? I dont think 16k/hour would be possible even 2 years ago, and especially now after their last update to combat scraping.

If you have rotating proxies, order a larger package and try scrape box. 200k/hr should be possible hypothetically.

 

[ManiB]

I'm currently scraping ~16k accounts per hour per IP. I would like to use proxies to duplicate this and scrape 10x more.

That's a huge number. i tried scraping with my own mobile net and got blocked after a few thousand scrapes. You might wanna try Instadub. they have a good scraper and you can divide the list to several parts and allot different accounts and proxies to give you results.
Mind telling us which country based 4g proxies are you using?

 

[arcangel]

Are you using rotating proxies? I dont think 16k/hour would be possible even 2 years ago, and especially now after their last update to combat scraping.

If you have rotating proxies, order a larger package and try scrape box. 200k/hr should be possible hypothetically.

How do you use scrapebox to scrape these types of accounts?

 

[turelink]

I think you have to use the rotating residential proxies for those type of scraping.
Some providers offer the mobile IPs like luminati

 

[aleia]

For that amount of scrape I think you would need a fabric on proxies lol

 

[TwitterPanel]

[QUOTE = "perrymason, post: 12489732, üye: 1190913"] mükemmel proxy'ler bulmanız gerekecek ve günümüzde IG ile kolay değil. [/ QUOTE]
Kesinlikle sana katılıyorum.

 

[liyong601c]

Pick an rotating residential IP provider！

 

[Thibault]

That's a huge number. i tried scraping with my own mobile net and got blocked after a few thousand scrapes. You might wanna try Instadub. they have a good scraper and you can divide the list to several parts and allot different accounts and proxies to give you results.
Mind telling us which country based 4g proxies are you using?

France

How do you use scrapebox to scrape these types of accounts?

Good Question

Are you using rotating proxies? I dont think 16k/hour would be possible even 2 years ago, and especially now after their last update to combat scraping.

If you have rotating proxies, order a larger package and try scrape box. 200k/hr should be possible hypothetically.

You're right, I've done more tests and it appears I'm doing less effectively : 5k/hour/IP

For that amount of scrape I think you would need a fabric on proxies lol

2 millions per day would be 20/25 accounts & IPs on my end, so not that much!

Pick an rotating residential IP provider！

I've tried and it's actually better, thanks!


I think the issue I'm having is my own internet connection get throttled for doing too many HTTP requests, via my own IP or some proxies.

 

[rootjazz]

As suggested above to scrape 2million profiles a day you need to use a RESIDENTIAL / reverse / backconnect package where you access a pool of proxies.

You would be wanting to scrape from the website not the mobile API in order to not require IG accounts. However, you can only pull 200 or less profile pages per IP in a time period before IG blocks you and you need to log in to view the profile pages.

Based way to do it is, use your mobile API bot to scrape the results, then feed those results into your web page email scraper module

 

[rootjazz]

I'm currently scraping ~16k accounts per hour per IP. I would like to use proxies to duplicate this and scrape 10x more.

No you are not. It is NOT possible to scrape 16k profiles in an hour from a single IP. Probably you don't realise it, but the request is being redirected to the IG login page instead of the profile page. Check the logs of the program you are using which should output the final response url and status during the request and it will confirm what I say