rootjazz
Elite Member
- Dec 21, 2012
- 3,166
- 1,190
You can definitely do more than 200 per hour and be safe.Hi, I would like to figure out if you would be interested in an Instagram scraper with this features:
My idea is to create a scraper which scrapes the followers of an specific account. To do this, the scraper would require a list of IG accounts (ID and pass) due to each account would scrape 200 followers profiles max per hour to be safe and avoid ban for a lot of requests using the unofficial IG API
Disagree there, maybe you can do that for 1 minute, but you do 200 a minute, for every minute, you WILL have issues(each account can scrape 200 profiles per minute).
But sharing the requests is the right way to go about thisSo, using a rotation list of 60 accounts, in an hour the scraper would get info from 12.000 profiles (288.000 in a day if the profile has over that amount of followers, or if I add a profile rotator when finish with one).
A VPN is not the best option as it limits to one account per time as you will need to reset the VPN often, use proxies which are designed to be used at the application level, routing each request through a different IP at the same timeIn addition to that, if the scraper use a VPN, it would be able to scrape many profiles at the same time using a single own account (now, multiply the result 60 times if I had 60 accounts to run the scraper). That because the limit of 200 requests is for each instance.
The output, only for public profiles, would be a CSV file with this fields:
- User ID and URL
- Email*
- Phone number*
- Followers number
- Follows number
- Post numbers
- Description
* Email and phone numbers only if it’s available.
don't limit it, just convert the IG JSON response to CSV then you can output EVERYTHING IG has on the account (this is how the pro scrapers do it at the moment
Honestly, you would need to see it very cheap as programs already exist to do exactly what you specify and a lot LOT more.I would want to know if you like a software like this to develop one. ¿What price would you recommend? ¿What kind of software would you prefer, an .exe or a webapp? ¿Have you other interesting ideas?
Thank you for your support!
Why limit it to just followers scrapes?
Why limit the results?
If you want to do something that people want, try and do something with locations, working out the location of an account (IG doesn't give it to you). That is probably something you could sell and I haven't seen any product do that well yet
Regards,
Martin