Scraping twitter followers?

Inspired by another member (phpbuilt) on here I'm about to embark on building my own twitter follower scraper using the twitter API.

I've read the api limits call to something like 15 calls in 15 minutes. Each call return 200 followers. So, in an hour, using only one account I can scrape 12,000 followers. In a day 288k followers.

Let' say I want to run 4 or 5 instances of this php script at the same time ... whats the best way to do that? cron?

Thanks in advance.

Cron is a good way to go. 4 or 5 instances means the same amount of accounts and proxies, since using the same IP for all the accounts will get you banned. I've set my scrapper for 15 calls every 20 mins for obvious reasons.

I'd say running cron jobs would be the way to go with this, also why not use proxy's to run more than 1 account at a time? Here's an example I wrote for you on how to use proxies with PHP CURL, rather simple.


If you plan on using private proxies that require a Username & Password, add these two lines.

You are right about 15 request per 15 minutes, but each request returns not 200 but up to (if the user has fewer followers you will obviously get less) 5000 follower IDs.

So if you do the math in an hour you can get up to 300k followers and in one day up to 7200k so I don't see the point in running multiple instances scrappers.


You may try to run multiple scrappers if you will do user/lookup for those scrapped IDs which I assume you will. In this case, you can get information for 24k to 72k users in an hour.