Please help me with this twitter scraper script

Hey everyone,

So I found this script from like 2011 or something, which uses a Google doc spreadsheet with script to scrape the followers or following of a Twitter account.

So far I have been able to get it to scrap 5,000 users, but it will not go past that, and when I run it again, it just scrapes the same 5,000 again.

I have a client who is following 40,000 people and he would like his whitelist (people he never wants to unfollow) to be made up of only verified users, and this scraper has a column that shows if the user is verified.

Does anyone know of any code I can add to the script so that when it runs the second time, it starts up where it left off?

Here are the original links about the script:

https://mashe.hawksey.info/2011/03/export-twitter-followers/

https://mashe.hawksey.info/2011/08/export-more-twitter-followers/#comment-12920

Here is the link with the added script to try to get the scraping to continue without stopping (I'm not even sure if this is accomplishing anything, since Twitter is just doing the 5000 limit)

https://gist.github.com/mhawksey/1121340

Also, when I use the Google doc spreadsheet, when I am on the script editing page, and I click "Run", it comes up with this error:

ScriptProperties API is deprecated.
File: Code Line: 501
The API has been marked as deprecated which means that the feature should be avoided and may be removed in the future. Consider using an alternative solution.

This is line 501: if (!ScriptProperties.getProperty("cursor")){

But when I am in the spreadsheet and I use the spreadsheet's function to scrape the "friends" of the account, it will give me the first 5,000 people each time.

Thanks to anyone who can help!

You are probably hitting the Twitter API limits, take a look at them here -> https://dev.twitter.com/rest/public/rate-limits

Are you a developer ? If you are, take a look here http://stackoverflow.com/questions/...followers-from-a-twitter-account-with-node-js
If you are not a developer, have you considered paying somebody to have this done?

fidodido said:

You are probably hitting the Twitter API limits, take a look at them here -> https://dev.twitter.com/rest/public/rate-limits

Are you a developer ? If you are, take a look here http://stackoverflow.com/questions/...followers-from-a-twitter-account-with-node-js
If you are not a developer, have you considered paying somebody to have this done?

Click to expand...

Thanks fidodido. I'm not a developer, but I did have to create the twitter app to make this work. I don't understand programming/scripts, etc, so I can't fix this to make it work right. I believe it worked properly in 2011 when the guy made it.

I do understand twitter's API limits, but I was just wondering if anyone knew how I could edit the script so that when I ran it again, it would start back up where I left off once I hit the limit and waited, instead of it just doing the first 5,000 each time. I was hoping I could get the first 5,000, then wait and run it again to get 5,001-10,000, and so on...

So anyone on here know how to program scripts for Twitter scraping?