Free Twitter scraper

V

vinclancy

Regular Member
Joined
Jul 6, 2013
Messages
243
Reaction score
156
Hey all,

I've been pretty fed up with scrapers out there, so I'm just building my own. Right now it scrapes usernames or tweets from a recent keyword search, as many as you'd like at a time; I'll be adding more to this same link as I go. I plan on it being the best scraper out there, and I'll always keep it free.

Here it is: https://github.com/austenallred/twitter-scraper
 
Advertise on BHW
Keep this bot updated, in which programming language was writen?
 
UPDATE: I just added a tweet scraper for unlimited usernames and unlimited number of tweets (so long as you stay within Twitter's rate limits).

And it is written in ruby.
 
Do you know the limits for scraping users through the api? Nice share mate
 
jamie3000 said:
Do you know the limits for scraping users through the api? Nice share mate
Click to expand...
Just google it, it should be one of the top results. I can't remember right now but I think you can make 200 calls every 15mins. I'm not sure though.
 
Fantastic work, hope you keep this tool updated :)
 
I've set up a cron job to run it every 30 minutes. I automaticlaly scrape 22,500 usernames every half hour, and haven't had a problem
 
Any other features you guys would like?
 
How do I use this??? not too sure what I should be doing
 
Download the files, make sure you have Ruby installed, and follow the instructions in the readme. Add me on skype if you get lost.
 
lol OP. the newbies may need an .exe or this is going to build into a massive thread of confused noobs.

CyberAlien said:
Is this script scraping the data or using Twitter's API?
Click to expand...

CyberAlien, if you bothered to open the source or even spend 20 seconds looking at it, you can see it uses the API.

config.consumer_key = "YOUR_CONSUMER_KEY"
config.consumer_secret = "YOUR_CONSUMER_SECRET"
config.access_token = "YOUR_ACCESS_TOKEN"
config.access_token_secret = "YOUR_ACCESS_SECRET"
 
:) Follow the instructions in the readme or add me on skype and I can explain it
 
Thanks for the share! I also took a look at the book you're writing, can't wait to see it :cool:
 
skype id - austenallred (as you can see by clicking the skype button under every comment i post)
 
Also, I just added a simple follower file and function to the bot; soon it should be the only twitter bot you need.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top