Twitter Web Scraper?

ringtail

Registered Member
Joined
Dec 29, 2013
Messages
60
Reaction score
20
Hey guys,

I'm looking for a different kind of twitter scraper, would any of you be able to point me in the right direction?

Needed - Scrape the Name and Username of a user based on a search without having to have an account logged in or API credentials. Proxy rotation/decaptcha modules are a plus (just incase).

Ex. Scrape all Scott's and their twitter usernames from a url - https://twitter.com/search?q=scott&src=typd&mode=users

I want to be able to feed the URL so I can change the q='s param

Let me know if you know where to find one. All I can find are bots and other scrapers that require logging in or API credentials and that wont work due to high volume.

Thanks BHW!
 
If you are ready to spend some $$ PM me
 
If you have a budget I could create a custom bot for you, PM me with your skype details.
If you're looking for a free solution I actually don't know. There's lots of twitter bots out there and maybe some premium bots have this feature, but I doubt any free one do.
 
in your chrome install scraper plugin.

then enter the xpath of the elements you want to extract on a particular page.

Since twitter uses dom to load results , to scrape more scroll to bottom of the page before using the scraper plugin.
 
Thanks for the responses guys. @DIVOK: thanks dude, I looked into the extensions but they didn't seem to suit my need for this project.

Again, not looking for a bot (I have 'em) or a paid solution (I can hire a web scraper/data extractor off odesk). I ended up going with a one off solution using import.io. It's slow as FFFF but may get the job done as a proof of concept. If anyone else knows any other solutions, please let me know.
 
Back
Top