Twitter Scrapping Risk Usinng Selenium

[FlyingCow]

Hi guys, I'm currently working on a project that requires scraping Twitter data. I'm planning to use a third-party library like Selenium for this task. I wanted to know if it's safe to do so and if there are any risks of getting banned by Twitter?

 

[ReverseEngineering]

Since you plan to scrape only the risk of getting banned is lower, because you don't "spam" X with massive posts.

Selenium needs a lot of resources, why don't you simply do HTTPS requests with a script? You don't need any browser emulation for scraping.

 

[Sickfly]

Just use throw away accounts. Or verify some blue ones for 10 bucks and virtual credit cards. With blues I can scrape massively before getting a temporary restriction which only lasts like a few days

 

[nirose]

why don't you simply do HTTPS requests with a script?

Because their website is javascript heavy where content is loaded by javascript calls and you'd get nothing out of a "request" other than placeholder html text. You'd also have to manage cookies and authentication for that request and Twitter API is expensive af.

 

[BiggusD1ckus]

Just use throw away accounts. Or verify some blue ones for 10 bucks and virtual credit cards. With blues I can scrape massively before getting a temporary restriction which only lasts like a few days

any vcc provider you'd recommend for going blue?

 

[Ashk881]

Selenium needs a lot of resources, why don't you simply do HTTPS requests with a script? You don't need any browser emulation for scraping.

Because Twitter throws captchas randomly in between and it's rendered with Javascript. You can't scrape the content directly with plain http requests unless you are using the client-side api somehow.

 

[ReverseEngineering]

Because Twitter throws captchas randomly in between and it's rendered with Javascript. You can't scrape the content directly with plain http requests unless you are using the client-side api somehow.

Because their website is javascript heavy where content is loaded by javascript calls and you'd get nothing out of a "request" other than placeholder html text. You'd also have to manage cookies and authentication for that request and Twitter API is expensive af.

The trick is to do HTTPS requests based on the private X app API, not the website. Similar to TikTok, Instagram, Snapchat, ...

 

[Sickfly]

any vcc provider you'd recommend for going blue?

Pst dot net, or flexcards dot cards.. but pst is easier

 

[Deleted member 1197157]

The trick is to do HTTPS requests based on the private X app API, not the website. Similar to TikTok, Instagram, Snapchat, ...

Hi, can you DM me on - @floweroflights on Telegram?

 

[Veil123]

Try to use a modified library (e.g. https://github.com/ultrafunkamsterdam/undetected-chromedriver) as even though Twitter does not have any complex detection algorithms, they can detect the standard (unmodified) web driver.

 

[ReverseEngineering]

Hi, can you DM me on - @floweroflights on Telegram?

Sure.