Instagram Threads Scraper

[j3ffb3z0s]

Hi,
does anyone have any general idea on how to scrape Threads? Will I have to use a selenium script and a chrome browser, does headless work and so on? Would appreciate any help. Currently I am paying for a Rapid API module but he just upped the price from 100$ to 300$ per month and I'm not gonna pay since it's cheaper to make my own.

 

[Groot Agency]

Hi,
does anyone have any general idea on how to scrape Threads? Will I have to use a selenium script and a chrome browser, does headless work and so on? Would appreciate any help. Currently I am paying for a Rapid API module but he just upped the price from 100$ to 300$ per month and I'm not gonna pay since it's cheaper to make my own.

To scrape Threads, you don't necessarily need Selenium and Chrome. While Selenium in headless mode works, it can be slow and blocked by anti-bot measures. If an API is available or you can reverse-engineer the requests, that might be a more efficient option. If not, Selenium is a viable fallback, but consider the cost of maintenance and potential blocking.

 

[MR.Spuf]

Hi,
does anyone have any general idea on how to scrape Threads? Will I have to use a selenium script and a chrome browser, does headless work and so on? Would appreciate any help. Currently I am paying for a Rapid API module but he just upped the price from 100$ to 300$ per month and I'm not gonna pay since it's cheaper to make my own.

Apify is pretty good to find what you're looking for. They have free plan, monthly more than enough if you don't want to scrape large amounts. They have web app as API as well if you want to build an app.

Not affiliated tho, good luck

 

[Deleted member 173106]

What exactly you want to fetch? images, photos or something else?

 

[stephen1701]

@j3ffb3z0s You can try intercepting the endpoint using Scrapy or another method, though it might not work due to anti-bot technology. Alternatively, you could use Playwright or Selenium, which can yield better results with IP rotation.

If you would rather not deal with it, feel free to ask us. We can build a fully cloud-based solution for you that handles regular scraping.

 

[Proxy Cake]

You'll need an undetectable version of selenium as well as residential proxies, it should get you most of the way there

 

[Manual Growth]

Check Apify, it has everything that you need

 

[kingbotss]

I think, you will need app API, accounts to scrape. I think someone has threads API python

 

[j3ffb3z0s]

UPDATE: I created a scraper using python, selenium, BitBrowser a mobile proxy that rotates by a request and 10 Threads profiles. We are tracking 40 Thread accounts and I want to see how their posts perform constantly

 

[kingbotss]

Can't you make using app API, it would be faster

 

[charman]

Selenium can work, but bot detection might be an issue proxies and session management could help.

 

[kingbotss]

It would be slow. What if you want to track 1k profiles