Instagram Scraper

Hello Guys,

I skimmed a book about scraping and decided to make my own instagram scraper using selenium and an old instagram account.

The bot was supposed to:

1 - Scroll through entire profile
2 - Count each post comment and like
3 - Capture the post thumbnail and classify it as relevant or irrelevant to the niche
4 - Later it would repost the best images on my profile

It was working pretty well, however I think I scrolled too many posts trying to test the script I would guess around 12 profiles and 10000+ images and the account got banned lol. It seems that instagram is much more aggressive than the examples mentioned in the book that I read.

I want to create more fake accounts to keep testing, but I'm not very good at using proxies and going stealth. I'm also worried about my main account.

Anyone have any tips? I guess I should use puppeteer instead.

There are many great scrapers available online….use them and preserve your energy for other tasks

For public profies you can do scrapping without Log In.

I suppose you need to Log In for private profiles?

Regular users scrool a lot so this should not be a reason
for ban, only maybe too much actions like follow, unfollow, too many
comments or similar.

Instagram and other services can easily detect patterns even smaller ones.

IMO, Selenium is a large scale pattern. If you want success with Selenium scraping, you have to be unpredictable.

Audio fingerprinting, Canvas fingerprinting, etc are good starting points for this rat game.

Just make sure you're consistent with what you doing, and it'll pay off the effort.

davidsmurf said:

book about scraping

Click to expand...

Can you share what book?

WhiteHatCoder said:

For public profies you can do scrapping without Log In.

I suppose you need to Log In for private profiles?

Regular users scrool a lot so this should not be a reason
for ban, only maybe too much actions like follow, unfollow, too many
comments or similar.

Click to expand...

You can only scrape the first 12 posts, it then tells you to login. If you only visit the first page without login it also looks suspicious specially if youre using selenium.

slippery_slider said:

Can you share what book?

Click to expand...

"Web Scraping with Python", by O'Reilly editor, you can get it for free on zlib.

If you already know python, I suggest you skimming through it.

Selenium seems to be way too slow for scraping, why not use some just HTTP request based method?

woodburn said:

Selenium seems to be way too slow for scraping, why not use some just HTTP request based method?

Click to expand...

You cant scrape anything on any big social media website without loading javascript.

A http request wont load any javascript on their end.

How are you planning on doing step 3?

just use instagrapi(on github), its easier(if your have basic knowledge in python/api) and faster

That's easy, you can just use a pre trained object detection model built in TensorFlow, I used resNet50, but there are others more powerful.