Twitter Strategy: Followers?

Hi All,

I have a sort of a technical question to ask, would appreciate it if a Twitter expert can help me answer it...

So I have a Twitter account, I only follow people I like to hear from, brands or public figures, etc...

On the other hand, I also tweet a lot and many people add me... However I do NOT add them back (as most people who are looking to grow they follower base do), because I figured out a lot of them are just "spammers" or bots, just looking for people who will follow them back... They have no interest in what I tweet about (stuff related to my work), and I have no interest in hearing from them either...

Not adding them back results in them unfollowing me after a while... So over time, my follower base increases and them decreases constantly, resulting in almost no net increase...

Now what if I did decide to follow them back?
Would this affect the impression of my tweet negatively?

Meaning, most of those people I will follow back won't engage with my tweets, which means that Twitter won't help my tweets get more impressions (although I'm always using relevant hashtags)...

Am I correct with this assumption?

Well if you don't tweet that many and focus on quality over quantity you will get followers that are engaged with your content, imo more than 12 tws/day is spammy.