Yes I agree with you, and I still use the disavow file, but many Google "top contributors" in the google webmaster forum said to not use it, and that it is a tool exclusively designed to help people who have received penalties.
But I want to share a little strange thing that happened to me about this:
4 or 5 years ago I submitted a blog post on to a few websites and got the backlinks from those (although these sites don't accept guest posts anymore now, but they did back in the days). A few weeks later I saw a decline in my rankings. And in the google webmaster forum they told me that this was because of the backlinks "I built" with the guest post. They told me to disavow those. I disavowed them, and I got back my rankings in no time. Now when looking back at this story, I think this was just a coincidence. And it had nothing to do with the guest post backlinks. So a month ago I removed these backlinks from the disavow and again now saw a drop (which though could have been caused by something else). At the same time I also tried a period of a couple weeks without disavow file and my rankings seemed to improve.
But as you and those who have been in this business long enough know, it is really difficult to pinpoint what is causing what on a short term scale.
So in your disavow which websites do you have?
If it's a small new website in another language linking to you, do you keep it?
if it's a poorly designed website with shit UX but that seems from a honest person, do you keep it?
Me, for years, I disavowed everybody, unless you are a huge blog/website. And this has worked for me. But I always have the doubt if I should let other backlinks be, not disavow them, and that could make the situation even better.