I disavow links regularly (about once a quarter) and usually see ranking increases afterwards. I've noticed over the years that if traffic seems to be capped at a certain level, its likely to be time to disavow. Traffic usually peaks mid-week for the sites I have, and if there is no peak, traffic is capped (impressions it seems, not ranks). Disavowing the dross releases the cap.
I understand the arguments against disavowing - that you're admitting that you're building links, or that you could disavow a link that Google likes. However, I'm careful about what links I do build and I know what is spam. I don't disavow links I build because I only build good links. My disavow file contains about 5,000 domains. Many have been built by bots on .pw (or other cheap) domains. If you've a good website, other people are likely to build spam links to it for all sorts of reasons.
Disavow is only for links you can't remove otherwise. If you've bought a "Sherlock rides Lady Gaga" service, ask the service provider to remove the links before trying to disavow them. Disavow seems to be like no-indexing a link - you no longer get credit, but the anchor text remains. Anchor text is important, so you want to be monitoring that as well as the domains on which the links are. Disavowing won't solve an anchor text over-optimization problem.
I only disavow based on links shown in Webmaster Tools / Search Console. Because Google doesn't show all links and because it changes those it does show, I have whitelists and blacklists of domains that have featured in there. Using an Excel sheet and the VLOOKUP function, I can see which links are new in WMT.
I've found that if you disavow based on a list of links from ahrefs, you're more likely to disavow links that aren't harming you. The links Google shows in WMT seem to include the worst offenders.
I manually check every single link. Yes, it takes time, but its the only way of knowing what is spam.
Your link profile doesn't have to be perfect. Google seems to tolerate some spam.
Just as with any link to your site, you need Google to crawl it before it takes effect. I use speed-links to do this.