I would go even further than this - increase in competition is a great sign. It means that many people think that this is a lucrative niche, which is definitely a sign that you need to enter the niche.
Obviously on-page SEO works, that's a real weird thing to say. What's not working is probably the fact that your competitors are doing it better. The way you put it makes me think, what else do you do than keywords? There is much that can be done with On-Page. Will structured data be obsolete? WIll accessibility be obsolete? Will pagespeed be obsolete? Will UX, engagement, bounce rate etc be obsolete as all of those depend on on-page? Probably not.
This long-tail or short kws or whatever is a (slowly) dying practice anyway. While Google still relies some on specific keywords, I think this will be eventually going away as their NLP models are becoming better and better understanding user intent and site content on a deep level.