it's backlink matter. So if you focus on spam link you will experience this going down in rank.
and my website have the same problems. Then just do something in disavow for not related link to anchortext
Has nothing to do with backlinks. It's a content quality update. Do you have high-quality content on the page that answers the E-A-T guidelines by Google? You'll rank higher. You don't follow the high-quality guidelines? You will derank. TLDR; do high-quality content and focus intent intenta and you'll rank higher than your competitors.