Adding a SSL to my main website, should I change the URL to all my backlinks on my PBN?

Morning SEO Experts,

I added a SSL certificate to my main website (https) and suffered keywords loss rankings.

I'm wondering if I should change the URL of all my backlinks on my PBN to include the (http(s)) ?

Is someone already experienced such issue ?

Thank you