Multilingual sitemap

We're making multilingual sitemap. We have lost all rankings the last 4-5 days and we believe that is due to lack of canonical links and faulty sitemap (maybe it looks like we only have duplicate content).

<loc>https://www.example.com/</loc>
<xhtml:link
rel="alternate"
hreflang="de"
href="https://www.example.com/de "/>
<xhtml:link
rel="alternate"
hreflang="fr"
href="https://www.example.com/fr "/>
<xhtml:link
rel="alternate"
hreflang="da"
href="https://www.example.com/da "/>
<xhtml:link
rel="alternate"
hreflang="pl"
href="https://www.example.com/pl "/>
<lastmod>2019-08-14T16:03:10+00:00</lastmod>
<priority>1.00</priority>
</url>
<url>
<loc>https://www.example.com/page1/</loc>
<xhtml:link
rel="alternate"
hreflang="de"
href="https://www.example.com/de/page1"/>
<xhtml:link
rel="alternate"
hreflang="fr"
href="https://www.example.com/no/page1"/>
<xhtml:link
rel="alternate"
hreflang="da"
href="https://www.example.com/da/page1"/>
<xhtml:link
rel="alternate"
hreflang="pl"
href="https://www.example.com/pl/page1" />
<lastmod>2019-08-14T16:26:24+00:00</lastmod>
<priority>0.80</priority>
</url>

Is this a good set-up? And will this keep rating on the alternates? Also, do you believe that wrong sitemap/canocicals might be the reason for significant drop in ratings?