Yoast making the website faster, sounds like creative (?) marketing?
I prefer The SEO Framework, much lighter and enough. A WP plugin can only do so much.
The role of the SEO plugin is very small in the grand scheme of things in my opinion. It is mostly for making sure you do not overlook something basic, such as giving a page a handcrafted title tag.
P.S.: In combination with Bongo, which I use for multi language, I had to switch to an explicit Sitemap plugin, because the sitemap functionality of SEO framework couldn't deal with Bongo correctly. The new sitemap plugin could, and the SEO framework automatically recognised the other plugin and stopped its own Sitemap. So, all good