It could be. If everything stays the same, then it should not matter. But if the theme change also changes some data on your site, it could affect your SEO. For example, a new theme could lower or increase your site speed, it could change structured data like Schema, or it could even change or break your headings (some themes have titles to be H2 be default for example).
So I would say that it's safe if you know what you are doing, but make sure that everything is as it was, or is a change for the better.