At the hosting side of things both the provider you buy from and the type of hosting product you buy matter.
Generally:
dedicated server > VPS > paid shared hosting >>> free hosting
Of course the further up you move the more skills and time you need to set it up.
Other than the server itself, which is something you can just throw money at to improve performance if you can afford it, you need to find where the bottleneck is.
As others have mentioned, depending on where the problem is (the theme,a plugin, images or other content, etc) the solution will differ.