You should deeply define what is your actual requirement.
1. Are you going to create a cluster to set up any Deep learning ML training models?
2. Create a VR Gaming development (both gaming development, hosting & rendering as a service)?
3. Preparing distributed cluster for integrating with Tensorflow, Pytorch, and similar engines?
4. Deploy HIGH Intense CPU/GPU application for the production sites in enterprises?
5. Crypto Mining (I don't think no VPS hosters with GPU won't support in the majority).?
6. Video Rendering or General purpose need?
Options are firstly figuring out the requirement i.e., listing out how much core GPU/CPU is required, Application to use, etc., so can the hardware specification can be gathered. Once the hardware is listed out, and its usage pattern you need either need to pick a service from public cloud GPU hosters like Alibaba, AWS, GCP, Azure, or even with Rackspace or any private VPU listers. everyone's terms and usage charges will be different.
Again, go for a TCO (Total Cost of Ownership) check for the annual year or bi-annual year pricing with GPU Cloud hosting companies. See if the prices match your requirement if not procure and deploy in your place (As these days GPU prices are dropped down heavily due to falling down in the crypto market).