I've recently drafted this sort of a check-list:
I'll expand on it:
---
1) do I understand from the specifications what I'm buying ?
Many service specifications are a bit complex and can get tricky.
2) what's the payment terms - any hidden fees or price shifts ?
What I don't like personally are these special promotions and then you get charged often much more and over the limit.
3) is the sales team easy to reach for clarifications?
This of course is a huge plus - if you send a request, you'll need a qualified answer, and ideally a timely answer.
4) if I'm not satisfied with the service, can I cancel and switch?
Top important - many hosts will lock you into long term contracts, that are hard to get out of if you are not satisfied with the service.
From more of a tech stand point:
5) what's the variety of services offered, will it be easy to upgrade, as my requirements grow?
Say that your traffic grows, you'll need additional services like CDN, or cloud, upgrade your server - solid host should be happy to oblige.
6) do they own their infrastructure?
Very important, it's easy to run a hosting service as a reseller, but of course, if you want something more complex, and get the right price / value ratio, you need to look into actual hosts that own their business, and go direct (not to pay the middleman).
7) where are the data centers located?
Not only regarding the connectivity, but also the legal landscape that may affect various types of online businesses (especially high risk).
8) is there a solid CDN option available?
As your site grows, if you want to do numbers, or even if you're a media buyer that wants a super fast and stable loading time on his landers. The pricing should be straightforward too.
And of course, how experienced is the team when it comes to assisting with your needs, or migrating your existing websites from another host. An experienced team can even advise you regarding any scripts or setup that you plan to run on your server.
---
So these would be a couple of things I would look into personally.
Plus of course track record and references.