It also depends on how large you want to go and what the primary purpose is.
If you want a host that will grow with you, look for track record, references, and what their services portfolio is like, if they own their infrastructure, if they can advise you with the setup and who manages their support.
Avoid too good to be true sounding deals as these usually result in extra charges in the next pay period and overselling the infrastructure.
Make you sure you know what you pay for and what you see is what you get.
If possible, avoid long term contracts unless you can properly trial the host and its plan performance.