Most VPS are indeed virtualized machine on a dedicated server. However, VM can also be created on your computer, thus that would be a little different than a datacenter-hosted VPS.
Of course, datacenter-hosted VPS would run 24/7, but make sure you pick a reliable provider, as the smaller ones tend to overload their machines or perhaps use old server hardware that would not run very quickly when operated. A VPS with quality SSD can outperform most old machines, even though the old machine uses 16-core processor and conventional rotational drive.
A self created VM would most likely be on your own computer - if you shutdown your computer, that gets shut off as well.
If you want a machine that runs 24/7, the datacenter-hosted VPS would be a good choice - provided it is from a good provider that is.