A VPS (at least in cases of how people use them on this forum) is a Windows server that lets you pretty much remote connect into it. You'll pretty much be in control of another desktop that doesn't sink your bandwidth, slow your daily tasks, and is dedicated for your purposes.
Its good for people doing a lot of backlinking and need a really fast connection (100mbps+). The one I have is great for that as I can run programs with 2-3x more threads (depending on the template) continuously without worrying that I won't be able to do anything on the PC due to 100% CPU usage.
At first I thought I could save a bit of money by doing it on my home PC as I recently upgraded my specs, and though it does a good job I am pretty glad that I have a VPS to do most of my stuff on. It's running continuously with various pieces of software.