Setting up Wordpress

Usually all Linux hosts provide MySQL databases. No need to separately buy databases.

If your host provides cPanel, you can use Fantastico or SimpleScripts in it to easily install WordPress. No need to manually install it.

I use HostGator which costs around $7 a month and it provides pretty much unlimited everything incl. databases.

webhostingpad is like 3 or 4 per month and it has cpanel. i've been using it for a while now and had little/no issues.

good advice from posters above. please google about wordpress security - especially if you install manually. use a plugin like wp-security-scan to look for holes.
you could have started with a free host that offers mysql, just to get the hang of it.
two more things noobs get caught in sooner or later:
- check your hosting plan (the small print) - you might have a limitation on your number of files, make sure to disable the catchall e-mail account to avoid reaching that because of spam
- backup your content and settings religiously - if you don't start with good practices it will be increasingly hard to develop them later (and expensive, since you'd start looking at this *only* after losing precious data).

Good luck!

Every linux hosting with cpanel comes with mysql. mysql is free. you don't need to buy. Check whether your hosting server is Linux based or Windows.