I'm in the Joomla camp. I've been a professional developer for quite a while, and, for me, the bottom line is that Joomla just shaves all sorts of time off of development.
It is true that the code that it generates is sometimes less than pretty. This is often the fault of the template as much as anything else, but it is actually pretty easy to work around.
The thing to remember is that most Joomla sites are bogged down with a LOT of external includes (CSS and JS mostly). If you want the site to really scream, once you have it dialed in from a visual and functional perspective, go in to the template and combine all of the CSS and JS. Use a CSS optimizer to strip out whitespace and any unused code as well. Finally, turn on the cache! Unless you have a site that needs to hit the DB all the time, the cache will seriously speed things up.
That being said, I have sometimes "flattened" a Joomla site in order to really max out the performance. Develop the site locally, and when you have it "just so", simply mirror it to your "real" Web server. The mirror will be HTML only, no database stuff (obviously this only works for sites that are essentially static in nature, so use some common sense) and so it will scream. The nice thing about this approach is that you can update the local version of the site taking advantage of the "ease" of the cms, and then update your "mirror", thus getting the best of both worlds for only minor inconvenience.
