From my experience, yes, it works exactly like that. That's why having your primary keywords in the domain name makes a LOT of sense, in my opinion. If you're creating niche sites, that's the most sensible approach.
I've personally seen sites rank on the first spot on google without having a single backlink with correct anchor text (an example keyword of one of these sites gets around 270 searches per day according to the google keyword tool).
The site only had backlinks with the url itself as anchor text and a few links which are probably semantically relevant (example: backlinks with anchor text "skin problems" for an acne site).
So, personally, I definitely believe it works that way.