spikyy says it... it's the google dance, we've all heard it (and seen it) before. I once got a clients site to literally #1 (low competition), then boom, it was gone. I was aghast, but the client told me to just keep on building links. So I did and the site came back, then stayed there for awhile. I guess one way to think about it is this... if you're king of the mountain, there will always be someone wanting to knock you off, so you must fight to stay there.l
And someone once against, when are there ENOUGH back links? Well think about it logically, if you build and build, promote and promote, then get to page 1 or number 1, would the inbound links suddenly stop? Probably not, they'd probably increase since more people would be talking about you. So keep it up, you're certainly on the right road.