You'll take a little dip for a while, but so long as you stay within your niche, it's not a big hit and you can recover from it easily enough. Make sure the domain has had continuous registration and that it has been indexed. The continuous registration may be debatable, but I stand by it that once the domain has dropped you're basically starting from scratch. As far as domain age is concerned, the oldest domain in existence still isn't any good to you if it's never been indexed...