Hey, I'm programming now a video site, this site is totally dynamic, content is created on request, for example: site.com/videos/election (where election is the keyword, matching videos for that will be displayed) site.com/tags/obama (where obama is the tag, videos tagged like that will be displayed) site.com/author/whatev (where whatev is the author, all videos by the author gets displayed) site.com/watch/xyz/video-title-here (xyz is the youtube code, video-title-here is the for the seo friendly url) All pages has links to relevant other pages within the site, if they get clicked the content get's generated for the request too. Hope you are not confused yet So, when a spider visits the site, they crawl the whole site based on the links on each page right? But what happens to dynamic content like that? The spider could hang out forever on the site crawling all links it finds. Just dont know if thats a good or bad thing. Tell me what you think, cheers.