Google search experts please help me out

IMShane

Junior Member
Joined
Sep 20, 2011
Messages
131
Reaction score
23
Let's take domain.com as an example, if I google "site:domain.com" all pages such as domain.com/about.html, domain.com/example/whats-up.html, etc. would appear as search results. If I only want results such as domain.com/xxxx and keep deeper urls like domain.com/xxxx/xxxx.html out, what parameters should I use?

Thanks

P.S.: I'm trying to scrape indexed pages of a site to a certain level (to domain.com/xxxx in this case).
 
Last edited:
Put a no index on each page you don't want indexed and disallow pages on your robots.txt
 
Put a no index on each page you don't want indexed and disallow pages on your robots.txt
Well, I think you misunderstood. I was just trying to scrape indexed results of a domain (only to domain.com/xxxx level). domain.com is not my site.
 
Back
Top