1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

disallow / deindex pages

Discussion in 'White Hat SEO' started by clutch5, Feb 12, 2012.

  1. clutch5

    clutch5 Registered Member

    Joined:
    Oct 31, 2009
    Messages:
    74
    Likes Received:
    12
    I have website and I want to disallow / deindex pages containing "x" keyword in permalinks

    example: domain.com/how-to-"x"

    can I do this using robots.txt or???
     
  2. misteryou.

    misteryou. Power Member

    Joined:
    Feb 1, 2012
    Messages:
    573
    Likes Received:
    109
    Block a specific web crawler from a specific web page ?

    User-agent: Googlebot
    Disallow: /no-google/blocked-page.html


    ???
     
  3. MBW007

    MBW007 Junior Member

    Joined:
    Feb 11, 2012
    Messages:
    157
    Likes Received:
    29
    Location:
    In the sandbox. Again.
    Disallow: /how-to-

    Will disallow everything beginning with /how-to- such as /how-to-fly/fly.html and /how-to-fly.php

    However, removing it from the index is another issue all together.