1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can I prevent Google crawling / indexing pages on scraper website?

Discussion in 'Black Hat SEO' started by Evan B123, Feb 20, 2013.

  1. Evan B123

    Evan B123 Newbie

    Joined:
    Feb 20, 2013
    Messages:
    1
    Likes Received:
    0
    Hello BlackHatWorld

    I am a long term viewer on BHW, primarily in the marketing forums. I read through some of the threads on the 'Cloaking' Forum but since I'm not technical myself I found these quite difficult to follow. I have an issue which I would be extremely grateful to gain some advice on....

    My issue:

    I have a site which scrapes video content - primarily from Youtube for a particular niche. My website automatically generates a title tag which is just about identical to the Youtube title tag.

    I don't want to be penalized or blacklisted by Google. Is there anyway that I can prevent these certain pages from being crawled/judged by Google? I know that I can set 'noindex' tags to all of these pages...

    I don't care if these pages aren't indexed in google (I only care about being indexed and ranked high for brand keywords) - I just need to make sure that my overall website / branded keywords won't be penalized as a result of having these titles the same as Youtube page titles.

    Any help or suggestions would be greatly appreciated.
    Evan
     
  2. executed

    executed Power Member

    Joined:
    Feb 19, 2012
    Messages:
    699
    Likes Received:
    200
  3. ok888

    ok888 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 23, 2010
    Messages:
    2,363
    Likes Received:
    649
  4. maecenas23

    maecenas23 Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 12, 2010
    Messages:
    1,070
    Likes Received:
    986
    Occupation:
    Full-time worker :D
    Location:
    #RIPCity
    The robots will do the job. The tutorial from executed is perfect for what you need.