1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How to reduce crawl rate?

Discussion in 'Black Hat SEO' started by deeminionz, Dec 6, 2012.

  1. deeminionz

    deeminionz Junior Member

    Joined:
    May 25, 2012
    Messages:
    126
    Likes Received:
    9
    Say I want to ensure that google, baidu, whatever do not crawl the URL more than once a week. How would I do so?
     
  2. HatIsBlacked

    HatIsBlacked Regular Member

    Joined:
    Dec 30, 2010
    Messages:
    224
    Likes Received:
    57
    set crawl delay through your robots.txt file