1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Block IG from crawling/spidering website in .htaccess

Discussion in 'Black Hat SEO' started by Spinethetic, Apr 14, 2016.

  1. Spinethetic

    Spinethetic Registered Member

    Joined:
    Dec 10, 2015
    Messages:
    91
    Likes Received:
    69
    Anybody have a code for .htaccess and robots.txt to block the Instagram crawl spider from crawling a website or redirect them?

    I ask because I'm using the same root domain across my IG accounts to track click performance for individual ig profiles but they all redirect to the same url.

    Best regards and stay blessed! :)
     
    • Thanks Thanks x 1
    Last edited: Apr 14, 2016
  2. phatzilla

    phatzilla Jr. VIP Jr. VIP

    Joined:
    Apr 9, 2009
    Messages:
    1,383
    Likes Received:
    1,023
    Figure out their IP range and or UAstring, then redirect based on those signals.