1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

stop google indexing my links page

Discussion in 'Black Hat SEO' started by nickjett, Feb 19, 2013.

  1. nickjett

    nickjett Junior Member

    Joined:
    Oct 28, 2010
    Messages:
    150
    Likes Received:
    9
    Could someone please tell me some ways i can prevent google from indexing one of my pages on my site. Im using a certain link exchange website. But i dont want google to index the my links page, so i get the full affect of a one way backlink.
     
  2. calebmann

    calebmann Power Member Premium Member

    Joined:
    Sep 12, 2010
    Messages:
    541
    Likes Received:
    298
    Occupation:
    FULL TIME INTERNET
    Location:
    Digitopia
    Code:
    <meta name="robots" content="noindex">
    plug that into your page.
     
  3. nickjett

    nickjett Junior Member

    Joined:
    Oct 28, 2010
    Messages:
    150
    Likes Received:
    9
    Any other ways, this way is quite easy to detect. Im pretty sure the link exchange site detects this particular way and then doesnt allow the link exchange.
     
  4. mrblackjack

    mrblackjack Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 6, 2011
    Messages:
    960
    Likes Received:
    552
    Occupation:
    I live alone, I work alone, I make money alone
    Location:
    G00gle LaNd
    well, in order to prevent Google from indexing your outbound links but than again to make them visible to the users, you need a server side script (so it wont be visible to the users who viewsource your page code) to handle the Google VS. Visitor detection.
    you need to pay attention to the Google Crawlers patterns available to you which are:
    1. Useragent string.
    2. Bot Ip net ranges.


    The UA string of Google bot will normally include the string 'google' and/or 'googlebot' - we need to grab this
    The IP ranges vary, and you'll need to conduct a research for grabbing those ip ranges - I'll attach here an IP ranges that you can start with.


    The process is pretty simple, you set a PHP script that checks the above parameters against the visitor to determine whether it's a user or a Google bot. If a visitor UA contains the 'google' string, and/or is within the IP ranges defined by you as of Google bot, it's Google bot so print different HTML code for it; otherwise, it's a real user so print the HTML code that presents the outbound links.


    Check out the following PHP code - it should be inserted into your site at the relevant page.


    PHP:
    <?php    $useragent     $_SERVER['HTTP_USER_AGENT'];    $ip         $_SERVER['REMOTE_ADDR'];            function checkUserAgent(){            global $useragent;            $useragents = array('Googlebot''google','AdsBot-Google','adsbot');                foreach($useragents as $string){                    if (stristr($useragent$string)){                        return true;                    }                }                                    }                    function checkGoogleIP (){            global $ip;                $parts explode('.',$ip);                if(                    ($parts[0] == 66     AND                     $parts[1] == 249     AND                     (64 <= $parts[2])     AND ($parts[2] <= 95))                  )                return true;        }                function checkGoogle(){            if ( checkGoogleIP() OR checkUserAgent() ){                // it's Google so output html code without links            }else{                // it's a Visitor so output html code with links            }                }                // fire the function        checkGoogle();?>
     
  5. innozemec

    innozemec Jr. VIP Jr. VIP

    Joined:
    Aug 19, 2011
    Messages:
    5,290
    Likes Received:
    1,799
    Location:
    www.Indexification.com
    Home Page:
  6. GiorgioB

    GiorgioB Supreme Member

    Joined:
    Feb 28, 2012
    Messages:
    1,288
    Likes Received:
    1,318
    Occupation:
    Making money
    Location:
    Touching the Sky
    just add the page URL to your robots.txt file as nofollow. or else i think you can do it from WMT
     
  7. mrblackjack

    mrblackjack Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 6, 2011
    Messages:
    960
    Likes Received:
    552
    Occupation:
    I live alone, I work alone, I make money alone
    Location:
    G00gle LaNd
    not good. since he wants to prevent users from checking on his outbound links. a quick robots.txt check will reveal his method - not good!
     
  8. innozemec

    innozemec Jr. VIP Jr. VIP

    Joined:
    Aug 19, 2011
    Messages:
    5,290
    Likes Received:
    1,799
    Location:
    www.Indexification.com
    Home Page:
  9. GiorgioB

    GiorgioB Supreme Member

    Joined:
    Feb 28, 2012
    Messages:
    1,288
    Likes Received:
    1,318
    Occupation:
    Making money
    Location:
    Touching the Sky
    users? I thought it was just the spiders he wanted to prevent from crawling and indexing (btw i meant noindex, not no follow). I did that for my admin URL for my websites, thought it was fine

    really? pages i told them not to index were not indexed...
     
  10. mrblackjack

    mrblackjack Jr. VIP Jr. VIP Premium Member

    Joined:
    Dec 6, 2011
    Messages:
    960
    Likes Received:
    552
    Occupation:
    I live alone, I work alone, I make money alone
    Location:
    G00gle LaNd


    Check out the bold & red. He wants one-way backlins thus needs to prevent from links exchangers to find out that their links will never get indexed - therefore, he needs to prevent them from finding out his trick, and prevent Google from indexing those links
     
  11. nickjett

    nickjett Junior Member

    Joined:
    Oct 28, 2010
    Messages:
    150
    Likes Received:
    9

    I have checked a few ips Google uses at seems to be anywhere between 66.249.73.... 66.249.75...., in your script the range seems to be a bit wider, but shouldnt be a problem. Im not very good with scripts, is the script you gave me complete?

    All i have to do is add that to a php page? and my outbound links wont show up to googlebot? How can i test it works?
     
  12. phpbuilt

    phpbuilt Jr. VIP Jr. VIP

    Joined:
    May 16, 2011
    Messages:
    1,650
    Likes Received:
    5,208
    Occupation:
    $ from websites I own.
    Location:
    putting monkeys in paypal
    First create a page called cloak.php. Save the following to it.

    Code:
    <?php
    
    /*
    
    NiX Cloak Copyright (C) 2012 MyProxyLists.com
    
    License Agreement: You may not use this code to sell any kind of cloaking services. Feel free to modify for your needs.
    
    DO NOT SHARE THIS CODE OUTSIDE BHW (blackhatworld.com). THE CODE IS BHW EXCLUSIVE!
    */
    
    function search_engine($ipaddr = '') {
    
        if (!valid_ip($ipaddr)) {
            return false;
        }
    
        /* Ranges for Google/Yahoo/Bing are 99.9% accurate as of 16th of November 2012 */
    
        $ranges = array(
            "89.207.224.0/21",
            "8.8.8.0/24",
            "8.8.4.0/24",
            "8.35.200.0/21",
            "8.35.192.0/21",
            "8.34.216.0/21",
            "8.34.208.0/21",
            "74.125.0.0/16",
            "72.14.192.0/18",
            "70.32.128.0/19",
            "66.249.64.0/19",
            "66.102.0.0/20",
            "64.233.182.0/23",
            "64.233.173.0/24",
            "64.233.172.0/24",
            "64.233.168.0/23",
            "64.233.160.0/19",
            "64.15.112.0/20",
            "216.58.192.0/19",
            "216.239.32.0/19",
            "209.85.128.0/17",
            "208.65.152.0/22",
            "208.117.224.0/19",
            "207.223.160.0/20",
            "199.223.232.0/21",
            "199.192.112.0/22",
            "193.142.125.0/24",
            "192.178.0.0/15",
            "173.255.112.0/20",
            "173.194.0.0/16",
            "172.217.0.0/16",
            "142.250.0.0/15",
            "113.197.105.0/24",
            "108.59.80.0/20",
            "108.177.0.0/17",
            "108.170.192.0/18",
            "192.197.157.0/24",
            "168.62.0.0/15",
            "168.61.0.0/16",
            "167.220.40.0/21",
            "157.60.31.0/24",
            "157.60.23.0/24",
            "157.58.90.0/24",
            "157.55.0.0/16",
            "157.56.0.0/16",
            "138.91.0.0/16",
            "137.135.0.0/16",
            "137.116.0.0/15",
            "134.170.0.0/16",
            "132.245.0.0/16",
            "131.253.8.0/24",
            "131.253.64.0/18",
            "131.253.62.0/23",
            "131.253.61.0/24",
            "131.253.6.0/24",
            "131.253.5.0/24",
            "131.253.32.0/20",
            "131.253.3.0/24",
            "131.253.24.0/21",
            "131.253.21.0/24",
            "131.253.18.0/24",
            "131.253.12.0/22",
            "131.253.1.0/24",
            "111.221.69.0/24",
            "111.221.66.0/24",
            "111.221.23.0/24",
            "111.221.112.0/21",
            "94.245.64.0/18",
            "70.37.135.0/24",
            "70.37.128.0/23",
            "70.37.128.0/18",
            "70.37.0.0/17",
            "65.55.86.0/23",
            "65.55.117.0/24",
            "65.54.92.0/23",
            "65.54.88.0/23",
            "65.52.0.0/14",
            "64.4.0.0/18",
            "216.32.240.0/22",
            "216.32.180.0/22",
            "213.199.128.0/18",
            "209.240.192.0/19",
            "209.185.128.0/22",
            "208.68.136.0/21",
            "207.82.250.0/23",
            "207.68.128.0/18",
            "207.46.98.0/24",
            "207.46.96.0/19",
            "207.46.77.0/24",
            "207.46.62.0/24",
            "207.46.58.0/24",
            "207.46.32.0/20",
            "207.46.128.0/17",
            "207.46.0.0/19",
            "206.191.224.0/19",
            "206.138.168.0/21",
            "205.248.96.0/19",
            "204.95.96.0/20",
            "204.79.197.0/24",
            "204.79.196.0/24",
            "204.79.195.0/24",
            "204.152.140.0/23",
            "199.74.210.0/24",
            "199.60.28.0/24",
            "199.30.16.0/20",
            "199.242.48.0/21",
            "199.103.90.0/23",
            "199.103.122.0/24",
            "198.49.8.0/24",
            "198.206.164.0/24",
            "198.200.130.0/24",
            "194.69.96.0/19",
            "193.149.64.0/19",
            "192.84.160.0/23",
            "192.84.159.0/24",
            "192.48.225.0/24",
            "98.139.96.0/22",
            "98.139.48.0/20",
            "98.139.104.0/22",
            "98.139.0.0/20",
            "98.137.64.0/20",
            "98.137.216.0/21",
            "98.137.208.0/21",
            "98.137.192.0/20",
            "98.136.64.0/18",
            "98.136.128.0/19",
            "8.12.146.0/24",
            "76.13.5.0/24",
            "76.13.238.0/23",
            "76.13.236.0/23",
            "76.13.232.0/23",
            "76.13.234.0/23",
            "76.13.232.0/22",
            "76.13.192.0/20",
            "76.13.215.0/24",
            "76.13.216.0/21",
            "76.13.224.0/21",
            "74.6.232.0/22",
            "74.6.236.0/22",
            "74.6.240.0/21",
            "74.6.64.0/19",
            "76.13.0.0/17",
            "76.13.128.0/18",
            "74.6.228.0/22",
            "74.6.156.0/23",
            "74.6.152.0/23",
            "74.6.148.0/23",
            "74.6.150.0/23",
            "74.6.0.0/19",
            "72.30.96.0/20",
            "72.30.64.0/19",
            "72.30.240.0/20",
            "72.30.224.0/21",
            "72.30.220.0/22",
            "72.30.216.0/22",
            "72.30.192.0/20",
            "72.30.128.0/18",
            "68.180.152.0/21",
            "67.28.115.0/24",
            "67.195.224.0/19",
            "67.195.192.0/19",
            "67.195.160.0/19",
            "66.228.188.0/22",
            "66.228.184.0/22",
            "66.218.64.0/19",
            "66.163.184.0/21",
            "216.109.112.0/20",
            "209.73.176.0/21",
            "209.73.160.0/20"
        );
    
        shuffle($ranges);
    
        foreach ($ranges as $cidr) {
    
            if (ip_in_block($ipaddr, $cidr)) {
                return true;
            }
        }
    
        return false;
    }
    
    function valid_ip($ipaddr = '') {
    
        if (preg_match("/^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/", $ipaddr)) {
    
            $parts = explode(".", $ipaddr);
    
            foreach ($parts as $part) {
    
                if (intval($part) > 255 || intval($part) < 0) {
                    return false;
                }
            }
    
            return true;
    
        } else {
            return false;
        }
    }
    
    function ip_in_block($ipaddr = '', $cidr = '') {
    
        if (valid_ip($ipaddr) && preg_match('/^([0-9]{1,3}\.)([0-9]{1,3}\.)([0-9]{1,3}\.)([0-9]{1,3})+(\/[0-9]{1,2})$/', $cidr, $m)) {
    
            $dq_host = $m[1] . $m[2] . $m[3] . $m[4];
            $cdr_nmask = strtok("$m[5]", "/");
            $bin_host = dqtobin($dq_host);
            $bin_net = (str_pad(substr($bin_host, 0, $cdr_nmask), 32, 0));
            $network = bintodq($bin_net);
    
    
            if (ip_in_network($ipaddr, $network, $cdr_nmask)) {
                return true;
            } else {
                return false;
            }
    
        } else {
            return false;
        }
    }
    
    function dqtobin($dqin) {
    
        $bin = null;
        $dq = explode(".", $dqin);
    
        for ($i = 0; $i < 4; $i++) {
            $bin[$i] = str_pad(decbin($dq[$i]), 8, "0", STR_PAD_LEFT);
        }
    
        return implode("", $bin);
    }
    
    function bintodq($binin = '') {
    
        $dq = null;
        $binin = explode(".", chunk_split($binin, 8, "."));
    
        for ($i = 0; $i < 4; $i++) {
            $dq[$i] = bindec($binin[$i]);
        }
    
        return implode(".", $dq);
    }
    
    function ip_in_network($ip, $net_addr, $net_mask) {
        $ip_binary_string = sprintf("%032b", ip2long($ip));
        $net_binary_string = sprintf("%032b", ip2long($net_addr));
        return (substr_compare($ip_binary_string, $net_binary_string, 0, $net_mask) === 0);
    }
    
    ?>
    Next, on your page that you want to restrict google/yahoo/bing from seeing, put this at the very top.

    Code:
    require_once("cloak.php");
    $remote_addr = $_SERVER['REMOTE_ADDR'];
    if (search_engine($remote_addr)) {
        $newurl = "http://".$_SERVER["HTTP_HOST"];
        Header( "HTTP/1.1 301 Moved Permanently" );
        Header( "Location: $newurl" );
        exit;
    }
    
    This will make Google/Yahoo/Bing see your link page as a 301 to your homepage (so you recover any lost PR, and Google doesn't see it).