1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How can I detect a search engine spider using PHP

Discussion in 'Cloaking and Content Generators' started by Physaux, Aug 12, 2010.

  1. Physaux

    Physaux Regular Member

    Joined:
    Sep 21, 2009
    Messages:
    201
    Likes Received:
    618
    Basically I need a function that I can call that returns true or false if a spider is viewing the page. What is the best way to do this? I know I can compare user-agents, but I figured I'd check with you guys before I go out and spend time creating a solution to a problem that might already be solved here. Any code I can grab that will do this for me?

    I'm entirely new to this, so please share even the basic ideas that I am missing, Thanks!
     
  2. littleg2008

    littleg2008 Senior Member

    Joined:
    Dec 3, 2009
    Messages:
    861
    Likes Received:
    421
    Location:
    Cambridgeshire, UK
    Do you want it in real time?

    I did think that cpanel has this as a built in feature when viewing your websites traffic?
     
  3. risefromdeath

    risefromdeath Power Member

    Joined:
    Jul 1, 2009
    Messages:
    650
    Likes Received:
    107
    user agent is the only way
     
  4. drkenneth

    drkenneth Executive VIP

    Joined:
    Nov 13, 2008
    Messages:
    285
    Likes Received:
    176
    Occupation:
    Developer/Entrepreneur
    Location:
    USA
    What are you using this for?

    If it's for giving extremely different content to bots and users you're going to want to get a good list of bot IPs and identify them that way. Otherwise, you risk getting found out and blacklisted.

    If it is simply for internal metrics, or some other small on-page modifications, then go for user agent. (But don't expect it to catch all the bots.)
     
  5. Black Eye

    Black Eye Newbie

    Joined:
    Aug 12, 2010
    Messages:
    14
    Likes Received:
    1
    use Cpanel, or if you want to put it on ur site, maybe ..

    just gugling for PHP script for SpiderBot visit