1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Facebook ads bot and robots.txt

Discussion in 'General PPC Discussion' started by lsmstd, Jan 6, 2013.

  1. lsmstd

    lsmstd Regular Member

    Joined:
    Jan 9, 2012
    Messages:
    388
    Likes Received:
    113
    I would like to block all spiders but let facebook's bot scrape my landing page.
    Do you know what is the user agent of their landing page scraper? Does it obey robots.txt?
    +rep/thanks for some answers
     
  2. robinimtiaj

    robinimtiaj Newbie

    Joined:
    Jan 2, 2013
    Messages:
    13
    Likes Received:
    0
    Occupation:
    Student
    Location:
    mymensingh
    i think robots now not working.
     
  3. lsmstd

    lsmstd Regular Member

    Joined:
    Jan 9, 2012
    Messages:
    388
    Likes Received:
    113
    You don't understand the question.