1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

[QUESTION] Bots and User-Agents ?

Discussion in 'C, C++, C#' started by jtrash01, Sep 25, 2015.

  1. jtrash01

    jtrash01 Regular Member

    Joined:
    Nov 5, 2013
    Messages:
    270
    Likes Received:
    117
    Location:
    BARCELONA, SPAIN
    I have scripts that assign random user-agents to each account for each instance.
    But I'm wondering If would be better to use one unique/specific user-agent string per account. And do not change it.

    What's better ?

    If you're experienced in this matter... any help would be appreciated !
     
  2. rootjazz

    rootjazz Jr. VIP Jr. VIP

    Joined:
    Dec 21, 2012
    Messages:
    684
    Likes Received:
    326
    Occupation:
    Developer
    Location:
    UK
    Home Page:
    When you log into X website in your browser as user Y, do you use a different browser each time? Do you change from mac to linux to windows to firefox to chrome to version 40 to version 13 to 26 all in a day?

    No. Your bots shouldn't either.


    But do the sites check? Probably not
     
  3. ekapek

    ekapek Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 2, 2010
    Messages:
    266
    Likes Received:
    47
    Home Page:
    It depends from site. More of them doesn't check useragent, they ofen check ips. But if you play with sites like google accounts they check useragents, ip and many more. Better for you is useragent permament for account
     
  4. rajanant

    rajanant Registered Member

    Joined:
    Sep 16, 2015
    Messages:
    62
    Likes Received:
    10
    Using one specific user-agent string per account is best solution. In case of google if you change your last browser configuration it might ask for verification (like phone number etc).
     
  5. jtrash01

    jtrash01 Regular Member

    Joined:
    Nov 5, 2013
    Messages:
    270
    Likes Received:
    117
    Location:
    BARCELONA, SPAIN
    With a bit of delay but, Many thanks for solving my doubt ! :)

    Make sense to have only one user agent per account. I made that change.

    Thank you again.
     
    Last edited: Oct 20, 2015
  6. mantic0re

    mantic0re Junior Member

    Joined:
    May 29, 2015
    Messages:
    118
    Likes Received:
    9
    Without any context it is impossible to answer. Obviously it depends on the website. If you want to bypass cloudflare for example, you have to use the same user agent on every request for that particular IP. But this is only one specific example.
     
  7. jtrash01

    jtrash01 Regular Member

    Joined:
    Nov 5, 2013
    Messages:
    270
    Likes Received:
    117
    Location:
    BARCELONA, SPAIN
    I'm mainly using Tumblr. And since Rootjazz owns a bot for that social network I though that I could trust him/her. Hehe
     
  8. nocare

    nocare Junior Member

    Joined:
    Apr 29, 2013
    Messages:
    164
    Likes Received:
    81
    Location:
    Deep Code
    Well for tumblr and other big, "wide-net" sites you can be certain they are cataloging your devices associated with an account. People will hop on with their desktops sure, but they also chill on the couch with a laptop/tablet(both?) and pick up a phone when its more convenient or they are not at home.

    So you aren't going to necessarily get flagged for changing user-agents, but what Rootjazz describes is activity that is flat out unnatural and that is kind of the point he was making. Bots trying to "secretly" mimic a human should not do bizarre things.

    I wouldn't use truly unique user agents for your bots, but some variety in genuine user agents may be a good way to avoid giving them a foothold to identify you as a bot user. EG: "user agent A + behavioral statistic C = definite bot".