1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Everything about bots. Huge Intro and discussions, kinds and more

Discussion in 'Black Hat SEO' started by elbeto, Apr 5, 2010.

  1. elbeto

    elbeto Registered Member

    Joined:
    May 1, 2009
    Messages:
    98
    Likes Received:
    75
    Occupation:
    Beer drinker
    Location:
    close to the laptop
    (English is not my first language)

    Hello buddies, After reading a lot of post of people asking for bots and making questions about which kind of bot is better, I decide to write this post about it.

    I have around 6 years experience programming what you called "bots". For a better understanding of the "bots", first we have to define the bot's kinds:

    1.- Web Bots: This kind of bots are based on web tasks automation, we can programm with them a browsing session, ftp download, rss download etc.
    2.- Mixed Bots: This kind of bots can mixed local and web bots. This one can manage web and local application.
    3.- Intelligent Bots: This kind of bots use Artifitial Intelligence to complete task, can be mixed also.
    4.- Another: Probably there is more kind, but that's not applicable to the goal of this post.


    Getting Deep:

    -------------------------------

    WEB BOTS:

    For this kind of bots there is plenty of options, Im going to cover here: Imacro, PHP/CURL, UBOT and Javascript, of course there is a lot more and if you want to complete this post, please do it...

    IMACRO: This Programming tool come in 2 types... A firefox Add on and StandAlone.
    Imacro Firefox and Internet Explorer Add on: This one is totally free you can download it from: https://addons.mozilla.org/en-US/firefox/addon/3863 and from: http://www.iopus.com/download/imacros-ie/ How to start? Well turn on the Imacro Record button and see what happened when you are browsing with these babies, saved the macro recorded and then play it to see all what you did before.
    Pros: - Easy to use. One click, One Macro recorded
    - easy to automate simple tasks.
    - Platform Independent

    Cons: - You can not programm conditionals, to make decisions based on situations.
    - You can not programm fucntions for error routines or Database Connections.
    - Very Limited Bots, the loops programming is very simple, you can not wait for images to display. You can not programm get data and then process it.
    - You can not add proxies
    - You can not change your user agent
    - If you want to make multithread you have to learn javascript first, to call multiple instances of the same macro. Of course you will not have the benefits of threads, memory share, etc

    Imacro StandAlone or Imacro Browser: This one cost 500$ but we are in blackhatworld and you can download it and activate it from here: http://www.blackhatworld.com/blackh...2329-get-iopus-imacros-scripting-edition.html This version is the perfect match for the addon version, if you do not know how to make something, record it first and then use it with the StandAlone, of course we still have the lack of implementing conditionals, but now we can integrate this baby with PHP Visual Basic or Javascript (and many others) to process the data.
    Pros: - Easy to use.
    - Documentation and examples on the web
    - Proxy Implementation
    - Cache clean implementation
    - Database Connection can be implemented with the supporting language (PHP, VB)
    - User Agent implementation
    Cons: - If you are using PHP you will need to install a webserver on your local machine. You can use wamp for this
    - Depends on another processing programming language
    - Do not have wait for image function, if your browsing suddenly stop (internet problem for example) the macro can give you an error
    - You have to know a compatible programming language to process the data and then come back to the macro running or start another macro to continue the process



    PHP/CURL
    and CURL: this is one of the best programming language if you want to build web bots, the PHP/CURL version can run whenever you have installed PHP (adding the curl library support), the standalone version called CURL, can run in many platforms by command line. PHP/CURL and CURL are focused on the http headers, you can fake almost everything, user agent, proxies, cookies, forms, ajax, javascript execution, referers. You do not have to navigate the whole web, download the images, etc, you just need to send http headers, analyze the answer from the server and continue with another http headers. No matters if the form is a mail composer or a login/password form with hidden fields, with PHP/CURL you can fake it using the same name fields. Now there is a implementation of simultaneous CURLs execution, so you can for example Send 50 facebook accounts to login to facebook in parallel. Yeah I use this for facebook hehehehehe, you can make wall posting, captcha solver using any api compatible with PHP, confirm friends, confirm groups, database connections, almost anything... If you are familiar Artifitial Intelligence you can make marvelous with this babies
    Cons: -The Learning Curve takes some time, special cases if you never programm with any language before.
    - You will need a live http header analyzer to get the form fields names and http headers send and response
    - You have to know how to use cookies and how to use regular expressions to get specific data from files or html code
    - You have to update the bot if something change in the http header


    UBOT: Speaking shortly its a tool which lets complete coding noobs make really complicated bots that will make their life easier. UBot uses Visual Scripting Language which you will be able to start to learn within 5 hours of watching detailed tutorial videos. It has inbult actions useful for bot making: email/names generation, captcha solving outsourcing and proxy flipping Great thing is the bots are compiled as standalone programs so later on you can give them to friend or even sell without worrying if they are compatible with someones computer.
    Cons: - For me the bots are really really slow.
    - To make simultaneous task (multiple bots running in parallel) is really annoying, time and memory/resource consuming
    - You have to know how to disable image loading if you want it a little bit faster
    - Only for windows (im not sure about this, but i can not find a linux/unix/mac version)
    - Not database support, you have to use the always slow and unefficient CSV files
    - Learning Curves, and basic programming knowledge
    - You will be slave of the bots, almost everytime the website change something, you have to change the bot

    Javascript : This is one of the hardest to learn, but also one of the most powerfull tools you can make almost anything on the web with javascript. First you have to know the HTML DOM: The HTML DOM - (Document Object Model) A basic understanding of the HTML DOM is essential for controlling webpages. This is because it's a "map" or "heirarchy" for accessing parts of the webpage. The HTML DOM is not language dependent, but rather the model for how the webpage document is constructed.

    Once you dominate the DOM you can access anything on the web with javascript, you can make database connection using AJAX, fake the user agent, referer, proxies, etc... I use this one in conjunction with autohotkey (explained below)


    MIXED BOTS:

    Winautomation: Ok, now we are talking about the mixed bots, the one that can automate our local tasks and web task. Winautomation has the ability to launch other programms, make conditionals, compare with images and connect to a few types of database, the built in excel support is good to keep small amount of data. You can generate executables and portable files. Has FTP and Send Mail support, the price around 300$, you can download it also using bittorrent. Or you can use thevault hehehe.
    Cons: - Normally get a lot of resources of your machine to operate
    - Depends on Microsoft Windows
    - Only a few database support implementation
    - Slow performance
    - You can not make http connection you depend of the browser and how to analyze what browser shows you

    AutoHotkey: This is one of the oldest and complete bot programming language. You can programm bots in mutiple ways, you can call dlls and stablish connections using COM wrappers (http headers, http, https, FTP, etc), the documentation is really huge is really fast, you can scrape, use regular expressions, emulate clicks, emulate navigation, easy to learn language, a lot of example on the web, easy to install, multithreading capability. You can make database connection using libraries, Image Wait functions, keyboard clicks emulation.
    Cons: - Windows dependent

    INTELLIGENT BOTS:


    My experience with this bots is using mix between C++ - Autohotkey and the other is using PHP/CURL. With the first one I have the neural network/genetic algorithm running on C++ and autohotkey makes the hardwork (C++ only takes decision based on the variables that autohotkey gave me). With PHP/CURL I use My PHP neural network implementation and get the data using the Curl, I just import the class and use it. What kind of bots can be made with this? well all the bots that require intelligent decisions, probabilistic, fuzzy logic based on experience for example (to give you or not a credit card) to invest or not in a stock, etc...
    Cons: - Neural Network / Genetic algorithm trainning.
    - Artifitial Intelligence Concepts


    At the end, you have to act my friend, this is the first step, which should you choose? depends on many factors... conditions? platforms? budget? time? When you start you will be requiring more features and you will have to jump from one to another.

    DISCLAIMER: I shared what I could have. I can answer few questions related to my knowledge. Dont flame, blame with asking proofs and other ACTUAL Inner details (Those who got brains. They can do stuff themselves after this) and please be respective to all :)

    Let the open discussion be up. Share your questions thanks buttons and answers guys!
     
    • Thanks Thanks x 23
  2. rootkat

    rootkat Registered Member

    Joined:
    Feb 6, 2010
    Messages:
    75
    Likes Received:
    116
    Occupation:
    Cat
    Location:
    3.14159
    This is great introductory information! Thanks. Got any links to full blown bot C++ tutorials?
     
  3. geteasymoneynow

    geteasymoneynow Regular Member

    Joined:
    Jan 17, 2010
    Messages:
    336
    Likes Received:
    290
    Occupation:
    Any Thing
    Location:
    With your girlfriend
    Liked this article really enjoyed it
    and your english is fine
     
  4. hitman247

    hitman247 Executive VIP Premium Member

    Joined:
    Oct 12, 2008
    Messages:
    739
    Likes Received:
    1,851
    Occupation:
    Full Time IM
    Location:
    Your Six O'clock
    Home Page:
    This is a really great information! Thank you for the share!
     
  5. elbeto

    elbeto Registered Member

    Joined:
    May 1, 2009
    Messages:
    98
    Likes Received:
    75
    Occupation:
    Beer drinker
    Location:
    close to the laptop
  6. elbeto

    elbeto Registered Member

    Joined:
    May 1, 2009
    Messages:
    98
    Likes Received:
    75
    Occupation:
    Beer drinker
    Location:
    close to the laptop
    Ey buddy thank you very much, if you use facebook please try my free bot and let me know what you think... The link to the bot is published on this thread.
     
  7. jazzc

    jazzc Moderator Staff Member Moderator Jr. VIP

    Joined:
    Jan 27, 2009
    Messages:
    2,468
    Likes Received:
    10,143
    Really nice article, sums it up :)

    Personally, I code my bots in VB.NET + Watin or VB.NET + Chilikat.

    This way I can handle all situations easily.
     
  8. elbeto

    elbeto Registered Member

    Joined:
    May 1, 2009
    Messages:
    98
    Likes Received:
    75
    Occupation:
    Beer drinker
    Location:
    close to the laptop
    @jazzc: Thanks for your nice comment my friend...
     
  9. stoned raider

    stoned raider Registered Member

    Joined:
    Aug 4, 2008
    Messages:
    94
    Likes Received:
    22
    i usually write bots in php with curl, really easy to do after some training

    also, some time ago, 8-9 months, not sure, someone shared here a great scanned book with really easy tutorials on C# bots , i really like the syntax of C# thou i never had enough time to get to it. i can upload it again if someones interested
     
    • Thanks Thanks x 1
    Last edited: Apr 12, 2010
  10. dearac

    dearac Junior Member

    Joined:
    Nov 15, 2008
    Messages:
    176
    Likes Received:
    165
    I would not mind taking a look at that.
     
  11. stoned raider

    stoned raider Registered Member

    Joined:
    Aug 4, 2008
    Messages:
    94
    Likes Received:
    22
    Code:
    http://rapidshare.com/files/374980933/CSHTTP.pdf.html
    there you go
     
    • Thanks Thanks x 7
  12. La Piovra

    La Piovra Newbie

    Joined:
    Nov 12, 2010
    Messages:
    14
    Likes Received:
    0
    Very good post