1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

How do I make a web bot?

Discussion in 'General Programming Chat' started by xelex, Nov 7, 2010.

  1. xelex

    xelex Junior Member

    Joined:
    Apr 11, 2009
    Messages:
    161
    Likes Received:
    12
    How do I make my program comunicate with the web?

    I want to code in C++(to practice the language), but can do it in java too.

    Basically this is exactly what I want. To do:

    I want my program to go to a given url/website. Browse to a needed category and then page, start a download from that page in a download manager, then once the download finishes, it goes to that same website, goes to a different category and starts a different download.

    I think I could make the backend for this to work.

    But I need: a way of my program to communicate with the web, and a way for my program to communicate with the download manager.

    any help will be appreciated ))))
     
  2. yousefgreat

    yousefgreat Regular Member

    Joined:
    Nov 14, 2008
    Messages:
    282
    Likes Received:
    180
    I guess uBot can get this done!
     
  3. xelex

    xelex Junior Member

    Joined:
    Apr 11, 2009
    Messages:
    161
    Likes Received:
    12
    I want to make my own in C++ to get the practice :)
     
  4. xpro

    xpro Regular Member

    Joined:
    Jan 21, 2009
    Messages:
    416
    Likes Received:
    16
    For java you can use URLConnection
     
  5. SleepieGirl

    SleepieGirl Regular Member

    Joined:
    Mar 7, 2009
    Messages:
    439
    Likes Received:
    290
    Occupation:
    Need Custom Programs? Message Me!!!
    Location:
    TenDollarBlog.com
    wow really?
    do you know any c++ or java?
    if you know the language you shouldnt have any problem browsing a website...
     
  6. DigitalGangster

    DigitalGangster Regular Member

    Joined:
    Jun 24, 2010
    Messages:
    387
    Likes Received:
    230
    Well i use winhttp.dll on all of my bots. Basically there are 2 ways you can do this. Either using winhttp.dll or using an embedded IE window in your program.
     
  7. madoctopus

    madoctopus Supreme Member

    Joined:
    Apr 4, 2010
    Messages:
    1,249
    Likes Received:
    3,498
    Occupation:
    Full time IM
    1. Learn how to do it
    2. Do it.

    Seriously now, you should ask in a C++ developer forum (maybe on Stack Overflow) if you want useful answers. Also try the C and C++ channels on IRC. Most people here are Internet marketers not C++ experts. Also, you may want to do some googling because the answers are there already. Just find some C++ HTML parsers and for the downloading part, will be easier to eliminate the download manager from your architecture as it won't be easy to use it from within your C++ program unless it has an exposed API. Or just use wget witc supports resuming and can be run from command line, hence interfaced from C++.

    I am not a C++ coder, so do ask on C++ forums and IRC maybe there are better ways.
     
  8. blakamia

    blakamia Junior Member

    Joined:
    Jan 25, 2010
    Messages:
    162
    Likes Received:
    343
    Dump C++. Use C# with WatiN. Vastly superior to most other botting libraries out there. If you're smart you can multithread it too.
     
  9. TheGarden

    TheGarden Registered Member

    Joined:
    May 12, 2010
    Messages:
    69
    Likes Received:
    40
    Occupation:
    Web Developer
    Location:
    UK
    If you're serious abotu scraping the web I wouldn't bother with a storngly typed language like C++, C# or Java. Go for a dynamic language like Python, Ruby, Perl, etc. They're much better suited to this sort of thing.

    Ruby
    • scrubyt.org
    • nokogiri.org

    Python