1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Create automation software

Discussion in 'General Programming Chat' started by JohnDoe1337, Mar 21, 2011.

  1. JohnDoe1337

    JohnDoe1337 Registered Member

    Joined:
    Feb 17, 2011
    Messages:
    59
    Likes Received:
    1
    Hello,

    I'm a master student in computer science and instead of buying expensive tools (XRumer, Scrapebox,...) I thought it would be fun to write my own programs. Now I'm wondering which language/tools would be best for this?
    I know Java (and C++,...) but honestly I don't know how to start with this since I haven't made a lot of web-based applications yet (only on a local network).
    I
     
  2. specopkirbs

    specopkirbs BANNED BANNED

    Joined:
    Nov 28, 2008
    Messages:
    920
    Likes Received:
    746
    i code in vb.net for simplicity but if i would you go for C++
     
  3. gnote

    gnote Registered Member

    Joined:
    Mar 10, 2009
    Messages:
    80
    Likes Received:
    6
    Occupation:
    Programmer
    Location:
    USA
    If you are that handy with C++ I would probably use that because you will get the maximum speed and number of threads. I wouldn't use Java. I personally use C# and it works great for HTTP bots. Probably wouldn't be very hard for you to learn. I would stay away from perl or python or any scripting languages, but I suppose that's user preference. I enjoy having a GUI with my threads displayed so I can see exactly what's going on in real time.

    First things first, you need to make yourself a raw socket SSL handler class that works with SOCKS proxies. Then you need to thread it. Whatever language this is easiest for you to accomplish is what I would use.
     
  4. satyawrat

    satyawrat Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2009
    Messages:
    922
    Likes Received:
    1,181
    Occupation:
    Hustler
    Location:
    Gurgaon
    Home Page:
    Xrumer was done in delphi, so you are better off to C/Cpp if you know them.
    Google "libcurl" for http request library...
    and you need a html parser for captcha and scraping and all other shit...
     
  5. JohnDoe1337

    JohnDoe1337 Registered Member

    Joined:
    Feb 17, 2011
    Messages:
    59
    Likes Received:
    1
    Already thanks for the useful replies. How did you start learning this web-based programming? Are there any books I should be aware of? (or online tutorials) Because SOCKS and SSL handler are not familiar to me (sockets are).
     
  6. satyawrat

    satyawrat Jr. VIP Jr. VIP

    Joined:
    Jul 8, 2009
    Messages:
    922
    Likes Received:
    1,181
    Occupation:
    Hustler
    Location:
    Gurgaon
    Home Page:
    I don't see many places employinh ssl to handle traffic when it comes to target sites like squidoo and such. So it should be worried about later on...
     
  7. gnote

    gnote Registered Member

    Joined:
    Mar 10, 2009
    Messages:
    80
    Likes Received:
    6
    Occupation:
    Programmer
    Location:
    USA
    I have never found any books on this type of thing, since there are few legitimate applications for the knowledge. You'll have to stick to google.

    For SSL I would google Org.Mentalis.Security
    I use those secure sockets in C#, but I'm not sure what the best option is in C++
    If you are familiar with regular sockets and async operations then you should be able to use that.
    As for SOCKS, you can just google the protocol and you will find info on how the packets work. If you are planning to create anonymous bots then you are going to need some type of proxy system, so setting up SOCKS connections would be a must.
     
  8. adbox

    adbox Power Member

    Joined:
    May 1, 2009
    Messages:
    658
    Likes Received:
    107
    Home Page:
    Come work for me and develop one of my needs that we could co-market together under my brandname. If you were able to put together this toy I have in my head I could market it under the HatnoHat label and I'd pay you 70% of all sales. It's not an backlinking software, but it is something I need for my marketing strategies and I'd like it to be a windows based software with connection capabilities. :rolleyes:
     
  9. NTG98

    NTG98 Junior Member

    Joined:
    May 27, 2009
    Messages:
    128
    Likes Received:
    12
    Occupation:
    Student
    Location:
    Texas
    I've developed some killer tools using 3 things: perl, wGet, and cUrl. Since wget and curl are basically the same, that's two things.

    The combination of these tools can do everything that imacros for firefox does, and for that matter xrumer. Once you learn how to GET with wGet and POST with curl, you can do a lot of the same stuff. Of course you also need a few proxies if youre going to be using google for scraping. And learning what an API is and what command line functionality does helps too. Here's how it has worked for me several times:

    I wget the target url, which returns the raw html of the web address. I then analyze it with perl, and then I have the values I am looking for. I then have the code write a POST request and then I run the post request, posting whatever it is I am looking for. With perl, you can set loops upon loops of get requests and analyze just about anything, and then by using post, you communicate with the target server. Perl organizes it all and analyzes the raw data.
     
  10. Autumn

    Autumn Elite Member

    Joined:
    Nov 18, 2010
    Messages:
    2,197
    Likes Received:
    3,041
    Occupation:
    I figure out ways to make money online and then au
    Location:
    Spamville
    Personally I would recommend the complete opposite! You can't beat the rapid development you get with a scripting language (I use Python and PHP), and for real industrial strength robustness you want to be running from the command line on Unix. Plus if you are programmer, open source *nix is a joy to work with.
     
    • Thanks Thanks x 1
  11. jason-l

    jason-l Regular Member

    Joined:
    Feb 21, 2011
    Messages:
    281
    Likes Received:
    83
    You want to choose a language that makes it easy/quick to run multiple threads...

    I guess I'm in the minority here.. As a Java programmer by trade (13 years paid java programming experience), I of course use Java :)

    I built some craigslist bots a few weeks ago using winautomation, and I'm currently re-building them using java.. and I've been building scrapers with java since 2001.
     
  12. jangedoo

    jangedoo Registered Member

    Joined:
    Jan 12, 2010
    Messages:
    59
    Likes Received:
    41
    i use vb.net, its easy and effective. I dont understand why most people don't like vb.net
     
  13. wes_one

    wes_one Junior Member

    Joined:
    Apr 25, 2011
    Messages:
    176
    Likes Received:
    75

    I agree on this one.Also there is softwares that make python code run up to 30x faster.

    Basically one point to say is that if it wasnt for python there wouldnt be for SEO and Internet Marketing(Google baby)
     
  14. Monrox

    Monrox Power Member

    Joined:
    Apr 9, 2010
    Messages:
    615
    Likes Received:
    579
    Scripting was never very useful to me for more complicated stuff or if you want to sell your creation. But that's just my experience.

    What's more important is that most tools (XRumer, Scrapebox to quote the OP) are not web based applications. They are desktop applications that do stuff connecting to the internet. A web based applications is something that in most simple terms does stuff in a user's browser and additionaly needs a supporting server. Examples are interactive games, chats, statistical analysis etc. A php developer must by default create such apps: he codes something and must use a webserver and a browser to get it working. Another such language is ASP. As far as I remember Python can handle both desktop and web based apps.

    Pick some easy language, Java's fine, C# or VB even better due to the countless code samples clogging the dev forums. The speed is irrelevant as the limiting factor is usually the actual connection bandwidth.
     
  15. wes_one

    wes_one Junior Member

    Joined:
    Apr 25, 2011
    Messages:
    176
    Likes Received:
    75
    C easy language?
    Python is easy but C is hard and complicated especially error handling
     
  16. I don't see much details in your question. Anyways applications are three types

    web based
    Rich Internet Apps (ex : market samurai) which rely mainly on webservices
    Desktop Apps

    I understand as a starter how you would feel lost. Pick up a framework. Start working. I haven't mentioned which framework and which language since I don't know much about your application or your coding abilities.

    Why C++ ? I mean yeah you can achieve anything using C++ but there are faster ways to code. That's what agile or RAD about. I would pick C++ only if I'm writing a Native app (ex : anti virus software , registry fixers etc , partitioning software). Start looking into WPF from Microsoft. You could also try WinForms. If you are planning for both win and mac , consider Mono (no wpf support yet). Sorry have zero experience with other languages / platform.

    In general Java is good for coding transaction systems where the business logic is really complex. Ex : Trading Software , Banking Websites etc. Just my 2 cents. As always everyone has their own opinion and few might not agree with above. Good luck. Just start somewhere. The good thing about writing your own program is in the long run you end with great ideas. Ideas + Coding Abilities = $$.

    Just two years back , I sold a simple CRM system for one client. Made around 12k that year. I took that project at skool then continued to work on it for one summer. I used win forms and c#. The client wanted to develop further but I work full time so can't take up 2nd employment. The reason I'm saying is because instead of putting lotta efforts in making a IM software you could make 10X with one proper business app and one client.
     
    • Thanks Thanks x 1
    Last edited by a moderator: May 24, 2011
  17. wu1239

    wu1239 Newbie

    Joined:
    Jun 4, 2011
    Messages:
    16
    Likes Received:
    0
    Python is easy and quick and clean.
    The python library is powerful and you could get everything you want.
    More additionally, GAE (Google App Engine) is free and it supports python/java.
    So, trust python.
     
  18. plumbum416

    plumbum416 Registered Member

    Joined:
    Mar 17, 2011
    Messages:
    93
    Likes Received:
    16
  19. moundown

    moundown BANNED BANNED

    Joined:
    Jun 24, 2010
    Messages:
    118
    Likes Received:
    32
    I would suggest Delphi. It's fast as C++ , and easy enough .....
    I use it for my bots and I can make 2 bots per day with it :p ..... Thats pretty fast!
     
  20. plumbum416

    plumbum416 Registered Member

    Joined:
    Mar 17, 2011
    Messages:
    93
    Likes Received:
    16
    That depends on how complex these bots tasks are ;)