Create automation software

Hello,

I'm a master student in computer science and instead of buying expensive tools (XRumer, Scrapebox,...) I thought it would be fun to write my own programs. Now I'm wondering which language/tools would be best for this?
I know Java (and C++,...) but honestly I don't know how to start with this since I haven't made a lot of web-based applications yet (only on a local network).
I

i code in vb.net for simplicity but if i would you go for C++

If you are that handy with C++ I would probably use that because you will get the maximum speed and number of threads. I wouldn't use Java. I personally use C# and it works great for HTTP bots. Probably wouldn't be very hard for you to learn. I would stay away from perl or python or any scripting languages, but I suppose that's user preference. I enjoy having a GUI with my threads displayed so I can see exactly what's going on in real time.

First things first, you need to make yourself a raw socket SSL handler class that works with SOCKS proxies. Then you need to thread it. Whatever language this is easiest for you to accomplish is what I would use.

Xrumer was done in delphi, so you are better off to C/Cpp if you know them.
Google "libcurl" for http request library...
and you need a html parser for captcha and scraping and all other shit...

Already thanks for the useful replies. How did you start learning this web-based programming? Are there any books I should be aware of? (or online tutorials) Because SOCKS and SSL handler are not familiar to me (sockets are).

JohnDoe1337 said:

Already thanks for the useful replies. How did you start learning this web-based programming? Are there any books I should be aware of? (or online tutorials) Because SOCKS and SSL handler are not familiar to me (sockets are).

Click to expand...

I don't see many places employinh ssl to handle traffic when it comes to target sites like squidoo and such. So it should be worried about later on...

I have never found any books on this type of thing, since there are few legitimate applications for the knowledge. You'll have to stick to google.

For SSL I would google Org.Mentalis.Security
I use those secure sockets in C#, but I'm not sure what the best option is in C++
If you are familiar with regular sockets and async operations then you should be able to use that.
As for SOCKS, you can just google the protocol and you will find info on how the packets work. If you are planning to create anonymous bots then you are going to need some type of proxy system, so setting up SOCKS connections would be a must.

JohnDoe1337 said:

Hello,

I'm a master student in computer science and instead of buying expensive tools (XRumer, Scrapebox,...) I thought it would be fun to write my own programs. Now I'm wondering which language/tools would be best for this?
I know Java (and C++,...) but honestly I don't know how to start with this since I haven't made a lot of web-based applications yet (only on a local network).
I

Click to expand...

Come work for me and develop one of my needs that we could co-market together under my brandname. If you were able to put together this toy I have in my head I could market it under the HatnoHat label and I'd pay you 70% of all sales. It's not an backlinking software, but it is something I need for my marketing strategies and I'd like it to be a windows based software with connection capabilities.

I've developed some killer tools using 3 things: perl, wGet, and cUrl. Since wget and curl are basically the same, that's two things.

The combination of these tools can do everything that imacros for firefox does, and for that matter xrumer. Once you learn how to GET with wGet and POST with curl, you can do a lot of the same stuff. Of course you also need a few proxies if youre going to be using google for scraping. And learning what an API is and what command line functionality does helps too. Here's how it has worked for me several times:

I wget the target url, which returns the raw html of the web address. I then analyze it with perl, and then I have the values I am looking for. I then have the code write a POST request and then I run the post request, posting whatever it is I am looking for. With perl, you can set loops upon loops of get requests and analyze just about anything, and then by using post, you communicate with the target server. Perl organizes it all and analyzes the raw data.

gnote said:

I would stay away from perl or python or any scripting languages, but I suppose that's user preference. I enjoy having a GUI with my threads displayed so I can see exactly what's going on in real time.

Click to expand...

Personally I would recommend the complete opposite! You can't beat the rapid development you get with a scripting language (I use Python and PHP), and for real industrial strength robustness you want to be running from the command line on Unix. Plus if you are programmer, open source *nix is a joy to work with.

You want to choose a language that makes it easy/quick to run multiple threads...

I guess I'm in the minority here.. As a Java programmer by trade (13 years paid java programming experience), I of course use Java

I built some craigslist bots a few weeks ago using winautomation, and I'm currently re-building them using java.. and I've been building scrapers with java since 2001.

i use vb.net, its easy and effective. I dont understand why most people don't like vb.net

Autumn said:

Personally I would recommend the complete opposite! You can't beat the rapid development you get with a scripting language (I use Python and PHP), and for real industrial strength robustness you want to be running from the command line on Unix. Plus if you are programmer, open source *nix is a joy to work with.

Click to expand...

I agree on this one.Also there is softwares that make python code run up to 30x faster.

Basically one point to say is that if it wasnt for python there wouldnt be for SEO and Internet Marketing(Google baby)

Scripting was never very useful to me for more complicated stuff or if you want to sell your creation. But that's just my experience.

What's more important is that most tools (XRumer, Scrapebox to quote the OP) are not web based applications. They are desktop applications that do stuff connecting to the internet. A web based applications is something that in most simple terms does stuff in a user's browser and additionaly needs a supporting server. Examples are interactive games, chats, statistical analysis etc. A php developer must by default create such apps: he codes something and must use a webserver and a browser to get it working. Another such language is ASP. As far as I remember Python can handle both desktop and web based apps.

Pick some easy language, Java's fine, C# or VB even better due to the countless code samples clogging the dev forums. The speed is irrelevant as the limiting factor is usually the actual connection bandwidth.

C easy language?
Python is easy but C is hard and complicated especially error handling

I don't see much details in your question. Anyways applications are three types

web based
Rich Internet Apps (ex : market samurai) which rely mainly on webservices
Desktop Apps

I understand as a starter how you would feel lost. Pick up a framework. Start working. I haven't mentioned which framework and which language since I don't know much about your application or your coding abilities.

Why C++ ? I mean yeah you can achieve anything using C++ but there are faster ways to code. That's what agile or RAD about. I would pick C++ only if I'm writing a Native app (ex : anti virus software , registry fixers etc , partitioning software). Start looking into WPF from Microsoft. You could also try WinForms. If you are planning for both win and mac , consider Mono (no wpf support yet). Sorry have zero experience with other languages / platform.

In general Java is good for coding transaction systems where the business logic is really complex. Ex : Trading Software , Banking Websites etc. Just my 2 cents. As always everyone has their own opinion and few might not agree with above. Good luck. Just start somewhere. The good thing about writing your own program is in the long run you end with great ideas. Ideas + Coding Abilities = $$.

Just two years back , I sold a simple CRM system for one client. Made around 12k that year. I took that project at skool then continued to work on it for one summer. I used win forms and c#. The client wanted to develop further but I work full time so can't take up 2nd employment. The reason I'm saying is because instead of putting lotta efforts in making a IM software you could make 10X with one proper business app and one client.

Python is easy and quick and clean.
The python library is powerful and you could get everything you want.
More additionally, GAE (Google App Engine) is free and it supports python/java.
So, trust python.

I totally agree with wu1239 but when it comes to OOP I really prefer Java and I started programming a bot in Java about 2 weeks ago.
Here I gave a litte advice on how I got started with HTTP programming in java
http://www.blackhatworld.com/blackhat-seo/other-languages/295237-java-bots.html#post3198516

I would suggest Delphi. It's fast as C++ , and easy enough .....
I use it for my bots and I can make 2 bots per day with it ..... Thats pretty fast!

I use it for my bots and I can make 2 bots per day with it ..... Thats pretty fast!

Click to expand...

That depends on how complex these bots tasks are