Roll your own bots and agents. Automate everything!

bartosimpsonio

Elite Member
Executive VIP
Jr. VIP
Joined
Mar 21, 2013
Messages
24,780
Reaction score
31,549
Interesting post, found via Reddit: http://www.reddit.com/r/SEO/comments/1hrivq/black_hat_challenge_learn_black_hat_seo_and_why/

---

For successful SEO work, computer programming is as important as online marketing itself. Learn the first steps towards creating your own robots, scripts and advanced SEO softwares.


Perl and Python are your friends. Both offer a wealth of free third party libraries for you to roll your own bots. Professional Black Hats love UNIX and all the tools that come with it for free. Mac OS X is one great type of UNIX, so are the various Linux distributions: Ubuntu, Debian, Fedora Core, Slackware and so on.
Microsoft Windows is still the number 1 OS but it is mostly geared towards the end user. Unfortunately, most of the SEO software available today is targetted at Windows, so you'll find yourself programming the .NET or lower level Windows API at some point.
Bot creation is what software developers call "high level programming". That is, you don't meddle with the innards(low level) of the OS in order to create bots. As such, you will likely use a high level language to write your own bots. Examples of high level languages include:

  • Javascript
  • Perl
  • Python
  • Ruby
Other programmable packages include:
  • iMacros for Firefox
  • Rebol
  • Web Macro Recorder for IE
...among others. Most automation follows a certain pattern: you perform a certain action by hand and suddenly notice you're doing it over and over again just changing a certain parameter. This kind of routine can easily be automated using any of the above listed languages/macro packages.
Perl

The Perl programming language is especially powerful as it offers a wealth of third party client libraries via the CPAN.org network. A simple client command such as cpan will give you entrance to thousands of useful WWW automation modules. Here are some of the most useful Perl modules for Web automation:

  • WWW::Mechanize - A complete web bot that can be programmed for anything, filling forms, browsing pages, checking broken links, etc.
  • WWW::Extractor - Extract(scrape) any information from any webpage.
  • LWP::UserAgent - A complete user agent with automatic cookie support, form submission, and capable of being programmed to do anything that can be done with a regular web browser.
Python

On Python Eggs you will find references and links to Python WWW automation resources.
Ruby

There are plenty of Ruby Gems to help you build useful robots in this awesome language. Ruby is the newer language among Python and Perl, and it brings several new concepts imported from the Functional Programming(FP) world. Ruby is an awesome language for all kinds of automation and works in Windows and Linux alike, though software development under UNIX is much more natural than in MS Windows.
iMacros

The iMacros add-on for Firefox is one of your most powerful friends for online automation. WE couldn't possibly exhaust every possibility this package offers, so we refer you to the iMacros wiki.
Under Windows

High level MS Windows programming is done in one of the managed languages available on the .NET framework. You can program in Visual Basic, C# or F# and create powerful software targetted at the Microsoft environment. Luckily some of the .NET software will run on Linux/UNIX by using the Mono open source framework, so you may be lucky in migrating some of your Windows software to UNIX. The Wine emulator is also capable of running some Windows software under UNIX. Some MS Windows SEO tools may be used in this fashion under a better enviroment for development such as that offered by Linux and Mac OS X.
 
I like Python personally. It's almost like programming in plain English!
 
Great post. Lot's of great results are attainable with imacros alone. Learning python on top of that makes sure you can do a lot of stuff surrounding SEO automatically.
 
Interesting article but the actual domain has only been registered 2 days.
Is it yours ?
j/k

W130SN, the master of WWW data!

I noticed that too, it's probably part of that contest. Surprised the contest people didn't register the .net....
 
So for someone looking to learn a first programming language, would you suggest python?

Not only because its easy, but because of all that open source that is out there. Personally, I believe the future in SEO and IM tools is not going to be desktop tools, That is getting old, it will have to be something that can scale.Why scale? because if you ask yourself most IM tools can't even multitask . Literally, they suck at multitasking all they can do is multithreading which is completely different than multitasking

By multitasking I mean, doing more than 1 different type of task at the same time . This is not the same as multithreading that comes with the majority of IM tools that all they do is do one task multiple times at the same time.

So the future in the next generation of tools will require something to be done in either ruby, python, java and perhaps c. It needs to be portable to be able to be ported on any machine. It would most likely be a mixture of SaaS and enterprise IM tools(tools that can scale across multiple machines without needing a GUI on each instance but controlled through a webpanel done in JS like JQUERY)

Learn python, and you could start building distributed seo systems that do more than one thing at the time and are all doing different tasks. Trust me you don't want to go the route of creating desktop SEO tools those are going to go out of style once people start creating more SaaS and enterprise systems, where bots are generally thought as workers doing different tasks at the same time and can be scaled horizontally.
 
Anyone know that coding program that's always being advertised on youtube?

I forgot what the site is called but it definitely sounds interesting.

It's a coding school for dummies. They hold your hand through every step of the way.

Will post when I find it.
 
Anyone know that coding program that's always being advertised on youtube?

I forgot what the site is called but it definitely sounds interesting.

It's a coding school for dummies. They hold your hand through every step of the way.

Will post when I find it.

Codecademy? Khan? Coursera? There are several of 'em...
 
Lots of good information. Knowing a programming language helps a ton in my daily projects. No waiting for some guy to do it for me!
 
Hi


can any one share video tutorials for python I tried in youtube but most of them are short , better if some share video series
 
hi

I found excellent video series for Python

h***p://youtube.c*m/watch?v=4Mf0h3HphEA&feature=share&list=ECEA1FEF17E1E5C0DA

Also I found that there are number of different component available to automate web browser but which is best specially to develop desktop bot ? Any suggestion

Thanks
 
Nice I will add the page to my collection of Good BH information to use in the future.
 
Back
Top