1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Learn how to code with seo20 | Lesson 1

Discussion in 'General Programming Chat' started by SEO20, Aug 6, 2013.

  1. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    Mjellooooo


    Welcome to my little code school where I hope to learn none-programmers to do simpler programming tasks to ease their daily IM life.
    I will try to keep it short and to the point.
    All lessons will be using blazing fast "Node js" as programming languge, which is OS independent :)


    ----------
    Step 1
    ----------
    Download and install a simple code-editor
    http://www.sublimetext.com


    ----------
    Step 2
    ----------
    Install Node JS on your local computer:
    http://nodejs.org -> press "install" and run :)


    ----------
    Step 3
    ----------
    Create a new folder easy to find on PC "c:\nodejs" would be a fine idea.


    ----------
    Step 4
    ----------
    Start a command-prompt and get into the newly created folder
    Code:
    c:
    cd nodejs
    

    ----------
    Step 5
    ----------
    Let's install a simple package we will be using på writen the folowing line:


    Code:
    npm install cheerio
    

    A lot more can be found here: https://npmjs.org




    We are now all setup to start coding with the the node and the installed package "cheerio".
    Cheerio is a Tiny, fast, and elegant implementation of core jQuery designed specifically for the server.




    ----------
    Step 6
    ----------
    In our nodejs folder create a file called cnn.js
    Open the file in your editor and paste this code:


    Code:
    var cheerio = require('cheerio');
    var http = require('http');
    
    
    var url ="http://edition.cnn.com/";
    
    
    function getIt(myData) {
        http.request(url).on('response', function(response) {
            var data = '';
            response.on("data", function (chunk) {
                data += chunk;
            });
            response.on('end', function () {
                myData(data);
            });
            }).end();
    }
    
    
    getIt(function(theData) {
        //console.log(theData);
        $ = cheerio.load(theData);
    
    
        $('ul.cnn_bulletbin li a').each(function() {
            console.log(this.text());        
            console.log('   ' + $(this).attr('href'));
        })
    });
    

    The code will scrape CNN's front page for headlines and their links.
    Before I go into how it works etc - let's see it running.


    In the nodejs folder write the following command to run your cnn.js code.


    Code:
    node cnn.js
    

    ----------
    Step 7
    ----------
    You have just created your first scraper :)


    All magic is in this line:
    Code:
    $('ul.cnn_bulletbin li a').each(function()
    

    It selects alle "UL" HTML-elements that has the class "cnn_bulletbin" select all those "li" elements and look for all "a" elements.
    console.log outputs the result to the screen.


    In next lession I will go into more details and take it a step further if people would like that.


    Enjoy
     
    • Thanks Thanks x 52
  2. morgan456

    morgan456 Jr. VIP Jr. VIP Premium Member

    Joined:
    Nov 26, 2010
    Messages:
    2,112
    Likes Received:
    247
    Gender:
    Male
    Occupation:
    SEO company
    Welcome you for sharing the information...! Thanks a lots dude..!
     
    • Thanks Thanks x 1
  3. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    Glad to help :)
     
  4. johnland

    johnland Junior Member

    Joined:
    Jul 12, 2010
    Messages:
    134
    Likes Received:
    29
    Thanks a lot for this, will love to see more details in your next lesson!
     
    • Thanks Thanks x 1
  5. umerjutt00

    umerjutt00 Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 28, 2011
    Messages:
    3,644
    Likes Received:
    1,904
    Occupation:
    Ninja
  6. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    So did you got it to run and all?
     
  7. dr.Hus

    dr.Hus Regular Member

    Joined:
    Oct 24, 2011
    Messages:
    265
    Likes Received:
    140
    Occupation:
    web design
    Location:
    www
    im still at work, will try as soon as i get to pc ... thanks for your time and effort Seo20
     
    • Thanks Thanks x 1
  8. HackZu

    HackZu Junior Member

    Joined:
    Nov 6, 2009
    Messages:
    115
    Likes Received:
    53
    Watching closely, amazingly fast. Following the example in the OP, I got all the URls including title in like 3 seconds. This comes in very handy for a project of mine! Repped
     
    • Thanks Thanks x 1
  9. phracktl

    phracktl Regular Member

    Joined:
    Sep 2, 2012
    Messages:
    218
    Likes Received:
    152
    Fantastic OP...you're really channelling the spirit of what computing was once all about -> empowering people

    Keep it coming - its really appreciated
     
    • Thanks Thanks x 2
  10. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    Big words - thanks.
     
  11. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    Thanks for the rep guys - node js is pretty awesome :)
     
  12. soothsayerpg

    soothsayerpg Power Member

    Joined:
    Feb 23, 2011
    Messages:
    584
    Likes Received:
    225
    Home Page:
    Already sub to this OP. Great Share!
     
  13. Asif WILSON Khan

    Asif WILSON Khan Executive VIP Premium Member

    Joined:
    Nov 10, 2012
    Messages:
    10,112
    Likes Received:
    28,543
    Gender:
    Male
    Occupation:
    Fun Lovin' Criminal
    Location:
    London
    Home Page:
    This looks like it could be a great little course. OP thanks and repped.
     
    • Thanks Thanks x 1
  14. SEO20

    SEO20 Elite Member

    Joined:
    Mar 25, 2009
    Messages:
    2,017
    Likes Received:
    2,259
    Any suggestions for next lesson? Thinking that it still should be scraping - maybe going through a list of URLs and scrape them for proxies?
    Ideas are welcome...
     
  15. slim_dusty

    slim_dusty Jr. VIP Jr. VIP Premium Member

    Joined:
    Jun 5, 2011
    Messages:
    392
    Likes Received:
    115
    Location:
    Middle earth
    Thanks seo20,

    I've been using seo tools for a while now, but have absolutely no programming knowledge. Programming has always been a little intimidating, don't know where to get started. It's like enjoying driving fast cars, and now finally being able to 'look under the hood'!

    Keep up the great work.
     
    • Thanks Thanks x 1
  16. christianbed

    christianbed Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 17, 2011
    Messages:
    1,341
    Likes Received:
    879
    Location:
    alert("Make Money")
    Home Page:
    Pretty cool...if I did it correctly, it spit out all the urls in cmd. For a total noob, knowing which of the three places to input code (cmd, nodejs, sublime text) is a challenge in itself. Anyway, it clearly displayed all the cnn headlines in the command prompt interpreter. Amidoingitrite?
     
  17. WORK@HOME

    WORK@HOME Senior Member

    Joined:
    Apr 25, 2013
    Messages:
    804
    Likes Received:
    362
    Location:
    Right Here
    Home Page:
    Nice tutorial.I look forward to more.
     
  18. igagcom

    igagcom Newbie

    Joined:
    May 21, 2013
    Messages:
    29
    Likes Received:
    4
    Man you are great! Tried python and many other things. It's never been so easy! Thank you very much.
     
    • Thanks Thanks x 1
  19. Kingu Buraddorei

    Kingu Buraddorei Junior Member

    Joined:
    Aug 4, 2013
    Messages:
    151
    Likes Received:
    60
    Exactly what I was looking for! Thanks seo20!
     
  20. fatjack'sblackhat

    fatjack'sblackhat Jr. VIP Jr. VIP Premium Member

    Joined:
    Jul 16, 2008
    Messages:
    611
    Likes Received:
    129
    Occupation:
    college student
    Location:
    Florida
    Please keep this thread up. Ive been looking for an activity to do like this to learn programming for a long time and the best part about this is its all real time so we can move along together as a group and help eachother! Thanks seo20