1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Tutorials: Python

Discussion in 'General Scripting Chat' started by Baybo.it, Aug 11, 2011.

  1. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    I'm an open source hacker by trade and I love sharing knowledge. I thought I'd start a thread about the cool things you can do using the Python language.

    I'll start off with some simple examples and later get into web frameworks, databasing, and eventually build an entire frameworked website. Questions are welcomed!

    First, how is Python installed?
    1. On Linux and Mac, it should be straight forward -- Python should be installed already on your machine.
    2. Windows is almost as simple. You go to python . org /getit and install the 2.7.2 executable

    How do you get started
    1. In Linux or OSX, simply open up a shell / terminal and type the word 'python' with no quotes and press enter.
    2. On Windows, you can either find Python 2.7.2 in your start menu, or open your good old cmd shell and run the Python program. Usually it's installed directly at the C: level and is easy to find.
    If there are any questions on getting started, just let me know. I'm happy to explain.

    Why Python

    So here's a good question. Why Python? Many of the folks here seem to love the . net frameworks, which is well and good, but many factors should go into choosing a programming language: robustness, paradigm, speed, complexity, openness, API, potential for cross-platform development, interoperability with other languages, and the overall popularity of the language. Python is not extremely fast, that's alright, because there are several extremely optimized frameworks for python development, like Django, webpy, and pylons.

    Why do I use Python? I have been programming for about eleven years and have tried C, C++, common lisp, scheme, clojure, java, c#, vb6, vbnet, javascript and nodejs, OCaml, python, perl, ruby, php, bash as well as a variety of other languages. In fact, I've written a lightweight language of my own based on OCaml for functional programming. I've learned first hand that different languages have different advantages.

    The JVM is great because it is cross-platform, and thus it is worth knowing a language like Java, Scala, or Clojure which compiles to JVM byte code. If you're writing something which has to be highly optimized and support million of concurrent users or threats, C has its obvious advantages with low level memory management and fine tune-able MPI. If you're writing a simple administrative script for your shell, might as well code it in bash or perl. But what about web apps?

    There's a reason I enjoy using Python for web apps, and it's not because ruby, php, perl, and asp can't accomplish similar tasks. I find Python to be the simplest with the cleanest conventions and syntax, the most powerful libraries, and the most developed web frameworks. Also, python is multi-paradigm which gives the programmer the ability to choose his or her own style without being restricted, be it functional, imperative, or strictly object oriented. I'll show you exactly how easy Python is to use.

    Objective: Generate a list of the numbers between 50 and 100 and print them in steps of 5.

    Code:
    for i in range(50,101)[::5]: print i
    
    While it would be best syntactic practice to put 'print i' on a new line with proper indentation, this is a pretty compelling example of how much you can accomplish with very little code in python. Not only that, but the code makes sense. By the end of this entire tutorial, I hope people will be able to launch their own full-scale website. After all, the technology I will be discussing (webpy, open source and written by Aaron Swartz) was originally designed as the back-end framework for reddit and held up to thousands of users, very well. For now, let's move on to the basics!

    Writing Something Useful
    Let's write a practical program in python that will actually accomplish something useful. How about defining a simple list?

    Code:
    x = [1,2,3,4]
    y = ["hi", "there", "how", "are", "you"]
    
    Notice we don't have to statically type any of our variables. We don't need to specify that x or y are a lists and furthermore, we don't have to tell the interpreter what kind of list x an y are. Meaning, we don't have to specify x is a list of type Integer and y is a list of type string (or char*, a character array if you're coming from C).

    This is one huge advantage of a language like Python. It is dynamically typed like perl (although perl uses sigils to represent $ scalars, at symbol for arrays, and % for associative arrays) -- php does similar, ruby, and many other scripting languages.

    What about iterating over a list? In Java or c++, your code would look something like this:

    Code:
    for (int x=0; x<100; x++) {
        cout << x << endl; // for c++, assume std libs included
        System.out.println(x); // java
    }
    
    For python, loops automatically iterate over lists and grab one element from the list at a type (assuming the list is populated with only atoms and no sublists or tuples)

    Code:
    mylist = range (100) # mylist is a list from 0 to 99, [0,1,2,...,98,99]
    mysum = 0
    for x in mylist:
        mysum += x
    print mysum
    
    This is a very simply program which could have been written more compactly in one tiny line using a map reduce list comprehension as:

    Code:
    sum(range(100))
    
    but the point is just to illustrate how the problem would be solved sequentially.

    So far we've gone over the basics of the language, what the syntax looks like, why python may be a good choice for your next web project, and hopefully you're excited about its capabilities.

    In my next post I hope to introduce dictionaries, explain the syntax further (for example, indentation does matter in python), and talk more about the type system.

    Comments, suggestions, feedback, and questions -- as always -- would be greatly appreciated. Hope you learned something useful!
     
    • Thanks Thanks x 17
    Last edited: Aug 12, 2011
  2. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Data Structures in Python
    In the last section of the tutorial, we discussed lists in python, or arrays. Arrays are systematically arranged blocks of memory which can store values at designated positions. There are many advantages to using arrays. For instance, retrieving a value from an array is very fast as the value can be reached directly based on a supplied position, without any heavy computation required for lookup.

    Imagine we have a list of 5 numbers, [1,2,3,4,5]. If we wanted to access the 0th element (computers start indexing at 0 because 0 is the smallest number a computer can represent using unsigned binary) then we'd only have to examine one value in the array, namely the block containing the value 1. For this reason, the worst case scenario for retrieving a value is that one memory location will have to be accessed. We formalize this by saying, retrieval or replacement of a single value within an array can be done, in the very worst case scenario, in one step, or in O(1).

    Let me explain what I mean when I say O(1) or big O of 1. Because not all computer programs deal with small lists, (think about how many data points facebook computes a day, for instance), we must think carefully about the properties of our data structures. How expensive is it to replace an element, to look up an element, to insert an element? We express the solution to these questions in a notation called Big O. The simplest way to think of big O notation is as follows: a big O value represents the WORST case scenario concerning how many items within a list or collection of items must be moved, repositioned, or modified in order for some operation to be performed. And what do I mean by operation? Here's an example, how many elements do we need to move in the list [1,3,2,4] for every item to be in ascending order? In this case, only 2 items have to be moved: the value 2 and 3 which have to be swapped. However, we can do better than this. We can use big O notation to express the worst case scenario possible (e.g. the maximum number of changes) to sort any list of size 'n' in ascending order. Well, let's reason this. What IS the worst case scenario for this job? The worst case scenario would mean that the list is ordered in such a way that the maximum number of elements are all in the wrong places. Given this definition, we can reason that any list which is in fully descending order is the worst case scenario because our objective is to turn the list into ascending order. With a list in descending order, EVERY element in the entire list of size 'n' will have to be repositioned. However, the important thing to note is, there is no ordering which will perform worse than the list we just described. Therefore, we can classify a worst case for ordering a list in ascending order. The solution is, the complexity classification (the worst case scenario, or big O) of ordering a list is 'n'. N items have to be moved, in the worst possible case.

    What can we learn from this about the array data structure? For arrays, the big O of inserting items into a list is 'n'. The worst scenario is the item will be added to the very beginning of the list, causing all 'n' items in the list to change their positions over one to fit the new element. This may not seem bad, but imagine inserting several values into lists which are millions of elements long.

    In the next tutorial, I'll describe some other data structures, like associate arrays / dictionaries, and linked lists. I'll explain the advantages of each, how what's unique about how they can be used in python.

    Like I promised in my original post, I'd like to explain a little more about Python's syntax so you can start writing more powerful programs of your own. In Python, indentation is used instead of brackets, and yes -- proper indentation does matter. This may seem a hindrance of your expressiveness to program, but as a seasoned programmer (having had to struggle with many unmaintainable, spaghetti code perl scripts) I have come to appreciate that Python's clean programming conventions and syntax help build easy-to-read, easy-to-write code.

    In order to understand python, we must first recall the difference between a compiled and an interpreted programming language. A compiled language, like C, takes human written source code and compiles it to machine code (all at once, in one process). Although python can be 'compiled' into pyc code, python is most commonly interpreted. What does this mean?

    An interpreted language behaves very similar to any command line you've ever used. Maybe you used a command line interface while interacting with an ATM at the bank, or maybe you use a command line all the time on your computer. Instead of reading the entire program at once and generating a machine representation which can be executed, Python reads your program line by line and interprets each step. It uses indentations as signals that different lines of codes should be grouped together and treated as similar entities (to learn more, look up programming type systems and scoping).

    That said, there are two ways to program in Python. First, you can interact directly with Python in the interactive command line environment called the REPL (read evaluate print loop). The REPL will read your command, evaluate it, print the result to your screen, and then loop back and wait for more input to be read in. REPLs are really convenient because you can program short programs very quickly without having to recompile every time you hit a little bug.

    But is this practical for a large program? How can you save big programs? You can actually type Python code in a text file and have the Python program process the entire text file line by line. In fact, this is traditionally how python code is executed. Just remember, even when you provide Python with an entire program file, it is still reading every line in the file, one by one, just as if you were typing each line at the REPL command line interface. This is a clear difference between interpreted and compiled languages.

    Because python is interpreted, it can do some pretty awesome things, like dynamic typing. Unlike C or Java where you have to tell the compiler what type of data type you would like to use (for instance, telling the compiler you want x to be an int or y to be a string) the interpreter can figure it out by using context.

    It seems we now know all the basics, we can continue with learning Python syntax. In case you're new to programming, syntax means the rules for a programming language, as opposed to semantics which are the meaning. In the English language, using an apostrophe in "don't" would be syntax, as would using an exclamation mark in a sentence to mean excitement! Semantics, on the other hand, determine meaning. If I were to say, "something mysterious is a foot", do I really mean an actual human foot? Okay, back to the syntax for python.

    To get started, all you have to do is enter a python command at the REPL:

    Code:
    5
    
    Notice that python acts very much like a calculator. What happens when you type the number 5 into a calculator and then press enter? You'll get 5 right back! That means numbers in python evaluate to themselves. Similarly...

    Code:
    5+5
    
    This evaluates to 10, like you'd expect.

    When doing conditionals in Python, we need to be careful with syntax
    Code:
    x = 10
    if x == 10:
        print "hi!"
    
    In the above code, the interpreter will set x=10 and remember it in memory. In the next line, there is a conditional which asks if x is equal to the value 10, and guess what, it is! But wait, what is with that colon. And how about the indentation? In python, you can use tabs or spaces for indentation but you need to be consistent with your spacing throughout your program or it will break. In the above code, the colon tells Python that a block of code is starting and a new context should be made. The result of this code is that the word "hi" will be printed to your screen (the test condition x == 10 turned out to be True and passed!)

    That's all for now. Next post will deal with data structures in greater depth.
     
    • Thanks Thanks x 3
    Last edited: Aug 11, 2011
  3. isabellamelendez

    isabellamelendez Regular Member

    Joined:
    Apr 24, 2011
    Messages:
    394
    Likes Received:
    81
    yo subscribed, i really would love to learn python. :)
     
  4. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Glad to hear it Isabella! If you need more information on getting python set up, I'd suggest python . org/about/gettingstarted/ and wiki.python. org/moin/BeginnersGuide. Also, if you're looking for some code which is a bit more advanced and practical which you could run, you can check out this Python web crawler code.activestate .co m/recipes/576551/ (remove all spaces in links)

    Tomorrow I hope to continue the tutorial.

    Some things I've programmed before and hope to cover later:
    1. Connecting to Twitter an Facebook via OAuth2
    2. Writing a website over webpy and apache
    3. Writing custom web crawlers to scrape emails
    4. Creating and using RESTful APIs over HTTP

    If there's any interest in one of these four concepts in particular, let me know.
     
    • Thanks Thanks x 1
    Last edited: Aug 11, 2011
  5. ~divinci

    ~divinci Registered Member

    Joined:
    Sep 23, 2009
    Messages:
    51
    Likes Received:
    11
    Occupation:
    Infastructure Reverse Engineer
    Location:
    Liverpool UK
    Hey M,

    Nice tutorials :) i am a c/c# man myself but thought i would let you know something that I heard from a colleague regarding Python.

    A guy called Dubno recently joined Bank of America who are paying him a couple of million a year. He is doing at BOA what he was paid to do at JPMorgan which was

    "led a team that designed and built a software platform to unify trading, risk and pricing applications for foreign exchange options, equities and commodities"

    So my mate who works for the bank was telling me that Dubno met all developers accross BOA in face to face meetings (lots of visits accross the world) and told them that to unify all their existing systems - they would have to use Python as the 'middleware' language!!

    Funny ey! So i guess that proves your love for the language is rightly so :)

    www . banktech . com / management-strategies / 224000052

    Added you on TWIT mate
     
  6. haxxaruz

    haxxaruz Registered Member

    Joined:
    Jun 12, 2011
    Messages:
    81
    Likes Received:
    19
    first of all, I really appreciate you doing such a tutorial. I'm a python lover myself; so if you're stuck into something just msg me. :) I would kindly try to provide you with answers. how long have you been into python?
    ooh really?
     
  7. syngenetic

    syngenetic Guest

    I want to focus on learning C++ and Python, but most universities in Georgia are teaching Java as their primary language. So I try to stick with a language that will help me in Georgia Tech and etc.
     
  8. ~divinci

    ~divinci Registered Member

    Joined:
    Sep 23, 2009
    Messages:
    51
    Likes Received:
    11
    Occupation:
    Infastructure Reverse Engineer
    Location:
    Liverpool UK
    lol good spot
     
  9. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Thread, sorry. ;o)
     
  10. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Syngenetic, I used C and Java for a handful of years for my research involving software verification and multi-threaded concurrent searching. I can understand wanting to focus on technologies that will help you in school. I guess it depends what your overall goal is -- as I suggested, some languages are better suited for specific tasks.

    Writing an entire web framework, or using an existing web server like Jetty, could also work for writing a scalable website, but that is not a strategy I would personally recommend, unless you have experience with other JVM languages like Clojure. Really, I think Clojure is the language of the future, however, the fact that it is a functional language and a lisp turns off a lot of people who are used to the imperative mindset.

    I'd be happy to start a discussion about why I specifically don't like Java as a language, but I think it will suffice to say learning Python and Java are not mutually exclusive. Learning multiple languages is important because most employers and and schools don't focus on learning languages within their institutions. They expect you to learn programming on your own. A good computer science class will instead focus on computational complexity analysis, algorithm design and classification, and data structure implementation.

    In order to be well rounded, I suggest you learn a statically typed, imperative, compiled (or byte compiled language) like C, C++, or Java, and also learn a dynamically typed, interpreted language like python, php, perl, or ruby. You should use what language is right for you, however, you should take time to educate yourself on the advantages and disadvantages of each language. Choosing the right language determines how far you can take a problem towards a solution. If the code becomes so mangled that you are having trouble making process, or if there aren't enough open APIs to continue developing, your project may lose momentum.

    Maybe a good exercise would be to take problems you've previously solved in Java and try to solve them in python or ruby as well. Or take a brand new problem and solve them both in java and python, side by side. Compare them and learn about the differences.

    Becoming immersed in the hacker mentality is the best way to successfully learn computer science. Good luck and hope these tips have been helpful!
     
  11. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    RESTful APIs
    There's been a lot of buzz recently about RESTful APIs. But what does it all mean?

    APIs as we know (or application programming interfaces, but no one really calls them that anymore... They're just 'APIs') are compact ways to use other people's code. Traditionally, you would install and include someone else's software package within your code and interact with it via a programming interface defined by the package's author. Of course, this process requires that you install their code and that you are using the same programming language as they are. RESTful APIs solve this problem. Instead of downloading, installing, and using someone else's package WITHIN your program (and hoping their software is available for your language), you can use your program to query a website (just like you query google for search results via a URL) and the website will internally use the API you'd like and return you the result data.

    A good example would be, instead of buying a mill saw, some blades, a nice workbench, and cutting wood in your house, making a mess, you can instead send the wood off to a company who will just return cleanly cut pieces of wood to your specifications.

    Sounds nice, but how does it work? In this tutorial, I'll show you how anyone can use a RESTful API without installing any software packages (assuming you already have the python installed and urllib/2).

    How does it work? It uses HTTP, just like a google request. For those unfamiliar, I'll describe the process at a high level. When you type a search on google, you are actually submitting a HTML form. A form is a special type of HTML tag which specifies instructions on how information should be passed from one computer (or server) to another, when an action such as 'submit' has been specified. Forms generally have two attribute fields, a method and an action. Your web browser examines the html form's attributes and uses this to determine how it should pass data to the recipient.

    A html form may look like this:

    Code:
    <html>
        <head>
            <title>Mek's form example</title>
        </head>
        <body>
            <form method="GET" action="some-url">
                <input type="text" name="username" />
                <input type="password" name="password" />
                <input type="submit" name="submit" />
            </form>
        </body>
    </html>
    
    First, before anyone jumps at my throat, yes, I am aware the method of this form is GET and that a 'password' field is being used. Whenever you are constructing a form, you have several options regarding what 'method' to use when transmitting data from location A to B. Each has their own advantages and disadvantages. I will explain this in greater detail later, however, in nearly all cases, if you are transmitting secure data like credit card information, passwords, or private keys, you will NOT want to use GET as your HTTP method. I will explain why this is the case using the code above.

    First, I'd like to explain what the code above does and what it means (and then I'll mention the difference between two of the methods of the HTTP protocol, GET and POST).

    In the code above, we create a form, just like any HTML tag, but we specify two fields, a method and an action. The method determines how we send data from A to B, as I said before. The action determines who the recipient of the information.

    In order to understand the difference between GET and POST, let's pretend the 'action' field in our code above was google, and not 'some-url'. I'll describe what's actually happening when a user (like you or I) visit the webpage containing the code above up to the point where the form is submitted and information is returned.
    1. A user goes to our website (with the code above) and enters in a username and password within the input fields.

    2. The user presses the submit button

    3. The user's web browser examine the form's attributes and determines how the information should be sent to the recipient (according to the method)

    3. a) If the method is GET, the browser understands you would like to GET a new webpage as a response to your request. As we know, the way to get a new webpage (or the way we request a webpage) is by asking for a URL.

    The browser constructs a URL from our data which looks like this:
    Code:
    something .co m / something? username=xxx & password=1234
    
    Sorry, you'll have to be creative and remove spaces in the example above as I can't use links yet in my posts.

    Anyways. You'll notice there is a question mark in the url. This question mark denotes the start of a query. A query is how special parameters are specified within a URL. They are used to tell the server you want specific pieces of information. In this case, we are passing the server two parameters through our URL (and we're asking to GET a webpage which fits that criteria). For a google search, when you type a query in the textbox, maybe we're searching for 'python' and press submit, your browser turns your request into something like:

    Code:
    google .co m /#q=python
    
    Why wouldn't you want to have a password sent to a server via GET? Notice using GET all the information transmitted is sent visibly through the URL, and you wouldn't want others to see your password. Then why the heck would you want to use GET every? And what alternative is there to GET? Good questions!

    GET is useful because it allows us to share links which contain very specific information. For instance, not only can I share a link to google, but I can share a link containing the exact query I used to get a google result! So there, GET actually is extremely useful. It's how we GET every web page we want! But when it comes to passing secure information, GET is not the best solution because it is passed visibly and the amount of information you can send within GET is (as per the specs) limited to 256 characters. Therefore, we POST information to a server when we want to give information to us.

    But wait, maybe you're thinking, "How come when I POST information like my credit card information on Amazon, I GET a new page?". This is another good question. When you POST information to a server, the server will handle your POST request how ever it is designed. A POST doesn't have to direct you to a new website, however, traditionally this is preferable, so the recipient server will redirect you.

    Ahh, so we have the basics of GET and POST taken care of, but why the heck is this important? We're talking about Python, right? RESTful APIs?

    Right, right. So, the way RESTful APIs work is, you can go into your browser and make a GET request by sending parameters manually (in your http url bar) to a server hosting a RESTful API, and that server will return you the correct data back.

    This is useful, but we'll take this a step further. We'll write a python script which simulates a GET request to a RESTful API server and then get capture the server's response in python so we can then use the returned data!

    Once you learn how to do this, you will be able to use any RESTful API on the web! You won't be restricted by your programming language. Powerful stuff.

    But there's one last thing I didn't discuss. Fine, we make a request to a server using their RESTful API, but what format is their returned data in? Chances are, the returned data will be available in a variety of formats. Two of the more popular formats are JSON and XML. In my opinion, dealing with XML is a pain in the arse, so I'm only going to discuss JSON (sorry! it's sleek and sexy).

    In order to understand JSON (javaScript Object Notation -- actually a lot easier than it sounds) we must first -- as I promised in my last post -- learn about python dictionaries (associative arrays). Aw, but we were just getting to the good part! Let's see some python! Fine, we can hold off on learning how and why JSON is awesome when we get to the part in our Python code where the server has actually sent us the JSON code as a response.

    So, without further ado, here's a snippet of python code which sends a GET request (without any data) to my website Baybo and retrieves a list of all the products on the platform. Oh, if you'd rather get info about our users instead, just replace the word 'products' in the url below with 'users':

    Code:
    import urllib2
    
    # Since I cannot post urls on the forums yet
    # this tiny hack will turn my 'not a url'
    # into a url! Yay.
    not_a_url = "http:;;baybo,it;api;products"
    url = not_a_url.replace(",",".").replace(";","/")
    
    # Now we have our url, let's open it in python
    response = urllib2.urlopen(url)
    
    # Now let's read that response into a variable
    html = response.read()
    
    # if you're feeling lazy, you could do this all in
    # one line and just type:
    # html = urllib2.urlopen(url).read()
    
    # Now the variable 'html' contains our 
    # JSON response but it's a String! Not JSON!
    
    # Oh well, guess we have to learn about JSON now!
    
    Also, as a side note, before I forget, notice this example doesn't pass an advanced GET query with variables (like our form example above). I'll show a second example which does pass data along with the URL after I explain JSON and after we do something useful with the data we get!

    ... And all this will get done after I eat some lunch! Happy hacking.
     
    • Thanks Thanks x 5
  12. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Okay, where did we leave off... We got a string representation of the JSON data we want from the server, now we want to tell python to parse this data as JSON so we can interact with it.

    In the last step we did
    Code:
    html = urllib2.urlopen(url).read()
    
    In order to take this html data (in string form) and convert it to JSON, we will need to import the 'json' module (should already be installed on your machine with python) and use the json.loads() function. This is what the entire program would look like:

    Code:
    import urllib2
    import json
    html = urllib2.urlopen(url).read()
    data = json.loads(html)
    
    The variable 'data' is now an actual data structure within Python that we can natively access. In this case, the data the server transmitted to us is an array (a python list) of dictionaries. Uh oh, now we really need to learn what dictionaries are, or we won't be able to use this data effectively!

    A dictionary is similar to a list/array in that it is a data structure for storing values. However, the method of indexing values and the values of an index are different than a list. In fact, the complexity analysis for the entire data structure is different! By this I mean, a dictionary (associative array) does not have the same complexity properties as a list: the computational cost (the number of steps) to perform insertion, deletion, and resizing over the data structure are different than lists.

    So, before we tackle the complexity properties of dictionaries, what the heck is a dictionary? Think of a dictionary like... Well, a dictionary! Or a phone book, if you prefer. A dictionary is a collection (unordered) of key-value pairs. In a dictionary (the books we're used to) a dictionary word is used to reference a definition. If you want to know the definition for a word, use use the word as the key, find this word, and knowing the word will give you access to the definition. Same goes for a phone book. You use someone's name as the key, and with it can find someone's number (the value).

    A dictionary in python looks like this:

    Code:
    mydict = { "key" : 3,
                    "mek" : 13}
    
    Values are access from a dictionary in a similar way as lists, however, the index is not a position (that is, not necessarily an integer), but a key. Here's an example of how I would access the value 13 in the above dictionary called 'mydict' using the key "mek" which is a string"

    Code:
    # This returns the value 3
    mydict["mek"]
    
    Ok! Dictionaries have very interesting properties with respect to complexity analysis... But I suspect you folks would rather continue the example right now and so something useful with your data / see a full example program. If you're interested in learning more about the data structures, let me know! This is useful information which can be applied to nearly any programming language and knowing this information is useful for CS job interviews and school.

    Anyways. Where did we leave off in our program? Ah yes, you just turned our data into JSON format and python automatically loaded this JSON into a native python data structure as a list of dictionaries which we can use!

    Code:
    # data is a list of dictionaries
    data = json.loads(html)
    
    # The first element of the list is a dictionary.
    # This will print out the dictionary so we can see
    # what keys and values it has.
    data[0]
    
    # This tells us the keys for the dictionary are:
    # "content", "name", "created", "slug", "price", "modified",
    # "users_id", "currency", "avatar", "id".
    
    # What does this mean? For every product on Baybo,
    # We can get all this information! If we wanted,
    # we could make our own website and advertise / affiliate
    # products with tiny script.
    
    # Q: How would we access the product id of the 1st product?
     # A: get the 0th product in the dictionary and 
    #     get the value where the key is "id"
    data[0]["id"]
    
    # We could write a loop to iterate over every product
    # and do something useful:
    
    # This will print out the name of every product
    for product in data:
        print data[product]["name"]
    
    Following these steps, you can use any RESTful API out there. I just used my personal RESTful API as an example, that way if you folks have any questions, I can help you solve your problems.

    Until next time, good luck and happy hacking!
     
    • Thanks Thanks x 3
  13. Baybo.it

    Baybo.it Registered Member

    Joined:
    Aug 9, 2011
    Messages:
    72
    Likes Received:
    39
    Occupation:
    Founder of Baybo.it
    Location:
    San Francisco
    Home Page:
    Setup your own dynamic website with templating and flexible url matching in under 50 lines of code. This tutorial will take you from start to finish. Note: The install instructions provided are for Linux Ubuntu. If there's demand, I'll explain how ez_install can be setup for windows and linux to install any third party python library.

    Frameworks
    There are many web frameworks for Python. Among these are Django, Tornado, Pylons, Grok, TurboGears, and Webpy. Descriptions of each are available at: wiki. python. org /moin/WebFrameworks. In this section I am going to describe how to get setup with webpy as it is extremely light-weight, intuitive, and relatively easy for those with minimal experience with web frameworks.

    Webpy
    Webpy is a standalone web framework which will allow you to manage, generate, and serve rich web applications. With webpy, you have the ability to match url addresses to unique procedures which can generate site specific content. For example, if you have a website called example. co m, you can easily specify a mapping for the urls example. co m/page1 and example. co m/page2 (e.g. the pattern /page1 and /page2) and define procedures to handle content rendering specific to each of these cases. You may wish to have ‘/page1' render a login page and ‘/page2' generate your homepage.

    III. Installation and Test
    1. install: sudo apt-get install python-webpy # or sudo easy_install web.py
    2. Open a text editor and paste:

    Code:
    import web
    # This maps any page on your website to the class ‘hello'
    # the (.*) will capture 1 var in the url after /hello
    # for example, it will capture /hello/mek
    # and if webpy notices the url matches /hello/anything
    # it will pass in the value of "anything" to the GET method.
    
    urls = (
        '/(.*)', 'hello', 
    )
    
    app = web.application(urls, globals())
    
    class hello:
        """
        This class captures and serves all requests for the url
        pattern /hello
        """
        # if the HTTP request used the GET protocol,
        # serve using this method:
        def GET(self, name):
            """
            As we learned from the previous post, this method will capture
            any GET request sent to our server (when a user requests a page)
            """"
            # Anything after /home will be stored in var 'name'
            # (passed in). However, if no name provided, after /hello
            # in url, set name to "dude"
            
            if not name:
                name = 'dude'
            
            return 'My website works, ' + name + '! Duh! Winning!'
    
    if __name__ == "__main__":
        app.run()
    
    Save As: test.py
    Run with: python test.py 8080 # or optionally, use sudo and run on port 80
    View: your locally accessible website in your favourite browser at localhost aka 127.0.0.1
    You're done: Winning!

    Explanation
    In the previous code, we specify a list of ‘urls' which are patterns mapping to classes. When a client requests a url from their web browser to your server (or in this case your local computer), the url they typed will be analyzed and processed by the patterns in your url list. In the example above, our urls only contain one url-pattern to class mapping; namely, every request matching the pattern ‘/(.*) (which is a regex pattern matching any url of the form ‘/' followed by 0 or more of any character) to the procedure ‘hello' (whose instructions are defined in the class ‘hello'). That is to say, whenever a user visits yourwebsite/anything, /anything will match our pattern ‘/
    (.*)' and instruct the ‘hello' class to issue a response. You can add more pattern/procedure pairs to the url list to serve multiple unique webpages. For more information, view the official webpy tutorial at webpy. org/docs / 0.3/tutorial or for completing specific tasks in webpy, check out the cookbook at webpy. org / cookbook/.
     
    • Thanks Thanks x 2
    Last edited: Aug 13, 2011
  14. dr_0x

    dr_0x Junior Member

    Joined:
    May 9, 2010
    Messages:
    141
    Likes Received:
    169
    Home Page:
  15. yesrams

    yesrams Regular Member

    Joined:
    Dec 16, 2009
    Messages:
    354
    Likes Received:
    75
    Location:
    @BHW
    Oh! I missed this thread. thanks for the good post. Waiting for next post.
    Great explanation .
     
    Last edited: Aug 16, 2011
  16. drey2k

    drey2k Power Member

    Joined:
    Jan 4, 2009
    Messages:
    551
    Likes Received:
    169
    Occupation:
    Finance guy
    Location:
    USSR 1943
    I have a couple questions about this:

    Code:
    import urllib2
    
    # Since I cannot post urls on the forums yet
    # this tiny hack will turn my 'not a url'
    # into a url! Yay.
    not_a_url = "http:;;baybo,it;api;products"
    url = not_a_url.replace(",",".").replace(";","/")
    
    # Now we have our url, let's open it in python
    response = urllib2.urlopen(url)
    
    # Now let's read that response into a variable
    html = response.read()
    
    # if you're feeling lazy, you could do this all in
    # one line and just type:
    # html = urllib2.urlopen(url).read()
    
    # Now the variable 'html' contains our 
    # JSON response but it's a String! Not JSON!
    
    # Oh well, guess we have to learn about JSON now!
    
    What is "import urllib2"?

    How do I actually run this code?

    Thx
     
  17. haxxaruz

    haxxaruz Registered Member

    Joined:
    Jun 12, 2011
    Messages:
    81
    Likes Received:
    19
    Urllib2 is an inbuilt-package of python.
    Code:
    hxxp://docs.python.org/library/urllib2.html
    Assuming you're on Windows:
    (if you were on Linux, you'd probably know how to run this code)
    Code:
    hxxp://docs.python.org/faq/windows#how-do-i-run-a-python-program-under-windows
    without wax,
    haxxaruz
     
    • Thanks Thanks x 1
  18. timothywcrane

    timothywcrane Power Member

    Joined:
    Apr 25, 2009
    Messages:
    590
    Likes Received:
    236
    Occupation:
    Internet Promotion Management
    Location:
    USA
    Home Page:
    Subscribed, and thanks given. I love Python. As a cut and paste coder, I love the live IDE. I also love the fact that knowing ANY amount of Python helps me in Jython/Sikuli IDE dev. If you have ever compared what is needed to program in Java as compared to Jython, you would understand the newbie's choice for the Jython option.

    I tell about Sikuli every chance I get, as it is Open source and Jython based (allowing for all python and Java call implimentation 100%). I believe it is BSD licensed.
     
  19. sweetie

    sweetie Registered Member

    Joined:
    Aug 3, 2011
    Messages:
    81
    Likes Received:
    3
    Occupation:
    web master
    Great in explanation Baybo, I program in PHP , but you really get me with python . Constructions look more compact than PHP :)
     
  20. drey2k

    drey2k Power Member

    Joined:
    Jan 4, 2009
    Messages:
    551
    Likes Received:
    169
    Occupation:
    Finance guy
    Location:
    USSR 1943
    Would def like to learn more about data structures!

    Your a fantastic teacher with these tutorials btw... really really well written.