1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Hello im new Programming [Python] - Question about Twitter

Discussion in 'Other Languages' started by maxibaby, Apr 21, 2013.

  1. maxibaby

    maxibaby Junior Member

    Joined:
    Apr 20, 2013
    Messages:
    103
    Likes Received:
    37
    Occupation:
    Student
    Location:
    Venezuela
    Hello so i started programming not so far ago, i started doing those udacity tutorials.
    So, im working in Python and i want to create a Twitter account via Python.

    What i've done so far? Well, as how i see things, twitter at twitter / signup displays a website with various forms.
    These have names like : user[name] , user , user[user_password] and user[screen_name]
    So what do i think ? well, what i need is just to send a POST method to this website with this info filled:

    I did this on a basic HTML forms website i made my own, and could sucessfully do it with something like this:

    Code:
    >>> import urllib
    >>> import urllib2
    >>> query_args = { input_name:input_value , input_name:input_value , etc..}
    >>> encoded_args = urllib.urlencode(query_args)
    >>> url = 'local' << fixed so i can post
    >>> print urllib2.urlopen(url, encoded_args).read()
    
    With this easy code i made up to fill a form in my website , witch was recorded in a database and then i could see it from browser.


    But for some reason this same method isn't working for twitter. Can someone tell me why it is , or where should i start seeing to improve
     
  2. DreamBigWorkHard

    DreamBigWorkHard Registered Member

    Joined:
    Nov 20, 2012
    Messages:
    92
    Likes Received:
    11
    Occupation:
    Daydreamer
    Firstly you need to capture all the headers info when you manually create accounts on twitter, then try your best to simulate all the server-client process.
     
  3. Gophering

    Gophering Junior Member Premium Member

    Joined:
    Mar 21, 2013
    Messages:
    115
    Likes Received:
    279
    Occupation:
    Unemployed
    Location:
    EU
    Get yourself a reverse proxy OP. Burpsuite is pretty much the industry standard (http://portswigger.net/index.html), however fiddler (win) or charles (mac) will do fine as well. Run the reverse proxy and perform a couple of HTTP requests on twitter, check your traffic logs and try to emulate them with your python client.

    For twitter specifically I'd recommend using a headless browser engine as they've become very anal about unofficial clients accessing their services. I've noticed certain JS code which runs client side and is most probably there to distinguish your requests from those that would be sent by a real browser. You could try using Spynner (QT based) or PhantomJS (JS based), I tend to stick with phanotm as its very lightweight and easy to work with.

    Hope this helps.
     
  4. YouFeelMeDawg?

    YouFeelMeDawg? BANNED BANNED

    Joined:
    Aug 10, 2011
    Messages:
    266
    Likes Received:
    371
    That is how you do it if you are using a browser or headless browser, but not with http requests.

    Reason is very simple, is because there aren't just 2 input fields on the twitter form for the login. You need to look under the hood, either examine the Form and grab all the input values(including the hidden input) or just sniff your traffic by simulating it on firefox then replicating on requests.

    Also I would recommend you use "requests" for python, is a much better wrapper.

    Remember, use Dom inspector from chrome or firebug and always check the source code if your doing it manually to see all the input fields you need to send over
    so your post to twitter will not only contain the visible input fields, but a lot of hidden inputs(you need to send those too).
     
  5. maxibaby

    maxibaby Junior Member

    Joined:
    Apr 20, 2013
    Messages:
    103
    Likes Received:
    37
    Occupation:
    Student
    Location:
    Venezuela
    Okay ((Remember im new to this ))

    So for what i understood with this method im using a headless browser, witch means im not sending headers to the server rigth ?

    So for what i checked this is the Form Data that gets sent to the server.

    Before i was only adding name, username, password and ail.

    And i dint know about that authenticity_token.
    i just found that it comes when i first make the request : there's that hidden input that has a value ( Seems that each form in the website has a specific authenticity_token, so i'm just gonna watch the one at the form where i fill the info )


    For the rest i tougth i didn't need to add those to my program since that way it would have the default values <- Am i wrong?

    By the way im trying to sign up not login (seems like login needs some Oauth app)

    Code:
    [LIST]
    [*][FONT=Consolas]authenticity_token=56cad992d4fd4a69e08b8df36c224b1d237bcd8&user%5Bname%5D=Gabmenez&user%5Bemil%5D=m&user%5Buser_password%5D=&user%5Bscreen_name%5D=maxiasfy2312&user%5Bremember_me_on_signup%5D=1&user%5Bremember_me_on_signup%5D=&user%5Buse_cookie_personalization%5D=1&asked_cookie_personalization_setting=1&context=&ad_id=&ad_ref=&user%5Bdiscoverable_by_email%5D=1&user%5Bsend_email_newsletter%5D=1[/FONT]
    [/LIST]
    
    Code:
    [LIST]
    [*][B]authenticity_token:[/B]
    [COLOR=#ffa500][FONT=Consolas]56cad992d4fd4a69e08b8df36c224b1d2379bcd8[/FONT][/COLOR]
    [*][B]user[name]:[/B]
    [*][B]user[emil]:[/B]
    [*][B]user[user_password]:[/B]
    [*][B]user[screen_name]:[/B]
    [*][B]user[remember_me_on_signup]:[/B]
    [FONT=Consolas]1[/FONT]
    [*][B]user[remember_me_on_signup]:[/B]
    [*][B]user[use_cookie_personalization]:[/B]
    [FONT=Consolas]1[/FONT]
    [*][B]asked_cookie_personalization_setting:[/B]
    [FONT=Consolas]1[/FONT]
    [*][B]context:[/B]
    [*][B]ad_id:[/B]
    [*][B]ad_ref:[/B]
    [*][B]user[discoverable_by_email]:[/B]
    [FONT=Consolas]1[/FONT]
    [*][B]user[send_email_newsletter]:[/B]
    [FONT=Consolas]1[/FONT]
    [/LIST]
    
    I didn't understand how would i use this: " I've noticed certain JS code which runs client side and is most probably there to distinguish your requests from those that would be sent by a real browser. You could try using Spynner (QT based) or PhantomJS (JS based), I tend to stick with phanotm as its very lightweight and easy to work with."

    And also this: That is how you do it if you are using a browser or headless browser, but not with http requests.
    I need to do http requests? i cant just do it this way ?
     
    Last edited: Apr 21, 2013
  6. xxNEBer

    xxNEBer Newbie

    Joined:
    Apr 13, 2013
    Messages:
    24
    Likes Received:
    15
    You should become familiar with Firebug so you can see all the GET and POST requests. You can see all the fields needed for the headers and data being posted via urllib.urlencode( data ). I have made plenty of Twitter bots with Python.
    In the future you may want to look into cookies with cookielib.MozillaCookieJar and proxy support with urllib2.ProxyHandler

    Edited spelling.
     
    Last edited: Apr 27, 2013
  7. xxNEBer

    xxNEBer Newbie

    Joined:
    Apr 13, 2013
    Messages:
    24
    Likes Received:
    15
    as for the authenticity_token, look into BeautifulSoup :)
     
  8. FJX

    FJX Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 13, 2011
    Messages:
    356
    Likes Received:
    186
    Location:
    0x90
    I use Tamper Data + Fiddler2 when doing this approach. And requests.