Youtube cookie issues

 

Results 1 to 20 of 20
Okay, I am trying to log on youtube using http web request but I am ...
  1. #1
    ultra.marine's Avatar
    ultra.marine is offline Registered Member
    Join Date
    Oct 2012
    Location
    Macedonia
    Posts
    79
    Thanks
    142
    Thanked 90 Times in 29 Posts

    Default Youtube cookie issues

    Okay, I am trying to log on youtube using http web request but I am getting this feedback from youtube "Your browser's cookie functionality is turned off. Please turn it on"


    Any help, hits or anything I will be thankful!

    Code:
           Dim tempCookies As New CookieContainer
            Dim postData As String = "continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26feature%3Dsign_in_button%26hl%3Den_US%26next%3D%252F%26nomobiletemp%3D1&service=youtube&dsh=-9120265060045154752&hl=en_US&GALX=xgj0d0m6phE&pstMsg=1&dnConn=&checkConnection=youtube%3A212%3A1&checkedDomains=youtube&timeStmp=&secTok=&_utf8=%E2%98%83&bgresponse=%21A0JW0zonfLaO7kT-7m88MYDfHgIAAABUUgAAAA4qAQsxnzRQHLCiYtetendJsDYVMbUPHMynXE9B_fvNVQJKhD2KyxW9E03fKTVJ1tcKd2KN4ivJz-O7_MzUHmL6v2NVMN-O10Ms5W6SrHvumwMXoNeozmvKHidnGmS1h-Y0ynR_JQl_wUoHydgg6J7OctiL9Zafvok5BSlj-KBRH-BUSLUrtoMuv9tna37N-xwRAQLiAmmOVyfQsF1uov3O-iHlB5bZL8RzszIWeiSJacz7dkCZvR3YbKR7NRF0fReiQtWQx4atJ2KRJ6ogScEYsa6RTqAXboEaqqyEsd1sE0pjmmoAOQXL8OngfKuTyCy7ac-ILQLszuQyspXGvpY_wqiHJIC1lCX_aYnynq0&Email=USERNAME%40MAIL.COM&Passwd=MYPASSWORD1&signIn=Sign+in&PersistentCookie=yes&rmShown=1"
    
    
            Dim encoding As New UTF8Encoding
            Dim byteData As Byte() = encoding.GetBytes(postData)
    
    
            Dim postReq As HttpWebRequest = DirectCast(WebRequest.Create("https://accounts.google.com/ServiceLoginAuth"), HttpWebRequest)
            postReq.Method = "POST"
            postReq.KeepAlive = True
    
    
            postReq.CookieContainer = New CookieContainer()
            postReq.ContentType = "application/x-www-form-urlencoded"
            postReq.Referer = "Https://accounts.google.com/ServiceLogin?uilel=3&service=youtube&continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26feature%3Dsign_in_button%26hl%3Den_US%26next%3D%252F%26nomobiletemp%3D1&passive=true&hl=en_US"
            postReq.UserAgent = "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.872.0 Safari/535.2"
            postReq.ContentLength = byteData.Length
    
    
            Dim postreqstream As Stream = postReq.GetRequestStream()
            postreqstream.Write(byteData, 0, byteData.Length)
            postreqstream.Close()
            Dim postresponse As HttpWebResponse
    
    
            postresponse = DirectCast(postReq.GetResponse(), HttpWebResponse)
            tempCookies.Add(postresponse.Cookies)
            loginCookie = tempCookies
            Dim postreqreader As New StreamReader(postresponse.GetResponseStream())
    
    
            Dim thepage As String = postreqreader.ReadToEnd
    
    
            RichTextBox1.Text = thepage




  2. #2
    gimme4free is offline Executive VIP
    Join Date
    Oct 2008
    Posts
    1,828
    Thanks
    358
    Thanked 1,854 Times in 555 Posts

    Default Re: Youtube cookie issues

    You are creating a new cookie container AFTER going to the login page, you should create a cookie container for the whole process / manage cookies in a class.

    Also this alone would still not be enough to login, you need to make sure to follow ALL of the Google URL's & parse out the Javascript redirects etc that Google run, there is quite a bit to do to login to YT properly using HTTPWebRequest while also detecting all potential responses & also Google login now uses BotGuard so there are certain pages that you will not be able to view without HTTPWebRequest without decoding BotGuard's requests (Good luck!!!).

    If you are struggling with a login with HTTPWebRequest then I recommend switching to WebBrowser as YT monitor more than just the individual calls to make actions so you will have to spend a long time adding in all of the AJAX calls that are made within a browser just making a simple action.


    Edit: Just noticed you are not visiting the login page at all, you are just posting the data, Google has a lot of variables that you need to parse & use when posting the login & also cookies are set when visiting the login page.

  3. The Following User Says Thank You to gimme4free For This Useful Post:

    ultra.marine (03-26-2013)

  4. #3
    ultra.marine's Avatar
    ultra.marine is offline Registered Member
    Join Date
    Oct 2012
    Location
    Macedonia
    Posts
    79
    Thanks
    142
    Thanked 90 Times in 29 Posts

    Default Re: Youtube cookie issues

    hehe thanks man for everything I think i am visiting the login page but i'll try and look around. I already have working program with web browsers but the problem is it can't be multithreaded so I thought to change to web requests... seems this will be harder that I thought. :/

    Thanks anyway

  5. #4
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    I already posted a working solution here:
    http://www.blackhatworld.com/blackha...n-using-c.html
    Translate it to VB

  6. The Following User Says Thank You to theMagicNumber For This Useful Post:

    esolution (06-16-2013)

  7. #5
    ultra.marine's Avatar
    ultra.marine is offline Registered Member
    Join Date
    Oct 2012
    Location
    Macedonia
    Posts
    79
    Thanks
    142
    Thanked 90 Times in 29 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    I already posted a working solution here:
    http://www.blackhatworld.com/blackha...n-using-c.html
    Translate it to VB
    I have seen it and tried it, it doesn't work m8. You don't even have cookie container, without cookies there is no use of post request at all.

  8. #6
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by ultra.marine View Post
    I have seen it and tried it, it doesn't work m8. You don't even have cookie container, without cookies there is no use of post request at all.
    Of course i am using cookies. The WebClientEx class inherits WebClient and add cookie support.

    And you haven't tried it at all, as i coded it yesterday, to help some ignorant fuck, who in return told me:
    "But i wanted HttpWebRequest not WebClient" - WebClient is wrapper around webrequest/webresponse.
    Last edited by theMagicNumber; 03-27-2013 at 05:43 PM.

  9. The Following User Says Thank You to theMagicNumber For This Useful Post:

    ultra.marine (03-27-2013)

  10. #7
    ultra.marine's Avatar
    ultra.marine is offline Registered Member
    Join Date
    Oct 2012
    Location
    Macedonia
    Posts
    79
    Thanks
    142
    Thanked 90 Times in 29 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    Of course i am using cookies. The WebClientEx class inherits WebClient and add cookie support.

    And you haven't tried it at all, as i coded it yesterday, to help some ignorant fuck, who in return told me:
    "But i wanted HttpWebRequest not WebClient" - WebClient is wrapper around webrequest/webresponse.
    You have some behavior issues, watch your language. You neither coded that for me, neither it's in vb.net, neither helps me understand where I am wrong.

    Sometimes you may believe people word around, as some of them are honest
    Code:
    http://img5.imageshack.us/img5/9043/sdfasf.png
    See the date where I downloaded and I deleted your program after trying it. You think I would bullshit without trying it ? You don't have to go on rage for nothing. Control your emotions

    However, after your post, I thought... hmmm i should re-try it and I found it working, that "always true bug was the reason I thought doesn't work", but actually works.

    Thanks, next time you can politely say things, not using "fucks and shits"... I thought we're all here to get and offer some help, or not?

  11. #8
    YouFeelMeDawg? is offline Banned - see signature
    Join Date
    Aug 2011
    Posts
    268
    Thanks
    21
    Thanked 353 Times in 134 Posts

    Default Re: Youtube cookie issues

    Well your example clearly doesn't work because, you have hard coded "dsh" and "GALX" values in your code.
    Those values are dynamic and are different on every single reload, in other words they are unique identifier.

    You obviously going to get problems when doing the post, you have to get those values from the html when you request the login page.
    This member has been permanently banned from BHW.

  12. The Following User Says Thank You to YouFeelMeDawg? For This Useful Post:

    ultra.marine (03-27-2013)

  13. #9
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by ultra.marine View Post
    You have some behavior issues, watch your language. You neither coded that for me, neither it's in vb.net, neither helps me understand where I am wrong.

    Sometimes you may believe people word around, as some of them are honest
    Code:
    http://img5.imageshack.us/img5/9043/sdfasf.png
    See the date where I downloaded and I deleted your program after trying it. You think I would bullshit without trying it ? You don't have to go on rage for nothing. Control your emotions

    However, after your post, I thought... hmmm i should re-try it and I found it working, that "always true bug was the reason I thought doesn't work", but actually works.

    Thanks, next time you can politely say things, not using "fucks and shits"... I thought we're all here to get and offer some help, or not?
    Hello,
    i apologize my behaviour was unacceptable, had a bad day.
    Anyway here is what you have to do:
    1) navigate to:
    Code:
    https://accounts.google.com/ServiceLogin?continue=http%3A%2F%2Fwww.youtube.com%2Fsignin&service=youtube&hl=en
    2) parse the <input> from <form> with ID gaia_loginform. There are 18 variables. You can use regex or xpath to do that. GALX and dsh are dynamic, so you can only parse these parameters and construct the post data.
    3) DON'T forget to Url Encode the data before posting it
    4) set AllowAutoRedirect to true and post the data to
    Code:
    https://accounts.google.com/ServiceLoginAuth
    5) DONE.

  14. The Following User Says Thank You to theMagicNumber For This Useful Post:

    tb303 (03-28-2013)

  15. #10
    YouFeelMeDawg? is offline Banned - see signature
    Join Date
    Aug 2011
    Posts
    268
    Thanks
    21
    Thanked 353 Times in 134 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    Hello,
    i apologize my behaviour was unacceptable, had a bad day.
    Anyway here is what you have to do:
    1) navigate to:
    Code:
    https://accounts.google.com/ServiceLogin?continue=http%3A%2F%2Fwww.youtube.com%2Fsignin&service=youtube&hl=en
    2) parse the <input> from <form> with ID gaia_loginform. There are 18 variables. You can use regex or xpath to do that. GALX and dsh are dynamic, so you can only parse these parameters and construct the post data.
    3) DON'T forget to Url Encode the data before posting it
    4) set AllowAutoRedirect to true and post the data to
    Code:
    https://accounts.google.com/ServiceLoginAuth
    5) DONE.
    ya but your forgetting one important thing.
    By default if you dont generate the "bgresponse" value instead of the "js_disabled" value that is set on, youtube will know that you don't have javascript enabled and will actually hurt you on the long run. In other words, now they can tell if your using a browser or http request because of that variable.
    This member has been permanently banned from BHW.

  16. #11
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by YouFeelMeDawg? View Post
    ya but your forgetting one important thing.
    By default if you dont generate the "bgresponse" value instead of the "js_disabled" value that is set on, youtube will know that you don't have javascript enabled and will actually hurt you on the long run. In other words, now they can tell if your using a browser or http request because of that variable.
    I didn't forget that. Yes, it is leaving a huge footprint. Maybe you will offer a solution to decode the JS ?
    I will deal with it shortly, but currently i am not interested in youtube.

  17. #12
    YouFeelMeDawg? is offline Banned - see signature
    Join Date
    Aug 2011
    Posts
    268
    Thanks
    21
    Thanked 353 Times in 134 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    I didn't forget that. Yes, it is leaving a huge footprint. Maybe you will offer a solution to decode the JS ?
    I will deal with it shortly, but currently i am not interested in youtube.
    The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

    Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

    Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
    So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
    Last edited by YouFeelMeDawg?; 03-28-2013 at 05:24 PM.
    This member has been permanently banned from BHW.

  18. #13
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by YouFeelMeDawg? View Post
    The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

    Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

    Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
    So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
    True.
    I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
    For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

    Have you used HtmlUnit(Java) the library has some sort of JS processor?

  19. #14
    YouFeelMeDawg? is offline Banned - see signature
    Join Date
    Aug 2011
    Posts
    268
    Thanks
    21
    Thanked 353 Times in 134 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    True.
    I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
    For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

    Have you used HtmlUnit(Java) the library has some sort of JS processor?
    I am switching mainly to Python basically, Java would be my second choice or ruby on rails. The thing is I am stepping away from C# or anything that uses things like .net framework or is only usable on a specific OS, like windows.

    My next project will hopefully be a youtube bot that is scalable on the cloud,starting with using a public cloud like amazon, rackspace, and eventually do my private/hybrid cloud, that means using vps/dedi with public clouds to make my own private/hybrid cloud. Doing it this way I can just use the vps/dedi as a factory of workers with celery(http://www.celeryproject.org/) and make the modules for the tasks(youtube, facebook,twitter, web 2.0).

    So I am taking literally like 6 months to learn python better and all of its useful libraries
    like boto, for amazon cloud services

    I need to learn py-redis, to use redis caching software and as a AMPQ broker for celery, then I have to learn mongoDB since mysql is really slow and its hard to scale at an enterprise level.On top of that, I have to use extensively webhooks to be able to connect a lot of 3rd party services together instead of polling their API for results,so thats another thing I will use extensevely

    In conclusion I will be using selenium to automate browser bots(chrome, firefox, ie) and about the htmlunit, I think the best would be to not use it because its not a full http wrapper that has PUT,GET,DELETE,POST. You can do GET nicely with htmlunit but not anything else.
    So I am still deciding what web framework to use with python.

    Now I am also want to add a ruled-based engine, the sad part is that JAVA has a lot of opensource, freeware options like Dross,Jess,JRuledEngine,OpenRules etc but for python I don't really have that many option.

    For python for a ruled-based engine I am looking at pyclips(the python module for the CLIPS ruled based engine developed by NASA), or Intellect 1.4. So thats another thing I am looking forward in learning so I can make my next bot sort of like a script based, where I can write rules instead of hard coding anything making it able to code an Expert System for IM bot that is deployable on a cloud instead of desktop-based.

    So ya, am not really coding or updating my yt bots to be honest. I am more learning about all of these other tools/libraries to make my own enterprise cloud based IM tool, then I can sell my software-as-a-service meaning, I can charge people per hour to use my bot with their proxies and accts, instead of releasing my bot and giving them the exe.

    All I know from youtube is from what I have coded extensevely in the past 2 years, lately these past 6 months I have just bee learning a lot about different 3rd party services I can use to make my next bot.
    Last edited by YouFeelMeDawg?; 03-28-2013 at 05:53 PM.
    This member has been permanently banned from BHW.

  20. #15
    theMagicNumber is offline Regular Member
    Join Date
    May 2010
    Posts
    295
    Thanks
    139
    Thanked 138 Times in 87 Posts

    Default Re: Youtube cookie issues

    You have interesting ideas and goals. I am also considering switching to Python, actually it will be more of a test, i would like to test .NET and Python performance for web automation.

    I think a cloud based solution will be too expensive than selling desktop bots, it will be harder to maintain and if your user base grows too fast, probably you won't have the resources to cope with that in time. I don't have a lot of experience with amazon EC, but i did some tests with google app engine and it is really expensive, it is cheaper to start with dedis.

    I am not familiar with celery, but it reminds me of Majestic12 project, they started as Distributed Search Engine and promised to develop open source search engine, but in the end they created a paid SEO intelligence service.

    About mongoDB, this is something worth checking. I am a performance freak and i made a few searches, it seems that mongoDB is way faster than MySQL.
    Also using SSDs for the databases will increase the performance a lot. For example my MS SQL database is 40%-50% faster than hosting it on a regular HDD.

    I am interested only in the javascript evaluator in HtmlUnit, basically decoding/emulating the JS is necessary before the POST, i tried some .NET solutions but they were working with really basic javascript.

    I started my ruled-based crawler/scraper 1-2 years ago as a simple project, at least it was simple on my drawing board. As time passed, it was getting more and more complicated, i dropped the project. But will definitely check pyclips.

  21. #16
    hatemachine's Avatar
    hatemachine is offline Jr. VIP
    Join Date
    Jan 2011
    Posts
    268
    Thanks
    147
    Thanked 1,059 Times in 336 Posts

    Default Re: Youtube cookie issues

    One cannot really compare MangoDB to MySQL because MangoDB is a NoSQL database. It's like comparing Apples to Oranges. I have tried HtmlUnit, and it's painfully slow, there's even a .NET wrapper and that's even slower, it's just terribly slow. Its javascript engine isn't that good either, there are no good solutions when it comes to javascript crap.
    soon.jpg

  22. #17
    safetys is offline Newbies
    Join Date
    May 2013
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by theMagicNumber View Post
    True.
    I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
    For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

    Have you used HtmlUnit(Java) the library has some sort of JS processor?
    Have you get the bgresponse value, I alse want to use the google keyword tool, But now, I have spent serval days, and still ca'nt get the bgreponse value, So........

  23. #18
    safetys is offline Newbies
    Join Date
    May 2013
    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by YouFeelMeDawg? View Post
    The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

    Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

    Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
    So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
    Have you sovle the bgresponse and login by httprequest, I really want to know.

  24. #19
    YouFeelMeDawg? is offline Banned - see signature
    Join Date
    Aug 2011
    Posts
    268
    Thanks
    21
    Thanked 353 Times in 134 Posts

    Default Re: Youtube cookie issues

    Quote Originally Posted by safetys View Post
    Have you sovle the bgresponse and login by httprequest, I really want to know.
    I haven't, it would require to translate the javascript function into some other language. What you can do though is basically do a hybrid solution, where you simply automate the browser to log in and grab the cookies after the login, and just insert the cookies in all of your youtube requests, so you can end up doing rating, commenting, uploading etc with cookies that are logged in.
    This member has been permanently banned from BHW.

  25. #20
    cipango is offline Junior Member
    Join Date
    Mar 2011
    Posts
    120
    Thanks
    46
    Thanked 24 Times in 15 Posts

    Default Re: Youtube cookie issues

    A hybrid solution is the way to go, the bgresponse is not easy to calculate.


Similar Threads

  1. Youtube compilation - Copyright issues
    By frdrik in forum Black Hat SEO
    Replies: 2
    Last Post: 02-23-2013, 09:52 AM
  2. Issues with linking Adsense to Youtube
    By imnotragoo in forum Adsense
    Replies: 0
    Last Post: 05-01-2012, 11:06 AM
  3. YouTube thumbs up issues?
    By jakeparker in forum Social Networking Sites
    Replies: 4
    Last Post: 01-19-2012, 05:06 PM
  4. YouTube Comment Issues.. Anyone else having troubles?
    By jakeparker in forum Social Networking Sites
    Replies: 4
    Last Post: 12-29-2011, 06:29 AM
  5. youTube Promotion issues
    By luke82817 in forum YouTube
    Replies: 7
    Last Post: 01-15-2009, 06:06 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  




BlackHatWorld on Twitter BlackHatWorld on FaceBook


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103