Youtube cookie issues

ultra.marine

ultra.marine

Registered Member
Joined
Oct 5, 2012
Messages
80
Reaction score
105
Okay, I am trying to log on youtube using http web request but I am getting this feedback from youtube "Your browser's cookie functionality is turned off. Please turn it on"


Any help, hits or anything I will be thankful!

Code: 
       Dim tempCookies As New CookieContainer
        Dim postData As String = "continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26feature%3Dsign_in_button%26hl%3Den_US%26next%3D%252F%26nomobiletemp%3D1&service=youtube&dsh=-9120265060045154752&hl=en_US&GALX=xgj0d0m6phE&pstMsg=1&dnConn=&checkConnection=youtube%3A212%3A1&checkedDomains=youtube&timeStmp=&secTok=&_utf8=%E2%98%83&bgresponse=%21A0JW0zonfLaO7kT-7m88MYDfHgIAAABUUgAAAA4qAQsxnzRQHLCiYtetendJsDYVMbUPHMynXE9B_fvNVQJKhD2KyxW9E03fKTVJ1tcKd2KN4ivJz-O7_MzUHmL6v2NVMN-O10Ms5W6SrHvumwMXoNeozmvKHidnGmS1h-Y0ynR_JQl_wUoHydgg6J7OctiL9Zafvok5BSlj-KBRH-BUSLUrtoMuv9tna37N-xwRAQLiAmmOVyfQsF1uov3O-iHlB5bZL8RzszIWeiSJacz7dkCZvR3YbKR7NRF0fReiQtWQx4atJ2KRJ6ogScEYsa6RTqAXboEaqqyEsd1sE0pjmmoAOQXL8OngfKuTyCy7ac-ILQLszuQyspXGvpY_wqiHJIC1lCX_aYnynq0&Email=USERNAME%40MAIL.COM&Passwd=MYPASSWORD1&signIn=Sign+in&PersistentCookie=yes&rmShown=1"


        Dim encoding As New UTF8Encoding
        Dim byteData As Byte() = encoding.GetBytes(postData)


        Dim postReq As HttpWebRequest = DirectCast(WebRequest.Create("https://accounts.google.com/ServiceLoginAuth"), HttpWebRequest)
        postReq.Method = "POST"
        postReq.KeepAlive = True


        postReq.CookieContainer = New CookieContainer()
        postReq.ContentType = "application/x-www-form-urlencoded"
        postReq.Referer = "Https://accounts.google.com/ServiceLogin?uilel=3&service=youtube&continue=http%3A%2F%2Fwww.youtube.com%2Fsignin%3Faction_handle_signin%3Dtrue%26feature%3Dsign_in_button%26hl%3Den_US%26next%3D%252F%26nomobiletemp%3D1&passive=true&hl=en_US"
        postReq.UserAgent = "Mozilla/5.0 (Windows NT 5.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.872.0 Safari/535.2"
        postReq.ContentLength = byteData.Length


        Dim postreqstream As Stream = postReq.GetRequestStream()
        postreqstream.Write(byteData, 0, byteData.Length)
        postreqstream.Close()
        Dim postresponse As HttpWebResponse


        postresponse = DirectCast(postReq.GetResponse(), HttpWebResponse)
        tempCookies.Add(postresponse.Cookies)
        loginCookie = tempCookies
        Dim postreqreader As New StreamReader(postresponse.GetResponseStream())


        Dim thepage As String = postreqreader.ReadToEnd


        RichTextBox1.Text = thepage
 
Advertise on BHW
You are creating a new cookie container AFTER going to the login page, you should create a cookie container for the whole process / manage cookies in a class.

Also this alone would still not be enough to login, you need to make sure to follow ALL of the Google URL's & parse out the Javascript redirects etc that Google run, there is quite a bit to do to login to YT properly using HTTPWebRequest while also detecting all potential responses & also Google login now uses BotGuard so there are certain pages that you will not be able to view without HTTPWebRequest without decoding BotGuard's requests (Good luck!!!).

If you are struggling with a login with HTTPWebRequest then I recommend switching to WebBrowser as YT monitor more than just the individual calls to make actions so you will have to spend a long time adding in all of the AJAX calls that are made within a browser just making a simple action.


Edit: Just noticed you are not visiting the login page at all, you are just posting the data, Google has a lot of variables that you need to parse & use when posting the login & also cookies are set when visiting the login page.
 
hehe thanks man for everything :D I think i am visiting the login page but i'll try and look around. I already have working program with web browsers but the problem is it can't be multithreaded so I thought to change to web requests... seems this will be harder that I thought. :/

Thanks anyway
 
ultra.marine said:
I have seen it and tried it, it doesn't work m8. You don't even have cookie container, without cookies there is no use of post request at all.
Click to expand...

Of course i am using cookies. The WebClientEx class inherits WebClient and add cookie support.

And you haven't tried it at all, as i coded it yesterday, to help some ignorant fuck, who in return told me:
"But i wanted HttpWebRequest not WebClient" - WebClient is wrapper around webrequest/webresponse.
 
Last edited:
theMagicNumber said:
Of course i am using cookies. The WebClientEx class inherits WebClient and add cookie support.

And you haven't tried it at all, as i coded it yesterday, to help some ignorant fuck, who in return told me:
"But i wanted HttpWebRequest not WebClient" - WebClient is wrapper around webrequest/webresponse.
Click to expand...

You have some behavior issues, watch your language. You neither coded that for me, neither it's in vb.net, neither helps me understand where I am wrong.

Sometimes you may believe people word around, as some of them are honest
Code: 
http://img5.imageshack.us/img5/9043/sdfasf.png

See the date where I downloaded and I deleted your program after trying it. You think I would bullshit without trying it ? You don't have to go on rage for nothing. Control your emotions ;)

However, after your post, I thought... hmmm i should re-try it and I found it working, that "always true bug was the reason I thought doesn't work", but actually works.

Thanks, next time you can politely say things, not using "fucks and shits"... I thought we're all here to get and offer some help, or not?
 
Well your example clearly doesn't work because, you have hard coded "dsh" and "GALX" values in your code.
Those values are dynamic and are different on every single reload, in other words they are unique identifier.

You obviously going to get problems when doing the post, you have to get those values from the html when you request the login page.
 
ultra.marine said:
You have some behavior issues, watch your language. You neither coded that for me, neither it's in vb.net, neither helps me understand where I am wrong.

Sometimes you may believe people word around, as some of them are honest
Code: 
http://img5.imageshack.us/img5/9043/sdfasf.png

See the date where I downloaded and I deleted your program after trying it. You think I would bullshit without trying it ? You don't have to go on rage for nothing. Control your emotions ;)

However, after your post, I thought... hmmm i should re-try it and I found it working, that "always true bug was the reason I thought doesn't work", but actually works.

Thanks, next time you can politely say things, not using "fucks and shits"... I thought we're all here to get and offer some help, or not?
Click to expand...

Hello,
i apologize my behaviour was unacceptable, had a bad day.
Anyway here is what you have to do:
1) navigate to:
Code: 
https://accounts.google.com/ServiceLogin?continue=http%3A%2F%2Fwww.youtube.com%2Fsignin&service=youtube&hl=en
2) parse the <input> from <form> with ID gaia_loginform. There are 18 variables. You can use regex or xpath to do that. GALX and dsh are dynamic, so you can only parse these parameters and construct the post data.
3) DON'T forget to Url Encode the data before posting it
4) set AllowAutoRedirect to true and post the data to
Code: 
https://accounts.google.com/ServiceLoginAuth
5) DONE.
 
theMagicNumber said:
Hello,
i apologize my behaviour was unacceptable, had a bad day.
Anyway here is what you have to do:
1) navigate to:
Code: 
https://accounts.google.com/ServiceLogin?continue=http%3A%2F%2Fwww.youtube.com%2Fsignin&service=youtube&hl=en
2) parse the <input> from <form> with ID gaia_loginform. There are 18 variables. You can use regex or xpath to do that. GALX and dsh are dynamic, so you can only parse these parameters and construct the post data.
3) DON'T forget to Url Encode the data before posting it
4) set AllowAutoRedirect to true and post the data to
Code: 
https://accounts.google.com/ServiceLoginAuth
5) DONE.
Click to expand...
ya but your forgetting one important thing.
By default if you dont generate the "bgresponse" value instead of the "js_disabled" value that is set on, youtube will know that you don't have javascript enabled and will actually hurt you on the long run. In other words, now they can tell if your using a browser or http request because of that variable.
 
YouFeelMeDawg? said:
ya but your forgetting one important thing.
By default if you dont generate the "bgresponse" value instead of the "js_disabled" value that is set on, youtube will know that you don't have javascript enabled and will actually hurt you on the long run. In other words, now they can tell if your using a browser or http request because of that variable.
Click to expand...
I didn't forget that. Yes, it is leaving a huge footprint. Maybe you will offer a solution to decode the JS ?
I will deal with it shortly, but currently i am not interested in youtube.
 
theMagicNumber said:
I didn't forget that. Yes, it is leaving a huge footprint. Maybe you will offer a solution to decode the JS ?
I will deal with it shortly, but currently i am not interested in youtube.
Click to expand...

The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
 
Last edited:
YouFeelMeDawg? said:
The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
Click to expand...
True.
I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

Have you used HtmlUnit(Java) the library has some sort of JS processor?
 
theMagicNumber said:
True.
I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

Have you used HtmlUnit(Java) the library has some sort of JS processor?
Click to expand...

I am switching mainly to Python basically, Java would be my second choice or ruby on rails. The thing is I am stepping away from C# or anything that uses things like .net framework or is only usable on a specific OS, like windows.

My next project will hopefully be a youtube bot that is scalable on the cloud,starting with using a public cloud like amazon, rackspace, and eventually do my private/hybrid cloud, that means using vps/dedi with public clouds to make my own private/hybrid cloud. Doing it this way I can just use the vps/dedi as a factory of workers with celery(http://www.celeryproject.org/) and make the modules for the tasks(youtube, facebook,twitter, web 2.0).

So I am taking literally like 6 months to learn python better and all of its useful libraries
like boto, for amazon cloud services

I need to learn py-redis, to use redis caching software and as a AMPQ broker for celery, then I have to learn mongoDB since mysql is really slow and its hard to scale at an enterprise level.On top of that, I have to use extensively webhooks to be able to connect a lot of 3rd party services together instead of polling their API for results,so thats another thing I will use extensevely

In conclusion I will be using selenium to automate browser bots(chrome, firefox, ie) and about the htmlunit, I think the best would be to not use it because its not a full http wrapper that has PUT,GET,DELETE,POST. You can do GET nicely with htmlunit but not anything else.
So I am still deciding what web framework to use with python.

Now I am also want to add a ruled-based engine, the sad part is that JAVA has a lot of opensource, freeware options like Dross,Jess,JRuledEngine,OpenRules etc but for python I don't really have that many option.

For python for a ruled-based engine I am looking at pyclips(the python module for the CLIPS ruled based engine developed by NASA), or Intellect 1.4. So thats another thing I am looking forward in learning so I can make my next bot sort of like a script based, where I can write rules instead of hard coding anything making it able to code an Expert System for IM bot that is deployable on a cloud instead of desktop-based.

So ya, am not really coding or updating my yt bots to be honest. I am more learning about all of these other tools/libraries to make my own enterprise cloud based IM tool, then I can sell my software-as-a-service meaning, I can charge people per hour to use my bot with their proxies and accts, instead of releasing my bot and giving them the exe.

All I know from youtube is from what I have coded extensevely in the past 2 years, lately these past 6 months I have just bee learning a lot about different 3rd party services I can use to make my next bot.
 
Last edited:
You have interesting ideas and goals. I am also considering switching to Python, actually it will be more of a test, i would like to test .NET and Python performance for web automation.

I think a cloud based solution will be too expensive than selling desktop bots, it will be harder to maintain and if your user base grows too fast, probably you won't have the resources to cope with that in time. I don't have a lot of experience with amazon EC, but i did some tests with google app engine and it is really expensive, it is cheaper to start with dedis.

I am not familiar with celery, but it reminds me of Majestic12 project, they started as Distributed Search Engine and promised to develop open source search engine, but in the end they created a paid SEO intelligence service.

About mongoDB, this is something worth checking. I am a performance freak and i made a few searches, it seems that mongoDB is way faster than MySQL.
Also using SSDs for the databases will increase the performance a lot. For example my MS SQL database is 40%-50% faster than hosting it on a regular HDD.

I am interested only in the javascript evaluator in HtmlUnit, basically decoding/emulating the JS is necessary before the POST, i tried some .NET solutions but they were working with really basic javascript.

I started my ruled-based crawler/scraper 1-2 years ago as a simple project, at least it was simple on my drawing board. As time passed, it was getting more and more complicated, i dropped the project. But will definitely check pyclips.
 
One cannot really compare MangoDB to MySQL because MangoDB is a NoSQL database. It's like comparing Apples to Oranges. I have tried HtmlUnit, and it's painfully slow, there's even a .NET wrapper and that's even slower, it's just terribly slow. Its javascript engine isn't that good either, there are no good solutions when it comes to javascript crap.
 
theMagicNumber said:
True.
I use browsers from time to time, google keyword tool mainly, but usually i am scraping google and i don't give a damn about footprints.
For youtube, i really don't have any recent experience and probably the bgresponse variable set to js_disabled may lead to a lot of banned accounts/videos.

Have you used HtmlUnit(Java) the library has some sort of JS processor?
Click to expand...

Have you get the bgresponse value, I alse want to use the google keyword tool, But now, I have spent serval days, and still ca'nt get the bgreponse value, So........:)
 
YouFeelMeDawg? said:
The solution I have is pretty much, automate the browser via selenium, grab the cookies, then plug them into my http bot. In other words, I am pretty much using a hybrid bot now, browser where I need it, and http request where I don't need it.

Decoding the JS will require a really good person with a good skillset, and if its done believe me it won't be cheap. By decoding that, you can pretty much make gmail,youtube, google accts creators by http request instead of browser.

Eventually, youtube will require that you submit a valid bgresponse on login and login won't be possible without a valid bgresponse, meaning http bots will be useless for logging in unless you are able to decode the bgresponse and implement it on your bot.
So who ever told the OP to go with http request with youtube now, told him to do it on the wrong time. If I was the OP I would go browser, or hybrid, but never full http request it leaves too many footprints and there is cleary a big difference on doing it browser style vs http request especially where JS code is used by youtube/google.
Click to expand...

Have you sovle the bgresponse and login by httprequest, I really want to know.
 
safetys said:
Have you sovle the bgresponse and login by httprequest, I really want to know.
Click to expand...

I haven't, it would require to translate the javascript function into some other language. What you can do though is basically do a hybrid solution, where you simply automate the browser to log in and grab the cookies after the login, and just insert the cookies in all of your youtube requests, so you can end up doing rating, commenting, uploading etc with cookies that are logged in.
 
A hybrid solution is the way to go, the bgresponse is not easy to calculate.
 
Advertise on BHW
You must log in or register to reply here.
Sign up now!

Main Menu

Home Main Forum List Black Hat SEO White Hat SEO BHW Newbie Guide Blogging Black Hat Tools Social Media Downloads

Marketplace

Account Selling Content / Copywriting EBooks / Methods Hosting Misc Proxies For Sale SEO - Link building SEO - Packages Social Media Social Media - Panels Web Design More

Making Money

Affiliate Programs Hire a Freelancer IM Journeys Making Money Pay Per Click Site Flipping Want to Buy

BlackHat World

Dispute Resolution Forum Suggestions & Feedback Introductions Lounge
Back
Top