how to scrape protected page?

extra

Power Member
Joined
May 18, 2007
Messages
533
Reaction score
102
Hello
I try to scrape password protected page.
I'm using httpwebrequest/response
but for soem reason it dont work. i try to store the cookies
in container and use it but it doesnt work.
Can anyone help me with source example for scraping protexted page in c sharp?

Thanks in advance
 
Is it a page you have to login to see? if yes are you using the correct right login string example login=username&passwd=password .. Also some pages have hidden input fields that have values you have to include in the login string.. You would have to parse these values out in include them in the login string..

I recommend using the firefox plugin Live HTTP Headers to capture the http login packet, If your not already using it.
 
You can take software like teleport pro, which can be used to scrap protected pages but you should know atleast login and password
 
Back
Top