1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Can't scrape data from Website

Discussion in 'C, C++, C#' started by MisterNick, Oct 31, 2014.

  1. MisterNick

    MisterNick Registered Member

    Joined:
    Oct 22, 2014
    Messages:
    54
    Likes Received:
    45
    Occupation:
    Programmer
    Location:
    Tbilisi,Georgia
    Hello guys,

    I'm trying to scrape html from "fakenamegenerator" but it's showing me an error.Can you please help me?

    I'm using this code:
    HttpWebRequest myRequest = (HttpWebRequest)WebRequest.Create("fakenamegenerator url");
    myRequest.Method = "GET";
    WebResponse myResponse = myRequest.GetResponse();
    StreamReader sr = new StreamReader(myResponse.GetResponseStream(), System.Text.Encoding.UTF8);
    string result = sr.ReadToEnd();
    sr.Close();
    myResponse.Close();
    textBox1.Text = result;

    For other sites it's perfectly working but for fakenamegenerator it's not.Can you please suggest me what to do?
    thanks
     
  2. rawr00

    rawr00 Newbie

    Joined:
    Feb 24, 2014
    Messages:
    7
    Likes Received:
    3
    No need for all that, honestly. You can make good use of HTTPClient or WebClient classes here.

    Code:
    
    private async Task<string> GetFakeNameSrcAsync()
    {
       using (var client = new WebClient())
        {
           string src = await client.DownloadStringTaskAsync("URL");
           return src;  
        }
    }
    
    
    The above is an example. I have no idea what kind of error you are getting in the response, but you should check the headers with Fiddler or something similar and add them to the request. It may just be the UserAgent, but you should be able to figure it out pretty easily by checking out the requests made with your browser.

    To add headers using the WebClient class, for example, is done like this:

    Code:
    
    client.Headers.Add("USERAGENT");
    client.Headers.Add("Accept-Encoding: gzip, deflate");
    
    
     
  3. Conadovan

    Conadovan Newbie

    Joined:
    Oct 24, 2014
    Messages:
    7
    Likes Received:
    5
    Please don't just post "It's showing me an error" and expect to get a helpful answer.

    Code support 101 is POST THE ERROR(S) YOU GET.

    Makes helping you 100 times easier than trying to figure out ourselves what error the code is going to throw and then figuring out ourselves how to fix it.