1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

VB.net web scrapping question

Discussion in 'Visual Basic .NET' started by christoss1959, Jun 1, 2011.

  1. christoss1959

    christoss1959 Senior Member

    Joined:
    Nov 25, 2010
    Messages:
    894
    Likes Received:
    1,150
    Home Page:
    Hey guys,
    I am quite experienced with VB.NET but I've stumbled upon something that got me puzzled for weeks now.
    My problem is some websites use the document.write method to generate some of their HTML content. In a regular browser the results appear fine and I can easily copy it but I can't find a way to obtain the results of this method from WebWagon or httpwebrequest or even webbrowser.
    Is there a trick I could use? Am I missing something? They are the method to generate urls if that is helpful.
     
  2. christoss1959

    christoss1959 Senior Member

    Joined:
    Nov 25, 2010
    Messages:
    894
    Likes Received:
    1,150
    Home Page:
    Found it this seems to do the trick
    Code:
        
     For Each ClientControl As HtmlElement In web.Document.Links
                ListBox.Items.Add(ClientControl.GetAttribute("href"))
    Next