1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Create Multiple Background workers with HttpWebRequest

Discussion in 'Visual Basic .NET' started by simpleonline1234, May 20, 2012.

  1. simpleonline1234

    simpleonline1234 Junior Member

    Joined:
    Jan 26, 2010
    Messages:
    169
    Likes Received:
    13
    Okay so I've dug around all day and found that using multiple "background workers" would do what I would need for my application.

    How can I create 4 background workers and then feed the urls to the webrequest from a listbox?
    Code:
    Dim request As HttpWebRequest = HttpWebRequest.Create(URLGOESERE)                 
    Dim response As HttpWebResponse                 
    response = CType(request.GetResponse, HttpWebResponse)                 
    Dim webstream As Stream = response.GetResponseStream                 
    Dim streamreader As New StreamReader(webstream, Encoding.UTF8)                 
    Dim html As String = streamreader.ReadToEnd                 
    Messagebox.show(html.tostring) 
    I'm basically needing to have each webrequest access the listbox, grab a URL, process the webrequest and then go grab the next url on file.

    I am trying to have that running 4x at a time until the sites are all looped through.

    I've read a background worker is good vs. multi threading because it's easier to code.
     
  2. omaigadlol

    omaigadlol Registered Member

    Joined:
    Oct 25, 2008
    Messages:
    65
    Likes Received:
    18
    What language is this? I don't mind researching something.. if I know what it is.
     
    • Thanks Thanks x 1
  3. jazzc

    jazzc Moderator Staff Member Moderator Jr. VIP

    Joined:
    Jan 27, 2009
    Messages:
    2,468
    Likes Received:
    10,148
    It 's VB.NET - Thread moved to the proper section btw :)
     
  4. andee

    andee Regular Member

    Joined:
    Jul 24, 2010
    Messages:
    218
    Likes Received:
    83
    lol I thought this would be easy for someone whos created the "Ferrari version of Xrumer and Scrapebox"

     
  5. simpleonline1234

    simpleonline1234 Junior Member

    Joined:
    Jan 26, 2010
    Messages:
    169
    Likes Received:
    13
    Did I say Ferrari version? Oopps...I mean't to say the Pinto version...haha...I will be the Ferrari version when I'm done with it..well...if I ever get down with it. haha
     
  6. accelerator_dd

    accelerator_dd Jr. VIP Jr. VIP Premium Member

    Joined:
    May 14, 2010
    Messages:
    2,441
    Likes Received:
    1,005
    Occupation:
    SEO
    Location:
    IM Wonderland
    Well I am more of a C# guy but I'll try to help out as much as I can.

    Instead of doing the threads handling yourself, why don't you work with a thread pool? I needed something similar around 6 months ago or so and that's how I handled it. You can queue as many tasks as you want, pass arguments, and they'd all get done. You would also limit the thread number.

    The key to start them and have the application running without it freezing would be making one new thread from which things would run - so if you want to stop the execution you just kill that thread, since it's the main thread for the queue, it would terminate all threads. Since its started from a separate thread, it wouldnt kill the application. There are a few other ways of doing it but they're a bit more complicated.

    Let me know if you need any more help.
     
  7. simpleonline1234

    simpleonline1234 Junior Member

    Joined:
    Jan 26, 2010
    Messages:
    169
    Likes Received:
    13
    Thanks accelerator_22. That's the idea I was hunting for.

    Cool beans...that makes perfect sense. I believe I will toss the webrequest for the captcha into the thread pool and then create an event to kill the thread by toggling the dispose on the captcha submit button which also sends a webrequest POST for create the account...thanks for your help.
     
  8. simpleonline1234

    simpleonline1234 Junior Member

    Joined:
    Jan 26, 2010
    Messages:
    169
    Likes Received:
    13
    I think I have it ready to wrap up into an EXE and then I hit a road block which turns into another road block.
     
  9. andee

    andee Regular Member

    Joined:
    Jul 24, 2010
    Messages:
    218
    Likes Received:
    83
    its all good mate, I was only having a laugh .

    forget the background workers and just use threads, Check out this post by smack

    http://www.blackhatworld.com/blackh...6-mulitthreading-httprequest.html#post2147637

    he provides easy to follow pseudo code on how to multithread a list of urls, its how I learnt to do it...... you wont be able to multi-thread a listbox either (i dont think anyway), read it into an arraylist or something first