1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Coding first bot - Couple of questions!

Discussion in 'C, C++, C#' started by TheeAriGrande, Aug 11, 2013.

  1. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    I just began coding my first bot this morning. I'm planning to have proxy support, multi-threading support, process logging, etc. I have a few questions that I'd love answered, if you guys don't mind. By the way, I'm coding in C#.

    How should I go about multi-threading? My thoughts say that I should have a thread pool.
    How should I actually process a file of proxies (or usernames/passwords)? I'm thinking that the file should be processed, the data validated, and then the data added to a list. From there, a method would add the list into a queue for the thread pool.

    I've been coding PHP for years now, and have been creating and using object oriented code for about a year. In PHP the script would usually process a file into and array and use a loop to do call the same method over and over again with different parameters. Would this somehow apply to C# to? The only thing different would be this: loop over a list of data and add the method with it's parameters to the thread pool queue?​

    I've done tons of searches on google, been at it for a few hours during the day. I couldn't find anything solid. Any help would be appreciated, even the tiniest bit.

    All the best, tAG.
     
    • Thanks Thanks x 2
    Last edited: Aug 11, 2013
  2. | ALIEN |

    | ALIEN | Regular Member

    Joined:
    Jan 28, 2013
    Messages:
    277
    Likes Received:
    131
    Hey I too am interested in making bots. Actually I know some programming but so far I was unable to find any tutorials on google. :/
     
  3. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    A good alternative is something like uBot if you want an easy way to create bots. I like getting down and dirty with code, so I'd rather code everything myself.

    A good site to check out is Programmr. It's like a code academy except it has other languages like C# and C++. Once you learn the basics you could get into more complicated stuff like OOP, multi threading, etc.
     
  4. tiyowan

    tiyowan Regular Member

    Joined:
    Aug 7, 2013
    Messages:
    250
    Likes Received:
    135
    OP, do you have experience in Java? It's the same concept essentially.

    Edited: Thread pools / Parallel task library - search for them on MSDN. I'd
    suggest going with the latter, since it takes care of allocating the optimum
    number of threads for each task.

    Best not to meddle around too low with thread pools when you can let the
    framework take care about it.

    - T.
     
    • Thanks Thanks x 1
    Last edited: Aug 11, 2013
  5. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    No, unfortunately not. Maybe I should have learned Java or python before C#. lol. But I wanted to dive right into winforms and bots.
     
  6. MarketerX

    MarketerX Regular Member

    Joined:
    Mar 7, 2010
    Messages:
    398
    Likes Received:
    120
    Use the threadpool.

    If you are using visual studio I've been doing it like this, but with VB.Net. Should be the same namespace with C in VS though.

    Threading.ThreadPool.QueueUserWorkItem(Sub() DoSomething())
     
    • Thanks Thanks x 1
  7. tiyowan

    tiyowan Regular Member

    Joined:
    Aug 7, 2013
    Messages:
    250
    Likes Received:
    135
    No worries mate. Google for Bradley L. Foley - A Very Simple Introduction
    to the Task Parallel Library (TPL) in C#.

    You seem sharp (no pun intended), you'll pick it up. Just go with the TPL
    and don't mess around with thread pools in my opinion.

    Peace,

    - T.
     
    • Thanks Thanks x 1
  8. JP1016

    JP1016 Junior Member

    Joined:
    Sep 1, 2012
    Messages:
    120
    Likes Received:
    36
    Occupation:
    Student
    Location:
    BHW :-)
    Home Page:
    • Thanks Thanks x 1
  9. JP1016

    JP1016 Junior Member

    Joined:
    Sep 1, 2012
    Messages:
    120
    Likes Received:
    36
    Occupation:
    Student
    Location:
    BHW :-)
    Home Page:
    it would be nice if someone is willing to share the source code of their bot for studying purpose
     
  10. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    Thanks for the replies MarketerX and tiyowan.

    I've decided to look further into TPL and implement it. I'll most likely process the data from the file into a list. And then from there I would loop through the data and start a new task.

    I'm not sure if there is a queue for the TPL. However, I just read about a factory or scheduler, so maybe that's what that is. lol.

    Thanks for the help guys.
     
  11. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    I might create some threads when I complete the bot. Share some code snippets and my thought process behind the code. As for sharing source code, when I have the time and experience (which won't be for a few months) i'll create a simple bot and share the code.
     
    • Thanks Thanks x 1
  12. MarketerX

    MarketerX Regular Member

    Joined:
    Mar 7, 2010
    Messages:
    398
    Likes Received:
    120
    No problem, are you trying to test proxy validity in a multithreaded way? If so, you can probably make it multithreaded by making a method which will check if the proxy is valid, then call that method in the QueueUserWorkItem method in the Threadpool class. (As my first post showed an example of.)

    Here's some more example code...in VB.Net though but the syntax is so similar you should be able to figure it out.

    Code:
    Public Class Test
    
    Public Property ProxyList As List(Of System.Net.WebProxy)
    Public Property ValidProxies As List(Of System.Net.WebProxy)
    
    Public Sub LoadProxiesFromFile(fileLocation As String)
         Dim fileContents As String = My.Computer.FileSystem.ReadAllText(fileLocation)
             Dim lines() As String = fileContents.Split(vbCrLf)
             For Each line As String In lines
                      Try
                               ProxyList.Add(New System.Net.WebProxy(line))
                      Catch
                      End Try
             Next
    End Sub
    
    Public Sub TestProxyListValidity(validationUrl As String, validationString As String)
       For Each proxy As WebProxy In ProxyList
               'Each proxy has it's validity tested in it's own threadpool thread.
               System.Threading.ThreadPool.QueueUserWorkItem(Sub()
                           Using wc As New System.Web.WebClient
                           Try
                                  wc.Proxy = proxy
                                  Dim page As String = wc.DownloadString(validationUrl)
                                  If page.Contains(validationString)
                                            'The proxy successfully requested the page, and it contains the  expected validation string, add it to the list of valid proxies.
                                            ValidProxies.Add(proxy)
                                  End If
                            Catch
                            'An error occured during the web request, the proxy is invalid and wont be added to the list of valid proxies.
                            End Try
                            End Using
                            End Sub)
         Next
    End Sub
    
    End Class
    Edit: So with this code, you'd call LoadProxiesFromFile with the location of your proxylist, each proxy on its own seperate line in the format IP:port. Any valid proxies will be added to the ProxyList property. Then calling the TestValidity method with a url and string that a valid web request to the page would have (such as the pages <title> html tag. Each proxy is used with the WebClient object, and if there are no errors and the page returned when using the proxy contains the pages title tag, the proxy is valid and added to the ValidProxies property. This all happens multithreaded because QueueUserWorkItem is used in this example.
     
    • Thanks Thanks x 1
    Last edited: Aug 11, 2013
  13. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    Thanks for going into detail. Really appreciate! I think I'm going to combine you're code with the TPL.
     
  14. aymenbnr

    aymenbnr Jr. VIP Jr. VIP Premium Member

    Joined:
    Oct 14, 2009
    Messages:
    200
    Likes Received:
    75
    Home Page:
    Use Parallel task for Multi threading , Thread loop can be a little bit confusing for new comers
     
    • Thanks Thanks x 1
  15. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    Thanks! I just implemented Parallel tasks into my bot so that it can check proxies faster. And I'm going to set up TPL for the other tasks my bot needs to do. And maybe set a max degree so it doesn't run too many threads.
     
  16. sm754

    sm754 Registered Member

    Joined:
    Mar 21, 2012
    Messages:
    93
    Likes Received:
    38
    Occupation:
    Farmer
    Location:
    Azerbaijan
    Make sure to use threadlocks when accessing/changing the proxy list; otherwise you'll run into weird crashes/glitches.
    Otherwise, it sounds simple to implement multithreading in that.
     
    • Thanks Thanks x 1
  17. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    Okay, I'll look into locking. I think (again I'm new to multi threading and TPL and all that) that glitches are caused when the method depends on class properties. But then again, I have no idea what I'm talking about.

    Posted via Topify using iPhone/iPad
     
  18. sm754

    sm754 Registered Member

    Joined:
    Mar 21, 2012
    Messages:
    93
    Likes Received:
    38
    Occupation:
    Farmer
    Location:
    Azerbaijan
    Most multithreading glitches are caused when non-atomic operations are run in parallel; you're half-way done modifying an object, and another thread tries to look at or mess with the same object in the middle of this process. Locking forces other threads to wait their turn.
     
    • Thanks Thanks x 1
  19. elhefe

    elhefe Newbie

    Joined:
    Jul 9, 2013
    Messages:
    41
    Likes Received:
    4
    sm754 is exactly right. Specifically, you'll want to lock around this line:

    Code:
    ValidProxies.Add(proxy)
    
    Threading is definitely called for in this application, but locking is inherently tricky (see deadlock). Have you thought about using a thread-safe data structure like System.Collections.Concurent, or maybe even a database that will handle it all for you? A database is overkill for this particular situation, but it may come in handy for logging, and all those other cool things you're going to be coding in the future. Plus it'll let you share data across multiple computers... Something to think about.

    Might I suggest mongodb, riak, or redis? Heck, mysql would work and you might even have it running already.
     
    • Thanks Thanks x 1
  20. TheeAriGrande

    TheeAriGrande Regular Member

    Joined:
    Jul 14, 2013
    Messages:
    270
    Likes Received:
    151
    Location:
    Candlestick Park
    I had that similar consequence in my mind. Wasn't exactly it, but I was close. Lol. Thanks for the help. Right now the parallel tasks are running fine, but if I run into glitches I'll look into the concurent namespace.

    Posted via Topify using iPhone/iPad