Windows Server for high CPU tasks - is this even possible??

Discussion in 'Web Hosting' started by examo1, Feb 5, 2018.

  1. examo1

    examo1 Newbie

    Joined:
    Dec 20, 2017
    Messages:
    18
    Likes Received:
    3
    Hi, I am using a custom script which requires 40 browsers to be open/running at roughly the same time (multi-threaded) plus some background processes. The data on the page is continuously updated and each browser consumes around 0.45 GB in memory whilst running.

    I am currently able to open around 6-8 browsers comfortably on my current machine, after which things start slowing down. My current specs are:

    Intel Core i7 2600 CPU @ 3.40GHz
    #Cores: 4
    #Threads: 8
    10 Mbps Internet Speed


    From reading similar threads, it looks like I need a dedicated server - maybe from wholesaleinternet.net/dedicated/ (thanks @djelica), but I am not sure what specs would work and if I should be running this over multiple servers rather than a single one at once?

    Any advice on server specs, or even other suppliers would be much help. Thanks.
     
    • Thanks Thanks x 2
  2. Viniyois

    Viniyois Junior Member

    Joined:
    Aug 2, 2017
    Messages:
    168
    Likes Received:
    117
    Gender:
    Male
    Yea, if you gonna use VPS for that, you gonna spend lots of money. Dedicated is what you need. you need high CPU + RAM and that's when it becomes problematic.

    I am dealing with the similar situation and in the long term having high speed internet + PC at home is cheaper option in my opinion (electricity bill + internet + PC price was much cheaper long term when I calculated). But you have to run it 24/7 at home and make sure connection is not down.

    You can get dedicated, but powerful stats cost quite a lot.
     
    • Thanks Thanks x 2
  3. HudsonWebServices

    HudsonWebServices Jr. VIP Jr. VIP

    Joined:
    Nov 28, 2015
    Messages:
    1,193
    Likes Received:
    229
    Location:
    Texas, US
    Home Page:
    Nailed it right on the head!
     
    • Thanks Thanks x 2
  4. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    Thank you for the credit, I appreciate it.

    While I'm here ill try to quickly give my two cents since i faced some similar tasks in the past.

    First, scalability is not always easy task. You may get better performance with better specs but you may not get improvement what you were expecting.

    Second, are you sure that you need browsers at all? Removing browsers from the setup and doing same thing browsers do programatically you can achieve massive improvement (10x to 100x or more) even on the same home machine. This of course is true if your task allows you to bypass browsers completely and that you are technically capable to do it. (For nerds: yes browsers can be fully excluded from anything, just some things are pain in the ass to do programatically that you want to use browser and not fck with exclusion)

    Your home pc specs are not bad (you should have mentioned ram too), but i suspect that you have HDD and not SSD. Trust me, you want SSD if you dont have one.

    And of course, if profits are big enough to pay server rent, then why not give it a go.
     
    • Thanks Thanks x 3
  5. HudsonWebServices

    HudsonWebServices Jr. VIP Jr. VIP

    Joined:
    Nov 28, 2015
    Messages:
    1,193
    Likes Received:
    229
    Location:
    Texas, US
    Home Page:
    I'd honestly go with wholesale as well, I'd grab whatever you can afford. They have very limited availability. This one is available , Dual Xeon E5-2670v2 96GB 2x480GB SSD Preconfig.
     
    • Thanks Thanks x 1
  6. examo1

    examo1 Newbie

    Joined:
    Dec 20, 2017
    Messages:
    18
    Likes Received:
    3
    Thanks guys, all this info is very helpful and much appreciated! The plan is to eventually have a better PC/setup at home, but as the initial investment for a server is low compared to all the upgrades/replacements - I would like to try a 'proof of concept' using this first, ideally.

    Unfortunately the browser setup is better suited for this as the site sends live data and updates it's structure regularly (encrypting packets, SSL, etc). The cost in time to manually review and adjust every time it's updated would end up costing more than the hardware itself.

    Thanks, that's the one I was looking at too. My current RAM is 8GB with 4 Cores/8 Thread (HDD) and as mentioned, handles around 6 browsers ok. Do you think this one should handle 40 browsers @ ~0.45 GB/Browser?

    My only worry with these servers is the OS, as I am developing using Python 3 and Windows 10 based tools, which won't be supported in Windows 2008 Enterprise. Is it allowed/fairly straight forward to change to Win10 on a dedicated server such as this?

    Edit: That ones out of stock now, but would still be interested in seeing your thoughts on the above please, as i'll pick up a similar one when it's back in stock. Thanks.
     
  7. MuayThai

    MuayThai Jr. VIP Jr. VIP

    Joined:
    Aug 25, 2015
    Messages:
    1,061
    Likes Received:
    333
    Home Page:
    IMHO, you need a machine with a huge amount of RAM; the CPU(s) is not that important for you.
     
    • Thanks Thanks x 1
  8. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    Server with that specs should be able to handle over 100 browsers running simultaneously. If you do the math 100 browsers x 0.45 gb ram each = 45gb, so you should have room for more if needed. Those two CPUs are beasts. So give it a go.

    For OS doubt. You can message them and simply ask if they can install windows 10 for you (they will replay in short time, they want you as a customer). In the past I used VPS with windows 7 installed (from other company tho), I don't see reason not to be able to have windows 10.
     
    • Thanks Thanks x 1
  9. NxtGenLedz

    NxtGenLedz Jr. VIP Jr. VIP

    Joined:
    May 7, 2014
    Messages:
    358
    Likes Received:
    104
    OP, what is your budget? Also, have you thought about buying an old server from ebay for your home? Would save you a ton. I'd imagine you'd want a very high cpu core count (im talking 24 cores) and lots of ram.
     
  10. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    One more thing: if you are starving for performance, and you are getting two SSD's included, you may request RAID 0 ssd setup. This setup allow discs to write to both at the same time (almost the same speed as writing to one) and reading from both in the same time (almost doubling reading speed). Just keep in mind that you need to do regular backups on this setup since if one disc fails, data on both is destroyed.

    Good luck and best wishes
     
    • Thanks Thanks x 1
  11. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    724
    Likes Received:
    545
    Location:
    Poland
    Go with dual Xeon 2670 or similar, ssd etc. For heavy cpu tasks dual Xeon 2670 is comparable to 3-5 i7 2600.
    Wholesale is one of cheapest dedi sellers on market, you wont find better deal.

    There is better option than RAID0 - get server with alot of ram and put bots on ramdrive. Nothing is comparable to ramdrive.

    Edit:
    Not true, im running similar setup across many servers since few years, CPU is always bottleneck, not ram.

    For example:
    Xeon 1230v3 cpu, 32 gigs of ram. Usually CPU is maxed when ram usage is around 14gb
    Dual Xeon 2660 - CPU is killed when 50 GB of ram is consumed.

    All depends what you are actually doing. If you just open website, sit there for 10 minutes and do nothing - you need ram.
    If you open website and you perform some tasks, clicks or page reloads - CPU usage will be high, ram will stay low.
     
    • Thanks Thanks x 2
    Last edited: Feb 6, 2018
  12. NxtGenLedz

    NxtGenLedz Jr. VIP Jr. VIP

    Joined:
    May 7, 2014
    Messages:
    358
    Likes Received:
    104
    • Thanks Thanks x 2
  13. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    724
    Likes Received:
    545
    Location:
    Poland
    • Thanks Thanks x 2
  14. WhcTech

    WhcTech BANNED BANNED

    Joined:
    Feb 6, 2018
    Messages:
    1
    Likes Received:
    0
    Gender:
    Male
    this can be interresting for someone who want to do proxy, remote desktop etc..
     
  15. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    Good suggestion for ramdrive, i completely forgot about it. RAID0 will increase computer performance overall and that includes boot time, programs startup time, anything that reads data from disc where ramdrive is not available (funny, even ramdrive startup time), and best of all no action is needed after initial setup.

    So to sum it up: Shitload of RAM + RAID0 + ramdrive.

    some serious performance porn going on here
     
    • Thanks Thanks x 1
  16. vinku

    vinku Jr. VIP Jr. VIP

    Joined:
    Jan 22, 2008
    Messages:
    1,543
    Likes Received:
    2,384
    Gender:
    Male
    Occupation:
    Making Money With Android
    Location:
    Poland
    Home Page:
    Depends what you do. It's possible to make...headless browser for your task. If it's bot, do that.
     
  17. examo1

    examo1 Newbie

    Joined:
    Dec 20, 2017
    Messages:
    18
    Likes Received:
    3
    I'm looking at around $100-120/month in server cost, obviously the cheaper the better! As this is more for a 'test case' type project - I would prefer renting a server for at least a month and if goes well and I see it being long term, going with a home based server as you recommended. Honestly though, i'm surprised to see that it would only take ~4 months to pay off for the hardware itself...

    Thanks for that, I understand the ram is within limits but as @satyr85 mentions, I feel that the CPU might cap out before reaching those limits. As when I ran some tests on my own desktop, I noticed the program running very slow even when RAM was available and found that with 6 browsers (2.7GB) + ~2.5GB for OS/background processes - total 5.2GB was pretty much my limit for this application, although I had 8 GB total available.

    Also thanks for the suggestion regarding OS, i've sent them a message and will keep you guys updated on their response - for future reference.

    As this is really for testing purposes, what are your guys thoughts on multiple smaller servers? Eg, something like this:
    Dual Xeon 5520 3TB Preconfig
    2.26GHz - 2 processors, 8 Cores / 16 threads, 24GB DDR3, 3TB SATA


    How many browsers do you think this setup could handle? As at $40 it won't be too bad for testing and I can pass data via websockets between 2-3 servers to get the same end result.
     
  18. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    Exactly what I wrote, but programming would be time consuming process as OP stated, and in most cases you dont want to fck for days with it, just want damn thing running asap.
     
    • Thanks Thanks x 1
  19. satyr85

    satyr85 Jr. VIP Jr. VIP

    Joined:
    Aug 7, 2011
    Messages:
    724
    Likes Received:
    545
    Location:
    Poland

    I would say nope. Its comparable to Xeon 1230v3 or little slower, so comparable to your current i7 CPU.
    + HDD, you wont benefit from hdd.
     
    • Thanks Thanks x 1
  20. djelica

    djelica Regular Member

    Joined:
    Jun 26, 2009
    Messages:
    208
    Likes Received:
    88
    Same what i thought. For your setup go with more cores as they will be able to dedicate to more processes so you will get better performance in the end.
     
    • Thanks Thanks x 1