1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Server admins - Nginx

Discussion in 'Black Hat SEO' started by devi, Mar 13, 2010.

  1. devi

    devi Newbie

    Joined:
    Mar 2, 2007
    Messages:
    21
    Likes Received:
    1
    Just wondering if anyone here has seen, or had any experience using a server callled Nginx (add .org to check it out). It is basically a proxy/reverse proxy/load balancer/web server all in one, depending on what you want it to do, and it has been said to drastically improve loading times when configured properly.

    I mostly took notice due to the configuration options. Check out the modules it has and you'll see what I mean.

    I know we have a lot of clever people here, so I'm curious as to what methods you see being useful. Take a particular look at the proxy config options, the upstream module, and the GeoIP module... if I am correct, you could theoretically use this server to simply serve static pages instantly, then route any other traffic to whichever server/page you wanted to.

    What are some creative config setups you could see potentially being created by using this?
     
  2. voyevoda

    voyevoda Regular Member Premium Member

    Joined:
    Mar 21, 2010
    Messages:
    217
    Likes Received:
    97
    Location:
    Eastern Front
    My company runs a top 1500 Alexa website using 12 instances of nginx as reverse proxies/load balancers.

    I also use it on all my servers for serving static content and proxying requests back to PHP (fcgi) or thin/mongrel (for Rails apps).

    nginx absolutely destroys Apache in almost every meaningful way. It's far easier to configure and uses a fraction of the memory/cpu, too. :)
     
  3. eurgbp

    eurgbp Regular Member

    Joined:
    Jan 14, 2009
    Messages:
    212
    Likes Received:
    23
    Home Page:
    i concur, i use nginx myself, i hate threaded apache,
    nginx has async event driven concept, it really makes difference
    here is my nginx.conf, might help you somehow.
    had trouble with understanding subdomains, but somehow figure it out

    nginx has got very good idea of rewrite conditions IFs

    user www-data;
    worker_processes 10;
    events{worker_connections 10240;}
    http
    {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    keepalive_timeout 5;
    send_timeout 120;
    gzip on;

    server
    {
    send_timeout 120;
    listen 80 default;


    if ($host ~ ^dedic.com$) {
    return 444;
    }

    if ($host ~* ^(.*\..*)$) {
    set $sub "";
    set $domain $host;
    }

    if ($host ~* ^(.*)\.(.*\..*)$) {
    set $sub $1;
    set $domain $2;
    }


    if ($host ~* ^www\.(.*\..*)$) {
    set $sub "";
    set $domain $1;
    }


    server_name _;
    server_name_in_redirect off;
    root /home/web/$domain/$sub;
    access_log /var/log/nginx/access.log combined;
    error_log /var/log/nginx/error.log error;


    location /
    {

    root /home/web/$domain/$sub;
    index index.php index.html index.htm;
    #try_files $uri $uri/ @fallback;

    if (!-e $request_filename)
    {
    rewrite ^(.*)$ /index.php?$1 last;
    break;
    }


    location ~ \.php$
    {

    if (!-e $request_filename)
    {
    rewrite ^(.*)$ /index.php?$1 last;
    break;
    }

    fastcgi_ignore_client_abort on;
    fastcgi_intercept_errors off;
    fastcgi_buffers 32 8k;
    fastcgi_connect_timeout 180;
    fastcgi_read_timeout 180;

    fastcgi_pass 127.0.0.1:9000;
    fastcgi_index index.php;
    fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
    include fastcgi_params;
    }



    }
    }
    }
     
  4. voyevoda

    voyevoda Regular Member Premium Member

    Joined:
    Mar 21, 2010
    Messages:
    217
    Likes Received:
    97
    Location:
    Eastern Front
    Nice.

    I also tweak the gzip compression:

    Code:
    gzip  on;
    gzip_comp_level 2;
    gzip_proxied any;
    gzip_types text/plain text/html text/css application/x-javascript text/xml application/xml application/xml+rss text/javascript;
    
     
  5. eurgbp

    eurgbp Regular Member

    Joined:
    Jan 14, 2009
    Messages:
    212
    Likes Received:
    23
    Home Page:
    kljova :0) might use it also ;)
     
  6. portalweb

    portalweb Supreme Member Premium Member

    Joined:
    Jan 7, 2010
    Messages:
    1,415
    Likes Received:
    547
    Occupation:
    Hard Core Engineer
    Location:
    New York
    Hi,

    It's nice to know you're nginx expert, which is rare around here.

    I use Lighttpd as it works well with native compiled cgi-bin scripts to power my sites.

    Since you're the best Nginx administrator, is there a way to run compiled scripts through nginx web server without external scripts?

    I read the nginx documentation that nginx requires "external" cgi-bin script to run perl/compiled scripts, which is the main reason for not using nginx.
     
  7. radi2k

    radi2k Junior Member

    Joined:
    Nov 29, 2009
    Messages:
    117
    Likes Received:
    34
    Location:
    Germany
    well of course you could also try lighttpd which has good performance, too. i'm running all my sites on a small virtual server with only 512 MB ram and 10 GB storage!

    for all of you: dont just enable gzip compression use ETags also for static content. this will reduce traffic and will prevent clients from requesting the same static element twice. first page load is a bit slower but every page request after that is much faster.

    @portalweb:
    i think its a power of webservers to run external cgi scripts and binaries. you can use different users for every domain or customer. so you can improve security very much by separating them. if one host gets hacked all the others arent affected of that. runniny server modules like php cant prevent that since the whole webserver and all of its "clients" run under the same user. i'm doing webhosting now for about 6 years and had a lot of configs already, i know what i'm talking about :)
     
  8. voyevoda

    voyevoda Regular Member Premium Member

    Joined:
    Mar 21, 2010
    Messages:
    217
    Likes Received:
    97
    Location:
    Eastern Front
    I normally just use something like this with a cachebuster on the querystring (most frameworks append them automatically to assets):

    Code:
    if ($request_uri ~* "\.(ico|css|js|gif|jpe?g|png)") {
        expires max;
    }
    
    When you use etags, the client's browser still has to send a HTTP request for every asset and get back a 304. By setting an expiration date in the future, their browsers won't even send a request for those assets on subsequent page views.


    Yeah, that does seem kind of weird. I only use fcgi with PHP, so I can't speak about using plain old CGI with any particular languages. I hardly see CGI used nowadays. :eek:
     
  9. radi2k

    radi2k Junior Member

    Joined:
    Nov 29, 2009
    Messages:
    117
    Likes Received:
    34
    Location:
    Germany
    yep, your right. but a http 304 requires no I/O operations :) that are all operations that can save time that your visitors and even search engines will enjoy. and of course there wont be any traffic except the request and the 304 response (just some bytes).

    no one uses the old CGI or he/she is stupid ;) fastcgi with suexec or even chrooted environments are the future. a local hoster here is going one step further: own xen host per customer. thats very ram/cpu intensive but that brings a really good security environment with it :)
     
  10. portalweb

    portalweb Supreme Member Premium Member

    Joined:
    Jan 7, 2010
    Messages:
    1,415
    Likes Received:
    547
    Occupation:
    Hard Core Engineer
    Location:
    New York
    I disagree. I use compiled cgi scripts (written in C) as it performs much faster than php for specialized applications. That is the reason why I'm using lighttpd.
     
  11. NoSuchAgency

    NoSuchAgency Junior Member

    Joined:
    Oct 14, 2009
    Messages:
    124
    Likes Received:
    61
    I use Nginx primarily to serve static files and proxy pass to FCGI (php-cgi or django). Other useful stuff are the FCGI cache, rate limiting, and GeoIP.

    You can write your own C modules and include them at compile-time if you really care about performance. Otherwise you can always use embedded perl.

    Performance hasn't been an issue for me yet and you really won't feel any differences vs. Apache until you hit the order of hundreds to thousands of requests per second. But configuration is cleaner, and it's much less memory intensive than Apache.

    Nginx is stable, fast, lightweight, and crushes Apache for all my purposes.
     
  12. webcatpl

    webcatpl Junior Member

    Joined:
    Aug 12, 2008
    Messages:
    101
    Likes Received:
    9
    We use nginx in two configs:

    1. We use it for static content and fwd dynamic (php) to apache. It's pretty good, apache is php-cached so it's pretty fast.

    2. We use it for both static and dynamic content, which is fwd to fastcgi.

    nginx is a must! + cache of course.
     
  13. voyevoda

    voyevoda Regular Member Premium Member

    Joined:
    Mar 21, 2010
    Messages:
    217
    Likes Received:
    97
    Location:
    Eastern Front
    A little known feature of nginx is that it can talk directly to memcached. This means you can have certain requests bypass your entire application stack and never hit disk (aside from writing to the log, if any).

    This enables you to handle a huge amount of traffic on a single small server!
     
  14. portalweb

    portalweb Supreme Member Premium Member

    Joined:
    Jan 7, 2010
    Messages:
    1,415
    Likes Received:
    547
    Occupation:
    Hard Core Engineer
    Location:
    New York
    Hi,

    I'm Lighttpd user (with few apache servers for low-end traffic due to darn complex rewrite rules).

    I would love to switch to Ngnix web servers to replace all lighttpd web servers on my multi- dedicated servers. I tried to make the specialized "stand-alone" compiled scripts (written in C to generate the complex math-related images) as well as Perl to run under ngnix - and could not make it to work due to the sketchy documentation when dealing with fcgi proxy (http://wiki.nginx.org/NginxFcgiExample).

    Since you Nginx pros are around here, I'll install nginx web server on "development" server and then configure it..to see how it goes and post the results here.
     
  15. webcatpl

    webcatpl Junior Member

    Joined:
    Aug 12, 2008
    Messages:
    101
    Likes Received:
    9
    Why do you use perl? It's not most efficient solution.....


     
  16. portalweb

    portalweb Supreme Member Premium Member

    Joined:
    Jan 7, 2010
    Messages:
    1,415
    Likes Received:
    547
    Occupation:
    Hard Core Engineer
    Location:
    New York
    Perl? To make long story short, I run my own mailing servers to distribute hundreds of thousands email addresses weekly on my several servers (whitelist, fully opt-in email addresses, mind you, as I have been doing it for 10+ years).

    Perl is considered as the most effective method for running in background via cron job for automated email distribution.

    By the way, did you know that cPanel still use perl?
     
  17. NoSuchAgency

    NoSuchAgency Junior Member

    Joined:
    Oct 14, 2009
    Messages:
    124
    Likes Received:
    61
    By that logic, everyone should be programming in assembly or C. :D

    What's wrong with perl anyways? Text manipulation is one of it's best-known strengths, which is what portalweb is using it for.
     
  18. radi2k

    radi2k Junior Member

    Joined:
    Nov 29, 2009
    Messages:
    117
    Likes Received:
    34
    Location:
    Germany
    @portalweb: which compiler do you use to transfer your scripts into c and then binary? is it that one, that facebook uses? they made their tools public some time ago.

    is someone running some sites using java and/or tomcat? would be interesting to me since i havent done this yet.
     
  19. jazzc

    jazzc Moderator Staff Member Moderator Jr. VIP

    Joined:
    Jan 27, 2009
    Messages:
    2,468
    Likes Received:
    10,147
    This is a great thread! I am tempted to go out and try nginx.

    What troubles me though, is tha I use Plesk. I see an article here http://cnedelcu.blogspot.com/2009/09/setting-up-nginx-as-reverse-proxy-on.html about nginx and Plesk9 configuration, but

    this has the disdvantange it might break on a future Plesk update.

    So, the real question is, is the some tool like Plesk that works with nginx natively?

    Also, do you prefer running nginx as a reverse proxy, or to use it as a full server on its own (for php)?
     
    Last edited: Apr 17, 2010
  20. NoSuchAgency

    NoSuchAgency Junior Member

    Joined:
    Oct 14, 2009
    Messages:
    124
    Likes Received:
    61
    You can't use nginx as an all-in-one-package. It has no embeded CGI support, let alone PHP interpreter (which Apache bundles). Nginx would have to proxy to something else.

    For serving PHP, you'd need to run php-cgi and then use nginx to proxy pass to it. Or alternatively proxy pass to Apache serving PHP.

    If you set up Plesk as described in the article, there's no problem at all, as Apache handles all the PHP and nginx just acts as a reverse proxy.
     
    • Thanks Thanks x 1