mbhmirc
Regular Member
- Feb 6, 2011
- 208
- 54
Hi All,
We know google is putting some importance on how fast your site loads, however most people still don't know that minor speed increases make percentage point conversions in sales! I'm going to do a bit of a brain dump here from a large social network site I used to run. Some of this will be VPS/Dedicated only and some will be usable on shared hosting. If I get time I'll split it out but for now this is just a brain dump.
N.B. Make backups/Have your host on the ready before making any of the below changes, i take no responsibility for what you break by following this advice.
First off NGINX! Most people run apache/nginx/whatever httpd server. Now I like to do a particular setup:
Small Blog sites/Sites in general
Nginx In reverse proxy -> Apache -> Mysql
If it's a heavy image site or larger site
Nginx ------> images.domain.com -> varnish -> Filesystem (will explain this more in a second)
|---->www.domain.com -> Apache -> Mysql
Yes thats one nginx reverse proxying both varnish and apache on different domains. Two reasons (you can use this on small sites to!) first off is browsers can do multiple connections at once. If it's on the same domain some are capped to as little as four connections! If you have lots of images i'd suggest breaking it up into images1.domain.com images2.domain.com css.domain.com js.domain.com where css and js are css and javascript respectivly. What this does is allow the browser to make four times as many requests, however this is for sites with lots of content due to the fact dns lookups also have overhead. To find out what works best for you is to do my fav thing and TEST TEST TEST! Best site i've found for this is http://www.webpagetest.org/ You only care about the first pass! That is what a fresh visitor visiting your site for the very first time will wait. You will have to learn a bit of dns and vhost magic to split these out. Tripple bonus if you get big enough you can use a CDN as you?ve ready split your business logic and static files
https://help.ubuntu.com/community/Nginx/ReverseProxy
https://www.varnish-cache.org/ (you could just use varnish but nginx is more secure imho)
Secondly, why have i put nginx then service then database you ask? Well this is a fun one nginx "drip feeds" the slower connection on the browser end, meaning apache can shove data to it at full speed. This in turn means less database connections as their completed faster and you have a server that can send more traffic out to the masses!
Why varnish? It stores the images in memory and serves regular ones really fast, again freeing up the file system to do file system type stuff
Offtopic, put your mysql on raid 10 with a raid card with battery backed up cache and at least six disks of sas 15k rpm. If you have lots of money SSD?s
Opcache for php, i love eaccelerator but it?s hard to get hold of now so use a supported one like apc. Whats this do? Pre-compiles your php code into computer code so that the code runs faster. If you wana burn facebook has a project to actually run it?s own custom php server but this is super advanced
Ok now for some nice cut and paste stuff:
Obviously not the bit called code
This basically turns on compression on the server and also tells it to keep these types of files for 7 days on the clients browser. Usually this should be 14 days but you might change your code a lot in the early days. If you have full httpd access i suggest you lower the compression as to not to burn so much cpu, usually i find 5-6 to be good ratios. I?ll not post it here as people not knowing what they are doing will get a nasty 500 error and blame me
If your lazy google pagespeed! It does all the css/js combining and minify and make your site pretty for browsers for you. You can find out more about it here https://developers.google.com/speed/pagespeed/mod
Lastly most people arn?t dba?s (i.e. database admins) so i suggest you use https://launchpad.net/mysql-tuning-primer
Last but not least cloudflare do a free CDN, i don?t know how good it is and i know you loose some control, however if you can?t afford a vps/good hardware right now it might be a stop gap https://www.cloudflare.com/plans
Reduce disk usage by mounting your disks with no atime (access time) most apps don?t need to know the last access time you can find out more here:
http://www.howtoforge.com/reducing-disk-io-by-mounting-partitions-with-noatime
Be careful with this as it could hose your vps/server if you don?t know what you're doing.
If your using wordpress use wp-supercache it'll take the load off the database and shove it to the filesystem. Again test with and without on your host as some invest in good mysql and some invest in good file systems. You can't just apply cache's to every host.
Check if your app can use memcache. It stores items in memory and serves them really fast, thanks livejournal! (their is a wordpress one!)
Keep your blog spamfree, akismet is a pay for service but saves precious cpu and emails! However their is a free alternative! http://antispam.typepad.com/info/get-started.html (if your smart you can even setup your own akismet like service and charge )
I think there's enough here to get people started. I?ve worked on projects from small blogs to a social network site with several million visitors a day (used to own) and just recently on a global tracking system for a electric truck company which uses mysql as the backend. This does millions of inserts per second. Sorry to have ranted on but hope theirs some good info mixed in this lot
We know google is putting some importance on how fast your site loads, however most people still don't know that minor speed increases make percentage point conversions in sales! I'm going to do a bit of a brain dump here from a large social network site I used to run. Some of this will be VPS/Dedicated only and some will be usable on shared hosting. If I get time I'll split it out but for now this is just a brain dump.
N.B. Make backups/Have your host on the ready before making any of the below changes, i take no responsibility for what you break by following this advice.
First off NGINX! Most people run apache/nginx/whatever httpd server. Now I like to do a particular setup:
Small Blog sites/Sites in general
Nginx In reverse proxy -> Apache -> Mysql
If it's a heavy image site or larger site
Nginx ------> images.domain.com -> varnish -> Filesystem (will explain this more in a second)
|---->www.domain.com -> Apache -> Mysql
Yes thats one nginx reverse proxying both varnish and apache on different domains. Two reasons (you can use this on small sites to!) first off is browsers can do multiple connections at once. If it's on the same domain some are capped to as little as four connections! If you have lots of images i'd suggest breaking it up into images1.domain.com images2.domain.com css.domain.com js.domain.com where css and js are css and javascript respectivly. What this does is allow the browser to make four times as many requests, however this is for sites with lots of content due to the fact dns lookups also have overhead. To find out what works best for you is to do my fav thing and TEST TEST TEST! Best site i've found for this is http://www.webpagetest.org/ You only care about the first pass! That is what a fresh visitor visiting your site for the very first time will wait. You will have to learn a bit of dns and vhost magic to split these out. Tripple bonus if you get big enough you can use a CDN as you?ve ready split your business logic and static files
https://help.ubuntu.com/community/Nginx/ReverseProxy
https://www.varnish-cache.org/ (you could just use varnish but nginx is more secure imho)
Secondly, why have i put nginx then service then database you ask? Well this is a fun one nginx "drip feeds" the slower connection on the browser end, meaning apache can shove data to it at full speed. This in turn means less database connections as their completed faster and you have a server that can send more traffic out to the masses!
Why varnish? It stores the images in memory and serves regular ones really fast, again freeing up the file system to do file system type stuff
Offtopic, put your mysql on raid 10 with a raid card with battery backed up cache and at least six disks of sas 15k rpm. If you have lots of money SSD?s
Opcache for php, i love eaccelerator but it?s hard to get hold of now so use a supported one like apc. Whats this do? Pre-compiles your php code into computer code so that the code runs faster. If you wana burn facebook has a project to actually run it?s own custom php server but this is super advanced
Ok now for some nice cut and paste stuff:
Obviously not the bit called code
Code:
<IfModule mod_deflate.c>
#activates gzip for core file types
AddOutputFilterByType DEFLATE text/css text/javascript application/x-javascript application/javascript text/x-component text/html text/richtext image/svg+xml text/plain text/xsd text/xsl text/xml image/x-icon application/json
</IfModule>
<IfModule mod_expires.c>
# Activate mod_expires for this directory
ExpiresActive on
# locally cache common image types for 7 days
ExpiresByType image/jpg "access plus 7 days"
ExpiresByType image/jpeg "access plus 7 days"
ExpiresByType image/gif "access plus 7 days"
ExpiresByType image/png "access plus 7 days"
# cache CSS files for 24 hours - disable with a # while making changes or control f5 to force refresh
ExpiresByType text/css "access plus 24 hours"
</IfModule>
# enf optimise stuff blockThis basically turns on compression on the server and also tells it to keep these types of files for 7 days on the clients browser. Usually this should be 14 days but you might change your code a lot in the early days. If you have full httpd access i suggest you lower the compression as to not to burn so much cpu, usually i find 5-6 to be good ratios. I?ll not post it here as people not knowing what they are doing will get a nasty 500 error and blame me
If your lazy google pagespeed! It does all the css/js combining and minify and make your site pretty for browsers for you. You can find out more about it here https://developers.google.com/speed/pagespeed/mod
Lastly most people arn?t dba?s (i.e. database admins) so i suggest you use https://launchpad.net/mysql-tuning-primer
Last but not least cloudflare do a free CDN, i don?t know how good it is and i know you loose some control, however if you can?t afford a vps/good hardware right now it might be a stop gap https://www.cloudflare.com/plans
Reduce disk usage by mounting your disks with no atime (access time) most apps don?t need to know the last access time you can find out more here:
http://www.howtoforge.com/reducing-disk-io-by-mounting-partitions-with-noatime
Be careful with this as it could hose your vps/server if you don?t know what you're doing.
If your using wordpress use wp-supercache it'll take the load off the database and shove it to the filesystem. Again test with and without on your host as some invest in good mysql and some invest in good file systems. You can't just apply cache's to every host.
Check if your app can use memcache. It stores items in memory and serves them really fast, thanks livejournal! (their is a wordpress one!)
Keep your blog spamfree, akismet is a pay for service but saves precious cpu and emails! However their is a free alternative! http://antispam.typepad.com/info/get-started.html (if your smart you can even setup your own akismet like service and charge )
I think there's enough here to get people started. I?ve worked on projects from small blogs to a social network site with several million visitors a day (used to own) and just recently on a global tracking system for a electric truck company which uses mysql as the backend. This does millions of inserts per second. Sorry to have ranted on but hope theirs some good info mixed in this lot
Last edited:
