Whats the biggest wordpress site you've ever successfully ran?

[HenryHavoc]

I'm currently working on a pretty massive project that will be creating wordpress posts via CSV files with my own data and I will probably end up with over 500,000 posts. Possibly a million. I know wordpress has its limitations with how large a site you can build (mainly because every time a post is viewed, it needs to query a database, and the more posts you have, the larger that database becomes).

Obviously shared hosts are out of the question. I was recommended to use Kinsta. Anyone have other recommendations?

So whats the biggest site you've built? I'd love to get some input on how you go about managing a site this big (server speed optimization, getting the site indexed, SEO, etc)

 

[roadhamster]

The pain in the ass with wordpress, and more CMS'es/ applications using databases, the more database entries (posts), the slower your site will become. Depending on your hardware sooner or later your website will become unresponsive. But for every problem there's a solution. When it's gonna be huge, get yourself a neat LinuxVPS, install Sphinxsearch and let Sphinx get the stress from the database. Wordpress has a Sphinx plugin btw, https://wordpress.org/plugins/wordpress-sphinx-plugin/ and, if configured right, the power of this tool is amazing. Drop on top of that Redis server (http://redis.io) with the Redis wordpress plugin for WP https://wordpress.org/plugins/search/redis/ and you're good to go to the next 500,000 posts and your users won't notice a difference in performance.
SphinxSearch is doing much much better than the wordpress standard search, so SEO point of view this is a good startingpoint.

I'm using Sphinx/ Redis on a website (non-wordpress) with a database with over 20 million records 6.5GB in size with full search capability and it's lightning fast. Sphinxsearch: http://sphinxsearch.com.

 

[HenryHavoc]

The pain in the ass with wordpress, and more CMS'es/ applications using databases, the more database entries (posts), the slower your site will become. Depending on your hardware sooner or later your website will become unresponsive. But for every problem there's a solution. When it's gonna be huge, get yourself a neat LinuxVPS, install Sphinxsearch and let Sphinx get the stress from the database. Wordpress has a Sphinx plugin btw, https://wordpress.org/plugins/wordpress-sphinx-plugin/ and, if configured right, the power of this tool is amazing. Drop on top of that Redis server (http://redis.io) with the Redis wordpress plugin for WP https://wordpress.org/plugins/search/redis/ and you're good to go to the next 500,000 posts and your users won't notice a difference in performance.
SphinxSearch is doing much much better than the wordpress standard search, so SEO point of view this is a good startingpoint.

I'm using Sphinx/ Redis on a website (non-wordpress) with a database with over 20 million records 6.5GB in size with full search capability and it's lightning fast. Sphinxsearch: http://sphinxsearch.com.

You are a god amongst men. What would you say is the most user friendly way to set up this combo? I talked with a developer from Kinsta which runs on google cloud and they said as long as I had caching enabled I would be able to host the site with them. I'm pretty familiar with WordPress and front end stuff but back end server and database stuff is still relatively new to me. It seems like you have some experience with stuff so I'd love to hear your input on getting this up and running.

 

[rafark]

The database is not the problem, in this case is wordpress that is terribly optimized.

Assuming you're not on a tight budget you should get a medium vps with enough ram and then run an in memory cache storage tool such as redis. You should also use op cache provided you have full control of th server. (VPS or dedicated).

Also, are you importing the posts? Are a programmer? If so, don't read all data into memory at once when importing, stream it line by line.

 

[HenryHavoc]

The database is not the problem, in this case is wordpress that is terribly optimized.

Assuming you're not on a tight buffer you should get a medium vps with enough ram and then run an in memory cache storage tool such as redis. You solhoiod also use op cache provide you have full control of th server. (VPS or dedicated).

Also, are you importing the posts? Are a programmer? If so, don't read all data into memory at once when importing, stream it line by line.

Are there any hosting solutions that offer this type of setup right out of the box?

I'm importing the posts from csv files via WP All Import. I'll probably end up breaking it all up with cron jobs so it's not running all at once.

 

[roadhamster]

I checked Kinsta, looks nice, but for the 100$ a month you can get yourself a very decent VPS/ dedicated machine and you have full control over your setup. Put somehting like webmin on it to control your websites/ webserver setup.
Importing posts with wp-allimport, I tried it but it failed with csv files with lots of entries. For this I'm using Mysql cmmand line, I can import a huge csv files in no time straight into the database.
What I'm trying to say: every choice you make has its pro's & cons. A fully self-managed VPS or dedicated can be a lot of work, but when it's running, you have full control over all the aspects of your server. When you go for a managed option, you depend on the hosting company and may hope that they will be cooperative in your wishes, it's up to you.