GPU clustering for Machine Learning?

Is there any actual guide on how to put together a GPU cluster for Machine Learning?

K80's, while not the most powerful now, are pretty cheap and seem like a good way to put together a GPU cluster in a single 42U for a decent price, at $150-200 a piece now.

I don't know of a guide that takes you from 0 to 100 but here are some hints. First of all have a look at how you would size 1 node, with the necessary storage and memory requirements for your use case (text, images, videos or other). Once you do that build two of them and connect them with a good network connection, best would be PCI/Infiniband but that can get expensive. I would then install something like Kubernetes on top of them and do some tests with regards to throughput and other metrics that would be important. Achieving parallel training on multiple GPU's will require you use specific ML libraries as some don't support it and have a look how you want to package your training pipelines.
I did this several times and have my own little home setup.
This can be done relatively quickly, although I recommend you test it first on 2 raspberry pis to check out the deployment part. Scaling vertically then will be a matter of figuring out some details. Good luck

check https://vast.ai/

On the AMD side, it has very little software support for their GPUs. On the hardware side, Nvidia has introduced dedicated tensor cores.