Many concentrate on the average retention rate percentage however, I have realized that the actual retention curve is more valuable. The exact moment when users stop watching tells more than just CTR, specially in case this is happenning prior to receiving all the promised information from the video. Correction of these faults worked better for me than constant optimisation of keywords.
One thing which had a major impact on my videos is that I did not limit myself to the video itself, rather paid attention to what happened after the video. This way, a video could have a good retention rate but not perform well because it was not leading to other videos or full filling the original purpose. I created better chains of topics among my uploads.
Another factor that made more sense for me was distinguishing between the videos that attracted viewers and the videos that caused viewers to return to the channel at a later point. There could be a video that would have gud statistics on its own, yet contribute little to the channel because viewers simply disappear after watching it. I begins to care about the topics that made people watch something else.