There is an enormous list of ranking factors, most of which are unknown; so can't tell you exactly what has happened. But let's start with the basics:
Firstly, not all channels can rank. If your channel is new, you haven't upload videos before, etc. you may have trouble ranking at all. It's a good idea to upload a test video targeting a very low comp keyword, optimize metadata as per standard methods and see if you are ranking. Essentially just warm up the channel first. Don't put effort and resources into a video if you haven't ranked any videos on the channel before as your efforts can be for nothing as you have found out. You may rank eventually now but you should build some authority so you can rank faster in the future.
Assuming your videos are indexing properly in search; when you upload a video you initially rank high due to freshness, boosting views and other metrics in the right way can increase your rank further. Once you are ranking well, if your video doesn't perform with the organic traffic it's getting from real keyword searches your ranking will fall.
For example, let's say your competitors have a better title and thumbnail then you which is resulting in more people who search the keyword clicking on their video over yours. Their higher click through rate (CTR) may result in a higher ranking.
Other user engagements work in a similar way. Let's say users who search your target keyword and click on your video are only watching your video for 60 seconds and the video that ranks above you is getting 90 second watch time. This lower expected watch time per view is a signal to YouTube's search ranking algorithm that your video isn't as relevant to the keyword, as high quality, etc. meaning you may not be able to outrank your competitor.
The info on the different ways you can optimize for search is out there, just gotta do the research
Boosting metrics in the right way and optimizing metadata can expose you to a wider organic audience through increased rankings in search and recommendations, however if your video doesn't perform well relative to your competition you aren't going to stick those rankings for very long.
I hope this gets you thinking in the right direction. Good luck!
