I was really into it in my teenage years. At some point I was able to do it every night (closer to the morning after my deep sleep) and I've had experiences that felt like hours. I've done pretty much anything I could think of, like eating food (yes, you can taste food in your dream, which is remarkable) to gradually learning skills that defy laws, like flying, and setting towns of fire. Met interesting people, even went on a quest to find the wisest person. Found him in a cave but he didn't want to speak to me. Probably because I set those towns on fire, haha.
You feel like everything in your dream is part of you (clearly), but you don't feel like you're consciously creating anything of it. You can of course introduce things to "your world" but this is not required. Everything is already ready for exploring. Some places left me speechless, and it really makes you question the creative potential of your mind. You can also find yourself in uncomfortable places, which happens more frequently with people that have nightmares. This is a good moment to tell yourself that your the creator and that you're powerful enough to make any change. Unlike the real world, the dream world adapts to the ways that you confidently believe in.
And no, never took any substances. A lot of people will market these pills of african root herbs. Never tried them and wouldn't suggest you do. It's more fun to gradually learn it yourself.
It goes away if you're not conscious about wanting to do it. I'll admit that I rarely lucid dream now and have little desire to since it takes away from a peaceful sleep.
I've only implemented two techniques that made me lucid dream: when you go to bed, instead of curling up and passing out while letting your brain run, lie on your back, clear your head, and just relax. Just be aware and grow your consciousness. Don't move (this is often very hard for people to do but you gotta learn it), and your body will eventually "fall asleep" while you stay wakeful.
Have patience and do this whenever you can. Do this for 30 minutes/1 hour and then go to sleep if you haven't already. When you wake up in the middle of the night, push yourself to repeat the process. It will be much easier to reach this state. The other technique is an addition to this one, which is to take a cold shower at 3AM and then repeating this method.
I've had moments where I remained wakeful while falling asleep, and the process is basically that while your brain starts creating a scenery of your dream, your body slowly fades to a point where you don't feel it at all. Then the scenery strengthens and your brain simulates your body in that scenery, and boom, you're lucid dreaming.
If you haven't already, I suggest you read up on REM sleep, or better yet, read Exploring The World of Lucid Dreaming by Dr Stephen LaBerge. Although I've read it after I had already learned it, it's a highly dense text that covers pretty much everything you need.
REM will basically explain why you are unlikely to lucid dream in the first 4-6 hours of your sleep.
Once you start doing it, all you're going to focus on is prolonging your experience, and a method that worked for me was rubbing my palms in my dream. No matter what happens, stay calm or it'll abrupt your experience.