Recent content by firetodust

Yes if you change the optimizer to sgd. Doing so really slows down training so change your learning rate to 0.001

Glad I found this thread. I've been working on a gpt2-based model for almost two months now. I'm using it to generate content for a niche site. It took a lot of tinkering to get the thing to generate relevant content for 1 out of 10 generated articles (anywhere between 300-1000 words) and I...