Let's Make An AI Content Generator Based On GPT-2 (The OpenAI Model)

[kboxer7]

I have been doing some experimenting with the algorithm. I have noted something interesting. I am able to output a list of probable text continuations, giving each a probability score of being the best one. Within that list, one of those sentences is almost always the perfect sentence to continue with - like really frackin good. The problem is that often, the best continuation isn't rated with the highest probability. I could choose manually but that would sort of defeat the purpose. It would allow me to write an article in like a minute that would normally take an hour but I don't want any input to be necessary. More to come.

If you have something that even takes 15 minutes of human intervention to write I'd be interested in contributing financially/licensing or purchasing it from you. I know its open source (the model) but I am not technologically inclined and it would be a fool's errand for me to learn yet another thing with my plate so full.

DM me if you'd like to chat.

 

[WarLords]

Thats an awesome project will keep my eyes here as well Good luck and lets see how it goes

 

[vin42]

I have been doing some experimenting with the algorithm. I have noted something interesting. I am able to output a list of probable text continuations, giving each a probability score of being the best one. Within that list, one of those sentences is almost always the perfect sentence to continue with - like really frackin good. The problem is that often, the best continuation isn't rated with the highest probability. I could choose manually but that would sort of defeat the purpose. It would allow me to write an article in like a minute that would normally take an hour but I don't want any input to be necessary. More to come.

Something interesting is brewing here.....I would want it even if I have to select the best sentence when writing articles...it would be hundred times better than writing the article manually which is tedious and boring since I am forced to do it.

 

[vin42]

I think a spinner will be easier to create using gpt2......any inputs?

 

[jadezoole]

I think a spinner will be easier to create using gpt2......any inputs?

I agree, first a reliable rewriter application would be good.

 

[ragnarkar]

https://talktotransformer.com/

This demo site returns complete garbage

Try this one instead.. esp if you know some Python: https://medium.com/@ageitgey/deepfaking-the-news-with-nlp-and-transformer-models-5e057ebd697d

 

[vin42]

OpenAI finally releases “dangerous” large-scale unsupervised language model GPT-2
https://medium.com/syncedreview/openai-releases-1-5-billion-parameter-gpt-2-model-c34e97da56c0

 

[Deleted member 657006]

the problem is this open project being also scanned via google to recognise the ai work.

so future posting will just get de indexed.

plus proof article writing will never be achieved with ai bot.

a ai spinner would be perfect because the proof written articles spun would work if the ai spinner becomes available .

 

[Harlet]

The good samples from gpt2 exhibit a strong selection bias. While it is fun to play with, you might get something truly useful 1/10 tries. Not really great for automated posting. However detecting GPT and other similar tools just isnt as simple as being 'scanned by Google', either. Even the tool that GPT2 made to limit it's own nefarious use is not up to the task of reliably detecting GPT2 and neither is Google.

I have doubts that you could even make a reliable manual spinner using it as it often will break down in its own thought loops and go completely off subject no matter how well trained it is.

Also the huge model cannot be trained and the reality is that outside of a few circumstances, a trained 355m model is going to be better than the massive 1.5b parameter model. The real reason they released it is simply to stay relevant as there are better options and stronger competitors now.

 

[theRevolt]

the problem is this open project being also scanned via google to recognise the ai work.

so future posting will just get de indexed.

plus proof article writing will never be achieved with ai bot.

a ai spinner would be perfect because the proof written articles spun would work if the ai spinner becomes available .

Nope, not really. See, there is gibberish ranking in lots of SERPs and the examples you get here are from an untrained model, so you could train the small model on a certain niche and you would get a better output.
Obviously, nobody will release a niche-specific highly trained model...

You may want to look up earnings reports, stock reporting etc... most of that is written by AI now and it ranks. That is obviously soft you're average spam blogger does not have access to.

 

[Harlet]

Nope, not really. See, there is gibberish ranking in lots of SERPs and the examples you get here are from an untrained model, so you could train the small model on a certain niche and you would get a better output.
Obviously, nobody will release a niche-specific highly trained model...

You may want to look up earnings reports, stock reporting etc... most of that is written by AI now and it ranks. That is obviously soft you're average spam blogger does not have access to.

Although you are right that many earnings reports and such are now written by AI, those are not really comparable in the long run as they are much more predictable and data driven than average written content. It is easier to create readable in trees and still they do require human supervision even if it's limited. My company uses this period

However, you are right. Smaller models that are better trained and better yet different models like transformer xl can create content that is rankable. Although it is not cut particularly difficult to train these models, the barrier to entry is having the baong the ability to produce clean data sets to do so which frankly must members do not have

 

[theRevolt]

Although you are right that many earnings reports and such are now written by AI, those are not really comparable in the long run as they are much more predictable and data driven than average written content. It is easier to create readable in trees and still they do require human supervision even if it's limited. My company uses this period

However, you are right. Smaller models that are better trained and better yet different models like transformer xl can create content that is rankable. Although it is not cut particularly difficult to train these models, the barrier to entry is having the baong the ability to produce clean data sets to do so which frankly must members do not have

Yes, they follow an Almost static framework with just a few minor variables changing,but still a good example that as long as there is some perceived value it can/will rank.
You're right, the average spammer just does not have the knowledge to do this , which is why we do not see it more and the ones that do of course do not just share it freely as you can easily sell that instead of say automated spun content ,or with great clean source data and proper training actually as solid content that would beat the very cheap writers.

All in all just very interesting and let's see what else is to come here

 

[0verflow]

Nice find, im actually working on a content analyser and content generator based around gpt

Kind of going along the rankings angle - good content ranks -> measure good content -> write good content

ill be releasing that tool in q1 next year - its really fun to play with

 

[LeadCloak]

Anymore good updates, mate? Keeping eye on this thread from start.

 

[JayDee1984]

There are ways to shift the module a bit a side from what it’s supposed to produce and make it SEO friendly along side with data inputs related to a given topic.

The training is expensive though, I heard someone say about $50k...

 

[Purush]

Anyone successfully used it?

 

[Harlet]

There are ways to shift the module a bit a side from what it’s supposed to produce and make it SEO friendly along side with data inputs related to a given topic.

The training is expensive though, I heard someone say about $50k...

Training a full paran model is like the 1.5b parameters because it requires so much processing power. But a properly trained 350m model can outperform the best broadly trained model.

The model training is expensive and not productive because it is more scientific in nature than practical. The technology isnt that good yet. The only money that would be good spent on this would be improving the generation algorithm and doing new and unique things with it which is expensive as machine learning is not something you outsource from a pennies an hour country and get good results

 

[JayDee1984]

Training a full paran model is like the 1.5b parameters because it requires so much processing power. But a properly trained 350m model can outperform the best broadly trained model.

The model training is expensive and not productive because it is more scientific in nature than practical. The technology isnt that good yet. The only money that would be good spent on this would be improving the generation algorithm and doing new and unique things with it which is expensive as machine learning is not something you outsource from a pennies an hour country and get good results

Thank you for your insights

 

[cpafreak]

Looks pretty advanced if we compare the quality of spin rewriters. Is there any .exe file or direct link to test it.
I found licence and .md file but can anybody tell on how to try this stuff?

 

[KevinRB]

GPT-2 (Recently discussed here) is currently the state-of-the-art machine learning model for text generation agents. The previous was probably BERT. OpenAI decided to not release their fully-trained model. Instead, they released a retarded (Literally) version. In order to train a model comparable to the one they didn't release would cost a lot of money - or would it? I may have a way to do it for free if I can jerry-rig Google Colab.

This thread is for the discussion of how to approximate, to the best of my/our ability, a kick-ass GPT-2.

I have setup a Github to accompany this thread here: https://github.com/TheDoctorAI/aiwriter

Contributions welcomed.

What amazes me, is that to date there has not been a SINGLE commercial script/software/plugin or subscription based service built around the GPT-2 releases....there is a hungry/insatiable market for content!.