This is too much...

Joined
Mar 11, 2013
Messages
5,542
Reaction score
9,769
1656930330804.png

This is too much. One freaking click. I wish I truly understood machine learning. My goal for next 10 years is to learn this s*it and have enough money to make use of it.

It doesn't make sense. One paragraph is about stones, another about PPC. hahaha

Where do I start learning this? From what I understand, I need to know how to analyze all that text and polish it somehow. Hard af.
 
Wait... What?
Sir, I am kindly asking for expertise. This is my novice trick.

I show content that won't work for any strategy.

I just want to take people at their mercy to guide me on the way and tell me step-by-step what I need to generate text that complies with all standards.

In other words, as a guy generating 10,000 words with one click, I want to make my articles cohesive now.

The mess you see in the attached image is due to the fact that it was created from a scraped article from the Internet.

Subsequently, individual paragraphs from that scraped article were the inspiration for GPT-3. Unfortunately, the script can't separate the information properly. It is too advanced.

So maybe a question. How should I create an outline builder that GPT-3 will understand in the form of prompts?

What technology will allow me to do that? I care about 90% accuracy. 10% I can outsource to an editor.

In general, what I give GPT-3 is trash. But I know I can give something better. I have tried many techniques. It doesn't work out!

The article must contain factual information that does not repeat itself even once throughout the article. It must be some kind of advanced technology.

On top of that, it needs to speak in one tone and not change from "they" to "you" every now and then.

There is a post on the forum where someone listed all those problems.
 
Last edited:
. I wish I truly understood machine learning.
Back in the day we used to have these things called Markov text generators. This looks like a grammatically correct version of the output those text generators used to produce.

So maybe a question. How should I create an outline builder that GPT-3 will understand in the form of prompts?
You train it. At its core it's as simple as passing a jsonl file like this:
JSON:
{
"prompt": "<outline body> \n\n\n",
"completion": "<text for the outline> END"
}
...
with the prompts being your outline and the completion being the text for that outline.

One way you would do this is; instead of long outlines with multiple instances, give it an H2 as the prompt and the accompanying paragraph after the H2 as the completion.

You can feed in several thousands of examples for fine-tuning. Since, as of now you can train gpt-3 with a dataset of about 80-100mb or roughly 2.5 million tokens.


What technology will allow me to do that? I care about 90% accuracy.
A single model, however finetuned, cannot achieve figures like that.

A model will always start producing irrelevant content when you ask it to produce more than a thousand tokens in one go. A Transformer models isn't an LSTM, it doesn't have memory. It burst outs the words is the most relevant to the previous words and that derails further Generation.

The way, long form generators do it is either by multi-model pipelines (one model takes your outline and generates h2s/small paragraphs, another model takes the H2/small paragraphs and expands them and so on).

Or by breaking generation with multiple prompt/generation phases (you input a small paragraph, model generates some sentences, that becomes the prompt again and model makes more sentences)
 
Or by breaking generation with multiple prompt/generation phases (you input a small paragraph, model generates some sentences, that becomes the prompt again and model makes more sentences)
That's what I've been doing thus far. However, I want to create that 90% automated workflow.

You can feed in several thousands of examples for fine-tuning. Since, as of now you can train gpt-3 with a dataset of about 80-100mb or roughly 2.5 million tokens.
How much money is that?

Where do you learn this stuff? I am looking for a go-to resource where I will learn how to gather data for datasets and I want to learn how models work.
 
Last edited:
That's what I've been doing thus far. However, I want to create that 90% automated workflow.


How much money is that?
It costs $0.03/1k tokens for fine-tuning. so 2500*0.3 or roughly 75$. Round up to around 100
 
It costs $0.03/1k tokens for fine-tuning. so 2500*0.3 or roughly 75$. Round up to around 100
Nice. I thought you will say $5k but this sounds like a good deal. I didn't even check it myself because I was afraid of what I would see.
 
Which tool is this if I may ask?
All GPT-3 based models give relevant output
 
Which tool is this if I may ask?
All GPT-3 based models give relevant output
I extracted keywords and sentences from scraped paragraphs and sent them as prompts to OpenAI like below.

Write a paragraph about [3 - 4 sentences from scraped paragraph] Keywords: [keywords extracted from scraped paragraph]

And the output is terrible. I wanted to avoid training but I only got pain back.

This type of prompt worked well for something like a product that AI doesn't know about. In this case it was a total failure.
 
Last edited:
Where do you learn this stuff? I am looking for a go-to resource
There isn't really a "single go-to resource". Ehatqt you have put there on the internet instead is several hundreds of thousands of resources for small chunks of information spread all over the internet.

Get good at googling. It's indeed a skill. Google even has a whole course about it
https://coursebuilder.withgoogle.com/sample/course

I want to learn how models work.
There are 2 approaches. One is the "math-heavy" approach, another is the "code-heavy" approach.

The math approach involves learning all the math first. You need to learn Linear Algebra, Probability and Statistics and then you can work out how deep learning models work out in theory. You then build models with code and frameworks.

The code first approach involves building a basic neural network from scratch. You learn all the math required to do that along the way.

Once you are comfortable with basic perceptron neural networks, you can start looking into multilayer architectures and more complex neural networks like CNNs and RNNs. You won't need them to build them from the ground up, once you understand how it all works you will learn to use a framework like PyTorch.
 
There isn't really a "single go-to resource". Ehatqt you have put there on the internet instead is several hundreds of thousands of resources for small chunks of information spread all over the internet.

Get good at googling. It's indeed a skill. Google even has a whole course about it
https://coursebuilder.withgoogle.com/sample/course


There are 2 approaches. One is the "math-heavy" approach, another is the "code-heavy" approach.

The math approach involves learning all the math first. You need to learn Linear Algebra, Probability and Statistics and then you can work out how deep learning models work out in theory. You then build models with code and frameworks.

The code first approach involves building a basic neural network from scratch. You learn all the math required to do that along the way.

Once you are comfortable with basic perceptron neural networks, you can start looking into multilayer architectures and more complex neural networks like CNNs and RNNs. You won't need them to build them from the ground up, once you understand how it all works you will learn to use a framework like PyTorch.
Great! I don't have much knowledge of any of these areas, but this will be what I can learn for years to come.

Mathematics is a great exercise for the mind. I sleep better when I study it. Although I stopped because I was busy.

I expected this kind of response. I used to do research before, but I pushed it out of my head a bit.

This fact of learning math overwhelmed me a bit when I was learning coding intensively, but everything will go well if I spread it out over time.
 
View attachment 216553

This is too much. One freaking click. I wish I truly understood machine learning. My goal for next 10 years is to learn this s*it and have enough money to make use of it.

It doesn't make sense. One paragraph is about stones, another about PPC. hahaha

Where do I start learning this? From what I understand, I need to know how to analyze all that text and polish it somehow. Hard af.
That's awesome your progress motivates me
For the last 3 months i have been learning python for this purpose and your thread simply showed me I still had a long way to go.
Keep you mate you are going to make it and btw, @reaaski you changed your profile picture?
 
That's awesome your progress motivates me
For the last 3 months i have been learning python for this purpose and your thread simply showed me I still had a long way to go.
Keep you mate you are going to make it and btw, @reaaski you changed your profile picture?
Yes, I changed my profile picture. This is just another businessman picture I found on the internet. This one is a little bit more elegant, but frankly I change it based on my mood.

1656946731348.png


I have a tool like this that lets me quickly do this stuff. It's kind of manual, but I can get 15 unique articles in 5 minutes, so much faster than offers on the internet.

I think it's something interesting in terms of UX but it lacks in terms of basics like my own training of models. Then coupling models together.

I have an idea to create more automated scenarios with it. Though it won't go far without nailing those basics and having money for investment in training.
 
Great! I don't have much knowledge of any of these areas, but this will be what I can learn for years to come.

Mathematics is a great exercise for the mind. I sleep better when I study it. Although I stopped because I was busy.

I expected this kind of response. I used to do research before, but I pushed it out of my head a bit.

This fact of learning math overwhelmed me a bit when I was learning coding intensively, but everything will go well if I spread it out over time.
Here are some resources to help you

Linear Algebra from ocw:
https://ocw.mit.edu/courses/18-06sc-linear-algebra-fall-2011/
Probability:
https://m.youtube.com/playlist?list=PL2SOU6wwxB0uwwH80KTQ6ht66KWxbzTIo
Statistics:
https://m.youtube.com/playlist?list=PL432AB57AF9F43D4F
Matrix methods for ml (teaches how deep learning works):
https://ocw.mit.edu/courses/18-065-...-processing-and-machine-learning-spring-2018/
Neural Networks from Scratch (book) -
https://nnfs.io/
 
Yes, I changed my profile picture. This is just another businessman picture I found on the internet. This one is a little bit more elegant, but frankly I change it based on my mood.

View attachment 216577


I have a tool like this that lets me quickly do this stuff. It's kind of manual, but I can get 15 unique articles in 5 minutes, so much faster than offers on the internet.

I think it's something interesting in terms of UX but it lacks in terms of basics like my own training of models. Then coupling models together.

I have an idea to create more automated scenarios with it. Though it won't go far without nailing those basics and having money for investment in training.
Honestly you are doing great just keep up I might pm you from time to time hope you don't get bothered
 
Yes, I changed my profile picture. This is just another businessman picture I found on the internet. This one is a little bit more elegant, but frankly I change it based on my mood.

View attachment 216577


I have a tool like this that lets me quickly do this stuff. It's kind of manual, but I can get 15 unique articles in 5 minutes, so much faster than offers on the internet.

I think it's something interesting in terms of UX but it lacks in terms of basics like my own training of models. Then coupling models together.

I have an idea to create more automated scenarios with it. Though it won't go far without nailing those basics and having money for investment in training.
That was so heavy back then.

Now I write only simple browsers addons.

It's easier to work with them when you want to create *custom news content* and grab unique stuff *of your own choice*.

No need for js frameworks. JS itself is enough. Not even jquery...

If you're not in corporate, frameworks are complicating your venture. Unless it's your own, custom framework for your own big venture.

I dream of my own framework for automation. Plus no-code tool. And then you let AI figure out all paths based on human written text, then you just provide corrections...

Would make you start new journey in another social media in matter of days instead of months.
 
Back
Top