[Journey] Reverse Engineering Google with AI (Fine-tuning only. Advanced level)

[4440]

{"prompt":"OUTLINE OF ARTICLE\n\n###\n\n","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work\nexplain cryptocurrency\ncryptocurrency explained\nwhat's crypto\nwhat is cryptocurrency and how it works\nwhat crypto\nwhat is crypto money\nunderstanding cryptocurrency\ncripto currency\nwhat is cryptocurrency used for\nwhat is crypto coin\nwhat is a crypto\ncrypto basics\nwhat is crypto currency and how does it work\nkrypto currency\ncrypto currency for dummies\nwhat cryptocurrency is this\ncrytocurrency\nencrypted currency\ncrypto what is it\nwhat is cripto\nwhat exactly is cryptocurrency\ncyrptocurrency\nwhat is cryto\ncryptocurrency basics\nis crypto\nencryption currency\nabout cryptocurrency\ncryptocurrency how it works\nwhat are crypto coins\nbasics of cryptocurrency\ncrypto curency\nwhat is krypto\nwhat is the purpose of cryptocurrency\nwhat is cryptocurrency for dummies\ncrypto currancy\ncryptocurrency explained simply\nwhat is a crypto coin\nall about cryptocurrency\nhow to understand cryptocurrency\nwhat is cryptocurrency and how it works?\ncrypto currency explained\ncryptocurrancy\nintro to cryptocurrency\nunderstanding crypto\nexplain cryptocurrency to me\ncrptocurrency\ncypto currency\nwhat is crypto cash\nintroduction to cryptocurrency\nwhat is crypto coins\npurpose of cryptocurrency\ncryptocurreny\nhow does cryptocurrency work?\ncrypto.currency\ncrypocurrency\ncryptocurrency explained for dummies\ncryptocurrency definition for dummies\ncrypto explained simply\nwhat is cyber currency\nexplain cryptocurrency for dummies\nhow cryptocurrency works?\nwhat is crpto\nwhat is crypto used for\nhow does cryptocurrency work in simple terms\nwhat is crypto currency in simple terms\ncryptocurrecy\nwhat are cryptos\nwhat are the cryptocurrencies\nsimple explanation of cryptocurrency\ncryptocurrency system\ncryptocurrency.\nhow to explain cryptocurrency\nwhat is crypto and how does it work\nexplain crypto\ncryptocurrency information\nwhat is cyrpto\nwhat us crypto\neverything you need to know about cryptocurrency\nok google what is cryptocurrency\nwhat is cryptocurrency in simple terms\nunderstanding crypto currency\nwhat is.crypto\ncryptocurrenc\nwhy is crypto\ncrypto information\ncrypto knowledge\nwhats a crypto\nwhat is cypto\nencrypted currency meaning\neverything to know about cryptocurrency\ncryptocurrency purpose\ncryptocurrency is\ncryptocurrency simple definition\ni don't understand cryptocurrency\ncrypo currency\ncryptocurrency introduction\ncryptocurrency how does it work\nis crypto a currency\ncryptocurrecny\nehat is crypto\ncryptocurrency?\nwhat to know about cryptocurrency\nwhat are crypto\ncrypto curreny\ncryptocurrency info\nintro to crypto\nunderstanding cryptocurrencies\nis cryptocurrency virtual currency\ncryprocurrency\ncrypto how it works\nbasics of crypto\ncrypto for dummies explained\nwhat does cryptocurrency do\nexplanation of cryptocurrency\ncrytpo currency\ncryptocurr\nwhat does crypto do\ncryptocurrency overview\nwhat are cryptocurrencies used for\ncrypotcurrency\nexplaining cryptocurrency\nunderstand cryptocurrency\nhey google what is cryptocurrency\nthe basics of cryptocurrency\ncryptocurrenxy\ncryptocurrent\nwhat does crypto\ncryptocurrncy\ncrypto current\ncryptocurrenty\nwho is crypto\nhow crypto currency works\nintroduction to crypto\ncryptomoney\n\"cryptocurrency\"\ncryptocurrency currency\ncryptocurrnecy\ntell me about cryptocurrency\ncrypto currecy\ncrypto currentcy\ncrypto digital currency\ncyrtocurrency\nur cryptocurrency\nabout crypto\ncrupto currency\nsimple definition of cryptocurrency\ncryotocurrency ###"}


The article outline you need to do yourself and write a program to generate an outline based on a webpage.

One year back I was trying to generate a TOC with a complexed algo, calculating the possibilities of what heading should be h2, h3 and the inheritance.
Not until a few months back I realise that you can just send all headings into a LLM and let it organize a new one like what you did.
The difference is, instead of training/fine-tuning one LLM like you did, I simply go with few-shots and prompts. It works.

 

[splishsplash]

One year back I was trying to generate a TOC with a complexed algo, calculating the possibilities of what heading should be h2, h3 and the inheritance.
Not until a few months back I realise that you can just send all headings into a LLM and let it organize a new one like what you did.
The difference is, instead of training/fine-tuning one LLM like you did, I simply go with few-shots and prompts. It works.

You have to think carefully about this in terms of your definition of "it works"

It works the same way that giving your headings to your Aunt works. Assuming your Aunt can write of course ;-)

But the point is, she'll organize like a human would from a natural language perspective.

An LLM is the same. It's trained on natural language, so it's essentially just like a regular human.

Even the "You are an SEO engineer. blah blah" type prompts aren't that useful due to the way LLMs work.

Things like "Write an article in the style of the rapper Nas" work well because it's good at this. In inference mode it's very creative and can change styles very easily. What it struggles to do is access very specific information. Especially complex information.

For example..

A massive LLM will have been consumed stock market data. But you aren't going to be able to ask it *based on that data* why X company went up or down in Y month.

If you ask it that, what you're going to get is an answer based on what it's learned in general by reading about company X and the stock market.

Ie, and this is a really important distinction and 99.9999% of the world fails to understand about LLMs..

The pattern recognition doesn't come from the stock market data, but instead comes from reading about what others write to similar questions. THAT is the pattern recognition in an LLM. In chatgpt the pattern is related to dialog. Patterns in how to respond to questions. I do believe they reason and aren't just simple prediction machines like simpler transformers like BERT are.

But chatgpt is still reasoning within the context of the patterns of responding to dialog.

It's reasoning within its bubble. It isn't looking for any patterns in data related to SEO and what actually ranks. Or stock market data.

What we're actually trying to do with finetuning is limit the patterns to just the raw data with the thing we're concerned with.

In this case, the outlines we would want for the finetune to give a keyword and get an outline wouldn't just be ones that make good sense from a natural language perspective, but ones that are statistically closer to what ranks for similar keywords.


I, personally have a lot of SEO experience, so if you give me a random keyword, I can produce something closer to what you'd expect to rank without ANY research, just based on pattern matching from years of seeing sites.

Ie, give me a keyword "Best Super Widescreen Monitor for Office Use"

Or give me "How to train a German Shepherd" and I can come up with something that'll do pretty well without even looking at what's currently ranking.

If you gave that to a non-SEO they might give you a lovely sounding article with nice headings, but would it be optimal for SEO? Doubtful, otherwise we would have never needed SEOs in the first place. Just hiring a writer to write an article would give us the perfect piece to rank.


So what we really want is multiple finetuned models trained on ranking data for their page class.

For the classification model you need to train 1 model, passing the class and the data because it needs to learn the differences between them in order to classify.

But, for the outline generators, we aren't interested in the differences between page classes, because there will already be enough differences for it to understand within the 1 class type(ie ecommerce product pages), so we want to keep the training as accurate as possible by doing only 1 model per class type.

 

[4440]

You have to think carefully about this in terms of your definition of "it works"

It works the same way that giving your headings to your Aunt works. Assuming your Aunt can write of course ;-)

But the point is, she'll organize like a human would from a natural language perspective.

An LLM is the same. It's trained on natural language, so it's essentially just like a regular human.

Even the "You are an SEO engineer. blah blah" type prompts aren't that useful due to the way LLMs work.

Things like "Write an article in the style of the rapper Nas" work well because it's good at this. In inference mode it's very creative and can change styles very easily. What it struggles to do is access very specific information. Especially complex information.

For example..

A massive LLM will have been consumed stock market data. But you aren't going to be able to ask it *based on that data* why X company went up or down in Y month.

If you ask it that, what you're going to get is an answer based on what it's learned in general by reading about company X and the stock market.

Ie, and this is a really important distinction and 99.9999% of the world fails to understand about LLMs..

The pattern recognition doesn't come from the stock market data, but instead comes from reading about what others write to similar questions. THAT is the pattern recognition in an LLM. In chatgpt the pattern is related to dialog. Patterns in how to respond to questions. I do believe they reason and aren't just simple prediction machines like simpler transformers like BERT are.

But chatgpt is still reasoning within the context of the patterns of responding to dialog.

It's reasoning within its bubble. It isn't looking for any patterns in data related to SEO and what actually ranks. Or stock market data.

What we're actually trying to do with finetuning is limit the patterns to just the raw data with the thing we're concerned with.

In this case, the outlines we would want for the finetune to give a keyword and get an outline wouldn't just be ones that make good sense from a natural language perspective, but ones that are statistically closer to what ranks for similar keywords.


I, personally have a lot of SEO experience, so if you give me a random keyword, I can produce something closer to what you'd expect to rank without ANY research, just based on pattern matching from years of seeing sites.

Ie, give me a keyword "Best Super Widescreen Monitor for Office Use"

Or give me "How to train a German Shepherd" and I can come up with something that'll do pretty well without even looking at what's currently ranking.

If you gave that to a non-SEO they might give you a lovely sounding article with nice headings, but would it be optimal for SEO? Doubtful, otherwise we would have never needed SEOs in the first place. Just hiring a writer to write an article would give us the perfect piece to rank.


So what we really want is multiple finetuned models trained on ranking data for their page class.

For the classification model you need to train 1 model, passing the class and the data because it needs to learn the differences between them in order to classify.

But, for the outline generators, we aren't interested in the differences between page classes, because there will already be enough differences for it to understand within the 1 class type(ie ecommerce product pages), so we want to keep the training as accurate as possible by doing only 1 model per class type.

Yeah, I just don't have the budget. If I have the budget I'll go train some local host LLMs for specific tasks too.
"It works" simply means it's a compromise for the moment, but the quality is still acceptable.

And, chatgpt still has very high benchmarks among all LLMs. With few-shot and prompting, I'm talking about multiple hand-selected, context-rich few-shot learning and recursive, CoT kind of prompts, chatgpt can have extremely high accuracy and performance that could come near to gpt4. Multiple papers have data that proves this.

Fine-tuning nor few-shot learning could teach LLM new knowledge. Few-shots are similar to fine-tuning which only shapes the output of LLM to a given pattern. We both know these. So my point is, few-shots are not as good as fine-tuning, but it is on a similar path.

For others who don't have the budget for fine-tuning a LLM like me, nicely designed prompts can get you nice results as well. The difference is, instead of investing money in fine-tuning a task-specific model, you need to invest time into pre-processing the data and coding your prompt designs.

GPT 3.5 is still one of the best LLMs out there with cheap rates, large parameters and it's fast. For people who don't have the budget or hardware for hosting their own models, my advice is to master GPT 3.5, coding skills and read papers. GPT 3.5 have the potential to get you at 90/100 high-quality results, you just learn your way to unlock it.

I highly respect your knowledge and sharing, no doubt.

 

[splishsplash]

Yeah, I just don't have the budget. If I have the budget I'll go train some local host LLMs for specific tasks too.
"It works" simply means it's a compromise for the moment, but the quality is still acceptable.

And, chatgpt still has very high benchmarks among all LLMs. With few-shot and prompting, I'm talking about multiple hand-selected, context-rich few-shot learning and recursive, CoT kind of prompts, chatgpt can have extremely high accuracy and performance that could come near to gpt4. Multiple papers have data that proves this.

Fine-tuning nor few-shot learning could teach LLM new knowledge. Few-shots are similar to fine-tuning which only shapes the output of LLM to a given pattern. We both know these. So my point is, few-shots are not as good as fine-tuning, but it is on a similar path.

For others who don't have the budget for fine-tuning a LLM like me, nicely designed prompts can get you nice results as well. The difference is, instead of investing money in fine-tuning a task-specific model, you need to invest time into pre-processing the data and coding your prompt designs.

GPT 3.5 is still one of the best LLMs out there with cheap rates, large parameters and it's fast. For people who don't have the budget or hardware for hosting their own models, my advice is to master GPT 3.5, coding skills and read papers. GPT 3.5 have the potential to get you at 90/100 high-quality results, you just learn your way to unlock it.

I highly respect your knowledge and sharing, no doubt.

few shots don't work when it comes to things as complex as SEO.

It would be like trying to few shot stockmarket prediction. It won't work.

You don't need budget.. https://github.com/qwopqwop200/GPTQ-for-LLaMa

4 bit llama. You can train 7b on an old 8GB graphics card, or rent a GPU for $1.40/day.

Or, use google colab free.

 

[tragicflaw]

Nice one, I haven't tried out the chat-gpt route but I think you have challanged me to try it out myself to rule it out

Apart from it, I didn't fully understand the pipeline. So the first part of the plan is to finetune a model to extract out possible high ranking keywords based on outlines, which gives you nice results. You are adding essentially a site classifier for model to be more fine grained, which is fine as well. Then, how exactly do you come up with an optimized outline? Is it, train another model to (real keywords,site classifcation) (model2) -> Outline OR single_keyword->outlines from page 1 -> kws_from_model1(outlines)->model2(kws,site classification->generated optimized outlines?


If it it is the second, why do we generate keywords from the outlines in the first place? can't we extract good ones already from the ranking pages and use the model(2) to generate an outline?

I might have missed it entirely but just curious I'm experienced in AI/LLM finetuning as well, you can hit me up anytime.

 

[splishsplash]

Yeah, I just don't have the budget. If I have the budget I'll go train some local host LLMs for specific tasks too.
"It works" simply means it's a compromise for the moment, but the quality is still acceptable.

And, chatgpt still has very high benchmarks among all LLMs. With few-shot and prompting, I'm talking about multiple hand-selected, context-rich few-shot learning and recursive, CoT kind of prompts, chatgpt can have extremely high accuracy and performance that could come near to gpt4. Multiple papers have data that proves this.

Fine-tuning nor few-shot learning could teach LLM new knowledge. Few-shots are similar to fine-tuning which only shapes the output of LLM to a given pattern. We both know these. So my point is, few-shots are not as good as fine-tuning, but it is on a similar path.

For others who don't have the budget for fine-tuning a LLM like me, nicely designed prompts can get you nice results as well. The difference is, instead of investing money in fine-tuning a task-specific model, you need to invest time into pre-processing the data and coding your prompt designs.

GPT 3.5 is still one of the best LLMs out there with cheap rates, large parameters and it's fast. For people who don't have the budget or hardware for hosting their own models, my advice is to master GPT 3.5, coding skills and read papers. GPT 3.5 have the potential to get you at 90/100 high-quality results, you just learn your way to unlock it.

I highly respect your knowledge and sharing, no doubt.

Also remember you absolutely dont need even 13b. 7b is so powerful. The instruct/dialog trained llama7bs outperform chatgpt in over half of tests. Its trained on 1 trillion tokens. Davinci was only trained on 400 billion tokens. This is huge. You basically don't NEED that many parameters. What you need is MORE training tokens.

7b and 13b are 1 trillion tokens, and 33b and 65b are trained on 1.4 trillion tokens. That's 3.5x as much as davinci.

So a 4 bit 7b is IMMENSELY powerful.

I'm getting results here with Curie, which is a 6b (iirc) model trained on 300 billion tokens.

It's just much more convenient and fast to train, but you'll have better results with a 4bit 7b llama, than you will with curie.

OpenAI is hugely overrated. It's not as good as you think. They just spent a huge amount of money on gathering training data and THIS was the key.

The reason 7b can outperform chatgpt on a lot of tasks is because they built training data with gpt4. This is the magic. But the model its self has the potential if you have the training data, which we now do thanks to openai.

So in summary, it's not openai's models that are unique. It was their training data.

https://bdtechtalks.com/2023/04/17/open-source-chatgpt-alternatives/
Tests show Vicuna outperforms LLaMA and Alpaca and is close to Bard and ChatGPT-4.. FOUR. That's how good this thing is. Vicuna outperforms chatgpt3.5 as judged by gpt4 in over 90% of cases. The general community has no idea how good these things are.

Nice one, I haven't tried out the chat-gpt route but I think you have challanged me to try it out myself to rule it out

Apart from it, I didn't fully understand the pipeline. So the first part of the plan is to finetune a model to extract out possible high ranking keywords based on outlines, which gives you nice results. You are adding essentially a site classifier for model to be more fine grained, which is fine as well. Then, how exactly do you come up with an optimized outline? Is it, train another model to (real keywords,site classifcation) (model2) -> Outline OR single_keyword->outlines from page 1 -> kws_from_model1(outlines)->model2(kws,site classification->generated optimized outlines?


If it it is the second, why do we generate keywords from the outlines in the first place? can't we extract good ones already from the ranking pages and use the model(2) to generate an outline?

I might have missed it entirely but just curious I'm experienced in AI/LLM finetuning as well, you can hit me up anytime.

Ok, so you always want to use real data to train.

It's different if we use gpt4 to create instruct series training samples or chat dialog then we can absolute get training data from gpt4.

This is what it was trained to do. Follow human instructions. Answer questions that humans ask.

It isn't trained to do anything related to SEO, so we would never use it for creating sample data to train SEO models otherwise you just get junk. Since the output from gpt4 is junk for SEO. It has no idea about actual live google ranking factors, which themselves are based on machine learning. We want to use machine learning to beat google's machine learning.

You must use LIVE DATA.

Everything I'm doing is live data.

People are getting a little confused and thinking I'm using chatgpt to generate outlines or keywords.

The only thing I had chatgpt do was give me ideas for search keywords. I gave it a few examples like "How does the stockmarket work", "Guide to content marketing". Ie, I was looking for tutorial/info style keywords to search for. That's the extent of chatgpt. I could have done this myself, but it was good enough at giving me ideas to search for.

So let's take 1 training sample. Let's say I choose "guide to content marketing".

Google it.

Top result is https://blog.hubspot.com/marketing/content-marketing

This is a real live ranking page.

Now, if this page ranks number 1 for this keyword, we can safely assume that the on-page for this is good, right?

We might have a page with perfect onpage that's ranking number 18. We don't know, do we? It could be 18 because it's not got enough backlinks. It's too new. The site it's on doesn't have enough topical authority. Any number of reasons. We basically DO NOT KNOW if the page ranking 18 is good or bad, so we can never use those.

We need to limit confounding variables. (That's a term in statistics for variables that you don't account for that influence the dependent and independent variables used and thus influence your cause and effect which is what you're trying to model, with your model in this case being used to predict outcomes)

Now, we take the outline of https://blog.hubspot.com/marketing/content-marketing

No chatgpt, no generating outlines. Why would we do that? That wouldn't be real data. We need real outlines of real ranking pages otherwise the model is fine tuned on garbage and won't give us meaningful data

Now, put https://blog.hubspot.com/marketing/content-marketing into ahrefs, export the top 5 keywords. Save the top 100, or 150. it depends really on the number of training samples.

I used 150 in the initial example which is too high for the 100 training samples used. More keywords, means more variation in the data, which means we need WAY more samples.

For 100 training samples it would have been better to train with 25 of the highest volume keywords.

For the bigger one I'm doing now I will experiment with this actually, but I'll probably go for the top 100 vol keywords

We could also test this using only the top 3, but my gut says top 5 is better because if it's ranking #1 for the main keyword you searched(Which should be a decent keyword, not a micro keyword, because if it's a micro keyword then we probably won't have many other useful keyword rankings. We want lots of longtails, so a more broad, slightly higher comp base keyword will give a page that's more likely to have 100+ keywords in the top 5)

And that's it.

Your training data is then just

OUTLINE

###

keywords


END


That's it.


Real outline, real keywords. Real data.


The next model is the classification model, so that I can then train 10-12 different models for the "get keywords from a page outline". So first I'd get a page, classify it, and then run the appropriate model. This is super important, because the type of keywords that will rank for a tutorial are radically different than the type that would rank for an ecom product page. The model needs to learn only for the page type.


Make sense now? Feel free to ask for clarification.

Jeez. Time wasting journey.

Yep, it's a huge waste of time to be running an AI research company and spend time finetuning models for SEO that can dramatically improve the ability to rank pages, build sites and make huge amounts of money. I mean it's not like I could ever create SaaS tools from any of this knowledge and sell them for $100's per month to 1000's of agencies and SEO's. Who would ever want AI SEO tools.. But oh well.

Machine learning after all has no application in the real world. I'd be better off just watching Netflix and playing World of Warcraft eh.

 

[Ghoztseo]

Also remember you absolutely dont need even 13b. 7b is so powerful. The instruct/dialog trained llama7bs outperform chatgpt in over half of tests. Its trained on 1 trillion tokens. Davinci was only trained on 400 billion tokens. This is huge. You basically don't NEED that many parameters. What you need is MORE training tokens.

7b and 13b are 1 trillion tokens, and 33b and 65b are trained on 1.4 trillion tokens. That's 3.5x as much as davinci.

So a 4 bit 7b is IMMENSELY powerful.

I'm getting results here with Curie, which is a 6b (iirc) model trained on 300 billion tokens.

It's just much more convenient and fast to train, but you'll have better results with a 4bit 7b llama, than you will with curie.

OpenAI is hugely overrated. It's not as good as you think. They just spent a huge amount of money on gathering training data and THIS was the key.

The reason 7b can outperform chatgpt on a lot of tasks is because they built training data with gpt4. This is the magic. But the model its self has the potential if you have the training data, which we now do thanks to openai.

So in summary, it's not openai's models that are unique. It was their training data.

https://bdtechtalks.com/2023/04/17/open-source-chatgpt-alternatives/
Tests show Vicuna outperforms LLaMA and Alpaca and is close to Bard and ChatGPT-4.. FOUR. That's how good this thing is. Vicuna outperforms chatgpt3.5 as judged by gpt4 in over 90% of cases. The general community has no idea how good these things are.




Ok, so you always want to use real data to train.

It's different if we use gpt4 to create instruct series training samples or chat dialog then we can absolute get training data from gpt4.

This is what it was trained to do. Follow human instructions. Answer questions that humans ask.

It isn't trained to do anything related to SEO, so we would never use it for creating sample data to train SEO models otherwise you just get junk. Since the output from gpt4 is junk for SEO. It has no idea about actual live google ranking factors, which themselves are based on machine learning. We want to use machine learning to beat google's machine learning.

You must use LIVE DATA.

Everything I'm doing is live data.

People are getting a little confused and thinking I'm using chatgpt to generate outlines or keywords.

The only thing I had chatgpt do was give me ideas for search keywords. I gave it a few examples like "How does the stockmarket work", "Guide to content marketing". Ie, I was looking for tutorial/info style keywords to search for. That's the extent of chatgpt. I could have done this myself, but it was good enough at giving me ideas to search for.

So let's take 1 training sample. Let's say I choose "guide to content marketing".

Google it.

Top result is https://blog.hubspot.com/marketing/content-marketing

This is a real live ranking page.

Now, if this page ranks number 1 for this keyword, we can safely assume that the on-page for this is good, right?

We might have a page with perfect onpage that's ranking number 18. We don't know, do we? It could be 18 because it's not got enough backlinks. It's too new. The site it's on doesn't have enough topical authority. Any number of reasons. We basically DO NOT KNOW if the page ranking 18 is good or bad, so we can never use those.

We need to limit confounding variables. (That's a term in statistics for variables that you don't account for that influence the dependent and independent variables used and thus influence your cause and effect which is what you're trying to model, with your model in this case being used to predict outcomes)

Now, we take the outline of https://blog.hubspot.com/marketing/content-marketing

No chatgpt, no generating outlines. Why would we do that? That wouldn't be real data. We need real outlines of real ranking pages otherwise the model is fine tuned on garbage and won't give us meaningful data

Now, put https://blog.hubspot.com/marketing/content-marketing into ahrefs, export the top 5 keywords. Save the top 100, or 150. it depends really on the number of training samples.

I used 150 in the initial example which is too high for the 100 training samples used. More keywords, means more variation in the data, which means we need WAY more samples.

For 100 training samples it would have been better to train with 25 of the highest volume keywords.

For the bigger one I'm doing now I will experiment with this actually, but I'll probably go for the top 100 vol keywords

We could also test this using only the top 3, but my gut says top 5 is better because if it's ranking #1 for the main keyword you searched(Which should be a decent keyword, not a micro keyword, because if it's a micro keyword then we probably won't have many other useful keyword rankings. We want lots of longtails, so a more broad, slightly higher comp base keyword will give a page that's more likely to have 100+ keywords in the top 5)

And that's it.

Your training data is then just

OUTLINE

###

keywords


END


That's it.


Real outline, real keywords. Real data.


The next model is the classification model, so that I can then train 10-12 different models for the "get keywords from a page outline". So first I'd get a page, classify it, and then run the appropriate model. This is super important, because the type of keywords that will rank for a tutorial are radically different than the type that would rank for an ecom product page. The model needs to learn only for the page type.


Make sense now? Feel free to ask for clarification.






Yep, it's a huge waste of time to be running an AI research company and spend time finetuning models for SEO that can dramatically improve the ability to rank pages, build sites and make huge amounts of money. I mean it's not like I could ever create SaaS tools from any of this knowledge and sell them for $100's per month to 1000's of agencies and SEO's. Who would ever want AI SEO tools.. But oh well.

Machine learning after all has no application in the real world. I'd be better off just watching Netflix and playing World of Warcraft eh.

Be practical, You afraid of big companies would steal your idea. So anyway you won't tell us even a little hint that what you doing, I'm a developer and I can understand that as well. Better if you keep working separately without writing this journey with lots of jargons. Well, you trying to market your name and campaign here. Don't get me wrong but this kinda feel like what you doing like a hypothetical case because you saying chatGPT is not good but idk man! people are making loads of bucks including me because I know How to fine tune properly :') . You maybe sounded like I know everything but you don't. Anyways good luck.

A big salute from me for your marketing techniques. Well, World of Warcraft is for kids maybe! Elden Ring, Warzone and Tears of Kingdom are the trend now.

 

[ww3]

looks interesting

 

[splishsplash]

Be practical, You afraid of big companies would steal your idea. So anyway you won't tell us even a little hint that what you doing, I'm a developer and I can understand that as well. Better if you keep working separately without writing this journey with lots of jargons. Well, you trying to market your name and campaign here. Don't get me wrong but this kinda feel like what you doing like a hypothetical case because you saying chatGPT is not good but idk man! people are making loads of bucks including me because I know How to fine tune properly :') . You maybe sounded like I know everything but you don't. Anyways good luck.

A big salute from me for your marketing techniques. Well, World of Warcraft is for kids maybe! Elden Ring, Warzone and Tears of Kingdom are the trend now.

What the hell are you waffling on about?

So anyway you won't tell us even a little hint that what you doing, I'm a developer and I can understand that as well.

I have given every single detail, from the data gathering, through giving the full data preparation code and commands to train. I even shared the hyper parameters used and the plans for the future tests with number of training samples, hyper parameters, data gather and data segmentation.

The ONLY thing I haven't shared was my function get an outline.

But I'm saying "It's an outline of a page". I'm sure people can work out how to get an outline..

You'd think I just wrote a few lines saying I fined tuned a model and here's the results, look at me. I've written 1000's of words and shared code and commands.

What exactly do you think I'm not sharing?

How utterly stupid to say I won't even share a "little hint". I've laid all out on a silver platter.

I'm a developer and I can understand that as well. Better if you keep working separately without writing this journey with lots of jargons

https://www.blackhatworld.com/seo/looking-for-a-dev-for-autogenerated-script.1356668/ <-- I thought you were a developer? Just another lie then?


This doesn't make sense. How can I share this without jargon? What do you want? Do you want me to just write a SaaS for you and set you up with access so you can "push a button".

This isn't made up jargon. It's very accurate technical terms related to machine learning. If you took a machine learning engineer from DeepMind, they wouldn't find any fault in my reasoning or use of jargon. It's very accurate. I know what I'm talking about. I study this stuff deeply and have been doing so since 2021. I didn't just read a couple of articles last week and then start writing.

Well, you trying to market your name and campaign here.

And it's bad to market your name by giving away free knowledge? I don't sell any courses and never will. I give away all knowledge for free. I will sell SaaS applications in the future, which I'll get an approved BST for, but I am sharing most parts of my journey here which will also allow other people to learn and benefit.

What a silly thing to say. What am I suppose to do, sit in my bedroom and stay quiet until I die? I like business. I like writing. I like putting my name out there. If you don't like it. Tough.

Don't get me wrong but this kinda feel like what you doing like a hypothetical case because you saying chatGPT is not good but idk man! people are making loads of bucks including me because I know How to fine tune properly :') . You maybe sounded like I know everything but you don't. Anyways good luck.

You know how to fine tune chatgpt properly do you?

You state that nonsense and then you quite rudely say I sound like I know what I'm talking about, but I don't really.

Please explain to me oh wise one, how you fine tune chatgpt?

I'd love to hear that one.

And why do you love following me around bhw attacking every post I make? Are you going for douche of the year award? You literally just appear everywhere I post and poke me.

 

[GustavoFung]

following

 

[SyntaxSurge]

People are getting a little confused and thinking I'm using chatgpt to generate outlines or keywords.

We can then give the finetuned model an outline, and it'll give us back keywords. Simple. There's no chatgpt, no answering, no supersetting of 3 sites. It's just the outline from 1 page that ranks for a main base keyword.

I completely understand what you're doing and it is just limited to getting keywords from the outline of Page 1 on SERP. But may I know what will you do to those extracted keywords? I know that you will use it to create articles for topical authority.

But my question is about the proccess of blog creation after extracting the keywords. Forget about the extraction of keywords from the outline of Page 1 on SERP, we already done that.

Suppose you already got the keywords, how will you make an outline then? The outline that I am talking about is not the outline for getting the keywords from page 1 of SERP, but rather the outline of your main article from your selected keyword. The actual content creation, not keyword finder.

1. When making the actual article, are you Copying the outline of TOP 1 result ONLY? or you choose or combine the outline from top 1 to 3 result, so your outline becomes superset of those 3 websites?

2. When answering the headings in the outline, are you using the chatgpt for each headings, So the responses are separated and the number would be the number of headings? OR you include all headings in the prompt to chatgpt, so the response would be only one?

Again, I am not talking about the extraction of keywords from SERP, but I am talking about the actual content creation, the aftermath of keyword research.

Sorry if this reply contains a lot of redundancy, but I kept on asking whether you are using chatgpt on article creation but your response is always about the extraction of keywords on SERP, ofcourse, you are not using chatgpt, neither do I, I am personally usinng semrush to export bulk keywords then use embeddings to sort and remove redundant keywords.

I am very sorry and Thank you very much

 

[DrBitcoin]

Yep, it's a huge waste of time to be running an AI research company and spend time finetuning models for SEO that can dramatically improve the ability to rank pages, build sites and make huge amounts of money. I mean it's not like I could ever create SaaS tools from any of this knowledge and sell them for $100's per month to 1000's of agencies and SEO's. Who would ever want AI SEO tools.. But oh well.

Machine learning after all has no application in the real world. I'd be better off just watching Netflix and playing World of Warcraft eh

AI SEO is starting to die bro

 

[splishsplash]

I completely understand what you're doing and it is just limited to getting keywords from the outline of Page 1 on SERP. But may I know what will you do to those extracted keywords? I know that you will use it to create articles for topical authority.

But my question is about the proccess of blog creation after extracting the keywords. Forget about the extraction of keywords from the outline of Page 1 on SERP, we already done that.

Suppose you already got the keywords, how will you make an outline then? The outline that I am talking about is not the outline for getting the keywords from page 1 of SERP, but rather the outline of your main article from your selected keyword. The actual content creation, not keyword finder.

1. When making the actual article, are you Copying the outline of TOP 1 result ONLY? or you choose or combine the outline from top 1 to 3 result, so your outline becomes superset of those 3 websites?

2. When answering the headings in the outline, are you using the chatgpt for each headings, So the responses are separated and the number would be the number of headings? OR you include all headings in the prompt to chatgpt, so the response would be only one?

Again, I am not talking about the extraction of keywords from SERP, but I am talking about the actual content creation, the aftermath of keyword research.

Sorry if this reply contains a lot of redundancy, but I kept on asking whether you are using chatgpt on article creation but your response is always about the extraction of keywords on SERP, ofcourse, you are not using chatgpt, neither do I, I am personally usinng semrush to export bulk keywords then use embeddings to sort and remove redundant keywords.

I am very sorry and Thank you very much

What I'll do with them?

You're asking my grand master plan?! ;-)

It's mostly just about understanding small pieces of the puzzle with multiple models and then using those to build tools.

In terms of actual uses for the first one, it's not the MOST useful, but it's an important first step.

When it's fully done it's fairly interesting to be able to give it an outline and have it tell you what keywords you are most likely to rank for.

This would actually help with coming up with lots of longtail anchors too.

Actually a big bonus with this is you can refine your article based on it.

So for example, we ran it on the "how to write fb ads" article that's already #1 and we got some interesting longtails back like


"example of a good facebook ad"
"strong headline for a facebook ad"
"how to personify in facebook ad"
"good ad format for facebook"
"how to write a facebook page post" <-- This here, is an example of a related article we could create and internally link which is coming back because it's highly relevant but shouldn't necessarily be part of that page.

The others.. Those would make excellent additions to rank for more longtails in the article

Adding sections with an example, talking about the ad format, how to make the facebook ad more personable and maybe a few examples of strong headlines(getting both those keywords in there)

Using semrush to extract the keywords is spot on.

You don't need to use embeddings. I wouldn't use embeddings.. Embeddings are NOT using transformers trained with SEO in mind. They are sentence transformers with natural language, so they can remove things they shouldn't.

You don't NEED to remove anything redundant, that's the point. Google does this for you.

Anything outside the top 5 is redundant. That is the magic of this. Google is telling you the relevant keywords for this outline..


And as for your question "How do we create the outline"

We don't.

That's not what this model does. Input is the outline, output is keywords.

If you want outline from a keyword that's a totally different finetune for another day.

Also you said a few times "when making the article".. What article? We aren't making any articles.

This is really simple.

We give it an outline, we get keywords. Done. There's no article. It's just getting keywords from a single outline.

 

[KHer0]

Be practical, You afraid of big companies would steal your idea. So anyway you won't tell us even a little hint that what you doing, I'm a developer and I can understand that as well. Better if you keep working separately without writing this journey with lots of jargons. Well, you trying to market your name and campaign here. Don't get me wrong but this kinda feel like what you doing like a hypothetical case because you saying chatGPT is not good but idk man! people are making loads of bucks including me because I know How to fine tune properly :') . You maybe sounded like I know everything but you don't. Anyways good luck.

A big salute from me for your marketing techniques. Well, World of Warcraft is for kids maybe! Elden Ring, Warzone and Tears of Kingdom are the trend now.

Don't you have someone in your life to waste his time instead of ours!

One reply here worths more than your whole account.

From your replies it shows how dumb you are to benefit from this journey! However, don't spread your dumbness here and if you don't understand something then keep it to yourself instead of embarrassing yourself in public.

This is one of the greatest journies that I have ever seen in the forum, reminds me of zen10 not completed journey.

@splishsplash keep going and never let this thread die. You are doing remarkable work here. There isn't much replies as you are exploring new areas, so no one really has experience on it.

I am learning a lot from this, although I would have preferred if AI would help us on the off page side as this is where seems the real competition is.

However, the model you are working one could be a huge breakthrough in the whole SEO industry if it achieves noticable difference with end results.

Again, this is my main fear, I don't think algorithm spend much time measuring on-page signals... you got some page and your visitors are happy? great, lets move on to off page.

What I mean, I am afraid the end results of this model won't give noticable difference compared to normal written human articles.

However, if we think about it as developing an AI content writer which has better understanding of on-page seo metrics than humans! WOW! Think of the possibilities!!!

And to think this breakthrough comes from BHW, you should be considered national treasure for this forum for choosing it to publish your research. Just, don't forget us when you finally make it .. maybe some lifetime subscriptions for your first subscribers ^_^

Anyway, whatever happens, please don't stop updating this journey

PS: like I told you before, not eveyone deserves your attention! In some cases, ignoring is the best reply

 

[splishsplash]

This is one of the greatest journies that I have ever seen in the forum, reminds me of zen10 not completed journey.

@splishsplash keep going and never let this thread die. You are doing remarkable work here. There isn't much replies as you are exploring new areas, so no one really has experience on it.

Thanks for the kind words and the support!

I am learning a lot from this, although I would have preferred if AI would help us on the off page side as this is where seems the real competition is.

However, the model you are working one could be a huge breakthrough in the whole SEO industry if it achieves noticable difference with end results.

Again, this is my main fear, I don't think algorithm spend much time measuring on-page signals... you got some page and your visitors are happy? great, lets move on to off page.

What I mean, I am afraid the end results of this model won't give noticable difference compared to normal written human articles.

However, if we think about it as developing an AI content writer which has better understanding of on-page seo metrics than humans! WOW! Think of the possibilities!!!

And to think this breakthrough comes from BHW, you should be considered national treasure for this forum for choosing it to publish your research. Just, don't forget us when you finally make it .. maybe some lifetime subscriptions for your first subscribers ^_^

Anyway, whatever happens, please don't stop updating this journey

PS: like I told you before, not eveyone deserves your attention! In some cases, ignoring is the best reply

On-page is definitely very very important.

In my experience, the most powerful way to rank and get traffic is siloing/topical authority building.

This isn't strictly on-page though.. But, it does make a big difference.

Just not so much for smaller longtails that have a very specific user intention like "Why does my persian cat headbutt me".. There is not much you can do to optimize that An H1, a couple fo H2's, a few paragraphs and you're golden.

On-page becomes more important for medium tails, and obviously broad.

I've done a lot of analyses over the years. So so much.. I'm almost like a fine tuned model myself ;-)

What I can tell you with the bigger keywords, more medium tail, is that you'll sometimes see 1 or 2 pages that rank in the top 10 with smaller, weaker looking content, but, every time I see this, those pages have BOTH strong topical authority and strong links. Always both. If it's weak links and strong topical authority, you will see meatier on-page with many more topics/entities covered on the page. If it's weak topical authority and very strong links, you will also see the very strong on-page.

This is what's common for broad/medium tails :-

Strong links, medium topical authority and strong on-page.

Weak links, high topical authority and strong on-page.

Strong links, high topical authority and weak on-page.

Just those 3. That's the common patterns I've seen a lot over the past 3-4 years since topical authority has become prominent. 2017-2018 was the era of links and authority sites. Awful time in SEO when it was dominated by links. 2019 is when topical authority became important. Coincidentally Sep 2019 was the first introduction of the first transformer model BERT. Or Oct it might have been. I can't remember.

So the main takeaway here is I don't often see an 'average' on-page for the medium tail and broad keywords.

I also don't often see no to low topical authority and strong links. This is a rare one and it's why I say topical authority is king in SEO.

The exact one I'm fine tuning just now though is for getting keywords from outlines.. It's not inherently useful. It's just *interesting* from a research point of view because we can easily verify the results by plugging them into keyword.com and it gives us a nice basis for seeing "can we actually reverse engineer google".

But, after having done it I'm seeing some surprising uses.

I'll re-quote what I wrote above in case you missed it(I'm writing a LOT!)

Actually a big bonus with this is you can refine your article based on it.

So for example, we ran it on the "how to write fb ads" article that's already #1 and we got some interesting longtails back like


"example of a good facebook ad"
"strong headline for a facebook ad"
"how to personify in facebook ad"
"good ad format for facebook"
"how to write a facebook page post" <-- This here, is an example of a related article we could create and internally link which is coming back because it's highly relevant but shouldn't necessarily be part of that page.

The others.. Those would make excellent additions to rank for more longtails in the article

Adding sections with an example, talking about the ad format, how to make the facebook ad more personable and maybe a few examples of strong headlines(getting both those keywords in there)

So we could use this for a tool that gives you suggestions for more sections on an article.

We could refine this more by building an actual app that takes the results, looks at the page and then suggests actual headings.

THIS would be what chatgpt would be good for. Give it the page outline and ask it to add in another h2 or h3 that would cover the extra keywords.

So you'd give it for example the

"example of a good facebook ad"
"strong headline for a facebook ad"
"how to personify in facebook ad"
"good ad format for facebook"

In fact let's try it.

This will give away my outline format, but fuck it.. Anyone can work it out, it's not hard.

So I tried chatgpt for this..

Seriously, chatgpt is CRAP for SEO.

The results aren't even worth posting. Total waste of time. Maybe people think it's good because they don't know what good on-page looks like, but from my perspective it's repetitive.

Instead, I tried with gpt4. I like its results a lot. Here's the result and prompt

Prompt

Here is an outline for an article


title:9 Tips to Write the Best Facebook Ads Ever (with Examples)
h6:Recent Articles
h6:How Will ChatGPT & AI Impact PPC? 6 Expert Predictions
h6:6 Absolute Best Strategies to Grow Your Digital Marketing Agency in 2023
h6:Google Marketing Live 2023: The 11 Biggest Updates You Can’t Miss
h4:Reader Favorites
h4opular Articles
h6:Featured Tools
h4PC University
h4:Guides
h4ata & Benchmarks
h4:WordStream
h4:LocaliQ
h1:9 Tips to Write the Best Facebook Ads Ever (with Examples)
h2:Why Facebook Advertising Works So Well
h2:The Two Types of Facebook Ads
h2:But Does Copywriting for Facebook Ads Really Matter?
h2:9 Strategies for Writing Your Best Facebook Ads
h3:1. Use Facebook Targeting to Narrow Your Audience, then Write to It
h3:2. Write Different Facebook Ads for Different People
h3:3. Ensure Your Facebook Ad Copy Goes with Your Visual
h3:4. Stay Focused with One Call to Action
h3:5. Keep it Short and Lead with Value
h3:6. Use Simple Language that’s Easy to Understand
h3:7. Be Upfront About the Numbers
h3:8. Get Some Perspective
h3:9. Test your Ad Copy
h2:Write Copy That Drives Results on Facebook



I want you to re-write this outline to include new sections so we can cover answers for the following questions

"example of a good facebook ad"
"strong headline for a facebook ad"
"how to personify in facebook ad"
"good ad format for facebook"

Highlight the new sections by adding NEW::: in front of them.

GPT4 Reply

Sure, here's the revised outline incorporating the new sections, with each addition marked by "NEW:::":

h1: 9 Tips to Write the Best Facebook Ads Ever (with Examples)

h2: Why Facebook Advertising Works So Well

h2: The Two Types of Facebook Ads

h2: But Does Copywriting for Facebook Ads Really Matter?

h2: 9 Strategies for Writing Your Best Facebook Ads

h3: 1. Use Facebook Targeting to Narrow Your Audience, then Write to It

h3: 2. Write Different Facebook Ads for Different People

h3: 3. Ensure Your Facebook Ad Copy Goes with Your Visual

h3: 4. Stay Focused with One Call to Action

h3: 5. Keep it Short and Lead with Value

h3: 6. Use Simple Language that’s Easy to Understand

h3: 7. Be Upfront About the Numbers

h3: 8. Get Some Perspective

h3: 9. Test your Ad Copy

h2: Write Copy That Drives Results on Facebook

NEW::: h2: Diving Deeper: Answering Your Specific Facebook Ad Questions

NEW::: h3: What Makes a Good Facebook Ad: A Comprehensive Example

NEW::: h3: Crafting a Strong Headline for Your Facebook Ad

NEW::: h3: Incorporating Personification in Your Facebook Ad

NEW::: h3: Selecting the Best Ad Format for Facebook

h6: Recent Articles

h6: How Will ChatGPT & AI Impact PPC? 6 Expert Predictions

h6: 6 Absolute Best Strategies to Grow Your Digital Marketing Agency in 2023

h6: Google Marketing Live 2023: The 11 Biggest Updates You Can’t Miss

h4: Reader Favorites

h4: Popular Articles

h6: Featured Tools

h4: PPC University

h4: Guides

h4: Data & Benchmarks

h4: WordStream

h4: LocaliQ

You could even have it automatically written too, so the power level has just gone up 2 notches.

1) Identify potential longtails missing from articles on your site. Apply this to a site with 2k articles and you're going to get a huge traffic increase. Right now it needed me to pick them out, but it wouldn't be hard to use AI to pick out the best candidates.

2) gpt4 gives us new headings and where to insert. Fully automatic.

3) AI writer writes the new content.

Boom.. Automatically get 10-20% increase in traffic minimum for your site with zero work and zero human input. And that's just from the first fine tuned model.

As for your second question about links..

That's to come, don't worry

Oh boy, I have so many ideas for this. I have no idea what kind of results we'll get and I don't want to get overconfident, but I think we could see some amazing things in terms of identifying the links that are causing sites to rank, and the links that are doing nothing, then extrapolating those patterns to tools that can automatically assess any potential backlink.

Combine that with another tool I have in the works which is a fully automated outreach AI(does everything from find sites to link on, email, negotiate, pay, write the content, send the content, check the link is live, create the reports. It's like an outreach staff member, except AI) then you have a fully automated link building solution that will only build links that have a positive impact..

 

[DrBitcoin]

Thanks for the kind words and the support!



On-page is definitely very very important.

In my experience, the most powerful way to rank and get traffic is siloing/topical authority building.

This isn't strictly on-page though.. But, it does make a big difference.

Just not so much for smaller longtails that have a very specific user intention like "Why does my persian cat headbutt me".. There is not much you can do to optimize that An H1, a couple fo H2's, a few paragraphs and you're golden.

On-page becomes more important for medium tails, and obviously broad.

I've done a lot of analyses over the years. So so much.. I'm almost like a fine tuned model myself ;-)

What I can tell you with the bigger keywords, more medium tail, is that you'll sometimes see 1 or 2 pages that rank in the top 10 with smaller, weaker looking content, but, every time I see this, those pages have BOTH strong topical authority and strong links. Always both. If it's weak links and strong topical authority, you will see meatier on-page with many more topics/entities covered on the page. If it's weak topical authority and very strong links, you will also see the very strong on-page.

This is what's common for broad/medium tails :-

Strong links, medium topical authority and strong on-page.

Weak links, high topical authority and strong on-page.

Strong links, high topical authority and weak on-page.

Just those 3. That's the common patterns I've seen a lot over the past 3-4 years since topical authority has become prominent. 2017-2018 was the era of links and authority sites. Awful time in SEO when it was dominated by links. 2019 is when topical authority became important. Coincidentally Sep 2019 was the first introduction of the first transformer model BERT. Or Oct it might have been. I can't remember.

So the main takeaway here is I don't often see an 'average' on-page for the medium tail and broad keywords.

I also don't often see no to low topical authority and strong links. This is a rare one and it's why I say topical authority is king in SEO.

The exact one I'm fine tuning just now though is for getting keywords from outlines.. It's not inherently useful. It's just *interesting* from a research point of view because we can easily verify the results by plugging them into keyword.com and it gives us a nice basis for seeing "can we actually reverse engineer google".

But, after having done it I'm seeing some surprising uses.

I'll re-quote what I wrote above in case you missed it(I'm writing a LOT!)





So we could use this for a tool that gives you suggestions for more sections on an article.

We could refine this more by building an actual app that takes the results, looks at the page and then suggests actual headings.

THIS would be what chatgpt would be good for. Give it the page outline and ask it to add in another h2 or h3 that would cover the extra keywords.

So you'd give it for example the

"example of a good facebook ad"
"strong headline for a facebook ad"
"how to personify in facebook ad"
"good ad format for facebook"

In fact let's try it.

This will give away my outline format, but fuck it.. Anyone can work it out, it's not hard.

So I tried chatgpt for this..

Seriously, chatgpt is CRAP for SEO.

The results aren't even worth posting. Total waste of time. Maybe people think it's good because they don't know what good on-page looks like, but from my perspective it's repetitive.

Instead, I tried with gpt4. I like its results a lot. Here's the result and prompt

Prompt


GPT4 Reply


You could even have it automatically written too, so the power level has just gone up 2 notches.

1) Identify potential longtails missing from articles on your site. Apply this to a site with 2k articles and you're going to get a huge traffic increase. Right now it needed me to pick them out, but it wouldn't be hard to use AI to pick out the best candidates.

2) gpt4 gives us new headings and where to insert. Fully automatic.

3) AI writer writes the new content.

Boom.. Automatically get 10-20% increase in traffic minimum for your site with zero work and zero human input. And that's just from the first fine tuned model.

As for your second question about links..

That's to come, don't worry

Oh boy, I have so many ideas for this. I have no idea what kind of results we'll get and I don't want to get overconfident, but I think we could see some amazing things in terms of identifying the links that are causing sites to rank, and the links that are doing nothing, then extrapolating those patterns to tools that can automatically assess any potential backlink.

Combine that with another tool I have in the works which is a fully automated outreach AI(does everything from find sites to link on, email, negotiate, pay, write the content, send the content, check the link is live, create the reports. It's like an outreach staff member, except AI) then you have a fully automated link building solution that will only build links that have a positive impact..

what if you just feed it with the top 3 results and let your llm do something similar to that?
that way you can output content similar to the top3 in any phrases/kws

 

[4440]

Also remember you absolutely dont need even 13b. 7b is so powerful. The instruct/dialog trained llama7bs outperform chatgpt in over half of tests. Its trained on 1 trillion tokens. Davinci was only trained on 400 billion tokens. This is huge. You basically don't NEED that many parameters. What you need is MORE training tokens.

7b and 13b are 1 trillion tokens, and 33b and 65b are trained on 1.4 trillion tokens. That's 3.5x as much as davinci.

So a 4 bit 7b is IMMENSELY powerful.

I'm getting results here with Curie, which is a 6b (iirc) model trained on 300 billion tokens.

It's just much more convenient and fast to train, but you'll have better results with a 4bit 7b llama, than you will with curie.

OpenAI is hugely overrated. It's not as good as you think. They just spent a huge amount of money on gathering training data and THIS was the key.

The reason 7b can outperform chatgpt on a lot of tasks is because they built training data with gpt4. This is the magic. But the model its self has the potential if you have the training data, which we now do thanks to openai.

So in summary, it's not openai's models that are unique. It was their training data.

https://bdtechtalks.com/2023/04/17/open-source-chatgpt-alternatives/
Tests show Vicuna outperforms LLaMA and Alpaca and is close to Bard and ChatGPT-4.. FOUR. That's how good this thing is. Vicuna outperforms chatgpt3.5 as judged by gpt4 in over 90% of cases. The general community has no idea how good these things are.




Ok, so you always want to use real data to train.

It's different if we use gpt4 to create instruct series training samples or chat dialog then we can absolute get training data from gpt4.

This is what it was trained to do. Follow human instructions. Answer questions that humans ask.

It isn't trained to do anything related to SEO, so we would never use it for creating sample data to train SEO models otherwise you just get junk. Since the output from gpt4 is junk for SEO. It has no idea about actual live google ranking factors, which themselves are based on machine learning. We want to use machine learning to beat google's machine learning.

You must use LIVE DATA.

Everything I'm doing is live data.

People are getting a little confused and thinking I'm using chatgpt to generate outlines or keywords.

The only thing I had chatgpt do was give me ideas for search keywords. I gave it a few examples like "How does the stockmarket work", "Guide to content marketing". Ie, I was looking for tutorial/info style keywords to search for. That's the extent of chatgpt. I could have done this myself, but it was good enough at giving me ideas to search for.

So let's take 1 training sample. Let's say I choose "guide to content marketing".

Google it.

Top result is https://blog.hubspot.com/marketing/content-marketing

This is a real live ranking page.

Now, if this page ranks number 1 for this keyword, we can safely assume that the on-page for this is good, right?

We might have a page with perfect onpage that's ranking number 18. We don't know, do we? It could be 18 because it's not got enough backlinks. It's too new. The site it's on doesn't have enough topical authority. Any number of reasons. We basically DO NOT KNOW if the page ranking 18 is good or bad, so we can never use those.

We need to limit confounding variables. (That's a term in statistics for variables that you don't account for that influence the dependent and independent variables used and thus influence your cause and effect which is what you're trying to model, with your model in this case being used to predict outcomes)

Now, we take the outline of https://blog.hubspot.com/marketing/content-marketing

No chatgpt, no generating outlines. Why would we do that? That wouldn't be real data. We need real outlines of real ranking pages otherwise the model is fine tuned on garbage and won't give us meaningful data

Now, put https://blog.hubspot.com/marketing/content-marketing into ahrefs, export the top 5 keywords. Save the top 100, or 150. it depends really on the number of training samples.

I used 150 in the initial example which is too high for the 100 training samples used. More keywords, means more variation in the data, which means we need WAY more samples.

For 100 training samples it would have been better to train with 25 of the highest volume keywords.

For the bigger one I'm doing now I will experiment with this actually, but I'll probably go for the top 100 vol keywords

We could also test this using only the top 3, but my gut says top 5 is better because if it's ranking #1 for the main keyword you searched(Which should be a decent keyword, not a micro keyword, because if it's a micro keyword then we probably won't have many other useful keyword rankings. We want lots of longtails, so a more broad, slightly higher comp base keyword will give a page that's more likely to have 100+ keywords in the top 5)

And that's it.

Your training data is then just

OUTLINE

###

keywords


END


That's it.


Real outline, real keywords. Real data.


The next model is the classification model, so that I can then train 10-12 different models for the "get keywords from a page outline". So first I'd get a page, classify it, and then run the appropriate model. This is super important, because the type of keywords that will rank for a tutorial are radically different than the type that would rank for an ecom product page. The model needs to learn only for the page type.


Make sense now? Feel free to ask for clarification.






Yep, it's a huge waste of time to be running an AI research company and spend time finetuning models for SEO that can dramatically improve the ability to rank pages, build sites and make huge amounts of money. I mean it's not like I could ever create SaaS tools from any of this knowledge and sell them for $100's per month to 1000's of agencies and SEO's. Who would ever want AI SEO tools.. But oh well.

Machine learning after all has no application in the real world. I'd be better off just watching Netflix and playing World of Warcraft eh.

Yes I am aware of the 4-bit and 7B versions. They can run on colab or locally and can do powerful things.
I was planning to host and test some Vicuna, local-llama small weights some days ago, but soon I give up and decide to master and sharpen my gpt 3.5 skill first and slowly move to local LLMs.

LLMs are developing so fast, new models come out every single day, claiming higher performance with lower parameters. Chasing new LLMs and consuming new information every day leaves me anxious and brings no real production. I learn new things(not even learn, I just hear about new LLMs coming out) and want to try new things. In the end, production is zero.

That said, I'd rather set up a full set-up gpt 3.5, leave it running auto-pilot on the cloud without worrying and then try new LLMs. So production and the cost-of-testing-new-LLMs can happen concurrently, the opportunity cost is acceptable for me.

 

[splishsplash]

what if you just feed it with the top 3 results and let your llm do something similar to that?
that way you can output content similar to the top3 in any phrases/kws

That's next, but it's not a good idea to do it with the top 3.

That would be for example 1000 keywords and 3000 training samples.

It's better to do 3000 keywords, only take the top 1 result, and you then have 3000 keywords and 3000 training samples. More variety, and it's only learning from the TOP TOP result.

 

[DrBitcoin]

That's next, but it's not a good idea to do it with the top 3.

That would be for example 1000 keywords and 3000 training samples.

It's better to do 3000 keywords, only take the top 1 result, and you then have 3000 keywords and 3000 training samples. More variety, and it's only learning from the TOP TOP result.

I just remembered that google likes to pace 1 site in #1 in serps for several keywords, for 3k related keywords, we should actually expect way less than 3k unique urls

 

[GenesisOne]

This is a really advanced SEO guide.
Thanks for sharing @splishsplash