[Journey] Reverse Engineering Google with AI (Fine-tuning only. Advanced level)

[SyntaxSurge]

Thanks, If I learn fine-tuning, I would use it instead of gpt3.5. My only problem in fine-tuning is which data should I feed into it.

I am currently fine-tuning gpt3.5 directly into the prompt, instructing it what to do and what pattern to output.

May I know what is the difference between fine-tuning it in command line vs fine-tuning the result on the prompts? Thanks

 

[splishsplash]

Thanks, If I learn fine-tuning, I would use it instead of gpt3.5. My only problem in fine-tuning is which data should I feed into it.

I am currently fine-tuning gpt3.5 directly into the prompt, instructing it what to do and what pattern to output.

May I know what is the difference between fine-tuning it in command line vs fine-tuning the result on the prompts? Thanks

That's not finetuning, that's just 1-shot, 2-shot, 5-shot prompting.

It doesn't physically alter the model's weights(its brain)


It's not like us, where if you show me something once or twice, I will learn..

When you're using chatgpt like that you're running it in what's called "inference mode". It's like your brain becomes completely STATIC/FROZEN in its current state.

Fine-tuning is specifically, doing further training on the model.

To do it with OpenAI you need to follow the guide here - https://platform.openai.com/docs/guides/fine-tuning

 

[UnusualSubstance]

This is going to be a pure-gold journey! Most won't even know that this is gold, lol.

 

[splishsplash]

Here is the actual screenshot from the 2nd test article now keyword.com updated properly



That's just crazy-good. 45 of the 70 keywords it generated are in the top 3, and it did this only from a page outline.

That's really hard to do even do as a human.

Its ability to give you keywords that are in the top 3, most of the time #1, but fairly different to the title is astounding.

I mean the title of the page https://www.wordstream.com/blog/ws/best-facebook-ads, is "9 Tips to Write the Best Facebook Ads Ever"

and it gives back keywords like "how to write facebook marketing ads", "how to be one of the best facebook advertisers", "writing facebook ads that work", "facebook ad copy critique for effective copy" etc..

I can't wait to see what's possible with training to give it a keyword then getting back the outline. It's going to pretty much make any on-page SEO optimization tool obsolete and superfluous.

In a way here we are poking under the hood of Google and seeing all the keywords that it considers exactly spot on relevancy wise. This kind of information can be used to build up a strong relevancy picture and be used to optimize pages to rank as well as a separate model for giving outlines.

And another interesting thing, if we look at the keywords that aren't in the top 100 like

"strong headline for facebook ad"

"example of a good facebook ad"

"good ad format for facebook"

"what is the best word to use in a facebook ad"

These are all pretty fucking nice keywords that you could add sections into the existing article to broaden the relevancy and add more "topical depth" to the article. These aren't "random", unrelated keywords. As a human I can see.. Actually, these would fit really well in the article, but they DONT cover them. Maybe they SHOULD?

 

[SyntaxSurge]

Thanks, if possible, may I get just one example of your fine-tunning data? I'll just use it as a basis as I haven't tried fine-tuning yet. Thanks
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

 

[splishsplash]

Thanks, if possible, may I get just one example of your fine-tunning data? I'll just use it as a basis as I haven't tried fine-tuning yet. Thanks
{"prompt": "<prompt text>", "completion": "<ideal generated text>"}

{"prompt":"OUTLINE OF ARTICLE\n\n###\n\n","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work\nexplain cryptocurrency\ncryptocurrency explained\nwhat's crypto\nwhat is cryptocurrency and how it works\nwhat crypto\nwhat is crypto money\nunderstanding cryptocurrency\ncripto currency\nwhat is cryptocurrency used for\nwhat is crypto coin\nwhat is a crypto\ncrypto basics\nwhat is crypto currency and how does it work\nkrypto currency\ncrypto currency for dummies\nwhat cryptocurrency is this\ncrytocurrency\nencrypted currency\ncrypto what is it\nwhat is cripto\nwhat exactly is cryptocurrency\ncyrptocurrency\nwhat is cryto\ncryptocurrency basics\nis crypto\nencryption currency\nabout cryptocurrency\ncryptocurrency how it works\nwhat are crypto coins\nbasics of cryptocurrency\ncrypto curency\nwhat is krypto\nwhat is the purpose of cryptocurrency\nwhat is cryptocurrency for dummies\ncrypto currancy\ncryptocurrency explained simply\nwhat is a crypto coin\nall about cryptocurrency\nhow to understand cryptocurrency\nwhat is cryptocurrency and how it works?\ncrypto currency explained\ncryptocurrancy\nintro to cryptocurrency\nunderstanding crypto\nexplain cryptocurrency to me\ncrptocurrency\ncypto currency\nwhat is crypto cash\nintroduction to cryptocurrency\nwhat is crypto coins\npurpose of cryptocurrency\ncryptocurreny\nhow does cryptocurrency work?\ncrypto.currency\ncrypocurrency\ncryptocurrency explained for dummies\ncryptocurrency definition for dummies\ncrypto explained simply\nwhat is cyber currency\nexplain cryptocurrency for dummies\nhow cryptocurrency works?\nwhat is crpto\nwhat is crypto used for\nhow does cryptocurrency work in simple terms\nwhat is crypto currency in simple terms\ncryptocurrecy\nwhat are cryptos\nwhat are the cryptocurrencies\nsimple explanation of cryptocurrency\ncryptocurrency system\ncryptocurrency.\nhow to explain cryptocurrency\nwhat is crypto and how does it work\nexplain crypto\ncryptocurrency information\nwhat is cyrpto\nwhat us crypto\neverything you need to know about cryptocurrency\nok google what is cryptocurrency\nwhat is cryptocurrency in simple terms\nunderstanding crypto currency\nwhat is.crypto\ncryptocurrenc\nwhy is crypto\ncrypto information\ncrypto knowledge\nwhats a crypto\nwhat is cypto\nencrypted currency meaning\neverything to know about cryptocurrency\ncryptocurrency purpose\ncryptocurrency is\ncryptocurrency simple definition\ni don't understand cryptocurrency\ncrypo currency\ncryptocurrency introduction\ncryptocurrency how does it work\nis crypto a currency\ncryptocurrecny\nehat is crypto\ncryptocurrency?\nwhat to know about cryptocurrency\nwhat are crypto\ncrypto curreny\ncryptocurrency info\nintro to crypto\nunderstanding cryptocurrencies\nis cryptocurrency virtual currency\ncryprocurrency\ncrypto how it works\nbasics of crypto\ncrypto for dummies explained\nwhat does cryptocurrency do\nexplanation of cryptocurrency\ncrytpo currency\ncryptocurr\nwhat does crypto do\ncryptocurrency overview\nwhat are cryptocurrencies used for\ncrypotcurrency\nexplaining cryptocurrency\nunderstand cryptocurrency\nhey google what is cryptocurrency\nthe basics of cryptocurrency\ncryptocurrenxy\ncryptocurrent\nwhat does crypto\ncryptocurrncy\ncrypto current\ncryptocurrenty\nwho is crypto\nhow crypto currency works\nintroduction to crypto\ncryptomoney\n\"cryptocurrency\"\ncryptocurrency currency\ncryptocurrnecy\ntell me about cryptocurrency\ncrypto currecy\ncrypto currentcy\ncrypto digital currency\ncyrtocurrency\nur cryptocurrency\nabout crypto\ncrupto currency\nsimple definition of cryptocurrency\ncryotocurrency ###"}


The article outline you need to do yourself and write a program to generate an outline based on a webpage.

 

[SyntaxSurge]

{"prompt":"OUTLINE OF ARTICLE\n\n###\n\n","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work\nexplain cryptocurrency\ncryptocurrency explained\nwhat's crypto\nwhat is cryptocurrency and how it works\nwhat crypto\nwhat is crypto money\nunderstanding cryptocurrency\ncripto currency\nwhat is cryptocurrency used for\nwhat is crypto coin\nwhat is a crypto\ncrypto basics\nwhat is crypto currency and how does it work\nkrypto currency\ncrypto currency for dummies\nwhat cryptocurrency is this\ncrytocurrency\nencrypted currency\ncrypto what is it\nwhat is cripto\nwhat exactly is cryptocurrency\ncyrptocurrency\nwhat is cryto\ncryptocurrency basics\nis crypto\nencryption currency\nabout cryptocurrency\ncryptocurrency how it works\nwhat are crypto coins\nbasics of cryptocurrency\ncrypto curency\nwhat is krypto\nwhat is the purpose of cryptocurrency\nwhat is cryptocurrency for dummies\ncrypto currancy\ncryptocurrency explained simply\nwhat is a crypto coin\nall about cryptocurrency\nhow to understand cryptocurrency\nwhat is cryptocurrency and how it works?\ncrypto currency explained\ncryptocurrancy\nintro to cryptocurrency\nunderstanding crypto\nexplain cryptocurrency to me\ncrptocurrency\ncypto currency\nwhat is crypto cash\nintroduction to cryptocurrency\nwhat is crypto coins\npurpose of cryptocurrency\ncryptocurreny\nhow does cryptocurrency work?\ncrypto.currency\ncrypocurrency\ncryptocurrency explained for dummies\ncryptocurrency definition for dummies\ncrypto explained simply\nwhat is cyber currency\nexplain cryptocurrency for dummies\nhow cryptocurrency works?\nwhat is crpto\nwhat is crypto used for\nhow does cryptocurrency work in simple terms\nwhat is crypto currency in simple terms\ncryptocurrecy\nwhat are cryptos\nwhat are the cryptocurrencies\nsimple explanation of cryptocurrency\ncryptocurrency system\ncryptocurrency.\nhow to explain cryptocurrency\nwhat is crypto and how does it work\nexplain crypto\ncryptocurrency information\nwhat is cyrpto\nwhat us crypto\neverything you need to know about cryptocurrency\nok google what is cryptocurrency\nwhat is cryptocurrency in simple terms\nunderstanding crypto currency\nwhat is.crypto\ncryptocurrenc\nwhy is crypto\ncrypto information\ncrypto knowledge\nwhats a crypto\nwhat is cypto\nencrypted currency meaning\neverything to know about cryptocurrency\ncryptocurrency purpose\ncryptocurrency is\ncryptocurrency simple definition\ni don't understand cryptocurrency\ncrypo currency\ncryptocurrency introduction\ncryptocurrency how does it work\nis crypto a currency\ncryptocurrecny\nehat is crypto\ncryptocurrency?\nwhat to know about cryptocurrency\nwhat are crypto\ncrypto curreny\ncryptocurrency info\nintro to crypto\nunderstanding cryptocurrencies\nis cryptocurrency virtual currency\ncryprocurrency\ncrypto how it works\nbasics of crypto\ncrypto for dummies explained\nwhat does cryptocurrency do\nexplanation of cryptocurrency\ncrytpo currency\ncryptocurr\nwhat does crypto do\ncryptocurrency overview\nwhat are cryptocurrencies used for\ncrypotcurrency\nexplaining cryptocurrency\nunderstand cryptocurrency\nhey google what is cryptocurrency\nthe basics of cryptocurrency\ncryptocurrenxy\ncryptocurrent\nwhat does crypto\ncryptocurrncy\ncrypto current\ncryptocurrenty\nwho is crypto\nhow crypto currency works\nintroduction to crypto\ncryptomoney\n\"cryptocurrency\"\ncryptocurrency currency\ncryptocurrnecy\ntell me about cryptocurrency\ncrypto currecy\ncrypto currentcy\ncrypto digital currency\ncyrtocurrency\nur cryptocurrency\nabout crypto\ncrupto currency\nsimple definition of cryptocurrency\ncryotocurrency ###"}


The article outline you need to do yourself and write a program to generate an outline based on a webpage.

So, I need to make a fine-tune data for each article? Because what if the article is not related to crypto?

Also, is this how it works:
If my prompt is "Please Make an OUTLINE OF ARTICLE about how to make money online"
it's answer is related to the fine-tuned data right?

The article outline you need to do yourself and write a program to generate an outline based on a webpage.

Yeah, the format of my gpt3.5 prompt for outline is like this:

Develop a comprehensive {blog_language} outline for a long-form article for the title of "{title}" and a focus keyword of "{keyword}" It should feature maximum of {headings_count} engaging headings and subheadings (H2, H3, H4, ...) that are detailed, mutually exclusive, collectively exhaustive and cover the entire topic. Use "{keyword}" in subheading(s) like H2, H3, H4, etc.. only if relevant. Use "{keyword}" in the content. My articles should contain jokes, anecdotes, internal links, definitions, counterarguments, examples, statistics, historical facts, and quotes, ONLY IF APPLICABLE AND WHEN THE USER INTENT IS MATCHED. Format your outline using HTML tags like <h2>, <h3>, <p>, and etc. Do not add <h1> tag. Do not add title in the headings. Remove You May Also Like and other articles or topic recommendations, do not include it. You can remove or exclude the headings that are not fitted, necessary, or related to the title and keyword. You can also include FAQs in the outline. The headings and questions should be related to "{keyword}".

Your response should be related to any of these categories: [{blog_categories}]
Choose only ONE category, then make an outline based on the given title and keyword.

Also, add images in the outline, only if necessary and relevant. Do not add images at the end. The images should be in between contents. Insert images in between paragraphs or in between headings and outline as long as these images are relevant. Also add SEO optimized image alt text that contains the focused keyword.

Only add ONE <h2>Conclusion</h2>

Add <h2>References</h2>, add trusted links to relevant texts, with this format: <a href="YOUR TRUSTED LINK HERE" target="_blank">RELEVANT TEXT HERE</a>. Again, you should only write the outline of the article. Do not explicitly add the words "Steps" and Numberings in the headings. For example, Do not add "Step 1, Step 2, and so on" Do not also add "Method 1, Method 2, and so on" Do not also add "Heading 1, Heading 2, and so on" Do not also add "Subheading 1, Subheading 2, and so on" Do not also add "Subtitle 1, Subtitle 2, and so on" Do not explicitly state these in the headings and paragraphs. Do not also add numberings like "1., 2., 3., and so on" instead, you can use lists if there are numberings in the headings. The <h2>Conclusion</h2> and <h2>References</h2> should be located at the very end of the outline.

All of your responses should be written in {blog_language}
Answer the question in the introduction. Make your response direct to the point also Answer the questions direct to the point. Answer Yes or No, then explanation if possible or give the direct answer that readers are looking for.
For example, if readers want template for letters, then provide example templates for letters. Just also give what the readers are looking for based on the keyword. Give examples.


The format of your outline is:

<h2>Title 1</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

<img src="" alt="PLACE THE IMAGE ALT TEXT HERE">

<h2>Title 2</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

Add direct to the point examples to your outline. Add step by step process. Make sure to answer the keywords and questions directly to the point. You can base or combine the headings from the given articles below, so your outline becomes a superset of the top articles:

{headings_of_top_1_search_result}
{headings_of_top_2_search_result}
{headings_of_top_3_search_result}

But yeah, it consumes a lot of tokens. I wonder how can I fine-tune that. Thanks

 

[nohupt]

jeez man, tell us how you REALLY feel LMAO

but i'm jk, i am 100% with you. this is going to be a great journey thread.

 

[splishsplash]

So, I need to make a fine-tune data for each article? Because what if the article is not related to crypto?

Also, is this how it works:
If my prompt is "Please Make an OUTLINE OF ARTICLE about how to make money online"
it's answer is related to the fine-tuned data right?



Yeah, the format of my gpt3.5 prompt for outline is like this:

Develop a comprehensive {blog_language} outline for a long-form article for the title of "{title}" and a focus keyword of "{keyword}" It should feature maximum of {headings_count} engaging headings and subheadings (H2, H3, H4, ...) that are detailed, mutually exclusive, collectively exhaustive and cover the entire topic. Use "{keyword}" in subheading(s) like H2, H3, H4, etc.. only if relevant. Use "{keyword}" in the content. My articles should contain jokes, anecdotes, internal links, definitions, counterarguments, examples, statistics, historical facts, and quotes, ONLY IF APPLICABLE AND WHEN THE USER INTENT IS MATCHED. Format your outline using HTML tags like <h2>, <h3>, <p>, and etc. Do not add <h1> tag. Do not add title in the headings. Remove You May Also Like and other articles or topic recommendations, do not include it. You can remove or exclude the headings that are not fitted, necessary, or related to the title and keyword. You can also include FAQs in the outline. The headings and questions should be related to "{keyword}".

Your response should be related to any of these categories: [{blog_categories}]
Choose only ONE category, then make an outline based on the given title and keyword.

Also, add images in the outline, only if necessary and relevant. Do not add images at the end. The images should be in between contents. Insert images in between paragraphs or in between headings and outline as long as these images are relevant. Also add SEO optimized image alt text that contains the focused keyword.

Only add ONE <h2>Conclusion</h2>

Add <h2>References</h2>, add trusted links to relevant texts, with this format: <a href="YOUR TRUSTED LINK HERE" target="_blank">RELEVANT TEXT HERE</a>. Again, you should only write the outline of the article. Do not explicitly add the words "Steps" and Numberings in the headings. For example, Do not add "Step 1, Step 2, and so on" Do not also add "Method 1, Method 2, and so on" Do not also add "Heading 1, Heading 2, and so on" Do not also add "Subheading 1, Subheading 2, and so on" Do not also add "Subtitle 1, Subtitle 2, and so on" Do not explicitly state these in the headings and paragraphs. Do not also add numberings like "1., 2., 3., and so on" instead, you can use lists if there are numberings in the headings. The <h2>Conclusion</h2> and <h2>References</h2> should be located at the very end of the outline.

All of your responses should be written in {blog_language}
Answer the question in the introduction. Make your response direct to the point also Answer the questions direct to the point. Answer Yes or No, then explanation if possible or give the direct answer that readers are looking for.
For example, if readers want template for letters, then provide example templates for letters. Just also give what the readers are looking for based on the keyword. Give examples.


The format of your outline is:

<h2>Title 1</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

<img src="" alt="PLACE THE IMAGE ALT TEXT HERE">

<h2>Title 2</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

Add direct to the point examples to your outline. Add step by step process. Make sure to answer the keywords and questions directly to the point. You can base or combine the headings from the given articles below, so your outline becomes a superset of the top articles:

{headings_of_top_1_search_result}
{headings_of_top_2_search_result}
{headings_of_top_3_search_result}

But yeah, it consumes a lot of tokens. I wonder how can I fine-tune that. Thanks

No, why would you have chatgpt3 generate an outline That's just hallucinated nonsense. It has nothing to do with what actually ranks on google.

You need outlines from real pages that are ranked #1 along with all the page 1 keywords they rank for. You need REAL data. Finetuning curie with hallucinated chatgpt data won't do anything. You could already get hallucinated output from chatgpt now.


I did a test just now with the fb ad guide page.

So, I crafted a better prompt for chatgpt, gave it the outline and this is the keywords it gave me :-

Facebook advertising copywriting best practices
Tips for writing Facebook ad headlines
Writing compelling Facebook ad descriptions
Facebook ad copywriting examples and tips
How to write attention-grabbing Facebook ads
Improving Facebook ad conversions with copywriting
Crafting persuasive Facebook ad campaigns
Writing effective Facebook ad call-to-actions
Facebook ad writing techniques for better results
Optimizing Facebook ad copy for higher engagement
Strategies for writing killer Facebook ads
Facebook ad copy that converts
Creating impactful Facebook ad messaging
Secrets to writing irresistible Facebook ads
Facebook ad copywriting hacks
Writing persuasive Facebook ad copy that sells
Tips for writing Facebook ad headlines that convert
Facebook advertising tips for copywriters
Writing Facebook ads that stand out
Crafting compelling Facebook ad stories
How to write engaging Facebook ad content
Facebook ad copywriting tricks for success
Writing effective Facebook ad bullet points
Facebook ad copywriting formulas for high conversions
Tips for writing Facebook carousel ads
Facebook ad copywriting for e-commerce
Writing impactful Facebook ad visuals
Strategies for writing Facebook video ads
Writing Facebook ads for lead generation
Facebook ad copywriting for mobile users
Tips for writing persuasive Facebook ad offers
Writing Facebook ads for local businesses
Facebook ad copywriting for B2B marketing
Writing Facebook ads for brand awareness
Tips for writing dynamic Facebook ads
Writing Facebook ads for retargeting campaigns
Facebook ad copywriting for event promotion
Writing Facebook ads for app installs
Tips for writing effective Facebook ad headlines
Facebook ad copywriting for nonprofit organizations
Writing Facebook ads for real estate marketing
Facebook ad copywriting for holiday campaigns
Writing Facebook ads for restaurant promotions
Tips for writing Facebook ads for fashion brands
Writing Facebook ads for travel and tourism
Facebook ad copywriting for healthcare industry
Writing Facebook ads for educational institutions
Tips for writing Facebook ads for fitness and wellness
Writing Facebook ads for automotive industry

Facebook ad copywriting for professional services

Now, if you put these into keyword.com a lot of them are actually top 3 for the page. HOWEVER..

Look at them. They aren't really like search keywords are there? They're just phrases. The ones that are top 3, are just phrases based on the outline, so naturally they do rank for the page.

Things like "tips for writing facebook ad headlines", and "facebook advertising copywriting best practices" or "writing effective facebook ad call-to-actions.

It's just taking keywords from the outline and phrasing them.

Then you have crap like

"writing impactful facebook ad visuals"

"tips for writing facebook ads for fitness and wellness"

Compare that with my fine tuned model. Even visually eyeball them

how to write facebook ads
how to write a facebook ad
write a facebook ad
how to write ads for facebook
writing facebook ads
writing effective facebook ads
best practices for writing facebook ads
writing ads for facebook
how to write facebook marketing ads
writing ads on facebook
how to write a facebook marketing ad
writing ad for facebook
how to be an ad writer for facebook
how to become a better facebook advertising writer
writing effective facebook ads
how to create facebook ads
how to be a better ad writer for facebook
how to write a facebook page post
write an effective facebook ad
how to create facebook ads that get lots more people
write your facebook ads
how to get consumers to click in a facebook ad?
how to write an effective facebook ad
example of a good facebook ad
write a facebook ad post
how to improve facebook ad campaigns
ad writing for facebook
best practices and strategies for writing effective facebook ads
how to improve facebook ads
example of good facebook advertisement
best practice for writing facebook ads
strong headline for facebook ad
how to write facebook ads that work
how to write an effective advert for facebook
facebook ad writing tips
appropriate ad format for facebook
how to create facebook ads that convert
advertisement writing for facebook
facebook ad copy critique for effective copy
how to reduce spam on facebook
facebook ads online content review
how to write effective facebook ads
how to personify in facebook ad
how to write a good facebook ad
best practices in facebook ads
how to become the best facebook ad writer
2dcreative carr lahig cood
ad creating for facebook
what is the best word to use in a facebook ad?
how to improve facebook ads that don't convert?
best facebook ad practices
best practices for writing great facebook ads
writing facebook ads that work
how to write your facebook ad
good ad format for facebook
how to be one of the best facebook advertisers?
facebook ads copy
copywriting a facebook ad
writing facebook ad copy
help writing facebook ads
how to write the best facebook ad
write facebook advertisements
number 1 content for facebook ad
best advertisment for facebook ads
writing an effective facebook ad
facebook ads best practices
best practice for writing a facebook ad
best practices for facebook ad copy
how to write an effective facebook ad post
good ad format for facebook ads
best practices for facebook advertising

These LOOK like keywords.


Here's a further test..

If we pop the chatgpt ones in ahrefs keyword explorer we get https://share.getcloudapp.com/v1uWKxnb

Nothing, absolutely nothing. Not a single one is even a 0-10 vol.

If we put the finetuned model's keywords in, we get

https://share.getcloudapp.com/xQuEdANX
1/3 of them have volume.

Even the other 2/3 that don't, they are "search"'ish. The chatgpt one reads more like a bunch of article titles.

So while a keyword like "good ad format for fb" doesn't appear to have volume in ahrefs, you can clearly see that it's a micro-topic you could add into the article, and the actual search term might be something like "whats a good ad format for facebook". In fact it is, that's a 0-10 vol. So that "good ad format for fb" will probably have about 10-15 0-10 vol longtail searches..

Where-as stuff like "writing impactful facebook ad visuals" or "strategies for writing killer facebook ads" are just fluffy type titles/sub-headings with lots of adjectives.

That's the thing with chatgpt. It gives you a lot of fancy adjectives and rephrases some parts of the outline, but it's not giving you actual micro-topics(ie, keywords representing microtopics) for that article.

So, you absolutely MUST when finetuning get source data from google.

That means, you search for something like "how to become a notary in Texas", then you get all the keywords on page 1, and you get the outline. You repeat that 100+ times and you fine-tune that data. Then to get more advanced you classify each page type, and you fine-tune with the classification class.

 

[TechEinstien]

Thanks for this kind of thread. This is gold.

 

[SyntaxSurge]

No, why would you have chatgpt3 generate an outline That's just hallucinated nonsense. It has nothing to do with what actually ranks on google.

You need outlines from real pages that are ranked #1 along with all the page 1 keywords they rank for. You need REAL data. Finetuning curie with hallucinated chatgpt data won't do anything. You could already get hallucinated output from chatgpt now.


I did a test just now with the fb ad guide page.

So, I crafted a better prompt for chatgpt, gave it the outline and this is the keywords it gave me :-





Now, if you put these into keyword.com a lot of them are actually top 3 for the page. HOWEVER..

Look at them. They aren't really like search keywords are there? They're just phrases. The ones that are top 3, are just phrases based on the outline, so naturally they do rank for the page.

Things like "tips for writing facebook ad headlines", and "facebook advertising copywriting best practices" or "writing effective facebook ad call-to-actions.

It's just taking keywords from the outline and phrasing them.

Then you have crap like

"writing impactful facebook ad visuals"

"tips for writing facebook ads for fitness and wellness"

Compare that with my fine tuned model. Even visually eyeball them




These LOOK like keywords.


Here's a further test..

If we pop the chatgpt ones in ahrefs keyword explorer we get https://share.getcloudapp.com/v1uWKxnb

Nothing, absolutely nothing. Not a single one is even a 0-10 vol.

If we put the finetuned model's keywords in, we get

https://share.getcloudapp.com/xQuEdANX
1/3 of them have volume.

Even the other 2/3 that don't, they are "search"'ish. The chatgpt one reads more like a bunch of article titles.

So while a keyword like "good ad format for fb" doesn't appear to have volume in ahrefs, you can clearly see that it's a micro-topic you could add into the article, and the actual search term might be something like "whats a good ad format for facebook". In fact it is, that's a 0-10 vol. So that "good ad format for fb" will probably have about 10-15 0-10 vol longtail searches..

Where-as stuff like "writing impactful facebook ad visuals" or "strategies for writing killer facebook ads" are just fluffy type titles/sub-headings with lots of adjectives.

That's the thing with chatgpt. It gives you a lot of fancy adjectives and rephrases some parts of the outline, but it's not giving you actual micro-topics(ie, keywords representing microtopics) for that article.

So, you absolutely MUST when finetuning get source data from google.

That means, you search for something like "how to become a notary in Texas", then you get all the keywords on page 1, and you get the outline. You repeat that 100+ times and you fine-tune that data. Then to get more advanced you classify each page type, and you fine-tune with the classification class.

Thanks
I now understand that you are using API to get the keywords for each ranking pages on SERP, I thought you scrape each websites then fetch the heading tags (H1, H2, H3, H4, H5, H6) then used the fetched heading tags to the fine tuning of the model.

That means, you search for something like "how to become a notary in Texas", then you get all the keywords on page 1, and you get the outline. You repeat that 100+ times and you fine-tune that data. Then to get more advanced you classify each page type, and you fine-tune with the classification class.

1. Does that mean you are appending other search query/main keywords unrelated to "how to become a notary in Texas" to the fined-tuned model? And you are doing this for 100+ times foreach different keywords OR foreach keyword variation of the main keyword?
Different keywords Like this:

{"prompt":"OUTLINE OF ARTICLE\n\n#1\n\n","completion":" how to become a notary in Texas\nhow does notary in Texas work\nwhat is notary in Texas....}
{"prompt":"OUTLINE OF ARTICLE\n\n#2\n\n","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work....}
{"prompt":"OUTLINE OF ARTICLE\n\n#3\n\n","completion":" fb ad guide\nhow to write facebook ads\nhow to write a facebook ad....}

Repeat this 100+ times for each different main keywords use different prompts

2. Are you using the example fine-tuned model above that contains the keywords of first page results that have been repeated for each query/main keyword 100+ times, to make different articles that contains different keywords? Or before you make that article you are appending the main keywords data (keywords in the SERP) to the current fine-tuned model?

how to write facebook ads
how to write a facebook ad
write a facebook ad
how to write ads for facebook
writing facebook ads
writing effective facebook ads
best practices for writing facebook ads
writing ads for facebook
how to write facebook marketing ads
writing ads on facebook
how to write a facebook marketing ad
writing ad for facebook
how to be an ad writer for facebook
how to become a better facebook advertising writer
writing effective facebook ads
how to create facebook ads
how to be a better ad writer for facebook

3. Lets say you already have the lists of keywords for page 1 on SERP, as what you posted above. Now, how can you make the outline for an article out of it? What I mean by outline is the heading tags (H1, H2, H3, H4, H5, H6). Because once you have an outline, you can generate answers for each headings using gpt. In my current case, I am instructing the gpt to format the outline directly on the prompt like this and I will fetch it using beautifulsoup in python:

The format of your outline is:

<h2>Title 1</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

<img src="" alt="PLACE THE IMAGE ALT TEXT HERE">

<h2>Title 2</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

3. Since you are using fine-tuning, I assume you aren't using gpt4 nor gpt3.5 as these models doesn't support fine-tuning, except text-davinci-003. My question is, how are you dealing with long answers? Or have you ever encounter exceed of token limits? It's pretty normal in my case as I am using long prompts.

Also, sometimes gpt generates long answers, resulting an incomplete response because of the limit. In chat models like gpt3.5 and gpt4, there is an option there to continue the response. But as far as I know, because I used text-davinci-003 before, you can not continue incomplete response there.

Sorry if I have a lot of questions, Thanks for your response

 

[splishsplash]

Thanks
I now understand that you are using API to get the keywords for each ranking pages on SERP, I thought you scrape each websites then fetch the heading tags (H1, H2, H3, H4, H5, H6) then used the fetched heading tags to the fine tuning of the model.


1. Does that mean you are appending other search query/main keywords unrelated to "how to become a notary in Texas" to the fined-tuned model? And you are doing this for 100+ times foreach different keywords OR foreach keyword variation of the main keyword?
Different keywords Like this:

{"prompt":"OUTLINE OF ARTICLE\n\n#1\n\n","completion":" how to become a notary in Texas\nhow does notary in Texas work\nwhat is notary in Texas....}
{"prompt":"OUTLINE OF ARTICLE\n\n#2\n\n","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work....}
{"prompt":"OUTLINE OF ARTICLE\n\n#3\n\n","completion":" fb ad guide\nhow to write facebook ads\nhow to write a facebook ad....}

Repeat this 100+ times for each different main keywords use different prompts

2. Are you using the example fine-tuned model above that contains the keywords of first page results that have been repeated for each query/main keyword 100+ times, to make different articles that contains different keywords? Or before you make that article you are appending the main keywords data (keywords in the SERP) to the current fine-tuned model?

how to write facebook ads
how to write a facebook ad
write a facebook ad
how to write ads for facebook
writing facebook ads
writing effective facebook ads
best practices for writing facebook ads
writing ads for facebook
how to write facebook marketing ads
writing ads on facebook
how to write a facebook marketing ad
writing ad for facebook
how to be an ad writer for facebook
how to become a better facebook advertising writer
writing effective facebook ads
how to create facebook ads
how to be a better ad writer for facebook

3. Lets say you already have the lists of keywords for page 1 on SERP, as what you posted above. Now, how can you make the outline for an article out of it? What I mean by outline is the heading tags (H1, H2, H3, H4, H5, H6). Because once you have an outline, you can generate answers for each headings using gpt. In my current case, I am instructing the gpt to format the outline directly on the prompt like this and I will fetch it using beautifulsoup in python:

The format of your outline is:

<h2>Title 1</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

<img src="" alt="PLACE THE IMAGE ALT TEXT HERE">

<h2>Title 2</h2>
<p>Subtitle 1</p>
<p>Subtitle 2</p>

3. Since you are using fine-tuning, I assume you aren't using gpt4 nor gpt3.5 as these models doesn't support fine-tuning, except text-davinci-003. My question is, how are you dealing with long answers? Or have you ever encounter exceed of token limits? It's pretty normal in my case as I am using long prompts.

Also, sometimes gpt generates long answers, resulting an incomplete response because of the limit. In chat models like gpt3.5 and gpt4, there is an option there to continue the response. But as far as I know, because I used text-davinci-003 before, you can not continue incomplete response there.

Sorry if I have a lot of questions, Thanks for your response

1. I do scrape the websites and use the heading tags.

You're overcomplicating this.

It's just 2 things

prompt: outline of a webpage

completion: all the keywords that webpage ranks for on page 1

That's it. Nothing more.

2. See above. You're massively overcomplicating. It's just a page outline and the keywords it ranks for. That's it.

3. You don't make the outline of an article from keywords. You scrape the page and extract the outline with your code using a lib of your choice like bs4. You don't instruct chatgpt to do anything. Forget chatgpt. It has nothing to do with this. You aren't generating outlines with chatgpt. That's totally useless and hallucinated garbage from chatgpt.

4. You can't fine tune text-davinci-003, or text-davinci-002 either. Only the base davinci, curie, babbage and ada. davinci-002, 003 and 3.5 are already fine tuned. gpt4 is also fine tuned. We don't have access to a base gpt4.

How I deal with long answers: Truncate your context window of course You prep the data to fit and be less than 2048 tokens.

I'm not sure what you mean by "gpt generates long answers". You aren't using gpt to generate anything. You are getting the outline of the page, FROM the page.. The actual outline of the page, not chatgpt's version of what it thinks would be an outline. If you did that, you wouldn't have real live ranking examples, you'd just have a chatgpt hallucination. This is crucial..

 

[Ashk881]

single product review - A review of a single product

best reviews - Multiple reviews/top10/best X page

news - A news article

faq - A FAQ page of some sort

forum - A forum post

tutorial - I want to differentiate between informational posts and tutorials guides. A tutorial would be like the big guides you find when you type things like "beginners guide to seo", or "how to do content marketing". These are not really 'informational', they are tutorials which are different. Informational is more your "how to groom a persian cat" etc

service - Service pages. Plumbers, seo services.. Any sort of business/person selling a service.

recipe - cooking recipes

That's all I can think of for now, but please make suggestions if you think I've missed something

You might need another class for sparse content. These would be things like Profile Pages, company contact pages, checkout pages, Real Estate Listings, etc. Mostly user generated or some "autogenerated" content that don't really fit into either of informational, services or
Forum pages.

You might find some more classes on
https://schema.org/WebPage
I tried solving exactly something like this last year. Tho, my first attempt was using purely statistical and small DL nlp techniques (LDAs, small RoBerta models etc) and the second was using a huge LLM, with yandex's YaLM model.

Neither gave good results. The LLM one especially was too under-trained (both the base model and my fine-tuning) since it was just so huge, 100 billion parameters
I wouldn't even afford runing a decent run on it.

Looks like small models with chinchilla optimal training like llama are really the best suited for these kind of tasks. The base models are trained on so much more tokens that even the smaller models that are cheap on inference side perform Really well, and fine tuning them makes it even better.

I've blown out my grant tho so can't really play these stuff anymore.
Hope you enjoy this journey

 

[SyntaxSurge]

prompt: outline of a webpage

completion: all the keywords that webpage ranks for on page 1

I see, so you are just focusing on Getting keywords from a page outline. I thought you are also talking about the actual content creation.

so the format is like this:

Where Title is H2 and, Subtitle is either H3, H4, H5, H6:

{"prompt":"Title 1\nSubtitle 1\nTitle 2\nSubtitle 2","completion":" how to become a notary in Texas\nhow does notary in Texas work\nwhat is notary in Texas....}
{"prompt":"Title 1\nSubtitle 1\nTitle 2\nSubtitle 2","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work....}

I'm not sure what you mean by "gpt generates long answers". You aren't using gpt to generate anything. You are getting the outline of the page, FROM the page.. The actual outline of the page, not chatgpt's version of what it thinks would be an outline. If you did that, you wouldn't have real live ranking examples, you'd just have a chatgpt hallucination. This is crucial..

I am talking about the actual content creation where you answer the headings of the outline.
1. When answering the headings in the outline, are you using the chatgpt for each headings, So the response number would be the number of headings? OR you include all headings in the prompt to chatgpt, so the response would be only one?

2. Do you have a separate fine-tuned model for answering each headings Or you just use the same fine-tuning on outline creation?

3. When making the actual article, are you Copying the outline of TOP 1 result ONLY? or you choose or combine the outline from top 1 to 3 result, so your outline becomes superset of those 3 websites? Because if that is the case, without chatgpt, you need to do this manually, not automated.

Thanks

 

[splishsplash]

You might need another class for sparse content. These would be things like Profile Pages, company contact pages, checkout pages, Real Estate Listings, etc. Mostly user generated or some "autogenerated" content that don't really fit into either of informational, services or
Forum pages.

You might find some more classes on
https://schema.org/WebPage
I tried solving exactly something like this last year. Tho, my first attempt was using purely statistical and small DL nlp techniques (LDAs, small RoBerta models etc) and the second was using a huge LLM, with yandex's YaLM model.

Neither gave good results. The LLM one especially was too under-trained (both the base model and my fine-tuning) since it was just so huge, 100 billion parameters
I wouldn't even afford runing a decent run on it.

Looks like small models with chinchilla optimal training like llama are really the best suited for these kind of tasks. The base models are trained on so much more tokens that even the smaller models that are cheap on inference side perform Really well, and fine tuning them makes it even better.

I've blown out my grant tho so can't really play these stuff anymore.
Hope you enjoy this journey

That's great advice, thank-you!

Yeah those models wouldn't cut it. You should have gone for something like neox-1.3b or gptj-6b -- They'd do the job. LLaMA is in another league though.

You don't need much to train lit-llama-7b though. You can train it on a 24GB 3090 if you use FP8. ( I think, you'll need to check. I do bf16 but I rent multiple A100 80GB's so I don't really play with smaller weights )

There's also 4 bit training working for it with GPTQ 4 bit quantization which apparently has performance similar to even an uncompressed fp16 model.

You're then talking being able to run 7b on something ridiculously small like an 8GB to 12GB card

https://twitter.com/x/status/1654917326381228033

- 30b is being fine tuned with 4 bits on a 24GB card, which is just a 3090. You can rent them for $0.30/hr. Or for 7b you can train that on a free google colab.

Or rent something here https://cloud.vast.ai/create/

You can get an 8GB 1070 for $0.058/hr. That's $1.4 per day.


A 12GB RTX A2000 for $0.093/hr - $2.3/day

You got options bro. Don't give up You don't need big bucks to do some cool things with machine learning now. Open source is CRAZY ADVANCED. I'm just doing openai because it's a baseline, and it's FAST. Even 8xA100 80GB's take about 2 hours to finetune with 30k training samples. Which is ok I guess. I haven't done 30k samples on curie yet, but 100 samples took 4 mins, and most of that was waiting time. It probably wouldn't take more than an hour. It's just MUCH easier/quicker though for testing and lets you focus more on the results than the engineering.

I see, so you are just focusing on Getting keywords from a page outline. I thought you are also talking about the actual content creation.

so the format is like this:

Where Title is H2 and, Subtitle is either H3, H4, H5, H6:

{"prompt":"Title 1\nSubtitle 1\nTitle 2\nSubtitle 2","completion":" how to become a notary in Texas\nhow does notary in Texas work\nwhat is notary in Texas....}
{"prompt":"Title 1\nSubtitle 1\nTitle 2\nSubtitle 2","completion":" cryptocurrency\nhow does crypto work\nwhat is cryptocurrency and how does it work....}

I am talking about the actual content creation where you answer the headings of the outline.
1. When answering the headings in the outline, are you using the chatgpt for each headings, So the response number would be the number of headings? OR you include all headings in the prompt to chatgpt, so the response would be only one?

2. Do you have a separate fine-tuned model for answering each headings Or you just use the same fine-tuning on outline creation?

3. When making the actual article, are you Copying the outline of TOP 1 result ONLY? or you choose or combine the outline from top 1 to 3 result, so your outline becomes superset of those 3 websites? Because if that is the case, without chatgpt, you need to do this manually, not automated.

Thanks

You keep talking about chatgpt here..

It has *ZERO*.

Let me say that again..

It has ***ZERO*** to do with anything here.

I am not using chatgpt to generate any data. Forget chatgpt completely for this task. It may as well not exist.

There is no answering either. Nothing is being answered..

We are fine tuning with page outlines and keywords that page ranks for. This is it. Nothing more.

We can then give the finetuned model an outline, and it'll give us back keywords. Simple. There's no chatgpt, no answering, no supersetting of 3 sites. It's just the outline from 1 page that ranks for a main base keyword.


You keep asking me the same questions over and over, and talking about chatgpt when I keep saying it's not using chatgpt.

I think I've answered you now about 4 times.

Plus, I wrote about, what, 5k to 10k words in the original posts.. I couldn't be any more detailed. I've even given you all the code except the code to get an outline for a page, which you can write yourself with bs4.

I've literally given you every single bit of code, with detailed instructions and every single command to make this work.

The ONLY thing you have to do is create your own get_outline.py, and def get_outline(url): function that returns whatever you decide to use as an outline, save that in get_outline.py, replace the path in the dir_path variable - https://share.getcloudapp.com/Apug6RRJ

That's it. You have EVERY OTHER bit of working code to prepare the data. It generates the entire training file, then the commands to fine tune.

I couldn't put this anymore on a plate. It's fully working code and commands.

All you need is a single get_outline() function that returns whatever text you decide to use as your page outline.

 

[DrBitcoin]

You got options bro. Don't give up You don't need big bucks to do some cool things with machine learning now. Open source is CRAZY ADVANCED. I'm just doing openai because it's a baseline, and it's FAST. Even 8xA100 80GB's take about 2 hours to finetune with 30k training samples. Which is ok I guess. I haven't done 30k samples on curie yet, but 100 samples took 4 mins, and most of that was waiting time. It probably wouldn't take more than an hour. It's just MUCH easier/quicker though for testing and lets you focus more on the results than the engineering.

true. there are options. and you can always buy openai accounts for more $$$ savings!

I like your approach, but I like mine as it's more spammy and still gets the results

 

[splishsplash]

true. there are options. and you can always buy openai accounts for more $$$ savings!

I like your approach, but I like mine as it's more spammy and still gets the results

I don't advocate that, but it doesn't work with finetuning. Your model only exists on your account, so you lose it once the account runs dry.

 

[Whisperken]

Following

 

[splishsplash]

Quick Update

Most of today was spent on my new siloing guide which you can read here - https://www.blackhatworld.com/seo/seo-siloing-2023-the-last-siloing-guide-youll-ever-need.1502244/

Did some planning for some future models and AI toys too.

Planned out a tool to grade sites for topical authority. This one is reliant on a finetuned model to get sub-topics from pages on a site, then grade the entire site for each sub-topic so sites topical authority can be compared.

The plan with it is to do relative grading. Ie, it won't work on a site by its self, since you have nothing to compare to. How can you classify topical authority on its own? You can't really. It's always relative.

So if you give it 10 sites, then it'll get the sub-topics from every page, do a total for each site, then grade them relative to each other so you can say "Ok site C has a 100 score persian cat grooming. Ie, it's the strongest, perfect score, 100. So site A might then have a score of 15, site B would have 47 and so on" This way you can compare your site to any other site and get a topical score to see what you lack, and by how much.


HTML Parse Fine Tune

Train MPT-7B to to parse things out of html.

Prompt: Question + raw HTML

Completion: The data that satisfies the question.

This will be a really useful one. You can just give this any html, and ask it, with an instruction to give you data back.

MPT-7B has a context window of 65k so you can feed it HUGE html files unlike even gpt4.( And gpt 4 is way too expensive anyway for this to be practical )

Get broad-topic and sub-topics for page

This one will be used to get the sub-topics for the topical authority tool for pages.

Classify user intent for a keyword

This one will let you give it a keyword and it'll tell you in plan English what the user intent is. Powerful for using before you create an article to make sure the first paragraph in the article solves the user intent quickly. This is how you keep bounce rates down. The user scans the first sentence or 2 and if it doesn't solve, they often hit back. If it does then they read on.

Tomorrow I'll start gathering data for the page classification finetune and hopefully start finetuning it by Wednesday evening at the latest.

 

[Obsidian Knight]

Wow, this is actually really really interesting! I'm following along closely.