- Oct 9, 2013
- 3,471
- 14,472
You wouldnt want 3k related. Thats a bad sample. You want 3k distinctive and you do one fine tuned model per page classification
[Journey] Reverse Engineering Google with AI (Fine-tuning only. Advanced level)
- Thread starter splishsplash
- Start date
You wouldnt want 3k related. Thats a bad sample. You want 3k distinctive and you do one fine tuned model per page classification
codingMinds
Regular Member
- Apr 14, 2019
- 282
- 55
Noob question- Is there any guide I follow to get outline of the page? I am going to start a new website with ai articles and I think if I get keywords following your approach then I can create good articles. BTW the way you are sharing all the details is amazing. Thanks
tragicflaw
Newbie
- Mar 29, 2023
- 16
- 4
Yep, these all make sense. I understand why you train with *real* data from real search results, totally valid points. I only didn't understand how we use this trained model afterwards. Because after all, we train the model to output keywords *conditioned* on real outlines, although good but still requires an outline in inference time as well. So how do we go from keyword to optimized outline is the part I'm missing. Just to test my understanding and make it easier for you reply/explain, let me think of few ways to do it and you tell me if i'm getting it anything that resembles your planWe could also test this using only the top 3, but my gut says top 5 is better because if it's ranking #1 for the main keyword you searched(Which should be a decent keyword, not a micro keyword, because if it's a micro keyword then we probably won't have many other useful keyword rankings. We want lots of longtails, so a more broad, slightly higher comp base keyword will give a page that's more likely to have 100+ keywords in the top 5)
And that's it.
Your training data is then just
OUTLINE
###
keywords
END
That's it.
Real outline, real keywords. Real data.
The next model is the classification model, so that I can then train 10-12 different models for the "get keywords from a page outline". So first I'd get a page, classify it, and then run the appropriate model. This is super important, because the type of keywords that will rank for a tutorial are radically different than the type that would rank for an ecom product page. The model needs to learn only for the page type.
Make sense now?Feel free to ask for clarification.
Before that though, I highly recommend you train(condition) the classifiers and kw generation in a single model. Because model capacity is sufficent to do it and extra information for why a certain page might be a certain class might help with kw generation, at worst it won't do nothing and you can still try seperate models. You could do it like this:on train time, input ={ Outline, classification label} , generation = {top kws} and on inference time, input ={Outline}, generation ={classification_label, kws}. Remember to put classification label after the outline so we can predict it if it's not given during the inference time.
Back to my understanding:
- What we have is kw generator, conditional on outlines
1- So given a keyword and a generated outline from some other model, you can get this outline and predict it's keywords. Given these keywords, you can score how good this outline is by getting their volume data etc. Good approach, but you are still not getting an optimal outline, you can only rank how good an outline is. You can potentially make it iterative based on the score, so good idea if this is it.
2- Given a keyword, we can do a search on it and get top results. Then, we can get its top keywords from ahrefs and with an another model generate an outline. In this scenario, we don't really need kw generator model, since we have access to ground truth from real keywords with ahrefs. I guess we can uncover unused kws with a model,
again if im missing anything, that's probably on me but thanks for your detailed explanations
There is really untapped potential in open source LLMS and making them work for your specific case, while everyone is wrestling with chatgpt, so kudos. If you have a discord group or something you discuss these stuff with people, and would like to invite me as well, I'm down to discussing this stuff more 
KHer0
Elite Member
- Mar 22, 2011
- 2,465
- 2,981
@tragicflaw
This first model isn't for production. Its to test the theory, how good is AI really is at predicting the keywords an article should rank for. Basically, replicating google crawler for the on-page part.
Why? because at first you need something to compare to, to know the quality of your AI.
He simply trained his AI to read an article and guess which keywods it should rank for just like google algorithm does! Now you can test it on actual pages and compare its real ranking to the AI output. This way you can tell how good your AI really is.
Now when you write new article which isn't even published yet, you can make your AI tell you which keywords to target and which keywords you should rank for.
If you use it for page 2 and page 3 as well, then it can tell you related topics which you create new articles about.
Finally, with those new generated keywords, you could ask ChatGPT to create new paragraphs based on the keywords you found. Which should help you rank for more keywords, and increase your traffic by a good margin.
So, from this first model alone, you are able to replicate google crawler, how it sees your article when it visits it, and how it decides which keywords you trying to rank and which are you actually worth ranking for
This first model isn't for production. Its to test the theory, how good is AI really is at predicting the keywords an article should rank for. Basically, replicating google crawler for the on-page part.
Why? because at first you need something to compare to, to know the quality of your AI.
He simply trained his AI to read an article and guess which keywods it should rank for just like google algorithm does! Now you can test it on actual pages and compare its real ranking to the AI output. This way you can tell how good your AI really is.
Now when you write new article which isn't even published yet, you can make your AI tell you which keywords to target and which keywords you should rank for.
If you use it for page 2 and page 3 as well, then it can tell you related topics which you create new articles about.
Finally, with those new generated keywords, you could ask ChatGPT to create new paragraphs based on the keywords you found. Which should help you rank for more keywords, and increase your traffic by a good margin.
So, from this first model alone, you are able to replicate google crawler, how it sees your article when it visits it, and how it decides which keywords you trying to rank and which are you actually worth ranking for
- Oct 9, 2013
- 3,471
- 14,472
I already gave away the outline structure, so fuck it, I'll give you guys the code. It's nothing revolutionary. I just can't give away the code and exact training format for everything I do otherwise I will be equipping competitors. It's not the individuals doing stuff at home for their own sites I'm concerned about. It's companies and people with big money who'll compete with my future AI SaaS products. My tech just now is *way* ahead of everyone else in the marketing space. That's mostly because there are no hardcore devs in this space, and it's almost impossible to get real AI engineers.
You now have complete code. Follow the instructions in the main posts and here's your get_outline() function
I find myself dropping in here and consuming everything you write way to often. Can't imagine how long it takes for you to write it! (If it's not a fine tuned models of your own persona
Not long, I can type fast
Long posts take me 5-10 mins to write. Really long ones 20-25 mins. I can write about 1000 words in 10 minutes. Actual typing speed is 120-170 wpm depending on what I'm writing.For this, we'll train with
Just to test my understanding and make it easier for you reply/explain, let me think of few ways to do it and you tell me if i'm getting it anything that resembles your plan
on train time, input ={ Outline, classification label} , generation = {top kws} and on inference time, input ={Outline}, generation ={classification_label, kws}. Remember to put classification label after the outline so we can predict it if it's not given during the inference time.
Back to my understanding:
- What we have is kw generator, conditional on outlines
1- So given a keyword and a generated outline from some other model, you can get this outline and predict it's keywords. Given these keywords, you can score how good this outline is by getting their volume data etc. Good approach, but you are still not getting an optimal outline, you can only rank how good an outline is. You can potentially make it iterative based on the score, so good idea if this is it.
2- Given a keyword, we can do a search on it and get top results. Then, we can get its top keywords from ahrefs and with an another model generate an outline. In this scenario, we don't really need kw generator model, since we have access to ground truth from real keywords with ahrefs. I guess we can uncover unused kws with a model,
again if im missing anything, that's probably on me but thanks for your detailed explanations
Sometimes it's better to give more varied data and sometimes it's not.. In general finetuning is REALLY different to how we interact with an instruct trained model like chatgpt.
chatgpt is trained to recognize patterns in questions and answers. It's very broad and what you're training for is helping it to ACCESS the meat and bones in the LLM through instructions/prompts. You're helping it make sense of all the data, knowledge and reasoning capabilities within the model.
This is VERY different to a problem where you have external data that you want to understand.
You have to remember that our SEO data is not contained within the model. We are training it to pattern match.
In cases where our goal IS a classification model, ie, give the page type, ecom category, ecom product, info etc, it's beneficial to train one model with examples of each type because it will make it easier for the model to learn what type X is by seeing examples of non-type X's.
The output of these models is just a class. It's 1-2 words. If there's 12 classes, then there are only 12 options it needs to choose from.
Now, if you've got a model you're training to produce OUTLINES, from a single keyword. Always a single, never try to train this with multiple keywords. It's too much data for the model. Finetuning successfully is about feeding it higher quality data that make the patterns more obvious.
If you give it huge inputs and want huge outputs you're going to struggle with pattern recognition. Remember this isn't like training chatgpt for instructions. chatgpt is not doing pattern recognition on data, it's learning to recognize patterns of what good answers look like to help it access its own internal abilities. This is so different than what we're doing.
The more complex the data, the more samples you need.
It's vastly easier to give 1000 examples of a core keyword, and an outline and learn what outlines look for the core keyword.
And we don't need to give extra keywords, there's no need. If you have 5 pages ranking for "how to do content marketing", then they will have a lot of overlap with certain keywords. Each will have a group of longtails that only they rank for, but what we're looking for the model to find is the primary components that all outlines have in common for a particular class. We don't want to include the outlier components of a page for this type of model.
You could take that outline generated, and then pass it into the reverse model(The one we are training now), and it'll give you a huge range of keywords that an outline like that could rank for. NOW you have your huge range of outlier longtails and can tweak that outline.
So in essence..(And you've helped me discover this which is why I love this thread. I learn way way more when I explain things to people. That's one of the main reasons I do this. It's like a chain of thought prompt where I get to analyse my reasoning)
In order to create the best outline, we will have BOTH models.
First the one we've not trained yet, with the input: keyword, output: outline
Then we take the output of that, put it into the input of the original model in my first posts and get a range of keywords back to enhance the outline with.
Also to add, if we train that model with multiple keywords, instead of 1, then we end up with superfluous data.
Does it matter if the article that's #1 for "how to write facebook ads" also ranks for "examples of a good facebook ad" ?
Does giving that keyword and others help? Or just confuse the model with too much data and make it hard to narrow in on the pattern.
I'm not saying the model couldn't handle it. It probably could, but you might end up needing 1 million training examples to train something like this.
Maybe you might even be better training from scratch and just having a model that only understands language from the perspective of keywords and outlines.
Technically you don't need to first train a model on normal language. You could just train it purely on SEO data. It will still learn language, but just in a different, non-human way. The problem with that is cost. To train something like llama-7b from scratch, if you do it with 1 trillion tokens of SEO data, you'll need 2048 A100's for 21 days.
That would cost $2,281,144.32 at coreweave.com's prices of $2.21/hr per 80GB A100. That's a lot, especially to test a theory
If anyone wants to test that, feel free to wire me over $2.5 mil and we'll make it happen ;-)
You'd also have to get 1 trillion tokens worth of ranking data.
I'm pretty sure if you did that you'd have reverse engineered Google mind you
The only problem is you'd need to train a new model every 6 months to a year because they update a lot.
The other big problem with that is with a model like llama has a context window size of 2048. You could in theory increase it to 32k, but the computation required would be silly given its attention window mechanism would make it prohibitive.
You would need to be able to include all information about a site in 1 training sample otherwise you wouldn't have a model that can predict ranking. Just one that has a sort of language understanding.
Ie, you'd need to include full data in each training sample, like full article, keywords it ranks for, positions, outbound links, inbound links and data on each link. Then you'd be able to do accurate predictions about something's ranking potential.
Yep this is spot on!
tragicflaw
Newbie
- Mar 29, 2023
- 16
- 4
Yep, gotcha. That's what I was trying to clarify.
On seperate classifier model topic, I'd still argue for a single model. In my experience, joint training of relevant objectives is almost always superior to seperate models. (i.e. joint training of LLMs on both code + language is superior to training only on a single of them, both on coding and language benchmarks. Or Mask-RCNN model gets better if you jointly train classifier+detector+pixel segmentations). If the objectives are totally not relevant(surprisingly hardly the case), the model learns to seperate the objectives in the network at the last layers. It only becomes a problem with tiny networks due to capacity. Up to you, just my 2 cents
reading your post made me think of an idea, which can bring everything together
- What is the objective here? To write articles that rank well in google. To do that, we try to mimic the google output by training a model that outputs the google keywords that rank well(measured by ahrefs). It's essentially proxy objective, but we know by experience and intuition that it's a good one. i.e. whatever score google assigns to our articles, it correlates strongly with this keyword scoring. let this google scoring function be G(x) and our scoring function F(outline). We assume with previous intuiton that they are very close.
- What is the objective in Chatgpt, or chat models? To produce text with given prompt such that, it produces human prefered text. Let human prefered text be the function F(prompt) and our model G(prompt). F(prompt) is essentially a black box, who the hell knows what's going on in there. But we want to learn to mimic it. We get around this by mimicing a proxy metric, human rank preference data from data labelers. It turns out, if we mimic this, we end up with a function that resembles F(prompt) as well.
It's essentially the same thing. Both objectives are the same, in the sense that we have no clue what they really are but we can only get close to them by mimicking a proxy.
So to close the loop completely, and to go directly from a keyword -> google optimized outline, I suggest the following strategy(which will turn out to be the same as RLHF or chatgpt, but automated in our case
)Given an outline, have a scoring model(reward model in RLHF). We already have this, from an otuline we can generate keywords and calculate how many of them are good, thus end up with a final score. let it be a float between 0 and 1.
I don't think we need to do RLHF or PPOs necessarily, so I'll go with the simpler supervised version. Note that that migh be necessary as discussed in here: .
For every training example, for given keyword, have LLM generate an output. Then calculate how good this outline is with the current model. Calculate loss between them as MSE(1,calculated_score), backprop this loss in the LLM. Repeat this for all examples, hopefully we get a model that tries to get a maximum 1 score from the scoring model, which in turn would optimize our outline
hope this helps and i might try a similar thing myself. Let me know if you try out as well. It really would be a final model
DrBitcoin
Failed to resolve dispute
- Mar 27, 2023
- 551
- 391
what are you referring to by "distinctive" keywords?
by 3k related keywords, im referring to keywords like:
how to lose weight
how to reduce fat
Not sure what you mean by distinctive keywords
- Oct 9, 2013
- 3,471
- 14,472
Yep, gotcha. That's what I was trying to clarify.
On seperate classifier model topic, I'd still argue for a single model. In my experience, joint training of relevant objectives is almost always superior to seperate models. (i.e. joint training of LLMs on both code + language is superior to training only on a single of them, both on coding and language benchmarks. Or Mask-RCNN model gets better if you jointly train classifier+detector+pixel segmentations). If the objectives are totally not relevant(surprisingly hardly the case), the model learns to seperate the objectives in the network at the last layers. It only becomes a problem with tiny networks due to capacity. Up to you, just my 2 cents
reading your post made me think of an idea, which can bring everything together
Hmm.
You're right here with this helping with broader LLM training. Maybe I'm on the wrong track even with finetuning and it would be better to do just 1 model with all the classes. In fact, I think you're definitely right, because it should in fact help to teach the model variances between the classes and give it a better understanding of each class from that. They do work in mysterious ways we don't understand.
- What is the objective here? To write articles that rank well in google. To do that, we try to mimic the google output by training a model that outputs the google keywords that rank well(measured by ahrefs). It's essentially proxy objective, but we know by experience and intuition that it's a good one. i.e. whatever score google assigns to our articles, it correlates strongly with this keyword scoring. let this google scoring function be G(x) and our scoring function F(outline). We assume with previous intuiton that they are very close.
- What is the objective in Chatgpt, or chat models? To produce text with given prompt such that, it produces human prefered text. Let human prefered text be the function F(prompt) and our model G(prompt). F(prompt) is essentially a black box, who the hell knows what's going on in there. But we want to learn to mimic it. We get around this by mimicing a proxy metric, human rank preference data from data labelers. It turns out, if we mimic this, we end up with a function that resembles F(prompt) as well.
It's essentially the same thing. Both objectives are the same, in the sense that we have no clue what they really are but we can only get close to them by mimicking a proxy.
So to close the loop completely, and to go directly from a keyword -> google optimized outline, I suggest the following strategy(which will turn out to be the same as RLHF or chatgpt, but automated in our case
Given an outline, have a scoring model(reward model in RLHF). We already have this, from an otuline we can generate keywords and calculate how many of them are good, thus end up with a final score. let it be a float between 0 and 1.
I don't think we need to do RLHF or PPOs necessarily, so I'll go with the simpler supervised version. Note that that migh be necessary as discussed in here: .
For every training example, for given keyword, have LLM generate an output. Then calculate how good this outline is with the current model. Calculate loss between them as MSE(1,calculated_score), backprop this loss in the LLM. Repeat this for all examples, hopefully we get a model that tries to get a maximum 1 score from the scoring model, which in turn would optimize our outline
hope this helps and i might try a similar thing myself. Let me know if you try out as well. It really would be a final model
This is gold.
In the same way that davinci-001 was trained on instruct series prompts, and then what was generated from those instruct prompts was then used as a proxy object to further finetune to bridge between G(x) and F(x) where G(x) is optimal output for a human, and F(x) is what the instruct model produced..
This can be applied to every model we finetune in the SEO space.
Holy shit.. You've just given me a way to make my AI writer go from above human level to god-tier.. This is pretty hard to implement, so I doubt any other AI writer companies in the space will even attempt this. My ultimate goal anyway is to create 10,000+ ultra high quality AI sites and dominate the low-med niche space. To do that there's a LOT of components since it all needs to be 100% automated. My SaaS businesses are only side projects. The real money is going to be dominating the space these final 5 years or so while SEO still works, and also capture users in some other form so if the sites eventually stop receiving traffic I still have users.
We should keep the research open anyway. There's more benefit with open research than closed. Even OpenAI are starting to get crushed by the innovation coming from the open source community. The above is an example. I don't think I would have thought of what you did, and you might not have thought about it without my input. Maybe there's other people out there that will add to the insight and it'll get somewhere even better the more that's shared. It's not like 1000 people are going to suddenly be able to implement a god-tier AI site creation system and create 100's of thousands of AI sites
I agree "distinctive" isn't a very accurate word in the context.
I'm using distinctive to mean choosing keywords that give you a broad sample that's representative of as many different types of keywords as possible.
So you definitely wouldn't want 3k related keywords. That wouldn't be useful.
It would be like trying to train a vision model to recognize animals, and only giving it pictures of black male Siamese cats.
We want a training sample that represents the population(statistics term population)
Ie, you want "how to lose weight", "how the stock market works", "how to groom your dog", "why should you lift weights", "guide to running a marathon"..
You want to in fact exclude keywords that are too similar.
Remember, 3k keywords is for 3k training samples. It's not 3k keywords in 1 training sample.
myonletter
Registered Member
- Nov 26, 2020
- 51
- 20
Great seeing you attempt to reverse-engineer Google's search blackbox with LLMs.
- Oct 9, 2013
- 3,471
- 14,472
Update
Facing some challenges with gathering the training data for the classification model.
Unfortunately gpt4 isn't able to classify reliably with my now, 25 classes, and there's no way I'm going to do 15k manually.
I was using this
That's the url, first few hundred words from <p>'s and the outline.
It gets about 70% correct, but that's no use. I need 99% correct otherwise the training data is completely fucking useless. Training data is EVERYTHING. Your fine tuned model is only as good as the training data.
gpt3.5 is even worse. It gets about 60% correct. It depends on the page.. For things like category pages/info/tutorial both get them right 90% of the time, but even 90% isn't good enough, and what I need to be able to do here is give the training data generator like 2000 search terms and have it scrape and categorize the top 10 from each. I'm also using gpt4 to give me keywords that are most likely to give me certain classes, which it's fairly good at. This guarantees I get the right pages..
I've come up with a couple of solutions
The first is instead of giving gpt4 multiple classes I'll just ask it to classify 1 class and give it a couple of examples. I'll tell it to say "I DONT KNOW" if it doesn't know. This means I at least won't end up with false positives which is the danger.
The downside to this alone is I still miss out on the more unique templates that I would like within the training data.
I'm limited to a prompt size of 2048 tokens with curie for fine tuning(Even davinci for fine tuning since its davinci-001. It's 2048). This severely limits my options.
So my second part of the solution is I'm going to add in extra summarized data about a web page.
So far this is what I've come up with, but I'm adding more data points.
(Excuse the code, it's not particularly nice design wise. I wouldn't write code like this for a production system that needs to be maintained, but for these quick data gathering programs time is of the essence!)
Code in the next post. Can't fit it all into 1 post
Facing some challenges with gathering the training data for the classification model.
Unfortunately gpt4 isn't able to classify reliably with my now, 25 classes, and there's no way I'm going to do 15k manually.
I was using this
I will give you the outline of a webpage.
I want them classified into one of the following categories:
info - informational style article
tutorial - guide/tutorial
ecom cat - ecommerce category page
ecom product - single ecommerce product page
best X - a best X type page
reviews top 10 - X reviews, top 10 X. Different than best which is more best 2-4 products
single product review - a review of just 1 product
news - news article. Reporting on current events in the world.
faq - a faq
forum - a forum post
service - a service being sold
recipe - a cooking recipe
homepage - a homepage that doesn't fall into another category
blog cat - a blog category/silo/tag page
directory - a directory page/list of links
profile - a profile link, business or person
gallery - A page with only images, or 80-90% images. A gallery
contact - A contact page
about - An about page. About a company/product
careers - a page with jobs
team - a page with team members for a company
video - A page with videos or a single video
legal - legal documents like privacy policy etc
portfolio - A portfolio page
Affiliate page
paa - People also ask page
Also write out your confidence score out of 10 that scores how confident you are that the category is correct. If you are ABSOLUTELY CERTAIN, then score 10, if you are certain, and there's a tiny chance you might be wrong, score 9, if you are confident it's correct, but there's a slight chance you're wrong, score 7 or 8. If you are fairly confident, but there's a not insignificant chance you are wrong, then score it 5 to 6. If you are not quite sure and making a guess you feel is a good guess, score it 3 to 4. If you have no confidence in your guess and feel it's essentially like rolling a dice, then score it 1 to 2.
Don't explain, just give a category and confidence. Examples of results:
info:8
blog cat:9
Here's the outline:
url: https://www.searchlogistics.com/learn/seo/how-search-engines-work/
sample paragraph content from page: Home>Learn>SEO>How Do Search Engines Work?Search engines work by simply crawling billions of pagesusing the web crawlers they have developed. These are commonly referred to assearchengine spidersorbots.A search engines spider thennavigates the web by following links on a new web page it discoversto find new pages and so forth.This is an important piece of knowledge that many new SEOs miss out on:But understanding how search engines work isparamount!Why?Because you need to know how the system works in order to try and leverage it!You can’t fix a car’s engine problem without knowing what’s going on under the hood…… and thesame rules applyfor all search engines.But, you don’t need to knoweverythingabout search engine algorithms either.I’m going to take you through how search engines work step by step. Let’s start with the search engineessentialsto lay the foundation for a successfulSEOcareer.What Will I Learn?Google’s search engine works around these two main functions:We will be looking at these in more detail in a moment.
outline of page:
title:How Do Search Engines Work In 2022? What You Need To Know
h5:Getting Started
h3:SEO Checklist: 45x Ways To Increase Your Search Traffic
h3:What Is SEO And Why You MUST Pay Attention To It!
h3:How Many Backlinks Do I Need To Rank?
h3:How Do Search Engines Work? What You Need To Know
h3:These Are The Best SEO Tools That Money Can Buy
h3:27 Google Tools You Should Know About
h5:SEO Case Studies
h3:From Google Penalty To 5x Search Traffic
h3:Growing Revenue $17,122 To $92,119 Per Month
h3:How To Triple Ecommerce Revenue
h3:+73% Search Traffic With These 6 Fixes
h3:Ecommerce SEO Case Study: 3x Revenue In 90 Days
h3:How To 14x Search Traffic In 8x months
h5:Keyword Research
h3
h3:These 7x Types Of Keywords Will Increase Your Search Traffic
h3:How To Use Buyer Keywords To Boost Your Sales
h3:Keyword Research: The What, Why & How
h3:The Best Keyword Research Tools (And How To Use Them)
h3:Google Keyword Planner: How To Use It The Right Way
h5:Content Creation
h3:How To Write Killer Website Content That Attracts People
h3:18 SEO Copywriting Hacks That Get Instant Results
h3:How To Write A Listicle That Attracts Traffic & Backlinks
h3:How To Increase Search Visibility With FAQ Schema
h5:On Page SEO
h3:What Is On Page SEO? How To Perfectly Optimise Your Page
h3:6x Free Ways To Increase Website Speed (and search traffic!)
h3:The Ultimate On Page SEO Checklist To Increase Rankings
h3:The Hidden SEO Ranking Factor You’re Probably Deleting
h3:How To Increase Search Visibility With FAQ Schema
h3:3x Ways A Silo Structure Will Boost Your Search Traffic
h5:Link Building
h3:15x Incredible Link Building Strategies
h3:Link Building Services: What You REALLY Need To Know
h3:Backlink Analysis – The Easiest Ways To Build Links
h3
h3:8x Ways To Build Powerful Edu Backlinks
h3:Testimonial Link Building: Powerful Homepage Links
h5
h3:7 Google Penalty Checker Tools
h3:The Step By Step Process To Recover From Any Penalty
h3:Google Page Experience Update Checklist
h3
h3:How To Get Out Of Google Sandbox As Quickly As Possible
h3:How To Improve Your Core Web Vitals In 15 Minutes
h5:General SEO
h3:How To Exclude Words From Google Search Results
h3:SEO Checklist: 45x Ways To Increase Your Search Traffic
h3:Testing The 12 Fastest WordPress Hosting Providers
h3:Learn How To Find Powerful Expired Domains Step By Step
h3:The Fastest & Easiest Way To Get Wikipedia Backlinks
h3:The Ultimate Guide To Search Engine Submission
h5:SEO Tools & Reviews
h3:SEMRush Review: How To Increase Search Traffic
h3:Surfer SEO Review - How I Used It To Win The #1 Position
h3:NitroPack Review & Case Study
h3:WPX Hosting Review – Faster Than You Might Think
h3:My Kinsta Hosting Review & Case Study
h3:SEO Powersuite Review: 24x Ways To Increase Search Traffic
h1:How Do Search Engines Work In 2023? What You Need To Know
h5:4 Ways To Increase Your Search Traffic
h6:A TRUSTWORTHY Link Building Strategy Starts With Us....
h2:How Google Works & How They Rank Your Page
h5:Crawling and Indexing
h3:What Is Crawling?
h5:Why does Google do this?
h4:How Does A Crawler Work?
h3:What Is Indexing?
h3:Why Do Some Pages Show Up Higher Than Others?
h2:How Other Search Engines Work And How They Differ
h3:How Amazon Works
h2:What Do Search Engines Want?
h2:Wrapping It Up
h2:You Might Also Like
h4:Speed Up Magento 2 With These 10x Practical Tips
h4:3 Ways To Check Domain Ownership History Easily
h4:Free SEO Tools That Are Actually Worth Using
h4:Google’s New SEO Starter Guide: What You Need To Know
h2:What Are Your Thoughts?
h3:26 Responses
h2:Leave a ReplyCancel reply
h3:My Free Link Building TrainingWill Show You How
That's the url, first few hundred words from <p>'s and the outline.
It gets about 70% correct, but that's no use. I need 99% correct otherwise the training data is completely fucking useless. Training data is EVERYTHING. Your fine tuned model is only as good as the training data.
gpt3.5 is even worse. It gets about 60% correct. It depends on the page.. For things like category pages/info/tutorial both get them right 90% of the time, but even 90% isn't good enough, and what I need to be able to do here is give the training data generator like 2000 search terms and have it scrape and categorize the top 10 from each. I'm also using gpt4 to give me keywords that are most likely to give me certain classes, which it's fairly good at. This guarantees I get the right pages..
I've come up with a couple of solutions
The first is instead of giving gpt4 multiple classes I'll just ask it to classify 1 class and give it a couple of examples. I'll tell it to say "I DONT KNOW" if it doesn't know. This means I at least won't end up with false positives which is the danger.
The downside to this alone is I still miss out on the more unique templates that I would like within the training data.
I'm limited to a prompt size of 2048 tokens with curie for fine tuning(Even davinci for fine tuning since its davinci-001. It's 2048). This severely limits my options.
So my second part of the solution is I'm going to add in extra summarized data about a web page.
So far this is what I've come up with, but I'm adding more data points.
(Excuse the code, it's not particularly nice design wise. I wouldn't write code like this for a production system that needs to be maintained, but for these quick data gathering programs time is of the essence!)
Code in the next post. Can't fit it all into 1 post
- Oct 9, 2013
- 3,471
- 14,472
To install the necessary libs, save the below as requirements.txt then run
pip3 install -r requirements.txt
- Oct 9, 2013
- 3,471
- 14,472
This is easier. Formatting is lost on bhw which is death for python - https://github.com/tbelfort/ai-seo-tools/blob/main/get_structure_for_classify_webpage.py
And here's the code that builds the training data with chatgpt3.5 or gpt4 - https://github.com/tbelfort/ai-seo-tools/blob/main/gather_page_classes_with_chatgpt.py
and this is the google_search.py that it imports for scraping google. You need a serper.dev API key for this - https://github.com/tbelfort/ai-seo-tools/blob/main/google_search.py
And here's the code that builds the training data with chatgpt3.5 or gpt4 - https://github.com/tbelfort/ai-seo-tools/blob/main/gather_page_classes_with_chatgpt.py
and this is the google_search.py that it imports for scraping google. You need a serper.dev API key for this - https://github.com/tbelfort/ai-seo-tools/blob/main/google_search.py
Last edited:
- Oct 9, 2013
- 3,471
- 14,472
Pretty much done, but I've had one final idea which could be absolutely killer.
I'm going to reduce a page's html to shorthand like this
<div>
<p><img></img><bold></bold></p>
<p></p>
</div>
<div>
</div>
<div>
<img></img>
<img></img>
<img></img>
<img></img>
<p></p>
</div>
<p></p>
<p></p>
<img></img>
<img></img>
we get a string with :-
div(p(img,bold),p)
div
div(img,img,img,img,p)
p
p
img
img
Reduce to shorthand
div(p(img,bold),p),div,div(img{4},p),p{2},img{2}
This is ridiculously hard to code though. I need to code my own recursive html parser to do this :-/
Once that's done then we can finally start trying to gather the training data for this.
I'm going to reduce a page's html to shorthand like this
<div>
<p><img></img><bold></bold></p>
<p></p>
</div>
<div>
</div>
<div>
<img></img>
<img></img>
<img></img>
<img></img>
<p></p>
</div>
<p></p>
<p></p>
<img></img>
<img></img>
we get a string with :-
div(p(img,bold),p)
div
div(img,img,img,img,p)
p
p
img
img
Reduce to shorthand
div(p(img,bold),p),div,div(img{4},p),p{2},img{2}
This is ridiculously hard to code though. I need to code my own recursive html parser to do this :-/
Once that's done then we can finally start trying to gather the training data for this.
- Oct 9, 2013
- 3,471
- 14,472
Parser done and working!
Produces the exact template in the above post.
I'm not going to shorten img,img,img,img to img{4}. Probably better for the model to see it as img,img,img,img
Just going to add a few more tweaks to the get_structure_for_classify_webpage.py then I'll publish it so anyone can use it as a base for anything they like. It's pretty powerful and gives you a ton of parsed info about a web page you can play around with and use for your own fine tunings.
Here's the parser bit for the template though if anyone's interested to see the code for it You can use this code as a base for any sort of tree based parser.
def html_body_shorthand(html):
# Parse HTML data
soup = BeautifulSoup(html, 'html.parser')
# Replace <p> tags with the appropriate heading tags
for i in range(1, 7):
for p_tag in soup.find_all('p'):
_replace_p_with_heading(p_tag, i)
content_tree = { "_parent": {}, "_children": [], "_name": "body" }
current_node = content_tree
shorthand_tags = soup.body.find_all(lambda tag: tag.name != "script", recursive=False)
for tag in shorthand_tags:
_build_parse_tree(current_node, tag)
template_string = _traverse_parse_tree_build_string(current_node)
print(template_string)
exit(0)
def _replace_p_with_heading(tag, heading_level):
p_class = tag.get('class')
if p_class and any(h_class in p_class for h_class in [f'h{heading_level}', f'heading{heading_level}', f'header{heading_level}', f'header-{heading_level}', f'heading-{heading_level}']):
tag.name = f'h{heading_level}'
tag.attrs = {}
def _traverse_parse_tree_build_string(node):
string = ""
for child_node in node["_children"]:
string += child_node["_name"]
if len(child_node["_children"]) > 0:
string += "("
string += _traverse_parse_tree_build_string(child_node)
string += "),"
else:
string += ","
return string.rstrip(",")
def _build_parse_tree(node, tag):
node["_children"].append({ "_parent": node, "_children": [], "_name": tag.name})
if tag.children:
for child_tag in tag.children:
if child_tag.name is None:
continue
else:
_build_parse_tree(node["_children"][-1], child_tag)
Produces the exact template in the above post.
I'm not going to shorten img,img,img,img to img{4}. Probably better for the model to see it as img,img,img,img
Just going to add a few more tweaks to the get_structure_for_classify_webpage.py then I'll publish it so anyone can use it as a base for anything they like. It's pretty powerful and gives you a ton of parsed info about a web page you can play around with and use for your own fine tunings.
Here's the parser bit for the template though if anyone's interested to see the code for it
You can use this code as a base for any sort of tree based parser.def html_body_shorthand(html):
# Parse HTML data
soup = BeautifulSoup(html, 'html.parser')
# Replace <p> tags with the appropriate heading tags
for i in range(1, 7):
for p_tag in soup.find_all('p'):
_replace_p_with_heading(p_tag, i)
content_tree = { "_parent": {}, "_children": [], "_name": "body" }
current_node = content_tree
shorthand_tags = soup.body.find_all(lambda tag: tag.name != "script", recursive=False)
for tag in shorthand_tags:
_build_parse_tree(current_node, tag)
template_string = _traverse_parse_tree_build_string(current_node)
print(template_string)
exit(0)
def _replace_p_with_heading(tag, heading_level):
p_class = tag.get('class')
if p_class and any(h_class in p_class for h_class in [f'h{heading_level}', f'heading{heading_level}', f'header{heading_level}', f'header-{heading_level}', f'heading-{heading_level}']):
tag.name = f'h{heading_level}'
tag.attrs = {}
def _traverse_parse_tree_build_string(node):
string = ""
for child_node in node["_children"]:
string += child_node["_name"]
if len(child_node["_children"]) > 0:
string += "("
string += _traverse_parse_tree_build_string(child_node)
string += "),"
else:
string += ","
return string.rstrip(",")
def _build_parse_tree(node, tag):
node["_children"].append({ "_parent": node, "_children": [], "_name": tag.name})
if tag.children:
for child_tag in tag.children:
if child_tag.name is None:
continue
else:
_build_parse_tree(node["_children"][-1], child_tag)
- Oct 9, 2013
- 3,471
- 14,472
Now I've ran it on wolfofblogstreet.com/scraper-test-11 which is a much more complex page.
Output is
python get_structure_for_classify_webpage.py https://wolfofblogstreet.com/scraper-test-11
https://wolfofblogstreet.com/scraper-test-11/a,div(div(section(div(div(div(div(div(div(button(span,span,span),div(ul(li(a),li(a),li(a),li(a),li(a)),div(div(a(img)),button)),div)))))))),div(div(div(main(div(section(div(div(div(div(div(a(img))),section(div(div(div(div(div(p)))))),section(div(div(div(div(div(div(a(span(span))))))))),section(div(div(div(div(div(h1)),div(div(h2,p,ul(li,li),p,p,h3,p,h3,p,h2,p(a),p(i,a),p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p)))))),section(div(div(div(div(div(h2)),div(div(div(div(div(div(div,div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a))))),div(div(div(a(img)),div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a))))),div(div(div(a(img)),div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a)))))),div))))))),section(div(div(div(div(div(h2)),div(div(div(div(img),div(div(h4),div))))))))))))))),script,script)),div(section(div(div(div(section(div(div(div(div(div(a(img))))))),section(div(div(div(div(div(div(div(div(ul(li(a),li(a),li(a),li(a))))))))),div(div(div(div(h3)),div(div(ul(li(a(span(i),span)),li(a(span(svg(style,g(path,path,path,path))),span))))))),div(div(div(div(ul(li(a(span(i),span)),li(a(span(i),span)),li(a(span(i),span))))))))),section(div(div(div(div(div(p)))))))))))),link,span,svg,svg
If anyone wants to help feel free to check that against the source of the page and see if you can find a mistake. I'm looking now! Need to confirm it's valid before deploying and gathering training data
Output is
python get_structure_for_classify_webpage.py https://wolfofblogstreet.com/scraper-test-11
https://wolfofblogstreet.com/scraper-test-11/a,div(div(section(div(div(div(div(div(div(button(span,span,span),div(ul(li(a),li(a),li(a),li(a),li(a)),div(div(a(img)),button)),div)))))))),div(div(div(main(div(section(div(div(div(div(div(a(img))),section(div(div(div(div(div(p)))))),section(div(div(div(div(div(div(a(span(span))))))))),section(div(div(div(div(div(h1)),div(div(h2,p,ul(li,li),p,p,h3,p,h3,p,h2,p(a),p(i,a),p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p,p)))))),section(div(div(div(div(div(h2)),div(div(div(div(div(div(div,div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a))))),div(div(div(a(img)),div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a))))),div(div(div(a(img)),div(h2(a),div(span(a),span(span),div,span(svg(path,path)),span(a)))))),div))))))),section(div(div(div(div(div(h2)),div(div(div(div(img),div(div(h4),div))))))))))))))),script,script)),div(section(div(div(div(section(div(div(div(div(div(a(img))))))),section(div(div(div(div(div(div(div(div(ul(li(a),li(a),li(a),li(a))))))))),div(div(div(div(h3)),div(div(ul(li(a(span(i),span)),li(a(span(svg(style,g(path,path,path,path))),span))))))),div(div(div(div(ul(li(a(span(i),span)),li(a(span(i),span)),li(a(span(i),span))))))))),section(div(div(div(div(div(p)))))))))))),link,span,svg,svg
If anyone wants to help feel free to check that against the source of the page and see if you can find a mistake. I'm looking now! Need to confirm it's valid before deploying and gathering training data
Kirito Kun
Junior Member
- Jun 16, 2022
- 125
- 119
Hi @splishsplash , I understand maybe 50% of what you write, but I am making an effort to learn.
Your posts are the most appreciated by me on BHW. There are many here who know exactly what they are doing , but in my opinion none can hold a candle to you.
I just want to say a few kind words because I feel your journey gets too little attention for what you have accomplished here.
Thank you for your tireless and detailed updates
Your posts are the most appreciated by me on BHW. There are many here who know exactly what they are doing , but in my opinion none can hold a candle to you.
I just want to say a few kind words because I feel your journey gets too little attention for what you have accomplished here.
Thank you for your tireless and detailed updates
CryptoGuys
Registered Member
- Apr 17, 2023
- 54
- 45
ChatGPT works great! and your public PBN will be dead in a few month! get ready man , AI is eating the world, you're dead and you can't do shit about it! Look for a new job before it's too late!
- Oct 9, 2013
- 3,471
- 14,472
Tell me more about this ChatGPT. I haven't heard of it..
- Mar 8, 2009
- 3,247
- 12,747
What is the point of posting this? I don't understand some of you who are so negative and can't stand someone doing well in their life. Please, keep the abusive and negative comments to yourself, as this is one of the more interesting Journeys I have seen. Again, there is no reason to put this type of comment in this thread, as it's filled with anger.
Let's be a community and support one another as more people will post journeys like this, and if we don't, most will keep it to themselves, and we will not learn a thing from it.