[Journey] 1 million UVs/month in 12 months using AI generated content. Let's do it!

[RowdySadjit]

I'm getting a bit stuck on automation of the content scraping part.

I can easily find a couple of low comp keywords to target but when I put them into search google all I get is a load of varied results with no clear uniformity so I have no idea how I will go about programming that?

I am having to pick through manually with a fine toothcomb skimming through each page on the results page for a keyword and looking through the article for a relevant bit of text here or there. Still haven't found any clear method to the madness as yet. Since they are underserved terms there are just scrappy little bits and pieces from youtube results, reddit answers, forum posts or educational pdfs and suchlike.

I have installed the newspaper3k library but results in serps are more often not even articles and when I just tried to load a url into it just now it wasn't even able to pick up any content as it must be in a different format than it is capable of reading (this was for some educational/university course url). Isn't it a catch22 that if you have a bunch of neat articles that you could scrape then the keyword phrase will not be low competition so other marketers are already there? So if there are a bunch of good articles we can scrape it means the keyword is already competitive, but if it isn't it means there is no good info to scrape since it doesn't exist yet on google.

Having trouble with this part. Not sure how to solve.

 

[Alexion]

I'm getting a bit stuck on automation of the content scraping part.

I can easily find a couple of low comp keywords to target but when I put them into search google all I get is a load of varied results with no clear uniformity so I have no idea how I will go about programming that?

I am having to pick through manually with a fine toothcomb skimming through each page on the results page for a keyword and looking through the article for a relevant bit of text here or there. Still haven't found any clear method to the madness as yet. Since they are underserved terms there are just scrappy little bits and pieces from youtube results, reddit answers, forum posts or educational pdfs and suchlike.

I have installed the newspaper3k library but results in serps are more often not even articles and when I just tried to load a url into it just now it wasn't even able to pick up any content as it must be in a different format than it is capable of reading (this was for some educational/university course url). Isn't it a catch22 that if you have a bunch of neat articles that you could scrape then the keyword phrase will not be low competition so other marketers are already there? So if there are a bunch of good articles we can scrape it means the keyword is already competitive, but if it isn't it means there is no good info to scrape since it doesn't exist yet on google.

Having trouble with this part. Not sure how to solve.

I think you overestimate what low comp keywords mean in these AI cases.

The kind of keywords you are finding, are the ones you actually have a shot at for a money site where you actually research and write the articles.

Whereas, the low comp keywords the guys are using are not THAT low, where the first page is already populated. You just ride out the traffic, and get SOME part of it. In volume, that translates to traffic. That's why volume is important.

If i were to pick any tech blog with 3,000 articles, and start rewriting everything by hand, after some time i will eventually steal SOME of that website's traffic as more competition is appearing in the serps.

Also, keep in mind that, some of the best content, and some of the best answers to questions is found on reddit, quora, StackOverflow, etc. You just have to combine it and make it nice.

If it were that easy, everyone would be doing it.

 

[RowdySadjit]

I think you overestimate what low comp keywords mean in these AI cases.

The kind of keywords you are finding, are the ones you actually have a shot at for a money site where you actually research and write the articles.

Whereas, the low comp keywords the guys are using are not THAT low, where the first page is already populated. You just ride out the traffic, and get SOME part of it. In volume, that translates to traffic. That's why volume is important.

If i were to pick any tech blog with 3,000 articles, and start rewriting everything by hand, after some time i will eventually steal SOME of that website's traffic as more competition is appearing in the serps.

Also, keep in mind that, some of the best content, and some of the best answers to questions is found on reddit, quora, StackOverflow, etc. You just have to combine it and make it nice.

If it were that easy, everyone would be doing it.

Yes I had considered this and was almost going to write another paragraph saying the same but wanted people to come with their own responses first. I was thinking that, given the ease which content is found detailed by other posters in this thread it indicates that they are not targetting such low comp terms.

I am just coming off of studying morten's course which he advises to go to the lowest of the low, of course that is totally white hat so a different game, but I used that as my starting point.

@spectrejoe has said in his journey he is going for 0 competition for a lot of his keywords so I will be interested to hear his take on it.

If not going for 'no' comp then what should be the search criteria for choosing which keywords to save and make articles on? What I have done so far is to tell the bot to only save search terms where there exists either a quora or reddit result in the serps. So if we are not going after little to NO comp as this method does, I wonder rather what criteria I should choose to save keywords which will classify them as low but not too low as to have no content available to scrape.

Is it more just a case of scraping keywords indiscriminately in a given niche and making articles as fast as you can and seeing what sticks?

 

[RowdySadjit]

Couldn't edit above post but just found this from @Sartre from earlier in the thread:

So this is a 4 week old site, most articles were posted 2 weeks ago. I think this is very promising. But also I have a very good method for finding zero-competition keywords(reddit/quora/forums on top of results), that's why I'm getting clicks so fast without backlinking much. Also surprised that got some snippets for these keywords already, hence the high CTRs.

So it indicates he is indeed going for the lowest ones...interested to hear what he will say on the matter.

 

[mdevo]

Yes I had considered this and was almost going to write another paragraph saying the same but wanted people to come with their own responses first. I was thinking that, given the ease which content is found detailed by other posters in this thread it indicates that they are not targetting such low comp terms.

I am just coming off of studying morten's course which he advises to go to the lowest of the low, of course that is totally white hat so a different game, but I used that as my starting point.

@spectrejoe has said in his journey he is going for 0 competition for a lot of his keywords so I will be interested to hear his take on it.

If not going for 'no' comp then what should be the search criteria for choosing which keywords to save and make articles on? What I have done so far is to tell the bot to only save search terms where there exists either a quora or reddit result in the serps. So if we are not going after little to NO comp as this method does, I wonder rather what criteria I should choose to save keywords which will classify them as low but not too low as to have no content available to scrape.

Is it more just a case of scraping keywords indiscriminately in a given niche and making articles as fast as you can and seeing what sticks?

You are overthinking everything. Why won't you just make your app and then think about keywords you want to target?
Your starting point is to get working app that can do X, Y and Z with given keyword. If it can do those steps with all keywords, then you can think about targetting low comp keywords.
No comp keywords aint really that "no comp". There is always a competition, but it's not always that strong.

 

[Sartre]

Couldn't edit above post but just found this from @Sartre from earlier in the thread:


So it indicates he is indeed going for the lowest ones...interested to hear what he will say on the matter.

same as above - no need to overthink it. just go from zero competition and go higher once you run out of keywords. It's automated content anyway.

 

[Shaybib]

same as above - no need to overthink it. just go from zero competition and go higher once you run out of keywords. It's automated content anyway.

Yes. I don't get why people bother with keyword resesrch with autoblogs. Just use all keywords you find and hope for the best.

 

[spectrejoe]

Yes. I don't get why people bother with keyword resesrch with autoblogs. Just use all keywords you find and hope for the best.

I tried that approach and it works but takes a lot longer to see results. Going for low comp keywords first gets u traffic even on new websites.

Issue is, how the fuck are you gonna figure out which keywords are low competition and which ones aren't when you have thousands of them? lmao

 

[Accountstealth]

I tried that approach and it works but takes a lot longer to see results. Going for low comp keywords first gets u traffic even on new websites.

Issue is, how the fuck are you gonna figure out which keywords are low competition and which ones aren't when you have thousands of them? lmao

How about scraping the low competiton ones (usually low volume, right?) first with an auto scraper like SEO Minion.
It scrapes PAA which usually are low comp low volume and that should be it. No need to reinvent the wheel

 

[spectrejoe]

How about scraping the low competiton ones (usually low volume, right?) first with an auto scraper like SEO Minion.
It scrapes PAA which usually are low comp low volume and that should be it. No need to reinvent the wheel

I thought about it but I think it would be a bit weird for my AI sites using the snippet spam method (where the articles are literally just grabbing 10-30 snippets). Which means that each of these questions would likely get mentioned a lot of times I assume

 

[Accountstealth]

I thought about it but I think it would be a bit weird for my AI sites using the snippet spam method (where the articles are literally just grabbing 10-30 snippets). Which means that each of these questions would likely get mentioned a lot of times I assume

Not necessarily. Nowdays a lot of this questions have a dedicated postd for them. Although im not that good at automation it just makes sense to go for it.
Just my 2 cents tho

 

[mdevo]

I thought about it but I think it would be a bit weird for my AI sites using the snippet spam method (where the articles are literally just grabbing 10-30 snippets). Which means that each of these questions would likely get mentioned a lot of times I assume

But you don't have to use this particular method. Just target longtails from PAA's and add your own content to them.

 

[Accountstealth]

But you don't have to use this particular method. Just target longtails from PAA's and add your own content to them.

Exactly

 

[RowdySadjit]

Yes but that comes back to the original issue I raised of the long tails not having any decent content to scrape except the PAA answer. So PAA answer seems like the only valid choice there for those really low comp ones.

 

[mdevo]

Yes but that comes back to the original issue I raised of the long tails not having any decent content to scrape except the PAA answer. So PAA answer seems like the only valid choice there for those really low comp ones.

Actually websites that are included as feature within PAA have decent content. You just have to figure out a smart way to scrape valid sentences from this content.
Moreover, some of those snippets are not that accurate to the given question. I am sure that sometimes you can scrape better sentences from article than Google included in PAA box.
It's the hardest thing and also @Sartre mentioned that somewhere in this thread.

 

[RowdySadjit]

Actually websites that are included as feature within PAA have decent content. You just have to figure out a smart way to scrape valid sentences from this content.
Moreover, some of those snippets are not that accurate to the given question. I am sure that sometimes you can scrape better sentences from article than Google included in PAA box.
It's the hardest thing and also @Sartre mentioned that somewhere in this thread.

That is what I was saying, since google chose it it is already the best answer according to Google so it is like they already done the work for you.

I am gonna focus on these for the content. Hey we already know it works from all those copy and paste websites like were posted earlier who are making a killing with it without any type of paraphrasing/ai at all!

 

[Sartre]

That is what I was saying, since google chose it it is already the best answer according to Google so it is like they already done the work for you.

I am gonna focus on these for the content. Hey we already know it works from all those copy and paste websites like were posted earlier who are making a killing with it without any type of paraphrasing/ai at all!

PBNs are key for these sites

 

[hwp994108018]

It's more efficient to pay for the API and have the results within a few minutes for 20k keywords that I need at that point



this is really cool and impressive.

Right now we are working on a theme that is very flexible/randomized with different color pallettes and looks like it's a very legit site like Very Well Health etc. trying to target all EAT factors from the Google manual review handbook https://static.googleusercontent.co....com/en//searchqualityevaluatorguidelines.pdf

Right now scaling and improving the system is my goal, not making pennies. My goal is to get these sites into Mediavine/Adthrive, with slight editing. My white hat sites are already making a full-time income so I'm not pressed for making money. I wouldn't monetize a site that is getting less than 50k US impressions/month. I tried and got some sites into Ezoic just as a proof of contept but it's small money and a shitty network so we removed these ads.

“Ezoic is a terrible network”...
Although I have found a better network, but in reality, many people want to join but are rejected

 

[RowdySadjit]

@Sartre Are you still following the 'faq schema' format of just doing single sentence or two in a bulletpoint fashion, like how you did in that image example article you showed of 'running knee pain' earlier? Do you still rely on that to make the articles seem passable or did you evolve past that and are now able to make convincing pieces in any article format?

In my testing I have found that the paraphrasing falls apart when trying to paraphrase a whole informational essay/blogger style of 1000-1500 word post because, while the paraphrasing looks legit for a sentence or two, it goes to pieces the longer a paragraph is when the following sentence must follow the internal logic of the previous one.

So I wondered if your bot has become so advanced now that you can write whole articles that would look human passable in that format or rather you are still working around that by rather setting up the article structures so as to avoid large paragraphs which may poke holes in the ai's capabilities.

I am much more interested in the idea of making high quality articles which will stand the test of time, at a lower output, rather than just blasting churn and burn junk.

 

[hwp994108018]

But also I have a very good method for finding zero-competition keywords(reddit/quora/forums on top of results), that's why I'm getting clicks so fast without backlinking much. Also surprised that got some snippets for these keywords already, hence the high CTRs.

Learning and improving every day is a pleasure.

I've been building websites for a while and have built more than 10 websites. When I read your post a second time, I finally gained a deeper understanding of the key keywords.

If reddit/quora/forum, etc. can be ranked at the top of the search results, then the degree of competition is really low, which is better than any keyword tool to give.
This is also better than any "keyword method", no longer have to think about domain authority , page authority， backlinks, internal links, content word count...