SpectreJoe's Autogenerated Content to 700$/month. [Self-coded Bot]

[nelida kornilova]

it`s worth to try
do you start to monetize it now

 

[TheOverlord]

Looking forward to your update bro

 

[spectrejoe]

Update, Day: 29
Total articles posted so far: 15265 (+7129)

Not much to say really, nothing happened so far that's interesting.

Updated my bot and content is no longer an issue, it's now 100% readable and 100% unique, downside is only being able to upload ~1k-1.5k posts a day.

Google has been EXTREMELY slow at indexing the posts, apparently, there is a certain limit for new sites and it seems like I've hit it, in the past 14 days only like 200 posts have been indexed.
I stopped posting until they start getting indexed because there's no point adding 500k articles right now if they aren't getting indexed. + I've been using bots for other projects that have been making me nearly 100$ a day and that's consuming all the RAM in my computer so for now I don't have the ability to run all these programs at the same time in my desktop.

Once Google starts indexing more often I'll continue posting new articles.

I already expected it to be slow so no surprises here, I predict starting next month site will start booming.

 

[ANewBeginning]

Google has been EXTREMELY slow at indexing the posts, apparently, there is a certain limit for new sites and it seems like I've hit it

That's probably due to the crawl budget man

 

[Gogame1]

Google Index around 12,000 to 15,000 pages and stop indexing further posts.
Also i noticed Google start deindexing posts.

I have a website where i made around 200000+ posts in a span of 2 months on an expired domain.
Its been 2 years since i made those posts and Google indexed 30,000 posts till last 2 months back.

Suddenly last month Google indexed another 35,000 posts.
After that once again Google is deindexing posts.

 

[forwardedlandlines]

I'm actually developing my own high tech content drip system, so it's been fun to read this one. Mine will be starting out much slower (over many domains, that also interlink), letting every bit of content soak (and slowly speed up the rate), and though automated, have high quality content at the top for dwell time, and have a strong structure in place for the rest of the content, that will not be spun but paraphrased. By structure I mean a solid structure of h1/2/3, ul/li, maps, address, faqs, index menu, strong/italic, references, youtube, royalty free images... Also site speed will be scored at 100%, because the framework is custom built and will employ the best speed methods, and UI experience methods, like responsiveness and precalculating the layout boundaries to prevent CLS (Cumulative Layout Shift), and other aspects.

Also backlink automation will be in place, like a high quality GSA campaign with recaptcha solving on every newly dripped post.

Eventually the whole content network of this system is intended to SEO boost 1 single business site, but I believe the satellite domains will themselves become traffic drivers as well...

 

[Shaybib]

Are you interlinking your posts? It would help indexing

 

[TheOverlord]

Google Index around 12,000 to 15,000 pages and stop indexing further posts.
Also i noticed Google start deindexing posts.

I have a website where i made around 200000+ posts in a span of 2 months on an expired domain.
Its been 2 years since i made those posts and Google indexed 30,000 posts till last 2 months back.

Suddenly last month Google indexed another 35,000 posts.
After that once again Google is deindexing posts.

Did you have proper SEO setup for google crawlers? Interlinking, social signals, and just getting your link out there I mean. Also a good sitemap structure. If you did this and they still didn't index that's pretty strange.

 

[Gogame1]

Did you have proper SEO setup for google crawlers? Interlinking, social signals, and just getting your link out there I mean. Also a good sitemap structure. If you did this and they still didn't index that's pretty strange.

Yes, I have proper SEO setup for Google crawlers with sitemap and interlinking.

 

[TheOverlord]

Yes, I have proper SEO setup for Google crawlers with sitemap and interlinking.

did it have lots of duplicate content?

 

[galusx]

Did HTTP or HTTPS really matter for index pages? I just start up my 100k sites without drip post, let`s see how it turns out.

 

[spectrejoe]

That's probably due to the crawl budget man

Yea most likely because it clearly says it knows there are 15k+ posts due to sitemap but when I check the link they show it always says not crawled yet.

Google Index around 12,000 to 15,000 pages and stop indexing further posts.
Also i noticed Google start deindexing posts.

I have a website where i made around 200000+ posts in a span of 2 months on an expired domain.
Its been 2 years since i made those posts and Google indexed 30,000 posts till last 2 months back.

Suddenly last month Google indexed another 35,000 posts.
After that once again Google is deindexing posts.

That's weird, hows ur content quality? if its highly duplicate then it explains the slow rate

I'm actually developing my own high tech content drip system, so it's been fun to read this one. Mine will be starting out much slower (over many domains, that also interlink), letting every bit of content soak (and slowly speed up the rate), and though automated, have high quality content at the top for dwell time, and have a strong structure in place for the rest of the content, that will not be spun but paraphrased. By structure I mean a solid structure of h1/2/3, ul/li, maps, address, faqs, index menu, strong/italic, references, youtube, royalty free images... Also site speed will be scored at 100%, because the framework is custom built and will employ the best speed methods, and UI experience methods, like responsiveness and precalculating the layout boundaries to prevent CLS (Cumulative Layout Shift), and other aspects.

Also backlink automation will be in place, like a high quality GSA campaign with recaptcha solving on every newly dripped post.

Eventually the whole content network of this system is intended to SEO boost 1 single business site, but I believe the satellite domains will themselves become traffic drivers as well...

Damn, that sounds REALLY intricate and complex, mine does post with H2 and such but that's all, no images (besides the featured image of the post) or anything.

Are you interlinking your posts? It would help indexing

Nope, unfortunately don't know how to do it in an automated way unless there was a plugin for Wordpress that auto does it.

Did HTTP or HTTPS really matter for index pages? I just start up my 100k sites without drip post, let`s see how it turns out.

Not too sure but don't think that's the issue. I'll be switching over to https again soon though to give it a try now that I've slowed down post publishing + got proxies

 

[bsantos]

Hey would you be able to share the code? I have been very involved in automation and scrapping lately I think I can help and we could colaborate? I think i saw somewhere you were from Portugal thats awesome me too! Loving this journey so far keep it up!

 

[bobbyeiken]

In my experience googlebot crawl around 12500-15000 pages in 30 days.
Since you use new domains, googlebot might limit your crawl budget.
There's no way google willing to crawl all you pages without limit, that will be pricey.
I think the logic for this is Google must have already implemented an algorithm to detect and prevent this kind of spam site.
My recommendation is you should drip feed the articles or spread it to multiple domains.

 

[TheOverlord]

In my experience googlebot crawl around 12500-15000 pages in 30 days.
Since you use new domains, googlebot might limit your crawl budget.
There's no way google willing to crawl all you pages without limit, that will be pricey.
I think the logic for this is Google must have already implemented an algorithm to detect and prevent this kind of spam site.
My recommendation is you should drip feed the articles or spread it to multiple domains.

Google has explicitly said drip feeding is kinda useless even for hundreds of thousands of pages

I think putting the links around the web in social media and directories would be the best move

 

[Ignatius O]

Are you using python for your bots?
If so are you using selenium module or requests module

 

[spectrejoe]

Hey would you be able to share the code? I have been very involved in automation and scrapping lately I think I can help and we could colaborate? I think i saw somewhere you were from Portugal thats awesome me too! Loving this journey so far keep it up!

I can't, at least not yet, I'm still tweaking the code very often and I'd need to do some extra changes before releasing it to public and ye I'm from Portugal Almada

In my experience googlebot crawl around 12500-15000 pages in 30 days.
Since you use new domains, googlebot might limit your crawl budget.
There's no way google willing to crawl all you pages without limit, that will be pricey.
I think the logic for this is Google must have already implemented an algorithm to detect and prevent this kind of spam site.
My recommendation is you should drip feed the articles or spread it to multiple domains.

Yea it's crawl limit but don't think drip feed posts will help at all if google wont crawl them anyways.

Google has explicitly said drip feeding is kinda useless even for hundreds of thousands of pages

I think putting the links around the web in social media and directories would be the best move

That's what I was thinking as well but I'd have to figure out a way to automate adding every post I make to social media or social bookmark sites or smth.
Not too sure if it's gonna be easy to do that.

Are you using python for your bots?
If so are you using selenium module or requests module

requests mostly using selenium for other programs not related to this project though

 

[Shaybib]

Nope, unfortunately don't know how to do it in an automated way unless there was a plugin for Wordpress that auto does it.

There is plugin contextual related posts that might help. Also you can export sitemap to csv and add to your post "you might also like:" and add random 5 links with anchors that is if you are posting by your own program

 

[bsantos]

Hey any update on the journey? Was really excited!

 

[jamie3000]

Are you using last modified attribute in your sitemap? Because if not Google will be spending half your crawl budget recrawling / rechecking pages it's already crawled once. I remember this from a 100k site I made when I monitored the Apache logs.