How to index millions of pages?

TheOverlord

Jr. VIP
Jr. VIP
Joined
Sep 1, 2014
Messages
121
Reaction score
97
I mean shitty pages, like spun and translated content. How do I get it to index faster?
 

sir.rankalot

Junior Member
Joined
Sep 25, 2020
Messages
121
Reaction score
80
1. Having a strong server that can handle a huge crawl budget (search google for crawl budget)
2. Make sure your site has a good structure that google can crawl easily.
3. Do a sitemap then submit it to the search console. If you cannot, or don't know how to generate sitemaps for million pages I will show you.
4. Do backlinks
5. Get referral traffics (many ways: social, high-quality guest post ...)
 

VampireDamon

Regular Member
Joined
May 26, 2021
Messages
244
Reaction score
60
I mean shitty pages, like spun and translated content. How do I get it to index faster?
The first important thing to consider for indexing is the uniqueness and the quality of the content.
Since you are trying to index translated and spun content make sure they are readable, grammar error-free, and plagiarism-free.
Try using Google Webmaster Tools which will request an increased crawl rate, if you have not tried it yet.
Try to get qualified links to accelerate the process of indexing.
Try formating a crawl loop.

I hope this will help you.
 

jamie3000

Jr. VIP
Jr. VIP
Joined
Jun 30, 2014
Messages
4,592
Reaction score
3,015
The first important thing to consider for indexing is the uniqueness and the quality of the content.
Since you are trying to index translated and spun content make sure they are readable, grammar error-free, and plagiarism-free.
Try using Google Webmaster Tools which will request an increased crawl rate, if you have not tried it yet.
Try to get qualified links to accelerate the process of indexing.
Try formating a crawl loop.

I hope this will help you.

A crawl loop?
 

kabopon

Registered Member
Joined
Jun 2, 2021
Messages
57
Reaction score
24
Pretty much as others have mentioned: Backlinks, referral traffics, a well built site (and sitemap) and a good niche can help you stand out.
 

Piss

Regular Member
Joined
Apr 24, 2021
Messages
226
Reaction score
109
The first important thing to consider for indexing is the uniqueness and the quality of the content.
Since you are trying to index translated and spun content make sure they are readable, grammar error-free, and plagiarism-free.
Try using Google Webmaster Tools which will request an increased crawl rate, if you have not tried it yet.
Try to get qualified links to accelerate the process of indexing.
Try formating a crawl loop.

I hope this will help you.
He said millions of pages, man.
 

VampireDamon

Regular Member
Joined
May 26, 2021
Messages
244
Reaction score
60
A crawl loop?
Yes.
Copy a list of all of your site's links and paste them all into a PHP file that was called from every page. Then you go to the Google Search Console and try Google to get a URL and crawl every single link on that URL's page.
Since you have so many connections and the pages to which you connect likewise have a lot of links, Google gets caught in a loop and scans your site considerably faster.
 

Ignatius O

Regular Member
Joined
Jun 27, 2016
Messages
267
Reaction score
57
You have to discover this on your own. I don't think anybody will be willing to give you the right solution to this
 

johndig

Newbie
Joined
Jan 18, 2021
Messages
7
Reaction score
1
Try linkbox pro. Sometimes it helps. But the best way to fix low-quality pages in the index is to create permanent 301 redirects to each of them. This is only one working method I found.
 

terrycody

Jr. VIP
Jr. VIP
Joined
Sep 29, 2012
Messages
4,202
Reaction score
1,789
Even you can index, you will finally get a 4 result penalty, mostly.
 
Top