How to force google to crawl your URLs?

[jamie3000]

I saw a while ago someone mentioned a trick where you submit a sitemap to Google and 301 the URLs in the sitemap to a website that you want crawling. eg money site, web2.0s, gsa links etc.

Does anyone know if this is what indexer services are doing now? And does this work at scale, e.g 50000 pages.

Thanks for any info

 

[MisterXYZ]

How is this different from submitting sitemap of your website if you are trying to index website?

 

[PBN Systems]

How is this different from submitting sitemap of your website if you are trying to index website?

Because he's trying to get links crawled that aren't on his site.

 

[jamie3000]

How is this different from submitting sitemap of your website if you are trying to index website?

Because you can use the crawl budget of multiple websites if I understand it correctly

 

[MisterXYZ]

Because he's trying to get links crawled that aren't on his site.

No he is trying to index own website.

Because you can use the crawl budget of multiple websites if I understand it correctly

But Google said crawl budget does not come into picture for less 100,000 or 1 million pages. Though you can not believe anything coming from Google.

 

[jamie3000]

No he is trying to index own website.



But Google said crawl budget does not come into picture for less 100,000 or 1 million pages. Though you can not believe anything coming from Google.

I've got 16,000 ish pages and Google is crawling 1k a month which is nothing when you're waiting to hit the button to scale it to one million.

I think I only have two options build more links which in theory should increase my crawl budget, that seems to be how it works.

Or find a way to force crawl my pages.

 

[Myst3ry]

Post tweets with popular hashtags(haven't tried it, just have saved it for later)

 

[jamie3000]

Post tweets with popular hashtags(haven't tried it, just have saved it for later)

Not sure I can post my domain 50,000 times but I'll certainly look into it, thanks

I'll do the 301 thing with an old domain and check my Apache logs to see what the crawl bot does.

 

[AngelSeo]

speed-links.net seems to be still working (mostly).

 

[SirLouen]

Because you can use the crawl budget of multiple websites if I understand it correctly

This is just theory. Crawl budget itself doesn't exist.
It's just common jargon to designate the fact that Google will not crawl thousands of sub pages of a site that doesn't have a huge reputation because it has to potentially optimize his power to crawl other more relevant sites.

You cannot force anything and now even less that Google is still having indexing issues with the full migration of GWT to GSC

Although you can "suggest" Google to crawl your site, by using multiple methods, like the ones mentioned above (indexers, using 2.0 with certain rep, using the indexing tool for you own site...)

 

[AngelSeo]

This is just theory. Crawl budget itself doesn't exist.
It's just common jargon to designate the fact that Google will not crawl thousands of sub pages of a site that doesn't have a huge reputation because it has to potentially optimize his power to crawl other more relevant sites.

You cannot force anything and now even less that Google is still having indexing issues with the full migration of GWT to GSC

Although you can "suggest" Google to crawl your site, by using multiple methods, like the ones mentioned above (indexers, using 2.0 with certain rep, using the indexing tool for you own site...)

That is correct - you can "force" Google to crawl the pages, but thats it - the indexing part.. well.

 

[jamie3000]

That is correct - you can "force" Google to crawl the pages, but thats it - the indexing part.. well.

My pages index at 94% ish...once they're actually crawled. So it's the crawling bit I'm struggling with rather than the indexing

 

[Deleted member 1319755]

speed-links.net seems to be still working (mostly).

What’s the indexing rate for you and when last did it work for you?

 

[AngelSeo]

What’s the indexing rate for you and when last did it work for you?

I use it all the time, but is kinda hard to tell the indexing rate as really depends of the property you are trying to index. But I would say that around 40-50% of the shits are indexed usually on the first try and I do a second one a week later that give me maybe 10% percent. So probably around 50-60% of the links are indexed.

 

[PrayingMantis]

I use it all the time, but is kinda hard to tell the indexing rate as really depends of the property you are trying to index. But I would say that around 40-50% of the shits are indexed usually on the first try and I do a second one a week later that give me maybe 10% percent. So probably around 50-60% of the links are indexed.

Can you please tag the service provider? I have issue making payments to index my links.

 

[andyrilflamewest]

speed-links.net seems to be still working (mostly).

Not for me. About 25%