How can i bulk check indexing?
- Thread starter online28
- Start date
SiouxMarketing
Newbie
- May 15, 2020
- 38
- 19
Just to understand correctly, you're wanting to know how to check the status of a URL in Google index in bulk correct?
If so, are these all the same domain, or are these bulk URLS all different domains?
If so, are these all the same domain, or are these bulk URLS all different domains?
Stas Va
Junior Member
- Oct 1, 2019
- 164
- 60
The problem with bulk indexation checking is Google captcha.
So you should use Crawler + CapMonster (or alternative) + rotating proxies. And there also should be logic.
I'll provide Tuesday a working solution.
Visual Eagle
Elite Member
- Dec 11, 2008
- 2,630
- 2,282
Find the right proxy provider for SB.. and use site: with 2-3 seconds delay.
Always works for me but i am checking around 100-200 links only with 10 proxies.
Always works for me but i am checking around 100-200 links only with 10 proxies.
online28
Supreme Member
- Feb 24, 2011
- 1,464
- 237
Which crawler should I use? Thanks
online28
Supreme Member
- Feb 24, 2011
- 1,464
- 237
Yes, any solution? Thanks
SiouxMarketing
Newbie
- May 15, 2020
- 38
- 19
You did not answer the question I asked. I asked if it was one domain or multiple domains?
online28
Supreme Member
- Feb 24, 2011
- 1,464
- 237
Sorry, multiple domains, they are web 2.0
- Apr 4, 2014
- 6,464
- 4,864
I use http://stratage.ms/en/indexchecker/ + tor but it's not 100% accurate as rarely it shows 'not indexed' whereas on a manual check it's indexed and the limit is 50 URLs per run.
- Jan 25, 2009
- 6,378
- 3,774
Scrapebox works fine, its the proxies that are the issue. For example try a test with no proxies.
you need different proxies is all, storm proxies is no longer good with googe, at least thats what many users say.
online28
Supreme Member
- Feb 24, 2011
- 1,464
- 237
Which provider is good for Google now? Should i still buy rotating proxies for crawling? Thanks
- Jan 25, 2009
- 6,378
- 3,774
Rotating proxies are good for like scraping from search engines, but they tend to be from all sorts of sources from all sorts of locations around the world.
When it comes to index checking google has tons of data centers and your geo location of the IP determines where you connect and what results you get.
So you can literally use different ips around the world and get a yes and a no as indexed for the same url and both are valid. So you have to navigate that, although ideally your index checking with an ip relevant to where you want to rank. Like if you want to rank in google france, then you use a France IP and thus you get results relevant to google france
That said I would personally buy private proxies for indexing and of course you can use those for scraping too. Although of course you can use rotating proxies for scraping.
Here is an article I wrote on this concept, its for rank tracking but the information is relevant to index checking.
http://scrapeboxfaq.com/why-does-ra...ank-then-what-i-see-in-a-browser-or-elsewhere
My 2 cents.