Nigel Farage
BANNED
- Feb 8, 2012
- 565
- 1,502
Some of those websites load fast and some of them load slow and in order to have "live" harvesting we will have to crawl whole world wide web every single time the analysis is being performed which is not realistically possible to do within an acceptable period of time with technologies that exist right now.
That's another thing I don't understand. This notion that a crawler must crawl every single website on the web. I find that difficult to understand for several reasons.
First, the whole web isn't indexed by Google. So if you have backlinks on sites that aren't indexed, it doesn't matter if they aren't found or reported, because they don't count anyways. Second, Google by definition not only presents all the backlinks that are indexed, but it presents them in order of what it thinks is relevant, which ought to be a pretty good indicator of what it thinks is relevant. I assume there has to be more to backlink checking and analysis than simply doing a Google search, but it seems to me that simply running Google would be a good start.
I don't understand why credible, commonly-used software (like SEO Spyglass) cannot manage to report the existence of backlinks that are on page 3 of Google. Now in this thread I am getting the message that your software is independently crawling the entire web, and because of this, "X", "Y" and "Z", and I just cannot understand why Google cannot be used to find out what backlinks Google itself has indexed.
Last edited: