1. This website uses cookies to improve service and provide a tailored user experience. By using this site, you agree to this use. See our Cookie Policy.
    Dismiss Notice

Pages with no visits, Crawl Budget Optimisation, and meta-noindex

Discussion in 'White Hat SEO' started by scenari0, Jan 29, 2020.

  1. scenari0

    scenari0 Newbie

    Joined:
    Jan 16, 2020
    Messages:
    14
    Likes Received:
    0
    One of my website has 500,000+ webpages, and I started to see that the SERP ranking of such pages started to be eroded with the time.

    I've been checking, and around 10,000 of the webpages had no visit in the last six months. I've read about the concept of "Crawl Budget Optimisation", and I'd like to know your opinion:

    • Can the ranking-drop be related to the Crawl Budget?
    • Is it worth inserting a "<meta name=robots content=noindex>" tag in each one of the pages with no visits?
    • Any recommended tool to handle/automate the server logs?

    Thank you very much.
     
  2. W Mann

    W Mann Newbie

    Joined:
    Jan 14, 2020
    Messages:
    14
    Likes Received:
    1
    I'm in a simlar scenario with one of my larger sites - did a log file analysis and found Google bots trying to crawl hundreds of 404 pages. Not sure how to cut them or stop them short of a 410 on each as they're non-existant (either old or a result of Magento) and there's no internal or external backlinks. Frustrating as it does seem to be wasting crawl budget. Super interested to hear opinions on how to stop the bots from attempting to find/crawl these pages..