1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Sitemap Questions

Discussion in 'White Hat SEO' started by Scorpion Ghost, Feb 17, 2014.

  1. Scorpion Ghost

    Scorpion Ghost Senior Member

    Joined:
    Mar 22, 2013
    Messages:
    915
    Likes Received:
    180
    I'm working with a blogger blog and last night i decided to take the leap and get into sitemap/robots.txt/robots header tags...

    First i submitted my blog to Webmasters Tools, and it got configured ok. I got 10 posts on the blog, webmaster tools found 10 posts indexed, all good. I also know how to submit a second or third or more sitemaps when i reach over 500 or over 1000 articles on my blog. Because blogger sitemap/feed is limited to 500 posts, you gotta submit a new sitemap for each 500 posts on your blog. Anyway, i did that and i figured it out.

    I was planning to go and submit my sitemap to Bing/Yahoo next, but i can't move on until i clear up some things:

    -------------------------------------

    1. Resubmitting Sitemaps
    When you make changes to your Sitemap, you can resubmit it using your Google Webmaster Tools account.

    Resubmit a Sitemap using Webmaster Tools


    1. On the Webmaster Tools Home page, click the site you want.
    2. Under Crawl, click Sitemaps.
    3. Select the Sitemap(s) you want to resubmit, and then click the Resubmit button.

    Really? I gotta resubmit it every time i "make changes" to my site? And what's "changes exactly"? A new article? A change of color with CSS on the navigation tabs? Deleted footer gadget? Is this really necessary?

    -----------------------------------------

    2. Submit a Sitemap using robots.txt

    This is from google's website, and they first show you how to submit a sitemap on webmaster tools, and then they show you how to submit it in your robots.txt file as well.

    For blogspot blogs, there are some codes in the robots.txt file by default, here they are:

    User-agent: Mediapartners-Google
    Disallow:

    User-agent: *
    Disallow: /search
    Allow: /

    Sitemap:
    http://example.blogspot.com/feeds/posts/default?orderby=UPDATED

    Is it really necessary to submit the Sitemap to the robots.txt also?

    And in addition to that, i read that the blogger's sitemap (and the feedburner feed) only display the 25 latest posts by default, so it's wise to insert your sitemap in there in order to increase that amount so that the crawlers can see all of your posts each time they visit your site...

    But on the other hand, if i'm submitting my sitemap to google and bing/yahoo, then they will already see my entire sitemap. But then, if i don't go and resubmit it every time i make a change on my blog, then maybe they'll miss something...

    Oww shit... Confused :sucks:
     
  2. ooo

    ooo Newbie

    Joined:
    Jan 4, 2017
    Messages:
    8
    Likes Received:
    0
    Thank you for asking this. Specifically, I'm wondering if/when it's necessary to add your sitemap to the robots.txt file.

    Is it redundant to add your sitemap url to the robots.txt file?
     
  3. Scorpion Ghost

    Scorpion Ghost Senior Member

    Joined:
    Mar 22, 2013
    Messages:
    915
    Likes Received:
    180
    Hey,

    Frankly, I don't even remember if I added it to the robots.txt or not.

    But if I were to start a blog today, I'd submit the sitemap to Google and Bing/Yahoo and Yandex webmaster tools, and that would be that. I wouldn't add it in the robots.txt.

    So that's the advice I'd give right now :)