1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

What is the difference between crawling and indexing

Discussion in 'Black Hat SEO' started by Absurtuk, Jan 7, 2014.

  1. Absurtuk

    Absurtuk Regular Member

    Joined:
    Mar 1, 2013
    Messages:
    239
    Likes Received:
    31
    Hey I've searched the entire forum for this question but nothing comes useful to me .
    So , in many link indexing services they say "we are gonna crawl your backlink"
    so what is the difference between crawling and indexing ?
    Crawling like , google bot comes to your site and just fetch the data and not indexing your page?
     
  2. Tosmekop

    Tosmekop Supreme Member

    Joined:
    Oct 24, 2011
    Messages:
    1,208
    Likes Received:
    813
    Think of it like a book. The author of a book will spend time researching different areas of interest for his/her book. They are essentially crawling for data/information. They [The search-bots] see what's out on the internet (as it relates to your question), by crawling. As for the index, also like a book -- or what you'd find at the back of the book, will point to where on this particular website (that's been crawled), the information may be located. It's basically the SERPs' index, after they've crawled your site, that your site becomes added to. That is how you can type things into the search textboxes, to obtain the results of whatever it was you were searching for. Hope this helps. :)
     
    • Thanks Thanks x 3
  3. saqib1234

    saqib1234 Newbie

    Joined:
    Nov 23, 2013
    Messages:
    47
    Likes Received:
    6
    Crawling is that search engine spiders visit your website. Indexing is that they consider their visit worthwhile and record their visit in search engine.
     
    • Thanks Thanks x 1
  4. wahabfun

    wahabfun Junior Member

    Joined:
    Dec 25, 2009
    Messages:
    121
    Likes Received:
    22
    Crawling occurs when the google robot traced URL from other pages and when it crawled in the url then it fetches its details like content, links etc and passed this info to other engines which is content filtration engine. If this filtration engine found the content useful then it index the url.

    Conclusion: Crawler only visits the site and collect the details like content,links etc and then passed these info to content filtration engine. Then if content filtration engine founds the content useful then it will pass the content and url to the indexing engine which will further index the url.
     
  5. Ballan

    Ballan Junior Member

    Joined:
    Oct 29, 2013
    Messages:
    105
    Likes Received:
    17
    As an aside, you can check indexing in google with the site:your domain.com operator

    Bot crawling you can check in your server logs.
     
  6. jollyman

    jollyman Jr. VIP Jr. VIP Premium Member

    Joined:
    Aug 29, 2013
    Messages:
    1,815
    Likes Received:
    286
    Gender:
    Male
    Occupation:
    Digital marketing
    Location:
    Somewhere around BHW
    Crawling is the process of search engine crawler visits your site and fetch the data.
    Indexing is actually positioning the fetched data in the search engine data base.
     
    • Thanks Thanks x 1