You built a blog a while ago, but where did you build the backlinks?
Backlinks indicate how many links are pointing TO your site from different sites, rather than how many links are on your own site.
The links that are pointing to your site on other websites need to be crawled and indexed by google, and be (do) follow links in order to show as a backlink.
If they are crap links like spam posts on low value blogs then they can also drop from Google and will no longer be shown as a backlink when you analyze your website.