1. This site uses cookies. By continuing to use this site, you are agreeing to our use of cookies. Learn More.

Website Stopped Indexing

Discussion in 'BlackHat Lounge' started by mickyfu, May 8, 2015.

  1. mickyfu

    mickyfu Jr. VIP Jr. VIP

    Joined:
    Dec 14, 2011
    Messages:
    7,223
    Likes Received:
    20,787
    Occupation:
    King Of Crypto C
    Location:
    Solihull Young Offenders
    One of my websites is not indexing any more, and has not for the past 20 days or so. I have spent ages looking into it, but can see no obvious problems, anyone got any clues. The only thing I can think of is I used a "free" theme from here, this seems to tie in with when it stopped indexing. Since then I bought the theme and installed it, but have had no changes. I have run new pages through indexers and everything else. Google is still crawling the site everyday.

    Any ideas?

    The theme is Sahifa Wordpress
     
  2. Brad100

    Brad100 Supreme Member

    Joined:
    Nov 9, 2014
    Messages:
    1,348
    Likes Received:
    967
    Gender:
    Male
    Damnit, Charlie Floates messing with you again, you need to get a hold of this kid Micky, seriously.

    On a serious note, is the site using one of them new TLDs? because a couple of sites I set up on domain names ending with .party etc are not indexing either, that might be the problem.
     
    • Thanks Thanks x 1
  3. mickyfu

    mickyfu Jr. VIP Jr. VIP

    Joined:
    Dec 14, 2011
    Messages:
    7,223
    Likes Received:
    20,787
    Occupation:
    King Of Crypto C
    Location:
    Solihull Young Offenders
    No, the domain is a .com and a year or so old. It has not been spammed or anything either.
     
  4. archon10

    archon10 BANNED BANNED

    Joined:
    Oct 10, 2011
    Messages:
    1,181
    Likes Received:
    1,668
    It's hard to say but let's see.

    Couple of things to check:

    1) Is WMT showing any errors? A lot of times, there is some change to the host's firewall that blocks Google
    2) Robots is good?
    3) Can you view logs to see if Google is even hitting the server? If not, then maybe firewall issues.
    4) Have you checked for hacks like referrer hacks?
    5) .htaccess is ok?

    If you haven't already done it, I would first see if Google is even reaching the server by viewing logs and if not, then the host probably made a change to the firewall. No errors in WMT either?

    How often do you normally get indexed? You could also submit a sitemap to WMT to see if you can kick Google into motion. There is the tiny chance it's a bug on their end. If it continues and you're at your wit's end, you can PM the domain to me and I'll see if I can get some help. Can't promise anything and if it's a blackhat domain, I probably can't help. :)

    And no, I won't try to sell you SEO.
     
    • Thanks Thanks x 1
  5. mickyfu

    mickyfu Jr. VIP Jr. VIP

    Joined:
    Dec 14, 2011
    Messages:
    7,223
    Likes Received:
    20,787
    Occupation:
    King Of Crypto C
    Location:
    Solihull Young Offenders
    I don't add any of my sites to GWT.
    Robots are fine.
    Logs look fine.
    Can't see any sign of it being hacked.
    Google is still crawling.

    The pages used to get indexed in a matter of minutes - a hour. Just reinstalled Wordpress, can't think of anything else it could possibly be.
     
  6. leena

    leena Elite Member

    Joined:
    Apr 13, 2010
    Messages:
    2,233
    Likes Received:
    827
    Did you try any link indexer to index it?
     
    • Thanks Thanks x 1
  7. peetrike

    peetrike Power Member

    Joined:
    Aug 19, 2012
    Messages:
    585
    Likes Received:
    219
    Location:
    Estonia
    I´ve experienced strange things lately about indexing. Different browsers give different results about indexing. For example Firefox gave me 4 days that the domain isn`t indexed but Chrome showed it as indexed. Some days Google shows strange indexing results too - for example site which has 200k indexed content Google is giving me only 5k indexed. Few days later it`s all correct again. On another brand new domain different browsers give different results - one gives 180k, another 106k and third one gives 130k. But not bad at all considering that the domain has 1m+ pages. Also it`s funny when Google has completely deindexed some domains then it still crawls these.

    Cheers
     
    • Thanks Thanks x 1
  8. mickyfu

    mickyfu Jr. VIP Jr. VIP

    Joined:
    Dec 14, 2011
    Messages:
    7,223
    Likes Received:
    20,787
    Occupation:
    King Of Crypto C
    Location:
    Solihull Young Offenders
    Yes I have tried indexers and gone through all the basic things.
     
  9. IMpossible

    IMpossible Supreme Member

    Joined:
    Apr 15, 2012
    Messages:
    1,344
    Likes Received:
    302
    Occupation:
    Internet Marketing Guru
    Location:
    Somewhere on earth
    Just use GWT for 1-2 days and see what kind of errors/suggestions you get.
     
    • Thanks Thanks x 1
  10. mickyfu

    mickyfu Jr. VIP Jr. VIP

    Joined:
    Dec 14, 2011
    Messages:
    7,223
    Likes Received:
    20,787
    Occupation:
    King Of Crypto C
    Location:
    Solihull Young Offenders
    Is it possible to completely get rid of GWT after you have used it?
     
  11. IMpossible

    IMpossible Supreme Member

    Joined:
    Apr 15, 2012
    Messages:
    1,344
    Likes Received:
    302
    Occupation:
    Internet Marketing Guru
    Location:
    Somewhere on earth
    When you login to GWT, the first thing you'll see is a list of all your websites that are in use.
    You can simply just delete it from there on.
     
  12. andyf91

    andyf91 Senior Member

    Joined:
    Aug 18, 2013
    Messages:
    1,029
    Likes Received:
    274
    Occupation:
    Internet Marketing
    Location:
    Florida
    Im having a similar problem as you for a domain its been over a month and still nothing ill keep you posted if i find anything!
     
  13. archon10

    archon10 BANNED BANNED

    Joined:
    Oct 10, 2011
    Messages:
    1,181
    Likes Received:
    1,668
    If Google is reaching the pages, then really WMT will just tell you if there are any errors such as 500, 503, 404, so technically the logs should tell you the same if you want to avoid WMT.

    Where WMT is helpful is if Google can't reach the pages at all such as a DNS issue (put your site into intodns.com to check) , or firewalls are blocking it. You would never know those issues because you would just see no crawls.

    However, it could be possible that the firewall is blocking just some of Google's IPs and perhaps Google is taking a conservative approach and not indexing the pages.

    Also, since you have a hunch that it's the theme, try using Rex Swain's HTTP viewer or web-sniffer.net and put Google's useragent in there to see if different content displays based on user agent.
     
  14. ryce889

    ryce889 Regular Member

    Joined:
    Jul 23, 2008
    Messages:
    491
    Likes Received:
    152
    Location:
    NY, NY
    Hey, I'm actually also using Sahifa's theme and noticing issues with indexing. The blog post URLs aren't being indexed at all, however tags and categories are. I've just recently switched my theme - will let you know if that changes anything.
     
  15. dannyslimited

    dannyslimited Registered Member

    Joined:
    Jan 10, 2015
    Messages:
    78
    Likes Received:
    48
    Location:
    USA
    Make sure your site map or maps is configured correctly.
     
  16. archon10

    archon10 BANNED BANNED

    Joined:
    Oct 10, 2011
    Messages:
    1,181
    Likes Received:
    1,668

    interesting...
    since the OP doesn't use WMT, I'm wondering if the theme is sniffing user agent/IP or something like that and throwing errors at Google that you don't see.
     
  17. ryce889

    ryce889 Regular Member

    Joined:
    Jul 23, 2008
    Messages:
    491
    Likes Received:
    152
    Location:
    NY, NY
    The sitemaps appear to be configured correctly. When I submit the URLs to Bing to index, they show up, however when I use Google to index the URLs, nothing happens.
     
  18. pxoxrxn

    pxoxrxn Supreme Member

    Joined:
    Dec 21, 2011
    Messages:
    1,398
    Likes Received:
    2,073
  19. terrycody

    terrycody Supreme Member

    Joined:
    Sep 29, 2012
    Messages:
    1,462
    Likes Received:
    401
    Occupation:
    marketer
    Location:
    Hell
    Hey mickyfu

    Still remember me? You replied to my post about my case: expired domain content to my new site not working well and 0 traffic after 1 month working ---

    ---------
    http://www.blackhatworld.com/blackh...osts-zero-traffic-what-wrong.html#post7885133
    ---------

    And I think I am facing the same problem as you, site still review ZERO traffic per day, already 1.5 month, and has 180++ posts on it!

    And I has noticed some strange thing like you, sites new posts stop indexing by Google, I forget the day, seems like around half month ago when this strange thing happened.

    So today I run site: mydomain.com on Google, 96 URL indexed! ( note: I have 180++post)

    And I go to Bing.com, site my domain: Only Home page got indexed!?!


    WTF??!


    Like you guys saying, I use a no-responsive theme shared before in BHW, though I installed WPtouch to mobile it, but I think I should change the theme ASAP, maybe the theme has some wired problem, who knows!

    If anyone figured out why this happen, please let me know, thank you all! :p
     
  20. ok888

    ok888 Elite Member

    Joined:
    Nov 23, 2010
    Messages:
    2,369
    Likes Received:
    654
    Are anyone of you guys using new domain Im facing the same, but i think the issue is google / new domain

    Im using Sahifa as well but I'm having this problem on another site with different theme