About three weeks ago, my blog was dropped from Google’s search index. It had a significant negative impact on my site’s traffic and on my Google PageRank. I thought it might be helpful to document what I have learned so far in case this ever happens to you.
First of all, I didn’t really know what hit me until one of my Twitter followers sent me a direct message: “Do you realize that your blog is no longer in Google’s search index and your PageRank is zero!” I’m sure the color drained from my face.
I immediately went to Google and tried to search for my own name. I got my Twitter profile page and a few images, but—sure enough—my blog was missing. I tried a few other searches for more specific content. Nothing.
I then checked my Google PageRank. It had dropped from five to ZERO!
Needless to say, this was disconcerting, especially since I know that about 13 percent of my blog’s traffic comes directly from Google search. I had not looked at Google Analytics in several weeks. However, when I checked it, I discovered that my search traffic had taken a nose dive beginning on June 22.
I then contacted my Web designer and developer, John Saddington. In typical fashion, he dropped what he was doing and began working on it. He and I spent the next two days, on and off, trying to figure out what had happened.
After a considerable about of sleuthing, we discovered that:
- Google’s verification code was missing. In order to verify your site and begin tracking it, Google asks you to insert a special code in the HTML head section of your blog’s home page or, alternatively, to upload a special HTML verification file. I had previously inserted the code. However, it was now missing. As a result, the Google robots considered my blog an “unverified site.”
- My robots.txt file had been modified. This file tells the Google search spiders what pages to index. If the file is missing, Google indexes your entire site. However, you may want Google to ignore certain pages or entire sections of your blog. My file had explicit commands to ignore EVERYTHING on my site. Nothing was being indexed.
- The functions.php file in my WordPress theme had been altered. Although we corrected the two problems mentioned above, it took us another day to discover that the head section of my blog had a “noindex,nofollow” meta tag in it. This, too, tells the Google search spiders to ignore the entire site. We finally traced this back to the functions.php file in my blog’s theme. It had this function in it, which we removed:
We are still not sure how these files were modified. There are at least three possibilities:
- Someone gained unauthorized access to my blog by hacking the password or exploiting a permissions setting.
- I unknowingly granted access to a hacker. I have hired a few coders here and there to do work on various aspects of my blog. It is possible that one of them modified these files. Why? I have no idea.
- I installed a plugin that made the modifications. Admittedly, I am always tinkering with my site. I try out a lot of plugins, so it’s possible I installed something that created the problem then uninstalled it.
I will probably never know how it happened. However, I am determined to protect myself from having it happen again.
Here’s how you can protect yourself as well:
- Search for yourself on Google. It’s probably a good idea to do this periodically to make sure your blog is still in their index.
- Monitor your PageRank. The easiest way to do this is via Google Toolbar for Firefox. You can also use any of the PageRank tools available online, like PR Checker.
- View your home page source code. Depending on the browser you are using, you can view the source code behind the page. Check the main page of your site and make sure you don’t have a “noindex,nofollow” meta tag.
- Use a strong password. I was using a pretty strong password, but I am now using 1Password to generate a completely random, very strong password. It would now be almost impossible to hack.
- Be cautious about giving access to your blog’s backend. I have a hard time imaging that someone I had previously hired did this, but I am going to be more cautious going forward.
- Research third-party plugins before installing them. I am going to pay more careful attention to user ratings and comments on plugins before I try them. It might also be worth doing a Google search to see if other bloggers have reported problems with the plugin you are considering.
- Make sure your permissions are set correctly. This is a bit more technical and applies specifically to WordPress, but make sure that the permissions on your wp-content folder are not set to 777. This setting would allow anyone to write to the files in that folder. If in doubt, ask your webmaster to verify the settings.
Note also that if your blog is dropped from Google Search, you will have to request that Google reconsider your site. You will then have to be patient. This is the hard part—at least for me.
It took almost a week for my site to reappear in Google’s search results. It took two weeks for my PageRank to be restored. Thankfully, everything is back as it was!