Every day there seems to be people on various forums, IM discussion groups and SEO sites where someones site was dropped from Google, was banned from Google, was deindexed from Google, received a Google slap, etc. Call it what you will, but to see your traffic suddenly drop is scary to say the least. Usually people are scared and confused and looking for answers as fast as possible.
So I made a list of all the things that could happen to your site or page in a Google search and why it might have happened.
First, let’s clarify some terms here on what might have happened to you:
- Deindexed – When your domain is completely removed from Google. Also known as Banned.
- Penalized – When your domain or page still exists but none of your pages can be found through very direct search queries. This penalty can be automatic through the Google algorithm or manually applied by a Google Quality Engineer.
- Sunk – Your domain or page wasn’t Deindexed or Penalized, but the traffic you were getting from Google suddenly drops dramatically. Also known as Sent To Hell and Sandboxed.
How To Figure Out The Extent Of The Damage
Remember, Google ranks PAGES, not SITES. For example, http://electronplumber.com is a PAGE. You can call it a domain or a site, but as far as Google is concerned, it’s a single page. And http://electronplumber.com/about is a PAGE, and http://electronplumber.com/awesomepost is a PAGE. Google ranks pages. I’ve seen a sites homepage be penalized out of the index, but other pages on that same domain still rank fine.
Check With Google’s WebMaster Tools
Go to webmaster.google.com. If you haven’t setup an account and claimed your site, do it. Google will tell you exactly if there are any technical issues with your site here that may be causing your issues.
How To Tell If You Were Deindexed
Perform A Site Search – Do a “site:yourdomain.com” search in Google, where “yourdomain.com” is your domain. For example, I would type the following exactly into a search box, without the quotes: “site:electronplumber.com”.
The pages that come back are all the pages Google has in their database for your site. People call this your INDEXED PAGES. If your site doesn’t show and you get this message “Your search – site:electronplumber.com – did not match any documents.” you’ve been deindexed. Note that ANY typo or mistake will cause you to lose a lot of sleep over nothing. If all my pages are at blog.electronplumber.com or a sub-folder like electronplumber.com/blog, you need to do a site: search there. Check them all before you panic.
Don’t forget that Google’s main aim is to provide quality content and the best results for searchers. Typically a complete deindexing occurs from one of these things:
- Major Navigation Problems – This happens far more often than people realize in the IM world. Your pop-up opt-in or script that keeps people from using their back button to leave your site correctly WILL get your site deindexed by Google.
- Duplicate Content – Eventually a site that features ONLY duplicate content from shopping sources such as Amazon or Shopzilla will be deindexed.
- Offline For Too Long – Your site was offline multiple times for long enough for Google to deem it 100% no longer active. Google does NOT want sites in it’s results that don’t exist anymore. You may not have noticed your site ever being down, but if your server reboots at the same time Google tries to crawl it and a line goes down to your hosting provider the next time Google tries to crawl it, your site could get temporarily declared dead. This happens far more often than people think. It may take days or weeks to become reindexed. Only time will tell here.
- Technical Difficulties – Your site is blocked for indexing by a robots.txt file. Most people don’t realize that some automatic WordPress installs have a setting to not allow search engines to find your site.
- Manual Action – Your site received a manual review by a Google Search Quality Engineer and was found in serious violation of the Google Webmaster guidelines and includes a great many things you could have done wrong.
- Malware Links – If you use autoblogging content on your site, you could be hosting links to malware or virus sites without knowing it. Go through each link on your site and make sure none of them link to sites that Google reports as hosting malware.
As a general rule, if the ONLY redeeming value for your site is to earn YOU money while providing no value to your visitor, there is a good chance Google will eventually deindex you. YOU may think that your awesome site layout helps people find products faster or looks better, but if you copied all the descriptions for products right from Amazon, you haven’t added value in Google’s eyes.
How To Tell If You Were Penalized
Perform a Domain Search Test – If your site passes the index test above but is not showing up for searches, you might have been Penalized. Do a “yourdomain.com” and a “www.yourdomain.com” search in Google, not including the quotes. Does your site appear at the top for at least one of those? Any site that hasn’t been hit with a Google penalty should appear as the first results for an exact match search.
Perform a Content Search Test – If your page passes the domain search test above, try grabbing a full sentence or two from the page in question and searching for that text. If your page isn’t first or second, you’ve might have been penalized. Go through all the results. Does your page appear anywhere in that search? Does it appear dead last behind other pages that don’t contain the exact sentence you are searching for? Then you most certainly have been penalized.
You may have been Penalized by a Google Search Quality Engineer. If your site has violated the Google Webmaster guidelines but still has some redeeming value, your site may be marked in the Google Search system with a penalty to all it’s rankings for some period of time. If this happens to you, take a long hard look at your site. The main reasons for being Penalized are:
- Linking To Bad Neighborhoods – Linking OUT to bad sites such as gambling, porn, hacks, torrents, and the like are seen as bad for your site.
- Buying links – Tons of debate on if buying links can hurt your site. Most people say the worst that can happen here is you don’t get credit for the bought links. If not, then I could buy links to my competition report them to Google and have them penalized. If your site obviously only has paid backlinks though, that might get you into trouble.
- Selling links – Being caught selling links will definitely hurt your site.
- Engaging in linking schemes – Basically trying to hide buying or selling links by using three way trades and such.
- Sneaky Redirects – Attempting to redirect visitors but not search engine robots.
- Hiding Content – Serving different links or content to search robots than human visitors.
- Unnatural Linking Pattern – Your backlinks triggered some algorithmic flag known only to Google. For example, too many links to a new site with too little content. Note that I have never seen a site that was Deindexed for backlinks alone. Not one. It’s far more likely that your linking worked, which caused your site to rise in the SERPS, which caused your competition to notice violations with you site and report it to Google.
Many people call this being Sandboxed, but there is much debate over exactly what Sandboxing is that I don’t want to get into here. Check the Wikipedia page on the Sandbox Effect in the Resources section below. Your only option here is to fix whatever you think Google caught you doing (you’ll have to guess unfortunately) and either wait it out for a few months and hope it comes back, or submit your site to Google for Reconsideration.
How To Tell If You Were Sunk
If you weren’t Deindexed and weren’t Penalized but your site’s traffic from Google keyword searches has dropped dramatically, your site might have Sunk.
There is no manual action by Google, no removal from the index. But keywords you were previously ranking well for are no longer ranking well. There are MANY reasons why your site may have sunk in the Google rankings, but some of the most frequent are:
- Site Redesign – Internal links count and a site redesign may have removed what Google considers to be “aged” links, to the determent of your rankings
- Technical Errors – Setting your robots.txt by accident to prevent indexing on some inner pages can have a VERY negative effect on your rankings
- Loss of Backlinks – If you rely on a few high value backlinks that suddenly disappear, your rank will drop.
- Loss of Backlink Value – I once had a site of mine linked in an article on the NPR All Tech Considered blog, a PR 7 page. They only update once every few weeks, so for about three months while the article stayed on the front page I had a VERY powerful backlink. My search traffic to that site increased about 25%. Then three months later after the article dropped off the front page, my search traffic dropped back down again. And on the flip side, one of my bigger sites started as a WordPress.com free site. After a year I wised up and transferred it to my own hosted domain, paying WordPress.com $12 to redirect everything from my old site to my new site. The next year I decided not to pay the $12 again since I was no longer getting traffic from the redirects. A few weeks later, my search traffic dropped close to 50%! It was no longer getting credit for those nice aged backlinks that were previously redirected. I coughed up the $12 and a few days later my traffic went back to normal.
- Lack of Backlink Growth – If you have been doing lots of backlink building and then suddenly drop off.
- Lack of Fresh Content – Google loves updates and fresh content gives your entire site a boost. Some people call this QDF or “Query Deserves Freshness”
- Competition – Someone is now outranking you.
- Google Update – Like the famed “Google Farmer Update”, sometimes Google just changes their algorithm and you can get hit.
- Too Much Duplicate Content, Not Enough Links – I’ve seen this happen to many autoblog sites. After a few hundred articles without enough backlinks, you’ll trigger some math formula where Google gets wise and you’ll see search traffic plummet across the board.
- Outbound Link Junk – Too many outbound links can dilute any internal links you have.
- Penalized - Anything from the Penalized or Deindexed list above can cause your site to Sink instead of a harsher penalty.
- Loss of Honeymoon Effect – Google actually gives every new site a chance to be discovered. For the first few months your new site with new content will get a boost in the SERPS (Search Engine Ranking Positions) to give it a chance of being discovered and becoming popular, even when it has no links. When this effect goes away, your rankings will drop.
- Slow Site – Google takes into account site speed and loading times for their algorithms. If you are on a shared server, make sure your sites is optimized for speed as much as possible.
People Talk About The Sandbox. Is My Site In The Sandbox?
You will hear people claim that your new site is in something called the Google Sandbox. The theory is that a new site that has too many backlinks created will get put in a box by Google for 6-12 months and not appear high up in the search results.
Matt Cutts, head of the Quality Team at Google has stated publicly that there IS NO GOOGLE SANDBOX, but there are parts of the Google algorithm that may lead people to believe that such a thing exists.
Google wants a quality experience for visitors of its search engine. People like new and fresh content. Putting new sites that go viral “in a sandbox” is NOT in Google’s best interest.
The most likely explanation for The Sandbox effect is this:
- New unique content site gets created and gets a giant fresh content bonus (called QDF or Query Deserves Freshness) since everything is brand new.
- User builds tons of keyword stuffed spammy low quality backlinks to said new site. The link profile for that site looks unnatural to the algorithm. The Google algorithm very likely then IGNORES these low quality links for new sites.
- A month or two later, the site is Sunk to the depths of the search rankings once it loses its QDF bonus and only has ignored backlinks. Site owner thinks they have been sandboxed and stops building links, which makes the problem worse.
If Google was to penalize a site for having too many backlinks, anyone could take a competitors site out of the search results by spamming hundreds of thousands of low quality backlinks at it. The general consensus among SEO experts is that Google will ignore those low quality backlinks, not penalize you for them.
The lessons here?
- Make sure your backlinks look natural (links to words like “here” and “this site” and full URLs like “http://www.examplesite.com/article1″
- Make sure you create enough high quality links to any new site to counteract these effects.
Should I Resubmit My Site For Reconsideration?
Do NOT submit your site for Reconsideration unless you have identified what is wrong first and fixed it. You are basically asking for a Google Quality Engineer to come take a manual look at your site. Have you ever bought any links? Sold any links? Is your site keyword stuffed? Is your site mostly to earn you affiliate commissions? I’d guess 90% of the people reading this do not really want a Google engineer taking a manual look at their site. A reconsideration request is like asking for an IRS audit without being 100% sure you are owed money. It will just end badly.
If you have received a manual action, you can usually tell. Use the Google toolbar and check if your PageRank is N/A or is grayed out. Check the Google Webmaster report for your site.
If you have been penalized/deindexed/sunk because the Google algorithm had an issue with your site, asking for reconsideration is just asking for trouble. Google will NEVER tell you what is wrong with your site. Figure it out using the lists above, fix it, then wait a few weeks before even considering submitting a reconsideration request.
Google Basics – Webmaster Tools Help – Google