Website Suddenly Disappears from Google Search

Introduction: Recently a website of mine suddenly disappears from Google search result pages. I followed a course of actions to find out the possible causes and make changes to the website with the hope to get out of the Google penalty. Today, the website suddenly resurfaces with the original search result rankings.

During the previous several weeks, I did see a gradual increase in the ranking of the website for several important keywords. But one day, out of the blue, my website disappeared from the Google search result pages for the same keyword. Not a gradual failing but a sudden disappearance. Since most of the traffic to the website was referred by Google at that time, the disappearance would have a fatal impact on the website.

Why the Website Has Disappeared from Google?

This is the most pressing question for any website owners or marketers. Because of Google penalty? Because of server error? Because of robot.txt setting? There are hundreds of possibilities but Google would not tell you why (at least not in an easy to understand way). Being the webmaster, you need to really seek it out yourself.

I tried a number of actions to determine what went wrong and I recorded the steps below in case anyone would find it useful. The first few steps deal with the technical part of the website while the later parts deal with the contents of the website.

Update: My website was hit by Google Fred algo update a moment ago, losing more than 90% of the referral traffic from Google. Read here for what I have done to recover the website from Google Fred penalty. Please note that I have also gone through the steps listed here first.

Step 1 – Check if the Website is still in Google Index

This step helps to check whether Google has removed your website from its index completely. Go to Google and type site:[your domain url].

Reasons for the website not showing up:

• error in the robots.txt
• you should have committed something viewed as a crime in Google’s eyes and your website should have been banned by Google for severe breaching of the Google webmaster guidelines. This is a major issue. Otherwise, you might have offended Google lightly and likely get a rank suppression (there are rumours on the internet that Google might suppress your ranking by 50 or so). But that should be rare.

 

Step 2 – Check the Google Webmaster Tools for Warning and Error Messages

In case you have not used the Google Webmaster Tools before, the Google Webmaster Tools are a FREE tool provided by Google for webmasters to understand the visibility of the website on Google search and to receive messages from Google over penalties. Follow the instructions to set up your Google Webmaster Tools account now.

Access the Google Webmaster Tools account and find out if Google has sent you any message over your website. You can also check for abnormalities at these locations specified below. If your website gets banned by Google completely, you will likely find a message here. According to Google Search Engineer Matt Cutts, Google sends out around 400,000 manual action notices every month.

• Search Traffic > Manual Actions
• Security Issues

If penalty messages are found, do follow the advices by Google closely until all the problems have been fixed. Then you might file a reconsideration request with an apology letter to ask Google to include your website back into the search results.

Step 3 – Check the robot.txt Setting

The robot.txt is a simple plain text file which a detrimental effects on the search engine spider. It can be used to exclude part of the website OR all of the website from the search engine index.

The robot.txt is usually located at the root of the web server. Google Webmaster Tools also provides a simple tool for you to test the setting of the robot.txt at

• Crawl > Blocked URLs

You can press the test button at the bottom of the page to see if all the pages of the website are blocked. If so, you just need to change the robot.txt to allow search engine bots to access and index your pages as follows and ask Google to crawl your whole site again.

User-agent: *
Allow: /
Disallow: /hidden-directory

Step 4 – Check if Google Crawls the Website

Go to the Crawl Stats page of the Google Webmaster Tools:

• Crawl > Crawl Stats

to look at the graphs to see if Google has crawled your web pages lately. If the graphs indicate that your website has not been crawled by Google lately, there might be a problem with the web server on which your website is being hosted. In that case, seek technical help from the server administrator.

You can also try to fetch your webpages as Google to see if the contents can be read by Google at:

• Crawl > Fetch as Google

Press the red “Fetch” button and look for the fetch status, a green tick with the word “Success” indicates that your web server is functioning properly

Step 5 – Check for Low Quality Inbound Link

Google always refines its search engine algorithms to provide only quality results in the search results pages. They have taken action to track down paid link building (an act to buy links from others with a view to gain higher rankings in Google). The webmasters need to check for all the inbound link to see if they have many low quality inbound links to the website at:

• Search Traffic > Links to Your Site

How to determine if a link is low quality? Though there are many tools to help you, all of them are very expensive. The simplest way is to search for the web page the link is on to see if it is in Google’s index. If not, there is a high possibility that the link is considered a low quality link. You may:

• ask the owner of that site to remove the link from the website (preferred)
• use the disavow tool by Google (Google makes it clear that simply disavow the link is not enough, the webmaster to try to ask the website in question to remove the link whenever possible, understand more about disavow)

How to determine if a link is unnatural? Google may or may not tell you with manual action warning but a sudden sharp decrease in the search engine ranking does tell something about Google’s view on that. This article details the steps taken to find and remove unnatural links.

Step 6 – Check for Duplicate Contents

Google has indicated that they might or might not give penalties to sites carrying duplicate contents. Duplicate contents are block of text that are found not only on one webpage. Some websites just copy contents illegally from other websites to grow their contents and Google hates this. That’s why Google will “try” to find the original source of the content and remove other duplicate contents from the search result page.

In this step, we need to check the internet to see if the content of the website is being copied by others. The free tool Copyscape Plagiarism Checker comes in handy. Just enter your page URL individual to check if your contents appear in others’ website. If others have copied you, you might:

• send an email to the webmaster urging the content to be removed
• file a complaint with Google to remove the search record

Step 7 – Submit to Google Index Again

After going through all the steps above, you might still not be able to find the real cause for the website disappearance. You might just try to ask Google to include your website back by “Submit to Index” action.

• Crawl > Fetch as Google and then click the “Submit to Index” button with the option “URL and all linked pages”
• Sometimes you may see an error message in a red box. Try submitting again until Google has received your request.

Why the Website Suddenly Disappears from Google Search?

It is hoped that through the above steps, the webmaster should be able to find the reason for the sudden drop out of the search result records and take appropriate actions to remedy the situation. It is said that Google will reassess the websites to decide whether the penalties should be kept or not from time to time. The shortest time to lift the penalty is 48 hours while the longest time is forever.

However, things are not so clear as the above steps involve certain guessworks. It is possible that you will not find any wrongdoings on your side yet you are penalised. After searching though the internet using Google (ironically), I have found some further reasons:

• the link building work is too aggressive that Google puts the website on quarantine, this is especially applicable to new websites. It is said that Google will find human to visit the website to judge the quality of the website case by case.
• some algorithm errors after an update of the search algorithms (you might search the internet for Google update + [month and year] to see if there is a recent algorithm change)

The Result

Since Google accounts for more than 67% of internet search traffic, the disappearance from Google would have detrimental effects on the traffic of a website. If you have a website suddenly disappears from Google search, don’t panic. Do your normal content updates and link building as usual. As long as you publish quality contents, human visitors will know that and they will continue to share your URLs. Google will soon notice that and lift the ban on you.

The website got out of the ban after two weeks. But I have learnt to depend less on Google to reduce the damages should Google put another ban on my website.

Update: The website disappeared again a week later. After submitting the website to Google index, it re-appeared. Seems something weird has happened. This is usually the case when Google is updating its algorithm. If so, the only thing we can do is to wait.

It is hoped that this article would help to investigate why your Website Suddenly Disappears from Google Search and the actions needed to take in order to bring it back. It really takes time.

Update: My website was hit by Google Fred algo update a moment ago, losing more than 90% of the referral traffic from Google. Read here for what I have done to recover the website from Google Fred penalty.