Traditional (Hard) 404 Errors
Simply put, 404 errors are ‘Not Found‘ error message. It is the standard HTTP response code, indicating that communication was made with the server but the page could not be found. A 404 error often occurs when a page has been moved or deleted. Such errors are very common on websites- especially large sites with hundreds or thousands of pages. Although they can be confusing to users, they are not the end of the world. They just need to be managed and their impact minimized.
Soft 404 Errors
A Soft 404 Error occurs when communication is made with the server, but a standard web page is returned with the “200 OK” response code. Often, it returns the homepage of the site with the error message- really confusing the visitor. Additionally, search engines may crawl and index the page as Google states below:
“furthermore search engines may spend much of their time crawling and indexing non-existent, often duplicative URLs on your site. This can negatively impact your site’s crawl coverage—because of the time Googlebot spends on non-existent pages, your unique URLs may not be discovered as quickly or visited as frequently.”
– Google Webmaster Blog, Aug. 12, 2008
Soft 404 errors are discouraged, because they confuse users and search engines. Users see a page with all the branding and design of your site, but instead of content they get an error code that they don’t understand. In the case of hard 404 errors, at least they are told that the page doesn’t exist and savvy internet users usually hit the back button in the browser (although often they just leave, hence the custom 404 page, see below). But with a 200 ok error they will see a bunch of stuff they don’t understand, like the server info, content type, etc.
What is An SEO to Do?
So, you know 404 errors are not preferred, and that soft 404 errors are even worst. 404 errors are never a good user experience and search engines don’t want to reward sites that have several of these errors (although Google makes clear it’s not a direct ranking factor- bottom line, they understand 404’s are going to happen).What is to be done? Let’s take a look at some helpful options.
Google Webmaster Tools Will Help
Use Fetch as Googlebot– a tool provided by Google, under Diagnostics on your sites Webmaster Tools page, that lets you see a page as the Googlebot crawler sees it. You will be able to determine if the correct 404 Page Not Found error is being returned for the specific URL. Also, Soft 404 errors are now included on your ‘crawl errors’ report in Google Webmaster Tools. You can correct the soft errors by following Google’s outlined steps below:
1. For the soft 404s, determine whether the URL:
a. Contains the correct content and properly returns a 200 response
b. Should 301 redirect to a more accurate URL
c. Doesn’t exist and should return a 404 or 410 response
2. Confirm that you’ve configured the proper HTTP Response by using Fetch as Googlebot in Webmaster Tools
3. If you now return 404s, you may want to customize your 404 page to aid y our users. Google’s custom 404 widget can help.
Create a customized 404 error page. Any large site will eventually have a 404 error or two, it’s not a matter of if- but when. So, create a customized 404 error page to explain the situation to those who stumble upon the page. Read our post about handling and customizing your 404 error pages to improve the user experience.
Helpful References:
– Google Webmaster Central: Soft 404 Errors
– Google Webmaster Central Blog: Farewell to Soft 404s
– Google Webmaster Central Blog: Crawl Errors Now Reports Soft 404s- Fixing soft 404 errors
Feedback: When do you see traditional and soft 404 errors? How do you avoid soft 404 errors? What are your favorite custom 404 error pages?