asymptoticdesign.co.uk - tips for web design and SEO

HTTP status in the server header

The HTTP response status code in the server header indicates to user agents (bots and browsers) the status of HTTP response following the HTTP request of URLs. Some common HTTP status codes are:

It is important for a website to return correct HTTP status response, for example HTTP status response 500 (Internal Server Error) in the server header when there is a server error at generating the requested URL. If a URL that causes a server error is returning HTTP status 500 (Internal Server Error) in the server header, it is not indexed by search engines, but if it returns HTTP status response 200 (OK) it can be indexed in search results, and usually it will contain mainly an error message, that would be of no use to people visiting the site. A correct HTTP status response at server error can be implemented with a comprehensive error catching algorithm in the server-side scripts used by a website to generate content.

Some problems in the way search engines deal with HTTP status response of URLs can be caused by the way the HTTP request and HTTP response deal with the type of HTTP protocol, HTTP/1.1 or HTTP/1.0. For example Google bots make usually HTTP/1.1 requests, Yahoo HTTP/1.0 requests, some MSN bots make HTTP/1.1 requests and others make HTTP/1.0 requests, and there might be some difference in which they deal with HTTP status response codes specific to HTTP/1.0 or HTTP/1.1 protocols, like HTTP status response 410 (Gone), that is specific to the HTTP/1.1 protocol.

reference links

tools