Website optimisation - redirecting and deleting pages

Introduction

It helps to notify search engines when you move or remove a page, change websites, or when you have multiple domains to deal with. So when you:

create a new website on your existing domain with new page names
reorganise your website, maybe putting things in folders
delete a page because it’s no longer wanted
have a .com domain you want to redirect to .co.uk or vice-versa
have a primary domain of (say) mybiz.com that you want to appear when people put in the hyphenated my-biz.com
deprecate one website domain for another - typically when merging or splitting businesses

you will have to think about notifying search engines.

The mechanics

There is a mechanism built into web servers for telling browsers and search engines when a page has moved, or gone, through the use of HTTP response codes. There are two important ones for this topic:

301 Moved permanently – the page has been moved to a new location permanently
410 Gone – the page is no longer available and never will be again

The search engines understand these things implicitly. However the first of these might be invisible to a human visitor but in the second case there has to be a page to show the resource is now missing.

Implementing redirects and deletions

The implementation will depend upon your web server type but in principle the server detects requests for pages that are no longer around and responds with a web page that has the appropriate HTTP status code. In the case of a redirection, the server will load the correct page once the page with the redirection response has been sent. You may have spotted that when a redirection occurs, two web pages are sent but you only get to see the last.

On Microsoft’s IIS 7 and later, you employ the URL Rewrite facility to either map individual pages one by one – this is appropriate if you have created a new website with a different technology and the pages are similarly, but not exactly, named; or you can do a blanket change using wildcards or regular expressions; or a mix of both. It can get a bit technical so you may need to seek advice.

SEO

Moving a page

If you put in place a redirection for a specific page, the search engines should pick up on this and you will not have your ranking penalised by dint of that. Of course if the content of your new page has changed substantially, then the ranking may change - that's quite normal.

Replacing your old website with a new version

This is a time when lots of pages can change names though the content might be broadly, or even exactly, the same. A change in technology or a move to clean URLs may require a page name change and if you don’t put in the appropriate page redirections it’s perfectly possible that your website could fall down the rankings.

Moving your website to a different domain

If you are making significant changes to your business - say merging - the two websites you had before may be replaced by one. The redundant website should either have a server-level redirect to the new website or you could put a page up on the old website that notified the visitor for 5 or 10 seconds that a change has occurred after which it pushes them to the new one.

Alternative domain names

Google is particularly keen on not having duplicate websites and if you use multiple domains then you should make sure that they map onto a single domain using redirections.

If, for whatever reason, you have to have two domain names live, as we have with the alternative spellings of Montpelier/Montpellier:

allthemontpeliers.org
allthemontpelliers.org

then you can use a mechanism within the web page to tell the search engine which is the canonical version. In each page we put in something similar to the following where the highlighted page-name below was set to the correct value:

How long to leave redirections in place

Redirections

The search engines should pick up redirections quite quickly, but a visitor who has bookmarked a page may take a year or more to come back, so it’s a question of how valuable you think that those returning visitors will be.

In principle, it’s best to get rid of redirections as soon as you can because it lowers your maintenance burden. If you have a busy site, then three months is probably OK whereas a low traffic site might have to wait six months or more.

If you add new pages to your site, make sure that the names don’t clash with extant redirections.

Deleted pages

Our experience is that some search engine indexes can hang onto page references for along time even when marked as Gone. We're talking a year or more here. That said, there is no specific penalty for having a paged marked as 404 Not Found, so once again you keep it as long as you are prepared to shoulder the maintenance effort.