Want to remove URL from Googlebot search but can't recreate exact URL

Posted on 2011-10-31
Last Modified: 2013-11-18
I'm in charge of maintaining a website that has three underlying CMS MS Access databases and recently deleted a number of old records from the CMS as they were either very old news stories or job vacancies being publicised.

Google Webmaster Tools is showing that we have severe health issues with the site citing a number of currently "unreachable" sites (500 error).

I have gone to use the "Remove URL" tool but Google's support pages state:

To remove a page or image, you must do one of the following:
      1•      Make sure the content is no longer live on the web. Requests for the page must return an HTTP 404 (not found) or 410 status code.
      2•      Block the content using a robots.txt file.
      3•      Block the content using a meta noindex tag.

These are my perceived issues in meeting the above criteria:

1.  Can't create a page with URLs in questions and give them generic 404 error/410 error content as part of the URL is made up of the database record's ID field.  The field in question has an AutoNumber datatype so I can't manually enter new IDs to recreate the URLs.

2.  I am not keen on blocking with robots.txt as the URLs are sequentially numbered and (my understanding is) if I state that I want to block URL then anything with this URL that has a sequential number starting with 7 will be blocked going forward, i.e: ?id=70, ?id=700 etc and I don't wish for those to be blocked.

3.  Don't think I can block content using a meta noindex tag for the reasons given in 1.

I would appreciate guidance with this as I don't want the website's ranking to suffer in anyway due to erroneous URLs.

Many thanks in advance.
Question by:CDIT_Solutions
    LVL 53

    Expert Comment

    I assume you are sending the 500 responses because the query to the database gets an error.  Change the query so that on error it re-directs to your 404 page.
    LVL 12

    Accepted Solution

    Another strategy that you could use would be to send a 301 redirect response code and then redirect Google/Bing search engine bots AND actual users from that "old" page that you've removed to your website's home page, or to a page created to take advantage of users coming into your website.

    If your "old" pages had any search engine link juice with other websites pointing to those pages, this will redirect some of that "juice" within Google/Bing's algorithm to help out your website.

    This is generally a good (best) method to enhance the Search Engine Optimization (SEO) value of those old pages in case anyone has links out on the internet pointing to them.

    Let us know of any other follow-up questions...

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Highfive + Dolby Voice = No More Audio Complaints!

    Poor audio quality is one of the top reasons people don’t use video conferencing. Get the crispest, clearest audio powered by Dolby Voice in every meeting. Highfive and Dolby Voice deliver the best video conferencing and audio experience for every meeting and every room.

    Foreword (July, 2015) Since I first wrote this article, years ago, a great many more people have begun using the internet.  They are coming online from every part of the globe, learning, reading, shopping and spending money at an ever-increasing ra…
    With a software solution, you always get to control email signatures and have the final say on the design - not an unrelated department or the uninterested end user.
    This tutorial walks through the best practices in adding a local business to Google Maps including how to properly search for duplicates, marker placement, and inputing business details. Login to your Google Account, then search for "Google Mapmaker…
    The viewer will learn how to look for a specific file type in a local or remote server directory using PHP.

    761 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    11 Experts available now in Live!

    Get 1:1 Help Now