Solved

Strange URL's in Google's Webmaster Tools  ASP.NET

Posted on 2013-02-07
4
445 Views
Last Modified: 2013-02-07
Googles webmaster tools has several hundred "not found" errors on my site, (my site is IIS7 and uses a lot of URL Re-writes)
The 404 errors are the result of Google spidering pages on my site that are in the form of:
www.mysite.com/mypage.aspx/

The trailing dash "/" after the .aspx is what is screwing up all the relative hyperlinks on the page.  There should be no trailing dash.
www.mysite.com/mypage.aspx  = CORRECT
www.mysite.com/mypage.aspx/ =  How in the world did Google "find" this incorrect page?

 I do not know how Google reached any page on my site with a trailing dash.  What kind of code behind (I use VB.NET) can I use to purify my page's URL to remove a trailing dash?  I'm thinking there has to be some kind of code I can use to scan the incoming URL, strip a trailing dash, and tell Google what the true URL is?  

Even better...(this would be extra credit, virtual of course :)  )....is there any way to find out how Google is reaching these pages?  I"d love to know what page(s) linked to pages with trailing dashes.  But the problem is that if Google finds even a single page with a trailing dash, suddenly, all relative hyperlinks are messed up, and Google starts a cascading, exponential spidering process of messed up pages.
0
Comment
Question by:arthurh88
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
4 Comments
 
LVL 26

Accepted Solution

by:
Alan Warren earned 500 total points
ID: 38865928
Hi Arthur,
had a similar issue a few months back, ended up incorporating something like this into my master page code behind:
        ' 2012Apr05: Getting requests for invalid paths (867 on the day)
        ' e.g. /browse.aspx/util/profiles/video/help/video/profiles/viewprofile.aspx
        ' redirecting using an absolute path can cause the  HTTP_ASPFILTERSESSIONID to be appended to the url
        ' http://books.google.com.ph/books?id=k0nZmXc-yQgC&pg=PA318&lpg=PA318&dq=HTTP_ASPFILTERSESSIONID+appended+to+url&source=bl&ots=Pv4K6d4g2N&sig=0cC0C094uVVpiI3sj4vIZqtnHg8&hl=en&sa=X&ei=8FXrUOvRCIfrkgW05YCYDA&redir_esc=y#v=onepage&q=HTTP_ASPFILTERSESSIONID%20appended%20to%20url&f=false

        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            context.Response.StatusCode = 404
            context.Response.StatusDescription = "Page not found"
            context.Response.Status = "404 Page not found"
            context.Response.Redirect("~/", True)
        End If

Open in new window

Alan
0
 

Author Comment

by:arthurh88
ID: 38866155
fantastic.  and here is my code that works like a charm, based off of what you said


 Dim url As String = Request.RawUrl
        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            Context.Response.StatusCode = 404
            Context.Response.StatusDescription = "Page not found"
            Context.Response.Status = "404 Page not found"
            Context.Response.Redirect(Strings.Left(url, Strings.InStr(url, ".aspx/") + 4), True)
        End If
0
 
LVL 26

Expert Comment

by:Alan Warren
ID: 38866205
I like the use of the Left() function to trim off the redundancy!

Success with your app!

Alan ";0)
0
 

Author Comment

by:arthurh88
ID: 38866241
yea i had to use the left cuz some urls in Google's "not found" list go on and on

www.mysite.com/mypage.aspx/blahblah/blahblah/(keeps going lol)
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

ASP.Net to Oracle Connectivity Recently I had to develop an ASP.NET application connecting to an Oracle database.As I am doing it first time ,I had to solve several problems. This article will help to such developers  to develop an ASP.NET client…
Real-time is more about the business, not the technology. In day-to-day life, to make real-time decisions like buying or investing, business needs the latest information(e.g. Gold Rate/Stock Rate). Unlike traditional days, you need not wait for a fe…
How to Install VMware Tools in Red Hat Enterprise Linux 6.4 (RHEL 6.4) Step-by-Step Tutorial

749 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question