Solved

Strange URL's in Google's Webmaster Tools  ASP.NET

Posted on 2013-02-07
4
440 Views
Last Modified: 2013-02-07
Googles webmaster tools has several hundred "not found" errors on my site, (my site is IIS7 and uses a lot of URL Re-writes)
The 404 errors are the result of Google spidering pages on my site that are in the form of:
www.mysite.com/mypage.aspx/

The trailing dash "/" after the .aspx is what is screwing up all the relative hyperlinks on the page.  There should be no trailing dash.
www.mysite.com/mypage.aspx  = CORRECT
www.mysite.com/mypage.aspx/ =  How in the world did Google "find" this incorrect page?

 I do not know how Google reached any page on my site with a trailing dash.  What kind of code behind (I use VB.NET) can I use to purify my page's URL to remove a trailing dash?  I'm thinking there has to be some kind of code I can use to scan the incoming URL, strip a trailing dash, and tell Google what the true URL is?  

Even better...(this would be extra credit, virtual of course :)  )....is there any way to find out how Google is reaching these pages?  I"d love to know what page(s) linked to pages with trailing dashes.  But the problem is that if Google finds even a single page with a trailing dash, suddenly, all relative hyperlinks are messed up, and Google starts a cascading, exponential spidering process of messed up pages.
0
Comment
Question by:arthurh88
  • 2
  • 2
4 Comments
 
LVL 26

Accepted Solution

by:
Alan Warren earned 500 total points
ID: 38865928
Hi Arthur,
had a similar issue a few months back, ended up incorporating something like this into my master page code behind:
        ' 2012Apr05: Getting requests for invalid paths (867 on the day)
        ' e.g. /browse.aspx/util/profiles/video/help/video/profiles/viewprofile.aspx
        ' redirecting using an absolute path can cause the  HTTP_ASPFILTERSESSIONID to be appended to the url
        ' http://books.google.com.ph/books?id=k0nZmXc-yQgC&pg=PA318&lpg=PA318&dq=HTTP_ASPFILTERSESSIONID+appended+to+url&source=bl&ots=Pv4K6d4g2N&sig=0cC0C094uVVpiI3sj4vIZqtnHg8&hl=en&sa=X&ei=8FXrUOvRCIfrkgW05YCYDA&redir_esc=y#v=onepage&q=HTTP_ASPFILTERSESSIONID%20appended%20to%20url&f=false

        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            context.Response.StatusCode = 404
            context.Response.StatusDescription = "Page not found"
            context.Response.Status = "404 Page not found"
            context.Response.Redirect("~/", True)
        End If

Open in new window

Alan
0
 

Author Comment

by:arthurh88
ID: 38866155
fantastic.  and here is my code that works like a charm, based off of what you said


 Dim url As String = Request.RawUrl
        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            Context.Response.StatusCode = 404
            Context.Response.StatusDescription = "Page not found"
            Context.Response.Status = "404 Page not found"
            Context.Response.Redirect(Strings.Left(url, Strings.InStr(url, ".aspx/") + 4), True)
        End If
0
 
LVL 26

Expert Comment

by:Alan Warren
ID: 38866205
I like the use of the Left() function to trim off the redundancy!

Success with your app!

Alan ";0)
0
 

Author Comment

by:arthurh88
ID: 38866241
yea i had to use the left cuz some urls in Google's "not found" list go on and on

www.mysite.com/mypage.aspx/blahblah/blahblah/(keeps going lol)
0

Featured Post

DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

For those of you who don't follow the news, or just happen to live under rocks, Microsoft Research released a beta SDK (http://www.microsoft.com/en-us/download/details.aspx?id=27876) for the Xbox 360 Kinect. If you don't know what a Kinect is (http:…
International Data Corporation (IDC) prognosticates that before the current the year gets over disbursing on IT framework products to be sent in cloud environs will be $37.1B.
This Micro Tutorial will give you a basic overview how to record your screen with Microsoft Expression Encoder. This program is still free and open for the public to download. This will be demonstrated using Microsoft Expression Encoder 4.
Internet Business Fax to Email Made Easy - With  eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, f…

912 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

25 Experts available now in Live!

Get 1:1 Help Now