Solved

Strange URL's in Google's Webmaster Tools  ASP.NET

Posted on 2013-02-07
4
437 Views
Last Modified: 2013-02-07
Googles webmaster tools has several hundred "not found" errors on my site, (my site is IIS7 and uses a lot of URL Re-writes)
The 404 errors are the result of Google spidering pages on my site that are in the form of:
www.mysite.com/mypage.aspx/

The trailing dash "/" after the .aspx is what is screwing up all the relative hyperlinks on the page.  There should be no trailing dash.
www.mysite.com/mypage.aspx  = CORRECT
www.mysite.com/mypage.aspx/ =  How in the world did Google "find" this incorrect page?

 I do not know how Google reached any page on my site with a trailing dash.  What kind of code behind (I use VB.NET) can I use to purify my page's URL to remove a trailing dash?  I'm thinking there has to be some kind of code I can use to scan the incoming URL, strip a trailing dash, and tell Google what the true URL is?  

Even better...(this would be extra credit, virtual of course :)  )....is there any way to find out how Google is reaching these pages?  I"d love to know what page(s) linked to pages with trailing dashes.  But the problem is that if Google finds even a single page with a trailing dash, suddenly, all relative hyperlinks are messed up, and Google starts a cascading, exponential spidering process of messed up pages.
0
Comment
Question by:arthurh88
  • 2
  • 2
4 Comments
 
LVL 26

Accepted Solution

by:
Alan Warren earned 500 total points
Comment Utility
Hi Arthur,
had a similar issue a few months back, ended up incorporating something like this into my master page code behind:
        ' 2012Apr05: Getting requests for invalid paths (867 on the day)
        ' e.g. /browse.aspx/util/profiles/video/help/video/profiles/viewprofile.aspx
        ' redirecting using an absolute path can cause the  HTTP_ASPFILTERSESSIONID to be appended to the url
        ' http://books.google.com.ph/books?id=k0nZmXc-yQgC&pg=PA318&lpg=PA318&dq=HTTP_ASPFILTERSESSIONID+appended+to+url&source=bl&ots=Pv4K6d4g2N&sig=0cC0C094uVVpiI3sj4vIZqtnHg8&hl=en&sa=X&ei=8FXrUOvRCIfrkgW05YCYDA&redir_esc=y#v=onepage&q=HTTP_ASPFILTERSESSIONID%20appended%20to%20url&f=false

        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            context.Response.StatusCode = 404
            context.Response.StatusDescription = "Page not found"
            context.Response.Status = "404 Page not found"
            context.Response.Redirect("~/", True)
        End If

Open in new window

Alan
0
 

Author Comment

by:arthurh88
Comment Utility
fantastic.  and here is my code that works like a charm, based off of what you said


 Dim url As String = Request.RawUrl
        If InStr(url, ".aspx/", CompareMethod.Text) > 0 Then
            Context.Response.StatusCode = 404
            Context.Response.StatusDescription = "Page not found"
            Context.Response.Status = "404 Page not found"
            Context.Response.Redirect(Strings.Left(url, Strings.InStr(url, ".aspx/") + 4), True)
        End If
0
 
LVL 26

Expert Comment

by:Alan Warren
Comment Utility
I like the use of the Left() function to trim off the redundancy!

Success with your app!

Alan ";0)
0
 

Author Comment

by:arthurh88
Comment Utility
yea i had to use the left cuz some urls in Google's "not found" list go on and on

www.mysite.com/mypage.aspx/blahblah/blahblah/(keeps going lol)
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Creating an analog clock UserControl seems fairly straight forward.  It is, after all, essentially just a circle with several lines in it!  Two common approaches for rendering an analog clock typically involve either manually calculating points with…
Problem Hi all,    While many today have fast Internet connection, there are many still who do not, or are connecting through devices with a slower connect, so light web pages and fast load times are still popular.    If your ASP.NET page …
Internet Business Fax to Email Made Easy - With eFax Corporate (http://www.enterprise.efax.com), you'll receive a dedicated online fax number, which is used the same way as a typical analog fax number. You'll receive secure faxes in your email, fr…
Get a first impression of how PRTG looks and learn how it works.   This video is a short introduction to PRTG, as an initial overview or as a quick start for new PRTG users.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now