asked on

URL-rewriting for multilingual search engine optimization of ASP.net site with localization

I have created a multilingual website in ASP.net with some of the localized text pulled from a database and some from resource files and it works like a charm.

The site first checks whether the visitor has cookie information about the preferred language. If not, it checks the Page.Request.UserLanguages for the preferred language and tries to match that. It then also attempts to place a cookie. If the visitor wants yet another language, he/she can select a different language through a linkbutton (JavaScript) to change the language selected and the site also updates the cookie accordingly.

A possible problem is that the url of any particular page remains the same irrespective of the language served. This seems to be a problem for search engines because they would have different content (the same content in a different language) for the same url.

Also, if we don't know anything about the preferred language and the visitor is not making any selection, we will always and only serve english. Even if the spider would be able to select a language through the linkbutton we would continue to serve english because we can't place a cookie and a spider has no "UserLanguage" parameter in its requests. (Or does it?)

Now, as a consequence, I believe that I have to handle the user's conscious language selection through a plain html link with no JavaScript for spiders to select the language because spiders don't process JavaScript.

I also assume that I have to make each page available with different url for different languages being served. Of course, I don't actually want to create separate pages for the same stuff. What is the best way do rewrite the urls? I hope there is some way to request this in the web config or global.asax file with some instruction like:

if request.path contains "/de/" replace "/de/" serve document found at "(path found after "/de/")" but leave all links in the document unchanged.

This way, I could test for the requested urls what language they ask for and set the current thread's culture accordingly. I assume this would work much smoother than with request parameters (as in "...aspx?language=de") because such request parameters won't be in the links within the served page (or can they?) and some spiders might have trouble with parameters (the "dynamic pages problem").

Now, please tell me how to do such an url rewriting so that, for example, all german pages have a url with /de/ after the domain name (and english pages with an /en/) but I don't actually have to create and manage all those subdirectories.

Thank you!

ASKER CERTIFIED SOLUTION

ihenry

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ddetering

ASKER

Funny - I didn't receive notification about this answer... Just wanted to post my new approach which I hope will work and solve my problems in a better way than rewriting the url. Nevertheless, ihenry, I'll check your answer and award the points to you if you deserve them ;)

So, as a matter of fact, I am directing multiple international domains to the site anyway. The simple answer is that I can leave the language selection javascript-based _in order_ to prevent search robots from using it. For each Application_BeginRequest (or will it be sufficient for each Session_Start) event I first check whether there is a cookie with language preferences (robots don't have a cookie). If not, I see whether there is a preferred UserLanguage that I can match (robots don't have that either). If not, I check whether the client requests an international domain:

ElseIf CurrentPage.Request.ServerVariables("server_name").ToString.IndexOf("domain.de") > -1 Then
Thread.CurrentThread.CurrentCulture = New CultureInfo("de-DE")

Else (and finally,) I serve the default english content.

As a result, I can submit the international domains and be confident that the robots will index my site only once for each language, and each language has a different url for the same page in the search engine results and the urls in the search engine results also match the expectation of most search engine users.

Possible problems will only arise once I add languages that don't match as nicely an international domain, e.g. Spanish for hispanics in the US or french for francophones in Canada. I guess I will then finally have to apply url rewriting.

ddetering

ASKER

I think this will work when I eventually will need it, but I haven't tried it yet. I particularly like the first article. However, I had thought that this url rewriting would work with just a few lines of code in the global.asax, but I guess not...