I was given a specific list of url's that I need to spider with a .NET, VB Program. The problem is that many of the sites are in a foreign language.
The question is, is there a way for my spider to detect the page's language? Maybe through the server's response headers? Also, what is a good option for automatic translations? I thought about Systran, but really don't know if it is a good, cost effective option.
Thanks a lot!