ASP.NET gets 401 Error on Apache FTP Server

One of our data suppliers is changing their FTP server from an IBM Domino HTTP server to an IBM Apache HTTP server. Username and password are required.

Our current production VB.NET 4.0 code cannot access the Apache server, getting an error:  (401) Unauthorized
I can login to the new server in IE11.

Our current code:
Dim pageHtml As String
Dim methodName As String = "ExtractFile.GetPageHtml: "

Dim myCred As New NetworkCredential(My.Settings.FTP_User, My.Settings.FTP_Password)
Dim wClient As New WebClient
wClient.Credentials = myCred
Dim pageData As Byte() = wClient.DownloadData(My.Settings.FTP_URL)
pageHtml = System.Text.Encoding.ASCII.GetString(pageData)
myCred = Nothing
wClient = Nothing

Open in new window


I also tried this with Digest. Same result:
Dim pageHtml As String = String.Empty
Dim methodName As String = "ExtractFile.GetPageHtml: "
Dim Url As String = "https://test.vandalay.org/ftp/filename"
Dim wc As New System.Net.WebClient
Dim cc As CredentialCache = New CredentialCache
cc.Add(New Uri(Url), "Digest", New System.Net.NetworkCredential("username", "password", "vandalay.org"))
wc.Credentials = cc
Dim Eek As String = wc.DownloadString(Url) 

Open in new window

4ncidAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

4ncidAuthor Commented:
Anyone?
Bob LearnedCommented:
What type of authentication is the FTP site using?  I would have assumed that your current code would have worked.  You might be able to use an HTTP debugger, like Fiddler, to see how it works with IE11.  The web browser can negotiate with the FTP, and provide the correct authentication, that you would need to duplicate in code.
4ncidAuthor Commented:
The web/FTP site is run by the state of NC and they offer just about zero support. They told me the credentials are the same as the production server, and if it does work my code is at fault.
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

Bob LearnedCommented:
Do you know how to debug a web session with Fiddler?
4ncidAuthor Commented:
I have not used it before but just installed and looking through it right now. I can see my https session to the FTP server, but unsure where to find authentication statements.
Dave BaldwinFixer of ProblemsCommented:
In IE11, are you connecting thru 'ftp://' or 'https://'?  And how are you entering your credentials?
Bob LearnedCommented:
Start by looking at the request headers.
4ncidAuthor Commented:
https://test.something.org/ftp/filename

Logon popup window is presented. see attached
Logon-box.jpg
Dave BaldwinFixer of ProblemsCommented:
And after you are logged in, does the url start with 'ftp' or 'https'?
4ncidAuthor Commented:
URL stays as shown above, starting with https.
Bob LearnedCommented:
That looks like basic authentication, which is part of the request headers.

Example:

string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(user + ":" + password));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;

Open in new window

4ncidAuthor Commented:
Copy of Fiddler log attached.  Line 3 is when IE11 has attached & logon screen is presented.  By line 13 it appears to have authenticated and is downloading con tent.
Request-headers.jpg
Bob LearnedCommented:
Under the Cookies / Login section you can see the basic authentication header value that I described earlier.
4ncidAuthor Commented:
You are referring to these lines? Unsure how that relates to the .NET code.

AOC-FTP-cookies.jpg
Bob LearnedCommented:
You have a WebClient, and you need to pass the Basic Authentication values for user name and password, so I showed you the two lines required.  You just need to replace user and password with the values that you need.

string credentials = Convert.ToBase64String(Encoding.ASCII.GetBytes(user + ":" + password));
client.Headers[HttpRequestHeader.Authorization] = "Basic " + credentials;

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
4ncidAuthor Commented:
Turns out the new server expects a "/" at the end of the URL.  The browsers would automatically add, but the .NET code crashed.

Thanks for the input.
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Apache Web Server

From novice to tech pro — start learning today.