Making web requests from behind a proxy

Posted on 2011-03-11
Last Modified: 2012-05-11
Hi, Experts,

This will be a long post, but I figure you'll do better with more information than with less.

The general questions:
How do I set a .NET 2.0 (C#, if it matters) project to use the user's default proxy settings?  Since I don't have any access to any proxy with authentication required, is there any way to test proxy support?

The details:
I have some software that makes calls to a web service.  It works perfectly fine in all of my tests.  A customer bought it, and it was failing with HTTP error 407 - proxy authentication required.

I needed to learn about .NET and proxy support.  I found an outdated msdn article here.  It tells me the process that .NET 2.0 goes through to figure out proxy settings.

In what little I can find on the web, I've been very surprised that WebRequest doesn't automatically look at your proxy settings.  StackOverflow shows a way that isn't overly inconvenient - simply setting the following in your app.config:
       <proxy autoDetect="true" />

Open in new window

Obviously, if this was the perfect solution, I'd question why Microsoft doesn't set proxy autoDect to true as the default.
But it wasn't the perfect solution.  I sent the new application to my customer, and he reported that he got a timeout error.  I'm guessing that some part of the proxy autodetection timed out, so the whole sequence failed.

I like the idea of fixing it in the app.config rather than in the C# code directly, so I tried removing the autodetect setting and instead using the useSystemDefault setting, (i.e.)
<proxy useSystemDefault="True" />
I would be fairly certain that this should work, but the customer reported that the WebRequest again returned a timeout error.

It is extremely possible that he didn't run the newest program I've sent him.  He's a regular user, not a programmer, but I don't know any other way to work on the proxy issue other than through him.  I can ask him to uninstall and reinstall the latest, just to be sure, but he's now out of his office until Monday.
Over the weekend, I'd like to come up with one final test for him - i.e., one that works.

I know that I can also rewrite all the WebRequests to include a Proxy setting, such as this example on stackoverflow.  Since there are several web requests, I've been reluctant to try that, unless I was certain of what proxy settings to use.

If I could unit test this on my own, I'd be ok with shooting in the dark and playing trial and error until I got it working, but having to send each test to a user and having each test fail is making me look worse and worse.

I know the customer can surf the web ok, so he has his web-browser's proxy set up correctly.

Does anyone have a recommendation as to how I can do my own testing?  I need to be behind a proxy that requires authentication.  I've considered using web-anonymizers, but they don't require authentication.

Does anyone have any experience in this and can tell me the proper way to set the proxy?

What's the difference between WebRequest.DefaultWebProxy and WebRequest.GetSystemWebProxy?  The msdn documentation (yes, I know this if for .NET 4.0.  The documentation for earlier versions says far less, which is part of my confusion.)  says that DefaultWebProxy checks the app.config then the IE proxy settings while GetSystemWebProxy only checks the IE proxy settings.

What happens if the user is using a different browser?  Do all the proxy settings occur in the same place in the registry, or can it be that his firefox runs fine and his IE is not set up properly?

That's a torrent of questions, but all highly related.  If anyone feels this is too much, I'm happy to branch it off into separate questions.

Thanks for reading this, and thanks in advance for any help you can offer.
Question by:Carnou
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2

Expert Comment

ID: 35119494
LVL 10

Author Comment

ID: 35119611
Thanks for the effort.

I saw that thread before posting here.  As tjmoore responded to the accepted answer, there are a few problems with using the command line solution - namely that the users are in a restricted environment, and I shouldn't expect them to run a command-line to make my program work.

That post refers to GetDefaultProxy, which has since been depracated by DefaultWebProxy and GetSystemWebProxy.

I can see if the user is capable of running netsh winhttp.  It won't be the ideal solution, but it may be acceptable.

The ProxyElement class seems to match what I was setting via the config file.  Do you know more about it?

If the user can run netsh winhttp, and it succeeds for him, and no one has any better ideas, I'll give this credit.  I'm still hoping someone could give me more answers than questions, though.

Accepted Solution

r3nder earned 500 total points
ID: 35123320
Basically you can run netsh commands for winhttp in a batch file or script - for that matter you can call the batch file in C# (it is just a couple of lines of code) and no on is the wiser

Here is an example of the C# used to call the batch

public void connectHTTP()
            string DriveLetter = Path.GetPathRoot(Environment.GetFolderPath(Environment.SpecialFolder.System));
            string Username = txtBxUser.Text.ToString();
            string Password = txtBxPass.Text.ToString();
            string args = @"/c HTTP.bat " + Username + " " + '"' + Password + '"';

            System.Diagnostics.Process p = new System.Diagnostics.Process();

            p.StartInfo.Arguments = args;
            p.StartInfo.UseShellExecute = false;
            p.StartInfo.FileName = DriveLetter + "windows\\system32\\cmd.exe";


Open in new window

Congratulations! You’re Certified – Now What?

Starting a new career can be overwhelming. Becoming certified in your field of expertise is a great start, but where do you go from here?  Here are some tips to help you on your career journey.

LVL 10

Author Comment

ID: 35167440
For the record, before moderators force this closed, the question isn't abandoned.  I've sent the suggestion that r3nder provided a link to to my customer, but I haven't heard back from the customer as to the results.

I asked two general questions in my original post:
"How do you set a .NET project to use the default proxy settings?"  r3nder sort of answered this, though the link provided would more accurately answer "How do you configure Windows to forward its proxy settings to .NET."  I'm waiting for the customer's response to know if r3nder's answer is a good one.  I still don't think the answer is "right" in that it doesn't make sense for me to have to force an execution of a batch file with every program I write that accesses the web.  There should be a simple setting within .NET itself.

The second question was: "How do you test proxy settings when you yourself are not behind an authenticated proxy?"  This one was more important, as it would allow me to test r3nder's answer and/or do experiments on my own.  It has not had any answer.

Mods - should I branch the second question off as a separate but related question?

Expert Comment

by:Tom Knowlton
ID: 36116040
I've requested that this question be deleted for the following reason:

This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
LVL 10

Author Comment

ID: 36116032
Sorry, I actually was planning to close this this weekend.  

The customer stopped working with me, and the issue eventually got abandoned by my company, so I was never able to verify r3nder's answer.

The first link was worthless to me - my posting should have made it obvious that simply copying a link from a basic google search wasn't enough.

The code that r3nder submitted may have been useful; since I wasn't able to test, I don't really know.  I'll close the question and reward points to r3nder - the answer may be useful to someone else, so it's worth adding into the knowledge base, I suppose.  I can't award an A for this, though.

LVL 10

Author Comment

ID: 36116041
I've never had to deal with a closure request, and didn't realize I had to click on Object.  Please see my note above.  I would accept r3nder's comment 35123320 for full points, though I feel it should be awarded a C.  Due to my customer no longer participating, I wasn't able to validate the answer, and no one addressed my underlying question of how to test as if I was behind a proxy when I'm not behind one.

LVL 10

Author Closing Comment

ID: 36116042
See my comments above for my grading and reasoning.

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

A long time ago (May 2011), I have written an article showing you how to create a DLL using Visual Studio 2005 to be hosted in SQL Server 2005. That was valid at that time and it is still valid if you are still using these versions. You can still re…
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
This tutorial will teach you the special effect of super speed similar to the fictional character Wally West aka "The Flash" After Shake : All lightning effects with instructions : http://www.mediaf…
Sometimes it takes a new vantage point, apart from our everyday security practices, to truly see our Active Directory (AD) vulnerabilities. We get used to implementing the same techniques and checking the same areas for a breach. This pattern can re…

632 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question