Still celebrating National IT Professionals Day with 3 months of free Premium Membership. Use Code ITDAY17


Making web requests from behind a proxy

Posted on 2011-03-11
Medium Priority
Last Modified: 2012-05-11
Hi, Experts,

This will be a long post, but I figure you'll do better with more information than with less.

The general questions:
How do I set a .NET 2.0 (C#, if it matters) project to use the user's default proxy settings?  Since I don't have any access to any proxy with authentication required, is there any way to test proxy support?

The details:
I have some software that makes calls to a web service.  It works perfectly fine in all of my tests.  A customer bought it, and it was failing with HTTP error 407 - proxy authentication required.

I needed to learn about .NET and proxy support.  I found an outdated msdn article here.  It tells me the process that .NET 2.0 goes through to figure out proxy settings.

In what little I can find on the web, I've been very surprised that WebRequest doesn't automatically look at your proxy settings.  StackOverflow shows a way that isn't overly inconvenient - simply setting the following in your app.config:
       <proxy autoDetect="true" />

Open in new window

Obviously, if this was the perfect solution, I'd question why Microsoft doesn't set proxy autoDect to true as the default.
But it wasn't the perfect solution.  I sent the new application to my customer, and he reported that he got a timeout error.  I'm guessing that some part of the proxy autodetection timed out, so the whole sequence failed.

I like the idea of fixing it in the app.config rather than in the C# code directly, so I tried removing the autodetect setting and instead using the useSystemDefault setting, (i.e.)
<proxy useSystemDefault="True" />
I would be fairly certain that this should work, but the customer reported that the WebRequest again returned a timeout error.

It is extremely possible that he didn't run the newest program I've sent him.  He's a regular user, not a programmer, but I don't know any other way to work on the proxy issue other than through him.  I can ask him to uninstall and reinstall the latest, just to be sure, but he's now out of his office until Monday.
Over the weekend, I'd like to come up with one final test for him - i.e., one that works.

I know that I can also rewrite all the WebRequests to include a Proxy setting, such as this example on stackoverflow.  Since there are several web requests, I've been reluctant to try that, unless I was certain of what proxy settings to use.

If I could unit test this on my own, I'd be ok with shooting in the dark and playing trial and error until I got it working, but having to send each test to a user and having each test fail is making me look worse and worse.

I know the customer can surf the web ok, so he has his web-browser's proxy set up correctly.

Does anyone have a recommendation as to how I can do my own testing?  I need to be behind a proxy that requires authentication.  I've considered using web-anonymizers, but they don't require authentication.

Does anyone have any experience in this and can tell me the proper way to set the proxy?

What's the difference between WebRequest.DefaultWebProxy and WebRequest.GetSystemWebProxy?  The msdn documentation (yes, I know this if for .NET 4.0.  The documentation for earlier versions says far less, which is part of my confusion.)  says that DefaultWebProxy checks the app.config then the IE proxy settings while GetSystemWebProxy only checks the IE proxy settings.

What happens if the user is using a different browser?  Do all the proxy settings occur in the same place in the registry, or can it be that his firefox runs fine and his IE is not set up properly?

That's a torrent of questions, but all highly related.  If anyone feels this is too much, I'm happy to branch it off into separate questions.

Thanks for reading this, and thanks in advance for any help you can offer.
Question by:Carnou
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 5
  • 2

Expert Comment

ID: 35119494
LVL 10

Author Comment

ID: 35119611
Thanks for the effort.

I saw that thread before posting here.  As tjmoore responded to the accepted answer, there are a few problems with using the command line solution - namely that the users are in a restricted environment, and I shouldn't expect them to run a command-line to make my program work.

That post refers to GetDefaultProxy, which has since been depracated by DefaultWebProxy and GetSystemWebProxy.

I can see if the user is capable of running netsh winhttp.  It won't be the ideal solution, but it may be acceptable.

The ProxyElement class seems to match what I was setting via the config file.  Do you know more about it?

If the user can run netsh winhttp, and it succeeds for him, and no one has any better ideas, I'll give this credit.  I'm still hoping someone could give me more answers than questions, though.

Accepted Solution

r3nder earned 1000 total points
ID: 35123320
Basically you can run netsh commands for winhttp in a batch file or script - for that matter you can call the batch file in C# (it is just a couple of lines of code) and no on is the wiser

Here is an example of the C# used to call the batch

public void connectHTTP()
            string DriveLetter = Path.GetPathRoot(Environment.GetFolderPath(Environment.SpecialFolder.System));
            string Username = txtBxUser.Text.ToString();
            string Password = txtBxPass.Text.ToString();
            string args = @"/c HTTP.bat " + Username + " " + '"' + Password + '"';

            System.Diagnostics.Process p = new System.Diagnostics.Process();

            p.StartInfo.Arguments = args;
            p.StartInfo.UseShellExecute = false;
            p.StartInfo.FileName = DriveLetter + "windows\\system32\\cmd.exe";


Open in new window

More Than Just A Video Library

Train for your certification. Learn the latest DevOps tools. Grow your skillset to do better work.

At Linux Academy, we release new training modules every week so you'll always be up to date on the latest tech.

LVL 10

Author Comment

ID: 35167440
For the record, before moderators force this closed, the question isn't abandoned.  I've sent the suggestion that r3nder provided a link to to my customer, but I haven't heard back from the customer as to the results.

I asked two general questions in my original post:
"How do you set a .NET project to use the default proxy settings?"  r3nder sort of answered this, though the link provided would more accurately answer "How do you configure Windows to forward its proxy settings to .NET."  I'm waiting for the customer's response to know if r3nder's answer is a good one.  I still don't think the answer is "right" in that it doesn't make sense for me to have to force an execution of a batch file with every program I write that accesses the web.  There should be a simple setting within .NET itself.

The second question was: "How do you test proxy settings when you yourself are not behind an authenticated proxy?"  This one was more important, as it would allow me to test r3nder's answer and/or do experiments on my own.  It has not had any answer.

Mods - should I branch the second question off as a separate but related question?

Expert Comment

by:Tom Knowlton
ID: 36116040
I've requested that this question be deleted for the following reason:

This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
LVL 10

Author Comment

ID: 36116032
Sorry, I actually was planning to close this this weekend.  

The customer stopped working with me, and the issue eventually got abandoned by my company, so I was never able to verify r3nder's answer.

The first link was worthless to me - my posting should have made it obvious that simply copying a link from a basic google search wasn't enough.

The code that r3nder submitted may have been useful; since I wasn't able to test, I don't really know.  I'll close the question and reward points to r3nder - the answer may be useful to someone else, so it's worth adding into the knowledge base, I suppose.  I can't award an A for this, though.

LVL 10

Author Comment

ID: 36116041
I've never had to deal with a closure request, and didn't realize I had to click on Object.  Please see my note above.  I would accept r3nder's comment 35123320 for full points, though I feel it should be awarded a C.  Due to my customer no longer participating, I wasn't able to validate the answer, and no one addressed my underlying question of how to test as if I was behind a proxy when I'm not behind one.

LVL 10

Author Closing Comment

ID: 36116042
See my comments above for my grading and reasoning.

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Wouldn’t it be nice if you could test whether an element is contained in an array by using a Contains method just like the one available on List objects? Wouldn’t it be good if you could write code like this? (CODE) In .NET 3.5, this is possible…
Today I had a very interesting conundrum that had to get solved quickly. Needless to say, it wasn't resolved quickly because when we needed it we were very rushed, but as soon as the conference call was over and I took a step back I saw the correct …
Add bar graphs to Access queries using Unicode block characters. Graphs appear on every record in the color you want. Give life to numbers. Hopes this gives you ideas on visualizing your data in new ways ~ Create a calculated field in a query: …
Have you created a query with information for a calendar? ... and then, abra-cadabra, the calendar is done?! I am going to show you how to make that happen. Visualize your data!  ... really see it To use the code to create a calendar from a q…

715 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question