Solved

Making web requests from behind a proxy

Posted on 2011-03-11
9
886 Views
Last Modified: 2012-05-11
Hi, Experts,

This will be a long post, but I figure you'll do better with more information than with less.

The general questions:
How do I set a .NET 2.0 (C#, if it matters) project to use the user's default proxy settings?  Since I don't have any access to any proxy with authentication required, is there any way to test proxy support?

The details:
I have some software that makes calls to a web service.  It works perfectly fine in all of my tests.  A customer bought it, and it was failing with HTTP error 407 - proxy authentication required.

I needed to learn about .NET and proxy support.  I found an outdated msdn article here.  It tells me the process that .NET 2.0 goes through to figure out proxy settings.

In what little I can find on the web, I've been very surprised that WebRequest doesn't automatically look at your proxy settings.  StackOverflow shows a way that isn't overly inconvenient - simply setting the following in your app.config:
<configuration>
   <system.net>
     <defaultProxy>
       <proxy autoDetect="true" />
     </defaultProxy>
  </system.net>
</configuration> 

Open in new window

Obviously, if this was the perfect solution, I'd question why Microsoft doesn't set proxy autoDect to true as the default.
But it wasn't the perfect solution.  I sent the new application to my customer, and he reported that he got a timeout error.  I'm guessing that some part of the proxy autodetection timed out, so the whole sequence failed.

I like the idea of fixing it in the app.config rather than in the C# code directly, so I tried removing the autodetect setting and instead using the useSystemDefault setting, (i.e.)
<proxy useSystemDefault="True" />
I would be fairly certain that this should work, but the customer reported that the WebRequest again returned a timeout error.

It is extremely possible that he didn't run the newest program I've sent him.  He's a regular user, not a programmer, but I don't know any other way to work on the proxy issue other than through him.  I can ask him to uninstall and reinstall the latest, just to be sure, but he's now out of his office until Monday.
Over the weekend, I'd like to come up with one final test for him - i.e., one that works.

I know that I can also rewrite all the WebRequests to include a Proxy setting, such as this example on stackoverflow.  Since there are several web requests, I've been reluctant to try that, unless I was certain of what proxy settings to use.

If I could unit test this on my own, I'd be ok with shooting in the dark and playing trial and error until I got it working, but having to send each test to a user and having each test fail is making me look worse and worse.

I know the customer can surf the web ok, so he has his web-browser's proxy set up correctly.

Does anyone have a recommendation as to how I can do my own testing?  I need to be behind a proxy that requires authentication.  I've considered using web-anonymizers, but they don't require authentication.

Does anyone have any experience in this and can tell me the proper way to set the proxy?

What's the difference between WebRequest.DefaultWebProxy and WebRequest.GetSystemWebProxy?  The msdn documentation (yes, I know this if for .NET 4.0.  The documentation for earlier versions says far less, which is part of my confusion.)  says that DefaultWebProxy checks the app.config then the IE proxy settings while GetSystemWebProxy only checks the IE proxy settings.

What happens if the user is using a different browser?  Do all the proxy settings occur in the same place in the registry, or can it be that his firefox runs fine and his IE is not set up properly?

That's a torrent of questions, but all highly related.  If anyone feels this is too much, I'm happy to branch it off into separate questions.

Thanks for reading this, and thanks in advance for any help you can offer.
0
Comment
Question by:Carnou
  • 5
  • 2
9 Comments
 
LVL 6

Expert Comment

by:r3nder
ID: 35119494
0
 
LVL 10

Author Comment

by:Carnou
ID: 35119611
Thanks for the effort.

I saw that thread before posting here.  As tjmoore responded to the accepted answer, there are a few problems with using the command line solution - namely that the users are in a restricted environment, and I shouldn't expect them to run a command-line to make my program work.

That post refers to GetDefaultProxy, which has since been depracated by DefaultWebProxy and GetSystemWebProxy.

I can see if the user is capable of running netsh winhttp.  It won't be the ideal solution, but it may be acceptable.

The ProxyElement class seems to match what I was setting via the config file.  Do you know more about it?

If the user can run netsh winhttp, and it succeeds for him, and no one has any better ideas, I'll give this credit.  I'm still hoping someone could give me more answers than questions, though.
0
 
LVL 6

Accepted Solution

by:
r3nder earned 500 total points
ID: 35123320
Basically you can run netsh commands for winhttp in a batch file or script - for that matter you can call the batch file in C# (it is just a couple of lines of code) and no on is the wiser

Here is an example of the C# used to call the batch

Thanks
r3nder
public void connectHTTP()
        {
            string DriveLetter = Path.GetPathRoot(Environment.GetFolderPath(Environment.SpecialFolder.System));
            string Username = txtBxUser.Text.ToString();
            string Password = txtBxPass.Text.ToString();
            string args = @"/c HTTP.bat " + Username + " " + '"' + Password + '"';

            System.Diagnostics.Process p = new System.Diagnostics.Process();

            p.StartInfo.Arguments = args;
            p.StartInfo.UseShellExecute = false;
            p.StartInfo.FileName = DriveLetter + "windows\\system32\\cmd.exe";

            p.Start();
            p.WaitForExit();
            
        }

Open in new window

0
 
LVL 10

Author Comment

by:Carnou
ID: 35167440
For the record, before moderators force this closed, the question isn't abandoned.  I've sent the suggestion that r3nder provided a link to to my customer, but I haven't heard back from the customer as to the results.

I asked two general questions in my original post:
"How do you set a .NET project to use the default proxy settings?"  r3nder sort of answered this, though the link provided would more accurately answer "How do you configure Windows to forward its proxy settings to .NET."  I'm waiting for the customer's response to know if r3nder's answer is a good one.  I still don't think the answer is "right" in that it doesn't make sense for me to have to force an execution of a batch file with every program I write that accesses the web.  There should be a simple setting within .NET itself.

The second question was: "How do you test proxy settings when you yourself are not behind an authenticated proxy?"  This one was more important, as it would allow me to test r3nder's answer and/or do experiments on my own.  It has not had any answer.

Mods - should I branch the second question off as a separate but related question?
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 5

Expert Comment

by:knowlton
ID: 36116040
I've requested that this question be deleted for the following reason:

This question has been classified as abandoned and is closed as part of the Cleanup Program. See the recommendation for more details.
0
 
LVL 10

Author Comment

by:Carnou
ID: 36116032
Sorry, I actually was planning to close this this weekend.  

The customer stopped working with me, and the issue eventually got abandoned by my company, so I was never able to verify r3nder's answer.

The first link was worthless to me - my posting should have made it obvious that simply copying a link from a basic google search wasn't enough.

The code that r3nder submitted may have been useful; since I wasn't able to test, I don't really know.  I'll close the question and reward points to r3nder - the answer may be useful to someone else, so it's worth adding into the knowledge base, I suppose.  I can't award an A for this, though.

0
 
LVL 10

Author Comment

by:Carnou
ID: 36116041
I've never had to deal with a closure request, and didn't realize I had to click on Object.  Please see my note above.  I would accept r3nder's comment 35123320 for full points, though I feel it should be awarded a C.  Due to my customer no longer participating, I wasn't able to validate the answer, and no one addressed my underlying question of how to test as if I was behind a proxy when I'm not behind one.

0
 
LVL 10

Author Closing Comment

by:Carnou
ID: 36116042
See my comments above for my grading and reasoning.
0

Featured Post

Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

Join & Write a Comment

More often than not, we developers are confronted with a need: a need to make some kind of magic happen via code. Whether it is for a client, for the boss, or for our own personal projects, the need must be satisfied. Most of the time, the Framework…
Calculating holidays and working days is a function that is often needed yet it is not one found within the Framework. This article presents one approach to building a working-day calculator for use in .NET.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
This tutorial demonstrates a quick way of adding group price to multiple Magento products.

746 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

12 Experts available now in Live!

Get 1:1 Help Now