Http Web Requests

Posted on 2004-10-22
Last Modified: 2008-01-09
My application needs to retrieve market data, using HTTP protocol, from a vendor, as a batch process.  I am planning to build an application using httpRequest httpResponse classes.  I have a problem when I post data to the vendors URL, the first request is challenged by "Enter Network Password" dialog box.  I have used Proxy object to supply network credentials, but that was for letting my request go out to the internet.  In my case the remote host is challenging me.  How can I supply these credentials to the remote proxy server in a non-interactive mode.  I have contemplated getting a window handle of the dialog box and manipulating it, but I am not comforatble with that approach.
Question by:sai7843
    LVL 5

    Expert Comment

    Something like this...

          System.Net.HttpWebRequest request = new System.Net.HttpWebRequest();
          System.Net.CredentialCache cache = new System.Net.CredentialCache();
          cache.Add(new Uri(proxy.Url), "Negotiate", new System.Net.NetworkCredential("username","password","domainname") );
          request.Credentials = cache;
    LVL 5

    Expert Comment


    Author Comment

    Hi TomasX2,
                       I am already using this to provide credentials for my request to go out to internet thru proxy servers on my end of net work.  But the "Enter NetWok Password" box is appearing when my request reaches the remote host i.e., the remote host is trying to authenticate me with a dialog box.  This is the problem, I am facing.  To see it, please navigate to
    LVL 5

    Expert Comment

    Are you setting the credentials for both the request object and the request´s proxy object.

            System.Net.HttpWebRequest request;
            request.Credentials = System.Net.CredentialCache.DefaultCredentials;
            request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;              
    LVL 5

    Expert Comment


    Author Comment

    Hi TomaX2
                         The following is a snapshot of my code.  
                          I am getting 401 return code from remote host

    username="abcdefg";    /// These two are required by the remote host
    password="xyxyxyx";  ////

    // Instantiate the custom Basic authentication module.
    CustomBasic customBasicModule = new CustomBasic();
    // Unregister the standard Basic authentication module.

    // Register the custom Basic authentication module.
    // Create the Web request object.
    HttpWebRequest req = (HttpWebRequest) WebRequest.Create(url);

    ///The following is required so that my request could get on to the internet,
    //because the corporate firewall does not allow traffic to get out without proper
    WebProxy objProxy;
    NetworkCredential objCredentials;
    objProxy = new WebProxy("", 8080);
    objCredentials = new NetworkCredential("user1", "welcome", "companyDomain");
    objProxy.Credentials = objCredentials;
    req.Proxy = objProxy;
    // Define the request access method.
    req.Method = "POST";
    req.Credentials = new NetworkCredential(username, password);   //Required by remotehost            
    StreamWriter objStream = new StreamWriter(req.GetRequestStream(), Encoding.ASCII);

    // Issue the request.
    HttpWebResponse result = (HttpWebResponse) req.GetResponse();

    //Console.WriteLine("\nAuthentication Succeeded:");

    // Store the response.
    Stream sData = result.GetResponseStream();

    ////++++++++++++++++++++++++++++Helper class+++++++++++++++++++++++

    public class CustomBasic : IAuthenticationModule

          private string m_authenticationType ;
          private bool m_canPreAuthenticate ;

          // The CustomBasic constructor initializes the properties of the customized
          // authentication.
          public CustomBasic()
                  m_authenticationType = "Basic";
                  m_canPreAuthenticate = false;

          // Define the authentication type. This type is then used to identify this
          // custom authentication module. The default is set to Basic.
          public string AuthenticationType
                return m_authenticationType;

          // Define the pre-authentication capabilities for the module. The default is set
          // to false.
          public bool CanPreAuthenticate
                     return m_canPreAuthenticate;

    // The checkChallenge method checks whether the challenge sent by the HttpWebRequest
    // contains the correct type (Basic) and the correct domain name.
    // Note: The challenge is in the form BASIC REALM="DOMAINNAME";
    // the Internet Web site must reside on a server whose
    // domain name is equal to DOMAINNAME.
          public bool checkChallenge(string Challenge, string domain)
                bool challengePasses = false;

                String tempChallenge = Challenge.ToUpper();

                    // Verify that this is a Basic authorization request and that the requested                   // domain is correct.
                    // Note: When the domain is an empty string, the following code only
                             // checkswhether the authorization type is Basic.

                   if (tempChallenge.IndexOf("BASIC") != -1)
                if (domain != String.Empty)
                      if (tempChallenge.IndexOf(domain.ToUpper()) != -1)
                            challengePasses = true;
                            // The domain is not allowed and the  
                                                                    //authorization type is Basic.
                            challengePasses = false;
                      // The domain is a blank string and the authorization type  
                                                    //is Basic.
                      challengePasses = true;

                      return challengePasses;

    // The PreAuthenticate method specifies whether the authentication implemented
    // by this class allows pre-authentication.
    // Even if you do not use it, this method must be implemented to obey to the rules
    // of interface implementation.
    // In this case it always returns false.
          public Authorization PreAuthenticate(WebRequest request, ICredentials  
                return null;

    // Authenticate is the core method for this custom authentication.
    // When an Internet resource requests authentication, the WebRequest.GetResponse
    // method calls the AuthenticationManager.Authenticate method. This method, in
    // turn, calls the Authenticate method on each of the registered authentication
    // modules, in the order in which they were registered. When the authentication is
    // complete an Authorization object is returned to the WebRequest.
          public Authorization Authenticate(String challenge, WebRequest request, ICredentials credentials)
                Encoding ASCII = Encoding.ASCII;        

                // Get the username and password from the credentials
                NetworkCredential MyCreds = credentials.GetCredential(request.RequestUri, "Basic");        

                // Verify that the challenge satisfies the authorization requirements.
                bool challengeOk = checkChallenge(challenge, MyCreds.Domain);

                if (!challengeOk)
                      return null;

                // Create the encrypted string according to the Basic authentication
                                    // format as follows:
          // a)Concatenate the username and password separated by colon;
          // b)Apply ASCII encoding to obtain a stream of bytes;
          // c)Apply Base64 encoding to this array of bytes to obtain the encoded
                // authorization.
          string BasicEncrypt = MyCreds.UserName + ":" + MyCreds.Password;

          string BasicToken = "Basic " + Convert.ToBase64String(ASCII.GetBytes(BasicEncrypt));

          // Create an Authorization object using the encoded authorization above.
          Authorization resourceAuthorization = new Authorization(BasicToken);

          return resourceAuthorization;

    Expert Comment

    Try WebClient as follows

    string uriString = "The URL";
    string input = "data";
    WebClient appClient = new WebClient();
    string userId = ConfigurationSettings.AppSettings["webShareUserId"];
    string password = ConfigurationSettings.AppSettings["webSharePassword"];
    string domain = ConfigurationSettings.AppSettings["webShareDomain"];
    appClient.Credentials = (ICredentials)new System.Net.NetworkCredential(userId,password,domain);

    Author Comment

    I have opened a ticket with Microsoft regarding this issue.  It turns out that I was having problems because the remote server was not formatting its headers according to the RFC specifications.  IE itself tolerates this, but HttpWebRequest classes could not cope with this.  Microsoft has recommended using .NET framwork 1.1 and using configuration setting to allow unsafeHeaderParsing, which solved the problem.

    Accepted Solution

    PAQed with points refunded (125)

    Community Support Moderator

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Learn The Basics of Ethical Hacking & Pen Testing

    Computer and network security is one of the fastest growing and most essential industries in technology, meaning companies will pay big bucks for ethical hackers. This is the perfect course to leap into this lucrative career, learning how to use ethical hacking to reveal ...

    Introduction                                                 Was the var keyword really only brought out to shorten your syntax? Or have the VB language guys got their way in C#? What type of variable is it? All will be revealed.   Also called…
    This article is for Object-Oriented Programming (OOP) beginners. An Interface contains declarations of events, indexers, methods and/or properties. Any class which implements the Interface should provide the concrete implementation for each Inter…
    This video is in connection to the article "The case of a missing mobile phone (". It will help one to understand clearly the steps to track a lost android phone.
    Illustrator's Shape Builder tool will let you combine shapes visually and interactively. This video shows the Mac version, but the tool works the same way in Windows. To follow along with this video, you can draw your own shapes or download the file…

    933 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    17 Experts available now in Live!

    Get 1:1 Help Now