Http Web Requests

My application needs to retrieve market data, using HTTP protocol, from a vendor, as a batch process.  I am planning to build an application using httpRequest httpResponse classes.  I have a problem when I post data to the vendors URL, the first request is challenged by "Enter Network Password" dialog box.  I have used Proxy object to supply network credentials, but that was for letting my request go out to the internet.  In my case the remote host is challenging me.  How can I supply these credentials to the remote proxy server in a non-interactive mode.  I have contemplated getting a window handle of the dialog box and manipulating it, but I am not comforatble with that approach.
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Something like this...

      System.Net.HttpWebRequest request = new System.Net.HttpWebRequest();
      System.Net.CredentialCache cache = new System.Net.CredentialCache();
      cache.Add(new Uri(proxy.Url), "Negotiate", new System.Net.NetworkCredential("username","password","domainname") );
      request.Credentials = cache;
sai7843Author Commented:
Hi TomasX2,
                   I am already using this to provide credentials for my request to go out to internet thru proxy servers on my end of net work.  But the "Enter NetWok Password" box is appearing when my request reaches the remote host i.e., the remote host is trying to authenticate me with a dialog box.  This is the problem, I am facing.  To see it, please navigate to
Cloud Class® Course: CompTIA Cloud+

The CompTIA Cloud+ Basic training course will teach you about cloud concepts and models, data storage, networking, and network infrastructure.

Are you setting the credentials for both the request object and the request´s proxy object.

        System.Net.HttpWebRequest request;
        request.Credentials = System.Net.CredentialCache.DefaultCredentials;
        request.Proxy.Credentials = System.Net.CredentialCache.DefaultCredentials;              
sai7843Author Commented:
Hi TomaX2
                     The following is a snapshot of my code.  
                      I am getting 401 return code from remote host

username="abcdefg";    /// These two are required by the remote host
password="xyxyxyx";  ////

// Instantiate the custom Basic authentication module.
CustomBasic customBasicModule = new CustomBasic();
// Unregister the standard Basic authentication module.

// Register the custom Basic authentication module.
// Create the Web request object.
HttpWebRequest req = (HttpWebRequest) WebRequest.Create(url);

///The following is required so that my request could get on to the internet,
//because the corporate firewall does not allow traffic to get out without proper
WebProxy objProxy;
NetworkCredential objCredentials;
objProxy = new WebProxy("", 8080);
objCredentials = new NetworkCredential("user1", "welcome", "companyDomain");
objProxy.Credentials = objCredentials;
req.Proxy = objProxy;
// Define the request access method.
req.Method = "POST";
req.Credentials = new NetworkCredential(username, password);   //Required by remotehost            
StreamWriter objStream = new StreamWriter(req.GetRequestStream(), Encoding.ASCII);

// Issue the request.
HttpWebResponse result = (HttpWebResponse) req.GetResponse();

//Console.WriteLine("\nAuthentication Succeeded:");

// Store the response.
Stream sData = result.GetResponseStream();

////++++++++++++++++++++++++++++Helper class+++++++++++++++++++++++

public class CustomBasic : IAuthenticationModule

      private string m_authenticationType ;
      private bool m_canPreAuthenticate ;

      // The CustomBasic constructor initializes the properties of the customized
      // authentication.
      public CustomBasic()
              m_authenticationType = "Basic";
              m_canPreAuthenticate = false;

      // Define the authentication type. This type is then used to identify this
      // custom authentication module. The default is set to Basic.
      public string AuthenticationType
            return m_authenticationType;

      // Define the pre-authentication capabilities for the module. The default is set
      // to false.
      public bool CanPreAuthenticate
                 return m_canPreAuthenticate;

// The checkChallenge method checks whether the challenge sent by the HttpWebRequest
// contains the correct type (Basic) and the correct domain name.
// Note: The challenge is in the form BASIC REALM="DOMAINNAME";
// the Internet Web site must reside on a server whose
// domain name is equal to DOMAINNAME.
      public bool checkChallenge(string Challenge, string domain)
            bool challengePasses = false;

            String tempChallenge = Challenge.ToUpper();

                // Verify that this is a Basic authorization request and that the requested                   // domain is correct.
                // Note: When the domain is an empty string, the following code only
                         // checkswhether the authorization type is Basic.

               if (tempChallenge.IndexOf("BASIC") != -1)
            if (domain != String.Empty)
                  if (tempChallenge.IndexOf(domain.ToUpper()) != -1)
                        challengePasses = true;
                        // The domain is not allowed and the  
                                                                //authorization type is Basic.
                        challengePasses = false;
                  // The domain is a blank string and the authorization type  
                                                //is Basic.
                  challengePasses = true;

                  return challengePasses;

// The PreAuthenticate method specifies whether the authentication implemented
// by this class allows pre-authentication.
// Even if you do not use it, this method must be implemented to obey to the rules
// of interface implementation.
// In this case it always returns false.
      public Authorization PreAuthenticate(WebRequest request, ICredentials  
            return null;

// Authenticate is the core method for this custom authentication.
// When an Internet resource requests authentication, the WebRequest.GetResponse
// method calls the AuthenticationManager.Authenticate method. This method, in
// turn, calls the Authenticate method on each of the registered authentication
// modules, in the order in which they were registered. When the authentication is
// complete an Authorization object is returned to the WebRequest.
      public Authorization Authenticate(String challenge, WebRequest request, ICredentials credentials)
            Encoding ASCII = Encoding.ASCII;        

            // Get the username and password from the credentials
            NetworkCredential MyCreds = credentials.GetCredential(request.RequestUri, "Basic");        

            // Verify that the challenge satisfies the authorization requirements.
            bool challengeOk = checkChallenge(challenge, MyCreds.Domain);

            if (!challengeOk)
                  return null;

            // Create the encrypted string according to the Basic authentication
                                // format as follows:
      // a)Concatenate the username and password separated by colon;
      // b)Apply ASCII encoding to obtain a stream of bytes;
      // c)Apply Base64 encoding to this array of bytes to obtain the encoded
            // authorization.
      string BasicEncrypt = MyCreds.UserName + ":" + MyCreds.Password;

      string BasicToken = "Basic " + Convert.ToBase64String(ASCII.GetBytes(BasicEncrypt));

      // Create an Authorization object using the encoded authorization above.
      Authorization resourceAuthorization = new Authorization(BasicToken);

      return resourceAuthorization;
Try WebClient as follows

string uriString = "The URL";
string input = "data";
WebClient appClient = new WebClient();
string userId = ConfigurationSettings.AppSettings["webShareUserId"];
string password = ConfigurationSettings.AppSettings["webSharePassword"];
string domain = ConfigurationSettings.AppSettings["webShareDomain"];
appClient.Credentials = (ICredentials)new System.Net.NetworkCredential(userId,password,domain);
sai7843Author Commented:
I have opened a ticket with Microsoft regarding this issue.  It turns out that I was having problems because the remote server was not formatting its headers according to the RFC specifications.  IE itself tolerates this, but HttpWebRequest classes could not cope with this.  Microsoft has recommended using .NET framwork 1.1 and using configuration setting to allow unsafeHeaderParsing, which solved the problem.
PAQed with points refunded (125)

Community Support Moderator

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today

From novice to tech pro — start learning today.

Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.