Sending data to remote webPage

Posted on 2009-12-20
Last Modified: 2012-05-08
I would like to see a working example of sending data to a web page from c#.
For example, with the yahoo mail login at:

I created an FTP application which logs in to remote payor sites to download and log files (eg, MVP, CHPHP, etc.  Some sites however cannot be accomplished through FTP or Moveit so I need to open an instance of IE, send a username and password and click submit all programatically from c#(no 3rd party pay for software please).

Please keep in mind I am running an SSIS script this is not an ASP project.

I tried many many examples on doing this but have yet to find one that works.  Thanks for your help.
Question by:Greg85374
    LVL 23

    Expert Comment

    Have you tried using the WebRequest/Response object's? You can envoke a POST to a page using these obejects.
    LVL 3

    Expert Comment

    here is a quick and rough c# project to bring down the response of a web page.
    You should have no probs running that. If you do use something like this I would suggest having it running on its on its own thread

    Author Comment

    Thanks for the speedy responses.
    Presto, yes, I tried due diligence before posting the question.

     Saber, from what I can see, you are basically loading a web page and reading in the source.      Here is what I did that to my surprise did the exact same thing(returned same results):

    Please note, if you run the example you showed me you will see where I got this from (perhaps yahoo is a bad test as I found out it has customized authentication)
    <th id='thun'><label for="username">Yahoo! ID:</label></th></tr>
    <td><input name="passwd" id="passwd" value="program!" size="17" class="yreg_ipt" type="password" maxlength="64"></td>

    so using these two input fields I did:

    public void Test3(string url, string username, string password)
                string param1 = username;
                string param2 = password;

                ASCIIEncoding encoding = new ASCIIEncoding();
                string postData = string.Format("userName={0}&passwd={1}", param1, param2);

                byte[] buffer = encoding.GetBytes(postData);

                // Prepare web request...
                HttpWebRequest myRequest =  (HttpWebRequest)WebRequest.Create(url);

                // We use POST ( we can also use GET )
                myRequest.Method = "POST";

                // Set the content type to a FORM
                myRequest.ContentType = "application/x-www-form-urlencoded";

                // Get length of content
                myRequest.ContentLength = buffer.Length;

                // Get request stream
                Stream newStream = myRequest.GetRequestStream();

                // Send the data.
                newStream.Write(buffer, 0, buffer.Length);

                // Close stream

                // Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
                HttpWebResponse myHttpWebResponse = (HttpWebResponse)myRequest.GetResponse();

                // Display the contents of the page to the console.
                Stream streamResponse = myHttpWebResponse.GetResponseStream();

                // Get stream object
                StreamReader streamRead = new StreamReader(streamResponse);
                Char[] readBuffer = new Char[256];

                // Read from buffer
                int count = streamRead.Read(readBuffer, 0, 256);

                while (count > 0)
                    // get string
                    String resultData = new String(readBuffer, 0, count);

                    // Write the data

                    // Read from buffer
                    count = streamRead.Read(readBuffer, 0, 256);


                // Release the response object resources.

                // Close response

    Of course as I got the same results as you did back, I did not even move on to trying to figure out how to submit the form.

    LVL 3

    Expert Comment

    I am pretty sure the problem is more about the yahoo authentication method. Are you giving up on solving it this way?

    Finally, is it possible to work at this from the other side and have the servers upload to a central point?

    Author Comment

    Sorry, I found what I believe to be a better example at:

    this is a test login site.

    Author Comment

    Hi Saber, got your message as soon as I posted that,  The clearest answer I can give you is that I am trying to automatically log into a vendors site from the hospital I work at.  I do not have any access beyond that of what I would have logging here at the Experts Exchange.

    Author Comment

    by:Greg85374 is the HTML code behind a web page on a remote vendors site (user name, password, and checkbox).  What I am trying to do is populate the values of the aforementioned controls programatically via c#.

                                  <div id="pnlLogin" style="height:64px;">
                                                                      <input name="Username" type="text" id="Username" /></STRONG></P>
                                                                      <input name="Password" type="password" id="Password" /></STRONG></P>
                                                          <P><STRONG>Please Note: </STRONG>Medicaid recipient level data is confidential and
                                                                is protected by state and federal laws and regulations. It can be used only for
                                                                the purposes directly connected to the administration of the Medicaid program.
                                                                You are required to read, understand and comply with these regulations. There
                                                                are significant state, civil and federal criminal penalties for violations.<BR>
                                                                <A href="" target="_blank">View Medicaid Confidentiality Regulations.</A></P>
                                                                <P><input id="chkbxAgree" type="checkbox" name="chkbxAgree" /><label for="chkbxAgree">I have read and I agree to the Medicaid Confidentiality Regulations</label></P>
                                                                <P><input type="image" name="ibAgreeLogin" id="ibAgreeLogin" src="images/emedny/btn_agreelogin_w.gif" alt="" border="0" /></P>

    LVL 3

    Expert Comment

    Whats the forms id/name?
    And I assume that it isnt https at this stage?

    Accepted Solution

    Actually, I most figured this out.

      private mshtml.IHTMLDocument2 document = null;
      private SHDocVw.InternetExplorer ie = null;

           public void OpenUrl(string url)
                object o = System.Reflection.Missing.Value;
                ie = new InternetExplorer();
                ie.Visible = true;
                //set event handler before calling navigate----duh...took me 30 minutes
                docCompleteHandler = new  DWebBrowserEvents2_DocumentCompleteEventHandler(this.OnIEDocumentComplete);
                ie.DocumentComplete += docCompleteHandler;

                ie.Navigate(url, ref o, ref o, ref o, ref o);
                while (ie.ReadyState != SHDocVw.tagREADYSTATE.READYSTATE_COMPLETE) { System.Threading.Thread.Sleep(20); }
            }// end public void OpenUrl

    As you see I am using the OnDocumentComplete event

    from here I can get what I need.  Unfortunatly it seems once I get my document click event I can no longer enter data into the webpage.

    I am using thefollowing:

    private bool ClickEventHandler(mshtml.IHTMLEventObj e)
                IHTMLElement element = (IHTMLElement)this.document.elementFromPoint(e.clientX, e.clientY);

                if (element != null)

                    if (element.GetType() == typeof(mshtml.IHTMLInputElement))
                        if (lastInputElement != (IHTMLInputElement)element)
                            if (lastInputElement.value != null) { lastInputElementValue = lastInputElement.value.ToString(); }
                            lastInputElement = (IHTMLInputElement)element;
                        // dont care for testing

                    //if (element.getAttribute("type", 0) != null && element.getAttribute("type", 0).ToString().ToUpper() == "INPUT")
                    //    lastInputElement =(IHTMLInputElement)element;

                    //lastInputElement.LostFocus += new HtmlElementEventHandler(el_LostFocus);

                   // string message = string.Empty;

                   // if ( != null) { message += "id: " +; } else { message += "id: null"; }
                   // if (element.getAttribute("name", 0) != null) { message += Environment.NewLine + "name: " + element.getAttribute("name", 0).ToString(); } else { message += Environment.NewLine + "name: null"; }
                   // if (element.getAttribute("type", 0) != null && element.getAttribute("type", 0).ToString().Trim().Length > 1) { message += Environment.NewLine + "type: " + element.getAttribute("type", 0).ToString(); } else { message += Environment.NewLine + "type: null"; }
                   //// System.Windows.Forms.MessageBox.Show(message);

                }// end if- checking if element was found

                return true;
            }// end private bool ClickEventHandler

    as you can see, much of this is still test code and ugly but what I would like to do is two things:

    1.) when the user clicks on an input field, give them a message to identify it (to log to DB for auotmation purposes if the dont know html.

    2.) log the username and password used.  This is because our passwords expire.  they dont maintain the DB like the should so I need to know what password was used when it failed.


    Featured Post

    6 Surprising Benefits of Threat Intelligence

    All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

    Join & Write a Comment

    Microsoft goes to great lengths to ensure that the users don’t encounter issues while working with MS Outlook. But errors are inevitable and can occur when you least expect them. One of such errors which are encountered in Outlook is Error 0x800ccc1…
    The biggest nightmare for any Exchange Server Administrator is to keep the server running without any issue. But the problems often come and they need to be resolved efficiently and timely. Here are important troubleshooting points: Define the Pr…
    In this Experts Exchange video Micro Tutorial, I'm going to show how small business owners who use Google Apps can save money by setting up what is called a catch-all email address in their Gmail accounts. By using the catch-all feature, small busin…
    Access reports are powerful and flexible. Learn how to create a query and then a grouped report using the wizard. Modify the report design after the wizard is done to make it look better. There will be another video to explain how to put the final p…

    755 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    24 Experts available now in Live!

    Get 1:1 Help Now