[Last Call] Learn how to a build a cloud-first strategyRegister Now


Sending data to remote webPage

Posted on 2009-12-20
Medium Priority
Last Modified: 2012-05-08
I would like to see a working example of sending data to a web page from c#.
For example, with the yahoo mail login at: https://login.yahoo.com/config/login_verify2?&.src=ym.

I created an FTP application which logs in to remote payor sites to download and log files (eg, MVP, CHPHP, etc.  Some sites however cannot be accomplished through FTP or Moveit so I need to open an instance of IE, send a username and password and click submit all programatically from c#(no 3rd party pay for software please).

Please keep in mind I am running an SSIS script this is not an ASP project.

I tried many many examples on doing this but have yet to find one that works.  Thanks for your help.
Question by:Greg85374
  • 5
  • 3
LVL 23

Expert Comment

ID: 26093293
Have you tried using the WebRequest/Response object's? You can envoke a POST to a page using these obejects.

Expert Comment

ID: 26093367
here is a quick and rough c# project to bring down the response of a web page.
You should have no probs running that. If you do use something like this I would suggest having it running on its on its own thread

Author Comment

ID: 26101186
Thanks for the speedy responses.
Presto, yes, I tried due diligence before posting the question.

 Saber, from what I can see, you are basically loading a web page and reading in the source.      Here is what I did that to my surprise did the exact same thing(returned same results):

Please note, if you run the example you showed me you will see where I got this from (perhaps yahoo is a bad test as I found out it has customized authentication)
<th id='thun'><label for="username">Yahoo! ID:</label></th></tr>
<td><input name="passwd" id="passwd" value="program!" size="17" class="yreg_ipt" type="password" maxlength="64"></td>

so using these two input fields I did:

public void Test3(string url, string username, string password)
            string param1 = username;
            string param2 = password;

            ASCIIEncoding encoding = new ASCIIEncoding();
            string postData = string.Format("userName={0}&passwd={1}", param1, param2);

            byte[] buffer = encoding.GetBytes(postData);

            // Prepare web request...
            HttpWebRequest myRequest =  (HttpWebRequest)WebRequest.Create(url);

            // We use POST ( we can also use GET )
            myRequest.Method = "POST";

            // Set the content type to a FORM
            myRequest.ContentType = "application/x-www-form-urlencoded";

            // Get length of content
            myRequest.ContentLength = buffer.Length;

            // Get request stream
            Stream newStream = myRequest.GetRequestStream();

            // Send the data.
            newStream.Write(buffer, 0, buffer.Length);

            // Close stream

            // Assign the response object of 'HttpWebRequest' to a 'HttpWebResponse' variable.
            HttpWebResponse myHttpWebResponse = (HttpWebResponse)myRequest.GetResponse();

            // Display the contents of the page to the console.
            Stream streamResponse = myHttpWebResponse.GetResponseStream();

            // Get stream object
            StreamReader streamRead = new StreamReader(streamResponse);
            Char[] readBuffer = new Char[256];

            // Read from buffer
            int count = streamRead.Read(readBuffer, 0, 256);

            while (count > 0)
                // get string
                String resultData = new String(readBuffer, 0, count);

                // Write the data

                // Read from buffer
                count = streamRead.Read(readBuffer, 0, 256);


            // Release the response object resources.

            // Close response

Of course as I got the same results as you did back, I did not even move on to trying to figure out how to submit the form.

What does it mean to be "Always On"?

Is your cloud always on? With an Always On cloud you won't have to worry about downtime for maintenance or software application code updates, ensuring that your bottom line isn't affected.


Expert Comment

ID: 26101236
I am pretty sure the problem is more about the yahoo authentication method. Are you giving up on solving it this way?

Finally, is it possible to work at this from the other side and have the servers upload to a central point?

Author Comment

ID: 26101240
Sorry, I found what I believe to be a better example at: http://newtest.richardhealey.com/index.asp

this is a test login site.

Author Comment

ID: 26101251
Hi Saber, got your message as soon as I posted that,  The clearest answer I can give you is that I am trying to automatically log into a vendors site from the hospital I work at.  I do not have any access beyond that of what I would have logging here at the Experts Exchange.

Author Comment

ID: 26274398
OK...here is the HTML code behind a web page on a remote vendors site (user name, password, and checkbox).  What I am trying to do is populate the values of the aforementioned controls programatically via c#.

                              <div id="pnlLogin" style="height:64px;">
                                                                  <input name="Username" type="text" id="Username" /></STRONG></P>
                                                                  <input name="Password" type="password" id="Password" /></STRONG></P>
                                                      <P><STRONG>Please Note: </STRONG>Medicaid recipient level data is confidential and
                                                            is protected by state and federal laws and regulations. It can be used only for
                                                            the purposes directly connected to the administration of the Medicaid program.
                                                            You are required to read, understand and comply with these regulations. There
                                                            are significant state, civil and federal criminal penalties for violations.<BR>
                                                            <A href="http://www.emedny.org/MedConfidentialityReg.html" target="_blank">View Medicaid Confidentiality Regulations.</A></P>
                                                            <P><input id="chkbxAgree" type="checkbox" name="chkbxAgree" /><label for="chkbxAgree">I have read and I agree to the Medicaid Confidentiality Regulations</label></P>
                                                            <P><input type="image" name="ibAgreeLogin" id="ibAgreeLogin" src="images/emedny/btn_agreelogin_w.gif" alt="" border="0" /></P>


Expert Comment

ID: 26276746
Whats the forms id/name?
And I assume that it isnt https at this stage?

Accepted Solution

Greg85374 earned 0 total points
ID: 26293498
Actually, I most figured this out.

  private mshtml.IHTMLDocument2 document = null;
  private SHDocVw.InternetExplorer ie = null;

       public void OpenUrl(string url)
            object o = System.Reflection.Missing.Value;
            ie = new InternetExplorer();
            ie.Visible = true;
            //set event handler before calling navigate----duh...took me 30 minutes
            docCompleteHandler = new  DWebBrowserEvents2_DocumentCompleteEventHandler(this.OnIEDocumentComplete);
            ie.DocumentComplete += docCompleteHandler;

            ie.Navigate(url, ref o, ref o, ref o, ref o);
            while (ie.ReadyState != SHDocVw.tagREADYSTATE.READYSTATE_COMPLETE) { System.Threading.Thread.Sleep(20); }
        }// end public void OpenUrl

As you see I am using the OnDocumentComplete event

from here I can get what I need.  Unfortunatly it seems once I get my document click event I can no longer enter data into the webpage.

I am using thefollowing:

private bool ClickEventHandler(mshtml.IHTMLEventObj e)
            IHTMLElement element = (IHTMLElement)this.document.elementFromPoint(e.clientX, e.clientY);

            if (element != null)

                if (element.GetType() == typeof(mshtml.IHTMLInputElement))
                    if (lastInputElement != (IHTMLInputElement)element)
                        if (lastInputElement.value != null) { lastInputElementValue = lastInputElement.value.ToString(); }
                        lastInputElement = (IHTMLInputElement)element;
                    // dont care for testing

                //if (element.getAttribute("type", 0) != null && element.getAttribute("type", 0).ToString().ToUpper() == "INPUT")
                //    lastInputElement =(IHTMLInputElement)element;

                //lastInputElement.LostFocus += new HtmlElementEventHandler(el_LostFocus);

               // string message = string.Empty;

               // if (element.id != null) { message += "id: " + element.id.ToString(); } else { message += "id: null"; }
               // if (element.getAttribute("name", 0) != null) { message += Environment.NewLine + "name: " + element.getAttribute("name", 0).ToString(); } else { message += Environment.NewLine + "name: null"; }
               // if (element.getAttribute("type", 0) != null && element.getAttribute("type", 0).ToString().Trim().Length > 1) { message += Environment.NewLine + "type: " + element.getAttribute("type", 0).ToString(); } else { message += Environment.NewLine + "type: null"; }
               //// System.Windows.Forms.MessageBox.Show(message);

            }// end if- checking if element was found

            return true;
        }// end private bool ClickEventHandler

as you can see, much of this is still test code and ugly but what I would like to do is two things:

1.) when the user clicks on an input field, give them a message to identify it (to log to DB for auotmation purposes if the dont know html.

2.) log the username and password used.  This is because our passwords expire.  they dont maintain the DB like the should so I need to know what password was used when it failed.


Featured Post

New feature and membership benefit!

New feature! Upgrade and increase expert visibility of your issues with Priority Questions.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

How can MDaemon MRK files and MSG files work together? We can also say that, how worldclient and Outlook connecter works, using both application collectively you can view your mails, contacts, calendar, task, notes. In this article you can unders…
Performance in games development is paramount: every microsecond counts to be able to do everything in less than 33ms (aiming at 16ms). C# foreach statement is one of the worst performance killers, and here I explain why.
This tutorial gives a high-level tour of the interface of Marketo (a marketing automation tool to help businesses track and engage prospective customers and drive them to purchase). You will see the main areas including Marketing Activities, Design …
In this Experts Exchange video Micro Tutorial, I'm going to show how small business owners who use Google Apps can save money by setting up what is called a catch-all email address in their Gmail accounts. By using the catch-all feature, small busin…

829 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question