Urgent! - Using HttpWebRequest to Post to web page that submits a form using javascript

Posted on 2006-03-25
Medium Priority
Last Modified: 2012-08-14
I'm using HttpWebRequest to post to web page.  That web page submits itself using javascript.  My issue is that it seems that web page doesn't seem to be submitting using the javascript when calling the page with a HttpWebRequest.  When running the page in a brower the javascript submits the form correctly.  Does anyone have any ideas on how to solve this?  When adding some logging to the HttpWebRequest I can see that page is being loaded properly but just doesn't seem to be submitting itself.

Below is some of the code that I've tried but has not worked.  Most of this code has been cobbled together from other posts similar to this topic.

  Public Shared Function RetrieveHttpContent(ByVal Url As String, ByRef ErrorMessage As String) As String
    Dim MergedText As String = ""
    Dim Http As System.Net.WebClient = New System.Net.WebClient()
    ' Download the Web resource and save it into a data buffer.
      Dim Result() As Byte = Http.DownloadData(Url)
      MergedText = Encoding.Default.GetString(Result)
    Catch ex As Exception
      ErrorMessage = ex.Message
      Return Nothing
    End Try

    Return MergedText
  End Function

  Public Shared Function RetrieveHttpContent2(ByVal Url As String) As String
    Dim MergedText As String = ""
    Dim Http As System.Net.HttpWebRequest = WebRequest.Create(Url)
    Http.KeepAlive = False

    'Get the data as an HttpWebResponse object
    Dim resp As HttpWebResponse = Http.GetResponse()
    ' Download the Web resource and save it into a data buffer.
    'Convert the data into a string (assumes that you are requesting text)
    Dim sr As New StreamReader(resp.GetResponseStream())
    Dim results As String = sr.ReadToEnd()
    Return results
  End Function

'I also tried async:
  ' go to the website for the info
  Private Function getSite(ByVal HttpSite As Uri) As String
    Dim wreq As HttpWebRequest = CType(WebRequest.Create(HttpSite), HttpWebRequest)
    Dim rc As IAsyncResult = CType(wreq.BeginGetResponse(New AsyncCallback(AddressOf Me.RespCallback), wreq), IAsyncResult)
    Return "done"
  End Function

  Public Sub RespCallback(ByVal ar As IAsyncResult)
    Dim req As HttpWebRequest = CType(ar.AsyncState, HttpWebRequest)
    Dim resp As HttpWebResponse = CType(req.EndGetResponse(ar), HttpWebResponse)

    Dim BytesRead As Integer = 0
    Dim Buffer() As Char = New Char(512) {}

    Dim Reader As StreamReader = New StreamReader(resp.GetResponseStream(), System.Text.Encoding.UTF8)
    Dim Writer As StringWriter = New StringWriter()

    BytesRead = Reader.Read(Buffer, 0, 512)
    While BytesRead <> 0
      Writer.Write(Buffer, 0, 512)
      BytesRead = Reader.Read(Buffer, 0, 512)
    End While
    Dim htmlpage As String = Writer.ToString()
    'extractInfo(htmlpage, resp.ResponseUri.AbsoluteUri)
  End Sub
Question by:cdaly33
1 Comment
LVL 15

Accepted Solution

Solar_Flare earned 1500 total points
ID: 16295610
firstly, javascript has nothing do do with the actual submission of data to the web server - all it does is instruct the browser to do a postback.

By doing the postback with a httpWebRequest the javascript is irrelevant. All that matters is the post data(if its a post request) or the get data in the URL (if its a get request)

The following code is what I use when getting html pages within VB.NET

' Initialise the request
                Dim webRequest As Net.HttpWebRequest = Net.WebRequest.Create(MyURL)
                ' Obtain the response
                Dim Response As Net.HttpWebResponse
                    Response = webRequest.GetResponse()
                    Dim sReader As IO.StreamReader = New IO.StreamReader(Response.GetResponseStream)
                    Dim HTML As String = sReader.ReadToEnd

                    thisHTML = HTML
                    thisResponseHeaders = Response.Headers
                Catch webex As System.Net.WebException
                    thisErrorMessage = webex.Message
                 End Try

if the action you are trying to perform involves posting data to the webserver (eg form input boxes) then you need to include a byte array of a string of post variable (eg "id=1&submitted=true&action=find")

Featured Post


Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article discusses the ASP.NET AJAX ModalPopupExtender control. In this article we will show how to use the ModalPopupExtender control, how to display/show/call the ASP.NET AJAX ModalPopupExtender control from javascript, how to show/display/cal…
Sometimes in DotNetNuke module development you want to swap controls within the same module definition.  In doing this DNN (somewhat annoyingly) swaps the Skin and Container definitions to the default admin selections.  To get around this you need t…
Kernel Data Recovery is a renowned Data Recovery solution provider which offers wide range of softwares for both enterprise and home users with its cost-effective solutions. Let's have a quick overview of the journey and data recovery tools range he…
SQL Database Recovery Software repairs the MDF & NDF Files, corrupted due to hardware related issues or software related errors. Provides preview of recovered database objects and allows saving in either MSSQL, CSV, HTML or XLS format. Ensures recov…
Suggested Courses
Course of the Month9 days, 8 hours left to enroll

609 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question