Link to home
Start Free TrialLog in
Avatar of AlexPonnath
AlexPonnathFlag for United States of America

asked on

PHP processing webform

I was wondering if there is a quick and dirty way to take a webpage find a section like a form and get all
fields, field type and value and load it in to an array. I need that to compare it to my hard coded field list
to ensure names have not changed to avoid my code breaking
Avatar of Nicholas
Nicholas

Not a very simple thing, but there is this that you can parse thru the fields

http://simplehtmldom.sourceforge.net/

You would look for the form then get an array of the input type=text/checkbox etc fields
Avatar of Dave Baldwin
For testing purposes to get all of the fields in my forms, I use the very simple PHP program below as the temporary 'action' attribute.  It lists all of them for me.  Note that checkboxes that are not checked are Not included in the POST or GET data.
'postdump.php'
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
 "http://www.w3.org/TR/html4/loose.dtd">

<html>
<head>
<title>POST Dump</title>
</head>
<body>
<pre>

<?php 
var_dump($_POST);
 ?>

</pre>
</body>
</html>

Open in new window

I think he means a rendered web page...

Care to elaborate Alex
Avatar of AlexPonnath

ASKER

that doesn't work in my case, i have a vendor website which i log into and post data based on populating some forms.
But they are frequently changing forms inc field names etc. So i want to run kind of a pre-validation before i start posting
data to make sure we still have current field mappings etc.
SimpleHTMLDom may work but it depends on their authentication method - you would need to log into the site first then request the page
It sounds like you're coming at this from the wrong direction :-)

Your PHP script would generate the HTML form, right?  So when the HTML form is submitted, you go back to the canonical list of possible inputs that were used to generate the HTML.  If any of the inputs are missing, or if any additional inputs are present you're under attack.  Just discard the request.  Here is an example of the design pattern.
<?php // demo/temp_alexponnath.php
/**
 * https://www.experts-exchange.com/questions/29021253/PHP-processing-webform.html
 */
error_reporting(E_ALL);


// THE FIELDS WE WANT TO ACCEPT
$fields = [ 'name', 'email' ];


// IF THE REQUEST HAS BEEN POSTED
if (!empty($_POST))
{
    $post = [];
    foreach ($fields as $field)
    {
        $post[$field] = $_POST[$field];
    }
    if ($_POST !== $post)
    {
        trigger_error('Under attack, run like hell!', E_USER_ERROR);
    }

    // SANITIZE AND PROCESS THE VALUES IN $post HERE
    var_dump($post);
    exit;
}


// CREATE THE FORM INPUTS, ACCORDING TO YOUR RULES
$inputs = [];
foreach ($fields as $field)
{
    $inputs[] = $field . ': <input name="' . $field . '" />';
}
$inputs = implode(PHP_EOL, $inputs);

// CREATE THE FORM, USING THE INPUTS
$form = <<<EOF
<form method="post">
$inputs
<input type="submit" />
</form>
EOF;

echo $form;

Open in new window

Your PHP script would generate the HTML form, right?
Not his site
Check with them if they have an API to post data, if not and they're constantly changing field names ask them why they don't - seems weird to be constantly changing something like that
I looked at the website and tried the following code to look for all inputs but i am not getting anything. Also not sure how i would only select all inputs of a fiven form as there might be more then one form on the page

<?php

$html = file_get_contents('page1.html');

echo $html;
$html = str_get_html($output);
$inputs = $html->find('input'); 

    print_r($inputs);

?> 

Open in new window

No API for that side and no its not my side, its a vendor system which has forms i need to submit
What is the vendor?  Do you have a link to their online man pages?
It's not an easy thing to do
You will need to login to the site, if the site then requires cookies you will need to store that, when requesting the page you want you need to send the cookie to the site
As for the form - if they are changing the tag then you will need to use discovery for the correct form

Far beyond the scope of a simple question here.
Sounds like you want to "scrape" the vendor web site in order to identify the expected inputs.  Is that right?
Yes I want to load the page of my vendor and compare the fields and field names that simple. I have not ask nor do I need help to login or any other thing. Nicholas comments are absolutely useless as they do not address the issue or question. I ask how to find or isolate my string to a certain form portion of a webpage and then go and find all inputs nothing more nothing less
Alex: Can you please show us a link to the vendor page you want to process?  If not, can you please visit the page, use "view source" and show us the HTML document.  If we can see the input HTML document we may be able to give you a good design that will extract the input controls.  Thanks.
So here is the code i uses based on the docs of the link provided
<?php

include 'simplehtmldom.php';

$html = new simple_html_dom();

$html->load_file('page1.html');

echo $html;
$ret = $html->find('form[id=blockRequestForm]'); 

print_r($ret);

?>

Open in new window


when i run the code the page never stops to load and keeps filling the array so i must be something missing here because i cant see that the array is that big
<!DOCTYPE html>






<style type="text/css">
    @import url("css/jquery-ui-1.9.1.custom.css");
</style>



<script type="text/javascript" src="/pas/js/jquery/jquery-1.8.2.js"></script>
<script type="text/javascript" src="/pas/js/jquery/jquery-ui-1.9.1.custom.js"></script>   
<script type="text/javascript" src="/pas/js/neustar.js"></script>
 
<head>
    <meta http-equiv="X-UA-Compatible" content="IE=9; IE=8; IE=7; IE=EDGE" />
    <META HTTP-EQUIV="Pragma" CONTENT="no-cache"> 
    <title>New Block Request</title>
</head>

<script type="text/javascript" src="js/utilities.js"></script>

<script language="Javascript" type="text/javascript">
    <!--
    var section = "";
    var active = null;
    // -->
</script>







<script>
    var pasWin = null;
    var timerId = null;
    function defaultOnLoad(msg)
    {
        window.defaultStatus=msg;
    
        setSessionTimeOut();
    
    }

    function defaultUnLoad()
    {
        if (timerId != null)
        {
          clearTimeout(timerId);
        }
    }


    function displayTimeout()
    {
      var answer = confirm('Your session expires in less than 5 minutes.  Click ok to continue session.'
                +'  Pressing cancel will not remind you about the session timeout.');
      if(answer)
      {
        pasWin = open('/pas/extendsession.do?doNotSetTimer=true',
                    "displayWindow",
                    "width=500,height=325,maximize=false,innerwidth=100");
        pasWin.focus();
        clearTimeout(timerId);
        setSessionTimeOut();
      }
    }


    function setSessionTimeOut()
    {
      
      timerId = setTimeout('displayTimeout()',1500000);
    }
</script>




<body BGCOLOR="white" onLoad="defaultOnLoad('Pooling Administration System.')"  bgcolor="white" onUnLoad="defaultUnLoad()">

<table BGCOLOR="white" width="100%" height="100%" cellspacing="0"
       cellpadding="0" border="1">


    <tr height="10%">
        <td colspan="2" align="center">
<!--Start Header-->
<a href="#skip-header" class="bypass-nav"></a>
<!--Begin Top Banner Table-->


<table width="100%" border="0" cellspacing="0" cellpadding="0">
    <tr valign="top">
        <th align="center" class=banner colspan="2">Pooling Administration System</th>
    </tr>
    <tr>
        
        <td align="right">
            <div id="topnavcontainer">
                <ul id="navlist">
                    <li><a href="/pas/logout.do"><font color="blue"><b>Sign Out</b></font></a></li>
                </ul>
            </div>
        </td>
    </tr>
</table>

<!-- End Header -->

</td>
    </tr>
    <tr height="2%">
        <td colspan="2" align="center">









<table width="100%" border="0">
    <tr>
        <td align="left"><a href="/pas/inbox.do?refresh=false&method=viewWorkItems&userId=30355"> <img
                    src="images/EJBHome.gif" border="0">alex@vodex.co (SP)
            </a>
        <td>
        <td align="center">Time : 05/08/2017 06:34:00 PM EDT
        <td>
            
    </tr>
</table>


</td>
    </tr>
    <tr height="86%">
        <td width="20%" valign="top">







<script type="text/javascript" src="js/ua.js"></script>
<script type="text/javascript" src="js/ftiens4.js"></script>


<script>

    USETEXTLINKS=1
USEFRAMES=0
STARTALLOPEN=0
ICONPATH="/pas/images/"
HIGHLIGHT=1
PERSERVESTATE=1
HIGHLIGHT_COLOR="White"
HIGHLIGHT_BG="Gray"
PAS_FONT="<font size=2 color='Black'>"
CF="<font>"
WRAPTEXT=1



foldersTree = gFld(PAS_FONT + CF, "javascript:undefined" )
aux1 = insFld(foldersTree, gFld(PAS_FONT + "Individual Block Requests" + CF,"javascript:undefined"))
aux2 = insDoc(aux1,gLnk("S", PAS_FONT + "New Block Request"+ CF, "/pas/newBlockRequestStep1.do?reloadModel=N&tRndid=23712"))
aux3 = insDoc(aux1,gLnk("S", PAS_FONT + "New Block Reservation"+ CF, "/pas/newBlockReserveStep1.do?reloadModel=N&tRndid=23712"))
aux4 = insDoc(aux1,gLnk("S", PAS_FONT + "Assign/Cancel/Extend Reservation"+ CF, "/pas/createRsvrBlkLst.do?tRndid=23712"))
aux5 = insDoc(aux1,gLnk("S", PAS_FONT + "Block Effective Date Change"+ CF, "/pas/blkEffDateChange.do?reloadModel=N&tRndid=23712"))
aux6 = insDoc(aux1,gLnk("S", PAS_FONT + "Block Modification"+ CF, "/pas/BlkModPage1.do?tRndid=23712"))
aux7 = insDoc(aux1,gLnk("S", PAS_FONT + "Block Disconnect"+ CF, "/pas/disconnect_blk_rqst.do?tRndid=23712"))
aux8 = insDoc(aux1,gLnk("S", PAS_FONT + "Cancel Block Disconnect"+ CF, "/pas/cancel_disconnect_blk_rqst.do?tRndid=23712"))
aux9 = insDoc(aux1,gLnk("S", PAS_FONT + "Saved Block Requests"+ CF, "/pas/savedBlkRqstList.do?tRndid=23712"))
aux10 = insDoc(aux1,gLnk("S", PAS_FONT + "Modify Pending Request"+ CF, "/pas/modifyPendingRequest.do?tRndid=23712"))
aux11 = insDoc(aux1,gLnk("S", PAS_FONT + "Copy Block Request"+ CF, "/pas/copyBlockRequest.do?tRndid=23712"))
aux12 = insDoc(aux1,gLnk("S", PAS_FONT + "Attach Documents"+ CF, "/pas/attachDocsStep1.do?tRndid=23712"))
aux13 = insFld(foldersTree, gFld(PAS_FONT + "CO/NXX Code Requests" + CF,"javascript:undefined"))
aux14 = insDoc(aux13,gLnk("S", PAS_FONT + "New Code Request"+ CF, "/pas/newCodeRequestStep1.do?reloadModel=N&tRndid=23712"))
aux15 = insDoc(aux13,gLnk("S", PAS_FONT + "Code Modification"+ CF, "/pas/changeCodePartOneStep1.do?tRndid=23712"))
aux16 = insDoc(aux13,gLnk("S", PAS_FONT + "Code Disconnect"+ CF, "/pas/disconnectCodePartOneStep1.do?tRndid=23712"))
aux17 = insDoc(aux13,gLnk("S", PAS_FONT + "Copy Code Request"+ CF, "/pas/copyCodeRequestStep1.do?reloadModel=N&tRndid=23712"))
aux18 = insDoc(aux13,gLnk("S", PAS_FONT + "Multiple Code Request"+ CF, "/pas/multiCodeRequestStep1.do?reloadModel=N&tRndid=23712"))
aux19 = insFld(foldersTree, gFld(PAS_FONT + "Withdraw Pending Requests" + CF,"javascript:undefined"))
aux20 = insDoc(aux19,gLnk("S", PAS_FONT + "Withdraw Pending Request"+ CF, "/pas/withdrawPendingRequest.do?tRndid=23712"))
aux21 = insFld(foldersTree, gFld(PAS_FONT + "Intra SP Block Port Requests" + CF,"javascript:undefined"))
aux22 = insDoc(aux21,gLnk("S", PAS_FONT + "New ISP Block Port Request"+ CF, "/pas/BlkPortRequestPage1.do?tRndid=23712"))
aux23 = insDoc(aux21,gLnk("S", PAS_FONT + "Modify ISP Block Port Request"+ CF, "/pas/BlkPortRequestPage1.do?requestType=Modify&tRndid=23712"))
aux24 = insDoc(aux21,gLnk("S", PAS_FONT + "Disconnect ISP Block Port Request"+ CF, "/pas/BlkPortRequestPage1.do?requestType=Disconnect&tRndid=23712"))
aux25 = insFld(foldersTree, gFld(PAS_FONT + "PSTN Activation" + CF,"javascript:undefined"))
aux26 = insDoc(aux25,gLnk("S", PAS_FONT + "Confirm PSTN - Code"+ CF, "/pas/preConfirmPSTN.do?tRndid=23712"))
aux27 = insDoc(aux25,gLnk("S", PAS_FONT + "Delayed PSTN Activation "+ CF, "/pas/delayedPSTNActivation.do?tRndid=23712"))
aux28 = insFld(foldersTree, gFld(PAS_FONT + "Confirm Resources In Service" + CF,"javascript:undefined"))
aux29 = insDoc(aux28,gLnk("S", PAS_FONT + "Create Part4 - Block"+ CF, "/pas/part4FormStep0.do?reloadModel=N&tRndid=23712"))
aux30 = insDoc(aux28,gLnk("S", PAS_FONT + "Create Part4 - Dedicated Customer"+ CF, "/pas/part4FormDedCodeStep0.do?tRndid=23712"))
aux31 = insFld(foldersTree, gFld(PAS_FONT + "Submit Forecast" + CF,"javascript:undefined"))
aux32 = insDoc(aux31,gLnk("S", PAS_FONT + "Create/Modify Forecast"+ CF, "/pas/createForecast.do?tRndid=23712"))
aux33 = insFld(foldersTree, gFld(PAS_FONT + "Search Forms" + CF,"javascript:undefined"))
aux34 = insDoc(aux33,gLnk("S", PAS_FONT + "View Form"+ CF, "/pas/searchViewStep1.do?tRndid=23712"))
aux35 = insDoc(aux33,gLnk("S", PAS_FONT + "List Forms"+ CF, "/pas/listFormsStep1.do?reloadModel=N&tRndid=23712"))
aux36 = insFld(foldersTree, gFld(PAS_FONT + "Reports" + CF,"javascript:undefined"))
aux37 = insDoc(aux36,gLnk("S", PAS_FONT + "Forecast Report"+ CF, "/pas/forecastReportSelect.do?reloadModel=N&tRndid=23712"))
aux38 = insDoc(aux36,gLnk("S", PAS_FONT + "Disconnect And Donation Report"+ CF, "/pas/disconnectAndDonationReportSelect.do?reloadModel=N&tRndid=23712"))
aux39 = insDoc(aux36,gLnk("S", PAS_FONT + "Part 1/1A Report"+ CF, "/pas/part1AReportSelect.do?reloadModel=N&tRndid=23712"))
aux40 = insDoc(aux36,gLnk("S", PAS_FONT + "Part 1B Report"+ CF, "/pas/part1BReportSelect.do?reloadModel=N&tRndid=23712"))
aux41 = insDoc(aux36,gLnk("S", PAS_FONT + "Part 3 Report"+ CF, "/pas/part3ReportSelect.do?reloadModel=N&tRndid=23712"))
aux42 = insDoc(aux36,gLnk("S", PAS_FONT + "Part 4 Report"+ CF, "/pas/part4ReportSelect.do?reloadModel=N&tRndid=23712"))
aux43 = insDoc(aux36,gLnk("S", PAS_FONT + "Assignment Needing Part 4 Report"+ CF, "/pas/part4AssignmentReportSelect.do?reloadModel=N&tRndid=23712"))
aux57 = insDoc(aux36,gLnk("S", PAS_FONT + "Total Numbering Resources Report"+ CF, "/pas/totalNumberingResourcesReportSelect.do?reloadModel=N&tRndid=23712"))
aux58 = insFld(foldersTree, gFld(PAS_FONT + "User Profile" + CF,"javascript:undefined"))
aux59 = insDoc(aux58,gLnk("S", PAS_FONT + "Edit User Profile"+ CF, "/pas/UpdateExtUserProfile.do?tRndid=23712"))
aux60 = insDoc(aux58,gLnk("S", PAS_FONT + "Reset Password"+ CF, "/pas/PasswordReset.do?tRndid=23712"))

		initializeDocument()
	</script>

        </td>
        <td width="80%" valign="top">







<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
    <title>New Block Request</title>
    <script type="text/javascript" src="/pas/js/jquery/jquery.form.js"></script>
     <script language="Javascript">
        $(document).ready(function () {
            $("#selectedState").change(submitState);             
            $("#npa").change(submitForm);
            $("#appTypeNd").change(submitForm);

        });

        function submitForm() {
            
            document.blockRequestForm.method = "post";
            document.blockRequestForm.action = "/pas/newBlockRequestStep2.do?reloadModel=Y";
            document.blockRequestForm.submit();
        }
        function submitState() {
             $("#rateCenter").attr("value","");
            document.blockRequestForm.method = "post";
            document.blockRequestForm.action = "/pas/newBlockRequestStep2.do?reloadModel=Y";
            document.blockRequestForm.submit();
        }

        function validateState() {
            var stateAbbr = $("#selectedState").val();
           
            if (stateAbbr != null && stateAbbr == "") {
                alert("Please select a state ");
                return false;
            }
        }
    </script>
</head>
<body>
<form id="blockRequestForm" name="blockRequestForm" action="newBlockRequestStep2.do" method="POST" onsubmit="return validateState()">
    <br>
	<table width="80%" border="0" cellspacing="0" cellpadding="0" align="center">
		<tr height="28">
			<td colspan="2">&nbsp;</td>
		</tr>
		<tr>
			<td colspan="2" class="tableHeader1"><div align="center">Request Resources</div></td>
		</tr>
		<tr>
			<td colspan="2">&nbsp;</td>
		</tr>
	</table>
	
    <table width="100%" border="0" bgcolor="#FFFFFF">
        
            
        
             
    </table>
    <table width="80%" border="0" align="center" bgcolor="#FFFFFF">
        <tr align="center">
            <td colspan="2">
                <hr width="70%">
            </td>
        </tr>
        <tr>
            <td width="50%" align="right">State<font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%"><select id="selectedState" name="selectedState">
                
                
                    <option value="">Please Select</option>
                
                    <option value="CA">CALIFORNIA</option>
                

            </select></td>
        </tr>
        <tr>
            <td width="50%" align="right">NPA<font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%"><select id="npa" name="formBlkRqst.npaId">
                

                
                    <option value="">Please Select</option>

                
                    <option value="209">209</option>

                
                    <option value="213">213</option>

                
                    <option value="310">310</option>

                
                    <option value="323">323</option>

                
                    <option value="408">408</option>

                
                    <option value="415">415</option>

                
                    <option value="424">424</option>

                
                    <option value="442">442</option>

                
                    <option value="510">510</option>

                
                    <option value="530">530</option>

                
                    <option value="559">559</option>

                
                    <option value="562">562</option>

                
                    <option value="619">619</option>

                
                    <option value="626">626</option>

                
                    <option value="628">628</option>

                
                    <option value="650">650</option>

                
                    <option value="657">657</option>

                
                    <option value="661">661</option>

                
                    <option value="669">669</option>

                
                    <option value="707">707</option>

                
                    <option value="714">714</option>

                
                    <option value="747">747</option>

                
                    <option value="760">760</option>

                
                    <option value="805">805</option>

                
                    <option value="818">818</option>

                
                    <option value="831">831</option>

                
                    <option value="858">858</option>

                
                    <option value="909">909</option>

                
                    <option value="916">916</option>

                
                    <option value="925">925</option>

                
                    <option value="949">949</option>

                
                    <option value="951">951</option>

                
            </select></td>

        </tr>
        <tr>
            <td width="50%" align="right">Rate Center<font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%"><select id="rateCenter" name="formBlkRqst.rtCntrId">
                
                
                    <option value="">Please Select</option>
                
            </select></td>
        </tr>
        <tr>
            <td width="50%" align="right">OCN <font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%"><select id="ocn" name="formBlkRqst.ocn">
                
                
                    <option value="">Please Select</option>
                
                    <option value="320H">320H-VODEX COMMUNICATIONS CORPORATION - CA</option>
                
            </select></td>
        </tr>

        <tr>
            <td width="50%" align="right">Quantity of Blocks Requested <font
                    color="red">*</font> &nbsp; &nbsp;
            </td>
            <td width="50%"><input id="formBlkRqst.quan1kBlksRqstd" name="formBlkRqst.quan1kBlksRqstd" type="text" value="" size="4" maxlength="2"/></td>
        </tr>
        <tr>
            <td width="50%" align="right">If requesting more than one
                block, do you want multiple effective dates? <font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%">
                <input id="formBlkRqst.multEffDtNd1" name="formBlkRqst.multEffDtNd" type="radio" value="Y"/> Yes
                <input id="formBlkRqst.multEffDtNd2" name="formBlkRqst.multEffDtNd" type="radio" value="N" checked="checked"/> No
            </td>
        </tr>

        <tr>
            <td width="50%" align="right">If requesting more than one
                block, do you want multiple switches? <font color="red">*</font>
                &nbsp; &nbsp;
            </td>
            <td width="50%">
                <input id="formBlkRqst.multSwitchNd1" name="formBlkRqst.multSwitchNd" type="radio" value="Y"/> Yes
                <input id="formBlkRqst.multSwitchNd2" name="formBlkRqst.multSwitchNd" type="radio" value="N" checked="checked"/> No
            </td>
        </tr>
        <tr align="center">
            <td colspan="2">
                <hr width="70%">
            </td>
        </tr>
       <tr>
            <td colspan="4" align="center"><input type="submit" name="actionName"
                                                  value="Continue"/> <input type="button" name="Submit3"
                                                                            value="Cancel"
                                                                            onClick="cancelConfirm('Are you sure you want to cancel \n this request?',
                                    '/pas/inbox.do?refresh=true&method=viewWorkItems')">
            </td>
        </tr>
    </table>
</form>
</body>
</body>
</html></td>
    </tr>
    <tr height="2%">
        <td colspan="2" align="center"><!-- Start Footer -->
<table width="100%" border="0" cellspacing="0" cellpadding="0">
  <tr>
    <td align="center" valign="middle" height="2" colspan="3" bgcolor="#cccccc"></td>
  </tr>
  <tr>
    <td align="left" valign="middle" width="81">
	<a href="http://www.my.biz" target="_blank">
	  <img src="images/neustar_logo.gif" width="81" height="19" border="0" alt="www.neustar.biz">
	</a>
    </td>
    <td align="center" valign="middle" width="464">
      <span class="footer">&copy; 2014 NeuStar, Inc.</span>
      <a class="footer" target="_legal" href="/pas/Pool Additional Information - Legal.htm">Legal Notice</a> 
    </td>
    <td align="right" valign="middle" width="200">
      <p class="footer">Last updated:
        <script language="JavaScript">
	<!-- Begin
	  var months = new Array(13);
          months[1] = "January";
          months[2] = "February";
          months[3] = "March";
          months[4] = "April";
          months[5] = "May";
          months[6] = "June";
          months[7] = "July";
          months[8] = "August";
          months[9] = "September";
          months[10] = "October";
          months[11] = "November";
          months[12] = "December";
          var dateObj = new Date(document.lastModified);
          var lmonth = months[dateObj.getMonth() + 1];
          var date = dateObj.getDate();
          var fyear = dateObj.getYear();
          if (fyear < 2000)
	    fyear += 1900;
          document.write(lmonth + " " + date + ", " + fyear)
        // End -->
        </script>.</p>

	<!-- For browsers without JavaScript enabled
    <noscript>
    Javascript not enabled: Script displays date.
    </noscript>
    //-->
    </td>
  </tr>
</table>
<!-- End Footer -->
</td>
    </tr>
</table>
</body>
</html>

Open in new window

You need to login to the site first, which is why I said it depends on how the authentication is performed - otherwise your script will just be asked to login. You being logged in via a browser is seperate from a php script running

You could manually download the page and process it, but I assume you want this to be a click and run if not then very easy
i can login via curl just fin as i said before and yes i will run the code against
$output = curl_exec($ch);

Open in new window

but to test my code i logged in downloaded the page
and try to test against the page that simple
Is the page above you posted the actual HTML from the site?
Logging in via curl doesn't maintain a login.

Ray probably knows more about this than I for persisting a login thru curl, not anything I've ever done. But probably best to sort out the parsing first so you know that is working then work on the auto part of it.
ASKER CERTIFIED SOLUTION
Avatar of Julian Hansen
Julian Hansen
Flag of South Africa image

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Logging in via curl doesn't maintain a login.
Nope.  cURL can act exactly like a well-behaved HTTP browser, accepting and returning cookies and following redirection.  Cookies are the mechanism for a "login" and by returning them, a cURL session can maintain a login.
Now that I see the script posted here, I think this is too big a job for a Q-n-A setting.  There are too many options, rabbit holes, and unanswered questions.  And it looks like whoever built this thing is too smart by half -- there are at least 3 close-body HTML tags and numerous other violations of W3 standards.  It also looks like this is deliberately architected to make it difficult for you to use an automated POST.  I would be inclined to ask the publisher for an API.  Anything you can cobble together to scrape the web document is going to be very brittle and will probably need to be rewritten whenever the publisher makes a change to this web page.  Added bonus: If you use the API you will be at less risk of running afoul of the site's terms of use.

To help you get started, here is what I can see that needs to be done.

Write a cURL script that makes a POST-method request to newBlockRequestStep2.do in their web directory.

Choose values for each of these request variables and make a POST request string.
<form id="blockRequestForm" name="blockRequestForm" action="newBlockRequestStep2.do" method="POST" onsubmit="return validateState()">

<select id="selectedState" name="selectedState">
<select id="npa"           name="formBlkRqst.npaId">
<select id="rateCenter"    name="formBlkRqst.rtCntrId">
<select id="ocn"           name="formBlkRqst.ocn">

<input id="formBlkRqst.quan1kBlksRqstd" name="formBlkRqst.quan1kBlksRqstd" type="text" value="" size="4" maxlength="2"/></td>

<input id="formBlkRqst.multEffDtNd1" name="formBlkRqst.multEffDtNd" type="radio" value="Y"/> Yes
<input id="formBlkRqst.multEffDtNd2" name="formBlkRqst.multEffDtNd" type="radio" value="N" checked="checked"/> No

<input id="formBlkRqst.multSwitchNd1" name="formBlkRqst.multSwitchNd" type="radio" value="Y"/> Yes
<input id="formBlkRqst.multSwitchNd2" name="formBlkRqst.multSwitchNd" type="radio" value="N" checked="checked"/> No

<input type="submit" name="actionName" value="Continue"/>

Open in new window

Fire it up, get a response, later-rinse-repeat.

Recommend you post this requirement in E-E Gigs where you can get hands-on work from a professional developer.
I took the code posted here
and added the JavaScript solution here
Which produced this in the receiving script on the server
Array
(
    [selectedState] => 
    [formBlkRqst_npaId] => 
    [formBlkRqst_rtCntrId] => 
    [formBlkRqst_ocn] => 
    [formBlkRqst_quan1kBlksRqstd] => 
    [formBlkRqst_multEffDtNd] => N
    [formBlkRqst_multSwitchNd] => N
    [action] => http://mrcp9/ee/newBlockRequestStep2.do
)

Open in new window

The advantage with JavaScript is that it already has all the tools you need to scrape a form.