• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 380
  • Last Modified:

Why is this curl operation failing?

I have the following PHP script that is attempting to post a file to another server using CURL.  

<?php

	$post_url = "https://xxx.xxx.org/cme_put.jsp";

    $ch = curl_init();

    curl_setopt($ch, CURLOPT_VERBOSE, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)");
    curl_setopt($ch, CURLOPT_POST, true);
    curl_setopt($ch, CURLOPT_URL, $post_url );
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true); // follow redirects
    curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
	
    $post_array = array(
        'my_file'=>'@/home/username/mysite.org/xxx/test.xml',
        "upload"=>"Upload"
    );

    curl_setopt($ch, CURLOPT_POSTFIELDS, $post_array);
 
    $response = curl_exec($ch);

    echo "<pre>";
	print_r(curl_getinfo($ch));
	echo "</pre>";
    
    //echo curl_getinfo($ch);
	if (curl_error($ch))
		echo curl_error($ch);
	else
		echo $response;
		
	curl_close($ch);
?>

Open in new window


The response I get from the remote server is:

Array
(
    [url] => https://xxx.xxx.org/data/test.xml
    [content_type] => text/html; charset=iso-8859-1
    [http_code] => 404
    [header_size] => 515
    [request_size] => 397
    [filetime] => -1
    [ssl_verify_result] => 0
    [redirect_count] => 1
    [total_time] => 1.059611
    [namelookup_time] => 2.3E-5
    [connect_time] => 0.07374
    [pretransfer_time] => 0.147874
    [size_upload] => 0
    [size_download] => 296
    [speed_download] => 279
    [speed_upload] => 0
    [download_content_length] => 296
    [upload_content_length] => 0
    [starttransfer_time] => 0.33763
    [redirect_time] => 0.721975
)

Not Found

The requested URL /data/test.xml was not found on this server.

Open in new window


However, if I do a normal form post with a file upload field to https://xxx.xxx.org/cme_put.jsp it works fine.

What would be causing curl to fail?
0
Jason C. Levine
Asked:
Jason C. Levine
  • 8
  • 6
3 Solutions
 
Ray PaseurCommented:
[http_code] => 404

Script not found?
0
 
Ray PaseurCommented:
Here is my teaching example of a CURL POST.  See if it helps.  But I am thinking that a 404 response will not trip the CURL error handler.
<?php // RAY_curl_post_example.php
error_reporting(E_ALL);


// DEMONSTRATE HOW TO USE CURL POST TO START AN ASYNCHRONOUS PROCESS


function curl_post($url, $post_array, $timeout=2, $error_report=FALSE)
{
    // PREPARE THE POST STRING
    $post_string = NULL;
    foreach ($post_array as $key => $val)
    {
        $post_string .= urlencode($key) . '=' . urlencode($val) . '&';
    }
    $post_string = rtrim($post_string, '&');

    // PREPARE THE CURL CALL
    $curl = curl_init();
    curl_setopt( $curl, CURLOPT_URL,            $url         );
    curl_setopt( $curl, CURLOPT_HEADER,         FALSE        );
    curl_setopt( $curl, CURLOPT_POST,           TRUE         );
    curl_setopt( $curl, CURLOPT_POSTFIELDS,     $post_string );
    curl_setopt( $curl, CURLOPT_TIMEOUT,        $timeout     );
    curl_setopt( $curl, CURLOPT_RETURNTRANSFER, TRUE         );

    // EXECUTE THE CURL CALL
    $htm = curl_exec($curl);
    $err = curl_errno($curl);
    $inf = curl_getinfo($curl);

    // ON FAILURE
    if (!$htm)
    {
        // PROCESS ERRORS HERE
        if ($error_report)
        {
            echo "CURL FAIL: $url TIMEOUT=$timeout, CURL_ERRNO=$err";
            echo "<pre>\n";
            var_dump($inf);
            echo "</pre>\n";
        }
        curl_close($curl);
        return FALSE;
    }

    // ON SUCCESS
    curl_close($curl);
    return $htm;
}


// USAGE EXAMPLE CREATES ASSOCIATIVE ARRAY OF KEY=>VALUE PAIRS
$args["name"]  = 'Ray';
$args["email"] = 'Ray.Paseur@Gmail.com';

// ACTIVATE THIS TO SEE THE ARRAY OF ARGS
// var_dump($args);

// SET THE URL
$url = "http://LAPRBass.com/RAY_bounce_post.php";

// CALL CURL TO POST THE DATA
$htm = curl_post($url, $args, 3, TRUE);

// SHOW WHAT CAME BACK, IF ANYTHING
if ($htm)
{
    echo "<pre>";
    echo htmlentities($htm);
}
else
{
    echo "NO RESPONSE YET FROM $url -- MAYBE BECAUSE IT IS RUNNING ASYNCHRONOUSLY";
}

Open in new window

0
 
Jason C. LevineNo oneAuthor Commented:
I'm not sure, Ray.  The result is from the remote server so I have less of an ability to check on it.  I'm posting the array from the first script and echoing the response and it's complaining that the file we're trying to transfer isn't there.  

So curl is not doing what I think it's doing.  Why?
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
Jason C. LevineNo oneAuthor Commented:
Let me work through the sample, then.
0
 
Ray PaseurCommented:
If you want to post the URL I would be glad to try it from my end.  If you want to obscure it a little bit, that's OK, just tell me the way to decode it.
0
 
Dave BaldwinFixer of ProblemsCommented:
I've noticed on a number of questions recently that some sites are using cookies and javascript to make it difficult to use curl or non-browser methods to post data to them.  Use something like Fiddler or LiveHTTP Headers to see what goes on when you use Firefox to do the upload.
0
 
Ray PaseurCommented:
Hey Jason - I need to see the page that has the form on it.  Can you show me that one, please?  Thanks, ~Ray
0
 
Jason C. LevineNo oneAuthor Commented:
I will, but I'm away from the normal resources today.  Tomorrow.

Thanks to both of you for your help.
0
 
Ray PaseurCommented:
10-4.  I'll be here.  
0
 
Jason C. LevineNo oneAuthor Commented:
Ray,

Form source attached:


<?php 
	$post_url = "https://somewhere.org/cme_put.jsp";
?>

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>Insert title here</title>
</head>
<body>

<form method="post" enctype="multipart/form-data" action="<?php echo $post_url ?>">
<input type="file" name="xmlFile" />
<input type="submit" name="btnSubmit" />
</form>


<?php // echo getcwd() ?>

</body>
</html>

Open in new window

0
 
Ray PaseurCommented:
Am I on firm ground if I assume that you have tested with the form, and it works perfectly every time?
0
 
Ray PaseurCommented:
OK, I've been thinking about this a little and I'd like to suggest a design change.  Let me start by saying that I am not 100% certain about this, but what I am writing here feels kind of intuitively correct to me.  I'll try to explain.

You can prepopulate almost everything in HTML forms except for a file name.  HTML <input type="file" /> cannot be prepopulated because it would present a huge security hole.  If you could prepopulate that field with a path to a file on the client machine, it would enable the server to get files from a client machine without the client making an affirmative action to send the file.  So it is not allowed, at least not by the browser.

This may not be allowed by CURL, either.  

It's possible that the error arose because the original CURL post contained the wrong name for the input (my_file and upload, instead of xmlFile and btnSubmit).  In that case, the action script would not have found the expected inputs and it would naturally fail.  I am not sure about this.

If you cannot prepopulate the path to the file with CURL, the design change would go something like this.  Read the XML string, urlencode() the string, or base64_encode() if you're transmitting binary data, and attach the string to a POST method request to the SOMEWHERE.org site.  Obviously this will require a change in the action script at the SOMEWHERE.org server.  Where they are now expecting to receive an uploaded file, they will instead need to receive and decode the XML string from the POST data.

Sidebar note: The SOMEWHERE.org server sets a cookie that looks like this code snippet.  But since there is only one request in the script, my example is not returning the cookie.  So DaveBaldwin has a good point.  A better test than using POST for the first request would be to use GET to read the form. Then you can return the cookie with the POST.  If you want to post the URL of the form page I can make one more test using first a GET request for the form, then a POST request to submit the data.  But I doubt that will work for the security reasons outlined above.

Best regards, ~Ray
# Netscape HTTP Cookie File
# http://curl.haxx.se/rfc/cookie_spec.html
# This file was generated by libcurl! Edit at your own risk.

home.SOMEWHERE.org	FALSE	/	TRUE	0	JSESSIONID	AD21506918BA16276BDBF8FAD9256407

Open in new window

0
 
Jason C. LevineNo oneAuthor Commented:
Ray,

Yes, the form worked first time/every time.

We are investigating the possibility of the above.
0
 
Jason C. LevineNo oneAuthor Commented:
And after all that time, it turned out to be a misconfig.  Sigh.  Thanks for the great advice and starter scripts, Ray.
0
 
Ray PaseurCommented:
You're welcome, and thanks for the points!  All the best, ~Ray
0

Featured Post

Free Tool: Subnet Calculator

The subnet calculator helps you design networks by taking an IP address and network mask and returning information such as network, broadcast address, and host range.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

  • 8
  • 6
Tackle projects and never again get stuck behind a technical roadblock.
Join Now