Solved

bash send variable to php using curl

Posted on 2014-07-30
39
501 Views
Last Modified: 2014-08-09
I am using curl to get a file from the remote server as such;

On workstation side;
curl -s -k --key some.key --cert some.crt https://someserver/fileupdate.php -u "$USERNAME:$PASSWD" -o $UPDATE

Open in new window


On php side;

        if($_SERVER['REQUEST_METHOD'] === 'GET'){
        // open the update file and retrieve
                $file = fopen("updates/file-YC8268D.txt","r");

                if (!$file) trigger_error('UNABLE TO OPEN updates/file-YC8268D.txt FOR READ', E_USER_ERROR);
                while (!feof($file)){
                        echo fread($file,1024);
                }

                fclose($file);
        }

Open in new window



I have the name of the file I want as a variable but could I send the file name I am wanting from curl to the PHP app?

Also, once file is downloaded, delete it from server side.
0
Comment
Question by:projects
  • 21
  • 11
  • 7
39 Comments
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40231433
on the client side pass the filename as id in the url
curl -s -k --key some.key --cert some.crt https://someserver/fileupdate.php?id=file-YC8268D.txt -u "$USERNAME:$PASSWD" -o $UPDATE

Open in new window

server side:
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
	// get the filename passed as id
	$fn=isset($_REQUEST["id"])?$_REQUEST["id"]:"";
	//if no filename was passed raise the error
	if ($fn=="")
	{
		trigger_error('No filename was specified', E_USER_ERROR);
	}
	else
	{
		//prefix the file with the path
		$fn="updates/".$fn
		//open the file
		$file = fopen($fn,"r");
	
		if (!$file) 
		{
			trigger_error('UNABLE TO OPEN updates/file-YC8268D.txt FOR READ', E_USER_ERROR);
		}
		else
		{
			while (!feof($file))
			{
      	                         echo fread($file,1024);
			}
			fclose($file);
			//delete the file
			unlink($fn);
		}
        }
}

Open in new window

0
 

Author Comment

by:projects
ID: 40231931
The file seems to be hard coded in your trigger statement. I am sending a variable from curl.
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40231939
elaborate please
0
 
LVL 29

Expert Comment

by:fibo
ID: 40234179
@projects:
- Randy's code is hard-coded at line 19 only, so mou cant probably place $fn at this line
- note that since you are relying on the data provided by the user to determine what to transfer, you MUST sanitize the data (eg, you probably don't want that some script kiddies asks the source code of the php file where you have stored the password of your database)... at the bare minimum you should run a test on file extension requested and see that it is in some "whitelist", eg .txt, .pdf, .gif
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40234224
Just change that line to:
trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);

Open in new window

0
 

Author Comment

by:projects
ID: 40234759
I don't know enough about scripting to understand @Fibo.

Randy, this does nothing at all.
I rain this without the file in the directory then added the file into the directory.
The script never saw the file and never knew it was there.

The file name is being generated on the script side so it sends the file name to php.
Php should check in that directory to see if such a file name exists, if so, script downloads it then the file is deleted. If no file, keep looking and check again the next day.
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40234767
show me the curl command you ran to execute the script
0
 

Author Comment

by:projects
ID: 40234784
curl -s -k --key some.key --cert some.crt https://server/fileupdate.php?id=open-$USERNAME.txt -u "$USERNAME:$PASSWD" -o $UPDATE
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40234828
and did you try to run that url from a browser as well?
0
 

Author Comment

by:projects
ID: 40234931
No because the script makes up the file name. But, I'll give it a try manually and report.
0
 

Author Comment

by:projects
ID: 40234943
Works from the command line but doesn't pick up the file, no errors at all.
0
 

Author Comment

by:projects
ID: 40234953
Hmm, there was an error...
HTTP/1.0 500 Internal Server Error
0
 

Author Comment

by:projects
ID: 40234997
[01-Aug-2014 11:32:41 PHP Parse error:  syntax error, unexpected '$file' (T_VARIABLE) in fileupdate.php on line 57

Line 57 is

            $file = fopen($fn,"r");
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40235018
line above it
$fn="updates/".$fn

Open in new window

is missing the ending ;
$fn="updates/".$fn;

Open in new window

0
 

Author Comment

by:projects
ID: 40235046
Seems to work as advertised.

Only two things.

1: When the file doesn't exist in the directory, it is generating a php error which would be a lot of unnecessary logging. However, if someone was hacking, I'd like to know about it so is there some way of not logging if authenticated, otherwise log?

2: Fibo's command about script kiddies?
Is this safe as it is written now?
0
 
LVL 21

Expert Comment

by:Randy Poole
ID: 40235054
I am not sure what kind of authentication you are using.  Also to correct issue with someone trying to grab files in another directory change:
$fn="updates/".$fn;

Open in new window

to
$fn="updates/".str_replace("..","**",$fn);

Open in new window

this will produce an error if someone is trying to pass a file name like ?id=../../../somefilename to resend your directory tree
0
 

Author Comment

by:projects
ID: 40235125
The authentication is shown in my original message. It's a curl https connection using name/passwd.

But, how do I prevent the logs from being created each time the script checks unless it is an authorized connection?

PHP Warning:  ... failed to open stream: No such file or directory
PHP Fatal error:  UNABLE TO OPEN ... FOR READ
0
 

Author Comment

by:projects
ID: 40236415
I cannot accept a solution until I know it is a safe one. The constant logging is an issue also as this will created lots of useless activity and make it difficult to find actual problems.

Here is what I have so far;

Sender;
function firm_check()
{
        # Check for new file
        echo "Checking for update"
        curl -s -k --key sum.key --cert sum.crt https://server/update.php?id=$USERNAME.bin -u "$USERNAME:$PASSWD" -o
 $UPDATE

        for filename in $(ls /tmp); do
        if [ -e $filename ]
            then
                if [[ $filename =~ .bin$ ]]
                then
#                real command goes here
echo "There is an update"
                fi
            fi
        done
}

Open in new window


PHP Side;
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
        // get the filename passed as id
        $fn=isset($_REQUEST["id"])?$_REQUEST["id"]:"";
        //if no filename was passed raise the error
        if ($fn=="")
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        }
        else
        {
                //prefix the file with the path
                $fn="updates/".str_replace("..","**",$fn);
                //open the file
                $file = fopen($fn,"r");

                if (!$file)
                {
                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
                }
                else
                {
                        while (!feof($file))
                        {
                                 echo fread($file,1024);
                        }
                        fclose($file);
                        //delete the file
                        unlink($fn);
                }
        }
}

Open in new window


And of course, the non stop error in logs;
fopen(updates/0412E6A.txt): failed to open stream: No such file or directory in fileupdate.php on line 57
PHP Fatal error:  UNABLE TO OPEN updates/0412E6A.txt FOR READ in fileupdate.php on line 61

Open in new window


I may have missed something or edited a word/name incorrectly in my attempt to obfuscate but otherwise, everything is at it should be. It all works, other than I am not sure how safe this is nor how to get rid of the log for legitimate checks.
0
 
LVL 29

Expert Comment

by:fibo
ID: 40236992
- If you test that the file exists before opening it, then this will not raise an error. See http://php.net/manual/en/function.file-exis

- At line 4 I would use my usual shortcut:
$fn= trim( ' ' . @$_GET['id'] );
which works in all cases.

- checking rhat the filename has an authorized extension:
$okTypes = array( 'txt' , 'gif', 'pdf');
...
$fn=...
$fileOk=true;
$dotPos=strrpos($fn, '.');
If ( !$dotPos OR (0 == $dotPos) ) $fileOk=false;

if ($fileOk) {
$fileType=substr($fn, ($dotPos + 1));
$fileOk = in_array($fileType, $okTypes);
}
0
What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 29

Expert Comment

by:fibo
ID: 40237009
- Sorry, typing from my phone creates errors. The correct link is
http://fr2.php.net/manual/en/function.file-exists.php

- Also
$fileOk=true;
 $dotPos=strrpos($fn, '.');
 If ( !$dotPos OR (0 == $dotPos) ) $fileOk=false;

Open in new window

would be smarter as
$dotPos=strrpos($fn, '.');
$fileOk = $dotPos AND (1 <= $dotPos);

Open in new window

0
 

Author Comment

by:projects
ID: 40237628
Any chance you could just write this as my final output should look? I don't want to make any errors and I'm not 100% sure of some of the things you mention above.
0
 
LVL 29

Expert Comment

by:fibo
ID: 40238299
Complete code base would be something like
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
        // get the filename passed as id
        $fn= trim( ' ' . @$_GET['id'] ); // use GET since this is what we selected
        //if no filename was passed raise the error
        if ( '' == $fn)
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        } else
        {
                //sanitize filename
                // check extension
                $okTypes = array( 'txt' , 'gif', 'pdf'); // adapt to your case
                $dotPos=strrpos($fn, '.'); // strRpos; CAUTION:  we decide that file requests MUST have an extension
                $fileOk= //combined AND: will stop on first false
                	!(!$dotPos) //make it true or false
                	AND (1 <= $dotPos) ) 
                	AND in_array($substr($fn, ($dotPos + 1)), $okTypes)
                	AND ( '.' <> substr($fn, 0, 1) ) // if start with a '.' possible hack attempt
                	AND !strpos($fn, '/') // no / allowed in filename
                	;
                if !$fileOk { 
                	trigger_error("Illegal filename [$fn], possible hack attempt", E_USER_ERROR);
                } else {
	                //prefix the file with the path
	                $fn="updates/$fn";
	                //test file exists AND openable
	                if !file_exists($fn) {
	                	trigger_error("File $fn does not exist or cannot be accessesd", E_USER_ERROR);
	                } else {
	                	//open the file
		                $file = fopen($fn,"r");
		
		                if (!$file)
		                {
		                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
		                } else {
		                        while (!feof($file))
		                        {
		                                 echo fread($file,1024);
		                        } //while
		                        fclose($file);
		                        //delete the file
		                        unlink($fn);
		                } //else file
		              } // else exists
                } //else FileOK
        } //fn
} // GET

Open in new window

0
 

Author Comment

by:projects
ID: 40239456
Something like or actually safe to use? :)


PHP Parse error:  syntax error, unexpected ')' in fileupdate.php on line 58

AND (1 <= $dotPos) )                                    < Line 58
0
 
LVL 29

Expert Comment

by:fibo
ID: 40239471
Sorry for the extra )
Remove it!
I hav placed additional tests to enhance security.
0
 

Author Comment

by:projects
ID: 40239495
No problem.

I did remove it before posting but then I got this error;

PHP Parse error:  syntax error, unexpected '!', expecting '(' in fileupdate.php on line 63

            if !$fileOk {               < Line 63
0
 

Author Comment

by:projects
ID: 40242588
Are we going to complete this?
0
 
LVL 29

Expert Comment

by:fibo
ID: 40242809
Sorry, on the love with no Access to my computer.
Missing some () there.. I will check then post the correct answer
0
 

Author Comment

by:projects
ID: 40242929
I'd love to accept the solution and close this question so thanks.
0
 
LVL 29

Expert Comment

by:fibo
ID: 40243686
The correct program is
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
        // get the filename passed as id
        $fn= trim( ' ' . @$_GET['id'] ); // use GET since this is what we selected
        //if no filename was passed raise the error
        if ( '' == $fn)
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        } else
        {
                //sanitize filename
                // check extension
                $okTypes = array( 'txt' , 'gif', 'pdf'); // adapt to your case
                $dotPos=strrpos($fn, '.'); // strRpos; CAUTION:  we decide that file requests MUST have an extension
                $fileOk= //combined AND: will stop on first false
                	!(!$dotPos) //ensure it has one of the 2 values true or false
                	AND (1 <= $dotPos)
                	AND in_array($substr($fn, ($dotPos + 1)), $okTypes)
                	AND ( '.' <> substr($fn, 0, 1) ) // if start with a '.' possible hack attempt
                	AND !strpos($fn, '/') // no / allowed in filename
                	;
                if (!$fileOk) { 
                	trigger_error("Illegal filename [$fn], possible hack attempt", E_USER_ERROR);
                } else {
	                //prefix the file with the path
	                $fn="updates/$fn";
	                //test file exists AND openable
	                if (!file_exists($fn)) {
	                	trigger_error("File $fn does not exist or cannot be accessesd", E_USER_ERROR);
	                } else {
	                	//open the file
		                $file = fopen($fn,"r");
		
		                if (!$file)
		                {
		                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
		                } else {
		                        while (!feof($file))
		                        {
		                                 echo fread($file,1024);
		                        } //while
		                        fclose($file);
		                        //delete the file
		                        unlink($fn);
		                } //else file
		              } // else exists
                } //else FileOK
        } //else fn
} // GET

Open in new window

0
 

Author Comment

by:projects
ID: 40243932
Wonderful. Here is the result;

PHP Notice:  Undefined variable: substr in fileupdate.php on line 59
PHP Fatal error:  Function name must be a string in fileupdate.php on line 59

                      AND in_array($substr($fn, ($dotPos + 1)), $okTypes)  < Line 59
0
 
LVL 29

Expert Comment

by:fibo
ID: 40245514
Being unable to access my susal PHP checker lets me fail to spot all these typos.
This should do it now
<?php
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
        // get the filename passed as id
        $fn= trim( ' ' . @$_GET['id'] ); // use GET since this is what we selected
        //if no filename was passed raise the error
        if ( '' == $fn)
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        } else
        {
                //sanitize filename
                // check extension
                $okTypes = array( 'txt' , 'gif', 'pdf'); // adapt to your case
                $dotPos=strrpos($fn, '.'); // strRpos; CAUTION:  we decide that file requests MUST have an extension
                $fileOk= //combined AND: will stop on first false
                	!(!$dotPos) //ensure it has one of the 2 values true or false
                	AND (1 <= $dotPos)
                	AND in_array(substr($fn, ($dotPos + 1)), $okTypes)
                	AND ( '.' <> substr($fn, 0, 1) ) // if start with a '.' possible hack attempt
                	AND (!strpos($fn, '/')) // no / allowed in filename
                	;
                if (!$fileOk) { 
                	trigger_error("Illegal filename [$fn], possible hack attempt", E_USER_ERROR);
                } else {
	                //prefix the file with the path
	                $fn="updates/$fn";
	                //test file exists AND openable
	                if (!file_exists($fn)) {
	                	trigger_error("File $fn does not exist or cannot be accessesd", E_USER_ERROR);
	                } else {
	                	//open the file
		                $file = fopen($fn,"r");
		
		                if (!$file)
		                {
		                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
		                } else {
		                        while (!feof($file))
		                        {
		                                 echo fread($file,1024);
		                        } //while
		                        fclose($file);
		                        //delete the file
		                        unlink($fn);
		                } //else file
		              } // else exists
                } //else FileOK
        } //else fn
} // GET
?>

Open in new window

0
 

Author Comment

by:projects
ID: 40246511
It works!
The only problem which still remains is this error in the php logs which I need to to filter out when it is a legitimate check by an authorized user. The remote script is using an authenticated curl connection to check for the update. I need to not have error logs for legit connections and only for someone trying to do something nefarious :)
 

PHP Fatal error:  File update/68DDB672.txt does not exist or cannot be accessesd in fileupdate.php on line 70

Line 70 is your line 30.
0
 
LVL 29

Expert Comment

by:fibo
ID: 40247528
1 - As you guessed you could comment out this lifting instruction

2 - However, I would probably keep this logging. If a user tries to open a non-existent file, this hints at some problem that should be addressed
0
 

Author Comment

by:projects
ID: 40247730
1 Instead of commenting it out, what would be best is to not log if this is an authenticated connection. Otherwise, log because it means someone is messing around.

2
The problem is that many remotes are checking for this update many times daily so it is filling the logs up very quickly.
0
 
LVL 29

Expert Comment

by:fibo
ID: 40248319
1 - Code change would be to replace
                if (!$file)
                {
                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
                }
                else

Open in new window

with
                if ($file)

Open in new window


1bis - Regarding the check for "an authenticated connection"?
Your initial question just mentions the connection thru https. You can check that before any other code by inserting a test like
if ('' == @$_SERVER['HTTPS']) {
	trigger_error('Uncorrect protocol requested', E_USER_ERROR);
	exit();
}

Open in new window

which will generate the error message then stop execution

2 - Disk space is cheap, so logfile volume is not really a problem.
However, if you have that many errors that log volume is huge, this shows some problem somewhere: who is asking for inexistent files? why? which docs are they given?
This however is out of the scope of your initial question.
0
 

Author Comment

by:projects
ID: 40249709
There was one extra }  at
} // GET
Removing that fixed it.

However, I am still getting the error in the php logs.

PHP Fatal error:  File updates/8DDGEJU.txt does not exist or cannot be accessesd in fileupdate.php on line 70

Also, the command is changed?

 if (!$file)
to
 if ($file)

Lose the exclamation?
0
 

Author Comment

by:projects
ID: 40249785
This is how things look at this point. The only way I could get rid of the php error upon checking for a file which didn't exist, even by an authenticated curl connection was to disable the error message as seen in the code.

if($_SERVER['REQUEST_METHOD'] === 'GET')

                if ('' == @$_SERVER['HTTPS']) {
                trigger_error('Incorrect protocol requested', E_USER_ERROR);
                exit();
                }
{
        // get the filename passed as id
        $fn= trim( ' ' . @$_GET['id'] ); // use GET since this is what we selected
        //if no filename was passed raise the error
        if ( '' == $fn)
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        } else
        {
                //sanitize filename
                // check extension
                $okTypes = array( 'txt' , 'gif', 'pdf'); // adapt to your case
                $dotPos=strrpos($fn, '.'); // strRpos; CAUTION:  we decide that file requests MUST have an extension
                $fileOk=  //combined AND: will stop on first false
                        !(!$dotPos) //ensure it has one of the 2 values true or false
                        AND (1 <= $dotPos)
                        AND in_array(substr($fn, ($dotPos + 1)), $okTypes)
                        AND ( '.' <> substr($fn, 0, 1) ) // if start with a '.' possible hack attempt
                        AND (!strpos($fn, '/')) // no / allowed in filename
                        ;
                if (!$fileOk) {
                        trigger_error("Illegal filename [$fn], possible hack attempt", E_USER_ERROR);
                } else {
                        //prefix the file with the path
                        $fn="updates/$fn";
                        //test file exists AND openable
                        if (!file_exists($fn)) {
//                              trigger_error("File $fn does not exist or cannot be accessed", E_USER_ERROR);
                        } else {
                                //open the file
                                $file = fopen($fn,"r");

                                if ($file)
                                        while (!feof($file))
                                        {
                                                 echo fread($file,1024);
                                        } //while
                                        fclose($file);
                                        //delete the file
                                        unlink($fn);
                                } //else file
                              } // else exists
                } //else FileOK
        } //else fn
// GET

Open in new window

0
 
LVL 29

Accepted Solution

by:
fibo earned 500 total points
ID: 40250003
Seems I did not remove the right condition.
The revised code is below.
<?php
if ('' == @$_SERVER['HTTPS']) {
	trigger_error('Uncorrect protocol requested', E_USER_ERROR);
	exit();
}
if($_SERVER['REQUEST_METHOD'] === 'GET')
{
        // get the filename passed as id
        $fn= trim( ' ' . @$_GET['id'] ); // use GET since this is what we selected
        //if no filename was passed raise the error
        if ( '' == $fn)
        {
                trigger_error('No filename was specified', E_USER_ERROR);
        } else
        {
                //sanitize filename
                // check extension
                $okTypes = array( 'txt' , 'gif', 'pdf'); // adapt to your case
                $dotPos=strrpos($fn, '.'); // strRpos; CAUTION:  we decide that file requests MUST have an extension
                $fileOk= //combined AND: will stop on first false
                	!(!$dotPos) //ensure it has one of the 2 values true or false
                	AND (1 <= $dotPos)
                	AND in_array(substr($fn, ($dotPos + 1)), $okTypes)
                	AND ( '.' <> substr($fn, 0, 1) ) // if start with a '.' possible hack attempt
                	AND (!strpos($fn, '/')) // no / allowed in filename
                	;
                if (!$fileOk) { 
                	trigger_error("Illegal filename [$fn], possible hack attempt", E_USER_ERROR);
                } else {
	                //prefix the file with the path
	                $fn="updates/$fn";
	                //test file exists AND openable
	                if (@file_exists($fn)) {
//	                	trigger_error("File $fn does not exist or cannot be accessesd", E_USER_ERROR);
//	                } else {
	                	//open the file
		                $file = @fopen($fn,"r");
		
		                if !(!$file)
		                {
//		                        trigger_error('UNABLE TO OPEN '.$fn.' FOR READ', E_USER_ERROR);
//		                } else {
		                        while (!feof($file))
		                        {
		                                 echo fread($file,1024);
		                        } //while
		                        fclose($file);
		                        //delete the file
		                        unlink($fn);
		                } //if file
		              } // if exists
                } //else FileOK
        } //else fn
} // GET
?>

Open in new window

0
 

Author Closing Comment

by:projects
ID: 40250792
I so appreciate people who stick to something until it's done! Thank you very much.

By the way, the only thing that was still wrong with the solution was an extra exclamation mark which I removed.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

If I have to fix slow responding website my first thoughts are server side optimizations: the database may not be optimized or caching is not enabled, or things like that. We often overlook another major part of our web application: the client. We o…
What is Node.js? Node.js is a server side scripting language much like PHP or ASP but is used to implement the complete package of HTTP webserver and application framework. The difference is that Node.js’s execution engine is asynchronous and event…
This video teaches users how to migrate an existing Wordpress website to a new domain.
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now