Solved

Grep working on command but not working in shell script

Posted on 2014-02-06
16
945 Views
Last Modified: 2014-02-24
Hello,

I call a bash script using the following php code:
<?php
        if($_POST['formSearch'] == "Search")
        {
               system("sh /sysadm/shared/NCC/ftp_search.sh $_POST[TYPE] $_POST[SEARCH] $_POST[DATE] $_POST[SERVER]", $return_val);
        }
?>

Open in new window

It runs the script and I'm able to echo the date and the table header but it's not running the script, or if it is it's not outputting the the ftp_search.$DATE file.  I've run this grep command on the command line and it works.  I'm able to echo all variables and I've also opened up the permissions on all files and directories for now.  Any suggestions would be appreciated.
#!/bin/bash

# Utility to search FTP Logs for specified User and Date

SEARCH=$2
DATE=$3
SYSTEM=$4
TYPE=$1

echo $DATE
grep $SEARCH logs/$SYSTEM/ftp_log.$DATE > tmp/ftp_search.$DATE

echo '<table border="1" width="500"  font color="#0000FF" size="1" face="Arial" style="font-family: Arial; color: #0000FF; font-size: 8pt"  bgcolor="#99FF99" bordercolor="#CCFFCC" cellspacing="0" cellpadding="5">'

echo '<tr><td valign="top" align="center" bgcolor="#800000"> <font color="#FFFFFF"><b>Search</b></font></td></tr>'

Open in new window

Thank you,
Dave
0
Comment
Question by:dloszewski
16 Comments
 
LVL 109

Expert Comment

by:Ray Paseur
ID: 39839868
Normally you see something like this: $_POST[TYPE] written like this: $_POST['TYPE'] with quotes around the array index name.  Without the quotes, PHP things that TYPE is a defined constant, and that may be causing some confusion.

What are you trying to do, just in plain language (no technical explanation needed)?

Thanks, ~Ray
0
 

Author Comment

by:dloszewski
ID: 39839907
The PHP code is on a page called ftp_index.php which I access from the web.  I make a bunch of selections, ftp server name, type, date, and criteria and then when I hit the Search button on the form it calls the ftp_search.sh script that's in another directory.  I echo'd all the values that were passed in ftp_search.sh script and all values were echoing out correctly so they look like they're getting passed.  Just for the heck of it I tried without the quotes and it didn't make a difference. Although when I put quotes around the variables being passed the php page wouldn't even fun.  

Basically what I"m trying to do is just call a script that will run a grep on some ftp logs files and will output those matches to the webpage.
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39840091
Might be a "current directory" problem. Are you sure that "log" is a subdirectory of the directory that the script is running in?

Can you put in the full path to log/$SYSTEM?
0
DevOps Toolchain Recommendations

Read this Gartner Research Note and discover how your IT organization can automate and optimize DevOps processes using a toolchain architecture.

 
LVL 16

Expert Comment

by:AlexPace
ID: 39842766
When it runs in script does the process have the same file and folder permissions as when you run it from the command line?
0
 

Author Comment

by:dloszewski
ID: 39844177
yes, same permissions.  And yes, the log is in a subdirectory of the directory that the script is running in.  I've also tried putting the full paths in and it still did not make a difference.  I just checked the permissions again and it's all wide open.
0
 

Author Comment

by:dloszewski
ID: 39844198
Is there a way for me to debug it and have the debug/error log go to an outside file?

I've tried calling the script using a '#!/bin/bash -x 2>&1l out.log' and using set +x and -x but I get no out.log file.
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
ID: 39845233
Start the script with
  #!/bin/bash
  exec > /tmp/out.log 2>&1
Better to use an explicit path - it's not always obvious where a simple "out.log" file may end up!
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39845238
And then add "set -x"
0
 

Author Comment

by:dloszewski
ID: 39847099
That doesn't look to have created any file either.  Here's the full code for that file:
#!/bin/bash
exec > out.log 2>&1

# Utility to search FTP Logs for specified User and Date

SEARCH=$2
DATE=$3
SYSTEM=$4
TYPE=$1

#grep $TYPE logs/$SYSTEM/ftp_log.$DATE | grep $SEARCH > tmp/ftp_search.$$

set -x
echo $DATE
grep $SEARCH logs/$SYSTEM/ftp_log.$DATE > tmp/ftp_search.$DATE
echo test2
set +x

echo '<table border="1" width="500"  font color="#0000FF" size="1" face="Arial" style="font-family: Arial; color: #0000FF; font-size: 8pt"  bgcolor="#99FF99" bordercolor="#CCFFCC" cellspacing="0" cellpadding="5">'

echo '<tr><td valign="top" align="center" bgcolor="#800000"> <font color="#FFFFFF"><b>Search</b></font></td></tr>'

while read FTP_SEARCH
do
        rpt_start=`echo $FTP_SEARCH | awk -F"|" '{print $1}'`

        echo '<tr>'
        echo '    <td valign=top>'$rpt_start'</td>'
        echo '</tr>'

done < tmp/ftp_search.$DATE

echo '</table>'

rm tmp/ftp_search.$DATE

Open in new window

0
 
LVL 19

Expert Comment

by:simon3270
ID: 39847261
If it didn't create out.log, then I don't think it ran.

Did you try /tmp/out.log too?

I'd be vary cautious about having relative filenames and paths (your tmp/ftp_search really ought to be /tmp/ftp_search).  You may know that the script wil be run in the root directory, but if someone installes a new web server, that may not be the case. Also, I assume that this isn't runnign as the root user - in which case, the "exec > out.log 2>&1" may not be able to write to the current directory.

You said earlier that you'd managed to echo the parameters when calling this from php - could you do that same again, and this time output the value of the pwd command?
0
 

Author Comment

by:dloszewski
ID: 39864777
Ok, so I decided to move everything to one directory so there are no issues going between directories, unfortunately I'm still coming up with the same issue.  I also decided to forgo passing the parameters from PHP and instead just call the script so it would rule out any issues passing the parameters.  So now I have:

grep 4683 sslmftp1_messages_20131202 > ftp_search.14-02-10

Open in new window


$ pwd
/sysadm/shared/www/NCC

drwxr-xr-x  2 dloszews    126     4096 Feb 17 08:51 NCC

$ ls -l
total 63412
-rw-r--r-- 1 dloszews 126     1054 Feb 10 15:01 ftp_index.php
-rw-r--r-- 1 dloszews 126      960 Feb 16 14:44 ftp_search.sh
-rw-r--r-- 1 dloszews 126 64667581 Feb 10 15:00 sslmftp1_messages_20131202

Perhaps I'm looking at this the wrong way.  Instead of sending stdout from the sh file is it possible ot send the stdout to the browser from the PHP file so I can see whatever error this is creating? I'm assuming it has to do with some kind of permission issue with the grep, although I also tried running these from my home directory and I still had the same issue.  If I take that grep command in the script and run it from the command line it works fine.  If I put 'echo before' and 'echo after' before and after the grep command in the shell script then I get 'before after' in my browswer so I know it is calling the shell and making it past the grep command.
0
 

Author Comment

by:dloszewski
ID: 39864784
I have also tried to change the permissions of the files to make them wide open:

$ ls -l
total 63412
-rwxrwxrwx 1 dloszews 126     1054 Feb 10 15:01 ftp_index.php
-rwxrwxrwx 1 dloszews 126      960 Feb 16 14:44 ftp_search.sh
-rwxrwxrwx 1 dloszews 126 64667581 Feb 10 15:00 sslmftp1_messages_20131202
0
 

Author Comment

by:dloszewski
ID: 39870635
I'm running this on a different system where I have total control now and this still doesn't work
0
 

Author Comment

by:dloszewski
ID: 39870991
so I finally got into the httpd log and I'm getting the following on the grep:

$ cd httpd
ksh: cd: /var/log/httpd - Permission denied
0
 
LVL 19

Expert Comment

by:simon3270
ID: 39872823
I can't see any "cd" commands in your script - not sure what's going on.  Does a normal user have access to the log files you are searching?  In particular, does the apache user have access?  Do they have full access along the entire path to those files?  (it's no use having 777 permissions on the final directory if they are 700 earlier in the path and they don't own that directory).

I see you are still running "grep" on a bare file name, and writing to a bare file name.  Can you not add full paths to the filenames used?
0
 

Author Comment

by:dloszewski
ID: 39882507
decided to go a different route using awk since I could still not figure this out.  Thank you all for your help.
0

Featured Post

Master Your Team's Linux and Cloud Stack

Come see why top tech companies like Mailchimp and Media Temple use Linux Academy to build their employee training programs.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Nothing in an HTTP request can be trusted, including HTTP headers and form data.  A form token is a tool that can be used to guard against request forgeries (CSRF).  This article shows an improved approach to form tokens, making it more difficult to…
The viewer will learn how to create a basic form using some HTML5 and PHP for later processing. Set up your basic HTML file. Open your form tag and set the method and action attributes.: (CODE) Set up your first few inputs one for the name and …
In this fourth video of the Xpdf series, we discuss and demonstrate the PDFinfo utility, which retrieves the contents of a PDF's Info Dictionary, as well as some other information, including the page count. We show how to isolate the page count in a…

831 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question