Solved

Grep working on command but not working in shell script

Posted on 2014-02-06
16
897 Views
Last Modified: 2014-02-24
Hello,

I call a bash script using the following php code:
<?php
        if($_POST['formSearch'] == "Search")
        {
               system("sh /sysadm/shared/NCC/ftp_search.sh $_POST[TYPE] $_POST[SEARCH] $_POST[DATE] $_POST[SERVER]", $return_val);
        }
?>

Open in new window

It runs the script and I'm able to echo the date and the table header but it's not running the script, or if it is it's not outputting the the ftp_search.$DATE file.  I've run this grep command on the command line and it works.  I'm able to echo all variables and I've also opened up the permissions on all files and directories for now.  Any suggestions would be appreciated.
#!/bin/bash

# Utility to search FTP Logs for specified User and Date

SEARCH=$2
DATE=$3
SYSTEM=$4
TYPE=$1

echo $DATE
grep $SEARCH logs/$SYSTEM/ftp_log.$DATE > tmp/ftp_search.$DATE

echo '<table border="1" width="500"  font color="#0000FF" size="1" face="Arial" style="font-family: Arial; color: #0000FF; font-size: 8pt"  bgcolor="#99FF99" bordercolor="#CCFFCC" cellspacing="0" cellpadding="5">'

echo '<tr><td valign="top" align="center" bgcolor="#800000"> <font color="#FFFFFF"><b>Search</b></font></td></tr>'

Open in new window

Thank you,
Dave
0
Comment
Question by:dloszewski
16 Comments
 
LVL 108

Expert Comment

by:Ray Paseur
Comment Utility
Normally you see something like this: $_POST[TYPE] written like this: $_POST['TYPE'] with quotes around the array index name.  Without the quotes, PHP things that TYPE is a defined constant, and that may be causing some confusion.

What are you trying to do, just in plain language (no technical explanation needed)?

Thanks, ~Ray
0
 

Author Comment

by:dloszewski
Comment Utility
The PHP code is on a page called ftp_index.php which I access from the web.  I make a bunch of selections, ftp server name, type, date, and criteria and then when I hit the Search button on the form it calls the ftp_search.sh script that's in another directory.  I echo'd all the values that were passed in ftp_search.sh script and all values were echoing out correctly so they look like they're getting passed.  Just for the heck of it I tried without the quotes and it didn't make a difference. Although when I put quotes around the variables being passed the php page wouldn't even fun.  

Basically what I"m trying to do is just call a script that will run a grep on some ftp logs files and will output those matches to the webpage.
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
Might be a "current directory" problem. Are you sure that "log" is a subdirectory of the directory that the script is running in?

Can you put in the full path to log/$SYSTEM?
0
 
LVL 16

Expert Comment

by:AlexPace
Comment Utility
When it runs in script does the process have the same file and folder permissions as when you run it from the command line?
0
 

Author Comment

by:dloszewski
Comment Utility
yes, same permissions.  And yes, the log is in a subdirectory of the directory that the script is running in.  I've also tried putting the full paths in and it still did not make a difference.  I just checked the permissions again and it's all wide open.
0
 

Author Comment

by:dloszewski
Comment Utility
Is there a way for me to debug it and have the debug/error log go to an outside file?

I've tried calling the script using a '#!/bin/bash -x 2>&1l out.log' and using set +x and -x but I get no out.log file.
0
 
LVL 19

Accepted Solution

by:
simon3270 earned 500 total points
Comment Utility
Start the script with
  #!/bin/bash
  exec > /tmp/out.log 2>&1
Better to use an explicit path - it's not always obvious where a simple "out.log" file may end up!
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
And then add "set -x"
0
Better Security Awareness With Threat Intelligence

See how one of the leading financial services organizations uses Recorded Future as part of a holistic threat intelligence program to promote security awareness and proactively and efficiently identify threats.

 

Author Comment

by:dloszewski
Comment Utility
That doesn't look to have created any file either.  Here's the full code for that file:
#!/bin/bash
exec > out.log 2>&1

# Utility to search FTP Logs for specified User and Date

SEARCH=$2
DATE=$3
SYSTEM=$4
TYPE=$1

#grep $TYPE logs/$SYSTEM/ftp_log.$DATE | grep $SEARCH > tmp/ftp_search.$$

set -x
echo $DATE
grep $SEARCH logs/$SYSTEM/ftp_log.$DATE > tmp/ftp_search.$DATE
echo test2
set +x

echo '<table border="1" width="500"  font color="#0000FF" size="1" face="Arial" style="font-family: Arial; color: #0000FF; font-size: 8pt"  bgcolor="#99FF99" bordercolor="#CCFFCC" cellspacing="0" cellpadding="5">'

echo '<tr><td valign="top" align="center" bgcolor="#800000"> <font color="#FFFFFF"><b>Search</b></font></td></tr>'

while read FTP_SEARCH
do
        rpt_start=`echo $FTP_SEARCH | awk -F"|" '{print $1}'`

        echo '<tr>'
        echo '    <td valign=top>'$rpt_start'</td>'
        echo '</tr>'

done < tmp/ftp_search.$DATE

echo '</table>'

rm tmp/ftp_search.$DATE

Open in new window

0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
If it didn't create out.log, then I don't think it ran.

Did you try /tmp/out.log too?

I'd be vary cautious about having relative filenames and paths (your tmp/ftp_search really ought to be /tmp/ftp_search).  You may know that the script wil be run in the root directory, but if someone installes a new web server, that may not be the case. Also, I assume that this isn't runnign as the root user - in which case, the "exec > out.log 2>&1" may not be able to write to the current directory.

You said earlier that you'd managed to echo the parameters when calling this from php - could you do that same again, and this time output the value of the pwd command?
0
 

Author Comment

by:dloszewski
Comment Utility
Ok, so I decided to move everything to one directory so there are no issues going between directories, unfortunately I'm still coming up with the same issue.  I also decided to forgo passing the parameters from PHP and instead just call the script so it would rule out any issues passing the parameters.  So now I have:

grep 4683 sslmftp1_messages_20131202 > ftp_search.14-02-10

Open in new window


$ pwd
/sysadm/shared/www/NCC

drwxr-xr-x  2 dloszews    126     4096 Feb 17 08:51 NCC

$ ls -l
total 63412
-rw-r--r-- 1 dloszews 126     1054 Feb 10 15:01 ftp_index.php
-rw-r--r-- 1 dloszews 126      960 Feb 16 14:44 ftp_search.sh
-rw-r--r-- 1 dloszews 126 64667581 Feb 10 15:00 sslmftp1_messages_20131202

Perhaps I'm looking at this the wrong way.  Instead of sending stdout from the sh file is it possible ot send the stdout to the browser from the PHP file so I can see whatever error this is creating? I'm assuming it has to do with some kind of permission issue with the grep, although I also tried running these from my home directory and I still had the same issue.  If I take that grep command in the script and run it from the command line it works fine.  If I put 'echo before' and 'echo after' before and after the grep command in the shell script then I get 'before after' in my browswer so I know it is calling the shell and making it past the grep command.
0
 

Author Comment

by:dloszewski
Comment Utility
I have also tried to change the permissions of the files to make them wide open:

$ ls -l
total 63412
-rwxrwxrwx 1 dloszews 126     1054 Feb 10 15:01 ftp_index.php
-rwxrwxrwx 1 dloszews 126      960 Feb 16 14:44 ftp_search.sh
-rwxrwxrwx 1 dloszews 126 64667581 Feb 10 15:00 sslmftp1_messages_20131202
0
 

Author Comment

by:dloszewski
Comment Utility
I'm running this on a different system where I have total control now and this still doesn't work
0
 

Author Comment

by:dloszewski
Comment Utility
so I finally got into the httpd log and I'm getting the following on the grep:

$ cd httpd
ksh: cd: /var/log/httpd - Permission denied
0
 
LVL 19

Expert Comment

by:simon3270
Comment Utility
I can't see any "cd" commands in your script - not sure what's going on.  Does a normal user have access to the log files you are searching?  In particular, does the apache user have access?  Do they have full access along the entire path to those files?  (it's no use having 777 permissions on the final directory if they are 700 earlier in the path and they don't own that directory).

I see you are still running "grep" on a bare file name, and writing to a bare file name.  Can you not add full paths to the filenames used?
0
 

Author Comment

by:dloszewski
Comment Utility
decided to go a different route using awk since I could still not figure this out.  Thank you all for your help.
0

Featured Post

6 Surprising Benefits of Threat Intelligence

All sorts of threat intelligence is available on the web. Intelligence you can learn from, and use to anticipate and prepare for future attacks.

Join & Write a Comment

Batch, VBS, and scripts in general are incredibly useful for repetitive tasks.  Some tasks can take a while to complete and it can be annoying to check back only to discover that your script finished 5 minutes ago.  Some scripts may complete nearly …
Utilizing an array to gracefully append to a list of EmailAddresses
Learn the basics of strings in Python: declaration, operations, indices, and slicing. Strings are declared with quotations; for example: s = "string": Strings are immutable.: Strings may be concatenated or multiplied using the addition and multiplic…
This tutorial will teach you the core code needed to finalize the addition of a watermark to your image. The viewer will use a small PHP class to learn and create a watermark.

762 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

11 Experts available now in Live!

Get 1:1 Help Now