Help troubleshoot a Shell script

nohup ./globscan.sh 2> /tmp/glo.err &
  When I run the attached script, it gave an error message below:

# more /tmp/glo*err*
Sending output to nohup.out
/opt/uvscan/globscan.sh: [[: not found
/opt/uvscan/globscan.sh: syntax error at line 4: `(' unexpected
# more nohup.out
/opt/uvscan/globscan.sh: [[: not found
/opt/uvscan/globscan.sh: syntax error at line 4: `(' unexpected
./globscan.sh: [[: not found
./globscan.sh: syntax error at line 4: `(' unexpected


If I run it as follows, don't get any error:
  ksh -v /opt/globscan.sh

The permission of the script file is 755


What did I miss?  All lines end with  <LF>  as I checked using  "cat -ev ..."   or vi's  set line on
sunhuxAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sunhuxAuthor Commented:
Attached the script which I missed earlier
Avscan-errScript.txt
sunhuxAuthor Commented:
Typo:
If I run it as follows, don't get any error:
   ksh -v /opt/avscan/globscan.sh
      OR
   /opt/avscan/globscan.sh

Am using root in all cases
sunhuxAuthor Commented:
I tot it's due to the "For"  which I've just changed to "for" in line 3 but still giving the same error
10 Tips to Protect Your Business from Ransomware

Did you know that ransomware is the most widespread, destructive malware in the world today? It accounts for 39% of all security breaches, with ransomware gangsters projected to make $11.5B in profits from online extortion by 2019.

simon3270Commented:
"[[" is not available in all shells (Bourne shell, for example), so it would be best to specify the shell in your script.
Insert the first line
    #!/usr/bin/ksh
Replace /usr/bin/ksh with the path to your "ksh" program.

Alternatively, change the "[[ ]]" to "[ ]" - the "-e" test you use works in both.
simon3270Commented:
Also, I'd be careful of the "for scanfile in `find ....`" line - it may work on a test box with a few files, but on production boxes with many more files it may blow up.  I'd probably rewrite it as follows.  The grep -v line is hard to read, so put the values in a simple text file (and make some of the tests more explicit - for example, "grep -v .ctl" will remove any file with "ctl" within its name - the dot means "any character").  I've also remove the "cd" from within the loop (you are already in  that directory!) and modified the stderr redirection so that stdout and stderr go to the same file.  Because the input to the loop is taken from a pipe, variable settings within the loop will not be visible outside it, so I replaced the "var" setting with a log file containing one line per processed file, then reported the number of lines in the file at the end.  In the vscan6 line, I've also put double quotes round $scanfile, in case the file name contains spaces.

So, from the "var=0" line to the "Total files have been checked" line:
> fc.log
cat > grepv.ctl << EOF
\.sock
^/cdrom
^/boot
^/platf
^/proc
mnttab
^/sys
^/dev
^/net
\.dbf
\.arc
\.lock
\.ctl
\.rdo
[Oo]racle
[Mm]ysql
sulog
syslog
messages
EOF
find /* | grep -v -f grepv.ctl | while read scanfile; do
    echo $scanfile >> fc.log
    nice ./vscan6 -p=/opt/avscan "$scanfile" -NC -l=/opt/uvscan /tmp/scan |grep -v "libvsapi.so" |grep -v "+-------" |grep -v " | " |grep -v "been check" >> /opt/avscan/update.log 2>&1
done
echo "$(wc -l < fc.log) Total files have been checked." >> /opt/avscan/update.log

Open in new window

Edit: Missed the "<< EOF" bit.

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sunhuxAuthor Commented:
Thanks Simon;  I've seen cases where the following housekeeping command fails with a
message  "argument list too long" :
     rm `find /var/*.log -print`

I'll test it out tomorrow.  There's a possibility one of those 500 Solaris/Unix servers has
millions of files in it
sunhuxAuthor Commented:
Btw, does the latest script you've amended with "fc ..." work
for both Bash & Korn shells?   There's a chance we may use
Bourne (ie /bin/sh) but unlikely.  Nobody uses C-shell here
sunhuxAuthor Commented:
So I won't need the  "var=0" anymore or if it's still needed, we'll need to increment it for every file scanned:
      ((var=var+1))
simon3270Commented:
If the script has the #! line as the first line in the file, it will use that specified shell whatever the caller is using (even if they use csh).. That said, the "[ ]" version should work with most non-c  shells, as will.my bit of code.

You don't need the $var variable at all. The problem here is that because the while loop is in a pipeline (it takes its input from the output of another command), any variables you change within the loop do not affect the value outside the loop. So, you'd set var to 0 at the beginning and increment it inside the loop, but after the loop, the value would still be 0. I worked round this by adding the processed file names, one name per line, to a file. After the loop, the number of lines in the file gives the number of files processed.
sunhuxAuthor Commented:
Think there's a little syntax error in the last script:

-bash-3.2# nohup nice ./globscan.sh 2> /tmp/glo.err
-bash-3.2#
-bash-3.2# more /tmp/glo.err
Sending output to nohup.out
grep: illegal option -- f
Usage: grep -hblcnsviw pattern file . . .
sunhuxAuthor Commented:
Think it lies on this line:
find /* -type f -size -52428800c | grep -v -f grepv.ctl | while read scanfile; do
sunhuxAuthor Commented:
or was it intended to be
   grep -v -F grepv.ctl
sunhuxAuthor Commented:
or it was meant to be:
 grep -v -F=grepv.ctl
sunhuxAuthor Commented:
Sorry the last post has a typo ie should read
  or it was meant to be:
 grep -v -f=grepv.ctl
sunhuxAuthor Commented:
The following gave the same error:
 grep -v -f=grepv.ctl
sunhuxAuthor Commented:
http://unix.stackexchange.com/questions/83260/reading-grep-patterns-from-a-file

Also tried  "grep -F -f grepv.ctl"  & it gave the same error

with just "grep -f grepv.ctl"  (ie with  -v  removed), still gave same error
sunhuxAuthor Commented:
http://stackoverflow.com/questions/4024637/passing-a-file-with-multiple-patterns-to-grep
Not quite sure how I can implement the above with the syntax below:
  tr -d "\r" <match.txt | grep -F -f - text.txt
sunhuxAuthor Commented:
so the "-v" is supposed to be there, right?  It's to grep lines "not" containing the pattern in grepv.ctl

So I've amended grep to fgrep ie:
find /* -type f -size -52428800c | fgrep -v -f grepv.ctl | while read scanfile; do

Is the above correct?  So far, it did not complaint
sunhuxAuthor Commented:
With the above syntax, I just noticed it scanned Oracle... folder which I don't
want it to (as supposed to exclude "[Oo]racle"  in grepv.ctl)

 root 23079 20392   1 20:02:02 pts/2       0:01 ./vscan -p=/opt/uvscan /Oracle11g/doc/owb.112/e17130/img_text/exporting_

What did I miss?
sunhuxAuthor Commented:
Just spotted it scans "/boot" which it was not supposed to:
 ... pts/2     0:01  ./vscan -p=/opt/uvscan /boot/grub/menu.lst -NC -l=/opt/uvscan /tmp/scan
sunhuxAuthor Commented:
Looks like, we'll need to remove  those  "beginning", "Containing", "ending" characters ie
^
\
[ ]
simon3270Commented:
Sorry, been out all day and have only just come back to your many attempts!

Which OS are you on?  The "-f" flag is available in GNU grep.

The "tr" fix you found is only valid if you are using a Windows machine either to run the code, or to generate the shell script (and if you are having carriage-return problems, the script would have failed earlier than this).

The "fgrep" version won't, as you say, like the pattern matching strings - it requires only the exact text that you want to match.  Have you tried egrep instead?  That should be able to parse the pattern strings successfully.
sunhuxAuthor Commented:
I'm on Solaris 10 x86 (64bit).    By removing the
^  \  []    characters & using "fgrep", got it going.

I'm not at liberty to install any packages/software
on the Solaris x86 VMs as they belong to our
tenants.  

Found that the scan takes rather long & I'm now exploring
how I can optimize the coding : thought   "ls ..." command is
faster than "find ..." command, so is the code below faster:

Change
find /* -type f -size -52428800c | grep -v -f grepv.ctl | while read scanfile; do

cd /
ls -aR -type f -size -52428800c | grep -v -f grepv.ctl | while read scanfile; do

(am assuming ls  has the -type & -size options).  Back home now, so can't
test out the above code
sunhuxAuthor Commented:
>tried egrep instead?
Haven't got a chance to try egrep, what's the syntax like?
simon3270Commented:
For Solaris use /usr/xpg4/bin/grep - that supports the "-f" flag.

Sadly, "ls" doesn't support type or size - it just list contents and leaves analysis and filtering to other scripts.

The time taken for the scan is probably not because of the "find" which should be very fast) but the actual vscan6 call.
simon3270Commented:
egrep syntax is basically the same as grep and fgrep - it just supports extended regular expressions such as "abc|xyz" to look for "abc" or "xyz".
sunhuxAuthor Commented:
I just realized that this "find ..." coding for AV scanner to do the scan is not efficient because:
for each files to be scanned, the scanner will open the signature pattern file, library etc.

If we feed the folders' names to this scanner don't need to suffer this long scan timings
simon3270Commented:
Does vscan6 work on files or directories? I assumed files, because of the way you wrote the original script.

If so, you could use xargs to supply multiple file names:

find /* -type f | /usr/bin/xpg4/bin/grep -v -f grepv.ctl | xargs. /vscan6.. .. .. ..

Add the -size and vscan6 parameters as necessary (I'm on my phone at the moment, so cut and paste is hard!)

This will work as long as you can have a series of file names at the end of vscan6 command (I.e. you don't use $scanfile, but instead just have names of files to scan at the end of the command). If not, we'll have to rethink, snd I'll ned to be at my computer!
sunhuxAuthor Commented:
This vscan6 could take in files as well as directories:  unfortunately, it's much more
efficient if it take in a folder rather than a list of files under that folder because if
it take a list of files, it will read the signature pattern file plus a library file the number
of time as the number of files.

So if /usr has 9000 files, vscan6 will read the pattern+library files plus output a banner
to a logfile 9000 times ie each time before it scan a file, it will read the pattern & library
& after scanning that file, it will output a banner.

if it's given a folder, it only reads the pattern+library files once tho there's 9000 files
in usr.


As this is now a radically new requirement, will most appreciate if you can look at
it at the new question that I've raised:

http://www.experts-exchange.com/questions/28703371/Need-Shell-script-to-parse-thru-suitable-filesystems-files-folders-for-AV-scanning.html
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.