shell script urgent help

I can't get handle on two issues with this script:

issue # 1: if file path contains "&" sign its adding "     commit;" when it inserts values in oracle
issue # 2: even when it finds the checksum value of file ... its inserting in oracle that file not exist ....

new.txt file content:

"/opt/app/file1.txt"
"/opt/app/file2 &, some text.txt"

Open in new window



Script:

#!/bin/bash
FILES='new.txt'
cat $FILES |
while read LINE
   do
echo $LINE
LINE=${LINE//\"}
if [ -e "$LINE" ]; then   # test if the file exists
     DATA=`sha256sum "$LINE"`
if [ $? -eq 0 ]; then   #test if the prior command run was successful
set ${DATA} ; #do not place this variable in quotes or you will always get a single value the only issue this will run into is if there is a space in the path.
checksum=$1
file_path=$LINE
echo $checksum
 #    path=`sha256sum "$LINE"|awk '{print $2}'`

         sqlplus -s 'DB Details' <<EOF
         insert into CHECKSUM (path,checksum_value) values ('$file_path', '$checksum');
         commit;
         EXIT;
EOF
else
     echo "The command sha256sum encountered an error with $LINE"
fi
else
     echo "The file $LINE does not exist"
     sqlplus -s 'DB details' <<EOF
     insert into CHECKSUM (path,checksum_value,status) values ('$file_path', '', 'File not found');
     commit;
     EXIT;
EOF     
fi
done

Open in new window

LVL 5
CalmSoulAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

Qlemo"Batchelor", Developer and EE Topic AdvisorCommented:
& is a substitution var prefix in Oracle Sql*Plus.
Add
SET DEFINE OFF

Open in new window

before the INSERT to prevent that.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
CalmSoulAuthor Commented:
Thanks Qlemo

Any idea on issue number 2?
0
egarciatCommented:
Between line 7 and 8 try this:

#LINE="/opt/app/file2 &, some text.txt"
OFS=$IFS; IFS='&'; LINE=($LINE); IFS=$OFS;
LINE=${LINE[0]}

Open in new window

0
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

ozoCommented:
Do you just want to ignore everything after & or space in $LINE ?
${LINE%%[& ]*}
0
CalmSoulAuthor Commented:
Issue # 1 is fixed, but issue # 2 is pending
issue # 2: even when it finds the checksum value of file ... its inserting in oracle that file not exist ....
0
ozoCommented:
Did you remove the  &, some text.txt" from $LINE?
0
CalmSoulAuthor Commented:
for issue # 1, This worked

SET DEFINE OFF
0
ozoCommented:
${LINE%%[& ]*} should take care of both issue # 1 and issue # 2
0
CalmSoulAuthor Commented:
change this

LINE=${LINE//\"}

Open in new window

to

LINE=${LINE%%[& ]*}

Open in new window



I am getting all file not founds now
0
ozoCommented:
You want both to take care of both " and &
0
CalmSoulAuthor Commented:
I only want to take care of "...

I think you are not getting the second issue ... even file is found or not found it inserting in database using this query

 insert into CHECKSUM (path,checksum_value,status) values ('$file_path', '', 'File not found');
     commit;

Open in new window

0
CalmSoulAuthor Commented:
The problem is with if structure....
0
ozoCommented:
Did you also remove the ^M that was giving you problems earlier?
I don't see that in your script.
0
CalmSoulAuthor Commented:
Yes ^M has been removed
I  used cat filename | col -b > newfilename
to remove that ...
0
ozoCommented:
Are there any extra characters after any of the
EOF
in your script?
0
CalmSoulAuthor Commented:
no there is no more special charaters ... I am thinking of re-doing this in perl ....

another issue I found that it skips lines as well... I had 10,000 lines and it only proceed 6,000
0
ozoCommented:
No echo at all on the skipped  lines?
Anything unusual about those lines, or the preceding or following lines ?
0
CalmSoulAuthor Commented:
Here is the script now...

#!/bin/bash
FILES='new.txt'
cat $FILES |
while read LINE
   do
echo $LINE
LINE=${LINE//\"}
#LINE=${LINE%%[& ]*}
if [ -e "$LINE" ]; then   # test if the file exists
     DATA=`sha256sum "$LINE"`
if [ $? -eq 0 ]; then   #test if the prior command run was successful
set ${DATA} ; #do not place this variable in quotes or you will always get a single value the only issue this will run into is if there is a space in the path.
checksum=$1
file_path=$LINE
echo $checksum
         sqlplus -s '' <<EOF
        SET DEFINE OFF
        insert into CHECKSUM (path,checksum_value) values ('$file_path', '$checksum');
         commit;
         EXIT;
EOF
elif  [ $? -ne 0 ]; then
     echo "The command sha256sum encountered an error with $LINE"

else
     echo "The file $LINE does not exist"
     sqlplus -s '' <<EOF
SET DEFINE OFF
insert into CHECKSUM (path,checksum_value,status) values ('$file_path', '', 'File not found');
     commit;
     EXIT;
EOF
fi
fi
done

Open in new window


I can see them in echo ... script will randomly skip 3 or 4 files and provide checksum of last line ... I did cat | more on the file there are no special charaters ...
0
CalmSoulAuthor Commented:
^M is back in my file ... how to remove it easily ?
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.