Solved

Unix Script to read files and manipulate the values

Posted on 2009-04-02
9
345 Views
Last Modified: 2013-12-26
I have three files:-
a) yest-unr = list of items saved on yesterday
b) tod-unr = list of items saved today
c) yest-num = list of number of days for items listed on file `yest-unr` correspondignly

I want to write a scipt to read an item line by line for file `tod-unr` and search it on `yest-unr` list
and if it finds there... it should increment the corresponding number to it on `yest-num` file.
and finnaly the tod-unr file will replace the yest-unr file and the yest-unr should be deleted.

The aim of the task is to count the number of days an item occured on `tod-unr` file...
below is the code i tried but running with error... on my script.... c = count read from yest-num file.., nc= c+1 for the incremented, tod-unr = list of items for today, yest-unr= list of items from yesterday, yest-num= file contains c's of yest-unr
#!/bin/sh

c=$0

nc=$0

for i in `(cat tod-unr)`

do

    c= expr $c+1

    for j in `(cat yest-unr)`

        do

          if [$i == $j]; then

              yc=echo `sed -n '$c p' yest-num`

              typeset -i $yc

              nc= expr $yc+1

              echo $nc >> yest-num-new

          else

              nc=$1

              echo $nc >> yest-num-new

          fi

         done

done

mv tod-unr yest-unr

mv yest-num-new yest-num

Open in new window

0
Comment
Question by:mystockid
  • 6
  • 3
9 Comments
 
LVL 29

Expert Comment

by:MikeOM_DBA
Comment Utility


Why do you keep the count separate from the items?
Maybe this could be a better design:

awk 'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}

{$1 !~ "#"

  n=$2; for (f in t) if (f == $1) {n+=1; t[f]=1}

  print $1, n;}

  END {for (f in t) if (t[f] == 0) print f,1;}

' yest_unr|sort >yest_new_unr

echo "### Last updated: `date`" >yest_unr

cat yest_new_unr >>yest_unr

rm yest_new_unr

Open in new window

0
 

Author Comment

by:mystockid
Comment Utility
I tried your code and still running with syntax erro.

bash-2.05# ./ccm-sc-v3
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 2
awk: illegal statement near line 2
./ccm-sc-v3: yest-unr: execute permission denied
#!/bin/sh

awk 'BEGIN {while ((getline line < "tod-unr") > 0) t [line]=0;}

{$1 !~ "#"

  n=$2; for (f in t) if (f == $1) {n+=1; t[f]=1}

  print $1, n;}

  END {for (f in t) if (t[f] == 0) print f,1;}'

yest-unr | sort > yest-new-unr

echo "### Last Updated: `date`" > yest-unr

cat yest-new-unr >> yest-unr

rm yest-new-unr
 
 
 

bash-2.05# ./ccm-sc-v3

awk: syntax error near line 1

awk: illegal statement near line 1

awk: syntax error near line 2

awk: illegal statement near line 2

./ccm-sc-v3: yest-unr: execute permission denied

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
Comment Utility
What os are you using?
Use the "bash" shell (not sh), works for me:
 
 
 
 

#!/bin/bash

##

cat - <<EOF >yest_unr

aaaa 1

tttt 1

dddd 1

ssss 1

EOF

cat - <<EOF >tod_unr

tttt

dddd

uuuu

EOF
 

echo "+-----------------------------"

cat yest_unr|grep -v '#'|\

awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}

{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1;}

 print $1, n;}

 END {for (f in t) if (t[f] == 0) print f,1;}

' |sort >yest_new_unr

echo "### Last updated: `date`" >yest_unr

cat yest_new_unr >>yest_unr

cat yest_unr

rm yest_new_unr

##

## Execution:

$ ./o0

+-----------------------------

### Last updated: Fri Apr  3 12:25:25 EDT 2009

aaaa 1

dddd 2

ssss 1

tttt 2

uuuu 1

Open in new window

0
 

Author Comment

by:mystockid
Comment Utility
it is solaris OS 5.9 Generic_118558-14 sun4u sparc SUNW,Ultra-60
...

thanks for your time the expected output is:-
tttt 2
ddd 2
uuuu 1

0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 29

Expert Comment

by:MikeOM_DBA
Comment Utility

If you only need the count of the "todays" files, what happens to the previous count?

Example:

## tod_unr
tttt
dddd
uuuu

## yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1

If you discard "aaaa 1" and "ssss 1" you loose the count and if next day you receive these items the count would begin at 1 (one) again.

The modified script to do this would be:

#!/bin/bash

##

cat - <<EOF >yest_unr

aaaa 1

tttt 1

dddd 1

ssss 1

EOF

cat - <<EOF >tod_unr

tttt

dddd

uuuu

EOF
 

echo "+-----------------------------"

cat yest_unr|grep -v '#'|\

awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}

{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}

 END {for (f in t) if (t[f] == 0) print f,1;}

' |sort >yest_new_unr

echo "### Last updated: `date`" >yest_unr

cat yest_new_unr >>yest_unr

cat yest_unr

rm yest_new_unr

exit

##

## Execution:

$ ./o0

+-----------------------------

### Last updated: Sun Apr  5 08:14:11 EDT 2009

dddd 2

tttt 2

uuuu 1

Open in new window

0
 

Author Comment

by:mystockid
Comment Utility

I still running with syntax error. I tried to filter the error but couldn't. Here is the output.

Thanks,
Alem
bash-2.05# more tod_unr

tttt

dddd

uuuu

bash-2.05# more yest_unr

aaaa 1

tttt 1

dddd 1

ssss 1

bash-2.05# more ccm-sc-v5

#!/bin/bash

echo "+-----------------------------"

cat yest_unr | grep -v '#'|\

awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}

{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}

 END {for (f in t) if (t[f] == 0) print f,1;}

' | sort >yest_new_unr

echo "### Last updated: `date`" >yest_unr

cat yest_new_unr >>yest_unr

cat yest_unr

rm yest_new_unr

exit

##

## Execution:

bash-2.05# ./ccm-sc-v5

+-----------------------------

awk: syntax error near line 1

awk: illegal statement near line 1

### Last updated: Tue Apr  7 09:06:08 EDT 2009

bash-2.05#

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
Comment Utility

I copied and pasted your code, tested with bash 3.2 and it works:

-bash-3.2$ cat tod_unr

tttt

dddd

uuuu

-bash-3.2$ cat yest_unr

aaaa 1

tttt 1

dddd 1

ssss 1

-bash-3.2$ cat ccm-sc-v5

#!/bin/bash

echo "+-----------------------------"

cat yest_unr | grep -v '#'|\

awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}

{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}

 END {for (f in t) if (t[f] == 0) print f,1;}

' | sort >yest_new_unr

echo "### Last updated: Tue Apr  7 13:57:04 EDT 2009" >yest_unr

cat yest_new_unr >>yest_unr

cat yest_unr

rm yest_new_unr

exit

##

## Execution:

-bash-3.2$ ./ccm-sc-v5

+-----------------------------

### Last updated: Tue Apr  7 13:57:04 EDT 2009

dddd 2

tttt 2

uuuu 1

-bash-3.2$ 

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
Comment Utility

PS: You could maybe try gawk or nawk instead?

0
 
LVL 29

Accepted Solution

by:
MikeOM_DBA earned 500 total points
Comment Utility
-- OR -- in ksh without awk:
 

#!/bin/ksh

while read fn

do

  line=(`grep $fn yest_unr`)

  if [ ! -z ${line[0]} ]; then

    echo "${line[0]} `expr ${line[1]} + 1`"

  else

    echo "$fn 1"

  fi

done < tod_unr|sort >tod_new_unr

mv tod_new_unr yest_unr

Open in new window

0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Introduction: Database storage, where is the exe actually on the disc? Playing a game selected randomly (how to generate random numbers).  Error trapping with try..catch to help the code run even if something goes wrong. Continuing from the seve…
Recently, an awarded photographer, Selina De Maeyer (http://www.selinademaeyer.com/), completed a photo shoot of a beautiful event (http://www.sintjacobantwerpen.be/verslag-en-fotoreportage-van-de-sacramentsprocessie-door-antwerpen#thumbnails) in An…
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now