Solved

Unix Script to read files and manipulate the values

Posted on 2009-04-02
9
349 Views
Last Modified: 2013-12-26
I have three files:-
a) yest-unr = list of items saved on yesterday
b) tod-unr = list of items saved today
c) yest-num = list of number of days for items listed on file `yest-unr` correspondignly

I want to write a scipt to read an item line by line for file `tod-unr` and search it on `yest-unr` list
and if it finds there... it should increment the corresponding number to it on `yest-num` file.
and finnaly the tod-unr file will replace the yest-unr file and the yest-unr should be deleted.

The aim of the task is to count the number of days an item occured on `tod-unr` file...
below is the code i tried but running with error... on my script.... c = count read from yest-num file.., nc= c+1 for the incremented, tod-unr = list of items for today, yest-unr= list of items from yesterday, yest-num= file contains c's of yest-unr
#!/bin/sh
c=$0
nc=$0
for i in `(cat tod-unr)`
do
    c= expr $c+1
    for j in `(cat yest-unr)`
        do
          if [$i == $j]; then
              yc=echo `sed -n '$c p' yest-num`
              typeset -i $yc
              nc= expr $yc+1
              echo $nc >> yest-num-new
          else
              nc=$1
              echo $nc >> yest-num-new
          fi
         done
done
mv tod-unr yest-unr
mv yest-num-new yest-num

Open in new window

0
Comment
Question by:mystockid
  • 6
  • 3
9 Comments
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 24052211


Why do you keep the count separate from the items?
Maybe this could be a better design:

awk 'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}
{$1 !~ "#"
  n=$2; for (f in t) if (f == $1) {n+=1; t[f]=1}
  print $1, n;}
  END {for (f in t) if (t[f] == 0) print f,1;}
' yest_unr|sort >yest_new_unr
echo "### Last updated: `date`" >yest_unr
cat yest_new_unr >>yest_unr
rm yest_new_unr

Open in new window

0
 

Author Comment

by:mystockid
ID: 24053574
I tried your code and still running with syntax erro.

bash-2.05# ./ccm-sc-v3
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 2
awk: illegal statement near line 2
./ccm-sc-v3: yest-unr: execute permission denied
#!/bin/sh
awk 'BEGIN {while ((getline line < "tod-unr") > 0) t [line]=0;}
{$1 !~ "#"
  n=$2; for (f in t) if (f == $1) {n+=1; t[f]=1}
  print $1, n;}
  END {for (f in t) if (t[f] == 0) print f,1;}'
yest-unr | sort > yest-new-unr
echo "### Last Updated: `date`" > yest-unr
cat yest-new-unr >> yest-unr
rm yest-new-unr
 
 
 
bash-2.05# ./ccm-sc-v3
awk: syntax error near line 1
awk: illegal statement near line 1
awk: syntax error near line 2
awk: illegal statement near line 2
./ccm-sc-v3: yest-unr: execute permission denied

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 24061710
What os are you using?
Use the "bash" shell (not sh), works for me:
 
 
 
 

#!/bin/bash
##
cat - <<EOF >yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1
EOF
cat - <<EOF >tod_unr
tttt
dddd
uuuu
EOF
 
echo "+-----------------------------"
cat yest_unr|grep -v '#'|\
awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}
{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1;}
 print $1, n;}
 END {for (f in t) if (t[f] == 0) print f,1;}
' |sort >yest_new_unr
echo "### Last updated: `date`" >yest_unr
cat yest_new_unr >>yest_unr
cat yest_unr
rm yest_new_unr
##
## Execution:
$ ./o0
+-----------------------------
### Last updated: Fri Apr  3 12:25:25 EDT 2009
aaaa 1
dddd 2
ssss 1
tttt 2
uuuu 1

Open in new window

0
Comprehensive Backup Solutions for Microsoft

Acronis protects the complete Microsoft technology stack: Windows Server, Windows PC, laptop and Surface data; Microsoft business applications; Microsoft Hyper-V; Azure VMs; Microsoft Windows Server 2016; Microsoft Exchange 2016 and SQL Server 2016.

 

Author Comment

by:mystockid
ID: 24063463
it is solaris OS 5.9 Generic_118558-14 sun4u sparc SUNW,Ultra-60
...

thanks for your time the expected output is:-
tttt 2
ddd 2
uuuu 1

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 24071235

If you only need the count of the "todays" files, what happens to the previous count?

Example:

## tod_unr
tttt
dddd
uuuu

## yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1

If you discard "aaaa 1" and "ssss 1" you loose the count and if next day you receive these items the count would begin at 1 (one) again.

The modified script to do this would be:

#!/bin/bash
##
cat - <<EOF >yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1
EOF
cat - <<EOF >tod_unr
tttt
dddd
uuuu
EOF
 
echo "+-----------------------------"
cat yest_unr|grep -v '#'|\
awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}
{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}
 END {for (f in t) if (t[f] == 0) print f,1;}
' |sort >yest_new_unr
echo "### Last updated: `date`" >yest_unr
cat yest_new_unr >>yest_unr
cat yest_unr
rm yest_new_unr
exit
##
## Execution:
$ ./o0
+-----------------------------
### Last updated: Sun Apr  5 08:14:11 EDT 2009
dddd 2
tttt 2
uuuu 1

Open in new window

0
 

Author Comment

by:mystockid
ID: 24086835

I still running with syntax error. I tried to filter the error but couldn't. Here is the output.

Thanks,
Alem
bash-2.05# more tod_unr
tttt
dddd
uuuu
bash-2.05# more yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1
bash-2.05# more ccm-sc-v5
#!/bin/bash
echo "+-----------------------------"
cat yest_unr | grep -v '#'|\
awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}
{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}
 END {for (f in t) if (t[f] == 0) print f,1;}
' | sort >yest_new_unr
echo "### Last updated: `date`" >yest_unr
cat yest_new_unr >>yest_unr
cat yest_unr
rm yest_new_unr
exit
##
## Execution:
bash-2.05# ./ccm-sc-v5
+-----------------------------
awk: syntax error near line 1
awk: illegal statement near line 1
### Last updated: Tue Apr  7 09:06:08 EDT 2009
bash-2.05#

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 24090130

I copied and pasted your code, tested with bash 3.2 and it works:

-bash-3.2$ cat tod_unr
tttt
dddd
uuuu
-bash-3.2$ cat yest_unr
aaaa 1
tttt 1
dddd 1
ssss 1
-bash-3.2$ cat ccm-sc-v5
#!/bin/bash
echo "+-----------------------------"
cat yest_unr | grep -v '#'|\
awk  'BEGIN {while ((getline line < "tod_unr") > 0) t[line]=0;}
{n=$2; for (f in t) if ($1 == f) {n+=1; t[f]=1; print $1, n;}}
 END {for (f in t) if (t[f] == 0) print f,1;}
' | sort >yest_new_unr
echo "### Last updated: Tue Apr  7 13:57:04 EDT 2009" >yest_unr
cat yest_new_unr >>yest_unr
cat yest_unr
rm yest_new_unr
exit
##
## Execution:
-bash-3.2$ ./ccm-sc-v5
+-----------------------------
### Last updated: Tue Apr  7 13:57:04 EDT 2009
dddd 2
tttt 2
uuuu 1
-bash-3.2$ 

Open in new window

0
 
LVL 29

Expert Comment

by:MikeOM_DBA
ID: 24090150

PS: You could maybe try gawk or nawk instead?

0
 
LVL 29

Accepted Solution

by:
MikeOM_DBA earned 500 total points
ID: 24090329
-- OR -- in ksh without awk:
 

#!/bin/ksh
while read fn
do
  line=(`grep $fn yest_unr`)
  if [ ! -z ${line[0]} ]; then
    echo "${line[0]} `expr ${line[1]} + 1`"
  else
    echo "$fn 1"
  fi
done < tod_unr|sort >tod_new_unr
mv tod_new_unr yest_unr

Open in new window

0

Featured Post

Enterprise Mobility and BYOD For Dummies

Like “For Dummies” books, you can read this in whatever order you choose and learn about mobility and BYOD; and how to put a competitive mobile infrastructure in place. Developed for SMBs and large enterprises alike, you will find helpful use cases, planning, and implementation.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Powershell Command to create a csv file using UTF-8 encoding 3 102
Sed question 2 85
Script to modify multiple files in Ubuntu / CentOS 2 54
IE 11 + long running scripts 3 52
Introduction: Dialogs (1) modal - maintaining the database. Continuing from the ninth article about sudoku.   You might have heard of modal and modeless dialogs.  Here with this Sudoku application will we use one of each type: a modal dialog …
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.

773 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question