Need shell script to filter the data in a file based on certain condition.

non awk solution

#!/bin/sh
field=$1
value=$2

cut -f$field /some/file | grep -c $value

Hi Tintin,
you solution will give the number of records which satisfies the condition.
But it won't give me those records. I want the records also.

#!/bin/sh
field=$1
value=$2

count=`cut -f$field /some/file | grep -c $value`
line=`cut -f$field /some/file | grep  $value`
echo "$line - $count"

Hi Tintin,

My file has a delimeter like pipe or comma to seperate fields. You are not taking care of that. Because of that its not cutting the exact field from the file.

For example:
My file is test.txt whith the data as
a|bcd|cde|123  
c|bcd|abc|    
||bcd|swd      

cut -f2 "test.txt" | grep  "bcd" command is giving output as
a|bcd|cde|123
c|bcd|abc|    
||bcd|swd    

It should not give the 3rd record in which the "bcd" is occuring in 3rd column.

Whoops, forgot the -d flag to cut

cut -f2 -d'|' ......

But stil ther ia some problem with your commands.
It won't give the complete record.

try to execute the your command
line=`cut -d'|' -f$field /some/file | grep  $value`
you are cutting a particular field from the file and searching for the value.
The input to the grep is the column only. How can you get the complete record?

I have tried with the folloing command.

cut -f2 -d "|" test.txt | sort | uniq -c

The output of this command is
1
2 bcd

This will give the number of records and the field value. For number of records, this is fine.
But the problem is it wont displays those records.

you need a trick like below  ( add magic field at start as extra field, grep , then cut it out )



$ cat x1.txt
A|ABC|YZQ|0093
B|CDE|233|0987
C|ABC|656|0001
C|AXC|344|2345
C|AFC|ABC|8982
C|ABC|551|0002

$ cat x2.ksh
#!/bin/ksh

P1=$1

if [ "${2}HH" == "HH" ]; then
  P2=1
else
  P2=$2
fi

cut -f$P2 -d"|" x1.txt  | sed -e "s/$/|/" > x1_extra.txt
paste  x1_extra.txt x1.txt | sed -e "s/ //" >  x1_total.txt
grep "^${P1}|" x1_total.txt  | cut -f2- -d"|"
echo Count of ${P1} at field ${P2} is `grep "^${P1}|" x1_total.txt|wc -l`


$ x2.ksh ABC 2
A|ABC|YZQ|0093
C|ABC|656|0001
C|ABC|551|0002
Count of ABC at field 2 is 3

$  x2.ksh AXC 2
C|AXC|344|2345
Count of AXC at field 2 is 1

$ x2.ksh ABC 3
C|AFC|ABC|8982
Count of ABC at field 3 is 1

I see what you want now (borrowing from ahoffmans's suggestion).

#!/bin/sh
FILE=/some/file.txt

field=$1
value=$2

awk -F'|' '($"'$field'"=="'$value'"){print}' $FILE
count=`cut -f$field $FILE | grep -c $value`
echo "Total matches for $value: $count"

Hi Tintin,
I am getting desired result with your code. One more concern is can we parameterise the delimeter in the awk also? ( we can parametrise the delimeter in the cut command)
Because my files are some times comma delimeted and some times pipe delimeted.

Hi HamdyHassan,
I have not check your code. I will check and let you know whether it is working fine or not.

regards,
Sharath

-----  some times comma delimeted and some times pipe delimeted
Yes you can do that

instead of -d"|"  
you use  -d$MyDel

make sure you check $3 as I did for $2, may be you can use default "|" in case $3 is empty
if [ "${2}HH" == "HH" ]; then
  P2=1
else
  P2=$2
fi

also when you can at command prompt, you need to put  " "  around |  otherwise shell consider it as pipline
$ x2.ksh ABC 2 "|"

hmm, you can write millions of lines of error prone code in any language you can imagine ...
What is the problem with my simple, understandable awk suggestion?

Hi Tintin,
your script is working fine. Can you elobarate it to search for records with null string in certain column. till now we displayed records with "ABC" at field num. 2, can't we display records with " " ( null value) at field num.2?

Be careful using the word null, as it often mistakingly gets used to mean empty or blank value.  In your case, do you mean blank value?

Thanks all of you for overhelming response.