Solved

Shell - Command to filter and count lines

Posted on 2014-02-18
6
310 Views
Last Modified: 2014-02-19
Hi,

I have a list of files:

.
TrafficErrorEvents20140128.9583
TrafficErrorEvents20140212.0604
.
.

that each have several of these lines:

20140218171207#20140218171207#6698072736#MDMD3#mdm1a#1#6698072737#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177388039#IMEI=70306000300926#MSISDN=59177388039#TerminalId=70306000300926#
.
.
20140218171207#20140218171207#6698072742#MDMD3#mdm1a#1#6698072743#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177250681#IMEI=01384100820529#MSISDN=59177250681#TerminalId=01384100820529#


So, what I need is:

A command that would scan these files (something with a wildcard TrafficErrorEvents201401* for example)

And outputs a count of the IMEI=70306000300926 that are equal....

So for example of the lines above I would have:
70306000300926  1
01384100820529  1

Is this something that can be done?

Tks,
Joao
0
Comment
Question by:joaotelles
  • 2
  • 2
  • 2
6 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39868727
If the value in question is always preceeded by "IMEI=" and always terminated with "#MSISDN"
you can try this:

awk -F"IMEI=|#MSISDN" '/IMEI=/ {A[$2]+=1} END {for(n in A) print n,A[n]}' TrafficErrorEvents201401*
0
 
LVL 84

Expert Comment

by:ozo
ID: 39868734
perl -lne '++$c{$_} for /IMEI=(\d+)/g;END{print "$_\t$c{$_}" for keys %c}' TrafficErrorEvents201401*
0
 

Author Comment

by:joaotelles
ID: 39868870
Sorry I forgot to mention, in these files I have a bunch of other diferent lines.. so I get the lines I need I have to filter by MDMD3..

Can you guys add this?

For example on my two lines, I have MDMD3 in both of them.... (#MDMD3#)

20140218171207#20140218171207#6698072736#MDMD3#mdm1a#1#6698072737#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177388039#IMEI=70306000300926#MSISDN=59177388039#TerminalId=70306000300926#
.
.
20140218171207#20140218171207#6698072742#MDMD3#mdm1a#1#6698072743#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177250681#IMEI=01384100820529#MSISDN=59177250681#TerminalId=01384100820529#
0
Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

 
LVL 84

Accepted Solution

by:
ozo earned 350 total points
ID: 39868901
perl -lne '/#MDMD3#/&&/IMEI=(\d+)/&&++$c{$1};END{print"$_\t$c{$_}"for keys%c}' TrafficErrorEvents201401*

or, if #MDMD3# always precedes IMEI=
perl -lne '/#MDMD3#.*IMEI=(\d+)/&&++$c{$1};END{print"$_\t$c{$_}"for keys%c}' TrafficErrorEvents201401*
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 150 total points
ID: 39868924
So there are lines containing "IMEI=" which you want to ignore if they don't also contain "#MDMD3#"?

awk -F"IMEI=|#MSISDN" '/IMEI=/&&/#MDMD3#/ {A[$2]+=1} END {for(n in A) print n,A[n]}' TrafficErrorEvents201401*
0
 

Author Closing Comment

by:joaotelles
ID: 39871401
Tks. Both worked just fine.
0

Featured Post

Is Your Active Directory as Secure as You Think?

More than 75% of all records are compromised because of the loss or theft of a privileged credential. Experts have been exploring Active Directory infrastructure to identify key threats and establish best practices for keeping data safe. Attend this month’s webinar to learn more.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
Over the years I've spent many an hour playing on hardened, DMZ'd servers, with only a sub-set of the usual GNU toy's to keep me company; frequently I've needed to save and send log or data extracts from these server back to my PC, or to others, and…
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.

911 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

21 Experts available now in Live!

Get 1:1 Help Now