?
Solved

Shell - Command to filter and count lines

Posted on 2014-02-18
6
Medium Priority
?
322 Views
Last Modified: 2014-02-19
Hi,

I have a list of files:

.
TrafficErrorEvents20140128.9583
TrafficErrorEvents20140212.0604
.
.

that each have several of these lines:

20140218171207#20140218171207#6698072736#MDMD3#mdm1a#1#6698072737#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177388039#IMEI=70306000300926#MSISDN=59177388039#TerminalId=70306000300926#
.
.
20140218171207#20140218171207#6698072742#MDMD3#mdm1a#1#6698072743#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177250681#IMEI=01384100820529#MSISDN=59177250681#TerminalId=01384100820529#


So, what I need is:

A command that would scan these files (something with a wildcard TrafficErrorEvents201401* for example)

And outputs a count of the IMEI=70306000300926 that are equal....

So for example of the lines above I would have:
70306000300926  1
01384100820529  1

Is this something that can be done?

Tks,
Joao
0
Comment
Question by:joaotelles
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
6 Comments
 
LVL 68

Expert Comment

by:woolmilkporc
ID: 39868727
If the value in question is always preceeded by "IMEI=" and always terminated with "#MSISDN"
you can try this:

awk -F"IMEI=|#MSISDN" '/IMEI=/ {A[$2]+=1} END {for(n in A) print n,A[n]}' TrafficErrorEvents201401*
0
 
LVL 84

Expert Comment

by:ozo
ID: 39868734
perl -lne '++$c{$_} for /IMEI=(\d+)/g;END{print "$_\t$c{$_}" for keys %c}' TrafficErrorEvents201401*
0
 

Author Comment

by:joaotelles
ID: 39868870
Sorry I forgot to mention, in these files I have a bunch of other diferent lines.. so I get the lines I need I have to filter by MDMD3..

Can you guys add this?

For example on my two lines, I have MDMD3 in both of them.... (#MDMD3#)

20140218171207#20140218171207#6698072736#MDMD3#mdm1a#1#6698072737#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177388039#IMEI=70306000300926#MSISDN=59177388039#TerminalId=70306000300926#
.
.
20140218171207#20140218171207#6698072742#MDMD3#mdm1a#1#6698072743#3#ConfigurationName=CP_WIBTSD#ConfigurationVersion=5.2#DestinationAddress=59177250681#IMEI=01384100820529#MSISDN=59177250681#TerminalId=01384100820529#
0
Get real performance insights from real users

Key features:
- Total Pages Views and Load times
- Top Pages Viewed and Load Times
- Real Time Site Page Build Performance
- Users’ Browser and Platform Performance
- Geographic User Breakdown
- And more

 
LVL 84

Accepted Solution

by:
ozo earned 1400 total points
ID: 39868901
perl -lne '/#MDMD3#/&&/IMEI=(\d+)/&&++$c{$1};END{print"$_\t$c{$_}"for keys%c}' TrafficErrorEvents201401*

or, if #MDMD3# always precedes IMEI=
perl -lne '/#MDMD3#.*IMEI=(\d+)/&&++$c{$1};END{print"$_\t$c{$_}"for keys%c}' TrafficErrorEvents201401*
0
 
LVL 68

Assisted Solution

by:woolmilkporc
woolmilkporc earned 600 total points
ID: 39868924
So there are lines containing "IMEI=" which you want to ignore if they don't also contain "#MDMD3#"?

awk -F"IMEI=|#MSISDN" '/IMEI=/&&/#MDMD3#/ {A[$2]+=1} END {for(n in A) print n,A[n]}' TrafficErrorEvents201401*
0
 

Author Closing Comment

by:joaotelles
ID: 39871401
Tks. Both worked just fine.
0

Featured Post

Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
In a previous video, we went over how to export a DynamoDB table into Amazon S3.  In this video, we show how to load the export from S3 into a DynamoDB table.
Suggested Courses
Course of the Month13 days, 19 hours left to enroll

800 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question