Shell - count of lines using part of it as filter

Hi,

I have several files with timestamp like this:

ig1a20130918.log
ig1a20130919.log
ig1a20130920.log
ig1a20130921.log
ig1a20130922.log
ig1a20130923.log
ig1a20130924.log
ig1a20130925.log

I need that the 7 days before files to be analized: so if I run today its from 24 to 18 days.

In the analize I need to:

1- Grep for this specific line:
2013-09-24 00:00:01.741 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59178421279 Host: 146.82.89.83
2013-09-24 00:00:01.900 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175365162 Host: 127.0.0.1
2013-09-24 00:05:56.646 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175679697 Host: aclwireless3.com

2- And do a count of the number of lines per different Host... So for example here I would have:
Host: 146.82.89.83 - 1
Host: 127.0.0.1 - 1
Host: aclwireless3.com - 1

Is this possible?

Can it be done for this line as well?  - same analizes..

2013-09-24 00:00:01.903 [ 8868] Statistic  WichaHandler         Web Server response received. MSISDN: 59175365162 Host: 127.0.0.1

Tks,
Joao


2-
joaotellesAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

ozoCommented:
Grep for which specific line?  It looks like you have examples of 4 distinct lines.
0
joaotellesAuthor Commented:
Doing a grep for this for example:

Statistic  WichaHandler         Sending Web Server request. MSISDN: 59178421279 Host:

And once you have these lines as output:
013-09-24 00:00:01.741 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59178421279 Host: 146.82.89.83
2013-09-24 00:00:01.900 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175365162 Host: 127.0.0.1
2013-09-24 00:05:56.646 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175679697 Host: aclwireless3.com

Divide it per Host by doing a count of each of them: - like this:

Host: 146.82.89.83 - 1
Host: 127.0.0.1 - 1
Host: aclwireless3.com - 1

==

Same thing for this:
2013-09-24 00:00:01.903 [ 8868] Statistic  WichaHandler         Web Server response received. MSISDN: 59175365162 Host: 127.0.0.1

===

Tks,
Joao
0
ozoCommented:
grep 'Statistic  WichaHandler         Sending Web Server request. MSISDN: 59178421279 Host:'  ig1a201309{1[89],2[1-4]}.log | awk '{print $NF}' | sort | uniq -c

However, that would not match
2013-09-24 00:00:01.900 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175365162 Host: 127.0.0.1
or
2013-09-24 00:05:56.646 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175679697 Host: aclwireless3.com
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Bootstrap 4: Exploring New Features

Learn how to use and navigate the new features included in Bootstrap 4, the most popular HTML, CSS, and JavaScript framework for developing responsive, mobile-first websites.

joaotellesAuthor Commented:
The grep is only for this:
Sending Web Server request. MSISDN:

So I will have this as output:
013-09-24 00:00:01.741 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59178421279 Host: 146.82.89.83
2013-09-24 00:00:01.900 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175365162 Host: 127.0.0.1
2013-09-24 00:05:56.646 [ 8868] Statistic  WichaHandler         Sending Web Server request. MSISDN: 59175679697 Host: aclwireless3.com

SOrry if it was confused...
0
TintinCommented:
Just change ozo's grep to

grep 'Sending Web Server request. MSISDN:'
0
joaotellesAuthor Commented:
Will it get the 7 last days of the timestamped files? (excluding the current day - so from 18 to 24 if today is 25.

ig1a20130918.log
ig1a20130919.log
ig1a20130920.log
ig1a20130921.log
ig1a20130922.log
ig1a20130923.log
ig1a20130924.log
ig1a20130925.log

grep 'Statistic  WichaHandler         Sending Web Server request. MSISDN:'  ig1a201309{1[89],2[1-4]}.log | awk '{print $NF}' | sort | uniq -c
0
TintinCommented:
You can confirm by doing

ls -1 ig1a201309{1[89],2[1-4]}.log
0
ozoCommented:
{1[89],2[1-4]} was an example that will only work on the 25th and would have to be modified to work on other days.
To automatically generate a list that works whatever the current day happens to be, you may want to use the technique of http:/Q_28247409.html
0
skullnobrainsCommented:
ls ig1a*.log | sort | tail -n 7

or using find

find . -name ig1a\*.log -mtime +6

....

something like this

sed -ne 's/.*Statistic  WichaHandler         Sending Web Server request. MSISDN:.*[[:space:]]\([^[:space:]]*\)$/\1/p' $(find . -name ig1a\*.log -mtime +6) | sort | uniq -c
0
joaotellesAuthor Commented:
Tks.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Shell Scripting

From novice to tech pro — start learning today.