[Okta Webinar] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

AWK and Sort

Posted on 2004-12-01
4
Medium Priority
?
438 Views
Last Modified: 2013-12-27
I have a awk command I need to modify, I need to:

1) find all lines that have refuesed in them,usally in column 18 but not always.
2) find all lines that have the same value in column 7
3) print to screen one copy of each different value it finds in column 7 and the number of times it reoccured in the file,
   and print out column 6, 7, and 18.
   


Command i need to modify:

awk ' /refused/ {print $6, $7, $18}' log_file | sort | uniq -c | sort -nr


Example of what want it to look like:



22-Nov-2004 01:35:54.44 tcp_in              Q 5 jcfaxhgibblyu@yyhmail.com rfc822;find@com find@com /mail/06/queue/tcp_in/002/ZR0I7Z00AF6W3UDF.00 <FYCQIMPNYZQGGRYOZKLQHMEM@wonet.com>  TCP active open: Failed connect()    Error: Connection refused


22-Nov-2004 01:35:54.44 tcp_in              Q 5 jcfaxhgibblyu@yyhmail.com rfc822;newmail.com newmail.com /mail/06/queue/tcp_in/002/ZR0I7Z00AF6W3UDF.00 <FYCQIMPNYZQGGRYOZKLQHMEM@wonet.com>  TCP active open: Failed connect()    Error: Connection refused


22-Nov-2004 01:35:54.44 tcp_in              Q 5 mymail.com rfc822;find@com find@com /mail/06/queue/tcp_in/002/IUYGHTREAF6W3UDF.00 <OIUYGMPNYZQGGRYOIUYHKJ@wonet.com>  TCP active open: Failed connect()    Error: Connection refused


22-Nov-2004 01:35:54.44 tcp_in              Q 5 jcfaxhgibblyu@yyhmail.com rfc822;byrd@villahermosa.com byrd@villahermosa.com /mail/06/queue/tcp_in/002/ZR0I7Z00AF6W3UDF.00 <FYCQIMPNYZQGGRYOZKLQHMEM@wonet.com>  TCP active open: Failed connect()    Error: Connection refused




2 jcfaxhgibblyu@yyhmail.com rfc822;find@com find@com
1 jcfaxhgibblyu@yyhmail.com rfc822;newmail.com
1 jcfaxhgibblyu@yyhmail.com rfc822;byrd@villahermosa.com


Thanks,
0
Comment
Question by:bt707
  • 2
  • 2
4 Comments
 
LVL 21

Accepted Solution

by:
tfewster earned 2000 total points
ID: 12718317
Using the -f option to "uniq" to ignore the first field will work - As long as you're certain that's what you want ;-)

awk ' /refused/ {print $6, $7, $18}' log_file | sort -k 2 | uniq -c -f 1 | sort -nr
0
 

Author Comment

by:bt707
ID: 12718620
that what i was looking at but can't seem to get it working.

what I need to do is to look at the 7th feild and find all lines that have the 7th field duplicated then print out how many times that this same value appeared
in the 7th field.


Thanks,
0
 
LVL 21

Expert Comment

by:tfewster
ID: 12719992
Now I'm not sure what the problem is; My amendment to your command does the following:
- Sorts the output using the second field, i.e. rfc822;address@domain to bring them all together (This IS the seventh field of the log_file).
- Does a "uniq" on the second (and subsequent) fields. If two ADJACENT lines contain the same second (and subsequent) fields, only one line is printed plus a total.  Note that as the first field is ignored in the "uniq", some of the "from" addresses (field 6 in the log, i.e. field 1 in the extract) are lost

In your example output, you seem to be extracting fields 6, 7 & 8 (not 18) as the email address is repeated (or is it a domain? I'm not sure). But the principle still stands...I tested it on your input and got the sort/summary you posted in your example.  Or am I missing something?
0
 

Author Comment

by:bt707
ID: 12720565
No i'm just not explaining it very good i guess.

What you put did work great for what I was looking for, I'm now trying to modify the results from that, I was printing out the other lines, but they are not really neccessary.

I'll close this one and open up new question for the next part of what I am trying to do,

This part is working great, your command was just right, i was trying to make it to complicated and missed it.


Thanks again,
0

Featured Post

VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Installing FreeBSD… FreeBSD is a darling of an operating system. The stability and usability make it a clear choice for servers and desktops (for the cunning). Savvy?  The Ports collection makes available every popular FOSS application and packag…
I promised to write further about my project, and here I am.  First, I needed to setup the Primary Server.  You can read how in this article: Setup FreeBSD Server with full HDD encryption (http://www.experts-exchange.com/OS/Unix/BSD/FreeBSD/A_3660-S…
Learn how to get help with Linux/Unix bash shell commands. Use help to read help documents for built in bash shell commands.: Use man to interface with the online reference manuals for shell commands.: Use man to search man pages for unknown command…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Suggested Courses
Course of the Month19 days, 1 hour left to enroll

834 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question