[2 days left] What’s wrong with your cloud strategy? Learn why multicloud solutions matter with Nimble Storage.Register Now

x
?
Solved

Unix/Linux Help for the below command

Posted on 2011-09-30
9
Medium Priority
?
469 Views
Last Modified: 2013-11-22
Hi,

Can anyone pleas let me know  how the below command works.

awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort |uniq
0
Comment
Question by:new_perl_user
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 4
  • 3
  • 2
9 Comments
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 36891941
Hi, it is a 'one liner' that processed the file HD_sync.log.10 with awk and sed and then sorts the output leaving only the unique entries.

Here's a description of what is happening. More details can be given if can post a sample HD_sync.log file.

awk'/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}'
1. Perform this awk command - for lines that match pattern STARTED or FINISHED - check if pattern /IMAGES[/]$/ is on this line, if so output the 2nd field of that line

HD_sync.log.10
2. on this file (your input file)

|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'
3. search beginning (^) of each line for pattern [^/]*[/]\(IMD.*\)\/IMAGES\/ and print the first matching pattern

|sort
4. sort the output

|uniq
5. keep only unique lines

Is this enough explanation for you? The above is really not 'a command' but a series of commands on the input file. You can view the output steps in between by doing these separate steps:

awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort |uniq
0
 

Author Comment

by:new_perl_user
ID: 36892020
Hi,
Thank you so much for the above explanation it is crystal clear. Also need a small help.  how can we tweak the above series of commands  to grab  data based on date(it should get yesterday's data from the file)  and write it to a log file.
0
 
LVL 38

Expert Comment

by:wesly_chen
ID: 36892663
> awk'/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}'
> 1. Perform this awk command - for lines that match pattern STARTED or FINISHED
No
In HD_sync.log.10, search the content between "STARTED" and "FINISHED", If any line contain "IMAGES/" or "IMAGES" at the end of line, then print the
second field of that line

> grab  data based on date(it should get yesterday
You need to provide the date format and the location of date in  a line, like
2011/09/29 or
2011-09-29
0
Get free NFR key for Veeam Availability Suite 9.5

Veeam is happy to provide a free NFR license (1 year, 2 sockets) to all certified IT Pros. The license allows for the non-production use of Veeam Availability Suite v9.5 in your home lab, without any feature limitations. It works for both VMware and Hyper-V environments

 

Author Comment

by:new_perl_user
ID: 36892746
I can't hardcode the date. so is there a way to  tell it  to look for data sysdate -1
0
 
LVL 38

Expert Comment

by:wesly_chen
ID: 36892807
> can't hardcode the date.
You just need to look at the HD_sync.log.10 and post the date/time format in that file (more than one for comparison)
0
 

Author Comment

by:new_perl_user
ID: 36892847
the date format in the file is:'Mon Sep 26 05:10:05 ' and this log file  gets data every hour  365 days . So everday I need to run the above command throught script and grab the data for yesterday . will it be possible, if so  can you please let me know.

Thanks,
0
 
LVL 38

Expert Comment

by:Gerwin Jansen, EE MVE
ID: 36893286
@wesley_chen
>> search the content between "STARTED" and "FINISHED"
You are absolutely correct, sorry about that.

About adding the date filter: I would add this:

grep "`date -d 'yesterday' +"%a %b %d"`"

then add the (somewhat modified) original line to this:

grep "`date -d 'yesterday' +"%a %b %d"`" | awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' | sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p' | sort | uniq

Open in new window


The 'grep' line will filter the lines matching 'yesterday' in this format:
Mon Sep 26
0
 

Author Comment

by:new_perl_user
ID: 36893336
oh my bad.  Yesterday in the sense I eas telling that it should get previous day data, but not grep for yestarday. It has to grep only date for previous day.

For example if I am executing the command tomorrow  it has to read the file and get today's data  as output.
0
 
LVL 38

Accepted Solution

by:
Gerwin Jansen, EE MVE earned 2000 total points
ID: 36893444
The grep command is using a date command that gets you yesterday's Day Month Date.

You can try this separate command if you like:

date -d 'yesterday' +"%a %b %d"

Open in new window


If you run this today (Fri Sep 30), it will give you Thu Sep 29 as output. Just try the above command.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

SSH (Secure Shell) - Tips and Tricks As you all know SSH(Secure Shell) is a network protocol, which we use to access/transfer files securely between two networked devices. SSH was actually designed as a replacement for insecure protocols that sen…
Active Directory replication delay is the cause to many problems.  Here is a super easy script to force Active Directory replication to all sites with by using an elevated PowerShell command prompt, and a tool to verify your changes.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.
In a recent question (https://www.experts-exchange.com/questions/29004105/Run-AutoHotkey-script-directly-from-Notepad.html) here at Experts Exchange, a member asked how to run an AutoHotkey script (.AHK) directly from Notepad++ (aka NPP). This video…
Suggested Courses

649 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question