Unix/Linux Help for the below command

Hi,

Can anyone pleas let me know  how the below command works.

awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort |uniq
new_perl_userAsked:
Who is Participating?

Improve company productivity with a Business Account.Sign Up

x
 
Gerwin Jansen, EE MVEConnect With a Mentor Topic Advisor Commented:
The grep command is using a date command that gets you yesterday's Day Month Date.

You can try this separate command if you like:

date -d 'yesterday' +"%a %b %d"

Open in new window


If you run this today (Fri Sep 30), it will give you Thu Sep 29 as output. Just try the above command.
0
 
Gerwin Jansen, EE MVETopic Advisor Commented:
Hi, it is a 'one liner' that processed the file HD_sync.log.10 with awk and sed and then sorts the output leaving only the unique entries.

Here's a description of what is happening. More details can be given if can post a sample HD_sync.log file.

awk'/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}'
1. Perform this awk command - for lines that match pattern STARTED or FINISHED - check if pattern /IMAGES[/]$/ is on this line, if so output the 2nd field of that line

HD_sync.log.10
2. on this file (your input file)

|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'
3. search beginning (^) of each line for pattern [^/]*[/]\(IMD.*\)\/IMAGES\/ and print the first matching pattern

|sort
4. sort the output

|uniq
5. keep only unique lines

Is this enough explanation for you? The above is really not 'a command' but a series of commands on the input file. You can view the output steps in between by doing these separate steps:

awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort
awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' HD_sync.log.10|sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p'|sort |uniq
0
 
new_perl_userAuthor Commented:
Hi,
Thank you so much for the above explanation it is crystal clear. Also need a small help.  how can we tweak the above series of commands  to grab  data based on date(it should get yesterday's data from the file)  and write it to a log file.
0
Build your data science skills into a career

Are you ready to take your data science career to the next step, or break into data science? With Springboard’s Data Science Career Track, you’ll master data science topics, have personalized career guidance, weekly calls with a data science expert, and a job guarantee.

 
wesly_chenCommented:
> awk'/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}'
> 1. Perform this awk command - for lines that match pattern STARTED or FINISHED
No
In HD_sync.log.10, search the content between "STARTED" and "FINISHED", If any line contain "IMAGES/" or "IMAGES" at the end of line, then print the
second field of that line

> grab  data based on date(it should get yesterday
You need to provide the date format and the location of date in  a line, like
2011/09/29 or
2011-09-29
0
 
new_perl_userAuthor Commented:
I can't hardcode the date. so is there a way to  tell it  to look for data sysdate -1
0
 
wesly_chenCommented:
> can't hardcode the date.
You just need to look at the HD_sync.log.10 and post the date/time format in that file (more than one for comparison)
0
 
new_perl_userAuthor Commented:
the date format in the file is:'Mon Sep 26 05:10:05 ' and this log file  gets data every hour  365 days . So everday I need to run the above command throught script and grab the data for yesterday . will it be possible, if so  can you please let me know.

Thanks,
0
 
Gerwin Jansen, EE MVETopic Advisor Commented:
@wesley_chen
>> search the content between "STARTED" and "FINISHED"
You are absolutely correct, sorry about that.

About adding the date filter: I would add this:

grep "`date -d 'yesterday' +"%a %b %d"`"

then add the (somewhat modified) original line to this:

grep "`date -d 'yesterday' +"%a %b %d"`" | awk  '/STARTED/,/FINISHED/{if($0~/IMAGES[/]$/){print $2}}' | sed -ne 's/^[^/]*[/]\(IMD.*\)\/IMAGES\//\1/p' | sort | uniq

Open in new window


The 'grep' line will filter the lines matching 'yesterday' in this format:
Mon Sep 26
0
 
new_perl_userAuthor Commented:
oh my bad.  Yesterday in the sense I eas telling that it should get previous day data, but not grep for yestarday. It has to grep only date for previous day.

For example if I am executing the command tomorrow  it has to read the file and get today's data  as output.
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.