Solved

Need to cut lines from top of text file

Posted on 2008-10-27
9
691 Views
Last Modified: 2013-12-27
HI all,

I need to cut all line above the the DASH line (----- -- -- -----)

I've been using the cut and tail, but undercertain circumstance the amount of line above the dash line changes.  So hard coding to cut the top 7 lines doesnt work at certain times.... and its causing me issues because I have a script  that outputs the data CSV format and loads into a mysql database.  Needless to say, its causing issues with the database when loading a bunch of dashs....

So, my question is there some way to tell the system to cut only the line above the dash line ( and cut he dash line as well. basiclly keep everything under the dash line)

bash-3.00# cat pcavanilla
Downloading xref file to /var/tmp/patchdiag.xref
Trying http://sunsolve.sun.com/patchdiag.xref (1/1)
Using /var/tmp/patchdiag.xref from Oct/24/08
Host: fxssun (SunOS 5.10/Generic_137112-08/i386/i86pc)
List: missing
 
Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
119964 10 < 11 ---  11 SunOS 5.10_x86: Shared library patch for C++_x86
bash-3.00#
 
here is what i'm doing now to cut the lines out and export to csv format

pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

Open in new window

0
Comment
Question by:jedblack
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 40

Expert Comment

by:omarfarid
ID: 22814358
try

loc=`cat file | grep -n '^--' | awk -F: '{ print $1 }'`
len=`cat file | wc -l`
lines=`expr $len - $loc`
tail -$lines file > newfile
0
 
LVL 22

Expert Comment

by:blu
ID: 22815029
You could also do it this way:

perl -ne 'print if /-----------/../IMPOSSIBLESTRING/' pcavanilla > newfile
0
 
LVL 1

Author Comment

by:jedblack
ID: 22816021
omarfarid,

I tried that, and it seemed to work...but this is the script i have in place right now to convert the PCA output to csv format, I've tried intergrating your code into the script...but it wont work.  Any ideas on how to modify the current script to include your solution?


bash-3.00# cat pca2csv.sh
#!/bin/bash
 
pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

Open in new window

0
Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 40

Expert Comment

by:omarfarid
ID: 22816098
what is the output of your script and how do you want in include mine into this?
0
 
LVL 22

Expert Comment

by:blu
ID: 22816184
You can easily do this with mine:

#!/bin/bash
 
pca -l |perl -ne 'print if /-----------/../IMPOSSIBLESTRING/' | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

But actually, it would be even better to just to it all in perl in the first place. What is the format of your input file? In your first mesage you show a download of the patchdiag list, but that doesn't really jive with what you show later.  What is the field separator on the lines you input? What do you what the output to look like? I see semicolons and commas in there.
0
 
LVL 1

Author Comment

by:jedblack
ID: 22817278
We use a script called PCA to patch our solaris machine.  However, there is no reporting function built into the script.  So, I engineered a method to migrate the output produced by the script and batch load into a MYSQL database.  Below is what the un-altered raw output looks like....

-bash-3.00$ ./pca -l
Downloading xref file to /var/tmp/patchdiag.xref
Trying http://sunsolve.sun.com/patchdiag.xref (1/1)
Using /var/tmp/patchdiag.xref from Oct/24/08
Host: pca (SunOS 5.10/Generic_118833-24/sparc/sun4u)
List: missing
 
Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
119254 42 < 59 RS-   5 SunOS 5.10: Install and Patch Utilities Patch
118666 -- < 17 -S- 111 JavaSE 5.0: update 16 patch (equivalent to JDK 5.0u16)
118676 -- < 03 --- 682 SunOS 5.10: patch for Solaris make and sccs utilities
118683 -- < 02 --- 410 SunOS 5.10: Patch for assembler
118735 01 < 02 --- 158 SunOS 5.10: patch usr/sbin/rpc.nisd_resolv
118777 -- < 13 R-- 164 SunOS 5.10: Sun GigaSwift Ethernet 1.0 driver patch
118814 -- < 01 --- 999 SunOS 5.10: patch platform/sun4u/kernel/tod/sparcv9/todsg
118824 -- < 02 --- 157 SunOS 5.10: patch usr/bin/sparcv9/sort
119578 29 < 30 R-- 739 SunOS 5.10: FMA Patch
118833 24 < 36 RS- 637 SunOS 5.10: kernel patch
118842 -- < 01 --- 999 SunOS 5.10: dada patch
118884 -- < 01 --- 999 SunOS 5.10: atomic.h patch
118918 19 < 24 RS- 630 SunOS 5.10: Solaris Crypto Framework patch
118981 -- < 03 --- 927 SunOS 5.10: Sun Quad FastEthernet qfe driver
119042 10 < 11 R-- 145 SunOS 5.10: svccfg & svcprop patch
119059 31 < 45 RS-  87 X11 6.6.2: Xsun patch
119090 -- < 31 R--  69 SunOS 5.10: Sun iSCSI Device Driver and Utilities
119130 26 < 33 R-- 621 SunOS 5.10: Sun Fibre Channel Device Drivers



Then I run this to turn the output in a CSV format to loading into the database.  Below is the command and the data output after running agaist the command
NOTE* part of the script inserts the date and hostname in the first two columns of the output
-------------------------
bash-3.00# cat pca2csv.sh
#!/bin/bash
 
pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

-------------------------


bash-3.00# cat pca2db.js
js; 10/22/08;118677;02;03;---;656;SunOS 5.10_x86: patch for make utilities
js; 10/22/08;118778;07;11;---;329;SunOS 5.10_x86: Sun GigaSwift 1.0
js; 10/22/08;118816;03;04;---;369;SunOS 5.10_x86: /usr/xpg4/bin/awk patch
js; 10/22/08;119091;22;26;---;278;SunOS 5.10_x86: Sun iSCSI Driver
js; 10/22/08;119281;08;15;---;273;CDE 1.6_x86: Runtime library for
js; 10/22/08;119398;04;05;---;617;SunOS 5.10_x86: Patch for America issues
js; 10/22/08;119471;07;09;---;509;SunOS 5.10_x86: Sun Enterprise Array utilitie
js; 10/22/08;119535;09;13;---;376;SunOS 5.10_x86: Flash Archive
js; 10/22/08;119964;07;08;---;714;SunOS 5.10_x86: Shared library for
js; 10/22/08;119975;05;07;---;546;SunOS 5.10_x86: fp plug-in cfgadm
js; 10/22/08;120347;05;08;---;484;SunOS 5.10_x86: Common Fibre HBA Host Bus Li
js; 10/22/08;120349;01;02;---;673;SunOS 5.10_x86: Fibre Channel Port
js; 10/22/08;120411;17;26;---;286;SunOS 5.10_x86: Internet/Intranet Input Framework
js; 10/22/08;120754;02;05;---;572;SunOS 5.10_x86: Microtasking libraries patch


So, now here is the problem, I've noticed that depending on certains situations the  amount of top lines above the DASH(---) line differ.  I have it hard coded into my script to remove the top 7 lines, but sometime that leaves or takes too many lines away.

That is why I asked my original question....

Omarfarid, your methods works, but i'm not sure how to implement that in my script.
Blu, your method works...but it still leaves the DASH line





0
 
LVL 6

Expert Comment

by:Talmash
ID: 22817386
egrep -B1 '^\-+$' file

tal
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 500 total points
ID: 22818747


pca -l > file

loc=`cat file | grep -n '^--' | awk -F: '{ print $1 }'`
len=`cat file | wc -l`
lines=`expr $len - $loc`
tail -$lines file > newfile
cat newfile  | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done
0
 
LVL 22

Expert Comment

by:blu
ID: 22821128
Okay, how about this:

#!/bin/perl
$,=';';
chomp ($h = `hostname`);
chomp ($d = `date '+%m/%d/%y'`);
while (<>) {
        if (/-----/) {$flag++;next;}
        next unless $flag;
        ($f1,$f2,$f3,$f4,$f5,$f6,$f7) = split(/\s+/,$_,7);
        print $h,$d,$f1,$f2,$f4,$f5,$f6,$f7;
}


Note that this is 3 processes created in total, while the script above uses 8 process plus 3 per line of input.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This tech tip describes how to install the Solaris Operating System from a tape backup that was created using the Solaris flash archive utility. I have used this procedure on the Solaris 8 and 9 OS, and it shoudl also work well on the Solaris 10 rel…
I have been running these systems for a few years now and I am just very happy with them.   I just wanted to share the manual that I have created for upgrades and other things.  Oooh yes! FreeBSD makes me happy (as a server), no maintenance and I al…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
Learn how to navigate the file tree with the shell. Use pwd to print the current working directory: Use ls to list a directory's contents: Use cd to change to a new directory: Use wildcards instead of typing out long directory names: Use ../ to move…

615 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question