Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Need to cut lines from top of text file

Posted on 2008-10-27
9
Medium Priority
?
694 Views
Last Modified: 2013-12-27
HI all,

I need to cut all line above the the DASH line (----- -- -- -----)

I've been using the cut and tail, but undercertain circumstance the amount of line above the dash line changes.  So hard coding to cut the top 7 lines doesnt work at certain times.... and its causing me issues because I have a script  that outputs the data CSV format and loads into a mysql database.  Needless to say, its causing issues with the database when loading a bunch of dashs....

So, my question is there some way to tell the system to cut only the line above the dash line ( and cut he dash line as well. basiclly keep everything under the dash line)

bash-3.00# cat pcavanilla
Downloading xref file to /var/tmp/patchdiag.xref
Trying http://sunsolve.sun.com/patchdiag.xref (1/1)
Using /var/tmp/patchdiag.xref from Oct/24/08
Host: fxssun (SunOS 5.10/Generic_137112-08/i386/i86pc)
List: missing
 
Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
119964 10 < 11 ---  11 SunOS 5.10_x86: Shared library patch for C++_x86
bash-3.00#
 
here is what i'm doing now to cut the lines out and export to csv format

pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

Open in new window

0
Comment
Question by:jedblack
  • 3
  • 3
  • 2
  • +1
9 Comments
 
LVL 40

Expert Comment

by:omarfarid
ID: 22814358
try

loc=`cat file | grep -n '^--' | awk -F: '{ print $1 }'`
len=`cat file | wc -l`
lines=`expr $len - $loc`
tail -$lines file > newfile
0
 
LVL 22

Expert Comment

by:Brian Utterback
ID: 22815029
You could also do it this way:

perl -ne 'print if /-----------/../IMPOSSIBLESTRING/' pcavanilla > newfile
0
 
LVL 1

Author Comment

by:jedblack
ID: 22816021
omarfarid,

I tried that, and it seemed to work...but this is the script i have in place right now to convert the PCA output to csv format, I've tried intergrating your code into the script...but it wont work.  Any ideas on how to modify the current script to include your solution?


bash-3.00# cat pca2csv.sh
#!/bin/bash
 
pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

Open in new window

0
Technology Partners: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 40

Expert Comment

by:omarfarid
ID: 22816098
what is the output of your script and how do you want in include mine into this?
0
 
LVL 22

Expert Comment

by:Brian Utterback
ID: 22816184
You can easily do this with mine:

#!/bin/bash
 
pca -l |perl -ne 'print if /-----------/../IMPOSSIBLESTRING/' | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

But actually, it would be even better to just to it all in perl in the first place. What is the format of your input file? In your first mesage you show a download of the patchdiag list, but that doesn't really jive with what you show later.  What is the field separator on the lines you input? What do you what the output to look like? I see semicolons and commas in there.
0
 
LVL 1

Author Comment

by:jedblack
ID: 22817278
We use a script called PCA to patch our solaris machine.  However, there is no reporting function built into the script.  So, I engineered a method to migrate the output produced by the script and batch load into a MYSQL database.  Below is what the un-altered raw output looks like....

-bash-3.00$ ./pca -l
Downloading xref file to /var/tmp/patchdiag.xref
Trying http://sunsolve.sun.com/patchdiag.xref (1/1)
Using /var/tmp/patchdiag.xref from Oct/24/08
Host: pca (SunOS 5.10/Generic_118833-24/sparc/sun4u)
List: missing
 
Patch  IR   CR RSB Age Synopsis
------ -- - -- --- --- -------------------------------------------------------
119254 42 < 59 RS-   5 SunOS 5.10: Install and Patch Utilities Patch
118666 -- < 17 -S- 111 JavaSE 5.0: update 16 patch (equivalent to JDK 5.0u16)
118676 -- < 03 --- 682 SunOS 5.10: patch for Solaris make and sccs utilities
118683 -- < 02 --- 410 SunOS 5.10: Patch for assembler
118735 01 < 02 --- 158 SunOS 5.10: patch usr/sbin/rpc.nisd_resolv
118777 -- < 13 R-- 164 SunOS 5.10: Sun GigaSwift Ethernet 1.0 driver patch
118814 -- < 01 --- 999 SunOS 5.10: patch platform/sun4u/kernel/tod/sparcv9/todsg
118824 -- < 02 --- 157 SunOS 5.10: patch usr/bin/sparcv9/sort
119578 29 < 30 R-- 739 SunOS 5.10: FMA Patch
118833 24 < 36 RS- 637 SunOS 5.10: kernel patch
118842 -- < 01 --- 999 SunOS 5.10: dada patch
118884 -- < 01 --- 999 SunOS 5.10: atomic.h patch
118918 19 < 24 RS- 630 SunOS 5.10: Solaris Crypto Framework patch
118981 -- < 03 --- 927 SunOS 5.10: Sun Quad FastEthernet qfe driver
119042 10 < 11 R-- 145 SunOS 5.10: svccfg & svcprop patch
119059 31 < 45 RS-  87 X11 6.6.2: Xsun patch
119090 -- < 31 R--  69 SunOS 5.10: Sun iSCSI Device Driver and Utilities
119130 26 < 33 R-- 621 SunOS 5.10: Sun Fibre Channel Device Drivers



Then I run this to turn the output in a CSV format to loading into the database.  Below is the command and the data output after running agaist the command
NOTE* part of the script inserts the date and hostname in the first two columns of the output
-------------------------
bash-3.00# cat pca2csv.sh
#!/bin/bash
 
pca -l | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done

-------------------------


bash-3.00# cat pca2db.js
js; 10/22/08;118677;02;03;---;656;SunOS 5.10_x86: patch for make utilities
js; 10/22/08;118778;07;11;---;329;SunOS 5.10_x86: Sun GigaSwift 1.0
js; 10/22/08;118816;03;04;---;369;SunOS 5.10_x86: /usr/xpg4/bin/awk patch
js; 10/22/08;119091;22;26;---;278;SunOS 5.10_x86: Sun iSCSI Driver
js; 10/22/08;119281;08;15;---;273;CDE 1.6_x86: Runtime library for
js; 10/22/08;119398;04;05;---;617;SunOS 5.10_x86: Patch for America issues
js; 10/22/08;119471;07;09;---;509;SunOS 5.10_x86: Sun Enterprise Array utilitie
js; 10/22/08;119535;09;13;---;376;SunOS 5.10_x86: Flash Archive
js; 10/22/08;119964;07;08;---;714;SunOS 5.10_x86: Shared library for
js; 10/22/08;119975;05;07;---;546;SunOS 5.10_x86: fp plug-in cfgadm
js; 10/22/08;120347;05;08;---;484;SunOS 5.10_x86: Common Fibre HBA Host Bus Li
js; 10/22/08;120349;01;02;---;673;SunOS 5.10_x86: Fibre Channel Port
js; 10/22/08;120411;17;26;---;286;SunOS 5.10_x86: Internet/Intranet Input Framework
js; 10/22/08;120754;02;05;---;572;SunOS 5.10_x86: Microtasking libraries patch


So, now here is the problem, I've noticed that depending on certains situations the  amount of top lines above the DASH(---) line differ.  I have it hard coded into my script to remove the top 7 lines, but sometime that leaves or takes too many lines away.

That is why I asked my original question....

Omarfarid, your methods works, but i'm not sure how to implement that in my script.
Blu, your method works...but it still leaves the DASH line





0
 
LVL 6

Expert Comment

by:Talmash
ID: 22817386
egrep -B1 '^\-+$' file

tal
0
 
LVL 40

Accepted Solution

by:
omarfarid earned 2000 total points
ID: 22818747


pca -l > file

loc=`cat file | grep -n '^--' | awk -F: '{ print $1 }'`
len=`cat file | wc -l`
lines=`expr $len - $loc`
tail -$lines file > newfile
cat newfile  | tail +7 | while read line
do
  echo `hostname`";"`date '+ %m/%d/%y'";"``echo $line | awk '{print $1";"$2";"$4";"$5";"$6";"$7,$8,$9,$10,$11,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'`> ./pca2db.`hostname`
 
done
0
 
LVL 22

Expert Comment

by:Brian Utterback
ID: 22821128
Okay, how about this:

#!/bin/perl
$,=';';
chomp ($h = `hostname`);
chomp ($d = `date '+%m/%d/%y'`);
while (<>) {
        if (/-----/) {$flag++;next;}
        next unless $flag;
        ($f1,$f2,$f3,$f4,$f5,$f6,$f7) = split(/\s+/,$_,7);
        print $h,$d,$f1,$f2,$f4,$f5,$f6,$f7;
}


Note that this is 3 processes created in total, while the script above uses 8 process plus 3 per line of input.
0

Featured Post

Free Tool: ZipGrep

ZipGrep is a utility that can list and search zip (.war, .ear, .jar, etc) archives for text patterns, without the need to extract the archive's contents.

One of a set of tools we're offering as a way to say thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Attention: This article will no longer be maintained. If you have any questions, please feel free to mail me. jgh@FreeBSD.org Please see http://www.freebsd.org/doc/en_US.ISO8859-1/articles/freebsd-update-server/ for the updated article. It is avail…
This tech tip describes how to install the Solaris Operating System from a tape backup that was created using the Solaris flash archive utility. I have used this procedure on the Solaris 8 and 9 OS, and it shoudl also work well on the Solaris 10 rel…
Learn how to find files with the shell using the find and locate commands. Use locate to find a needle in a haystack.: With locate, check if the file still exists.: Use find to get the actual location of the file.:
This video shows how to set up a shell script to accept a positional parameter when called, pass that to a SQL script, accept the output from the statement back and then manipulate it in the Shell.

876 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question