Advertisement

01.04.2008 at 01:08AM PST, ID: 23058031
[x]
Attachment Details
[x]
The Solution Rating System

With so many solutions, how can you tell which solutions are most likely to help you and which ones are not? To provide you with a tool to use, we rate our solutions based on various elements that most accurately determine if a solution is a quality solution. To explain what factors affect the solution rating, here are the elements we take into consideration when formulating our solution rating.

  • The Grade of the Solution
  • The Zone Rank of the Expert Providing the Solution
  • The Number of Author and Expert Comments
  • The Number of Experts Contributing
  • The Feedback of the Community

Your Input Matters
Because of the way the system is set up, the most important variable in this equation is you. As a member of Experts Exchange, you are able to cast your vote on the quality of the solutions in regard to how complete, accurate, helpful and easy to understand each solution is. When you provide your feedback, each rating is adjusted accordingly. So, if you see a solution that has a poor rating that you think is a good solution, let us know by rating it. As you do, the rating will be adjusted and will become more accurate for other members of our site.

If you have any suggestions that you would like to make for our rating system, please ask a question in the Suggestions Zone of Community Support.

Thank you!

Parse multiple line entries to produce single line output

Tags: ksh perl awk
Can I use awk or perl or other utility to make one output entry for a line starting with a date and 3 following lines with a filled parameter? so it looks like this?:

02-01-2008 03:34:50 201 0 0 mail@somewhere.com
02-01-2008 03:34:50 200 0 0
02-01-2008 03:35:23 201 0 0 mail@somewhere.com
02-01-2008 03:37:42 400 12 1

Input file:

[2008.01.02-03:34:50.170] aa.app.lib.Status$Req
[2008.01.02-03:34:50.172] aa.app.lib.Status$Response
header.function=[201]
header.completion=[0]
data.email=[mail@somewhere.com]
header.error=[0]
[2008.01.02-03:34:50.172] aa.app.lib.Status$Response
header.function=[200]
header.completion=[0]
header.error=[0]
[2008.01.02-03:35:23.822] aa.app.lib.Status$Req
[2008.01.02-03:35:23.822] app term server 0
[2008.01.02-03:35:23.823] Conn 1 server
[2008.01.02-03:35:23.856] Term use:1
[2008.01.02-03:35:23.860] aa.app.lib.Status$Req
[2008.01.02-03:35:23.862] aa.app.lib.Status$Response
header.function=[201]
header.completion=[0]
header.error=[0]
[2008.01.02-03:37:42.532] aa.app.lib.Status$Response
header.function=[400]
header.completion=[12]
data.email=[mail@somewhere.com]
header.error=[1]
Start your free trial to view this solution
Question Stats
Zone: Programming
Question Asked By: paulqna
Solution Provided By: ozo
Participating Experts: 2
Solution Grade: A
Views: 47
Translate:
Loading Advertisement...
01.04.2008 at 01:15AM PST, ID: 20580734

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.04.2008 at 01:38AM PST, ID: 20580776

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.04.2008 at 03:08AM PST, ID: 20581061

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.04.2008 at 08:32AM PST, ID: 20583647

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.04.2008 at 10:36AM PST, ID: 20584880

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.06.2008 at 11:37PM PST, ID: 20597513

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.06.2008 at 11:43PM PST, ID: 20597533

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 12:20AM PST, ID: 20597635

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 02:45AM PST, ID: 20598126

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 03:12AM PST, ID: 20598203

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 03:25AM PST, ID: 20598249

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 03:46AM PST, ID: 20598319

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 03:56AM PST, ID: 20598360

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 06:32AM PST, ID: 20599638

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 07:28AM PST, ID: 20600115

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 09:57AM PST, ID: 20601373

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 10:08AM PST, ID: 20601468

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 11:05AM PST, ID: 20602056

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
01.07.2008 at 12:37PM PST, ID: 20602974

Rank: Genius

All comments and solutions are available to Premium Service Members only.

Start your 7-day free trial and see for yourself why Experts Exchange is the easiest and most proven technology resource in the world. Get Started

Already a member? Login to view this solution.

 
 
Loading Advertisement...
Microsoft
  • Internet Protocols
  • Applications
  • Development
  • OS
  • Hardware
  • Windows Security
Apple
  • Operating Systems
  • Hardware
  • Programming
  • Networking
  • Software
Internet
  • Search Engines
  • File Sharing
  • WebTrends / Stats
  • Spy / Ad Blockers
  • Web Browsers
  • New Net Users
  • Web Development
  • Chat / IM
  • Anti Spam
  • Web Servers
  • Anti-Virus
  • Email Clients
Gamers
  • Tips
  • Online / MMORPG
  • Puzzle
  • Emulators
  • Action / Adventure
  • Role Playing
  • Consoles
  • Game Programming
  • Strategy
  • Sports
  • Misc
  • Computer Games
Digital Living
  • Hardware
  • Automotive
  • New Net Users
  • New Users
  • Software
  • Digital Music
  • Gaming World
  • Home Security
  • Apple
  • Networking Hardware
Virus & Spyware
  • Vulnerabilities
  • IDS
  • Encryption
  • Anti-Virus
  • Operating Systems Security
  • Software Firewalls
  • WebApplications
  • Cell Phones
  • Operating Systems
  • Internet
  • Hardware Firewalls
Hardware
  • Displays / Monitors
  • Handhelds / PDAs
  • Components
  • Peripherals
  • Laptops/Notebooks
  • Servers
  • Misc
  • Apple
  • Embedded Hardware
  • Networking Hardware
  • Storage
  • Desktops
  • New Users
Software
  • System Utilities
  • Industry Specific
  • Network Management
  • Photos / Graphics
  • Page Layout
  • VMware
  • Misc
  • Web Development
  • OS
  • CYGWIN
  • Voice Recognition
  • Virtualization
  • Message Queue
  • Quality Assurance
  • Security
  • Firewalls
  • MultiMedia Applications
  • Development
  • Database
  • Office / Productivity
  • Business Management
  • OS/2 Apps
  • Server Software
  • Internet / Email
ITPro
  • OS
  • Storage
  • Encryption
  • Operating Systems Security
  • Apple Hardware
  • Laptops & Notebooks
  • Servers
  • Networking Hardware
  • Peripherals
  • Devices
  • Displays / Monitors
  • WebTrends / Stats
  • Search Engines
  • Firewalls
  • Web Computing
  • WebApplications
  • IDS
  • Vulnerabilities
  • Email Clients
  • File Sharing
  • Spy / Ad Blockers
  • Web Browsers
  • Web Servers
  • Networking
  • Anti-Virus
  • Consulting
  • Chat / IM
  • Anti Spam
Developer
  • Web Servers
  • Web Browsers
  • Game Programming
  • Dev Tools
  • Industry Specific
  • Office / Productivity
  • Database
  • CYGWIN
  • Web Development
  • Search Engines
  • File Sharing
  • WebTrends / Stats
  • Programming
  • Content Management
  • Application Servers
  • Protocols
Storage
  • Removable Backup Media
  • Storage Technology
  • Servers
  • Grid
  • Remote Access
  • Backup / Restore
  • Misc
  • Hard Drives
OS
  • Miscellaneous
  • Security
  • Development
  • Linux
  • VMware
  • MainFrame OS
  • Unix
  • Apple
  • OS / 2
  • AS / 400
  • BeOS
  • Microsoft
  • VMS / OpenVMS
Database
  • Oracle
  • Miscellaneous
  • MySQL
  • Software
  • Sybase
  • Contact Management
  • PostgreSQL
  • Data Manipulation
  • Clarion
  • InterSystems Cache
  • Siebel
  • MUMPS
  • OLAP
  • SQLBase
  • SAS
  • GIS & GPS
  • 4GL
  • Berkeley DB
  • DB2
  • Informix
  • Interbase / Firebird
  • FoxPro
  • Reporting
  • LDAP
  • Filemaker Pro
  • MS SQL Server
  • dBase
  • MS Access
Security
  • Misc
  • Web Browsers
  • Software Firewalls
  • Operating Systems Security
  • File Sharing
  • Spy / Ad Blockers
  • Vulnerabilities
  • WebApplications
  • IDS
  • Anti-Virus
  • Encryption
  • Anti Spam
  • Email Clients
  • VPN
  • Chat / IM
Programming
  • Editors IDEs
  • Installation
  • Handhelds / PDAs
  • Multimedia Programming
  • System / Kernel
  • Automation
  • Algorithms
  • Game
  • Signal Processing
  • Project Management
  • Open Source
  • Database
  • Misc
  • Languages
  • Processor Platforms
  • Theory
Web Development
  • Scripting
  • Blogs
  • Web Servers
  • Software
  • Search Engines
  • Web Graphics
  • Web Services
  • Images
  • Internet Marketing
  • Images and Photos
  • Components
  • Document Imaging
  • Web Languages/Standards
  • Illustration
  • WebApplications
  • Fonts
  • WebTrends / Stats
  • Authoring
  • Digital Camera Software
  • Miscellaneous
Networking
  • Protocols
  • Apple Networking
  • Network Management
  • Message Queue
  • Application Servers
  • Content Management
  • File Servers
  • Email Servers
  • Misc
  • Java Editors & IDEs
  • Wireless
  • Networking Hardware
  • Backup / Restore
  • System Utilities
  • ISPs & Hosting
  • Web Servers
  • Storage Technology
  • Removable Backup Media
  • Servers
  • Web Computing
  • Broadband
  • Grid
  • OS / 2
  • Novell Netware
  • Unix Networking
  • Windows Networking
  • Security
  • Telecommunications
  • Operating Systems
  • Linux Networking
Other
  • Lounge
  • Business Travel
  • Community Support
  • New Net Users
  • Philosophy / Religion
  • Math / Science
  • Miscellaneous
  • URLs
  • Expert Lounge
  • Politics
  • Puzzles / Riddles
  • Automotive
Community Support
  • Suggestions
  • New to EE
  • New Topics
  • CleanUp
  • Announcements
  • General
  • Feedback
  • Input
  • EE Bugs
 
01.04.2008 at 01:15AM PST, ID: 20580734

Rank: Genius

Why not
02-01-2008 03:34:50 201 0 0 mail@somewhere.com
02-01-2008 03:34:50 200 0 0
02-01-2008 03:35:23 201 0 0
02-01-2008 03:37:42 400 12 1 mail@somewhere.com
 
01.04.2008 at 01:38AM PST, ID: 20580776

Rank: Genius

If you meant

02-01-2008 03:34:50 201 0 0 mail@somewhere.com
02-01-2008 03:34:50 200 0 0
02-01-2008 03:35:23 201 0 0
02-01-2008 03:37:42 400 12 1 mail@somewhere.com

you can get that with
perl -0 -ne 'print "$3-$2-$1 $4 $5 $6 $8 $7\n" while /^\[(\d+)\.(\d+)\.(\d+)-([\d:]+).*\n.*=\[(.*)].*\n.*=\[(.*)].*\n(?:.*=\[(.*)\].*\n)?.*=\[(.*)\]/mg'

If you meant

02-01-2008 03:34:50 201 0 0 mail@somewhere.com
02-01-2008 03:34:50 200 0 0
02-01-2008 03:35:23 201 0 0 mail@somewhere.com
02-01-2008 03:37:42 400 12 1

Then I don't understand the relationship between that and the input.
 
01.04.2008 at 03:08AM PST, ID: 20581061
This is correct, you were right:
02-01-2008 03:34:50 201 0 0 mail@somewhere.com
02-01-2008 03:34:50 200 0 0
02-01-2008 03:35:23 201 0 0
02-01-2008 03:37:42 400 12 1 mail@somewhere.com

I think I should make the question more generic:

Logentries may look like this:

entry 1
[2008.01.02-03:34:50.172] aa.bb aa bb aa bb cc
header.function=[201]
header.completion=[0]
header.set=[400]
file.out.=[/tmp/$param]
data.email=[mail@somewhere.com]
header.error=[1]

entry 2
[2008.01.02-03:35:50.172] aa.bb aa bb #$ aa bb cc 2
data.email=[mail@somewhere.com]
header.function=[201]
header.completion=[0]
header.error=[0]

entry 3
[2008.01.02-03:36:50.172] aa.bb 1 aa bb aa bb cc 99 $adf

entry 4
[2008.01.02-03:37:50.172] aa $aa bb 0 1 aa.bb
header.function=[201]
header.completion=[0]

So the amount of lines per logentrie could be different and the order of the logentries could be different but the output should always be:
timestamp functioncode completioncode errorcode emailaddress

and email address only if it exists in the logentry, just blank if not filled.

but showing at least all logentries which have  a functioncode completioncode and errorcode. and dont show the logentries without these 3 entries...
 
01.04.2008 at 08:32AM PST, ID: 20583647

Rank: Genius

Here is a way...
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
my %data;
while(<DATA>) {
	if(/^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/) {
		PrintData();
		$data{year} = $1;
		$data{month} = $2;
		$data{day} = $3;
		$data{time} = $4;
	}
	elsif(/header\.function=\[(\d+)\]/) {
		$data{function} = $1;
	}
	elsif(/header\.completion=\[(\d+)\]/) {
		$data{completion} = $1;
	}
	elsif(/header\.error=\[(\d+)\]/) {
		$data{error} = $1;
	}
	elsif(/data\.email=\[(.+)\]/) {
		$data{email} = $1;
	}
}
PrintData();
 
sub PrintData {
	no warnings;
	print "$data{month}-$data{day}-$data{year} $data{time} $data{function} $data{completion} $data{error} $data{email}\n"
	  if (exists($data{function}) and exists($data{completion}) and exists($data{error}));
	%data=();
}
Open in New Window
 
01.04.2008 at 10:36AM PST, ID: 20584880

Rank: Genius

are the
"entry 4" lines and the blank lines part of the file?
if so,
perl -00 -ne 'print "$3-$2-$1 $4 $5 $6 $7 $8\n" if /(?=.*^\[(\d+)\.(\d+)\.(\d+)-([\d:]+))(?=.*header\.function=\[(.*?)])(?=.*header\.completion=\[(.*?)])(?=.*header.error=\[(.*?)\])(?=.*data.email=\[(.*?)])?/ms' file
 
01.06.2008 at 11:37PM PST, ID: 20597513
ozo, it works but processing a file which is only just over 3mb takes with your pel::

real    7m53.81s
user    7m53.18s
sys     0m0.06s

And with a self written ksh script, with a lot of if's, sed, awk etc... statements it takes:

real    2m28.15s
user    0m54.07s
sys     2m24.13s

I am always told perl is much faster? Do you understand these differences in cpu times?

Adam314, Im too new to perl to be able to get that to work for now, should I insert a open_file and a close_file function and put a #!/usr/bin/perl at the beginning?
 
01.06.2008 at 11:43PM PST, ID: 20597533
More info about the hardware, the script runs on a server being almost 100% idle with 4 sparcv9 cpu's running at 1593MHz and plenty of free memory.
 
01.07.2008 at 12:20AM PST, ID: 20597635

Rank: Genius

Does the ksh script work with the
[2008.01.02-03:34:50.172] aa.bb aa bb aa bb cc
in an arbitrary position?
Does it need to?
Adam314's doesn't
I thought you wanted a single line program,
if it can be a multiple line program,  you might try something like

#!/usr/bin/perl
$/="";
my @k=qw(function completion error);
while( <> ){
    my %h=/^header\.(\w+)=\[(.*?)]/mg;
    print "$3-$2-$1 $4 @h{@k} ",/^data\.email=\[(.+)]/m,"\n" if exists$h{function} && exists$h{completion} && exists$h{error} && /^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/m;
}


#this might be optimized slightly if the [2008.01.02-03:34:50.172] aa.bb aa bb aa bb cc line is always present or if some lines are less likely to be present than others.
 
01.07.2008 at 02:45AM PST, ID: 20598126
The program does not require to be one line, only the output should be one liners.

The shell script I wrote is about 100 lines, so I was quite impressed that you were able todo it with one line :-)

The date/timestamp entry always preceeds the log entries containing parameters.

If I save your script info a file i.e. showlog.pl should I then be able to use it like this?:

cat file | showlog.pl

All the lines I look for are the ones which are most likely to be present, I already filter out the lines I do not want before I parse it with perl/ksh and use that "clean" file as input file.
 
01.07.2008 at 03:12AM PST, ID: 20598203

Rank: Genius

cat file | showlog.pl
yes, if showlog.pl is executable ans in your path
showlog.pl file
would also work
otherwise
cat file | perl showlog.pl
or
perl showlog.pl file

If there are no lines that are not ones that you look for, then something like
my @k=qw(function completion error email);
while( <> ){
    my %h=/^\w+\.(\w+)=\[(.*?)]/mg;
    print "$3-$2-$1 @h{@k}\n" if keys %h+!$h{email} == 4 && /^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/m;
}
may be slightly faster
but that version may not work if you have extra lines like
header.set=[400]
 
01.07.2008 at 03:25AM PST, ID: 20598249

Rank: Genius

#!/usr/bin/perl
$/="";
my @k=qw(function completion error email);
while( <> ){
    my %h=/^\w+\.(\w+)=\[(.*?)]/mg,email=>'',set=>'';  #if you know what all the extra lines are, you might also do it this way
   print "$3-$2-$1 $4 @h{@k}\n" if keys%h==5 && /^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/m
}
 
01.07.2008 at 03:46AM PST, ID: 20598319
This produces only one output entry, where there should be much more?
1:
2:
3:
4:
5:
6:
7:
#!/usr/bin/perl
$/="";
my @k=qw(function completion error);
while( <> ){
    my %h=/^header\.(\w+)=\[(.*?)]/mg;
    print "$3-$2-$1 $4 @h{@k} ",/^data\.email=\[(.+)]/m,"\n" if exists$h{function} && exists$h{completion} && exists$h{error} && /^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/m;
}
Open in New Window
 
01.07.2008 at 03:56AM PST, ID: 20598360

Rank: Genius

can you give an example of some entries that it should produce entries for?
I'm assuming that there are blank lines between entries like in your last example
 
01.07.2008 at 06:32AM PST, ID: 20599638
The input remains the same, or I do not understand your question?:

[2008.01.02-03:34:50.172] aa.bb aa bb aa bb cc
header.function=[201]
header.completion=[0]
header.set=[400]
file.out.=[/tmp/$param]
data.email=[mail@somewhere.com]
header.error=[1]
[2008.01.02-03:35:50.172] aa.bb aa bb #$ aa bb cc 2
data.email=[mail@somewhere.com]
header.function=[201]
header.completion=[0]
header.error=[0]
[2008.01.02-03:36:50.172] aa.bb 1 aa bb aa bb cc 99 $adf
[2008.01.02-03:37:50.172] aa $aa bb 0 1 aa.bb
header.function=[201]
header.completion=[0]

So not for example:
[2008.01.02-03:36:50.172] aa.bb 1 aa bb aa bb cc 99 $adf
<empty line>
[2008.01.02-03:37:50.172] aa $aa bb 0 1 aa.bb
header.function=[201]
header.completion=[0]
 
01.07.2008 at 07:28AM PST, ID: 20600115

Rank: Genius

The script I gave will process your input.  It expectes the date/time line to be the first in a record, but the other fields can be in any order.   If there isn't a function, completion, and error field, it will not print the record.

The only change needed would be to have it read from standard input instead of the DATA handle.  Save it in a file, for example showlog.pl.

You can then call it like this:
    cat file | perl showlog.pl > output.txt

For your example data, it produces this output:
01-02-2008 03:34:50 201 0 1 mail@somewhere.com
01-02-2008 03:35:50 201 0 0 mail@somewhere.com
1:
2:
3:
4:
5:
6:
7:
8:
9:
10:
11:
12:
13:
14:
15:
16:
17:
18:
19:
20:
21:
22:
23:
24:
25:
26:
27:
28:
29:
30:
31:
32:
33:
34:
35:
#!/usr/bin/perl
use strict;
use warnings;
use Data::Dumper;
 
my %data;
while(<>) {
	if(/^\[(\d+)\.(\d+)\.(\d+)\-([\d:]+)/) {
		PrintData();
		$data{year} = $1;
		$data{month} = $2;
		$data{day} = $3;
		$data{time} = $4;
	}
	elsif(/header\.function=\[(\d+)\]/) {
		$data{function} = $1;
	}
	elsif(/header\.completion=\[(\d+)\]/) {
		$data{completion} = $1;
	}
	elsif(/header\.error=\[(\d+)\]/) {
		$data{error} = $1;
	}
	elsif(/data\.email=\[(.+)\]/) {
		$data{email} = $1;
	}
}
PrintData();
 
sub PrintData {
	no warnings;
	print "$data{month}-$data{day}-$data{year} $data{time} $data{function} $data{completion} $data{error} $data{email}\n"
	  if (exists($data{function}) and exists($data{completion}) and exists($data{error}));
	%data=();
}
Open in New Window
 
01.07.2008 at 09:57AM PST, ID: 20601373
Using adams script on a simple old linux pc with perl 5.6.1 and a 233MHz pentium cpu it processes a 3MB file in:

real    0m17.609s
user    0m17.110s
sys     0m0.320s

Now I will test this again tomorrow on the solaris server to prove if the difference is in the script code or in the OS difference, but the difference between almost 8 minutes and 17 seconds is impressive.

Ozo, your new code either produces no output or just one line in total while using the same input file as for testing adams script.
 
01.07.2008 at 10:08AM PST, ID: 20601468

Rank: Genius

If there are no blank lines between entries, but each entry always starts with a bracketed date:
#!/usr/bin/perl
$/="\n[";
my @k=qw(function completion error);
while( <> ){
    my %h=/^header\.(\w+)=\[(.*?)]/mg;
print "$3-$2-$1 $4 @h{@k} ",/^data\.email=\[(.+)]/m,"\n" if exists$h{function} && exists$h{completion} && exists$h{error} && /\[?(\d+)\.(\d+)\.(\d+)\-([\d:]+)/;
}
Accepted Solution
 
01.07.2008 at 11:05AM PST, ID: 20602056
# date && time perl showlog_adam314.pl file1 >out.file1 && date
Mon Jan  7 20:01:12 CET 2008
real    0m18.008s
user    0m17.130s
sys     0m0.650s
Mon Jan  7 20:01:30 CET 2008
# date && time perl showlog_ozo.pl file1 >out.file1 && date
Mon Jan  7 20:01:38 CET 2008
real    0m8.666s
user    0m8.160s
sys     0m0.430s
Mon Jan  7 20:01:47 CET 2008

Now I will test it with the real log on the real server in the morning and create a seperate question with another 500 point so both of you get the amount of points you deserve :-)

 
01.07.2008 at 12:37PM PST, ID: 20602974

Rank: Genius

There is no need to create a seperate question to give me points.  I have no problems if you accept the code ozo gave you.  
 
 
20080716-EE-VQP-33 / EE_QW_2_20070628