Process log file with powershell.

Posted on 2011-05-10
Medium Priority
Last Modified: 2012-05-11
I have a log file which lists each package that was run by an ETL process. This is roughly in the format attached... What I need is the powershell logic to iterate through this file and tell me the starttime and endtime of each package so I can calculate how long each took. Between each of the package start/end statements are a lot of logging but for now I dont care about this. Any ideas?

If you could provide code for this example I think I could adapt this to my specific problem, unfortunately I am unable to post the complete logfile - but for those that are in the know it is a standard format ETL log.

PackageStart,ETLJOB,13/04/2011 09:00:04
   PackageStart,TASK1,13/04/2011 09:00:04
           PackageStart,TASK2,13/04/2011 09:00:04
           PackageEnd,TASK2,13/04/2011 09:00:04
   PackageEnd,TASK1,13/04/2011 09:00:04
   PackageStart,TASK3,13/04/2011 09:00:04
   PackageEnd,TASK3,13/04/2011 09:00:04
PackageEnd,ETLJOB,13/04/2011 09:00:04

Open in new window

Question by:Blowfelt82
LVL 13

Accepted Solution

soostibi earned 2000 total points
ID: 35728987
Here is my first try, update the path in line 2.
$taskdb = @{}
Get-Content C:\ee\package.txt | ?{$_.trim()}| %{
    $parts = $_.trim() -split ","
        $type = $parts[0]
        $name = $parts[1]
        $time = [datetime]::ParseExact($parts[2],"dd\/MM\/yyyy hh:mm:ss",$null)
    if($type -eq "Packagestart"){
        $taskdb.$name = @($time,$null,$null)
    elseif($type -eq "PackageEnd" -and $taskdb.$name){
$taskdb.keys | %{
    $taskdb.$_[2] = ($taskdb.$_[1] - $taskdb.$_[0]).totalmilliseconds
$taskdb.keys | %{
    New-Object -TypeName PSObject -Property @{
        Task = $_
        Start = $taskdb.$_[0]
        ElapsedTime = $taskdb.$_[2]
} | Sort-Object -Property start | Select-Object task, start, elapsedtime

Open in new window


Author Closing Comment

ID: 35736362
Thanks for the help, this answer was enough to set me on the right path.

Featured Post


Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

My attempt to use PowerShell and other great resources found online to simplify the deployment of Office 365 ProPlus client components to any workstation that needs it, regardless of existing Office components that may be needing attention.
There are times when we need to generate a report on the inbox rules, where users have set up forwarding externally in their mailbox. In this article, I will be sharing a script I wrote to generate the report in CSV format.
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an anti-spam), the admin…
Loops Section Overview

862 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question