Solved

Power shell script searching log files

Posted on 2013-06-27
11
428 Views
Last Modified: 2013-07-01
I have the following PS script that works except when something that is not a date is in the first field. I need to get by this error if anyone has any ideas on how to accomplish that.

$Date = (Get-Date -format yyyy-MM-dd)
ForEach($error in Get-Content C:\MonitorFolder\errorName.txt){
$error | % {
Select-String -path "C:\jboss-4.0.5.GA\server\default\log\server.log.2013-06-26" -Pattern $_ | where { (Get-Date((($_.Line).split("|")[0]).split(",")[0])) -gt (Get-Date).AddHours(-48) }
} | Out-File C:\MonitorFolder\ServerLogerrors.txt
$linesFound = (Get-Content C:\MonitorFolder\ErrorName.txt)
If ($linesFound.count -gt 0)
{Send-MailMessage -From 'Server_Log_Check@systrends.com' -To 'jimmy.lewis@systrends.com' `
  -SmtpServer 'smtp4.systrends.com' `
  -Subject $error' Testing  error Found on Border Reports server in server.log File (PS Script)' `
    -Body  @"
      Messages found in the last 30 minutes
      $(((Get-Content C:\MonitorFolder\Serverlogerrors.txt) | out-string) -join "`n")
"@
}}

The problem/exception occurs then the log file does not have the date as the first field.

2013-06-26 08:38:43,953 ERROR [STDERR]       at org.apache.tomcat.util.
2013-06-26 08:38:43,953 ERROR [STDERR]       at org.apache.tomcat.util.net.MasterSlav
2013-06-26 08:38:43,953 ERROR [STDERR]       at java.lang.Thread.run(Thread.java:619)
2013-06-26 08:38:43,953 ERROR [com.systrends.web.UserLogin] Password changing failed
javax.ejb.FinderException: java.sql.SQLException: No current row in the ResultSet.
      at com.systrends.postoffice.UsersEntityBean.ejbFindByLoginPassword(Unknown
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at

Throws the following error:

Get-Date : Cannot bind parameter 'Date'. Cannot convert value "javax.ejb.Finder
Exception: java.sql.SQLException: No current row in the ResultSet." to type "Sy
stem.DateTime". Error: "The string was not recognized as a valid DateTime. Ther
e is a unknown word starting at index 0."

It looks to me like the javax.ejb.FinderException being in the first row is causing the problem.

Any ideas on how to get around this problem?
0
Comment
Question by:jimmylew52
  • 6
  • 5
11 Comments
 
LVL 68

Expert Comment

by:Qlemo
Comment Utility
Sorry, but that script is a mess :( This is the least necessary change:
$Date = (Get-Date -format yyyy-MM-dd)
ForEach ($error in Get-Content C:\MonitorFolder\errorName.txt){
  $error | % {
    (Select-String -path "C:\jboss-4.0.5.GA\server\default\log\server.log.2013-06-26" -Pattern $_ ) -match '^\d{4}-\d{2}-\d{2}' |
      where { (Get-Date((($_.Line).split("|")[0]).split(",")[0])) -gt (Get-Date).AddHours(-48) }
  } | Out-File C:\MonitorFolder\ServerLogerrors.txt
  $linesFound = (Get-Content C:\MonitorFolder\ServerLogerrors.txt)
  If ($linesFound.count -gt 0)
  {
    Send-MailMessage -From 'Server_Log_Check@systrends.com' -To 'jimmy.lewis@systrends.com' `
      -SmtpServer 'smtp4.systrends.com' `
      -Subject $error' Testing  error Found on Border Reports server in server.log File (PS Script)' `
      -Body  @"
      Messages found in the last 30 minutes
      $(((Get-Content C:\MonitorFolder\Serverlogerrors.txt) | out-string) -join "`n")
"@
}}

Open in new window

BTW, line 7 was wrong (looking for the count of errorName.txt instead of ServerLogerrors.txt).
But even that will send a separate mail for each error category of errorName.txt, which is certainly not what you want.
0
 
LVL 1

Author Comment

by:jimmylew52
Comment Utility
I'm not supprised it is a mess. I am trying to learn this in my spare time, (not much of that).

Getting a different error:

You cannot call a method on a null-valued expression.
At C:\MonitorFolder\CheckServerLogFiles2.ps1:10 char:41
+       where { (Get-Date((($_.Line).split <<<< ("|")[0]).split(",")[0])) -gt (
Get-Date).AddHours(-48) }
    + CategoryInfo          : InvalidOperation: (split:String) [], RuntimeExce
   ption
    + FullyQualifiedErrorId : InvokeMethodOnNull
0
 
LVL 68

Expert Comment

by:Qlemo
Comment Utility
This one should be better:
$Date = (Get-Date -format yyyy-MM-dd)
ForEach ($error in Get-Content C:\MonitorFolder\errorName.txt)
{
  @(Select-String -path "C:\jboss-4.0.5.GA\server\default\log\server.log.2013-06-26" -Pattern $error) -match '^\d{4}-\d{2}-\d{2}' |
    where { (Get-Date $_.Line.split("|")[0].split(",")[0]) -gt (Get-Date).AddHours(-48) }
} | Out-File C:\MonitorFolder\ServerLogerrors.txt
$linesFound = Get-Content C:\MonitorFolder\ServerLogerrors.txt
If ($linesFound.count -gt 0)
{
  Send-MailMessage -From 'Server_Log_Check@systrends.com' -To 'jimmy.lewis@systrends.com' `
    -SmtpServer 'smtp4.systrends.com' `
    -Subject $error' Testing  error Found on Border Reports server in server.log File (PS Script)' `
    -Body  @"
      Messages found in the last 30 minutes
      $($linesFound -join "`n")
"@
}

Open in new window

0
 
LVL 1

Author Comment

by:jimmylew52
Comment Utility
Not as many errors but still throwing an error.

An empty pipe element is not allowed.
At C:\MonitorFolder\CheckServerLogFiles2.ps1:11 char:4
+ } | <<<<  Out-File C:\MonitorFolder\ServerLogerrors.txt
    + CategoryInfo          : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : EmptyPipeElement


I  have not seen the empty pipe error before.
0
 
LVL 68

Expert Comment

by:Qlemo
Comment Utility
Ok, that's my fault. foreach does not output anything into a pipeline. Anyway, I wanted to get rid of that loop, to allow for scanning the log file in one pass only, and here you go:
$logfile = 'C:\jboss-4.0.5.GA\server\default\log\server.log.2013-06-26'

$Date = Get-Date -format yyyy-MM-dd
@(Select-String -Path $logfile -SimpleMatch -Pattern (Get-Content C:\MonitorFolder\errorName.txt)) `
  -match '^\d{4}-\d{2}-\d{2}' |
  where { (Get-Date $_.Line.split("|")[0].split(",")[0]) -gt (Get-Date).AddHours(-48) } |
  Tee-Object -Variable linesFound | Out-File C:\MonitorFolder\ServerLogerrors.txt
If ($linesFound.count -gt 0)
{
  Send-MailMessage -From 'Server_Log_Check@systrends.com' -To 'jimmy.lewis@systrends.com' `
    -SmtpServer 'smtp4.systrends.com' `
    -Subject $error' Testing  error Found on Border Reports server in server.log File (PS Script)' `
    -Body  @"
      Messages found in the last 30 minutes
      $($linesFound -join "`n")
"@
}

Open in new window

I'm not sure you still need the output file for anything. If not, assigning the result of the long select-string line to a var (only) is much better, e.g. with
@(Select-String -Path $logfile -SimpleMatch -Pattern (Get-Content C:\MonitorFolder\errorName.txt)) `
  -match '^\d{4}-\d{2}-\d{2}' |
  where { (Get-Date $_.Line.split("|")[0].split(",")[0]) -gt (Get-Date).AddHours(-48) } |
  set-variable linesfound

Open in new window

BTW, I have switched Select-String to SimpleMatch because I assume you will not use regular expressions in your search pattern file.
0
Why You Should Analyze Threat Actor TTPs

After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

 
LVL 68

Expert Comment

by:Qlemo
Comment Utility
To elaborate on the empty pipe, try following examples:
foreach ($i in 1..3) { $i } 
foreach ($i in 1..3) { $i } | out-string
$(foreach ($i in 1..3) { $i }) | out-string

Open in new window

The latter builds a subexpression from foreach, and that always has a result. Somewhat strange, but important if you want to use IF, FOREACH and some other commands to return values instead of executing commands.
0
 
LVL 1

Author Comment

by:jimmylew52
Comment Utility
I understand most of the changes but still have a problem.

No errors are thrown but the error line in errorname.txt, that do exist in the log file, are not emailed to me. You script well beyond my abilities to  troubleshoot.
0
 
LVL 68

Expert Comment

by:Qlemo
Comment Utility
Did you try both versions of my code?
Could you post a small example (as files) for a log file and the errorname.txt?
0
 
LVL 1

Author Comment

by:jimmylew52
Comment Utility
Will get to this first thing this morning
0
 
LVL 1

Author Comment

by:jimmylew52
Comment Utility
Tried both versions this morning. The second version runs without errors but it does not send the email when it finds the error string.

Uploaded the requested files. Sorry for the delay, it has been very busy this morning.
errorName.txt
server.log.2013-06-26.txt
0
 
LVL 68

Accepted Solution

by:
Qlemo earned 500 total points
Comment Utility
None of the errors in the errorName.txt are in the log file, so nothing can be found.
In addition, you won't find something like javax.ejb.FinderException, as that doesn't have the timestamp prefixed. The Select-String lists that, but the -match eliminates anything without a timestamp prefix.
0

Featured Post

Threat Intelligence Starter Resources

Integrating threat intelligence can be challenging, and not all companies are ready. These resources can help you build awareness and prepare for defense.

Join & Write a Comment

Synchronize a new Active Directory domain with an existing Office 365 tenant
A procedure for exporting installed hotfix details of remote computers using powershell
Learn several ways to interact with files and get file information from the bash shell. ls lists the contents of a directory: Using the -a flag displays hidden files: Using the -l flag formats the output in a long list: The file command gives us mor…
When you create an app prototype with Adobe XD, you can insert system screens -- sharing or Control Center, for example -- with just a few clicks. This video shows you how. You can take the full course on Experts Exchange at http://bit.ly/XDcourse.

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now