We help IT Professionals succeed at work.

Script to extract log from many SMTP log

cawasaki
cawasaki asked
on
648 Views
Last Modified: 2012-05-10
Hi,

I need a script powershel or other to extract from my HUB exchange 2007 SmtpReceive LOG all line containt this:

"421 4.3.2 Service not available, closing transmission channel"

I have many file log in same folder, the script must go through all file.

Thanks for your help
Comment
Watch Question

Tomas ValentaIT Manager

Commented:
use in script
for /f "tokens=*" %%I in (dir target_folder /b /A-D ) do FIND /I "421 4.3.2 Service not available, closing transmission channel" %%I >> logfile.txt

Author

Commented:
hi Tominov,

Its a script? can you explain your script and how i can use it?

Thanks
Tomas ValentaIT Manager

Commented:
create batch file - smtplog.cmd with content:
for /f "tokens=*" %%I in (dir target_folder /b /A-D ) do FIND /I "421 4.3.2 Service not available, closing transmission channel" %%I >> logfile.txt

You must type correct target_ folder place and place for result file logfile.txt.
Explanation of the script:
for every line in output from command in bracket - dir..... - is done command FIND which print the whole line
where find the requested string - this output is directed to the file logfile.txt - line by line.

Author

Commented:
HUm, but i have many different log file and i need to find "421 4.3.2 Service not available, closing transmission channel" in all logs file at same times, your script can do this?

thanks
Tomas ValentaIT Manager

Commented:
this script go throw all files in directory and find in all files the requested string.
Do you have the log files in the same directory ?

Commented:
PowerShell solution:
dir *.log | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) | ft filename,linenumber, line

Open in new window

Author

Commented:
yes all my smtp log is in same directory at e:\smtplog

I need add the directory to the script or just lunch the script in directory?

thanks

Author

Commented:
hi soostibi,

your script list just a name of file, i need a script to export all line contain 421 4.3.2 Service not available, closing transmission channel
Tomas ValentaIT Manager

Commented:
if you launch it in this directory then you don't need to fill whole path.
But better is to fill full path. I correct my mistake in command - please copy and paste the following
part to your cmd file:
for /f "tokens=*" %%I in ('dir e:\smtplog /b /A-D') do FIND /I "421 4.3.2 Service not available, closing transmission channel" e:\smtplog\%%I >> e:\logfile.txt
The result file always save in another location.

Author

Commented:
Tominov

your script work perfectly, it is possible to add option to this script t export logfile with date, for exemple:

lofile09_09_10, because i need to schedule this script every 2 days.

Thanks

Commented:
No, my script extracts the filename, the linenumber and the line containing the given text. I updated your path in the script and left the formatting part, so now you'll see only the lines.

If you add this, you'll also see the other data:
| ft line, filename, linenumber
dir e:\smtplog\log | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel"))

Open in new window

Commented:
Sorry the * is missing, so the corrected one:
dir e:\smtplog\*log | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel"))

Open in new window

Tomas ValentaIT Manager

Commented:
Yes, I can, but be aware: in every export file will be the same data like in previous version + new data. It is really what you need ?

Commented:
This exports the data to yourpath\yourfile:
dir e:\smtplog\log | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) | select-object line, filename, linenumber | out-file e:\yourpath\yourfile.txt -append

Open in new window

Commented:
Sorry, the * disappeared again, so:
dir e:\smtplog\*log | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) | 
select-object line, filename, linenumber | out-file e:\yourpath\yourfile.txt -append

Open in new window

Tomas ValentaIT Manager

Commented:
New variant of your script include generation of export filename based on today's date.
The first line create variable currentdate based on current date (this must be done because
output from command date depend on international setting in operating system and one
of the varant is day/month/year and "/" is not allowed character in name of the file or directory.

for /f "tokens=2-4 delims=/ " %%i in ('date /t') do set currentdate=%%i-%%j-%%k
for /f "tokens=*" %%I in ('dir e:\smtplog /b /A-D') do FIND /I "421 4.3.2 Service not available, closing transmission channel" e:\smtplog\%%I >> e:\LOG-%currentdate%.txt

Author

Commented:
Hi,

tominov: Yes, I can, but be aware: in every export file will be the same data like in previous version + new data. It is really what you need ?

==> Yes

soostibi:

==> I will test your script

Author

Commented:
soostibi:

i just need to export all line in a txt or csv file not line number.

If the script find a line contain 421 4.3.2 Service not available, closing transmission channel, it copy all line to exported file.

thanks

Author

Commented:
soostibi:

Your script create a file like this: LOG-09-2010-.txt.

It is possible to add the day? because if i need to lunch a script every day

thanks

Commented:
This dumps all the filenames and line that include the given expression into a csv. If you run this from day to day, the new CSVs only include data from the logs, that were created after the previous CSV file. The name of the CSV file is like LOG-08-09-2010.csv.
Is that what you want?
$startdate = (Get-ChildItem e:\smtplog\*.csv | Sort-Object -Property lastwritetime | Select-Object -First 1).lastwritetime
dir e:\smtplog\*.log | ?{$_.lastwritetime -gt $startdate} | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) | 
	Select-Object filename, line | Export-Csv -Path "e:\smtplog\LOG-$(get-date -f 'dd-mm-yyy').log"

Open in new window

Author

Commented:
Tominov,


Your script create a file like this: LOG-09-2010-.txt.

It is possible to add the day? because if i need to lunch a script every day

thanks
Dale HarrisProfessional Services Engineer
CERTIFIED EXPERT

Commented:
Soostibi,

If the above script is executed, the 'dd-mm-yyy' will actually return day - minute - year.  Your best bet is:
(get-date).toshortdatestring

Which will return 9/9/2010 if it's ran today.

Regards,

Dale Harris
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

Author

Commented:
Soostibi:

This dumps all the filenames and line that include the given expression into a csv. If you run this from day to day, the new CSVs only include data from the logs, that were created after the previous CSV file. The name of the CSV file is like LOG-08-09-2010.csv.
Is that what you want

==>i need jut line that include given expression un my csv, not need filename.
and ok for the rest

I have error when i execute your powershellscript

Commented:
What error do you get? (I took the filename out of the result)
$startdate = (Get-ChildItem e:\smtplog\*.csv | Sort-Object -Property lastwritetime | Select-Object -First 1).lastwritetime    
dir e:\smtplog\*.log | ?{$_.lastwritetime -gt $startdate} | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) |     
        Select-Object line | Export-Csv -Path "e:\smtplog\LOG-$(get-date -f 'dd-MM-yyy').log"

Open in new window

Author

Commented:
here error:


[PS] C:\Windows\System32>$startdate = (Get-ChildItem e:\test\*.csv | Sort-Object
 -Property lastwritetime | Select-Object -First 1).lastwritetime dir c:\test\*.l
og | ?{$_.lastwritetime -gt $startdate} | Select-String ([regex]::escape("421 4.
3.2 Service not available, closing transmission channel")) Select-Object line |
Export-Csv -Path "c:\test\LOG-$(get-date -f 'dd-MM-yyy').log"

Unexpected token 'dir' in expression or statement.
At line:1 char:124
+ $startdate = (Get-ChildItem e:\test\*.csv | Sort-Object -Property lastwriteti
me | Select-Object -First 1).lastwritetime dir  <<<< c:\test\*.log | ?{$_.lastw
ritetime -gt $startdate} | Select-String ([regex]::escape("421 4.3.2 Service no
t available, closing transmission channel")) Select-Object line | Export-Csv -P
ath "c:\test\LOG-$(get-date -f 'dd-MM-yyy').log"

Commented:
The newline is missing from the code you runs. There should be a new line before "dir", after lastwritetime.

Author

Commented:
ok i get it work but:

1- the output file is empty, it only list the line in powershell mmc.

2- the script list file name, and i not need it, i need jut the entire line that include given expression in my csv, not need filename.

thanks for your help

Commented:
I think you also missed the pipe character (|) from the following place:
... channel")) | Select-Object ...
(It is hard to help, if you do not copy the exact code...)

Author

Commented:
i will test this tomorow
Tomas ValentaIT Manager

Commented:
What is output from command date /t ?

Author

Commented:
the output of date/t:

10/09/2010
Tomas ValentaIT Manager

Commented:
Yeah. My output on english version Windows 2003 server is Fri 09/10/2010.
Correction is only in first tokens parameter. tokens parameter is used for choosing
which values command retrieve - delimiters between valueses is in delim parameter (there
is / and space. Bellow is full script which will work in your international setting. If you move this
script to another system simply type date/t and based on result you only change tokens and/or
delims parameter of the first FOR command.

for /f "tokens=* delims=/ " %%i in ('date /t') do set currentdate=%%i-%%j-%%k
for /f "tokens=*" %%I in ('dir e:\smtplog /b /A-D') do FIND /I "421 4.3.2 Service not available, closing transmission channel" e:\smtplog\%%I >> e:\LOG-%currentdate%.txt

Author

Commented:
soostibi:

Your script work now, but i have an error when i lunsh the script:

Select-String : The file can not be read: C:\test\LOG-10-09-2010.log
At C:\test\PLIP.ps1:2 char:71
+ dir c:\test\*.log | ?{$_.lastwritetime -gt $startdate} | Select-String  <<<<
([regex]::escape("421 4.3.2 Service not available, closing transmission channel
")) |

Author

Commented:
Tominov:

Not work with new script:

C:\test>FIND /I "421 4.3.2 Service not available, closing transmission channel"
c:\test\test3.bat  1>>e:\test\LOG-10/09/2010
The system cannot find the path specified.
Tomas ValentaIT Manager
Commented:
Unlock this solution and get a sample of our free trial.
(No credit card required)
UNLOCK SOLUTION

Commented:
Can you open the C:\test\LOG-10-09-2010.log file with notepad? Maybe it is locked by some other processes...

Author

Commented:
Tominov:

Its work now, its possible to delete this part from output file:

---------- C:\TEST\LOG.LOG
2010-09-08T22:06:31.121Z,,08CD09A6F872EB85,1,10.200.4.228:25,10.33.22.21:50174,>,"421 4.3.2 Service not available, closing transmission channel",

---------- C:\TEST\PLIP.PS1
dir c:\test\*.log | ?{$_.lastwritetime -gt $startdate} | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel")) |    

---------- C:\TEST\PLOP.PS1
dir c:\test\*.log | ?{$_.lastwritetime -gt $startdate} | Select-String ([regex]::escape("421 4.3.2 Service not available, closing transmission channel"))

---------- C:\TEST\TEST.BAT

---------- C:\TEST\TEST2.BAT
for /f "tokens=*" %%I in ('dir c:\test /b /A-D') do FIND /I "421 4.3.2 Service not available, closing transmission channel" c:\test\%%I >> c:\test\LOG-%currentdate%.txt

---------- C:\TEST\TEST3.BAT
for /f "tokens=*" %%I in ('dir c:\test /b /A-D') do FIND /I "421 4.3.2 Service not available, closing transmission channel" c:\test\%%I >> C:\test\LOG-%currentdate%.txt

Author

Commented:
Soostibi:

Can you open the C:\test\LOG-10-09-2010.log file with notepad? Maybe it is locked by some other processes...

==>LOG-10-09-2010.log is the output file of your script!
Tomas ValentaIT Manager

Commented:
it is easy - you must have logfile and script saved in another location then smtplogs. Thats all.

Author

Commented:
Ok perfect,

The 2 script is OK.

Thanks tominov and Soostibi.
Unlock the solution to this question.
Thanks for using Experts Exchange.

Please provide your email to receive a sample view!

*This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

OR

Please enter a first name

Please enter a last name

8+ characters (letters, numbers, and a symbol)

By clicking, you agree to the Terms of Use and Privacy Policy.