Solved

Extract data from text file

Posted on 2014-03-06
14
421 Views
Last Modified: 2014-03-13
Hey Experts!  I need to check a log file on several servers daily.  Rather than checking them everyday, I was hoping I could run a script and dump out the most recent data.  Here is a brief sample of the data:

03/04/2014 19:07:45 --- SCHEDULEREC QUERY BEGIN
03/04/2014 19:07:45 --- SCHEDULEREC QUERY END
03/04/2014 19:07:45 Next operation scheduled:
03/04/2014 19:07:45 ------------------------------------------------------------
03/04/2014 19:07:45 Schedule Name:         DAILYINC_7PM
03/04/2014 19:07:45 Action:                Incremental
03/04/2014 19:07:45 Objects:              
03/04/2014 19:07:45 Options:              
03/04/2014 19:07:45 Server Window Start:   19:00:00 on 03/05/2014
03/04/2014 19:07:45 ------------------------------------------------------------
03/04/2014 19:07:45 Schedule will be refreshed in 12 hours.
03/05/2014 07:07:45 TSM Backup-Archive Client Version 6, Release 4, Level 0.11  
03/05/2014 07:07:45 Querying server for next scheduled event.
03/05/2014 07:07:45 Node Name: ---
03/05/2014 07:07:45 Session established with server ---: Linux/x86_64
03/05/2014 07:07:45   Server Version 6, Release 3, Level 4.0
03/05/2014 07:07:45   Server date/time: 03/05/2014 07:07:45  Last access: 03/04/2014 19:07:45

03/05/2014 07:07:45 --- SCHEDULEREC QUERY BEGIN
03/05/2014 07:07:45 --- SCHEDULEREC QUERY END
03/05/2014 07:07:45 Next operation scheduled:
03/05/2014 07:07:45 ------------------------------------------------------------
03/05/2014 07:07:45 Schedule Name:         DAILYINC_7PM
03/05/2014 07:07:45 Action:                Incremental
03/05/2014 07:07:45 Objects:              
03/05/2014 07:07:45 Options:              
03/05/2014 07:07:45 Server Window Start:   19:00:00 on 03/05/2014
03/05/2014 07:07:45 ------------------------------------------------------------
03/05/2014 07:07:45 Schedule will be refreshed in 12 hours.
03/05/2014 19:07:45 TSM Backup-Archive Client Version 6, Release 4, Level 0.11  
03/05/2014 19:07:45 Querying server for next scheduled event.
03/05/2014 19:07:45 Node Name: ----
03/05/2014 19:07:45 Session established with server ---: Linux/x86_64
03/05/2014 19:07:45   Server Version 6, Release 3, Level 4.0
03/05/2014 19:07:45   Server date/time: 03/05/2014 19:07:45  Last access: 03/05/2014 09:04:24

I would like the script to look for data with a certain date, set by me manually.  I need to copy the data found that matches the date (example 3/05/2014) to one, central log file.

So experts, how does one do this?  :)

Thanks in advance.
0
Comment
Question by:samiam41
  • 8
  • 3
  • 2
  • +1
14 Comments
 
LVL 28

Expert Comment

by:becraig
Comment Utility
Powershell
$value = "03/04/2014"
gc file.txt | select-string -pattern $value | out-file log-file.txt -append

Open in new window

#I added append if you were updating a log that you will continue to write to.
0
 
LVL 82

Expert Comment

by:oBdA
Comment Utility
Try something like this; you can pass the date either in the command line, or you'll be queried for it:
@echo off
setlocal
set LogFile=D:\Temp\test.log
set CentralLog=D:\Temp\test_Central.log
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
type "%LogFile%" | findstr.exe "^%FindDate%" >>"%CentralLog%"
if errorlevel 1 (
	echo Date not found!
	exit /b 1
)

Open in new window

Of course, the log file could be collected from several servers as well, while we're at it. Is the log file in the same path on all machines, or does the path differ?
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
becraig, thank you!  checking into it now.

oBdA, good to see you again.  There are several servers that have this log file and the log file is either on the C, D or E partition (sorry).  

Thanks for the help!
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
oBdA, if I run this from my pc, how/where would I specify the hostname of the other servers?
0
 
LVL 28

Assisted Solution

by:becraig
becraig earned 150 total points
Comment Utility
multiple servers

$report = @()
$logfile = c:\central-log.log
$value = "03/04/2014" 
#or we can even set this to today's date
$value = [string](get-date -Format MM/dd/yyyy)

gc serverlist.txt | % {
$report += gc \\$_\c$\directory\file.txt | select-string -pattern $value 
} 
$report | out-file $logfile  
#you can add -append if you need to add to an already present log file

Open in new window

0
 
LVL 82

Accepted Solution

by:
oBdA earned 350 total points
Comment Utility
Try it with this script; just add all log file locations after the [DATA] section:
@echo off
setlocal
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| find /i /n "[DATA]"') do set DataStart=%%a
set CentralLog=D:\Temp\test_Central.log
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	echo Processing '%%a' ...
	type "%%a" | findstr.exe "^%FindDate%" >>"%CentralLog%"
	if errorlevel 1 (
		echo Date not found in '%%a'.
	)
)
GOTO :EOF

[DATA]
D:\Temp\test.log
\\SomeServer\C$\Tivoli\tsm.log

Open in new window

0
 
LVL 51

Expert Comment

by:Bill Prew
Comment Utility
Here's a fairly simple approach if the names of the log files to be searched don't change day to day.  If you want to be prompted for the date then just use the logic for that already posted.
@echo off
setlocal

set CentralLog=Z:\temp\centrallog.txt
set LogFiles="C:\Temp\logfile1.txt" "D:\Temp\logfile1.txt" "E:\Temp\logfile1.txt"
set FindDate=01/01/2014

find "%FindDate%" %LogFiles% >> "%CentralLog%"

Open in new window

~bp
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 9

Author Comment

by:samiam41
Comment Utility
Hey BP!  Nice to see you again!

Can I run this script from my local pc and have it pull from a server?
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
@oBdA,

Here is the script I am using.  I only modified the entries below the DATA section.
@echo off
setlocal
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| find /i /n "[DATA]"') do set DataStart=%%a
set CentralLog=D:\Temp\test_Central.log
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	echo Processing '%%a' ...
	type "%%a" | findstr.exe "^%FindDate%" >>"%CentralLog%"
	if errorlevel 1 (
		echo Date not found in '%%a'.
	)
)
GOTO :EOF

[DATA]
\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log

Open in new window

I get the prompts for date and then get this error:

C:\Tools\Scripts>dsm_copy.bat
Enter date to search: 03/07/2014
Processing '\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log' ...
The system cannot find the path specified.

I tried using these in the DATA field and with two different servers (with and without the quotes).

"\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log"
\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
@BP,

I figured out I could build a batch script for each server and have it run at x time.  As long as the servers all have the drive mapped to the same network share and use z:, they can copy the necessary info.  Does that mean I need to be logged into the servers for this script to work?
0
 
LVL 82

Expert Comment

by:oBdA
Comment Utility
You don't need to put quotes around the path when you enter it in the [DATA] section; the script will take care of that.
Did you change line 11 (which could actually be moved to line 3) to the actual location of your central log file?
What happens when you enter dir "\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log" in a command prompt?
If you're running the script from Windows 7, did you start it from an elevated command prompt?
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
@oBdA, I found the error and it was on my side.  I never changed the "Set CentralLog" variable and with no D: drive, the script failed.  I made that change and it worked.  I did add the dir and the script wasn't happy so I took it out and it went back to working.  

Thank you sir!  Great to work with you again.  I want to test out the BP solution as well as the other one before closing out and awarding points.  I will close out by the end of today regardless.
0
 
LVL 9

Author Comment

by:samiam41
Comment Utility
**I am posting a follow up question to this if anyone wants to jump in while it is still "fresh" on your mind**

http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Batch/Q_28382929.html
0
 
LVL 9

Author Closing Comment

by:samiam41
Comment Utility
I appreciate everyone's time and assistance with this.  I took oBdA's code and am building off of it as it was the easiest (for me) to use and understand (I'm still a rook with scripting).  Thank you all and I hope to work with you again!

-Aaron
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

How to remove superseded packages in windows w60 or w61 installation media (.wim) or online system to prevent unnecessary space. w60 means Windows Vista or Windows Server 2008. w61 means Windows 7 or Windows Server 2008 R2. There are various …
Learn about cloud computing and its benefits for small business owners.
In this video, we discuss why the need for additional vertical screen space has become more important in recent years, namely, due to the transition in the marketplace of 4x3 computer screens to 16x9 and 16x10 screens (so-called widescreen format). …
This Micro Tutorial will give you a basic overview of Windows DVD Burner through its features and interface. This will be demonstrated using Windows 7 operating system.

743 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

20 Experts available now in Live!

Get 1:1 Help Now