Solved

Extract data from text file

Posted on 2014-03-06
14
432 Views
Last Modified: 2014-03-13
Hey Experts!  I need to check a log file on several servers daily.  Rather than checking them everyday, I was hoping I could run a script and dump out the most recent data.  Here is a brief sample of the data:

03/04/2014 19:07:45 --- SCHEDULEREC QUERY BEGIN
03/04/2014 19:07:45 --- SCHEDULEREC QUERY END
03/04/2014 19:07:45 Next operation scheduled:
03/04/2014 19:07:45 ------------------------------------------------------------
03/04/2014 19:07:45 Schedule Name:         DAILYINC_7PM
03/04/2014 19:07:45 Action:                Incremental
03/04/2014 19:07:45 Objects:              
03/04/2014 19:07:45 Options:              
03/04/2014 19:07:45 Server Window Start:   19:00:00 on 03/05/2014
03/04/2014 19:07:45 ------------------------------------------------------------
03/04/2014 19:07:45 Schedule will be refreshed in 12 hours.
03/05/2014 07:07:45 TSM Backup-Archive Client Version 6, Release 4, Level 0.11  
03/05/2014 07:07:45 Querying server for next scheduled event.
03/05/2014 07:07:45 Node Name: ---
03/05/2014 07:07:45 Session established with server ---: Linux/x86_64
03/05/2014 07:07:45   Server Version 6, Release 3, Level 4.0
03/05/2014 07:07:45   Server date/time: 03/05/2014 07:07:45  Last access: 03/04/2014 19:07:45

03/05/2014 07:07:45 --- SCHEDULEREC QUERY BEGIN
03/05/2014 07:07:45 --- SCHEDULEREC QUERY END
03/05/2014 07:07:45 Next operation scheduled:
03/05/2014 07:07:45 ------------------------------------------------------------
03/05/2014 07:07:45 Schedule Name:         DAILYINC_7PM
03/05/2014 07:07:45 Action:                Incremental
03/05/2014 07:07:45 Objects:              
03/05/2014 07:07:45 Options:              
03/05/2014 07:07:45 Server Window Start:   19:00:00 on 03/05/2014
03/05/2014 07:07:45 ------------------------------------------------------------
03/05/2014 07:07:45 Schedule will be refreshed in 12 hours.
03/05/2014 19:07:45 TSM Backup-Archive Client Version 6, Release 4, Level 0.11  
03/05/2014 19:07:45 Querying server for next scheduled event.
03/05/2014 19:07:45 Node Name: ----
03/05/2014 19:07:45 Session established with server ---: Linux/x86_64
03/05/2014 19:07:45   Server Version 6, Release 3, Level 4.0
03/05/2014 19:07:45   Server date/time: 03/05/2014 19:07:45  Last access: 03/05/2014 09:04:24

I would like the script to look for data with a certain date, set by me manually.  I need to copy the data found that matches the date (example 3/05/2014) to one, central log file.

So experts, how does one do this?  :)

Thanks in advance.
0
Comment
Question by:samiam41
  • 8
  • 3
  • 2
  • +1
14 Comments
 
LVL 29

Expert Comment

by:becraig
ID: 39910470
Powershell
$value = "03/04/2014"
gc file.txt | select-string -pattern $value | out-file log-file.txt -append

Open in new window

#I added append if you were updating a log that you will continue to write to.
0
 
LVL 84

Expert Comment

by:oBdA
ID: 39910501
Try something like this; you can pass the date either in the command line, or you'll be queried for it:
@echo off
setlocal
set LogFile=D:\Temp\test.log
set CentralLog=D:\Temp\test_Central.log
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
type "%LogFile%" | findstr.exe "^%FindDate%" >>"%CentralLog%"
if errorlevel 1 (
	echo Date not found!
	exit /b 1
)

Open in new window

Of course, the log file could be collected from several servers as well, while we're at it. Is the log file in the same path on all machines, or does the path differ?
0
 
LVL 9

Author Comment

by:samiam41
ID: 39910596
becraig, thank you!  checking into it now.

oBdA, good to see you again.  There are several servers that have this log file and the log file is either on the C, D or E partition (sorry).  

Thanks for the help!
0
U.S. Department of Agriculture and Acronis Access

With the new era of mobile computing, smartphones and tablets, wireless communications and cloud services, the USDA sought to take advantage of a mobilized workforce and the blurring lines between personal and corporate computing resources.

 
LVL 9

Author Comment

by:samiam41
ID: 39910616
oBdA, if I run this from my pc, how/where would I specify the hostname of the other servers?
0
 
LVL 29

Assisted Solution

by:becraig
becraig earned 150 total points
ID: 39910669
multiple servers

$report = @()
$logfile = c:\central-log.log
$value = "03/04/2014" 
#or we can even set this to today's date
$value = [string](get-date -Format MM/dd/yyyy)

gc serverlist.txt | % {
$report += gc \\$_\c$\directory\file.txt | select-string -pattern $value 
} 
$report | out-file $logfile  
#you can add -append if you need to add to an already present log file

Open in new window

0
 
LVL 84

Accepted Solution

by:
oBdA earned 350 total points
ID: 39910674
Try it with this script; just add all log file locations after the [DATA] section:
@echo off
setlocal
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| find /i /n "[DATA]"') do set DataStart=%%a
set CentralLog=D:\Temp\test_Central.log
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	echo Processing '%%a' ...
	type "%%a" | findstr.exe "^%FindDate%" >>"%CentralLog%"
	if errorlevel 1 (
		echo Date not found in '%%a'.
	)
)
GOTO :EOF

[DATA]
D:\Temp\test.log
\\SomeServer\C$\Tivoli\tsm.log

Open in new window

0
 
LVL 54

Expert Comment

by:Bill Prew
ID: 39911322
Here's a fairly simple approach if the names of the log files to be searched don't change day to day.  If you want to be prompted for the date then just use the logic for that already posted.
@echo off
setlocal

set CentralLog=Z:\temp\centrallog.txt
set LogFiles="C:\Temp\logfile1.txt" "D:\Temp\logfile1.txt" "E:\Temp\logfile1.txt"
set FindDate=01/01/2014

find "%FindDate%" %LogFiles% >> "%CentralLog%"

Open in new window

~bp
0
 
LVL 9

Author Comment

by:samiam41
ID: 39912505
Hey BP!  Nice to see you again!

Can I run this script from my local pc and have it pull from a server?
0
 
LVL 9

Author Comment

by:samiam41
ID: 39912629
@oBdA,

Here is the script I am using.  I only modified the entries below the DATA section.
@echo off
setlocal
if "%~1"=="" (
	set FindDate=
	set /p FindDate=Enter date to search: 
) else (
	set FindDate=%~1
)
if "%FindDate%"=="" goto :eof
for /f "tokens=1 delims=[]" %%a in ('type "%~f0" ^| find /i /n "[DATA]"') do set DataStart=%%a
set CentralLog=D:\Temp\test_Central.log
for /f "skip=%DataStart% delims=" %%a in ('type "%~f0"') do (
	echo Processing '%%a' ...
	type "%%a" | findstr.exe "^%FindDate%" >>"%CentralLog%"
	if errorlevel 1 (
		echo Date not found in '%%a'.
	)
)
GOTO :EOF

[DATA]
\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log

Open in new window

I get the prompts for date and then get this error:

C:\Tools\Scripts>dsm_copy.bat
Enter date to search: 03/07/2014
Processing '\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log' ...
The system cannot find the path specified.

I tried using these in the DATA field and with two different servers (with and without the quotes).

"\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log"
\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log
0
 
LVL 9

Author Comment

by:samiam41
ID: 39912695
@BP,

I figured out I could build a batch script for each server and have it run at x time.  As long as the servers all have the drive mapped to the same network share and use z:, they can copy the necessary info.  Does that mean I need to be logged into the servers for this script to work?
0
 
LVL 84

Expert Comment

by:oBdA
ID: 39912700
You don't need to put quotes around the path when you enter it in the [DATA] section; the script will take care of that.
Did you change line 11 (which could actually be moved to line 3) to the actual location of your central log file?
What happens when you enter dir "\\w2k3ps01\d$\Program Files\Tivoli\TSM\baclient\dsmsched.log" in a command prompt?
If you're running the script from Windows 7, did you start it from an elevated command prompt?
0
 
LVL 9

Author Comment

by:samiam41
ID: 39912726
@oBdA, I found the error and it was on my side.  I never changed the "Set CentralLog" variable and with no D: drive, the script failed.  I made that change and it worked.  I did add the dir and the script wasn't happy so I took it out and it went back to working.  

Thank you sir!  Great to work with you again.  I want to test out the BP solution as well as the other one before closing out and awarding points.  I will close out by the end of today regardless.
0
 
LVL 9

Author Comment

by:samiam41
ID: 39912767
**I am posting a follow up question to this if anyone wants to jump in while it is still "fresh" on your mind**

http://www.experts-exchange.com/Programming/Languages/Scripting/Shell/Batch/Q_28382929.html
0
 
LVL 9

Author Closing Comment

by:samiam41
ID: 39926149
I appreciate everyone's time and assistance with this.  I took oBdA's code and am building off of it as it was the easiest (for me) to use and understand (I'm still a rook with scripting).  Thank you all and I hope to work with you again!

-Aaron
0

Featured Post

PRTG Network Monitor: Intuitive Network Monitoring

Network Monitoring is essential to ensure that computer systems and network devices are running. Use PRTG to monitor LANs, servers, websites, applications and devices, bandwidth, virtual environments, remote systems, IoT, and many more. PRTG is easy to set up & use.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Use this article to create a batch file to backup a Microsoft SQL Server database to a Windows folder.  The folder can be on the local hard drive or on a network share.  This batch file will query the SQL server to get the current date & time and wi…
Learn about cloud computing and its benefits for small business owners.
This Micro Tutorial will teach you the basics of configuring your computer to improve its speed. It will also teach you how to disable programs that are running in the background simultaneously. This will be demonstrated using Windows 7 operating…
This Micro Tutorial will teach you how to the overview of Microsoft Security Essentials. This is a free anti-virus software that guards your PC against viruses, spyware, worms, and other malicious software. This will be demonstrated using Windows…

820 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question