asked on

Powershell Script to clean unwanted info from IIS log lines

I've created a powershell script to clean lines of code from IIS logs and copy the results to a new location like this (see Code)
So obviously I'm using a type command and passing a file name through an instance and using the same instance in the out-file to keep the file name the same.

this works fine to get the current date and open the file however each IIS folder I have (and there are four) have 250+ older files. I need to extract the full lines within the older files that contain certain IPs. I don't think I'm going to be able to use this little type command to do this. Can anyone help me with proper code?

I need to accomplish this:
Open a UNC path on another machine which contains all my log files, one at a time extract what I don't need (the entire line based on an IP found on that line) and then save the resulting file to a different location.

$date = Get-Date -format yyMMdd
$filename = "u_ex$date.log"
cd \
d:
cd iis1-w3svc1
(Type \\iisserver1\w3svc1\$filename) -notmatch "192.168.1.4" -notmatch "192.168.1.5" | out-file $filename
cd ..
cd iis1-w3svc2
(Type \\iisserver1\w3svc2\$filename) -notmatch "192.168.1.4" -notmatch "192.168.1.5" | out-file $filename
cd ..
cd iis2-w3svc1
(Type \\iisserver2\w3svc1\$filename) -notmatch "192.168.1.4" -notmatch "192.168.1.5" | out-file $filename
cd ..
cd iis2-w3svc2
(Type \\iisserver2\w3svc2\$filename) -notmatch "192.168.1.4" -notmatch "192.168.1.5" | out-file $filename

Open in new window

Rovastar

I wouldn't use POwershell for this I would use LogParser. It is design for things liek this.

http://www.microsoft.com/downloads/details.aspx?FamilyID=890cd06b-abf8-4c25-91b2-f8d975cf8c07&displaylang=en

http://forums.iis.net/default.aspx?GroupID=51

ThatVaiGuy

ASKER

Great, but even using logparser, what would the syntax look like?

Rovastar

Logparser just uses SQL to make the queries and you use teh feilds inside teh IIS logs

c-ip , for the client IP etc

I presume you know of the basics of SQL syntax.

Here is one query that will point you in the right direction.

http://weblogs.asp.net/steveschofield/archive/2008/02/28/logparser-look-for-certain-ip-s-between-a-timeframe.aspx

ALso there are some great examples in LOgParserLizard a GUI for logparser

http://www.lizard-labs.net/PageHtml.aspx?lng=2&PageId=18&PageListItemId=17

Rovastar

Also here are a few more examples

http://linuxlore.blogspot.com/2006/11/howto-use-microsofts-logparser-to.html

If you cannot get it from these examples. I'll post back with what you have got and I'll try and help further.

ThatVaiGuy

ASKER

Rovastar, I have to admit, I really wanted to do this in Powershell and absolutely hated you telling me to use something else. Now that I've taken a closer look at LogParser it really does seem to be the right tool to use to do ANYTHING with log files at all. I'd still like to use PS if I could, but I think this is going to meet my needs better than PS, less a PS expert comment on this thread an prove me wrong. Thanks for the suggestion, I'll comment back soon i'm sure with questions. I'm vaguely familiar with SQL syntax so I'm sure there will be questions. Thanks alot.

SOLUTION

Rovastar

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial

ThatVaiGuy

ASKER

Rovastar, you should learn powershell. Here is all the Syntax I needed to get this completed in Powershell:

Let's assume here that the log files are on a server called IISServer in the folder w3svc1, and I want to pull these two IP addresses: 192.168.1.1 and 192.168.1.2 and I want to save the results to c:\w3svctest\{filename}

See the attached code snippet. It's really simple.

$files = Get-ChildItem \\IISServer\w3svc1 -name
Foreach ($file in $files)
{
(type \\iisserver\w3svc1\$file) -notwith "192.168.1.1" -notwith "192.168.1.2" | out-file c:\w3svctest\$file
}

Open in new window

ASKER CERTIFIED SOLUTION

ThatVaiGuy

membership

This solution is only available to members.

To access this solution, you must be a member of Experts Exchange.

Start Free Trial