I need to generate a report from the rsync log on the server running the Daemon

Bassically I need to convert a large text file ever growing into some kind of useable report.  Additioally I would like to store the information in a data base.  Any Suggestions
vsimbcAsked:
Who is Participating?
 
mankowitzConnect With a Mentor Commented:
What format are you using for the file? The default log format is this:
    %o %h [%a] %m (%u) %f %l
    %i %h [%a] %m (%u) %f %l

You can use a perl script to put this into a db with something like this
while (<>)
{
    next unless ($day,$time,$op,$host,$module,$file,$bytes)
      = m#^ (\d+/\d\d/\d\d)\s+(\d\d:\d\d:\d\d)\s+\[\d+\]\s+(send|recv|[<>]f\S+)\s+
          (\S+)\s+\[\d+\.\d+\.\d+\.\d+\]\s+(\S+)\s+\(\S*\)\s+(.*)\s+(\d+) $ #x;
$sql = "INSERT INTO table (day, time, op, host, module, file, bytes) VALUES
             '$day','$time','$op','$host','$module','$file','$bytes')"
}

Your problem is that your report is continually growing, and you really want to insert stuff into the db in realtime, or at least in chunks. One solution is to rotate your log on a daily basis and simply process each chunk as it comes in. If you need finer granularity, you can rotate your log more often.
0
 
ai_ja_naiConnect With a Mentor Commented:
rotate your log every rsync usage adding a number on filename's tail with a bash script

!#/bin/bash
while (true) do
if [ -f /path/to/file ]; then wait 0;
else n$=$n+1;
fi
done
rsync blablaoptions --log-file=/path/to/file$n

This basically checks if a log with that tail number exists and searches for a valid number until it finds it and only at that point issues the rsync command with the logfile option enables.

Then, you may attach to the script some instructions like those my colegue pasted before
0
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.