Solved

Changing the output file of a process that is already executing KSH

Posted on 2003-12-10
9
920 Views
Last Modified: 2013-12-26
Hello all,

I have a questions regarding changing the output file name a process that is already started.

In other words, I am starting my process in in an indefinate loop, then every 24 hours,
I would like to change the output file name. I will show you my script, and maybe that will help:

Script
===========================
loop="1"; _loop=$(( ${loop} \* 60 ))
log_days=$(grep log_days ${ini}|cut -f2 -d\=)
cnt=0
while [ 1 ]; do
# Loop vars
cdate=$(date +'%Y%m%d')
chour=$(date +'%H')
 
# Clean up old files
log_days=$(grep log_days matt.ini|cut -f2 -d\=)
find . -name "ud_chk.KSH*" -mtime+${log_days} -exec rm -f {} \;
 
# Set log file name
case ${chour} in
 0[0]) if [ ${cnt} = 0 ]; then
           out_file="ud_chk.KSH.${cdate}.$$.log.0"
          elif ${cnt} > 0 ]; then
           cur_file=$(ls -t ud_chk.KSH*|head -n1)
           file_date=$(echo ${cur_file}|cut -f3 -d\.)
           file_cnt=$(echo ${cur_file}|cut -f6 -d\.)
           new_file_cnt=$(( ${file_cnt} \+ 1 ))
          if [ ${file_date} != ${cdate} ]; then
            print "\nCurrent log file is: ${out_file}>>${out_file}"
            print "The new log file is: ud_chk.KSH.${cdate}.$$.log.${new_file_cnt}">>${out_file}
            out_file="ud_chk.KSH.${cdate}.$$.log.${new_file_cnt}"
          fi
         fi ;;
    *) if [ ${cnt} = 0 ]; then
          out_file="ud_chk.KSH.${cdate}.$$.log.0"
         fi ;;
esac
ctime=$(date +'%H%M')
 case ${ctime} in
  #0[0-1][4][0]) # Check Corp user daily (perform all other checks also) ;;
  #0[3-4][1][0]) # Check REGNxx user dalies (perform all other checks also) ;;
    0[7][5][0-5]) print '\n$ctime is currently between 0750 & 0755'
                        print "ctime=${ctime}" ;;
    0[7][5][5-9]) print '\n$ctime is currently between 0755 & 0759'
                        print "ctime=${ctime}" ;;
                    *) if [ ${cnt} = 0 ]; then
                          write_cnt>${out_file} 2>&1
                        elif [ ${cnt} > 0 ]; then
                          print "This is round ${cnt} in the loop"; sleep 5
                        fi ;;
 esac
  (( cnt+=1 ))
done

===========================
Script

So, obviously, if the script is first starting, ( $cnt=0 ), then the log file will be:
out_file="ud_chk.KSH.${cdate}.$$.log.0"

However, if the program ( write_cnt is the function being executed and can be located
near the end of my script ) is already running, then I am checking $cdate to see if a new date has arrived
and if it has, then I want to change the log file name to reflect the new date, and be like
Original name:
ud_chk.KSH.20031210.12345.log.0
New name:
ud_chk.KSH.20031211.12345.log.1

So, I want to change the date and increment the last digit of the log file name by one each time it is changed.
I believe that you will agree that my script will perform all of the above requirements of changing the file name,
but I do not know how to force write_cnt() to use the new log file without stopping and restarting it.

Also, as you can see, I wish to print the old log file name and the new name into the existing log file before I change it. That way if you are looking at one of the log files and you reach the end, then you will know what the name of the next log file is that you need to look at.

If this dosn't make any sense to, then I will be happy to explain further.

Thanks for your time,
-Matt V
0
Comment
Question by:j8vy
  • 2
  • 2
  • 2
  • +3
9 Comments
 
LVL 9

Expert Comment

by:HamdyHassan
ID: 9915367
Hi

What this means
"how to force write_cnt() to use the new log file without stopping and restarting it"

If your process is running and writing to a log file, then you need to stop writing , opening another log file, start writing to

e.g.
you open a word document, and while writing to it, somebody else want to rename it, It won't work


Could you write in English the requirment just the logic?


0
 

Author Comment

by:j8vy
ID: 9915980
Ok, I'll just laugh it off .... but I am from Mississippi, and I am fairly sure that my primary language is English ... LOL LOL :) (Just Spanish on the side)

Anyway,

The loop will start:

cnt=0
while [ 1 ]; do
<some code here>
  Now we will execute the write_cnt() function and place its output into a file that will be $out_file
out_file="/path/to/out_file.date.pid.0"
   write_cnt>$out_file 2>&1 &
 (( cnt+=1 ))
done

Now as the loop executes, if the execution time passes a 24 hour period
(which it most defiantely will be running more than twenty four hours)
and I will determine this by placing write_cnt() into the background and continuing
the loop checking the current hour until a new day is reached. When this new day is reached
(preferably at 0000 or midnight you might say), I want to change the value of $out_file from
the original
out_file="/path/to/out_file.date.pid.0"
to
out_file="/path/to/out_file.date.pid.1"
write_cnt>$out_file 2>&1 &

so that now the value of $out_file has been changed and so on and so forth until the execution
stops or is stopped. What I want to do is force the process that I placed in the background (write_cnt)
to use the new value of $out_file.

I wanted to do this without stopping write_cnt and having to restart it, but if I need to stop it's execution
and restart it for it to use the new $out_file, then I will just have to do that.


Sorry for being unclear at first :)

-Matt
0
 
LVL 1

Expert Comment

by:mbekker
ID: 9919568
Hi Matt,

You could try one of the following, if it's possible:

- create a FIFO file with the same name as the output file
- start the program which writes to the FIFO file
- from another script, read from the FIFO file and redirect output
  to $outfile in the background (cat <file >$outfile &)
- after a while, kill the background process (e.g. kill %+)
- adjust the outfile variable and start the cat command again.

Another way is to copy the file to $fileout and empty the original one:

- cp -p file $outfile
- > file

There are two problems with the last solution. Between the cp command and '>' there could be something written to the outputfile. This output will be lost... If there's a moment where no output is generated it could be no problem.
The second problem could be not freeing the space in the filesystem from the original file. You have to check it with a 'df' before and after the cp and '>'. It depends on how the file is treated by the program.

Good luck!
0
 
LVL 51

Accepted Solution

by:
ahoffmann earned 350 total points
ID: 9931054
> .. I do not know how to force write_cnt() to use the new log file
write_cnt() needs to know how to do it.
For example most programs do that if they receive SIG_HUP signal.

Keep in mind that write_cnt() still writes to the file even if you remove it.
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 2

Expert Comment

by:zdes
ID: 9970476
Your standard output goes to file descriptor 1, stderr - to 2. To redirect it globally, run the following command:

exec 1> mylog.`date +%y%m%d-%H%M%S` 2>&1

just do it every 24 hours. In the example above the name of the file just contains date. This is just an example. You seem to have an elaborate rule about the file name.

also, you might want to flush stdout and err just before you redirect them by closing the descriptors:

exec 1>&- 2>&-

however, if you run some background processes, you might lose some output between the times you close the descriptors and open new again.
0
 

Expert Comment

by:harshalhayat
ID: 10010079
Matt,

Check out symbolic links (symlinks) and use them safely.
It is the best approach I feel.
Say

"a.out > a.stdout.${DATE} 2> a.stderr.${DATE}" can be changed with -
"a.out > a.stdout.link 2> a.stderr.link"
"ln -sf a.stdout.${DATE} a.stdout.link"
"ln -sf a.stderr.${DATE} a.stderr.link"

    - on every date change.

Harshal
0
 
LVL 1

Expert Comment

by:mbekker
ID: 10015035
Harshal,

The symbolic link trick won't always work.

If a process opens a file, the process gets a file descriptor. The file descriptor refers to the file via an inode or vnode or whatever... When a symbolic link is created with the same name, the inode will be different but the process still writes to the original inode. Since the inode is "gone", all data written to the file is lost.

I've put "" around gone because the inode still exists in the filesystem until the process closes the file. The inode doesn't have any filename which refers to it so the data will not be seen. (But it will fill up the filesystem!)

mbekker
0
 

Author Comment

by:j8vy
ID: 10091654
ahoffmann,
I have accepted your answer since I have incorporated your ideas into my solution.
My solution consist of the following:

I wrote a function named change_log()
change_log() performs several main operationss:
1. It checks to see if $PRG is running
2. If $PRG is running, change_log() sends a SIGHUP signal to $PRG. When $PRG
recieves the SIGHUP signal, the following occurrs: (the following trap statement is located inside $PRG)
trap 'print "\nINFORMATION :: `date +'%X'` ::\n${0##/*/} will now pause for 1 minute"
       print "while change_log() is being executed\n"; sleep 60' 1

As you can see, when $PRG recieves the SIGHUP signal, it will sleep for one minute.
This allows change_log() to copy the current log file to a new log file, and then empty the current log. Please read the code below in change_log(), and you will see how this flow works.

change_log() {
set_var
print "\nChanging log files for ${PRG}"
cdate=$(date +'%Y%m%d')
new_log_0="${ECS_LOG_PATH}/${PRG}.${cdate}.log.0"
ud_pid=$(ps -ef |grep ${PRG}|egrep -v 'grep|ps|run|start|stop'|awk '{print $2}')

# Clean up old files
find ${ECS_LOG_PATH} -name "${PRG}.${cdate}.log.*" -mtime +${log_ret} -exec rm -f {} \;
find /opt/incontrol/sf_scripts/output -name "pctm*" -mtime +${log_ret} -exec rm -f {} \;

# Change log files
ps -ef |grep ${PRG}|egrep -v 'grep|ps|run|start|stop'>/dev/null; rc=$?
if [ ${rc} = 0 ]; then
  kill -HUP ${ud_pid}
fi
if [ -a ${ECS_LOG_PATH}/${PRG}.${cdate}.log.* ]; then
  cur_log=$(ls -t ${ECS_LOG_PATH}/${PRG}.${cdate}.log.*|head -n1)
  log_cnt=$(echo ${cur_log}|cut -f4 -d\.)
  new_log_cnt=$(( ${log_cnt} \+ 1 ))
    print "\nchange_log() has been invovked!\nThe current log file will be renamed to:\n">>${log_file}
    print "${ECS_LOG_PATH}/${PRG}.${cdate}.log.${new_log_cnt}\n">>${log_file}
   cp -pf ${log_file} ${ECS_LOG_PATH}/${PRG}.${cdate}.log.${new_log_cnt}
   cat /dev/null>${log_file}
   print "\nchange_log() has been executed\n"
elif [ ! -a ${ECS_LOG_PATH}/${PRG}.${cdate}.log.* ]; then
  cp -pf ${log_file} ${new_log_0}
  cat /dev/null>${log_file}
  print "\nchange_log() has been executed\n"
fi  
}

I spawn my program ($PRG) and place it in the background:

rc2=0; ${PROG_PATH}/${PRG} ${log_file}>${log_file} 2>&1 &

Now I start a loop that will continue until rc2=2 or rc2>2, and when the time reaches 0500, change_log() is executed. This will effectively cause the log file to be changed every morning at 0500.

while [[ ${rc2} = 0 || ${rc2} = 1 ]]; do
 ctime=$(date +'%H%M')
  case ${ctime} in
    0[5][0][0]) change_log; sleep 60 ;;
  esac
 sleep 10; ps -fp $!>/dev/null; rc2=$?
done

Once again:
1. change_log() is executed at 0500 to change the output filename of $PRG
    a. change_log() sends a SIGHUP signal to $PRG
    b. $PRG traps that SIGHUP signal and causes $PRG to sleep for 60 secs
    c. change_log() will then remove all log files that are older than $ini/$LOG_FILE_RET
    d. change_log() will then check to see what log files have already been created
    e. change_log() creates a new log file with a name that is incremented by one larger than the last file name ( if change_log() is executed during the same day )
    E.g. Last log file: ud_chk.20040110.log.0
          New log file: ud_chk.20040110.log.1
if change_log() is executed on a new day, then the new log file name will always be:
          New log file: ud_chk.20040110.log.0
    f. change_log() will then ( cp -pf ) the current log file into the new log file
    g. change_log() will now empty the current log file ( cat /dev/null > $log_file )
    h. $PRG will resume after the 60 secs has passed from when the trap statement was executed.

I hope that my explanation makes sense, and that someone else would be able to use this solution.

I should also thank BMC software for the ideas on how to increment the log file
names. I noticed that their Control-M software names its log files this way, and I
wanted my software to work this way, as this naming convention makes log files
very easy to identify. This function ( change_log() ) was, however, writtien entirely
by myself. Please feel free to reuse it for your own programs.


-Matt V
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 10092222
seems to be well done :-)
Just a side note: sleep 60   might be insufficient if there are some filesystem problems somehow (NFS, disk, whatever)
In this case you might loose some messages in your log.
The only way to get this rare case also working propper is to have some kind of semaphore shared by your script and the program writing the log.
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

Suggested Solutions

Title # Comments Views Activity
Decoding 32 bit binary streams 6 31
Need Help INsttalling wget on Mavericks OS X 3 81
xyBalance chalenge 58 88
changeXy challenge 13 57
Introduction: Dynamic window placements and drawing on a form, simple usage of windows registry as a storage place for information. Continuing from the first article about sudoku.  There we have designed the application and put a lot of user int…
Introduction: Load and Save to file, Document-View interaction inside the SDI. Continuing from the second article about sudoku.   Open the project in visual studio. From the class view select CSudokuDoc and double click to open the header …
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
This video gives you a great overview about bandwidth monitoring with SNMP and WMI with our network monitoring solution PRTG Network Monitor (https://www.paessler.com/prtg). If you're looking for how to monitor bandwidth using netflow or packet s…

705 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now