Solved

Changing the output file of a process that is already executing KSH

Posted on 2003-12-10
9
928 Views
Last Modified: 2013-12-26
Hello all,

I have a questions regarding changing the output file name a process that is already started.

In other words, I am starting my process in in an indefinate loop, then every 24 hours,
I would like to change the output file name. I will show you my script, and maybe that will help:

Script
===========================
loop="1"; _loop=$(( ${loop} \* 60 ))
log_days=$(grep log_days ${ini}|cut -f2 -d\=)
cnt=0
while [ 1 ]; do
# Loop vars
cdate=$(date +'%Y%m%d')
chour=$(date +'%H')
 
# Clean up old files
log_days=$(grep log_days matt.ini|cut -f2 -d\=)
find . -name "ud_chk.KSH*" -mtime+${log_days} -exec rm -f {} \;
 
# Set log file name
case ${chour} in
 0[0]) if [ ${cnt} = 0 ]; then
           out_file="ud_chk.KSH.${cdate}.$$.log.0"
          elif ${cnt} > 0 ]; then
           cur_file=$(ls -t ud_chk.KSH*|head -n1)
           file_date=$(echo ${cur_file}|cut -f3 -d\.)
           file_cnt=$(echo ${cur_file}|cut -f6 -d\.)
           new_file_cnt=$(( ${file_cnt} \+ 1 ))
          if [ ${file_date} != ${cdate} ]; then
            print "\nCurrent log file is: ${out_file}>>${out_file}"
            print "The new log file is: ud_chk.KSH.${cdate}.$$.log.${new_file_cnt}">>${out_file}
            out_file="ud_chk.KSH.${cdate}.$$.log.${new_file_cnt}"
          fi
         fi ;;
    *) if [ ${cnt} = 0 ]; then
          out_file="ud_chk.KSH.${cdate}.$$.log.0"
         fi ;;
esac
ctime=$(date +'%H%M')
 case ${ctime} in
  #0[0-1][4][0]) # Check Corp user daily (perform all other checks also) ;;
  #0[3-4][1][0]) # Check REGNxx user dalies (perform all other checks also) ;;
    0[7][5][0-5]) print '\n$ctime is currently between 0750 & 0755'
                        print "ctime=${ctime}" ;;
    0[7][5][5-9]) print '\n$ctime is currently between 0755 & 0759'
                        print "ctime=${ctime}" ;;
                    *) if [ ${cnt} = 0 ]; then
                          write_cnt>${out_file} 2>&1
                        elif [ ${cnt} > 0 ]; then
                          print "This is round ${cnt} in the loop"; sleep 5
                        fi ;;
 esac
  (( cnt+=1 ))
done

===========================
Script

So, obviously, if the script is first starting, ( $cnt=0 ), then the log file will be:
out_file="ud_chk.KSH.${cdate}.$$.log.0"

However, if the program ( write_cnt is the function being executed and can be located
near the end of my script ) is already running, then I am checking $cdate to see if a new date has arrived
and if it has, then I want to change the log file name to reflect the new date, and be like
Original name:
ud_chk.KSH.20031210.12345.log.0
New name:
ud_chk.KSH.20031211.12345.log.1

So, I want to change the date and increment the last digit of the log file name by one each time it is changed.
I believe that you will agree that my script will perform all of the above requirements of changing the file name,
but I do not know how to force write_cnt() to use the new log file without stopping and restarting it.

Also, as you can see, I wish to print the old log file name and the new name into the existing log file before I change it. That way if you are looking at one of the log files and you reach the end, then you will know what the name of the next log file is that you need to look at.

If this dosn't make any sense to, then I will be happy to explain further.

Thanks for your time,
-Matt V
0
Comment
Question by:j8vy
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
  • 2
  • 2
  • +3
9 Comments
 
LVL 9

Expert Comment

by:HamdyHassan
ID: 9915367
Hi

What this means
"how to force write_cnt() to use the new log file without stopping and restarting it"

If your process is running and writing to a log file, then you need to stop writing , opening another log file, start writing to

e.g.
you open a word document, and while writing to it, somebody else want to rename it, It won't work


Could you write in English the requirment just the logic?


0
 

Author Comment

by:j8vy
ID: 9915980
Ok, I'll just laugh it off .... but I am from Mississippi, and I am fairly sure that my primary language is English ... LOL LOL :) (Just Spanish on the side)

Anyway,

The loop will start:

cnt=0
while [ 1 ]; do
<some code here>
  Now we will execute the write_cnt() function and place its output into a file that will be $out_file
out_file="/path/to/out_file.date.pid.0"
   write_cnt>$out_file 2>&1 &
 (( cnt+=1 ))
done

Now as the loop executes, if the execution time passes a 24 hour period
(which it most defiantely will be running more than twenty four hours)
and I will determine this by placing write_cnt() into the background and continuing
the loop checking the current hour until a new day is reached. When this new day is reached
(preferably at 0000 or midnight you might say), I want to change the value of $out_file from
the original
out_file="/path/to/out_file.date.pid.0"
to
out_file="/path/to/out_file.date.pid.1"
write_cnt>$out_file 2>&1 &

so that now the value of $out_file has been changed and so on and so forth until the execution
stops or is stopped. What I want to do is force the process that I placed in the background (write_cnt)
to use the new value of $out_file.

I wanted to do this without stopping write_cnt and having to restart it, but if I need to stop it's execution
and restart it for it to use the new $out_file, then I will just have to do that.


Sorry for being unclear at first :)

-Matt
0
 
LVL 1

Expert Comment

by:mbekker
ID: 9919568
Hi Matt,

You could try one of the following, if it's possible:

- create a FIFO file with the same name as the output file
- start the program which writes to the FIFO file
- from another script, read from the FIFO file and redirect output
  to $outfile in the background (cat <file >$outfile &)
- after a while, kill the background process (e.g. kill %+)
- adjust the outfile variable and start the cat command again.

Another way is to copy the file to $fileout and empty the original one:

- cp -p file $outfile
- > file

There are two problems with the last solution. Between the cp command and '>' there could be something written to the outputfile. This output will be lost... If there's a moment where no output is generated it could be no problem.
The second problem could be not freeing the space in the filesystem from the original file. You have to check it with a 'df' before and after the cp and '>'. It depends on how the file is treated by the program.

Good luck!
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
LVL 51

Accepted Solution

by:
ahoffmann earned 350 total points
ID: 9931054
> .. I do not know how to force write_cnt() to use the new log file
write_cnt() needs to know how to do it.
For example most programs do that if they receive SIG_HUP signal.

Keep in mind that write_cnt() still writes to the file even if you remove it.
0
 
LVL 2

Expert Comment

by:zdes
ID: 9970476
Your standard output goes to file descriptor 1, stderr - to 2. To redirect it globally, run the following command:

exec 1> mylog.`date +%y%m%d-%H%M%S` 2>&1

just do it every 24 hours. In the example above the name of the file just contains date. This is just an example. You seem to have an elaborate rule about the file name.

also, you might want to flush stdout and err just before you redirect them by closing the descriptors:

exec 1>&- 2>&-

however, if you run some background processes, you might lose some output between the times you close the descriptors and open new again.
0
 

Expert Comment

by:harshalhayat
ID: 10010079
Matt,

Check out symbolic links (symlinks) and use them safely.
It is the best approach I feel.
Say

"a.out > a.stdout.${DATE} 2> a.stderr.${DATE}" can be changed with -
"a.out > a.stdout.link 2> a.stderr.link"
"ln -sf a.stdout.${DATE} a.stdout.link"
"ln -sf a.stderr.${DATE} a.stderr.link"

    - on every date change.

Harshal
0
 
LVL 1

Expert Comment

by:mbekker
ID: 10015035
Harshal,

The symbolic link trick won't always work.

If a process opens a file, the process gets a file descriptor. The file descriptor refers to the file via an inode or vnode or whatever... When a symbolic link is created with the same name, the inode will be different but the process still writes to the original inode. Since the inode is "gone", all data written to the file is lost.

I've put "" around gone because the inode still exists in the filesystem until the process closes the file. The inode doesn't have any filename which refers to it so the data will not be seen. (But it will fill up the filesystem!)

mbekker
0
 

Author Comment

by:j8vy
ID: 10091654
ahoffmann,
I have accepted your answer since I have incorporated your ideas into my solution.
My solution consist of the following:

I wrote a function named change_log()
change_log() performs several main operationss:
1. It checks to see if $PRG is running
2. If $PRG is running, change_log() sends a SIGHUP signal to $PRG. When $PRG
recieves the SIGHUP signal, the following occurrs: (the following trap statement is located inside $PRG)
trap 'print "\nINFORMATION :: `date +'%X'` ::\n${0##/*/} will now pause for 1 minute"
       print "while change_log() is being executed\n"; sleep 60' 1

As you can see, when $PRG recieves the SIGHUP signal, it will sleep for one minute.
This allows change_log() to copy the current log file to a new log file, and then empty the current log. Please read the code below in change_log(), and you will see how this flow works.

change_log() {
set_var
print "\nChanging log files for ${PRG}"
cdate=$(date +'%Y%m%d')
new_log_0="${ECS_LOG_PATH}/${PRG}.${cdate}.log.0"
ud_pid=$(ps -ef |grep ${PRG}|egrep -v 'grep|ps|run|start|stop'|awk '{print $2}')

# Clean up old files
find ${ECS_LOG_PATH} -name "${PRG}.${cdate}.log.*" -mtime +${log_ret} -exec rm -f {} \;
find /opt/incontrol/sf_scripts/output -name "pctm*" -mtime +${log_ret} -exec rm -f {} \;

# Change log files
ps -ef |grep ${PRG}|egrep -v 'grep|ps|run|start|stop'>/dev/null; rc=$?
if [ ${rc} = 0 ]; then
  kill -HUP ${ud_pid}
fi
if [ -a ${ECS_LOG_PATH}/${PRG}.${cdate}.log.* ]; then
  cur_log=$(ls -t ${ECS_LOG_PATH}/${PRG}.${cdate}.log.*|head -n1)
  log_cnt=$(echo ${cur_log}|cut -f4 -d\.)
  new_log_cnt=$(( ${log_cnt} \+ 1 ))
    print "\nchange_log() has been invovked!\nThe current log file will be renamed to:\n">>${log_file}
    print "${ECS_LOG_PATH}/${PRG}.${cdate}.log.${new_log_cnt}\n">>${log_file}
   cp -pf ${log_file} ${ECS_LOG_PATH}/${PRG}.${cdate}.log.${new_log_cnt}
   cat /dev/null>${log_file}
   print "\nchange_log() has been executed\n"
elif [ ! -a ${ECS_LOG_PATH}/${PRG}.${cdate}.log.* ]; then
  cp -pf ${log_file} ${new_log_0}
  cat /dev/null>${log_file}
  print "\nchange_log() has been executed\n"
fi  
}

I spawn my program ($PRG) and place it in the background:

rc2=0; ${PROG_PATH}/${PRG} ${log_file}>${log_file} 2>&1 &

Now I start a loop that will continue until rc2=2 or rc2>2, and when the time reaches 0500, change_log() is executed. This will effectively cause the log file to be changed every morning at 0500.

while [[ ${rc2} = 0 || ${rc2} = 1 ]]; do
 ctime=$(date +'%H%M')
  case ${ctime} in
    0[5][0][0]) change_log; sleep 60 ;;
  esac
 sleep 10; ps -fp $!>/dev/null; rc2=$?
done

Once again:
1. change_log() is executed at 0500 to change the output filename of $PRG
    a. change_log() sends a SIGHUP signal to $PRG
    b. $PRG traps that SIGHUP signal and causes $PRG to sleep for 60 secs
    c. change_log() will then remove all log files that are older than $ini/$LOG_FILE_RET
    d. change_log() will then check to see what log files have already been created
    e. change_log() creates a new log file with a name that is incremented by one larger than the last file name ( if change_log() is executed during the same day )
    E.g. Last log file: ud_chk.20040110.log.0
          New log file: ud_chk.20040110.log.1
if change_log() is executed on a new day, then the new log file name will always be:
          New log file: ud_chk.20040110.log.0
    f. change_log() will then ( cp -pf ) the current log file into the new log file
    g. change_log() will now empty the current log file ( cat /dev/null > $log_file )
    h. $PRG will resume after the 60 secs has passed from when the trap statement was executed.

I hope that my explanation makes sense, and that someone else would be able to use this solution.

I should also thank BMC software for the ideas on how to increment the log file
names. I noticed that their Control-M software names its log files this way, and I
wanted my software to work this way, as this naming convention makes log files
very easy to identify. This function ( change_log() ) was, however, writtien entirely
by myself. Please feel free to reuse it for your own programs.


-Matt V
0
 
LVL 51

Expert Comment

by:ahoffmann
ID: 10092222
seems to be well done :-)
Just a side note: sleep 60   might be insufficient if there are some filesystem problems somehow (NFS, disk, whatever)
In this case you might loose some messages in your log.
The only way to get this rare case also working propper is to have some kind of semaphore shared by your script and the program writing the log.
0

Featured Post

Enroll in June's Course of the Month

June’s Course of the Month is now available! Experts Exchange’s Premium Members, Team Accounts, and Qualified Experts have access to a complimentary course each month as part of their membership—an extra way to sharpen your skills and increase training.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Here is how to use MFC's automatic Radio Button handling in your dialog boxes and forms.  Beginner programmers usually start with a OnClick handler for each radio button and that's just not the right way to go.  MFC has a very cool system for handli…
Introduction: Dialogs (2) modeless dialog and a worker thread.  Handling data shared between threads.  Recursive functions. Continuing from the tenth article about sudoku.   Last article we worked with a modal dialog to help maintain informat…
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
In this video, viewers are given an introduction to using the Windows 10 Snipping Tool, how to quickly locate it when it's needed and also how make it always available with a single click of a mouse button, by pinning it to the Desktop Task Bar. Int…

707 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question