Solved

Unix ftp script works sometimes good and sometimes not !?!!?!!?!

Posted on 2004-08-18
6
1,309 Views
Last Modified: 2013-12-26
Hi, all,

I have writtin an ftp script, and tested it, and it worked fine.

... Top of script not shown ...
#put the files
                  
#Create local control file
touch ${TMP_DIR}/`basename $CONTROL_FILE`

#Perform ftp
echo "
verbose
open $IP
user $LID $PWORD
cd $RFILE_PATH
ascii
put $LFILE $RFILE
ascii
put ${TMP_DIR}/`basename $CONTROL_FILE` $CONTROL_FILE
bye
"| ftp -i -n 2>&1 >> $REZ

#Delete local control file
rm ${TMP_DIR}/`basename $CONTROL_FILE`

... End of script not shown ...

Here is a good log it generates:

Verbose mode on.
Connected to 168.185.180.200.
220 ecaaprod FTP server (Version 1.1.214.4(PHNE_27765) Wed Sep  4 05:59:34 GMT 2002) ready.
Remote system type is UNIX.
Using binary mode to transfer files.
331 Password required for edsedi.
230 User edsedi logged in.
200 PORT command successful.
150 Opening ASCII mode data connection for /usr/bin/ls.
/baan/baan_prod/edi/061/mir/command/command.fil not found
226 Transfer complete.
221 Goodbye.
expected return value is found
Verbose mode on.
Connected to 168.185.180.200.
220 ecaaprod FTP server (Version 1.1.214.4(PHNE_27765) Wed Sep  4 05:59:34 GMT 2002) ready.
Remote system type is UNIX.
Using binary mode to transfer files.
331 Password required for edsedi.
230 User edsedi logged in.
250 CWD command successful.
200 Type set to A.
200 PORT command successful.
150 Opening ASCII mode data connection for INVRPT_100.061.
226 Transfer complete.
3048 bytes sent in 0.00 seconds (3990.03 Kbytes/s)
200 Type set to A.
200 PORT command successful.
150 Opening ASCII mode data connection for /baan/baan_prod/edi/061/mir/command/command.fil.
226 Transfer complete.
221 Goodbye.

As you see 2 files are send to the target, one of 3048 bytes, and the command.fil file has nothing in it, 0 bytes.
The script runs quite often, sometimes more times within seconds (but with other filenames, so nothing will ever be overwritten.

Sometimes (few times a week), it generates following log:
Verbose mode on.
Connected to 168.185.180.200.
220 ecaaprod FTP server (Version 1.1.214.4(PHNE_27765) Wed Sep  4 05:59:34 GMT 2002) ready.
Remote system type is UNIX.
Using binary mode to transfer files.
331 Password required for edsedi.
230 User edsedi logged in.
200 PORT command successful.
150 Opening ASCII mode data connection for /usr/bin/ls.
/baan/baan_prod/edi/061/mir/command/command.fil not found
226 Transfer complete.
221 Goodbye.
expected return value is found
Verbose mode on.
Connected to 168.185.180.200.
220 ecaaprod FTP server (Version 1.1.214.4(PHNE_27765) Wed Sep  4 05:59:34 GMT 2002) ready.
Remote system type is UNIX.
Using binary mode to transfer files.
331 Password required for edsedi.
230 User edsedi logged in.
250 CWD command successful.
200 Type set to A.
200 PORT command successful.
150 Opening ASCII mode data connection for INVRPT_100.061.
226 Transfer complete.
3048 bytes sent in 0.00 seconds (3885.85 Kbytes/s)
200 Type set to A.
221 Goodbye.

As you can see, the second put for the file command.fil is not in the log!!!  
200 PORT command successful.
150 Opening ASCII mode data connection for /baan/baan_prod/edi/061/mir/command/command.fil.
226 Transfer complete.

There is also no message like, file not found, or command failed or whatever, it seems like the second put is ignored.

PS: The ascii command between the first and the second put is not needed, but we put it there to check if it would make a difference.

Can anyone give me the reason and a solution for this, why the script sometimes ignores the second put???  I'd really would like to find out a reason why the put can be ignored without any error message.

Best Regards,
Tim
0
Comment
Question by:thimerion
  • 3
  • 3
6 Comments
 
LVL 20

Expert Comment

by:tfewster
ID: 11829529
I believe that
|ftp -i -n 2>&1 >> $REZ
should be
|ftp -i -n >> $REZ 2>&1

Your way, stderr is redirected to the device stdout _currently_ points to (e.g. a terminal) and stays pointed to that when stdout is redirected to $REZ;

The other way around, stdout is redirected to a file and then stderr is redirected to point to the same device.

So errors probably _were_ being generated, but not captured.  If these jobs were run from e.g. cron, any output that was not "properly" redirected would be emailed to the user.  Or a scheduling product like Maestro would capture such output in its stdlist logs.
0
 
LVL 1

Author Comment

by:thimerion
ID: 11829766
Hi tfewster,

Thanks for that!  I changed it in my script immediately.

Guess now we have to wait patiently :( until it fails again.

Hope we catch the fish here!  Fingers crossed.

Tim
0
 
LVL 20

Accepted Solution

by:
tfewster earned 500 total points
ID: 11830202
OK, if there's no way of testing it (e.g. sending files from a test environment to a test directory and forcing a failure).  Obviously you need to check that the change has not affected the normal running/logging!

Thinking ahead...
Why would the second "put" fail? Probably because the control file wasn't there to be sent, in which case you might have to go a long way back into the file production processes to diagnose the underlying problem.

Best wishes,
Tim F
0
What Should I Do With This Threat Intelligence?

Are you wondering if you actually need threat intelligence? The answer is yes. We explain the basics for creating useful threat intelligence.

 
LVL 1

Author Comment

by:thimerion
ID: 11830226
Hi tfewster,

We catched the error thanks to you!

If the script runs 2 times, the first one deletes the file before the other one sends it:-)

Script Run 1:
#Create local control file
touch ${TMP_DIR}/`basename $CONTROL_FILE`

Script Run 2:
#Create local control file
touch ${TMP_DIR}/`basename $CONTROL_FILE`

Script Run 1:
#Delete local control file
rm ${TMP_DIR}/`basename $CONTROL_FILE`

Script Run 2:
150 Opening ASCII mode data connection for INVRPT_100.061.
226 Transfer complete.
3048 bytes sent in 0.00 seconds (3786.98 Kbytes/s)
200 Type set to A.
/tmp/command.fil: No such file or directory
221 Goodbye.

You get all the points for this one.

Regards,
Tim
0
 
LVL 20

Expert Comment

by:tfewster
ID: 11830500
> The script runs quite often, sometimes more times within seconds

Damn, I should have guessed that possible cause ;-)

There are ways to check that you don't have 2 copies of the script running at the same time, e.g.

#!/usr/bin/ksh
if [ `ps -ef |grep scriptname|grep -v grep |wc -l` -ne 1 ]
then
  echo "Another instance of this script is running!
  exit 1
fi
(And then the rest of your script)
0
 
LVL 1

Author Comment

by:thimerion
ID: 11830641
Yep, but that will stop the second script from running, and it needs to run (sending other files).

We just do not delete the command.fil anymore now, so it can't be gone :-)

      #Delete local control file --> bad idea cause could be removing command.fil for next script
      #rm ${TMP_DIR}/`basename $CONTROL_FILE`

If that also will fail, we plan to use another dir (with date/time/process id stamp in it) to put the command.fil file.

Thanks,
Tim
0

Featured Post

IT, Stop Being Called Into Every Meeting

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

Introduction: Finishing the grid – keyboard support for arrow keys to manoeuvre, entering the numbers.  The PreTranslateMessage function is to be used to intercept and respond to keyboard events. Continuing from the fourth article about sudoku. …
Have you tried to learn about Unicode, UTF-8, and multibyte text encoding and all the articles are just too "academic" or too technical? This article aims to make the whole topic easy for just about anyone to understand.
This video will show you how to get GIT to work in Eclipse.   It will walk you through how to install the EGit plugin in eclipse and how to checkout an existing repository.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

17 Experts available now in Live!

Get 1:1 Help Now