Avatar of g46905
g46905

asked on 

UNIX script to move files from one location to the other?

In some cases (occur twice or thrice a day) our application reads the file while it is being written through FTP by our business partner, means if the file is 400 KB, and it is being written our application might grab it while only 200 KB of it is written, our applicatoin tries to process it, but as the file is incomplete it cannot process it and this causes us a substantial data loss.  Since our trading partner is not willing to make the changes, I would like to write a UNIX script that would move ( not copy ) from this location ( where the files are being FTPed at this moment to a different location. During this move, I want to make sure that we do not move any incomplete files ( ie the files that are still being written by the FTP process ). Can you please help me with a UNIX script as I am not very familiar with UNIX shell scripts?
Shell Scripting

Avatar of undefined
Last Comment
netmunky
Avatar of ozo
ozo
Flag of United States of America image

Is there a way to distinguish complete files from incomplete files?
Is the size always the same?  Is there a unique signature at the end?
Can you ftp a separate done indicator file after the first ftp is complete?
ASKER CERTIFIED SOLUTION
Avatar of nedfine
nedfine
Flag of India image

Blurred text
THIS SOLUTION IS ONLY AVAILABLE TO MEMBERS.
View this solution by signing up for a free trial.
Members can start a 7-Day free trial and enjoy unlimited access to the platform.
See Pricing Options
Start Free Trial
Avatar of netmunky
netmunky
Flag of United States of America image

usually the best way to ensure proper data integrity is to transfer an md5 file with the data file, then you can check the checksum before processing the file.
but if the source of the data file refuses to modify anything, you want to see if the ftp process still has the file open. best way may be to check open file descriptors (`lsof /path/to/file.ext` on linux, `fstat /path/to/file.ext` on *bsd)
Shell Scripting
Shell Scripting

The term 'shell' refers to a general class of text-based command interpreters most often associated with the UNIX and Linux operating systems. Popular shells include Bourne, Debian Almquist (dash), Korn (ksh), Bourne Again (bash) and the C shell family (csh). Some view the DOS 'cmd' prompt as a minimal shell of sorts. It is also possible to install Cygwin on Windows and emulate a full Unix environment with complete shell capabilities. Terminal emulators, such as xterm, GNOME Terminal and OS X Terminal, can be used to access shell.

11K
Questions
--
Followers
--
Top Experts
Get a personalized solution from industry experts
Ask the experts
Read over 600 more reviews

TRUSTED BY

IBM logoIntel logoMicrosoft logoUbisoft logoSAP logo
Qualcomm logoCitrix Systems logoWorkday logoErnst & Young logo
High performer badgeUsers love us badge
LinkedIn logoFacebook logoX logoInstagram logoTikTok logoYouTube logo