g46905
asked on
UNIX script to move files from one location to the other?
In some cases (occur twice or thrice a day) our application reads the file while it is being written through FTP by our business partner, means if the file is 400 KB, and it is being written our application might grab it while only 200 KB of it is written, our applicatoin tries to process it, but as the file is incomplete it cannot process it and this causes us a substantial data loss. Since our trading partner is not willing to make the changes, I would like to write a UNIX script that would move ( not copy ) from this location ( where the files are being FTPed at this moment to a different location. During this move, I want to make sure that we do not move any incomplete files ( ie the files that are still being written by the FTP process ). Can you please help me with a UNIX script as I am not very familiar with UNIX shell scripts?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
usually the best way to ensure proper data integrity is to transfer an md5 file with the data file, then you can check the checksum before processing the file.
but if the source of the data file refuses to modify anything, you want to see if the ftp process still has the file open. best way may be to check open file descriptors (`lsof /path/to/file.ext` on linux, `fstat /path/to/file.ext` on *bsd)
but if the source of the data file refuses to modify anything, you want to see if the ftp process still has the file open. best way may be to check open file descriptors (`lsof /path/to/file.ext` on linux, `fstat /path/to/file.ext` on *bsd)
Is the size always the same? Is there a unique signature at the end?
Can you ftp a separate done indicator file after the first ftp is complete?