Solved

unlink file succeed and no content for the file but the file is still  there

Posted on 2014-01-06
7
274 Views
Last Modified: 2014-01-21
I used Perl to detect a directory. If there is a new file coming, I copy this file to dest directory and then unlink this file from the src directory. Most cases are good. But in some rare cases, the file is still left in the src directory. Actually the left files had been copied to dest directory successfully. The left files are zero in size. This rare case usually happens in integral point clock. Does anyone has this problem before?
0
Comment
Question by:c11v11
7 Comments
 
LVL 8

Expert Comment

by:Surrano
ID: 39759440
There's probably some process holding the file open.
What filesystem(s) are we talking about?
Is the allocated space released in such occasions or still occupied by a "phantom" version of the file?

You may try using "lsof" or "lsof -X" to find what process keeps the file open, but if it's a momentary thing then maybe you'll never catch it. Even if you catch it, you have to understand how to alter the behaviour either the process in question or your script.

As a workaround, though, if it happens only between HH:00:00.000000 and HH:00:00.999999 then check the time before the unlink and if it falls in this interval then sleep one second. If you post a snippet of the script that does the unlink, I'll try to come up with a syntactically correct solution.
0
 
LVL 26

Expert Comment

by:wilcoxon
ID: 39759520
Is your directory NFS mounted (or some other remote file-system method)?  I've seen NFS do weird things (most often seemingly when it thinks a file is in use).
0
 
LVL 48

Expert Comment

by:Tintin
ID: 39760397
How are you checking that the file is complete, ie: not being written to before copying?
0
Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

 
LVL 8

Expert Comment

by:Surrano
ID: 39761370
Tintin's right; if it is not well-defined uncomfortable events may occur. If you can influence the sw creating the files, do something like create file "somefile.someuniqueid.tmp" and once complete, i.e. right after closing the file, rename it to "somefile.someuniqueid". Moving those files around should be piece of cake.

If you can't influence the software and you don't know its exact behaviour I think the best you can do is to check whether file is still open. Also, consider moving instead of copy+unlink if it makes sense in your use case.
0
 

Author Comment

by:c11v11
ID: 39774307
We are using NFS. The are 160 client  writing to this NFS share at the same time. I asked the developer if there is tmp file created when the clients wrote to the NFS share.  The developer said yes.  But I did not notice this. ( the file is very small, maybe it is not very easy to see this) This only happened on the integral point clock. That is very strange.Why does this only happen on integral point clock??
0
 
LVL 8

Accepted Solution

by:
Surrano earned 50 total points
ID: 39775855
Probably because a cron job runs at the same time, affecting either the files in question or the nfs share itself.
0
 

Author Closing Comment

by:c11v11
ID: 39798793
This is the problem. Thanks.
0

Featured Post

Find Ransomware Secrets With All-Source Analysis

Ransomware has become a major concern for organizations; its prevalence has grown due to past successes achieved by threat actors. While each ransomware variant is different, we’ve seen some common tactics and trends used among the authors of the malware.

Join & Write a Comment

Over the last ten+ years I have seen Linux configuration tools come and go. In the early days there was the tried-and-true, all-powerful linuxconf that many thought would remain the one and only Linux configuration tool until the end of times. Well,…
Linux users are sometimes dumbfounded by the severe lack of documentation on a topic. Sometimes, the documentation is copious, but other times, you end up with some obscure "it varies depending on your distribution" over and over when searching for …
Connecting to an Amazon Linux EC2 Instance from Windows Using PuTTY.
This demo shows you how to set up the containerized NetScaler CPX with NetScaler Management and Analytics System in a non-routable Mesos/Marathon environment for use with Micro-Services applications.

758 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

27 Experts available now in Live!

Get 1:1 Help Now