Apparent File deletion aberration.

I *seem* to have come across a situation :

With a FileReader and FileWriter pointing at an open File, as I understand things, both of these would need to be closed to release their file handles so that the file could be deleted (if required).

But some code I am trying closes only the Writer, then calls System.gc(), then calls a delete() on the file. The file is deleted, even though the Reader still has a line to it. If you then call a File.ready() on the file, or a File.read(), this even more bizarrely stops the deletion of the file again, and ready() will return 'true'.

In short, this doesn't seem like behaviour which I should be seeing. Is there any explanation?

import java.util.*;
import java.io.*;
import java.nio.file.Files;

class Strings {


public static void main(String[] args){

	try{
		File f = new File("C:/ee_q_code/nf.txt");// put your own file in here naturally
		FileWriter fw = new FileWriter(f);
		FileReader fr = new FileReader(f);
	try{
		if(f.exists()){
			for(int r=0;r<50000;r++){
				fw.write("Hello");
		}

		System.out.println(f.length());
	}

	}catch(Exception e){e.printStackTrace();}
	finally{fw.close();
                  System.gc();// See the effect of including or omitting this directive.
                  System.out.println((f.getAbsoluteFile()).delete());
        }
	//System.out.println(fr.read());// See the effect of keeping gc() and uncommenting this or the next line.
	//System.out.println(fr.ready());
	}catch(Exception all){all.printStackTrace();}


  }

}

Open in new window

LVL 17
krakatoaAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

CEHJCommented:
goose@vaio:/tmp$ java Strings
0
true
goose@vaio:/tmp$ java Strings
0
true
goose@vaio:/tmp$ java Strings
0
true
72

Open in new window

are the results of your commented directions on my system* except with the counter changed to 5
a. run original
b. run with System.gc commented out
c. run with System.gc in and System.out.println(fr.read()); uncommented


* Debian Wheezy 32bit x86
java version "1.7.0_25"
OpenJDK Runtime Environment (IcedTea 2.3.10) (7u25-2.3.10-1~deb7u1)
OpenJDK Client VM (build 23.7-b01, mixed mode, sharing)
0
krakatoaAuthor Commented:
Thanks CEHJ.

My results, referenced to your enumeration above, vary from yours and are :-

a. Same. viz: true (= file deleted, despite Reader still open.)
b. Different. viz: false (= file not deleted, as Reader still open.)
c. Different. viz: false (= file not deleted; the subsequent read() seems to defeat the effect of including System.gc() - (the 'a.' case)).

Java version 1.7.0_45
Windows 7 SP 1. 32 bit.

----------------------------

Also, in my opinion, (and code), you should only be seeing that char read() print (the 72 for 'H') being returned when the delete failed, not when the delete() method returned 'true'. This seems equally worrying.
0
girionisCommented:
My output is this

(run original)
245760
true

(run with System.gc commented out)
245760
false

(run with System.gc in and System.out.println(fr.read()); uncommented)
245760
false
72

If you don't use the System.gc(); you cannot delete the file since there are still references to it by the reader and writer. If you do use the System.gc(); then gc occurs and all the references to it are lost, so you can delete it.

The read indeed seems to defeat the effect of System.gc(). But bear in mind that the Java code is not always translated into bytecode with the same order of instructions, or even the bytecode is not always executed in the same order (the order of execution might be different as long as you get the same result at the end).

java version: 1.6.0_24 (64 bits).
Windows 7 Professional, service pack 1, 64 bits.
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
Introduction to Web Design

Develop a strong foundation and understanding of web design by learning HTML, CSS, and additional tools to help you develop your own website.

krakatoaAuthor Commented:
Thanks girionis.

I'm not of course qualified to pronounce on deeper why and wherefors, but it seems to me that this behaviour is potentially troublesome, even if there is a technical explanation, since the programmer may be to some extent relying on the file not being deleteable if an open resource has a handle to it.

I suppose a call to gc() could also be somewhere else in the JVM session, and so very distant from such events, which might give rise to some superficially deep-looking bug. Then do you  perhaps think that there is exposure too on the security front - if part of an app, or even the OS - thinks a file doesn't exist, even momentarily, then all sorts of cracks could appear in the system. I added a thread to the earlier code, laced with a wait() notify() semaphore, with the new thread looking for the file that gc() had permitted to be deleted. The thread reported that the file indeed was deleted. I wouldn't appreciate that greatly, if I had to explain this behaviour to someone else, as modest programmer myself. ;)
0
krakatoaAuthor Commented:
This is the code with a file-inspecting thread :

import java.util.*;
import java.io.*;


class Strings implements Runnable{

final static Object ob = new Object();
static File f;

public static void main(String[] args){

	try{
		f = new File("C:/ee_q_code/nf.txt");
		FileWriter fw = new FileWriter(f);
		FileReader fr = new FileReader(f);
	try{
		if(f.exists()){
			for(int r=0;r<50000;r++){
				fw.write("great");
		}

		System.out.println(f.length());
	}

	}catch(Exception e){e.printStackTrace();}
	//finally{fw.close();System.gc();System.out.println((f.getAbsoluteFile()).delete());}
	finally{fw.close();System.gc();System.out.println(f.delete());}

	synchronized (ob){

		Thread thread = new Thread(new Strings());

		thread.start();

		try{
		ob.wait();
		}catch(Exception e){}
	}

	//System.out.println(fr.read());
	//System.out.println(fr.ready());
	}catch(Exception all){all.printStackTrace();}


  }
	public void run(){

		synchronized (ob){

		if(f.exists()){System.out.println("File exists");}else{System.out.println("File deleted");}
		
		ob.notifyAll();

		}
	}

}

Open in new window


(I know you guys don't need the code, but I'd done it anyhow).
0
dpearsonCommented:
Part of the potential difference here is that you are relying (implicitly) on the behavior of the finalize method inside FileinputStream:

    protected void finalize() throws IOException {
        if ((fd != null) &&  (fd != FileDescriptor.in)) {

            /*
             * Finalizer should not release the FileDescriptor if another
             * stream is still using it. If the user directly invokes
             * close() then the FileDescriptor is also released.
             */
            runningFinalize.set(Boolean.TRUE);
            try {
                close();
            } finally {
                runningFinalize.set(Boolean.FALSE);
            }
        }
    }

This is the code that is doing the cleanup when you call System.gc().

However, 'finalize' itself is essentially an optional method - the garbage collector does not need to call it immediately.  Instead it puts them on a list of objects that need to be finalized and then calls finalize on the objects in that list (potentially after returning control to you - it's allowed to actually never call finalize() on the objects).

What's more, the methods inside a finalize() implementation are allowed to re-establish references to the rest of memory, causing the objects to be 'reincarnated' and no longer be garbage.  finalize() is a scary beast.

All in all, what this says is that if you fail to call close() on the reader, you are in the realm of "undefined behavior".  Which means Java is allowed to produce different results on different platforms and even on different runs within the same platform, because the correctness of your code now depends on the timing of when the finalize method is called.

Add to that the complexity of the different ways Windows can open files for sharing (which impacts the locking strategy): http://msdn.microsoft.com/en-us/library/windows/desktop/aa363874(v=vs.85).aspx

and you can see things get very complicated very quickly.  Java is of course mapping to a specific choice per O/S so again, you could get platform variations in when and how deletes are handled while a file handle is open.  I've not run into this so much with file handles, but I know sockets get very different treatment between Windows and Linux in terms of when a socket is already open on a port (which is similar to having two handles to a file).

All in all, I'd say the variation you're seeing is probably allowed by the Java spec - because you should never assume you can rely on the behavior of 'finalize', because basically the spec says you can't rely on it :)

Hope that helps,

Doug
0
krakatoaAuthor Commented:
Thanks for that contribution - which is very illuminating.
 It's a bit like discovering you are bankrupt or gravely ill, when you thought you had plenty in the bank on both counts - when the news comes, it's too late to do anything about it. ;)

Having said that, I haven't (fortunately) implemented anything (nor probably never will) on which my head depends which is impacted by this behaviour. But I can imagine there must be people who have, and others who could possibly exploit it to some nefarious ends. Either way, and whatever is going on under the hood, to me, a relative nobody in terms of programming, it's an outright bug. If the docs say that you can't delete a file when some handle persists, then that should be an absolute, and how the Java guys get around what the CPU does with its pile of instructions to get the job done should not really be the problem nor province of people "like me" so to speak. After all, if you did have money in the bank and thought you could spend it, then someone says something like "yes, well, you do have money in there, but not if the upstairs windows were opened during last Summer's heatwave, because that makes the money disappear into Schroedinger's Box" . . . then, although that might or might not be true, I for one would be tempted to look for another bank pretty swiftish.
0
CEHJCommented:
If the docs say that you can't delete a file when some handle persists, then that should be an absolute
Which docs are you referring to there?
0
dpearsonCommented:
If the docs say that you can't delete a file when some handle persists, then that should be an absolute

I think is also potentially a bit more complex than it looks.  You can imagine an OS allowing both - that is for the file to be deleted but for existing readers to continue to read from the file.  A bit like if I was reading a book in the library and somebody came in and deleted it from the index.  I can finish reading, but nobody new can check it out.  Doesn't mean the library is broken.  Also other libraries (OSes) might make different choices here.

Linux for instance has the concept of hard links which are essentially multiple 'references' to a file.  You can delete each link and the file remains on disk until the last one is gone.  So deletes can succeed, while not affecting the contents of the file (and presumably not necessarily other readers of that file).

Not saying that you're not right that this might be seen as a bug.  Just that file I/O and handles support a lot of concepts these days.

Doug
0
krakatoaAuthor Commented:
CEHJ -

I am going mostly by this :

Returns:
true if and only if the file or directory is successfully deleted; false otherwise

Of course, delete() *does* return the correct result each time, but the "if and only if" semantics seem to belie the fact that subsequent code can write to and read from, that very same "deleted" file. ;) To me - that's stretching having your cake and eating it a bit too far. ;)

Doug -

Yes, you can finish reading, as long as you don't mind reading a different ending to the one that everyone else got when they read the book. :)

I'll pretend it's not a bug. I'll also pretend I know what I'm talking about. ;)
0
krakatoaAuthor Commented:
The really scary thing is, as I already mentioned, that another VM process can call System.gc() (and possibly other methods, dunno, haven't investigated that), and on returning control to the caller, the file can be deleted.

In fact, from a test I just ran with the Writer *and* Reader still open, the file can still be deleted, with the accomplice call coming from a far process.
0
dpearsonCommented:
Yes, you can finish reading, as long as you don't mind reading a different ending to the one that everyone else got when they read the book. :)

Haha - love it :)

Doug
0
krakatoaAuthor Commented:
Yeah, and the library's still not broke either! The librarian instead just telling the next reader to read everything they have on the shelves pertaining to the Copenhagen interpretation of quantum mechanics, before they tackle their first choice War and Peace. This library doesn't like to disappoint . . . Ahem.  

:) ;)

Anyway, in this process, I seem to have re-written java.util.concurrent. Wow.
0
krakatoaAuthor Commented:
I haven't really closed this question, simply awarded points.
0
krakatoaAuthor Commented:
By the way, for anyone still interested, it seems to all depend on the length of the file you are trying to delete. That *might* explain CEHJ's results.

viz: if I make the write iterations 50000, then gc() overrides and you can delete the file. But if I make only 5000 writes, then it is not possible to delete it. So it appears there is something going on with the disk housekeeping and its propagation.
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
Java

From novice to tech pro — start learning today.