Session Directory on Terminal Servers

I have created a 2 sever network load balanced terminal server cluster.  Both have Windows 2003 Enterprise and I have created a Session Directory on a third server.  I used a GPO and have done all of the preliminary work such as adding these servers to the Session Driectory group and have started the session directory service on the 3rd server holding the database.  Each server reports in the event viewer that it has joined the Session Directory successfully.  This morning I connected to the cluster, opened a word document and started typing an example and after confirming which server I actually connected to I unplugged the network cable to that server.  When I tried to connect to the cluster again it would not let me connect.  I was under the impression that a session directory setup would provide high availabilty by allowing me to reconnect to another server in the cluster and putting me back into my word document or whatever else I was doing when the server "wnet down".  Not only did it not do that, I was prevented from connecting to the cluster as that user because it still saw my session as being open.  Am I missing something or is this how it's supposed to work?  If that's the case what is the real benefit of using a session directory on top of using network load balancing?  It seems like the session directory is working against me in this scenario as I am unable to get back to any server in the cluster.  Any thoughts or comment would be much appreciated.