?
Solved

Destroying XMLNode and XMLElement objects

Posted on 2006-05-11
5
Medium Priority
?
466 Views
Last Modified: 2006-11-18
I have a program where we create a very large (50MB+) XML file using C#.  It is taking a huge amount of memory in doing this.  It had become clear to use that the unreferenced objects were not being cleaned up and this the memory usage just kept growing.

We built a test program to verify this:

        /*----------------------------------------------------------------------
         * public void LoadDocument()
         *
         *
         * --------------------------------------------------------------------*/
        public void LoadDocument()
        {
            XmlDocument m_XmlData;
            XmlNode testnode;
            XmlElement testelement;

            m_XmlData = new XmlDocument();
            m_XmlData.LoadXml("<Test />");

            for (int i = 0; i < 2300; i++)
            {
                Trace.WriteLine("Running Loop " + i.ToString());

                testnode = m_XmlData.CreateElement("Test");
                testelement = m_XmlData.CreateElement("TestElement");
                testelement.InnerText = "Hi There";

                testnode.AppendChild(testelement);

                m_XmlData.GetElementsByTagName("Test").Item(0).AppendChild(testnode);
            }
        }

This function is called in a thread and when executed allocates about 4MB to preform.  When the function is done, and it exits the thread ends also.  However the memory allocation remains unchanged.  The memory allocated is not released.  This confirms our thought that memory was being allocated but not freed.   I know that garbage collection does not happen right away but how can I get these XMLNode and XMLElement objects to free after each interation of the loop?

When we run our larger XML file creator, it will create the 50 MB XML file but will allocate almost 1GB of RAM to do it.  It is hugely disproportionate.

We have tried having the garbage collection run at the end of each loop using GC.Collect() but it had no effect.

Any ideas out there?
0
Comment
Question by:jymmealey
  • 2
  • 2
4 Comments
 
LVL 11

Accepted Solution

by:
vo1d earned 2000 total points
ID: 16661927
you should not create your xml structure in memory first.
write it directly into the file by using the XmlWriter.
0
 
LVL 2

Author Comment

by:jymmealey
ID: 16662186
Writing it directly to the file is not an option for us at this point.  It would mean heavily re-writing and that can't happen right now.  If I can get the memory issue resolved there should be no need.

Jym
0
 
LVL 11

Expert Comment

by:vo1d
ID: 16668840
you cant solve that memoryissue because you are bulding a tree of xmlnodes internally.
so you have dependencies between parent nodes and childnodes.
what you could try to do is splitting your structure into logical blocks and create a list of references to memorystreams.
then in your loop, create the logical block with a reference to its parent node, write that block into a memorystream and compress that stream.
after creating your data itereate through your memorystream list, decrompress each memorystreamobject and write it to a file.
thats only an appproach.
0
 
LVL 2

Author Comment

by:jymmealey
ID: 16853421
I think vo1d is correct although I have not been able to fully explore all options.  As created elements are added to the document it seems to create some kind of lasting reference, although the object is no longer directly accessed by the document.  I have tested several times by chaning the element object after it is added and it make no effect on the document so it seems silly that the reference holds, but none the less it does.

I am using the XMLWriter, I had to alter the program a bit but I think it will work fine.

Thanks
Jym
0

Featured Post

How to Use the Help Bell

Need to boost the visibility of your question for solutions? Use the Experts Exchange Help Bell to confirm priority levels and contact subject-matter experts for question attention.  Check out this how-to article for more information.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction Although it is an old technology, serial ports are still being used by many hardware manufacturers. If you develop applications in C#, Microsoft .NET framework has SerialPort class to communicate with the serial ports.  I needed to…
Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
Screencast - Getting to Know the Pipeline
As many of you are aware about Scanpst.exe utility which is owned by Microsoft itself to repair inaccessible or damaged PST files, but the question is do you really think Scanpst.exe is capable to repair all sorts of PST related corruption issues?
Suggested Courses
Course of the Month14 days, 11 hours left to enroll

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question