Getting "Exception of type 'System.OutOfMemoryException' was thrown" when i try to load a xml....

Hi

Iam getting this exception when i try to load xaml file to a dataset (Exception of type 'System.OutOfMemoryException' was thrown) , the file iam loading is on 381 mb, so memmory are getting full when i try to load it....does any one know ho to fix this issue?
try
            {
                myDsBackup = new DsAmDb();
               
                XmlTextReader xmlReader = new XmlTextReader(GetBackupfile());
                XmlDataDocument xmlDs = new XmlDataDocument(myDsBackup);

                xmlDs.Load(xmlReader); ::::::GETTING EXCEPTIN HERE::::::
                m_intDbVersion = Convert.ToInt32(xmlDs.DataSet.Tables["DbVersion"].Rows[0].ItemArray[0]);
                myDsBackup = (DsAmDb)xmlDs.DataSet.Copy();
                xmlReader.Close();

                if (myOper.Engagement_ID != (int)myDsBackup.Engagement.Rows[0]["Engagement_ID"])
                {
                    MessageBox.Show(GetValue("engvsdbinconsistens"), GetValue("error"), MessageBoxButtons.OK, MessageBoxIcon.Error);
                    lstBoxEngagement.DataSource = null;
                    lstBoxClient.DataSource = null;
                    btnLoad.Enabled = false;
                    btnRestoreClient.Enabled = false;
                    this.Cursor = Cursors.Default;
                    return;
                }
            }

Open in new window

AdnanAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

DanielSVCommented:
381 Megabit or MegaByte?

How much RAM do you have available?

Also; Is XML really the best format to store your data in? I mean - 381MB is quite a lot of text!
0
_Katka_Commented:
Hi, try using the keyword 'using' as in:

using (XmlTextReader xmlReader = new XmlTextReader(GetBackupfile()))
using (XmlDataDocument xmlDs = new XmlDataDocument(myDsBackup))
{
                xmlDs.Load(xmlReader); ::::::GETTING EXCEPTIN HERE::::::
                m_intDbVersion = Convert.ToInt32(xmlDs.DataSet.Tables["DbVersion"].Rows[0].ItemArray[0]);
                myDsBackup = (DsAmDb)xmlDs.DataSet.Copy();
                xmlReader.Close();

                if (myOper.Engagement_ID != (int)myDsBackup.Engagement.Rows[0]["Engagement_ID"])
                {
                    MessageBox.Show(GetValue("engvsdbinconsistens"), GetValue("error"), MessageBoxButtons.OK, MessageBoxIcon.Error);
                    lstBoxEngagement.DataSource = null;
                    lstBoxClient.DataSource = null;
                    btnLoad.Enabled = false;
                    btnRestoreClient.Enabled = false;
                    this.Cursor = Cursors.Default;
                    return;
                }
}

regars,
Kate
0
AdnanAuthor Commented:
the file is on 381 MB (400 149 216 bytes), and i have 4 gb ram, xml is maybe not the best format to have data stored in but that is the way they have chosen to do here on my work.. :( ?!
0
Introduction to R

R is considered the predominant language for data scientist and statisticians. Learn how to use R for your own data science projects.

_Katka_Commented:
My guess is that you're not disposing of the data readers, try the suggestion above. It will release any used memory by the process of reading the data (could be huge sometimes). I guess this happens when this method is called multiple times. This should fix the memory leak.

regards,
Kate
0
AdnanAuthor Commented:
katka: when i try your codse iam getting this errpr when Error      22      'System.Xml.XmlDataDocument': type used in a using statement must be implicitly convertible to 'System.IDisposable'
0
_Katka_Commented:
Ok, I guess it is really out of memory.. Remember that you store those data in the dataset so you can be potentially using upto 800MB at that time. I'd first try to loading it directly to dataset via (considering that your DsAmDb is a descendant of the DataSet):

1) all at once differently

System.Xml.XmlTextReader xmlReader = new System.Xml.XmlTextReader(GetBackupfile());
myDsBackup.ReadXml(xmlReader);

2) or continually per node thus lower the need of intermediate memory

regards,
Kate
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
AdnanAuthor Commented:
oki now it worked i think....its reding the file and restoring the backup, i dident get any exception but iam wondring that it still will give me axception when iam trying to read large files.....??!
0
_Katka_Commented:
Sure, if you will try to load a file that is higher than physical + virtual memory, but not in XML (not a size of XML file) but in binary data, then it should.. In that case I'd try to increase the size of my virtual memory, or try to avoid the DB to grow larger then that. If you'll be dealing with huge DBs like >10GB, I'd consider a different approach.. not storing it in a variable (myDsBackup), but on the disk.

regards,
Kate
0
_Katka_Commented:
By 'on the disk' I meant copy the source XML file to a backup file on the disk, and then copy it back, if something fails on the main DB.

regards,
Kate
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C#

From novice to tech pro — start learning today.