Error reading xml

I am reading a huge xml file in my application. After reading few nodes I get

TransportRouteManager::interpretTransportRoute: Managed Exception: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. StackTrace:    at System.Net.GZipWrapperStream.Read(Byte[] buffer, Int32 offset, Int32 size)

   at System.Xml.XmlTextReaderImpl.ReadData()

   at System.Xml.XmlTextReaderImpl.ParseAttributeValueSlow(Int32 curPos, Char quoteChar, NodeData attr)

   at System.Xml.XmlTextReaderImpl.ParseAttributes()

   at System.Xml.XmlTextReaderImpl.ParseElement()

   at System.Xml.XmlTextReaderImpl.ParseElementContent()

   at System.Xml.XmlTextReaderImpl.Read()

   at System.Xml.XmlSubtreeReader.Read()

Open in new window


Unfortunately its random behavior. Can anyone help with this error please?
jim_imtechAsked:
Who is Participating?
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sarabandeCommented:
do you read the xml file from network or is it on a local disk?

how much is 'huge' and is it a 64-bit or a 32-bit windows?

note, an xml parser needs contiguous memory for parsing a subtree. if your system runs low on heap memory or many programs are running same time, a huge xml file could exhaust your resources.

Sara
0
jim_imtechAuthor Commented:
I read the downloaded the xml from internet using WbResp->GetResponseStream(). The xml is around 50 MB in size running on 32 bit windows. There is only this program running and it sometimes gives an error after an hour and sometimes within 15 mins.

Is this error because of memory?
0
sarabandeCommented:
yes, I would assume so.  on a 32-bit windows the total heapsize probably is less than 2 gb what is enough for 50mb files but not if the heap was heavily used for example by streaming operations of varying lengths. do you store the stream to a (temporary) local file or are you parsing directly from stream? if doing the latter you may consider to make it two steps, or even use a separate program/service for download. that has the advantage that all memory used for download was freed when you start parsing cause you could exit the download program before.

Sara
0

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
jim_imtechAuthor Commented:
Please guide me as to how to save Stream^ to file and then use the file? Even if I save the url to a file and then load the into a stream again to read, would the memory problem not be the same?
0
sarabandeCommented:
Even if I save the url to a file and then load the into a stream again to read, would the memory problem not be the same?
yes, it could be. however, the download itself would need resources additionally to that of your program. this part should be freed when you make it in two steps.

Please guide me as to how to save Stream^ to file and then use the file?
I am not experienced in writing managed code. you would use a FileStream object as far as I know. please look at the c# sample code at
http://msdn.microsoft.com/en-us/library/vstudio/36b93480(v=vs.90).aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1

Sara
0
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.