Read large XML file with Powershell

Posted on 2010-08-30
Last Modified: 2013-12-04
I have a 500mb+ xml file I need to break apart, but every time I do Get-Content file it is too large for memory. Is there any way to process each node in the root one at a time similar to reading a stream?

The error: Exception of type 'System.OutOfMemoryException' was thrown.
Question by:RyanAndres
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
LVL 65

Expert Comment

ID: 33561176

Author Comment

ID: 33563459
No, it uses $xml = [xml](gc file) ....

This is what I tried and it will load the entire file into memory.. which will cause the exception.
LVL 71

Accepted Solution

Chris Dent earned 500 total points
ID: 33565457

You should be able to initialise an instance of System.Xml.XmlReader using the StreamReader.


$Reader = New-Object IO.StreamReader("FullPathToFile")
$XmlReader = [Xml.XmlReader]::Create($Reader)

Since it's reading it as a stream rather than loading the entire file into memory it "should" work better. However, I cannot test that theory since I don't have any large XML files.


Author Closing Comment

ID: 33568419
Cool I used XmlTextReader and Writer.

Featured Post

How our DevOps Teams Maximize Uptime

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us. Read the use case whitepaper.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Synchronize a new Active Directory domain with an existing Office 365 tenant
Recently we ran in to an issue while running some SQL jobs where we were trying to process the cubes.  We got an error saying failure stating 'NT SERVICE\SQLSERVERAGENT does not have access to Analysis Services. So this is a way to automate that wit…
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA.…
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…

752 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question