Solved

Read large XML file with Powershell

Posted on 2010-08-30
4
2,811 Views
Last Modified: 2013-12-04
I have a 500mb+ xml file I need to break apart, but every time I do Get-Content file it is too large for memory. Is there any way to process each node in the root one at a time similar to reading a stream?

The error: Exception of type 'System.OutOfMemoryException' was thrown.
0
Comment
Question by:RyanAndres
  • 2
4 Comments
 
LVL 65

Expert Comment

by:rockiroads
ID: 33561176
0
 
LVL 8

Author Comment

by:RyanAndres
ID: 33563459
No, it uses $xml = [xml](gc file) ....

This is what I tried and it will load the entire file into memory.. which will cause the exception.
0
 
LVL 71

Accepted Solution

by:
Chris Dent earned 500 total points
ID: 33565457

You should be able to initialise an instance of System.Xml.XmlReader using the StreamReader.

e.g.

$Reader = New-Object IO.StreamReader("FullPathToFile")
$XmlReader = [Xml.XmlReader]::Create($Reader)

Since it's reading it as a stream rather than loading the entire file into memory it "should" work better. However, I cannot test that theory since I don't have any large XML files.

Chris
0
 
LVL 8

Author Closing Comment

by:RyanAndres
ID: 33568419
Cool I used XmlTextReader and Writer.
0

Featured Post

Industry Leaders: We Want Your Opinion!

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article will help you understand what HashTables are and how to use them in PowerShell.
This article explains how to prepare an HTML email signature template file containing dynamic placeholders for users' Azure AD data. Furthermore, it explains how to use this file to remotely set up a department-wide email signature policy in Office …
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an antispam), the admini…

685 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question