Solved

Read large XML file with Powershell

Posted on 2010-08-30
4
2,717 Views
Last Modified: 2013-12-04
I have a 500mb+ xml file I need to break apart, but every time I do Get-Content file it is too large for memory. Is there any way to process each node in the root one at a time similar to reading a stream?

The error: Exception of type 'System.OutOfMemoryException' was thrown.
0
Comment
Question by:RyanAndres
  • 2
4 Comments
 
LVL 65

Expert Comment

by:rockiroads
ID: 33561176
0
 
LVL 8

Author Comment

by:RyanAndres
ID: 33563459
No, it uses $xml = [xml](gc file) ....

This is what I tried and it will load the entire file into memory.. which will cause the exception.
0
 
LVL 70

Accepted Solution

by:
Chris Dent earned 500 total points
ID: 33565457

You should be able to initialise an instance of System.Xml.XmlReader using the StreamReader.

e.g.

$Reader = New-Object IO.StreamReader("FullPathToFile")
$XmlReader = [Xml.XmlReader]::Create($Reader)

Since it's reading it as a stream rather than loading the entire file into memory it "should" work better. However, I cannot test that theory since I don't have any large XML files.

Chris
0
 
LVL 8

Author Closing Comment

by:RyanAndres
ID: 33568419
Cool I used XmlTextReader and Writer.
0

Featured Post

Optimizing Cloud Backup for Low Bandwidth

With cloud storage prices going down a growing number of SMBs start to use it for backup storage. Unfortunately, business data volume rarely fits the average Internet speed. This article provides an overview of main Internet speed challenges and reveals backup best practices.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Create and license users in Office 365 in bulk based on a CSV file. A step-by-step guide with PowerShell script examples.
A brief introduction to what I consider to be the best editor for PowerShell.
Along with being a a promotional video for my three-day Annielytics Dashboard Seminor, this Micro Tutorial is an intro to Google Analytics API data.
This is Part 3 in a 3-part series on Experts Exchange to discuss error handling in VBA code written for Excel. Part 1 of this series discussed basic error handling code using VBA. http://www.experts-exchange.com/videos/1478/Excel-Error-Handlin…

770 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question