Go Premium for a chance to win a PS4. Enter to Win

x
?
Solved

Read large XML file with Powershell

Posted on 2010-08-30
4
Medium Priority
?
3,217 Views
Last Modified: 2013-12-04
I have a 500mb+ xml file I need to break apart, but every time I do Get-Content file it is too large for memory. Is there any way to process each node in the root one at a time similar to reading a stream?

The error: Exception of type 'System.OutOfMemoryException' was thrown.
0
Comment
Question by:RyanAndres
  • 2
4 Comments
 
LVL 65

Expert Comment

by:rockiroads
ID: 33561176
0
 
LVL 8

Author Comment

by:RyanAndres
ID: 33563459
No, it uses $xml = [xml](gc file) ....

This is what I tried and it will load the entire file into memory.. which will cause the exception.
0
 
LVL 71

Accepted Solution

by:
Chris Dent earned 2000 total points
ID: 33565457

You should be able to initialise an instance of System.Xml.XmlReader using the StreamReader.

e.g.

$Reader = New-Object IO.StreamReader("FullPathToFile")
$XmlReader = [Xml.XmlReader]::Create($Reader)

Since it's reading it as a stream rather than loading the entire file into memory it "should" work better. However, I cannot test that theory since I don't have any large XML files.

Chris
0
 
LVL 8

Author Closing Comment

by:RyanAndres
ID: 33568419
Cool I used XmlTextReader and Writer.
0

Featured Post

Veeam Disaster Recovery in Microsoft Azure

Veeam PN for Microsoft Azure is a FREE solution designed to simplify and automate the setup of a DR site in Microsoft Azure using lightweight software-defined networking. It reduces the complexity of VPN deployments and is designed for businesses of ALL sizes.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Simulator games are perfect for generating sample realistic data streams, especially for learning data analysis. It is even useful for demoing offerings such as Azure stream analytics, PowerBI etc.
Measuring Server's processing rate with a simple powershell command. The differences in processing rate also was recorded in different use-cases, when a server in free and busy states.
Exchange organizations may use the Journaling Agent of the Transport Service to archive messages going through Exchange. However, if the Transport Service is integrated with some email content management application (such as an anti-spam), the admin…
Loops Section Overview

916 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question