Handling very large data imports and exports with DataSet.WriteXml and ReadXml

I am investigating creating an import and export utility using the WriteXml and ReadXml methods on the DataSet class.

In my investigations I am trying to see if there is some way to batch up the data being processed, so that when exporting for example (WriteXml) there is a way to get the data in batches and process a batch at a time instead of trying to write all the data at once.

I can see the SqlDataAdapter has some batch properties and methods, but I can't find any explanation of how to use them if it they are suitable for what I want.

Essentially I would like to say "get xxx rows from the database, and write them to this file, then get the next xxx rows from the database and either append the existing file or write to a new file" and continue until all the rows in the table have been processed.
Who is Participating?
In previous version of DataAdapter (.Net 1.0 and 1.1) the SqlDataAdpater execute 1 command per DataRow. In ADO .Net 2.0 SqlDataAdapter has this porperty to set the number of concurrent commands to execute. You can set it to tell SqlDataAdapter to autoconfigure itself ( Default ) setting it to 0 or to you can set the Window Size to any number, for example 5000.
If you are building an Importer tool, i recomend the Integration Services of Sql Server 2005, it has a very performance set of componentes to do every thing than you want.
Question has a verified solution.

Are you are experiencing a similar issue? Get a personalized answer when you ask a related question.

Have a better answer? Share it in a comment.

All Courses

From novice to tech pro — start learning today.