Solved

vb.net - merge files

Posted on 2013-05-16
2
301 Views
Last Modified: 2013-05-30
Hello,
I have about 30 txt files, each file is about 5GB in size.
these text files are basically huge percause they are not compressed and they contains records line by line like this

name1560
name16598
name89865

I would like to know what would be the best approach to merge all these files into one?
0
Comment
Question by:XK8ER
2 Comments
 
LVL 69

Expert Comment

by:Éric Moreau
ID: 39174229
to get a 15 gb file? isn't it too big?
0
 
LVL 15

Accepted Solution

by:
angus_young_acdc earned 500 total points
ID: 39174315
That's going to be a massive file!  

Anyway what I would do is split the files down further so that any reads can manage the load.  Then, read each of the smaller files using a StreamReader and write the contents to a single file using StreamWriter (append true).  

It may struggle somewhat as it gets into the 10's of GB, but I've never had to create a text file so large so I can't confirm that.
0

Featured Post

3 Use Cases for Connected Systems

Our Dev teams are like yours. They’re continually cranking out code for new features/bugs fixes, testing, deploying, testing some more, responding to production monitoring events and more. It’s complex. So, we thought you’d like to see what’s working for us.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Suggested Solutions

Title # Comments Views Activity
Combine two datatables into one based on a value from second datatable 4 33
Data is not showing from images 15 37
Syntax Error 2 46
Visual Studio 2015 Source Code Control 10 36
It was really hard time for me to get the understanding of Delegates in C#. I went through many websites and articles but I found them very clumsy. After going through those sites, I noted down the points in a easy way so here I am sharing that unde…
This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
Along with being a a promotional video for my three-day Annielytics Dashboard Seminor, this Micro Tutorial is an intro to Google Analytics API data.
As a trusted technology advisor to your customers you are likely getting the daily question of, ‘should I put this in the cloud?’ As customer demands for cloud services increases, companies will see a shift from traditional buying patterns to new…

920 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

19 Experts available now in Live!

Get 1:1 Help Now