Script to Chomp a huge .txt file
Posted on 2012-03-28
I have a file located in C:\Temp named 'giantfile.txt'. It's about 2 million rows. The very first column has about 50 different values in it. I need to create smaller files from this file based on the value in that file. So if there are 50 different values I would end up with 50 text files etc...
Every row that contains the same value in that first column gets added to the new file. The new file will contain the name of the value in it. So if 'giantfile.txt' is:
Houston, Sam, Thompson, 303456789
Dallas, May, Jones, 123456789
Denver, Felix, Martinez, 8472346521
It would create 3 new files named: Houston.txt, Dallas.txt and Denver.txt and each file would contain the corresponding row. However, in real example each newly created file would be thousands of rows. The total rows in the new files added up would total the number of rows in 'giantfile.txt'. The destination of the new files would be C:\Temp\Completed
Prefer DOS or VB as I don't know Perl etc....
This is jus the first step. I will have other steps and will provide points necessary to achieve end goal. Hopefully we can work through this from beginning to end to ease the clients needs. They are stumbling around itn MS Access right now and I figured all this would be simple in scripting.....