Solved

VBSCRIPT Parse Large file into smaller files based on header info

Posted on 2009-04-08
7
595 Views
Last Modified: 2012-05-06
I need help creating a script that will read a large file that contains multiple records (no CR/LF) and, based on a known header ("1MHG"),  will create individual files.  The records can be long so I'm not sure using arrays will work.
Example: the file "ALL.TXT" contains 45 records in one long string. They all start with the header info "1MHG". Knowing that, we can find the start of the next record, because it will also start with "1MHG".  We do not know how the record ends.  I need to be able to strip each record, from the beginning of the "1MHG" occurance to the next occurance, and write 45 files. Each file will be named sequentially (file1,txt, file2.txt, etc). I have the code that opens the file and creates the output and names them sequentially. I just cannot get the part that parses the string and strips out the records.
Thanks
0
Comment
Question by:mannyms
  • 4
  • 3
7 Comments
 
LVL 25

Accepted Solution

by:
kevp75 earned 500 total points
Comment Utility
post your code?

basically what you'd be looking to do is a split on the string, and then loop through the array created.  So long as your machine doesn't have only 256M of RAM you should be all set with this method ;)

strVar = Split(objFile.ReadAll, "1MHG")
If IsArray(strVar) Then
     For i = 0 To Ubound(strVar) - 1
         set blah = blahblah.CreateTextFile(named....txt)
             blah.WriteLine(strVat(i))
         set blah = Nothing
      Next : i = Null
End If


(this is untested code, and the variables are unknow...please change what is necessary)
0
 
LVL 25

Expert Comment

by:kevp75
Comment Utility
is the header for the files always on the same line?   You can read in the first line for the value of the split...
0
 

Author Comment

by:mannyms
Comment Utility
The file is one Loooooong physical record with multiple logical records embedded, but they always start with the string "1MHG". They can be really large as well. My test file has only 45 records but is 1MB. We are expecting files that could potentially contain 100s if not 1000s or logical records. So the importance of doing this optimally is important.  
0
Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

 
LVL 25

Expert Comment

by:kevp75
Comment Utility
can you post your existing code?   Let's see if we can modify it to handle this.

Take out (or x out) anything incriminating :)
0
 

Author Comment

by:mannyms
Comment Utility
This is working well except that the "1MHG" is being stripped out of the array.

Dim objFSO, objFolder, objShell, objInputFile, objOutput, objFile, count_r

Dim strDirectory, strInputFile, strOutputFile

count_r = 1

tmpDate = Format(Date, "mmddyy")

tmpTime = Format(Time, "hhmmss")

'---determine actual input/output locations later---

strInputFile = "c:\all.al3"

strDirectory = "c:\AL3_" & tmpDate & tmpTime

strOutputFile = "\file" & count_r & ".al3"

'---------------------------------------------------

Set objFSO = CreateObject("Scripting.FileSystemObject")
 

If objFSO.FolderExists(strDirectory) Then

   Set objFolder = objFSO.GetFolder(strDirectory)

Else

   Set objFolder = objFSO.CreateFolder(strDirectory)

End If
 

Set objInputFile = Nothing

Set objOutputFile = Nothing

Set objFolder = Nothing
 

Const ForAppending = 8, ForReading = 1, ForWriting = 2
 

Set objInputFile = objFSO.OpenTextFile(strInputFile, ForReading)
 

strVar = Split(objInputFile.ReadAll, "1MHG")

If IsArray(strVar) Then

     For i = 1 To UBound(strVar)

        strOutputFile = "\file" & count_r & ".al3"

        If objFSO.FileExists(strDirectory & strOutputFile) Then

            '----add code = delete the file and recreate it.

            Set objFolder = objFSO.GetFolder(strDirectory)

        Else

            

            Set objOutputFile = objFSO.CreateTextFile(strDirectory & strOutputFile)

        End If

             objOutputFile.WriteLine (strVar(i))

             objOutputFile.Close

            Set objOutputFile = Nothing

            count_r = count_r + 1

      Next: i = Null

End If

Open in new window

0
 

Author Comment

by:mannyms
Comment Utility
Just got updated to a spec issue:(will create separate post if needed)
within the array for each record, starting at position 69 for 6 bytes, I must replace the existing data (almost always spaces) with "AWF   ". Positions 1-68 and positions 72 thru the end (variable) are written as is.

Example
(See code snippet for fixed length font)
1MHG161...IBM732PROFORM.....IBM9084212NJ................APPPAC............107X11

1MHG161...IBM732PROFORM.....IBM9084212NJ................APPPAC......AFW...107X11

--------------------------------------------------------------------^^^---------

Open in new window

0
 

Author Closing Comment

by:mannyms
Comment Utility
Provided the direction needed to complete. Excellent contributor.
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

I recently decide that I needed a way to make my pages scream on the net.   While searching around how I can accomplish this I stumbled across a great article that stated "minimize the server requests." I got to thinking, hey, I use more than one…
Well hello again!  Glad to see you've made it this far without giving up.  In this, the fourth installment of my popular series, I'm going to cover functions and subroutines, what they are, and why they are useful.  Just in case you stumbled onto th…
Excel styles will make formatting consistent and let you apply and change formatting faster. In this tutorial, you'll learn how to use Excel's built-in styles, how to modify styles, and how to create your own. You'll also learn how to use your custo…
In this seventh video of the Xpdf series, we discuss and demonstrate the PDFfonts utility, which lists all the fonts used in a PDF file. It does this via a command line interface, making it suitable for use in programs, scripts, batch files — any pl…

772 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

10 Experts available now in Live!

Get 1:1 Help Now