• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 454
  • Last Modified:

Parsing complicated flat files with C#

I have a task to read in various flat files with C# into a structured database table. The user will be going to a website to upload the file.  After uploading the file the C# code will parse the file breaking it up into variables to be inserted into an Oracle DB.   I am trying to come up with the most efficient solution to read in these files. The files are structured with a multiline headers and data that follows. I have read in csv files before but those were very straight forward and didn't have multiple data representations in the files.  I have included a couple examples of the files below.  I have been looking closely at using regular expressions to somehow read in this data but I am unsure whether this is the best approach.   Could someone give me some insight and/or examples as to the best approach.  The file sizes are going to go all the way up to 150MB so the user is already going to have to wait for an upload of these files.  I don't want them to have to wait much long to process them into the database.

This is the first file.  It contains a multiline header, a definition section and then a log section.  Logs can be from Log#:1 to Log#:n.  The log file lists an entry for each definition (e.g. "B:1")
***********************************************************************************************************

NAME:VMSMON
MODEL:4444
SWPN:xxxxx
LISTNAME:test
AC:    22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:xxxxx
PN:[Hss #2 Test]:xxxxx

DEFINITIONS:
DEF:B:1:30,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 1P
DEF:B:2:30,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:3:30,1520r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 6P
DEF:B:4:37,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 1P
DEF:B:5:37,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:6:37,1520r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 6P
DEF:B:7:44,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 1P
DEF:B:8:44,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 3P
DEF:B:9:44,1530r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 6P
DEF:B:10:51,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 1P
DEF:B:11:51,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 3P
DEF:B:12:51,1530r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Aft Rotor 6P
DEF:B:13:33,19050r,21225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 HSS 1P
DEF:B:14:33,16200r,17550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 HSSResonance
DEF:B:15:33,7350r,7650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 1P
DEF:B:16:33,14700r,15300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 2P
DEF:B:17:33,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:18:33,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:19:14,19050r,21225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 HSS 1P
DEF:B:20:14,16200r,17550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 HSSResonance
DEF:B:21:14,7200r,7350r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 #5BRG-FTF
DEF:B:22:14,8400r,8550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 #4BRG-FTF
DEF:B:23:14,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 1P
DEF:B:24:14,22350r,23325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 80Deg 1P
DEF:B:25:14,3825r,4050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Flex/Nfgov 1P
DEF:B:26:14,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Fuel Imp 1P
DEF:B:27:14,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1AxisC/D/Grot1P
DEF:B:28:44,6000r,6300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Blower 1P
DEF:B:29:44,12000r,12600r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Blower 2P
DEF:B:30:51,6000r,6300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Blower 1P
DEF:B:31:51,12000r,12600r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Blower 2P
DEF:B:32:33,38100r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 HSS 2P
DEF:B:33:33,65250r,68250r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM
DEF:B:34:14,38100r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 HSS 2P
DEF:B:35:14,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 2P
DEF:B:36:14,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 3P
DEF:B:37:14,107250r,114000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Flex/Nftac GM
DEF:B:38:45,30300r,31500r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APGM
DEF:B:39:45,60600r,63000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APGM 2P
DEF:B:40:52,30300r,31500r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APGM
DEF:B:41:52,60600r,63000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APGM 2P
DEF:B:42:31,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Planet 1P
DEF:B:43:31,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:44:31,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:45:36,19050r,21225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 HSS 1P
DEF:B:46:36,16200r,17550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 HSSResonance
DEF:B:47:36,7200r,7350r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 #5BRG-FTF
DEF:B:48:36,8400r,8550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 #4BRG-FTF
DEF:B:49:36,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 1P
DEF:B:50:36,22350r,23325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 80Deg 1P
DEF:B:51:36,3825r,4050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Flex/NfGov 1P
DEF:B:52:36,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Fuel Imp 1P
DEF:B:53:36,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2AxisC/D/Grot1P
DEF:B:54:45,7350r,7650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 1P
DEF:B:55:45,14700r,15300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 2P
DEF:B:56:45,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Planet 1P
DEF:B:57:45,4350r,4650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),SCAVP 1P
DEF:B:58:45,8775r,9225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),SCAVP 2P
DEF:B:59:45,3975r,4125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),2FCBP 1P
DEF:B:60:45,2550r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),ATLP 1P
DEF:B:61:45,5100r,5325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),ATLP 2P
DEF:B:62:45,6675r,7050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),AUHP 1P
DEF:B:63:45,13425r,14025r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),AUHP 2P
DEF:B:64:45,7800r,8400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),EG 1P
DEF:B:65:45,15600r,16800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),EG 2P
DEF:B:66:45,19080r,19800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),HSS 1P
DEF:B:67:52,7350r,7650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 1P
DEF:B:68:52,14700r,15300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 2P
DEF:B:69:52,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Planet 1P
DEF:B:70:52,4350r,4650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),SCAVP 1P
DEF:B:71:52,8775r,9225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),SCAVP 2P
DEF:B:72:52,3975r,4125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),2FCBP 1P
DEF:B:73:52,2550r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),ATLP 1P
DEF:B:74:52,5100r,5325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),ATLP 2P
DEF:B:75:52,6675r,7050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),AUHP 1P
DEF:B:76:52,13425r,14025r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),AUHP 2P
DEF:B:77:52,7800r,8400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),EG 1P
DEF:B:78:52,15600r,16800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),EG 2P
DEF:B:79:52,19080r,19800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),HSS 1P
DEF:B:80:31,30300r,31500r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),FPGM
DEF:B:81:31,60600r,63000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),FPGM 2P
DEF:B:82:31,65250r,68250r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM
DEF:B:83:36,38100r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 HSS 2P
DEF:B:84:36,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 2P
DEF:B:85:36,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 3P
DEF:B:86:36,107250r,114000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Flex/Nftac GM
DEF:B:87:46,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 2P
DEF:B:88:46,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 3P
DEF:B:89:46,33600r,43800r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Gerotor GM
DEF:B:90:46,23550r,30675r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Lube Pump 7
DEF:B:91:46,30000r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Lube Pump 9
DEF:B:92:53,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 2P
DEF:B:93:53,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 3P
DEF:B:94:53,33600r,43800r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Gerotor GM
DEF:B:95:53,23550r,30675r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Lube Pump 7
DEF:B:96:53,30000r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Lube Pump 9
DEF:B:97:33,186000r,193500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 1P
DEF:B:98:33,372000r,385500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 2P
DEF:B:99:33,130500r,135750r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM 2P
DEF:B:100:40,186000r,193500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 1P
DEF:B:101:40,372000r,385500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 2P
DEF:B:102:40,130500r,135750r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM 2P
DEF:B:103:45,186000r,193500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 1P
DEF:B:104:45,372000r,385500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 2P
DEF:B:105:45,537000r,552000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),Mixidler GM 1P
DEF:B:106:52,186000r,193500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 1P
DEF:B:107:52,372000r,385500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),CG GM 2P
DEF:B:108:52,537000r,552000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),Mixidler GM 1P
DEF:B:109:32,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 123 Brg
DEF:B:110:32,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 1P
DEF:B:111:32,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Fuel Imp 1P
DEF:B:112:32,5175r,6825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 AxisB 1P
DEF:B:113:32,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1AxisC/D/Grot1P
DEF:B:114:32,14850r,18975r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 RDS 1P
DEF:B:115:32,1275r,1800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis C FTF
DEF:B:116:32,1350r,1875r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis D FTF
DEF:B:117:32,2025r,2775r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis B FTF
DEF:B:118:40,19050r,21225r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 HSS 1P
DEF:B:119:40,16200r,17550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 HSSResonance
DEF:B:120:40,7350r,7650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 1P
DEF:B:121:40,14700r,15300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Spur Gear 2P
DEF:B:122:40,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:123:40,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:124:46,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 123 Brg
DEF:B:125:46,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 1P
DEF:B:126:46,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Fuel Imp 1P
DEF:B:127:46,5175r,6825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 AxisB 1P
DEF:B:128:46,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2AxisC/D/Grot1P
DEF:B:129:46,14850r,18975r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 RDS 1P
DEF:B:130:46,1275r,1800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis C FTF
DEF:B:131:46,1350r,1875r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis D FTF
DEF:B:132:46,2025r,2775r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis B FTF
DEF:B:133:53,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 123 Brg
DEF:B:134:53,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 1P
DEF:B:135:53,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Fuel Imp 1P
DEF:B:136:53,5175r,6825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 AxisB 1P
DEF:B:137:53,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2AxisC/D/Grot1P
DEF:B:138:53,14850r,18975r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 RDS 1P
DEF:B:139:53,1275r,1800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis C FTF
DEF:B:140:53,1350r,1875r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis D FTF
DEF:B:141:53,2025r,2775r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Axis B FTF
DEF:B:142:32,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 2P
DEF:B:143:32,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 3P
DEF:B:144:32,33600r,43800r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Gerotor GM
DEF:B:145:32,23550r,30675r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Lube Pump 7
DEF:B:146:32,30000r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Lube Pump 9
DEF:B:147:40,38100r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 HSS 2P
DEF:B:148:40,65250r,68250r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM
DEF:B:149:47,55500r,57750r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APU 1P
DEF:B:150:47,111000r,115500r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),APU 2P
DEF:B:151:7,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 2P
DEF:B:152:7,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 3P
DEF:B:153:38,30300r,31500r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),FPGM
DEF:B:154:38,60600r,63000r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),FPGM 2P
DEF:B:155:38,65250r,68250r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM
DEF:B:156:31,130500r,135750r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM 2P
DEF:B:157:38,130500r,135750r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),SBP GM 2P
DEF:B:158:46,141000r,184500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Axis B/C GM
DEF:B:159:46,193500r,252000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2RDS AxisB GM
DEF:B:160:53,141000r,184500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Axis B/C GM
DEF:B:161:53,193500r,252000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2RDS AxisB GM
DEF:B:162:7,7200r,7350r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 #5BRG-FTF
DEF:B:163:7,8400r,8550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 #4BRG-FTF
DEF:B:164:7,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 1P
DEF:B:165:7,22350r,23325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 80Deg 1P
DEF:B:166:7,3825r,4050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Flex/Nfgov 1P
DEF:B:167:7,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 123 Brg
DEF:B:168:38,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Planet 1P
DEF:B:169:38,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:170:38,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:171:47,7350r,8250r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),APU Zerol 1P
DEF:B:172:47,3450r,3900r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),APU Int Gear 1P
DEF:B:173:47,3900r,4350r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),APU Acc Gear 1P
DEF:B:174:47,5100r,5700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),APU Oil Gear 1P
DEF:B:175:27,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:176:27,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:177:27,7200r,8100r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 3P
DEF:B:178:27,750r,825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 3P
DEF:B:179:27,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 6P
DEF:B:180:31,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:181:38,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:182:49,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 1P
DEF:B:183:49,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:184:49,1520r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 6P
DEF:B:185:28,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 1P
DEF:B:186:28,765r,810r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 3P
DEF:B:187:28,1520r,1620r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 6P
DEF:B:188:32,141000r,184500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Axis B/C GM
DEF:B:189:32,193500r,252000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 RDS AxisB GM
DEF:B:190:39,141000r,184500r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Axis B/C GM
DEF:B:191:39,193500r,252000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 RDS AxisB GM
DEF:B:192:29,7200r,7350r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 #5BRG-FTF
DEF:B:193:29,8400r,8550r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 #4BRG-FTF
DEF:B:194:29,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 1P
DEF:B:195:29,22350r,23325r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 80Deg 1P
DEF:B:196:29,3825r,4050r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 Flex/NfGov 1P
DEF:B:197:29,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 123 Brg
DEF:B:198:39,8250r,11400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 123 Brg
DEF:B:199:39,21000r,27300r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 1P
DEF:B:200:39,5400r,7125r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Fuel Imp 1P
DEF:B:201:39,5175r,6825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 AxisB 1P
DEF:B:202:39,3600r,4425r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1AxisC/D/Grot1P
DEF:B:203:39,14850r,18975r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 RDS 1P
DEF:B:204:39,1275r,1800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis C FTF
DEF:B:205:39,1350r,1875r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis D FTF
DEF:B:206:39,2025r,2775r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 Axis B FTF
DEF:B:207:15,7800r,8400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 EG 1P
DEF:B:208:15,15600r,16800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#2 EG 2P
DEF:B:209:29,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 2P
DEF:B:210:29,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Comp GG 3P
DEF:B:211:39,42000r,54600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 2P
DEF:B:212:39,63000r,81900r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Comp GG 3P
DEF:B:213:39,33600r,43800r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Gerotor GM
DEF:B:214:39,23550r,30675r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Lube Pump 7
DEF:B:215:39,30000r,39600r,120000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Lube Pump 9
DEF:B:216:7,107250r,114000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#1 Flex/Nftac GM
DEF:B:217:29,107250r,114000r,600000r,1;0r,1,800,180,N,ips,F,1,1,N,(1,,,),#2 Flex/Nftac GM
DEF:B:218:1,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:219:1,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:220:1,7200r,8100r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 3P
DEF:B:221:1,750r,825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 3P
DEF:B:222:1,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 6P
DEF:B:223:8,7800r,8400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 EG 1P
DEF:B:224:8,15600r,16800r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),#1 EG 2P
DEF:B:225:20,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:226:20,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:227:20,7200r,8100r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 3P
DEF:B:228:20,750r,825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 3P
DEF:B:229:20,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 6P
DEF:B:230:13,2475r,2700r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 1P
DEF:B:231:13,4800r,5400r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 2P
DEF:B:232:13,7200r,8100r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Synch 3P
DEF:B:233:13,750r,825r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 3P
DEF:B:234:13,1500r,1650r,30000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Rotor 6P
DEF:B:235:49,19080r,19800r,60000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),HSS 1P
DEF:B:236:28,19080r,19800r,60000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),HSS 1P
DEF:UNITINDEX:1,PeakIPS,0.001000
ENDDEF:
LOG#: 1
DATE:03-04-2007
TIME:18:39:02
REG:0
ACH:1,-1
B:1,0,-1,-1,-1
B:2,0,-1,-1,-1
B:3,0,-1,-1,-1
B:4,17,-1,-1,-1
B:5,0,-1,-1,-1
B:6,0,-1,-1,-1
B:7,0,-1,-1,-1
B:8,0,-1,-1,-1
B:9,0,-1,-1,-1
B:10,0,-1,-1,-1
B:11,0,-1,-1,-1
B:12,0,-1,-1,-1
ACH:1,-1
B:13,2,-1,-1,-1
B:14,1,233,-1,-1
B:15,0,-1,-1,-1
B:16,0,-1,-1,-1
B:17,0,-1,-1,-1
B:18,0,-1,-1,-1
B:19,0,-1,-1,-1
B:20,0,216,-1,-1
B:21,0,-1,-1,-1
B:22,0,-1,-1,-1
B:23,0,280,-1,-1
B:24,0,-1,-1,-1
B:25,0,-1,-1,-1
B:26,0,-1,-1,-1
B:27,0,-1,-1,-1
B:28,0,-1,-1,-1
B:29,0,-1,-1,-1
B:30,0,-1,-1,-1
B:31,7,-1,-1,-1
ACH:1,-1
B:32,0,-1,-1,-1
B:33,0,-1,-1,-1
B:34,0,-1,-1,-1
B:35,0,-1,-1,-1
B:36,0,-1,-1,-1
B:37,0,-1,-1,-1
B:38,0,-1,-1,-1
B:39,0,-1,-1,-1
B:40,0,-1,-1,-1
B:41,0,-1,-1,-1
ACH:1,-1
B:42,0,-1,-1,-1
B:43,0,-1,-1,-1
B:44,0,-1,-1,-1
B:45,0,-1,-1,-1
B:46,0,216,-1,-1
B:47,0,-1,-1,-1
B:48,0,-1,-1,-1
B:49,0,280,-1,-1
B:50,0,-1,-1,-1
B:51,0,-1,-1,-1
B:52,0,-1,-1,-1
B:53,0,-1,-1,-1
B:54,0,-1,-1,-1
B:55,0,-1,-1,-1
B:56,0,-1,-1,-1
B:57,0,-1,-1,-1
B:58,0,-1,-1,-1
B:59,0,-1,-1,-1
B:60,0,-1,-1,-1
B:61,0,-1,-1,-1
B:62,0,-1,-1,-1
B:63,1,-1,-1,-1
B:64,3,-1,-1,-1
B:65,0,-1,-1,-1
B:66,0,-1,-1,-1
B:67,0,-1,-1,-1
B:68,0,-1,-1,-1
B:69,0,-1,-1,-1
B:70,0,-1,-1,-1
B:71,0,-1,-1,-1
B:72,0,-1,-1,-1
B:73,0,-1,-1,-1
B:74,0,-1,-1,-1
B:75,0,-1,-1,-1
B:76,2,-1,-1,-1
B:77,4,-1,-1,-1
B:78,0,-1,-1,-1
B:79,0,-1,-1,-1
ACH:1,-1
B:80,0,-1,-1,-1
B:81,0,-1,-1,-1
B:82,0,-1,-1,-1
B:83,0,-1,-1,-1
B:84,0,-1,-1,-1
B:85,6,-1,-1,-1
B:86,0,-1,-1,-1
B:87,0,-1,-1,-1
B:88,0,-1,-1,-1
B:89,0,-1,-1,-1
B:90,0,-1,-1,-1
B:91,0,-1,-1,-1
B:92,0,-1,-1,-1
B:93,0,-1,-1,-1
B:94,0,-1,-1,-1
B:95,0,-1,-1,-1
B:96,0,-1,-1,-1
ACH:1,-1
B:97,0,248,-1,-1
B:98,0,-1,-1,-1
B:99,0,-1,-1,-1
B:100,0,248,-1,-1
B:101,0,-1,-1,-1
B:102,0,-1,-1,-1
B:103,0,248,-1,-1
B:104,0,-1,-1,-1
B:105,0,-1,-1,-1
B:106,0,248,-1,-1
B:107,0,-1,-1,-1
B:108,0,-1,-1,-1
ACH:1,-1
B:109,18,-1,-1,-1
B:110,23,305,-1,-1
B:111,86,-1,-1,-1
B:112,86,-1,-1,-1
B:113,2,-1,-1,-1
B:114,6,-1,-1,-1
B:115,15,-1,-1,-1
B:116,15,-1,-1,-1
B:117,7,-1,-1,-1
B:118,0,-1,-1,-1
B:119,1,216,-1,-1
B:120,0,-1,-1,-1
B:121,0,-1,-1,-1
B:122,0,-1,-1,-1
B:123,1,-1,-1,-1
B:124,0,-1,-1,-1
B:125,0,280,-1,-1
B:126,18,-1,-1,-1
B:127,18,-1,-1,-1
B:128,0,-1,-1,-1
B:129,0,-1,-1,-1
B:130,18,-1,-1,-1
B:131,18,-1,-1,-1
B:132,2,-1,-1,-1
B:133,0,-1,-1,-1
B:134,0,280,-1,-1
B:135,5,-1,-1,-1
B:136,5,-1,-1,-1
B:137,3,-1,-1,-1
B:138,0,-1,-1,-1
B:139,0,-1,-1,-1
B:140,0,-1,-1,-1
B:141,0,-1,-1,-1
ACH:1,-1
B:142,0,-1,-1,-1
B:143,0,-1,-1,-1
B:144,9,-1,-1,-1
B:145,20,-1,-1,-1
B:146,9,-1,-1,-1
B:147,1,-1,-1,-1
B:148,0,-1,-1,-1
B:149,46,-1,-1,-1
B:150,11,-1,-1,-1
ACH:1,-1
B:151,4,-1,-1,-1
B:152,19,-1,-1,-1
B:153,0,-1,-1,-1
B:154,0,-1,-1,-1
B:155,0,-1,-1,-1
ACH:1,-1
B:156,0,-1,-1,-1
B:157,0,-1,-1,-1
B:158,0,-1,-1,-1
B:159,0,-1,-1,-1
B:160,0,-1,-1,-1
B:161,0,-1,-1,-1
ACH:1,-1
B:162,4,-1,-1,-1
B:163,1,-1,-1,-1
B:164,6,330,-1,-1
B:165,5,-1,-1,-1
B:166,1,-1,-1,-1
B:167,23,-1,-1,-1
B:168,1,-1,-1,-1
B:169,0,-1,-1,-1
B:170,0,-1,-1,-1
B:171,244,-1,-1,-1
B:172,16,-1,-1,-1
B:173,32,-1,-1,-1
B:174,15,-1,-1,-1
B:175,0,-1,-1,-1
B:176,2,-1,-1,-1
B:177,1,-1,-1,-1
B:178,1,-1,-1,-1
B:179,16,-1,-1,-1
ACH:1,-1
B:180,0,-1,-1,-1
B:181,0,-1,-1,-1
B:182,0,-1,-1,-1
B:183,0,-1,-1,-1
B:184,0,-1,-1,-1
B:185,0,-1,-1,-1
B:186,0,-1,-1,-1
B:187,0,-1,-1,-1
ACH:1,-1
B:188,11,-1,-1,-1
B:189,24,-1,-1,-1
B:190,14,-1,-1,-1
B:191,9,-1,-1,-1
ACH:1,-1
B:192,0,-1,-1,-1
B:193,5,-1,-1,-1
B:194,4,291,-1,-1
B:195,0,-1,-1,-1
B:196,0,-1,-1,-1
B:197,6,-1,-1,-1
B:198,19,-1,-1,-1
B:199,30,304,-1,-1
B:200,9,-1,-1,-1
B:201,13,-1,-1,-1
B:202,10,-1,-1,-1
B:203,2,-1,-1,-1
B:204,6,-1,-1,-1
B:205,6,-1,-1,-1
B:206,1,-1,-1,-1
B:207,10,-1,-1,-1
B:208,0,-1,-1,-1
ACH:1,-1
B:209,23,-1,-1,-1
B:210,14,-1,-1,-1
B:211,21,-1,-1,-1
B:212,12,-1,-1,-1
B:213,9,-1,-1,-1
B:214,47,-1,-1,-1
B:215,4,-1,-1,-1
ACH:1,-1
B:216,5,-1,-1,-1
ACH:1,-1
B:217,1,-1,-1,-1
ACH:1,-1
B:218,24,-1,-1,-1
B:219,31,-1,-1,-1
B:220,12,-1,-1,-1
B:221,24,-1,-1,-1
B:222,52,-1,-1,-1
B:223,3,-1,-1,-1
B:224,8,-1,-1,-1
B:225,29,-1,-1,-1
B:226,8,-1,-1,-1
B:227,10,-1,-1,-1
B:228,46,-1,-1,-1
B:229,265,-1,-1,-1
ACH:1,243
B:230,79,-1,-1,-1
B:231,14,-1,-1,-1
B:232,29,-1,-1,-1
B:233,176,-1,-1,-1
B:234,139,-1,-1,-1
ACH:1,242
B:235,0,-1,-1,-1
B:236,0,-1,-1,-1

***********************************************************************************************************
This is the second file. It is in a linear format with the log records separated by newlines
***********************************************************************************************************

NAME:VMSSPECT
MODEL:xxxxxx
SWPN:22222
LISTNAME:xxxxx
LISTID:387
LISTVER:1.150
AC:    22222
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111


LOG#: 1
LTYP:A
TID:Monitor
ADVIS: 1
AC_WT:    
C/G:    
FLT_HRS:      
OPER:  
CODE:  
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:21:10:27
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS:  ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:266
BINS:400
0
0
286
356
295
263
211
160
163
431
602
595
543
257
117
97
40
6
16
47
83
79
80
76
47
44
41
27
39
31
84
174
367
701
764
718
516
151
7
20
15
37
54
62
58
22
14
19
19
15
19
51
90
89
84
47
11
10
29
40
34
32
51
60
58
49
21
19
26
30
34
33
34
43
47
44
32
12
1
0
2
3
5
46
90
84
66
22
27
31
17
9
10
23
93
143
159
147
72
15
1
27
55
65
71
78
94
133
130
84
55
31
31
39
44
63
101
106
95
40
10
22
17
16
12
26
64
81
80
63
24
26
19
8
5
0
51
116
127
127
88
35
27
21
20
15
12
60
85
81
75
37
29
37
28
23
24
5
30
44
51
52
26
6
5
13
15
11
27
67
84
91
69
50
39
33
37
39
32
20
3
9
16
23
28
23
20
13
13
10
16
37
49
45
26
11
0
6
9
5
3
19
25
26
24
24
25
27
14
6
19
41
55
52
45
26
12
12
17
16
12
10
17
19
17
8
6
5
6
8
5
3
15
25
27
27
17
8
6
1
2
6
13
25
27
26
26
14
5
0
9
15
13
8
9
12
16
16
9
5
9
13
17
17
13
19
24
27
18
5
0
1
1
2
6
20
31
32
26
12
4
6
8
9
7
3
12
17
19
14
4
1
1
3
8
11
12
12
9
7
5
6
4
3
3
1
1
0
1
4
8
9
7
5
1
0
0
2
1
0
0
0
0
3
7
6
3
0
10
18
19
17
9
1
0
2
4
3
2
7
9
10
11
8
4
0
3
5
3
4
8
8
6
11
12
10
4
0
3
5
8
11
11
10
4
0
2
6
9
8
5
2
0
0
0
0
0
1
5
8
7
3
2
7
11
9
5
1
0
5
8
8
7
8
10
9
2
0
4
8
7


LOG#: 2
LTYP:A
TID:Monitor
ADVIS: 2
AC_WT:    
C/G:    
FLT_HRS:      
OPER:  
CODE:  
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:23:24:04
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS:  ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:265
BINS:400
0
0
239
131
212
224
195
160
10
449
716
725
................


***********************************************************************************************************
This is the third and final example.
***********************************************************************************************************
NAME:VMSFAULT
AC:2222222
MODEL: xxxxxxx
FORMAT:001
PN:SWPN 2222222
PN:CORE A008.0005
PN:PLATFORM 3333-00
PN:[Hss #1 Test]:222222
PN:[Hss #2 Test]:222222
PN:BPPN 15328-04 V01.07A CS=C3D043C5
PN:OPPN 15685-06 V01.24A CS=B19504B9 CFGID=SmartMode LOAD=0702040723
PN:DEXPN 15353-10 V0003.0007 CS=C9119471 CFGID=SmartMode LOAD=0509270754
PN:SPPN 12536-24 V04.14A CS=ADA7BA8E
PN:VIBRVID 1.150 SETUPCS=BDD08964
PN:ENGOP 17045-06 01.24A CS=3F462DD8
PN:XILINX 17040-01 01.00B
PN:ENGBP 01.00B
PN:ENGRVID 1.150

LOG#:01
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00003]EALM:ADVISORY LOG IS 0% of 88 BLK
OCCURRENCES: 1
LOG#:02
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00004]EENG:ENGINE CHECK LOG IS 1% of 800 BLK
OCCURRENCES: 1
LOG#:03
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00001]EMON:MONITOR LOG IS 0% of 15270 BLK
OCCURRENCES: 1
LOG#:04
DATE:03-05-2007
TIME:10:41:56
FDESC:W[22007]EENG:ENG PARM SIGNAL RANGE FAIL ENG PARM: E2-CHP, SECS = 100
OCCURRENCES: 5
LOG#:05
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E1-QRW, VALUE = +343
OCCURRENCES: 1
LOG#:06
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E2-QRW, VALUE = +429
OCCURRENCES: 1
0
sabarrow
Asked:
sabarrow
  • 8
  • 7
1 Solution
 
psdavisCommented:
You will definitely want to use Regular expressions to make this feasable.
First, write ones that encompass entire areas. Something that breaks up the text file into smaller sections.  You'll have one that capture the header, the definition, and the log.
 Then take each matching group and write regular expressions for each section.
0
 
sabarrowAuthor Commented:
What about the possibility of using an interpreter?  I've read other posts talk about this approach but haven't seen a good example yet.   The text files will eventually change format but not very often.  Over the past 5 years there has been three versions with additions of definitions which in turn add parameters to the log sections of the files.  My end goal is to get the parameters from the log sections into the database enabling the end user to do metrics on the data.  

If the regex approach is best I will head down that path I just want to make sure it is the "best path forward", the most efficient, and the most maintainable.  I haven't had much experience with regex so I've downloaded the expresso tool to toy around with it.  Pretty interesting so far.  Might take me awhile.  
0
 
Tony McCreathTechnical SEO ConsultantCommented:
I think your first step would be to work out and define an exact set of rules on how this data is formatted.

From a glance I don't think regex is needed (well not a complex one) but there seems to be some complex things going on in deciding what the next bit of data means.

Combined with your requirement to cope with large files, I would base my algorythm around a TextReader. read in data line by line and use sub routines to process chunks of data such as a LOG.

Each line seems to be based on a colon (:) seperated list of values which you can do easily with Split(':')
0
Concerto Cloud for Software Providers & ISVs

Can Concerto Cloud Services help you focus on evolving your application offerings, while delivering the best cloud experience to your customers? From DevOps to revenue models and customer support, the answer is yes!

Learn how Concerto can help you.

 
sabarrowAuthor Commented:
Okay,  check my logic here.  I envision having a class to read in the "NAME" value of each file.  This seems to be the common denominator across each file.  After I know which file I am working with I can then push the parsing of the file to a specific class custom to that particular file format.  So essentially I will have a class to read each file format.  These classes will utilize the TextReader base class to read line by line with subroutines to break up the lines.   Oh, not that it matters but the lines in the DEF files are delimited first by a colon (:) and then the rest of the line is csv (,).  

Also, There are three other files that have definitions defined in the first part of the file like my first example.  These definitions define the data in the log lines listed later in these files.  Since these definitions (and entire file structure) will only change probably once a year; do you think it would be a good idea to just define these in a DB table and ignore reading them in?  Basically I will look at the version of the file and pull the definitions to interpret the lines?
0
 
Tony McCreathTechnical SEO ConsultantCommented:
Sounds good to me.

I would be cautious of ignoring reading the definitions, it might save time now, but it quite often leads to issues later on.
0
 
sabarrowAuthor Commented:
One more thing.  This is probably really simple but in the definition files that have logs 1-n   How would I keep track of what log I'm on?   Also there is one file with the following format:

LOG#: 1
DATE:03-04-2007
TIME:18:35:25
P:0.00,,c,c,c,0.00,0.00,20.1,-5.38,0.00,0.00,19.6,-3.06,c,c,c,c,,,,,,,,,,,,,,,c,,,,-4.22,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.7,-5.40,0.00,0.00,19.6,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.23,0.00,c,c
P:0.00,100.06,c,c,c,0.00,0.00,20.0,-5.43,0.00,0.00,19.6,-3.06,c,c,c,c,,,,,,,,,,,,,,,c,,,,-4.24,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.3,-5.43,0.00,0.00,19.6,-3.05,,,,,,,,,,,,,,,,,,,,,,,-4.24,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.2,-5.42,0.00,0.00,19.5,-3.06,0.016,0.127,0.016,0.014,,,,,,,,,,,,,,,0.452,,,,-4.24,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.3,-5.33,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.20,0.00,c,c
P:0.00,100.28,c,c,c,0.00,0.00,19.5,-5.26,0.00,0.00,19.5,-3.06,0.015,0.127,0.016,0.014,,,,,,,,,,,,,,,0.571,,,,-4.16,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.4,-5.24,0.00,0.00,19.6,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.15,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.2,-5.23,0.00,0.00,19.5,-3.06,0.015,0.127,0.016,0.014,c,c,,,1.1,-0.4,c,-0.3,,,c,1.0,,,0.700,,,,-4.15,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.2,-5.24,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.15,0.00,c,c
P:0.00,100.42,c,c,c,0.00,0.00,20.5,-5.27,0.00,0.00,19.5,-3.06,0.015,0.126,0.016,0.014,,,,,,,,,,,,,,,0.749,,,,-4.17,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.5,-5.34,0.00,0.00,19.6,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.20,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.6,-5.34,0.00,0.00,19.5,-3.06,0.015,0.126,0.016,0.014,,,,,,,,,,,,,,,0.749,,,,-4.20,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.4,-5.33,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.20,0.00,c,c
P:0.00,100.53,c,c,c,0.00,0.00,20.6,-5.38,0.00,0.00,19.5,-3.06,0.015,0.126,0.017,0.014,,,,,,,,,,,,,,,0.749,,,,-4.22,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.6,-5.35,0.00,0.00,19.6,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.21,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.4,-5.28,0.00,0.00,19.5,-3.06,0.015,0.126,0.017,0.014,,,,,,,,,,,,,,,0.750,,,,-4.17,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.0,-5.28,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.17,0.00,c,c
P:0.00,100.65,c,c,c,0.00,0.00,20.6,-5.35,0.00,0.00,19.6,-3.06,0.016,0.126,0.017,0.014,0.033,0.033,,,1.0,-0.4,20.0,-0.3,,,22.0,0.9,,,0.750,,,,-4.20,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,20.7,-5.35,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.21,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,19.2,-5.30,0.00,0.00,19.4,-3.06,0.015,0.126,0.017,0.014,,,,,,,,,,,,,,,0.750,,,,-4.18,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,18.9,-5.65,0.00,0.00,19.5,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.36,0.00,c,c
P:0.00,100.71,c,c,c,0.00,0.00,19.5,-5.96,0.00,0.00,19.7,-3.05,0.016,0.126,0.016,0.014,,,,,,,,,,,,,,,0.751,,,,-4.51,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,21.3,-5.65,0.00,0.00,19.7,-3.05,,,,,,,,,,,,,,,,,,,,,,,-4.35,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,21.5,-5.39,0.00,0.00,19.8,-3.06,0.015,0.126,0.016,0.014,,,,,,,,,,,,,,,0.751,,,,-4.23,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,21.5,-5.39,0.00,0.00,19.8,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.23,0.00,c,c
P:0.00,100.79,c,c,c,0.00,0.00,22.6,-5.31,0.00,0.00,19.7,-3.06,0.015,0.126,0.017,0.014,,,,,,,,,,,,,,,0.751,,,,-4.18,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,23.0,-5.15,0.00,0.00,19.7,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.10,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,23.0,-5.13,0.00,0.00,19.7,-3.07,0.015,0.126,0.016,0.014,0.033,0.033,0.0,c,1.0,-0.4,20.1,-0.3,0.0,0.0,22.7,0.9,0.0,0.0,0.751,c,16.5,24.97,-4.10,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,22.9,-5.12,0.00,0.00,19.7,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.09,0.00,c,c
P:0.00,100.81,c,c,c,0.00,0.00,22.9,-5.12,0.00,0.00,19.7,-3.06,0.016,0.126,0.016,0.014,,,,,,,,,,,,,,,0.751,,,,-4.09,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,22.9,-5.17,0.00,0.00,19.7,-3.06,,,,,,,,,,,,,,,,,,,,,,,-4.11,0.00,c,c
P:0.00,,c,c,c,0.00,0.00,21.9,-5.18,0.00,0.00,19.7,-3.06,0.016,0.126,0.016,0.014,,,,,,,,,,,,,,,0.751,,,,-4.12,0.00,c,c
*********TO 138 LINES********
TIME:18:36:39


What method would you recommend to keep up with the date and time data for each log entry?  A time calculation has to be done.  Each record is taken at 1/2 a second.  

Sorry so many questions.  Just trying to get a good base before I press on.
0
 
Tony McCreathTechnical SEO ConsultantCommented:
an option would be to use a while loop to process the logs and track time updates in a local variable
0
 
sabarrowAuthor Commented:
How would you propose I right it to the database?  Insert each record when I read it or store in an array or some data set before putting it in the DB.  I also have to check that the data is not already in the database also before I insert it.  I cannot have two records in the DB with the same date time stamp
0
 
sabarrowAuthor Commented:
would it be better to read and store the data with each line read or build some sort of collection to upload after all lines are read.
0
 
Tony McCreathTechnical SEO ConsultantCommented:
You state this could be large, so you may want to avoid loading all lines.

Each subroutine could keep track of the data it requires, and when finshed generate the sql you need.

e.g. the log subroutine would be triggered if a line startas with "LOG". It would read lines until it has a whole log stored in its local variables, then create the sql and return.
0
 
sabarrowAuthor Commented:

Forgive my ignorance but what do you mean by "You state this could be large, so you may want to avoid loading all lines."?  Are you saying this could be a memory intensive way to read the file?  I'm  concerned about the time this method will take to complete. Essentially, I'm going to have to read all the lines of the file in so is this going to be a memory issue right? Would I create and destroy a new instance of TextReader with each subroutine to alleviate this?

As far as the subroutines keeping track of the data they require.  What do you recommend here?  Some sort of multidimensional array?  Do you have some example code in mind you could post?
0
 
Tony McCreathTechnical SEO ConsultantCommented:
Sorry, I read your comment "after all lines are read" as meaning you will load the whole file before processing it.

If your using the TextReader and processing as you go you should not have any memory problems.

Regarding keeping track of the data, each subroutine can do it in the way that bast suits the syntax.

I can only guess at the syntax from the sample data. e.g. the log sections might always be name: value lines until you reach a blank line or a line starting with "LOG". If so you could use a Dictionary<string, string> to sore the values. I'll try and get the time to do an example routine later.

0
 
Tony McCreathTechnical SEO ConsultantCommented:
This code does some basic parsing including the detection of a log section and the custom parsing of that. I've tested it on your third file.

As I said earlier. It is very hard to suggest code when I have very little clue about the syntax of the data.

Some notes on the code:

I have to do a line read ahead to determine what the next line means. I have supported this via passing the read ahead line in the process methods, along with the reader.

To signal the read ahead line needs re-processing I've used the moveToNextLine flag. Set this to false so the line gets re-processed.

A completed flag is used to help sub processes finish processing their block of data.

I've tried to establish a consistent pattern that you can use for each sub process you add.

Hope it helps
private void DoIt()
{
    FileStream stream = File.Open("ComplexFile3.txt", FileMode.Open);
 
    StreamReader reader = new StreamReader(stream);
 
    try
    {
        string line = reader.ReadLine();
 
        ProcessComplexFile(ref line, reader);
    }
    finally
    {
        reader.Close();
        stream.Dispose();
    }
}
 
private void ProcessComplexFile(ref string line, TextReader reader)
{
    while (line != null)
    {
        bool moveToNextLine = true;
 
        if (line.Contains(":")) //  valid name:value line to process
        {
            string name = line.Substring(0, line.IndexOf(':'));
            string value = line.Substring(line.IndexOf(':') + 1);
 
            switch (name)
            {
                case "LOG#":
                    moveToNextLine = ProcessLog(ref line, reader);
                    break;
                default: // a basic line
                    Debug.WriteLine("### " + name + "=" + value);
                    break;
            }
        }
        if (moveToNextLine)
            line = reader.ReadLine();
    }
    Debug.WriteLine("#########################################");
}
 
private bool ProcessLog(ref string line, TextReader reader)
{
    bool completed = false;
 
    int logNumber = -1;
    DateTime? date = null;
    TimeSpan? time = null;
    string fdesc = String.Empty;
    int occurances = 0;
 
    while (line != null && !completed)
    {
        bool moveToNextLine = true;
 
        if (line.Contains(":")) //  valid name:value line to process
        {
            string name = line.Substring(0,line.IndexOf(':'));
            string value = line.Substring(line.IndexOf(':')+1);
 
            switch (name)
            {
                case "LOG#":
                    if (logNumber == -1)
                        logNumber = Convert.ToInt32(value);
                    else
                    {
                        // second occurance of LOG which indicates the next log
                        completed = true; // so this method goes to the end
                        moveToNextLine = false; // so this LG line can be processed later
                    }
                    break;
                case "DATE":
                    date = Convert.ToDateTime(value);
                    break;
                case "TIME":
                    time = TimeSpan.Parse(value);
                    break;
                case "FDESC":
                    fdesc = value;
                    break;
                case "OCCURRENCES":
                    occurances = Convert.ToInt32(value);
                    break;
            }
        }
 
        if (moveToNextLine)
            line = reader.ReadLine(); // so current line can be processed by the next processor
    }
 
 
    // process log data
    Debug.WriteLine("*** logNumber=" + logNumber);
    Debug.WriteLine("*** date=" + (date == null ? "NULL" : date.ToString()));
    Debug.WriteLine("*** time=" + (time == null ? "NULL" : time.ToString()));
    Debug.WriteLine("*** fdesc=" + fdesc);
    Debug.WriteLine("*** occurances=" + occurances.ToString());
    Debug.WriteLine("*******************************");
 
    return false; // don't moveToNextLine as either end of file or start of next log
 
}

Open in new window

0
 
sabarrowAuthor Commented:
Two more questions and I'll put this to bed.  I know I'm probably wearing out my welcome but I truly appreciate your patience and guidance.   Normally I would have took what i have and ran with it but I have to get a prototype built to prove the concept.  If this can be completed it will save the company 2 million dollars.  I have around 10 different file types to read but I'm trying to get started with just the three I've shown you in order to get a prototype running.

Question 1.  In your code above.  After the values are obtained in "ProcessComplexFile" and "ProcessLog" which of the following would you recommend:  1. Store them in a collection and then iterate through the collection to insert into the database. 2. Create a data set and store the data there until ready to insert.  3. process each line of data into the table with each read.   Keep in mind I have to do a check on the table to make sure the data does not currently exist in the table.

Question 2.  My ignorance of string manipulation is shining through here but if you can give me some example code to read one or both of the files below I can customize and rewrite it to fit many of the other files.  I'm going to define each line below and attach two example files:

FILE # 1:  VXPMON.log

********** HEADER INFO*************
Can use the current code provided. Except for the last two lines.  There are two semicolons present in these lines
*****************************************
NAME:VMSMON
MODEL:4444
SWPN:33333
LISTNAME:test
AC:    22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111


********** Definition Lines*************
These repeat and are defined as below
*****************************************
DEF:B:1:30,255r,270r,6000r,1;0r,1,400,180,N,ips,F,1,1,N,(1,,,),Fwd Rotor 1P
DEF:B:a:bb,cccc,dddd,eeeee,f;ff,g,hhh,iii,j,kkk,l,m,n,o,(p,p,p,p),qqqqqqq

a = Band number
bb = Vib channel number
cccc = Start freq of band
dddd = End freq of band
eeeee = Max freq of spectrum
f;ff = Zoom factor
g = Azimuth channel
hhh = No. of FFT bins
iii = Cycle time(secs)
j = Phase included?
kkk = undefined1
l = undefined2
m = undefined3
n = undefined4
o = undefined5
(p,p,p,p) = unit index
qqqqqqq = Name of band

**********Last  Definition Line*************
There is one unique def line listed last
*****************************************
DEF:UNITINDEX:1,PeakIPS,0.001000
DEF:UNITINDEX:a,bbbbbb,cccccccc

a = Unit index number
bbbbbb = Unit name
cccccccc = Scale factor for unit
 


********** Log File Header*************
*****************************************
LOG#: 1
DATE:03-04-2007
TIME:18:39:02

********** Log File Entries*************
*****************************************
REG:0
REG:a

a = Regime number
----------------------------
ACH:1,243
ACH:a,b

a = channel number
b = RPM at start of log
----------------------------
B:1,0,-1,-1,-1
B:a,b,c,d,e

a = Band number
b = Band data point 1
c = Band data point 2
d = Band data point 3
e = Band data point 4


********************************************************************************
********************************************************************************
********************************************************************************

FILE # 2:  VXPSPEC.log

File # 2 is pretty much the same as File # 1 except for the log entries are built into one line

********** HEADER INFO*************
Can use the current code provided. Except for the two "PN" lines.  There are two semicolons present in these lines.
*****************************************

NAME:EMONITOR
MODEL:XXXXXX
SWPN:222222
LISTNAME:Eng(-06b)
AC:3333333
PN:[Eng #1]:444444
PN:[Eng #2]:444444
MRVID:1.5000
FORMAT:001
MNAME:test monitor
CTIME:0.500
LOG_FMT:TIME

********** Definition Lines*************
These repeat and are defined as below
*****************************************
DEF:P:1:AX-NR,0.50,1,1,(1.0,0.0,%)%Nr
DEF:P:a:bbbbb,cccc,d,e,(fff,ggg,hhh)iiii

a = Parameter number
bbbbb = Parameter channel ID
cccc = period in sec
d = # of averages before storing
e = Average type
fff = scale
ggg = bias
hhh = units
iiiii = name of parameter

********** Log File Header*************
*****************************************

LOG#: 1
DATE:03-04-2007
TIME:18:35:25


********** Log File Entries*************
*****************************************
P:0.00,,c,c,c,0.00,0.00,20.1,-5.38,0.00,0.00,19.6,-3.06,c,c,c,c,,,,,,,,,,,,,,,c,,,,-4.22,0.00,c,c

Each entry in these lines are directly relational to how man definition lines there are (e.g. DEF:P:1, DEF:P:2, DEF:P:3)  If there are 39 definition lines then there are 39 entries in these lines every time.)

The time entries between the records can be ignored and skipped over.

***********************************************************************************************

***********************************************************************************************


Again, your help is truly valued and appreciated. As stated before I
vxpmon.log
engmon.LOG
0
 
sabarrowAuthor Commented:
Tiggerito,

I contacted you on your personal website.  Check out my email there before you answer the last question.
0
 
Tony McCreathTechnical SEO ConsultantCommented:
Question 1:

I would store the data in a form that is most conviient, then save it to the database once I have gathered a full set of data.

This would depend a lot on the data structure you decide to use.

Question 2

I don't have time to go through all the structures you've defined. I also thing more info on the destination data structure would be needed to fully understand how this data needs to be parsed.
0

Featured Post

[Webinar] Cloud and Mobile-First Strategy

Maybe you’ve fully adopted the cloud since the beginning. Or maybe you started with on-prem resources but are pursuing a “cloud and mobile first” strategy. Getting to that end state has its challenges. Discover how to build out a 100% cloud and mobile IT strategy in this webinar.

  • 8
  • 7
Tackle projects and never again get stuck behind a technical roadblock.
Join Now