sabarrow
asked on
Parsing complicated flat files with C#
I have a task to read in various flat files with C# into a structured database table. The user will be going to a website to upload the file. After uploading the file the C# code will parse the file breaking it up into variables to be inserted into an Oracle DB. I am trying to come up with the most efficient solution to read in these files. The files are structured with a multiline headers and data that follows. I have read in csv files before but those were very straight forward and didn't have multiple data representations in the files. I have included a couple examples of the files below. I have been looking closely at using regular expressions to somehow read in this data but I am unsure whether this is the best approach. Could someone give me some insight and/or examples as to the best approach. The file sizes are going to go all the way up to 150MB so the user is already going to have to wait for an upload of these files. I don't want them to have to wait much long to process them into the database.
This is the first file. It contains a multiline header, a definition section and then a log section. Logs can be from Log#:1 to Log#:n. The log file lists an entry for each definition (e.g. "B:1")
**************************
NAME:VMSMON
MODEL:4444
SWPN:xxxxx
LISTNAME:test
AC: 22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:xxxxx
PN:[Hss #2 Test]:xxxxx
DEFINITIONS:
DEF:B:1:30,255r,270r,6000r
DEF:B:2:30,765r,810r,6000r
DEF:B:3:30,1520r,1620r,600
DEF:B:4:37,255r,270r,6000r
DEF:B:5:37,765r,810r,6000r
DEF:B:6:37,1520r,1620r,600
DEF:B:7:44,255r,270r,6000r
DEF:B:8:44,765r,810r,6000r
DEF:B:9:44,1530r,1620r,600
DEF:B:10:51,255r,270r,6000
DEF:B:11:51,765r,810r,6000
DEF:B:12:51,1530r,1620r,60
DEF:B:13:33,19050r,21225r,
DEF:B:14:33,16200r,17550r,
DEF:B:15:33,7350r,7650r,30
DEF:B:16:33,14700r,15300r,
DEF:B:17:33,2475r,2700r,30
DEF:B:18:33,4800r,5400r,30
DEF:B:19:14,19050r,21225r,
DEF:B:20:14,16200r,17550r,
DEF:B:21:14,7200r,7350r,30
DEF:B:22:14,8400r,8550r,30
DEF:B:23:14,21000r,27300r,
DEF:B:24:14,22350r,23325r,
DEF:B:25:14,3825r,4050r,30
DEF:B:26:14,5400r,7125r,30
DEF:B:27:14,3600r,4425r,30
DEF:B:28:44,6000r,6300r,30
DEF:B:29:44,12000r,12600r,
DEF:B:30:51,6000r,6300r,30
DEF:B:31:51,12000r,12600r,
DEF:B:32:33,38100r,39600r,
DEF:B:33:33,65250r,68250r,
DEF:B:34:14,38100r,39600r,
DEF:B:35:14,42000r,54600r,
DEF:B:36:14,63000r,81900r,
DEF:B:37:14,107250r,114000
DEF:B:38:45,30300r,31500r,
DEF:B:39:45,60600r,63000r,
DEF:B:40:52,30300r,31500r,
DEF:B:41:52,60600r,63000r,
DEF:B:42:31,1500r,1650r,30
DEF:B:43:31,2475r,2700r,30
DEF:B:44:31,4800r,5400r,30
DEF:B:45:36,19050r,21225r,
DEF:B:46:36,16200r,17550r,
DEF:B:47:36,7200r,7350r,30
DEF:B:48:36,8400r,8550r,30
DEF:B:49:36,21000r,27300r,
DEF:B:50:36,22350r,23325r,
DEF:B:51:36,3825r,4050r,30
DEF:B:52:36,5400r,7125r,30
DEF:B:53:36,3600r,4425r,30
DEF:B:54:45,7350r,7650r,30
DEF:B:55:45,14700r,15300r,
DEF:B:56:45,1500r,1650r,30
DEF:B:57:45,4350r,4650r,30
DEF:B:58:45,8775r,9225r,30
DEF:B:59:45,3975r,4125r,30
DEF:B:60:45,2550r,2700r,30
DEF:B:61:45,5100r,5325r,30
DEF:B:62:45,6675r,7050r,30
DEF:B:63:45,13425r,14025r,
DEF:B:64:45,7800r,8400r,30
DEF:B:65:45,15600r,16800r,
DEF:B:66:45,19080r,19800r,
DEF:B:67:52,7350r,7650r,30
DEF:B:68:52,14700r,15300r,
DEF:B:69:52,1500r,1650r,30
DEF:B:70:52,4350r,4650r,30
DEF:B:71:52,8775r,9225r,30
DEF:B:72:52,3975r,4125r,30
DEF:B:73:52,2550r,2700r,30
DEF:B:74:52,5100r,5325r,30
DEF:B:75:52,6675r,7050r,30
DEF:B:76:52,13425r,14025r,
DEF:B:77:52,7800r,8400r,30
DEF:B:78:52,15600r,16800r,
DEF:B:79:52,19080r,19800r,
DEF:B:80:31,30300r,31500r,
DEF:B:81:31,60600r,63000r,
DEF:B:82:31,65250r,68250r,
DEF:B:83:36,38100r,39600r,
DEF:B:84:36,42000r,54600r,
DEF:B:85:36,63000r,81900r,
DEF:B:86:36,107250r,114000
DEF:B:87:46,42000r,54600r,
DEF:B:88:46,63000r,81900r,
DEF:B:89:46,33600r,43800r,
DEF:B:90:46,23550r,30675r,
DEF:B:91:46,30000r,39600r,
DEF:B:92:53,42000r,54600r,
DEF:B:93:53,63000r,81900r,
DEF:B:94:53,33600r,43800r,
DEF:B:95:53,23550r,30675r,
DEF:B:96:53,30000r,39600r,
DEF:B:97:33,186000r,193500
DEF:B:98:33,372000r,385500
DEF:B:99:33,130500r,135750
DEF:B:100:40,186000r,19350
DEF:B:101:40,372000r,38550
DEF:B:102:40,130500r,13575
DEF:B:103:45,186000r,19350
DEF:B:104:45,372000r,38550
DEF:B:105:45,537000r,55200
DEF:B:106:52,186000r,19350
DEF:B:107:52,372000r,38550
DEF:B:108:52,537000r,55200
DEF:B:109:32,8250r,11400r,
DEF:B:110:32,21000r,27300r
DEF:B:111:32,5400r,7125r,3
DEF:B:112:32,5175r,6825r,3
DEF:B:113:32,3600r,4425r,3
DEF:B:114:32,14850r,18975r
DEF:B:115:32,1275r,1800r,3
DEF:B:116:32,1350r,1875r,3
DEF:B:117:32,2025r,2775r,3
DEF:B:118:40,19050r,21225r
DEF:B:119:40,16200r,17550r
DEF:B:120:40,7350r,7650r,3
DEF:B:121:40,14700r,15300r
DEF:B:122:40,2475r,2700r,3
DEF:B:123:40,4800r,5400r,3
DEF:B:124:46,8250r,11400r,
DEF:B:125:46,21000r,27300r
DEF:B:126:46,5400r,7125r,3
DEF:B:127:46,5175r,6825r,3
DEF:B:128:46,3600r,4425r,3
DEF:B:129:46,14850r,18975r
DEF:B:130:46,1275r,1800r,3
DEF:B:131:46,1350r,1875r,3
DEF:B:132:46,2025r,2775r,3
DEF:B:133:53,8250r,11400r,
DEF:B:134:53,21000r,27300r
DEF:B:135:53,5400r,7125r,3
DEF:B:136:53,5175r,6825r,3
DEF:B:137:53,3600r,4425r,3
DEF:B:138:53,14850r,18975r
DEF:B:139:53,1275r,1800r,3
DEF:B:140:53,1350r,1875r,3
DEF:B:141:53,2025r,2775r,3
DEF:B:142:32,42000r,54600r
DEF:B:143:32,63000r,81900r
DEF:B:144:32,33600r,43800r
DEF:B:145:32,23550r,30675r
DEF:B:146:32,30000r,39600r
DEF:B:147:40,38100r,39600r
DEF:B:148:40,65250r,68250r
DEF:B:149:47,55500r,57750r
DEF:B:150:47,111000r,11550
DEF:B:151:7,42000r,54600r,
DEF:B:152:7,63000r,81900r,
DEF:B:153:38,30300r,31500r
DEF:B:154:38,60600r,63000r
DEF:B:155:38,65250r,68250r
DEF:B:156:31,130500r,13575
DEF:B:157:38,130500r,13575
DEF:B:158:46,141000r,18450
DEF:B:159:46,193500r,25200
DEF:B:160:53,141000r,18450
DEF:B:161:53,193500r,25200
DEF:B:162:7,7200r,7350r,30
DEF:B:163:7,8400r,8550r,30
DEF:B:164:7,21000r,27300r,
DEF:B:165:7,22350r,23325r,
DEF:B:166:7,3825r,4050r,30
DEF:B:167:7,8250r,11400r,3
DEF:B:168:38,1500r,1650r,3
DEF:B:169:38,2475r,2700r,3
DEF:B:170:38,4800r,5400r,3
DEF:B:171:47,7350r,8250r,3
DEF:B:172:47,3450r,3900r,3
DEF:B:173:47,3900r,4350r,3
DEF:B:174:47,5100r,5700r,3
DEF:B:175:27,2475r,2700r,3
DEF:B:176:27,4800r,5400r,3
DEF:B:177:27,7200r,8100r,3
DEF:B:178:27,750r,825r,300
DEF:B:179:27,1500r,1650r,3
DEF:B:180:31,765r,810r,600
DEF:B:181:38,765r,810r,600
DEF:B:182:49,255r,270r,600
DEF:B:183:49,765r,810r,600
DEF:B:184:49,1520r,1620r,6
DEF:B:185:28,255r,270r,600
DEF:B:186:28,765r,810r,600
DEF:B:187:28,1520r,1620r,6
DEF:B:188:32,141000r,18450
DEF:B:189:32,193500r,25200
DEF:B:190:39,141000r,18450
DEF:B:191:39,193500r,25200
DEF:B:192:29,7200r,7350r,3
DEF:B:193:29,8400r,8550r,3
DEF:B:194:29,21000r,27300r
DEF:B:195:29,22350r,23325r
DEF:B:196:29,3825r,4050r,3
DEF:B:197:29,8250r,11400r,
DEF:B:198:39,8250r,11400r,
DEF:B:199:39,21000r,27300r
DEF:B:200:39,5400r,7125r,3
DEF:B:201:39,5175r,6825r,3
DEF:B:202:39,3600r,4425r,3
DEF:B:203:39,14850r,18975r
DEF:B:204:39,1275r,1800r,3
DEF:B:205:39,1350r,1875r,3
DEF:B:206:39,2025r,2775r,3
DEF:B:207:15,7800r,8400r,3
DEF:B:208:15,15600r,16800r
DEF:B:209:29,42000r,54600r
DEF:B:210:29,63000r,81900r
DEF:B:211:39,42000r,54600r
DEF:B:212:39,63000r,81900r
DEF:B:213:39,33600r,43800r
DEF:B:214:39,23550r,30675r
DEF:B:215:39,30000r,39600r
DEF:B:216:7,107250r,114000
DEF:B:217:29,107250r,11400
DEF:B:218:1,2475r,2700r,30
DEF:B:219:1,4800r,5400r,30
DEF:B:220:1,7200r,8100r,30
DEF:B:221:1,750r,825r,3000
DEF:B:222:1,1500r,1650r,30
DEF:B:223:8,7800r,8400r,30
DEF:B:224:8,15600r,16800r,
DEF:B:225:20,2475r,2700r,3
DEF:B:226:20,4800r,5400r,3
DEF:B:227:20,7200r,8100r,3
DEF:B:228:20,750r,825r,300
DEF:B:229:20,1500r,1650r,3
DEF:B:230:13,2475r,2700r,3
DEF:B:231:13,4800r,5400r,3
DEF:B:232:13,7200r,8100r,3
DEF:B:233:13,750r,825r,300
DEF:B:234:13,1500r,1650r,3
DEF:B:235:49,19080r,19800r
DEF:B:236:28,19080r,19800r
DEF:UNITINDEX:1,PeakIPS,0.
ENDDEF:
LOG#: 1
DATE:03-04-2007
TIME:18:39:02
REG:0
ACH:1,-1
B:1,0,-1,-1,-1
B:2,0,-1,-1,-1
B:3,0,-1,-1,-1
B:4,17,-1,-1,-1
B:5,0,-1,-1,-1
B:6,0,-1,-1,-1
B:7,0,-1,-1,-1
B:8,0,-1,-1,-1
B:9,0,-1,-1,-1
B:10,0,-1,-1,-1
B:11,0,-1,-1,-1
B:12,0,-1,-1,-1
ACH:1,-1
B:13,2,-1,-1,-1
B:14,1,233,-1,-1
B:15,0,-1,-1,-1
B:16,0,-1,-1,-1
B:17,0,-1,-1,-1
B:18,0,-1,-1,-1
B:19,0,-1,-1,-1
B:20,0,216,-1,-1
B:21,0,-1,-1,-1
B:22,0,-1,-1,-1
B:23,0,280,-1,-1
B:24,0,-1,-1,-1
B:25,0,-1,-1,-1
B:26,0,-1,-1,-1
B:27,0,-1,-1,-1
B:28,0,-1,-1,-1
B:29,0,-1,-1,-1
B:30,0,-1,-1,-1
B:31,7,-1,-1,-1
ACH:1,-1
B:32,0,-1,-1,-1
B:33,0,-1,-1,-1
B:34,0,-1,-1,-1
B:35,0,-1,-1,-1
B:36,0,-1,-1,-1
B:37,0,-1,-1,-1
B:38,0,-1,-1,-1
B:39,0,-1,-1,-1
B:40,0,-1,-1,-1
B:41,0,-1,-1,-1
ACH:1,-1
B:42,0,-1,-1,-1
B:43,0,-1,-1,-1
B:44,0,-1,-1,-1
B:45,0,-1,-1,-1
B:46,0,216,-1,-1
B:47,0,-1,-1,-1
B:48,0,-1,-1,-1
B:49,0,280,-1,-1
B:50,0,-1,-1,-1
B:51,0,-1,-1,-1
B:52,0,-1,-1,-1
B:53,0,-1,-1,-1
B:54,0,-1,-1,-1
B:55,0,-1,-1,-1
B:56,0,-1,-1,-1
B:57,0,-1,-1,-1
B:58,0,-1,-1,-1
B:59,0,-1,-1,-1
B:60,0,-1,-1,-1
B:61,0,-1,-1,-1
B:62,0,-1,-1,-1
B:63,1,-1,-1,-1
B:64,3,-1,-1,-1
B:65,0,-1,-1,-1
B:66,0,-1,-1,-1
B:67,0,-1,-1,-1
B:68,0,-1,-1,-1
B:69,0,-1,-1,-1
B:70,0,-1,-1,-1
B:71,0,-1,-1,-1
B:72,0,-1,-1,-1
B:73,0,-1,-1,-1
B:74,0,-1,-1,-1
B:75,0,-1,-1,-1
B:76,2,-1,-1,-1
B:77,4,-1,-1,-1
B:78,0,-1,-1,-1
B:79,0,-1,-1,-1
ACH:1,-1
B:80,0,-1,-1,-1
B:81,0,-1,-1,-1
B:82,0,-1,-1,-1
B:83,0,-1,-1,-1
B:84,0,-1,-1,-1
B:85,6,-1,-1,-1
B:86,0,-1,-1,-1
B:87,0,-1,-1,-1
B:88,0,-1,-1,-1
B:89,0,-1,-1,-1
B:90,0,-1,-1,-1
B:91,0,-1,-1,-1
B:92,0,-1,-1,-1
B:93,0,-1,-1,-1
B:94,0,-1,-1,-1
B:95,0,-1,-1,-1
B:96,0,-1,-1,-1
ACH:1,-1
B:97,0,248,-1,-1
B:98,0,-1,-1,-1
B:99,0,-1,-1,-1
B:100,0,248,-1,-1
B:101,0,-1,-1,-1
B:102,0,-1,-1,-1
B:103,0,248,-1,-1
B:104,0,-1,-1,-1
B:105,0,-1,-1,-1
B:106,0,248,-1,-1
B:107,0,-1,-1,-1
B:108,0,-1,-1,-1
ACH:1,-1
B:109,18,-1,-1,-1
B:110,23,305,-1,-1
B:111,86,-1,-1,-1
B:112,86,-1,-1,-1
B:113,2,-1,-1,-1
B:114,6,-1,-1,-1
B:115,15,-1,-1,-1
B:116,15,-1,-1,-1
B:117,7,-1,-1,-1
B:118,0,-1,-1,-1
B:119,1,216,-1,-1
B:120,0,-1,-1,-1
B:121,0,-1,-1,-1
B:122,0,-1,-1,-1
B:123,1,-1,-1,-1
B:124,0,-1,-1,-1
B:125,0,280,-1,-1
B:126,18,-1,-1,-1
B:127,18,-1,-1,-1
B:128,0,-1,-1,-1
B:129,0,-1,-1,-1
B:130,18,-1,-1,-1
B:131,18,-1,-1,-1
B:132,2,-1,-1,-1
B:133,0,-1,-1,-1
B:134,0,280,-1,-1
B:135,5,-1,-1,-1
B:136,5,-1,-1,-1
B:137,3,-1,-1,-1
B:138,0,-1,-1,-1
B:139,0,-1,-1,-1
B:140,0,-1,-1,-1
B:141,0,-1,-1,-1
ACH:1,-1
B:142,0,-1,-1,-1
B:143,0,-1,-1,-1
B:144,9,-1,-1,-1
B:145,20,-1,-1,-1
B:146,9,-1,-1,-1
B:147,1,-1,-1,-1
B:148,0,-1,-1,-1
B:149,46,-1,-1,-1
B:150,11,-1,-1,-1
ACH:1,-1
B:151,4,-1,-1,-1
B:152,19,-1,-1,-1
B:153,0,-1,-1,-1
B:154,0,-1,-1,-1
B:155,0,-1,-1,-1
ACH:1,-1
B:156,0,-1,-1,-1
B:157,0,-1,-1,-1
B:158,0,-1,-1,-1
B:159,0,-1,-1,-1
B:160,0,-1,-1,-1
B:161,0,-1,-1,-1
ACH:1,-1
B:162,4,-1,-1,-1
B:163,1,-1,-1,-1
B:164,6,330,-1,-1
B:165,5,-1,-1,-1
B:166,1,-1,-1,-1
B:167,23,-1,-1,-1
B:168,1,-1,-1,-1
B:169,0,-1,-1,-1
B:170,0,-1,-1,-1
B:171,244,-1,-1,-1
B:172,16,-1,-1,-1
B:173,32,-1,-1,-1
B:174,15,-1,-1,-1
B:175,0,-1,-1,-1
B:176,2,-1,-1,-1
B:177,1,-1,-1,-1
B:178,1,-1,-1,-1
B:179,16,-1,-1,-1
ACH:1,-1
B:180,0,-1,-1,-1
B:181,0,-1,-1,-1
B:182,0,-1,-1,-1
B:183,0,-1,-1,-1
B:184,0,-1,-1,-1
B:185,0,-1,-1,-1
B:186,0,-1,-1,-1
B:187,0,-1,-1,-1
ACH:1,-1
B:188,11,-1,-1,-1
B:189,24,-1,-1,-1
B:190,14,-1,-1,-1
B:191,9,-1,-1,-1
ACH:1,-1
B:192,0,-1,-1,-1
B:193,5,-1,-1,-1
B:194,4,291,-1,-1
B:195,0,-1,-1,-1
B:196,0,-1,-1,-1
B:197,6,-1,-1,-1
B:198,19,-1,-1,-1
B:199,30,304,-1,-1
B:200,9,-1,-1,-1
B:201,13,-1,-1,-1
B:202,10,-1,-1,-1
B:203,2,-1,-1,-1
B:204,6,-1,-1,-1
B:205,6,-1,-1,-1
B:206,1,-1,-1,-1
B:207,10,-1,-1,-1
B:208,0,-1,-1,-1
ACH:1,-1
B:209,23,-1,-1,-1
B:210,14,-1,-1,-1
B:211,21,-1,-1,-1
B:212,12,-1,-1,-1
B:213,9,-1,-1,-1
B:214,47,-1,-1,-1
B:215,4,-1,-1,-1
ACH:1,-1
B:216,5,-1,-1,-1
ACH:1,-1
B:217,1,-1,-1,-1
ACH:1,-1
B:218,24,-1,-1,-1
B:219,31,-1,-1,-1
B:220,12,-1,-1,-1
B:221,24,-1,-1,-1
B:222,52,-1,-1,-1
B:223,3,-1,-1,-1
B:224,8,-1,-1,-1
B:225,29,-1,-1,-1
B:226,8,-1,-1,-1
B:227,10,-1,-1,-1
B:228,46,-1,-1,-1
B:229,265,-1,-1,-1
ACH:1,243
B:230,79,-1,-1,-1
B:231,14,-1,-1,-1
B:232,29,-1,-1,-1
B:233,176,-1,-1,-1
B:234,139,-1,-1,-1
ACH:1,242
B:235,0,-1,-1,-1
B:236,0,-1,-1,-1
**************************
This is the second file. It is in a linear format with the log records separated by newlines
**************************
NAME:VMSSPECT
MODEL:xxxxxx
SWPN:22222
LISTNAME:xxxxx
LISTID:387
LISTVER:1.150
AC: 22222
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111
LOG#: 1
LTYP:A
TID:Monitor
ADVIS: 1
AC_WT:
C/G:
FLT_HRS:
OPER:
CODE:
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:21:10:27
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS: ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:266
BINS:400
0
0
286
356
295
263
211
160
163
431
602
595
543
257
117
97
40
6
16
47
83
79
80
76
47
44
41
27
39
31
84
174
367
701
764
718
516
151
7
20
15
37
54
62
58
22
14
19
19
15
19
51
90
89
84
47
11
10
29
40
34
32
51
60
58
49
21
19
26
30
34
33
34
43
47
44
32
12
1
0
2
3
5
46
90
84
66
22
27
31
17
9
10
23
93
143
159
147
72
15
1
27
55
65
71
78
94
133
130
84
55
31
31
39
44
63
101
106
95
40
10
22
17
16
12
26
64
81
80
63
24
26
19
8
5
0
51
116
127
127
88
35
27
21
20
15
12
60
85
81
75
37
29
37
28
23
24
5
30
44
51
52
26
6
5
13
15
11
27
67
84
91
69
50
39
33
37
39
32
20
3
9
16
23
28
23
20
13
13
10
16
37
49
45
26
11
0
6
9
5
3
19
25
26
24
24
25
27
14
6
19
41
55
52
45
26
12
12
17
16
12
10
17
19
17
8
6
5
6
8
5
3
15
25
27
27
17
8
6
1
2
6
13
25
27
26
26
14
5
0
9
15
13
8
9
12
16
16
9
5
9
13
17
17
13
19
24
27
18
5
0
1
1
2
6
20
31
32
26
12
4
6
8
9
7
3
12
17
19
14
4
1
1
3
8
11
12
12
9
7
5
6
4
3
3
1
1
0
1
4
8
9
7
5
1
0
0
2
1
0
0
0
0
3
7
6
3
0
10
18
19
17
9
1
0
2
4
3
2
7
9
10
11
8
4
0
3
5
3
4
8
8
6
11
12
10
4
0
3
5
8
11
11
10
4
0
2
6
9
8
5
2
0
0
0
0
0
1
5
8
7
3
2
7
11
9
5
1
0
5
8
8
7
8
10
9
2
0
4
8
7
LOG#: 2
LTYP:A
TID:Monitor
ADVIS: 2
AC_WT:
C/G:
FLT_HRS:
OPER:
CODE:
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:23:24:04
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS: ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:265
BINS:400
0
0
239
131
212
224
195
160
10
449
716
725
................
**************************
This is the third and final example.
**************************
NAME:VMSFAULT
AC:2222222
MODEL: xxxxxxx
FORMAT:001
PN:SWPN 2222222
PN:CORE A008.0005
PN:PLATFORM 3333-00
PN:[Hss #1 Test]:222222
PN:[Hss #2 Test]:222222
PN:BPPN 15328-04 V01.07A CS=C3D043C5
PN:OPPN 15685-06 V01.24A CS=B19504B9 CFGID=SmartMode LOAD=0702040723
PN:DEXPN 15353-10 V0003.0007 CS=C9119471 CFGID=SmartMode LOAD=0509270754
PN:SPPN 12536-24 V04.14A CS=ADA7BA8E
PN:VIBRVID 1.150 SETUPCS=BDD08964
PN:ENGOP 17045-06 01.24A CS=3F462DD8
PN:XILINX 17040-01 01.00B
PN:ENGBP 01.00B
PN:ENGRVID 1.150
LOG#:01
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00003]EALM:ADVISOR
OCCURRENCES: 1
LOG#:02
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00004]EENG:ENGINE CHECK LOG IS 1% of 800 BLK
OCCURRENCES: 1
LOG#:03
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00001]EMON:MONITOR
OCCURRENCES: 1
LOG#:04
DATE:03-05-2007
TIME:10:41:56
FDESC:W[22007]EENG:ENG PARM SIGNAL RANGE FAIL ENG PARM: E2-CHP, SECS = 100
OCCURRENCES: 5
LOG#:05
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E1-QRW, VALUE = +343
OCCURRENCES: 1
LOG#:06
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E2-QRW, VALUE = +429
OCCURRENCES: 1
This is the first file. It contains a multiline header, a definition section and then a log section. Logs can be from Log#:1 to Log#:n. The log file lists an entry for each definition (e.g. "B:1")
**************************
NAME:VMSMON
MODEL:4444
SWPN:xxxxx
LISTNAME:test
AC: 22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:xxxxx
PN:[Hss #2 Test]:xxxxx
DEFINITIONS:
DEF:B:1:30,255r,270r,6000r
DEF:B:2:30,765r,810r,6000r
DEF:B:3:30,1520r,1620r,600
DEF:B:4:37,255r,270r,6000r
DEF:B:5:37,765r,810r,6000r
DEF:B:6:37,1520r,1620r,600
DEF:B:7:44,255r,270r,6000r
DEF:B:8:44,765r,810r,6000r
DEF:B:9:44,1530r,1620r,600
DEF:B:10:51,255r,270r,6000
DEF:B:11:51,765r,810r,6000
DEF:B:12:51,1530r,1620r,60
DEF:B:13:33,19050r,21225r,
DEF:B:14:33,16200r,17550r,
DEF:B:15:33,7350r,7650r,30
DEF:B:16:33,14700r,15300r,
DEF:B:17:33,2475r,2700r,30
DEF:B:18:33,4800r,5400r,30
DEF:B:19:14,19050r,21225r,
DEF:B:20:14,16200r,17550r,
DEF:B:21:14,7200r,7350r,30
DEF:B:22:14,8400r,8550r,30
DEF:B:23:14,21000r,27300r,
DEF:B:24:14,22350r,23325r,
DEF:B:25:14,3825r,4050r,30
DEF:B:26:14,5400r,7125r,30
DEF:B:27:14,3600r,4425r,30
DEF:B:28:44,6000r,6300r,30
DEF:B:29:44,12000r,12600r,
DEF:B:30:51,6000r,6300r,30
DEF:B:31:51,12000r,12600r,
DEF:B:32:33,38100r,39600r,
DEF:B:33:33,65250r,68250r,
DEF:B:34:14,38100r,39600r,
DEF:B:35:14,42000r,54600r,
DEF:B:36:14,63000r,81900r,
DEF:B:37:14,107250r,114000
DEF:B:38:45,30300r,31500r,
DEF:B:39:45,60600r,63000r,
DEF:B:40:52,30300r,31500r,
DEF:B:41:52,60600r,63000r,
DEF:B:42:31,1500r,1650r,30
DEF:B:43:31,2475r,2700r,30
DEF:B:44:31,4800r,5400r,30
DEF:B:45:36,19050r,21225r,
DEF:B:46:36,16200r,17550r,
DEF:B:47:36,7200r,7350r,30
DEF:B:48:36,8400r,8550r,30
DEF:B:49:36,21000r,27300r,
DEF:B:50:36,22350r,23325r,
DEF:B:51:36,3825r,4050r,30
DEF:B:52:36,5400r,7125r,30
DEF:B:53:36,3600r,4425r,30
DEF:B:54:45,7350r,7650r,30
DEF:B:55:45,14700r,15300r,
DEF:B:56:45,1500r,1650r,30
DEF:B:57:45,4350r,4650r,30
DEF:B:58:45,8775r,9225r,30
DEF:B:59:45,3975r,4125r,30
DEF:B:60:45,2550r,2700r,30
DEF:B:61:45,5100r,5325r,30
DEF:B:62:45,6675r,7050r,30
DEF:B:63:45,13425r,14025r,
DEF:B:64:45,7800r,8400r,30
DEF:B:65:45,15600r,16800r,
DEF:B:66:45,19080r,19800r,
DEF:B:67:52,7350r,7650r,30
DEF:B:68:52,14700r,15300r,
DEF:B:69:52,1500r,1650r,30
DEF:B:70:52,4350r,4650r,30
DEF:B:71:52,8775r,9225r,30
DEF:B:72:52,3975r,4125r,30
DEF:B:73:52,2550r,2700r,30
DEF:B:74:52,5100r,5325r,30
DEF:B:75:52,6675r,7050r,30
DEF:B:76:52,13425r,14025r,
DEF:B:77:52,7800r,8400r,30
DEF:B:78:52,15600r,16800r,
DEF:B:79:52,19080r,19800r,
DEF:B:80:31,30300r,31500r,
DEF:B:81:31,60600r,63000r,
DEF:B:82:31,65250r,68250r,
DEF:B:83:36,38100r,39600r,
DEF:B:84:36,42000r,54600r,
DEF:B:85:36,63000r,81900r,
DEF:B:86:36,107250r,114000
DEF:B:87:46,42000r,54600r,
DEF:B:88:46,63000r,81900r,
DEF:B:89:46,33600r,43800r,
DEF:B:90:46,23550r,30675r,
DEF:B:91:46,30000r,39600r,
DEF:B:92:53,42000r,54600r,
DEF:B:93:53,63000r,81900r,
DEF:B:94:53,33600r,43800r,
DEF:B:95:53,23550r,30675r,
DEF:B:96:53,30000r,39600r,
DEF:B:97:33,186000r,193500
DEF:B:98:33,372000r,385500
DEF:B:99:33,130500r,135750
DEF:B:100:40,186000r,19350
DEF:B:101:40,372000r,38550
DEF:B:102:40,130500r,13575
DEF:B:103:45,186000r,19350
DEF:B:104:45,372000r,38550
DEF:B:105:45,537000r,55200
DEF:B:106:52,186000r,19350
DEF:B:107:52,372000r,38550
DEF:B:108:52,537000r,55200
DEF:B:109:32,8250r,11400r,
DEF:B:110:32,21000r,27300r
DEF:B:111:32,5400r,7125r,3
DEF:B:112:32,5175r,6825r,3
DEF:B:113:32,3600r,4425r,3
DEF:B:114:32,14850r,18975r
DEF:B:115:32,1275r,1800r,3
DEF:B:116:32,1350r,1875r,3
DEF:B:117:32,2025r,2775r,3
DEF:B:118:40,19050r,21225r
DEF:B:119:40,16200r,17550r
DEF:B:120:40,7350r,7650r,3
DEF:B:121:40,14700r,15300r
DEF:B:122:40,2475r,2700r,3
DEF:B:123:40,4800r,5400r,3
DEF:B:124:46,8250r,11400r,
DEF:B:125:46,21000r,27300r
DEF:B:126:46,5400r,7125r,3
DEF:B:127:46,5175r,6825r,3
DEF:B:128:46,3600r,4425r,3
DEF:B:129:46,14850r,18975r
DEF:B:130:46,1275r,1800r,3
DEF:B:131:46,1350r,1875r,3
DEF:B:132:46,2025r,2775r,3
DEF:B:133:53,8250r,11400r,
DEF:B:134:53,21000r,27300r
DEF:B:135:53,5400r,7125r,3
DEF:B:136:53,5175r,6825r,3
DEF:B:137:53,3600r,4425r,3
DEF:B:138:53,14850r,18975r
DEF:B:139:53,1275r,1800r,3
DEF:B:140:53,1350r,1875r,3
DEF:B:141:53,2025r,2775r,3
DEF:B:142:32,42000r,54600r
DEF:B:143:32,63000r,81900r
DEF:B:144:32,33600r,43800r
DEF:B:145:32,23550r,30675r
DEF:B:146:32,30000r,39600r
DEF:B:147:40,38100r,39600r
DEF:B:148:40,65250r,68250r
DEF:B:149:47,55500r,57750r
DEF:B:150:47,111000r,11550
DEF:B:151:7,42000r,54600r,
DEF:B:152:7,63000r,81900r,
DEF:B:153:38,30300r,31500r
DEF:B:154:38,60600r,63000r
DEF:B:155:38,65250r,68250r
DEF:B:156:31,130500r,13575
DEF:B:157:38,130500r,13575
DEF:B:158:46,141000r,18450
DEF:B:159:46,193500r,25200
DEF:B:160:53,141000r,18450
DEF:B:161:53,193500r,25200
DEF:B:162:7,7200r,7350r,30
DEF:B:163:7,8400r,8550r,30
DEF:B:164:7,21000r,27300r,
DEF:B:165:7,22350r,23325r,
DEF:B:166:7,3825r,4050r,30
DEF:B:167:7,8250r,11400r,3
DEF:B:168:38,1500r,1650r,3
DEF:B:169:38,2475r,2700r,3
DEF:B:170:38,4800r,5400r,3
DEF:B:171:47,7350r,8250r,3
DEF:B:172:47,3450r,3900r,3
DEF:B:173:47,3900r,4350r,3
DEF:B:174:47,5100r,5700r,3
DEF:B:175:27,2475r,2700r,3
DEF:B:176:27,4800r,5400r,3
DEF:B:177:27,7200r,8100r,3
DEF:B:178:27,750r,825r,300
DEF:B:179:27,1500r,1650r,3
DEF:B:180:31,765r,810r,600
DEF:B:181:38,765r,810r,600
DEF:B:182:49,255r,270r,600
DEF:B:183:49,765r,810r,600
DEF:B:184:49,1520r,1620r,6
DEF:B:185:28,255r,270r,600
DEF:B:186:28,765r,810r,600
DEF:B:187:28,1520r,1620r,6
DEF:B:188:32,141000r,18450
DEF:B:189:32,193500r,25200
DEF:B:190:39,141000r,18450
DEF:B:191:39,193500r,25200
DEF:B:192:29,7200r,7350r,3
DEF:B:193:29,8400r,8550r,3
DEF:B:194:29,21000r,27300r
DEF:B:195:29,22350r,23325r
DEF:B:196:29,3825r,4050r,3
DEF:B:197:29,8250r,11400r,
DEF:B:198:39,8250r,11400r,
DEF:B:199:39,21000r,27300r
DEF:B:200:39,5400r,7125r,3
DEF:B:201:39,5175r,6825r,3
DEF:B:202:39,3600r,4425r,3
DEF:B:203:39,14850r,18975r
DEF:B:204:39,1275r,1800r,3
DEF:B:205:39,1350r,1875r,3
DEF:B:206:39,2025r,2775r,3
DEF:B:207:15,7800r,8400r,3
DEF:B:208:15,15600r,16800r
DEF:B:209:29,42000r,54600r
DEF:B:210:29,63000r,81900r
DEF:B:211:39,42000r,54600r
DEF:B:212:39,63000r,81900r
DEF:B:213:39,33600r,43800r
DEF:B:214:39,23550r,30675r
DEF:B:215:39,30000r,39600r
DEF:B:216:7,107250r,114000
DEF:B:217:29,107250r,11400
DEF:B:218:1,2475r,2700r,30
DEF:B:219:1,4800r,5400r,30
DEF:B:220:1,7200r,8100r,30
DEF:B:221:1,750r,825r,3000
DEF:B:222:1,1500r,1650r,30
DEF:B:223:8,7800r,8400r,30
DEF:B:224:8,15600r,16800r,
DEF:B:225:20,2475r,2700r,3
DEF:B:226:20,4800r,5400r,3
DEF:B:227:20,7200r,8100r,3
DEF:B:228:20,750r,825r,300
DEF:B:229:20,1500r,1650r,3
DEF:B:230:13,2475r,2700r,3
DEF:B:231:13,4800r,5400r,3
DEF:B:232:13,7200r,8100r,3
DEF:B:233:13,750r,825r,300
DEF:B:234:13,1500r,1650r,3
DEF:B:235:49,19080r,19800r
DEF:B:236:28,19080r,19800r
DEF:UNITINDEX:1,PeakIPS,0.
ENDDEF:
LOG#: 1
DATE:03-04-2007
TIME:18:39:02
REG:0
ACH:1,-1
B:1,0,-1,-1,-1
B:2,0,-1,-1,-1
B:3,0,-1,-1,-1
B:4,17,-1,-1,-1
B:5,0,-1,-1,-1
B:6,0,-1,-1,-1
B:7,0,-1,-1,-1
B:8,0,-1,-1,-1
B:9,0,-1,-1,-1
B:10,0,-1,-1,-1
B:11,0,-1,-1,-1
B:12,0,-1,-1,-1
ACH:1,-1
B:13,2,-1,-1,-1
B:14,1,233,-1,-1
B:15,0,-1,-1,-1
B:16,0,-1,-1,-1
B:17,0,-1,-1,-1
B:18,0,-1,-1,-1
B:19,0,-1,-1,-1
B:20,0,216,-1,-1
B:21,0,-1,-1,-1
B:22,0,-1,-1,-1
B:23,0,280,-1,-1
B:24,0,-1,-1,-1
B:25,0,-1,-1,-1
B:26,0,-1,-1,-1
B:27,0,-1,-1,-1
B:28,0,-1,-1,-1
B:29,0,-1,-1,-1
B:30,0,-1,-1,-1
B:31,7,-1,-1,-1
ACH:1,-1
B:32,0,-1,-1,-1
B:33,0,-1,-1,-1
B:34,0,-1,-1,-1
B:35,0,-1,-1,-1
B:36,0,-1,-1,-1
B:37,0,-1,-1,-1
B:38,0,-1,-1,-1
B:39,0,-1,-1,-1
B:40,0,-1,-1,-1
B:41,0,-1,-1,-1
ACH:1,-1
B:42,0,-1,-1,-1
B:43,0,-1,-1,-1
B:44,0,-1,-1,-1
B:45,0,-1,-1,-1
B:46,0,216,-1,-1
B:47,0,-1,-1,-1
B:48,0,-1,-1,-1
B:49,0,280,-1,-1
B:50,0,-1,-1,-1
B:51,0,-1,-1,-1
B:52,0,-1,-1,-1
B:53,0,-1,-1,-1
B:54,0,-1,-1,-1
B:55,0,-1,-1,-1
B:56,0,-1,-1,-1
B:57,0,-1,-1,-1
B:58,0,-1,-1,-1
B:59,0,-1,-1,-1
B:60,0,-1,-1,-1
B:61,0,-1,-1,-1
B:62,0,-1,-1,-1
B:63,1,-1,-1,-1
B:64,3,-1,-1,-1
B:65,0,-1,-1,-1
B:66,0,-1,-1,-1
B:67,0,-1,-1,-1
B:68,0,-1,-1,-1
B:69,0,-1,-1,-1
B:70,0,-1,-1,-1
B:71,0,-1,-1,-1
B:72,0,-1,-1,-1
B:73,0,-1,-1,-1
B:74,0,-1,-1,-1
B:75,0,-1,-1,-1
B:76,2,-1,-1,-1
B:77,4,-1,-1,-1
B:78,0,-1,-1,-1
B:79,0,-1,-1,-1
ACH:1,-1
B:80,0,-1,-1,-1
B:81,0,-1,-1,-1
B:82,0,-1,-1,-1
B:83,0,-1,-1,-1
B:84,0,-1,-1,-1
B:85,6,-1,-1,-1
B:86,0,-1,-1,-1
B:87,0,-1,-1,-1
B:88,0,-1,-1,-1
B:89,0,-1,-1,-1
B:90,0,-1,-1,-1
B:91,0,-1,-1,-1
B:92,0,-1,-1,-1
B:93,0,-1,-1,-1
B:94,0,-1,-1,-1
B:95,0,-1,-1,-1
B:96,0,-1,-1,-1
ACH:1,-1
B:97,0,248,-1,-1
B:98,0,-1,-1,-1
B:99,0,-1,-1,-1
B:100,0,248,-1,-1
B:101,0,-1,-1,-1
B:102,0,-1,-1,-1
B:103,0,248,-1,-1
B:104,0,-1,-1,-1
B:105,0,-1,-1,-1
B:106,0,248,-1,-1
B:107,0,-1,-1,-1
B:108,0,-1,-1,-1
ACH:1,-1
B:109,18,-1,-1,-1
B:110,23,305,-1,-1
B:111,86,-1,-1,-1
B:112,86,-1,-1,-1
B:113,2,-1,-1,-1
B:114,6,-1,-1,-1
B:115,15,-1,-1,-1
B:116,15,-1,-1,-1
B:117,7,-1,-1,-1
B:118,0,-1,-1,-1
B:119,1,216,-1,-1
B:120,0,-1,-1,-1
B:121,0,-1,-1,-1
B:122,0,-1,-1,-1
B:123,1,-1,-1,-1
B:124,0,-1,-1,-1
B:125,0,280,-1,-1
B:126,18,-1,-1,-1
B:127,18,-1,-1,-1
B:128,0,-1,-1,-1
B:129,0,-1,-1,-1
B:130,18,-1,-1,-1
B:131,18,-1,-1,-1
B:132,2,-1,-1,-1
B:133,0,-1,-1,-1
B:134,0,280,-1,-1
B:135,5,-1,-1,-1
B:136,5,-1,-1,-1
B:137,3,-1,-1,-1
B:138,0,-1,-1,-1
B:139,0,-1,-1,-1
B:140,0,-1,-1,-1
B:141,0,-1,-1,-1
ACH:1,-1
B:142,0,-1,-1,-1
B:143,0,-1,-1,-1
B:144,9,-1,-1,-1
B:145,20,-1,-1,-1
B:146,9,-1,-1,-1
B:147,1,-1,-1,-1
B:148,0,-1,-1,-1
B:149,46,-1,-1,-1
B:150,11,-1,-1,-1
ACH:1,-1
B:151,4,-1,-1,-1
B:152,19,-1,-1,-1
B:153,0,-1,-1,-1
B:154,0,-1,-1,-1
B:155,0,-1,-1,-1
ACH:1,-1
B:156,0,-1,-1,-1
B:157,0,-1,-1,-1
B:158,0,-1,-1,-1
B:159,0,-1,-1,-1
B:160,0,-1,-1,-1
B:161,0,-1,-1,-1
ACH:1,-1
B:162,4,-1,-1,-1
B:163,1,-1,-1,-1
B:164,6,330,-1,-1
B:165,5,-1,-1,-1
B:166,1,-1,-1,-1
B:167,23,-1,-1,-1
B:168,1,-1,-1,-1
B:169,0,-1,-1,-1
B:170,0,-1,-1,-1
B:171,244,-1,-1,-1
B:172,16,-1,-1,-1
B:173,32,-1,-1,-1
B:174,15,-1,-1,-1
B:175,0,-1,-1,-1
B:176,2,-1,-1,-1
B:177,1,-1,-1,-1
B:178,1,-1,-1,-1
B:179,16,-1,-1,-1
ACH:1,-1
B:180,0,-1,-1,-1
B:181,0,-1,-1,-1
B:182,0,-1,-1,-1
B:183,0,-1,-1,-1
B:184,0,-1,-1,-1
B:185,0,-1,-1,-1
B:186,0,-1,-1,-1
B:187,0,-1,-1,-1
ACH:1,-1
B:188,11,-1,-1,-1
B:189,24,-1,-1,-1
B:190,14,-1,-1,-1
B:191,9,-1,-1,-1
ACH:1,-1
B:192,0,-1,-1,-1
B:193,5,-1,-1,-1
B:194,4,291,-1,-1
B:195,0,-1,-1,-1
B:196,0,-1,-1,-1
B:197,6,-1,-1,-1
B:198,19,-1,-1,-1
B:199,30,304,-1,-1
B:200,9,-1,-1,-1
B:201,13,-1,-1,-1
B:202,10,-1,-1,-1
B:203,2,-1,-1,-1
B:204,6,-1,-1,-1
B:205,6,-1,-1,-1
B:206,1,-1,-1,-1
B:207,10,-1,-1,-1
B:208,0,-1,-1,-1
ACH:1,-1
B:209,23,-1,-1,-1
B:210,14,-1,-1,-1
B:211,21,-1,-1,-1
B:212,12,-1,-1,-1
B:213,9,-1,-1,-1
B:214,47,-1,-1,-1
B:215,4,-1,-1,-1
ACH:1,-1
B:216,5,-1,-1,-1
ACH:1,-1
B:217,1,-1,-1,-1
ACH:1,-1
B:218,24,-1,-1,-1
B:219,31,-1,-1,-1
B:220,12,-1,-1,-1
B:221,24,-1,-1,-1
B:222,52,-1,-1,-1
B:223,3,-1,-1,-1
B:224,8,-1,-1,-1
B:225,29,-1,-1,-1
B:226,8,-1,-1,-1
B:227,10,-1,-1,-1
B:228,46,-1,-1,-1
B:229,265,-1,-1,-1
ACH:1,243
B:230,79,-1,-1,-1
B:231,14,-1,-1,-1
B:232,29,-1,-1,-1
B:233,176,-1,-1,-1
B:234,139,-1,-1,-1
ACH:1,242
B:235,0,-1,-1,-1
B:236,0,-1,-1,-1
**************************
This is the second file. It is in a linear format with the log records separated by newlines
**************************
NAME:VMSSPECT
MODEL:xxxxxx
SWPN:22222
LISTNAME:xxxxx
LISTID:387
LISTVER:1.150
AC: 22222
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111
LOG#: 1
LTYP:A
TID:Monitor
ADVIS: 1
AC_WT:
C/G:
FLT_HRS:
OPER:
CODE:
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:21:10:27
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS: ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:266
BINS:400
0
0
286
356
295
263
211
160
163
431
602
595
543
257
117
97
40
6
16
47
83
79
80
76
47
44
41
27
39
31
84
174
367
701
764
718
516
151
7
20
15
37
54
62
58
22
14
19
19
15
19
51
90
89
84
47
11
10
29
40
34
32
51
60
58
49
21
19
26
30
34
33
34
43
47
44
32
12
1
0
2
3
5
46
90
84
66
22
27
31
17
9
10
23
93
143
159
147
72
15
1
27
55
65
71
78
94
133
130
84
55
31
31
39
44
63
101
106
95
40
10
22
17
16
12
26
64
81
80
63
24
26
19
8
5
0
51
116
127
127
88
35
27
21
20
15
12
60
85
81
75
37
29
37
28
23
24
5
30
44
51
52
26
6
5
13
15
11
27
67
84
91
69
50
39
33
37
39
32
20
3
9
16
23
28
23
20
13
13
10
16
37
49
45
26
11
0
6
9
5
3
19
25
26
24
24
25
27
14
6
19
41
55
52
45
26
12
12
17
16
12
10
17
19
17
8
6
5
6
8
5
3
15
25
27
27
17
8
6
1
2
6
13
25
27
26
26
14
5
0
9
15
13
8
9
12
16
16
9
5
9
13
17
17
13
19
24
27
18
5
0
1
1
2
6
20
31
32
26
12
4
6
8
9
7
3
12
17
19
14
4
1
1
3
8
11
12
12
9
7
5
6
4
3
3
1
1
0
1
4
8
9
7
5
1
0
0
2
1
0
0
0
0
3
7
6
3
0
10
18
19
17
9
1
0
2
4
3
2
7
9
10
11
8
4
0
3
5
3
4
8
8
6
11
12
10
4
0
3
5
8
11
11
10
4
0
2
6
9
8
5
2
0
0
0
0
0
1
5
8
7
3
2
7
11
9
5
1
0
5
8
8
7
8
10
9
2
0
4
8
7
LOG#: 2
LTYP:A
TID:Monitor
ADVIS: 2
AC_WT:
C/G:
FLT_HRS:
OPER:
CODE:
SSET: 1
SNAME:Unknown Regm
DATE:03-04-2007
TIME:23:24:04
SPC#:67089
SDESC:#3 Sync Shaft Radial
VCH:27
ACH:1
FRQM:FREQ
FRQL:0h
FRQH:500.000000h
UNITS: ips
SF:0.001000
WIND:F
AVGS:1
AVGT:1
ISEL:AC_DIV1_X1
RPM:265
BINS:400
0
0
239
131
212
224
195
160
10
449
716
725
................
**************************
This is the third and final example.
**************************
NAME:VMSFAULT
AC:2222222
MODEL: xxxxxxx
FORMAT:001
PN:SWPN 2222222
PN:CORE A008.0005
PN:PLATFORM 3333-00
PN:[Hss #1 Test]:222222
PN:[Hss #2 Test]:222222
PN:BPPN 15328-04 V01.07A CS=C3D043C5
PN:OPPN 15685-06 V01.24A CS=B19504B9 CFGID=SmartMode LOAD=0702040723
PN:DEXPN 15353-10 V0003.0007 CS=C9119471 CFGID=SmartMode LOAD=0509270754
PN:SPPN 12536-24 V04.14A CS=ADA7BA8E
PN:VIBRVID 1.150 SETUPCS=BDD08964
PN:ENGOP 17045-06 01.24A CS=3F462DD8
PN:XILINX 17040-01 01.00B
PN:ENGBP 01.00B
PN:ENGRVID 1.150
LOG#:01
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00003]EALM:ADVISOR
OCCURRENCES: 1
LOG#:02
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00004]EENG:ENGINE CHECK LOG IS 1% of 800 BLK
OCCURRENCES: 1
LOG#:03
DATE:03-09-2007
TIME:11:44:57
FDESC:I[00001]EMON:MONITOR
OCCURRENCES: 1
LOG#:04
DATE:03-05-2007
TIME:10:41:56
FDESC:W[22007]EENG:ENG PARM SIGNAL RANGE FAIL ENG PARM: E2-CHP, SECS = 100
OCCURRENCES: 5
LOG#:05
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E1-QRW, VALUE = +343
OCCURRENCES: 1
LOG#:06
DATE:03-05-2007
TIME:10:41:57
FDESC:W[22004]EENG:ENG PARM XFR INPUT FAULT ENG PARM: E2-QRW, VALUE = +429
OCCURRENCES: 1
ASKER
What about the possibility of using an interpreter? I've read other posts talk about this approach but haven't seen a good example yet. The text files will eventually change format but not very often. Over the past 5 years there has been three versions with additions of definitions which in turn add parameters to the log sections of the files. My end goal is to get the parameters from the log sections into the database enabling the end user to do metrics on the data.
If the regex approach is best I will head down that path I just want to make sure it is the "best path forward", the most efficient, and the most maintainable. I haven't had much experience with regex so I've downloaded the expresso tool to toy around with it. Pretty interesting so far. Might take me awhile.
If the regex approach is best I will head down that path I just want to make sure it is the "best path forward", the most efficient, and the most maintainable. I haven't had much experience with regex so I've downloaded the expresso tool to toy around with it. Pretty interesting so far. Might take me awhile.
I think your first step would be to work out and define an exact set of rules on how this data is formatted.
From a glance I don't think regex is needed (well not a complex one) but there seems to be some complex things going on in deciding what the next bit of data means.
Combined with your requirement to cope with large files, I would base my algorythm around a TextReader. read in data line by line and use sub routines to process chunks of data such as a LOG.
Each line seems to be based on a colon (:) seperated list of values which you can do easily with Split(':')
From a glance I don't think regex is needed (well not a complex one) but there seems to be some complex things going on in deciding what the next bit of data means.
Combined with your requirement to cope with large files, I would base my algorythm around a TextReader. read in data line by line and use sub routines to process chunks of data such as a LOG.
Each line seems to be based on a colon (:) seperated list of values which you can do easily with Split(':')
ASKER
Okay, check my logic here. I envision having a class to read in the "NAME" value of each file. This seems to be the common denominator across each file. After I know which file I am working with I can then push the parsing of the file to a specific class custom to that particular file format. So essentially I will have a class to read each file format. These classes will utilize the TextReader base class to read line by line with subroutines to break up the lines. Oh, not that it matters but the lines in the DEF files are delimited first by a colon (:) and then the rest of the line is csv (,).
Also, There are three other files that have definitions defined in the first part of the file like my first example. These definitions define the data in the log lines listed later in these files. Since these definitions (and entire file structure) will only change probably once a year; do you think it would be a good idea to just define these in a DB table and ignore reading them in? Basically I will look at the version of the file and pull the definitions to interpret the lines?
Also, There are three other files that have definitions defined in the first part of the file like my first example. These definitions define the data in the log lines listed later in these files. Since these definitions (and entire file structure) will only change probably once a year; do you think it would be a good idea to just define these in a DB table and ignore reading them in? Basically I will look at the version of the file and pull the definitions to interpret the lines?
Sounds good to me.
I would be cautious of ignoring reading the definitions, it might save time now, but it quite often leads to issues later on.
I would be cautious of ignoring reading the definitions, it might save time now, but it quite often leads to issues later on.
ASKER
One more thing. This is probably really simple but in the definition files that have logs 1-n How would I keep track of what log I'm on? Also there is one file with the following format:
LOG#: 1
DATE:03-04-2007
TIME:18:35:25
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.06,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.28,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,100.42,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,100.53,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.65,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,18
P:0.00,100.71,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,100.79,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,23
P:0.00,,c,c,c,0.00,0.00,23
P:0.00,,c,c,c,0.00,0.00,22
P:0.00,100.81,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,22
P:0.00,,c,c,c,0.00,0.00,21
*********TO 138 LINES********
TIME:18:36:39
What method would you recommend to keep up with the date and time data for each log entry? A time calculation has to be done. Each record is taken at 1/2 a second.
Sorry so many questions. Just trying to get a good base before I press on.
LOG#: 1
DATE:03-04-2007
TIME:18:35:25
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.06,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.28,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,100.42,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,100.53,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,100.65,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,20
P:0.00,,c,c,c,0.00,0.00,19
P:0.00,,c,c,c,0.00,0.00,18
P:0.00,100.71,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,,c,c,c,0.00,0.00,21
P:0.00,100.79,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,23
P:0.00,,c,c,c,0.00,0.00,23
P:0.00,,c,c,c,0.00,0.00,22
P:0.00,100.81,c,c,c,0.00,0
P:0.00,,c,c,c,0.00,0.00,22
P:0.00,,c,c,c,0.00,0.00,21
*********TO 138 LINES********
TIME:18:36:39
What method would you recommend to keep up with the date and time data for each log entry? A time calculation has to be done. Each record is taken at 1/2 a second.
Sorry so many questions. Just trying to get a good base before I press on.
an option would be to use a while loop to process the logs and track time updates in a local variable
ASKER
How would you propose I right it to the database? Insert each record when I read it or store in an array or some data set before putting it in the DB. I also have to check that the data is not already in the database also before I insert it. I cannot have two records in the DB with the same date time stamp
ASKER
would it be better to read and store the data with each line read or build some sort of collection to upload after all lines are read.
You state this could be large, so you may want to avoid loading all lines.
Each subroutine could keep track of the data it requires, and when finshed generate the sql you need.
e.g. the log subroutine would be triggered if a line startas with "LOG". It would read lines until it has a whole log stored in its local variables, then create the sql and return.
Each subroutine could keep track of the data it requires, and when finshed generate the sql you need.
e.g. the log subroutine would be triggered if a line startas with "LOG". It would read lines until it has a whole log stored in its local variables, then create the sql and return.
ASKER
Forgive my ignorance but what do you mean by "You state this could be large, so you may want to avoid loading all lines."? Are you saying this could be a memory intensive way to read the file? I'm concerned about the time this method will take to complete. Essentially, I'm going to have to read all the lines of the file in so is this going to be a memory issue right? Would I create and destroy a new instance of TextReader with each subroutine to alleviate this?
As far as the subroutines keeping track of the data they require. What do you recommend here? Some sort of multidimensional array? Do you have some example code in mind you could post?
Sorry, I read your comment "after all lines are read" as meaning you will load the whole file before processing it.
If your using the TextReader and processing as you go you should not have any memory problems.
Regarding keeping track of the data, each subroutine can do it in the way that bast suits the syntax.
I can only guess at the syntax from the sample data. e.g. the log sections might always be name: value lines until you reach a blank line or a line starting with "LOG". If so you could use a Dictionary<string, string> to sore the values. I'll try and get the time to do an example routine later.
If your using the TextReader and processing as you go you should not have any memory problems.
Regarding keeping track of the data, each subroutine can do it in the way that bast suits the syntax.
I can only guess at the syntax from the sample data. e.g. the log sections might always be name: value lines until you reach a blank line or a line starting with "LOG". If so you could use a Dictionary<string, string> to sore the values. I'll try and get the time to do an example routine later.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Two more questions and I'll put this to bed. I know I'm probably wearing out my welcome but I truly appreciate your patience and guidance. Normally I would have took what i have and ran with it but I have to get a prototype built to prove the concept. If this can be completed it will save the company 2 million dollars. I have around 10 different file types to read but I'm trying to get started with just the three I've shown you in order to get a prototype running.
Question 1. In your code above. After the values are obtained in "ProcessComplexFile" and "ProcessLog" which of the following would you recommend: 1. Store them in a collection and then iterate through the collection to insert into the database. 2. Create a data set and store the data there until ready to insert. 3. process each line of data into the table with each read. Keep in mind I have to do a check on the table to make sure the data does not currently exist in the table.
Question 2. My ignorance of string manipulation is shining through here but if you can give me some example code to read one or both of the files below I can customize and rewrite it to fit many of the other files. I'm going to define each line below and attach two example files:
FILE # 1: VXPMON.log
********** HEADER INFO*************
Can use the current code provided. Except for the last two lines. There are two semicolons present in these lines
**************************
NAME:VMSMON
MODEL:4444
SWPN:33333
LISTNAME:test
AC: 22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111
********** Definition Lines*************
These repeat and are defined as below
**************************
DEF:B:1:30,255r,270r,6000r
DEF:B:a:bb,cccc,dddd,eeeee
a = Band number
bb = Vib channel number
cccc = Start freq of band
dddd = End freq of band
eeeee = Max freq of spectrum
f;ff = Zoom factor
g = Azimuth channel
hhh = No. of FFT bins
iii = Cycle time(secs)
j = Phase included?
kkk = undefined1
l = undefined2
m = undefined3
n = undefined4
o = undefined5
(p,p,p,p) = unit index
qqqqqqq = Name of band
**********Last Definition Line*************
There is one unique def line listed last
**************************
DEF:UNITINDEX:1,PeakIPS,0.
DEF:UNITINDEX:a,bbbbbb,ccc
a = Unit index number
bbbbbb = Unit name
cccccccc = Scale factor for unit
********** Log File Header*************
**************************
LOG#: 1
DATE:03-04-2007
TIME:18:39:02
********** Log File Entries*************
**************************
REG:0
REG:a
a = Regime number
--------------------------
ACH:1,243
ACH:a,b
a = channel number
b = RPM at start of log
--------------------------
B:1,0,-1,-1,-1
B:a,b,c,d,e
a = Band number
b = Band data point 1
c = Band data point 2
d = Band data point 3
e = Band data point 4
**************************
**************************
**************************
FILE # 2: VXPSPEC.log
File # 2 is pretty much the same as File # 1 except for the log entries are built into one line
********** HEADER INFO*************
Can use the current code provided. Except for the two "PN" lines. There are two semicolons present in these lines.
**************************
NAME:EMONITOR
MODEL:XXXXXX
SWPN:222222
LISTNAME:Eng(-06b)
AC:3333333
PN:[Eng #1]:444444
PN:[Eng #2]:444444
MRVID:1.5000
FORMAT:001
MNAME:test monitor
CTIME:0.500
LOG_FMT:TIME
********** Definition Lines*************
These repeat and are defined as below
**************************
DEF:P:1:AX-NR,0.50,1,1,(1.
DEF:P:a:bbbbb,cccc,d,e,(ff
a = Parameter number
bbbbb = Parameter channel ID
cccc = period in sec
d = # of averages before storing
e = Average type
fff = scale
ggg = bias
hhh = units
iiiii = name of parameter
********** Log File Header*************
**************************
LOG#: 1
DATE:03-04-2007
TIME:18:35:25
********** Log File Entries*************
**************************
P:0.00,,c,c,c,0.00,0.00,20
Each entry in these lines are directly relational to how man definition lines there are (e.g. DEF:P:1, DEF:P:2, DEF:P:3) If there are 39 definition lines then there are 39 entries in these lines every time.)
The time entries between the records can be ignored and skipped over.
**************************
**************************
Again, your help is truly valued and appreciated. As stated before I
vxpmon.log
engmon.LOG
Question 1. In your code above. After the values are obtained in "ProcessComplexFile" and "ProcessLog" which of the following would you recommend: 1. Store them in a collection and then iterate through the collection to insert into the database. 2. Create a data set and store the data there until ready to insert. 3. process each line of data into the table with each read. Keep in mind I have to do a check on the table to make sure the data does not currently exist in the table.
Question 2. My ignorance of string manipulation is shining through here but if you can give me some example code to read one or both of the files below I can customize and rewrite it to fit many of the other files. I'm going to define each line below and attach two example files:
FILE # 1: VXPMON.log
********** HEADER INFO*************
Can use the current code provided. Except for the last two lines. There are two semicolons present in these lines
**************************
NAME:VMSMON
MODEL:4444
SWPN:33333
LISTNAME:test
AC: 22222
MRVID:1.150
MNAME:Monitor
FORMAT:003
PN:[Hss #1 Test]:11111
PN:[Hss #2 Test]:11111
********** Definition Lines*************
These repeat and are defined as below
**************************
DEF:B:1:30,255r,270r,6000r
DEF:B:a:bb,cccc,dddd,eeeee
a = Band number
bb = Vib channel number
cccc = Start freq of band
dddd = End freq of band
eeeee = Max freq of spectrum
f;ff = Zoom factor
g = Azimuth channel
hhh = No. of FFT bins
iii = Cycle time(secs)
j = Phase included?
kkk = undefined1
l = undefined2
m = undefined3
n = undefined4
o = undefined5
(p,p,p,p) = unit index
qqqqqqq = Name of band
**********Last Definition Line*************
There is one unique def line listed last
**************************
DEF:UNITINDEX:1,PeakIPS,0.
DEF:UNITINDEX:a,bbbbbb,ccc
a = Unit index number
bbbbbb = Unit name
cccccccc = Scale factor for unit
********** Log File Header*************
**************************
LOG#: 1
DATE:03-04-2007
TIME:18:39:02
********** Log File Entries*************
**************************
REG:0
REG:a
a = Regime number
--------------------------
ACH:1,243
ACH:a,b
a = channel number
b = RPM at start of log
--------------------------
B:1,0,-1,-1,-1
B:a,b,c,d,e
a = Band number
b = Band data point 1
c = Band data point 2
d = Band data point 3
e = Band data point 4
**************************
**************************
**************************
FILE # 2: VXPSPEC.log
File # 2 is pretty much the same as File # 1 except for the log entries are built into one line
********** HEADER INFO*************
Can use the current code provided. Except for the two "PN" lines. There are two semicolons present in these lines.
**************************
NAME:EMONITOR
MODEL:XXXXXX
SWPN:222222
LISTNAME:Eng(-06b)
AC:3333333
PN:[Eng #1]:444444
PN:[Eng #2]:444444
MRVID:1.5000
FORMAT:001
MNAME:test monitor
CTIME:0.500
LOG_FMT:TIME
********** Definition Lines*************
These repeat and are defined as below
**************************
DEF:P:1:AX-NR,0.50,1,1,(1.
DEF:P:a:bbbbb,cccc,d,e,(ff
a = Parameter number
bbbbb = Parameter channel ID
cccc = period in sec
d = # of averages before storing
e = Average type
fff = scale
ggg = bias
hhh = units
iiiii = name of parameter
********** Log File Header*************
**************************
LOG#: 1
DATE:03-04-2007
TIME:18:35:25
********** Log File Entries*************
**************************
P:0.00,,c,c,c,0.00,0.00,20
Each entry in these lines are directly relational to how man definition lines there are (e.g. DEF:P:1, DEF:P:2, DEF:P:3) If there are 39 definition lines then there are 39 entries in these lines every time.)
The time entries between the records can be ignored and skipped over.
**************************
**************************
Again, your help is truly valued and appreciated. As stated before I
vxpmon.log
engmon.LOG
ASKER
Tiggerito,
I contacted you on your personal website. Check out my email there before you answer the last question.
I contacted you on your personal website. Check out my email there before you answer the last question.
Question 1:
I would store the data in a form that is most conviient, then save it to the database once I have gathered a full set of data.
This would depend a lot on the data structure you decide to use.
Question 2
I don't have time to go through all the structures you've defined. I also thing more info on the destination data structure would be needed to fully understand how this data needs to be parsed.
I would store the data in a form that is most conviient, then save it to the database once I have gathered a full set of data.
This would depend a lot on the data structure you decide to use.
Question 2
I don't have time to go through all the structures you've defined. I also thing more info on the destination data structure would be needed to fully understand how this data needs to be parsed.
First, write ones that encompass entire areas. Something that breaks up the text file into smaller sections. You'll have one that capture the header, the definition, and the log.
Then take each matching group and write regular expressions for each section.