Solved

64k string size limitation

Posted on 1998-09-23
10
199 Views
Last Modified: 2012-08-13
I am using VB 6

I have a log file (text format) that I am parsing. Here are the steps I am using:

1. Open the text file
2. Start the loop
3. Read the 1st line, and extract the needed info
4. Write the extracted data to a new file
5. Loop again until EOF

The program starts, and then errors. It looks like its a limitation of the 64K string size. If I go into the text file, and make it a 10-20 lines in length, it works fine.

The file I am extracting the info out of is 624K and will grow daily.

How can I overcome this limitation?
0
Comment
Question by:purphaze
10 Comments
 

Author Comment

by:purphaze
ID: 1436249
Edited text of question
0
 
LVL 2

Expert Comment

by:dabellei
ID: 1436250
   Open "file.txt" For Input As #1
   
    Do While Not EOF(nfic)
        Line Input #nfic, texte_share
        Rem  texte_share is a variable wich contain the text for the current line
    Loop
    Close #1



This should work even for big file.
0
 
LVL 2

Expert Comment

by:dabellei
ID: 1436251
replace nfic by 1 in the open statement
0
 
LVL 12

Expert Comment

by:mark2150
ID: 1436252
The LINE INPUT command presumes that there are occassional breaks in the line. A break is defined as a <CR>/<LF> pair. Not all programs place these at the end of the line. Look at the string you're being passed. Some log programs write a fixed length string every time they fire and do not supply the <CR>/<LF> pair. If this is the case then you'll have to change your OPEN and LINE INPUT statements with

OPEN "file.txt" FOR BINARY AS #1
DO WHILE NOT EOF(1)
    Buffer = SPACE( RecordLen )
    GET #1, , Buffer
.
LOOP

The RecordLen is a constant that is the exact number of bytes in one record.


0
 

Author Comment

by:purphaze
ID: 1436253
Here is the code I am using:
Open "c:\internet\data\log.txt" For Input As #1
Open "TESTFILE.TXT" For Output As #2
    Do While Not EOF(1)
        Line Input #1, strTemp
        intPos1 = InStr(1, strTemp, "EMPIRE\")
        intPos2 = InStr(intPos1, strTemp, " ")
        UserID = Mid(strTemp, intPos1 + 7, intPos2 - intPos1 - 7)
        Write #2, UserID
    Loop
    Close #1

If I make the log.txt file about 20 lines, it works. If I leave it the normal size (1400+ lines) it errors.

Note - each line in the log file is a different size. Some of the lines contain 284+ characters
0
What Is Threat Intelligence?

Threat intelligence is often discussed, but rarely understood. Starting with a precise definition, along with clear business goals, is essential.

 

Author Comment

by:purphaze
ID: 1436254
Here is the code I am using:
Open "c:\internet\data\log.txt" For Input As #1
Open "TESTFILE.TXT" For Output As #2
    Do While Not EOF(1)
        Line Input #1, strTemp
        intPos1 = InStr(1, strTemp, "EMPIRE\")
        intPos2 = InStr(intPos1, strTemp, " ")
        UserID = Mid(strTemp, intPos1 + 7, intPos2 - intPos1 - 7)
        Write #2, UserID
    Loop
    Close #1

If I make the log.txt file about 20 lines, it works. If I leave it the normal size (1400+ lines) it errors.

Note - each line in the log file is a different size. Some of the lines contain 284+ characters
0
 

Author Comment

by:purphaze
ID: 1436255
Unless I'm missing something (and that happens alot) it looks like I already implemented what you suggested. I tried to impement another Expert's suggestion mark1250, but it looks like each record must be the same size. I got close to my end result, as it would return 1402 records, but it is not completing the entire log.
0
 

Author Comment

by:purphaze
ID: 1436256
Unless I'm missing something (and that happens alot) it looks like I already implemented what you suggested. I tried to impement another Expert's suggestion mark1250, but it looks like each record must be the same size. I got close to my end result, as it would return 1402 records, but it is not completing the entire log.
0
 
LVL 1

Accepted Solution

by:
cd74013 earned 50 total points
ID: 1436257
I think the problem might be unrelated to the file records but to the logic of the program:

        intPos1 = InStr(1, strTemp, "EMPIRE\")
        intPos2 = InStr(intPos1, strTemp, " ")
       
if the string "EMPIRE\ " does not occur in the record read from the file then intpos1 will be set to zero.  In that case the next statement will try to find the substring from position zero in the string which will cause a run time error.

0
 

Author Comment

by:purphaze
ID: 1436258
OK, I manually searched the log file I am trying to extract info from, and there are about 6 lines in the log file  that do not contain the string "Empire\". Setup an "On Error" and it went thru the entire log file. The code I ended up with is:
Open "c:\internet\data\log.txt" For Input As #1
    Open "TESTFILE.TXT" For Append As #2
    Do While Not EOF(1)
        Line Input #1, strTemp
        On Error Resume Next
        intPos1 = InStr(1, strTemp, "EMPIRE\")
        intPos2 = InStr(intPos1, strTemp, " ")
        UserID = Mid(strTemp, intPos1 + 7, intPos2 - intPos1 - 7)
        Write #2, UserID
    Loop
       
    End

Now that I have a basic understanding of this, time to expand the extraction criteria
0

Featured Post

Highfive Gives IT Their Time Back

Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

Join & Write a Comment

There are many ways to remove duplicate entries in an SQL or Access database. Most make you temporarily insert an ID field, make a temp table and copy data back and forth, and/or are slow. Here is an easy way in VB6 using ADO to remove duplicate row…
Have you ever wanted to restrict the users input in a textbox to numbers, and while doing that make sure that they can't 'cheat' by pasting in non-numeric text? Of course you can do that with code you write yourself but it's tedious and error-prone …
Get people started with the process of using Access VBA to control Outlook using automation, Microsoft Access can control other applications. An example is the ability to programmatically talk to Microsoft Outlook. Using automation, an Access applic…
This lesson covers basic error handling code in Microsoft Excel using VBA. This is the first lesson in a 3-part series that uses code to loop through an Excel spreadsheet in VBA and then fix errors, taking advantage of error handling code. This l…

744 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

14 Experts available now in Live!

Get 1:1 Help Now