Solved

speed up additem

Posted on 2002-05-10
35
607 Views
Last Modified: 2010-05-02
I have over 66000 text line (reading from ASCII file) wants to display on the list box and each line is variable length.

The following is my routine.

Public Sub AddToListBox()
    Dim Mess() As String, I As Long, Counter As Long
    Dim Tmp As String, Length As Long, ListHwnd As Long, TextSize As Long

    On Error Resume Next
    LstMessages.Clear
    LstMessages.SetFocus
    ListHwnd = GetFocus()
    LstMessages.Visible = False
    DoEvents
    Me.ScaleMode = 3
    Mess = Split(Messages, vbCrLf)      ' contains all lines with vbcrlf delimiter
    Counter = UBound(Mess)
    Call LockWindowUpdate(ListHwnd)
    For I = 1 To Counter + 1
        Tmp = Mess(I - 1)
        SendMessage ListHwnd, LB_ADDSTRING, 0, ByVal Tmp
        Length = Me.TextWidth(Tmp) + 20
        TextSize = IIf(Length > TextSize, Length, TextSize)
    Next
    Call LockWindowUpdate(0&)
    SendMessage ListHwnd, LB_SETHORIZONTALEXTENT, TextSize, 0&
    LstMessages.Visible = True
    Me.ScaleMode = 1
End Sub


Even I replace the line
SendMessage ListHwnd, LB_ADDSTRING, 0, ByVal Tmp
with
LstMessages.additem Tmp

It tooks over 2 min on my computer (800MHz PIII).
Is any way that it can speed up the additem or using sendmessage.

I would not like to use any third party listbox.




Any idea?
0
Comment
Question by:EDDYKT
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 12
  • 6
  • 5
  • +8
35 Comments
 
LVL 70

Expert Comment

by:Éric Moreau
ID: 7002330
are you sure that you need 66000 lines in a listbox? what is the purpose of this listbox?
0
 
LVL 75

Expert Comment

by:Anthony Perkins
ID: 7002339
IMHO you have about 65,800 items too many.  It is time to re-design.

Anthony
0
 
LVL 28

Expert Comment

by:AzraSound
ID: 7002412
Gotta agree with everyone here...design needs reworking, not the functionality or code.  Can these items be further separated into groups?  Can they be fed to a database for quicker retrieval later?
0
Salesforce Has Never Been Easier

Improve and reinforce salesforce training & adoption using WalkMe's digital adoption platform. Start saving on costly employee training by creating fast intuitive Walk-Thrus for Salesforce. Claim your Free Account Now

 
LVL 16

Expert Comment

by:Richie_Simonetti
ID: 7002459
I must to agree too.
At this point, API is the fastest you can reach.
0
 
LVL 12

Expert Comment

by:roverm
ID: 7002466
I agree as well: Load the first (let's say) 200, then use a background thread to continue the loading of the rest.

D'Mzzl!
RoverM
0
 
LVL 22

Expert Comment

by:rspahitz
ID: 7002505
Rather than jump on the bandwagon...oh, what the heck...image the poor use who has to scroll through 66,000 items to find the desired one.

How about loading everything into a multi-line textbox and whenever the user clicks, determine the current location to determine the "selected" line?

Regardless of your implementation, how can you expect 66,000 lines of variable-length text to be split into small pieces and gathered together quickly?  String parsing is usually the slowest thing a computer does.
0
 
LVL 15

Expert Comment

by:ameba
ID: 7002514
My listbox can load only 32768 items.
If speed is important, you don't want to use Me.TextWidth or Iif.
GetFocus() - this was needed in VB1, now listboxes have hWnd property
If you use LockWindowUpdate, you don't need to set .Visible property
0
 
LVL 16

Expert Comment

by:Richie_Simonetti
ID: 7002752
As ameba pointed, not use textwidth. If you need to call
SendMessage ListHwnd, LB_SETHORIZONTALEXTENT, TextSize, 0&
use the loop on a similar fashion:

Dim ret As Long, sItmText As String
Dim i As Long, lResult As Long, lItmMax As Long
With List1
    For i = 0 To .ListCount - 1
        lResult = SendMessage(.hwnd, LB_GETTEXTLEN, i, ByVal 0)
        If (lResult > lItmMax) Then
            sItmText = .List(i)
            lItmMax = lResult
        End If
    Next i
    ret = SendMessage(.hwnd, LB_SETHORIZONTALEXTENT, lResult, ByVal 0)
End With
even adding it to your existing loop.
But take in mind all other comments regardind redesing.
0
 
LVL 2

Expert Comment

by:selim007
ID: 7003460
1st if ur source is a text file try using the Memory Mapping Method which will accelerate the reading process from the file.
2nd there's no way to load these items in a listbox so fast.
beside the list box won't be able to support that number of items unless you wanna design ur own one
y don't u try integrating HTML under your VB form.
i mean simply use the Microsoft Internet Controls OCX in your application, the load your file as HTML and make a link for each item. of course it's ur code that must create the link and obviously u must use some scripts like java or vb script... it 's not an easy way but it works if u r insisting on having 66000 items

0
 
LVL 1

Expert Comment

by:olx
ID: 7003614
EDDYKT

please hide the listbox before add items to it.
show it again when the adding ends.

doing that it will take from 39 to 8 seconds to add 30000 things to a listbox

but i still think in rspahitz comment "image the poor use who has to scroll through 66,000 items to find the desired one."

hope this helps
0
 
LVL 1

Expert Comment

by:olx
ID: 7003618
i personally think that additem is faster.
but ask Azrasound about it.
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7007154
OK, Let me explain more,
I've an OPC application that print out the system messages (error message) into the output log file. When the new log file exceed 1M size, I will delete the old one and rename the new to old log file.

I've another VB program that read both old and new ASCII file and display on the listbox. Each message will be timestamp and the user should be easy to go through the list box to check the progress of the program.

To AzraSound:
I did try to use database but the write performance from OPC application is poor. However the read is fast if I use database grid. The OPC application should be fast enough to write all log to log file without any delay.

To roverm:
I'm not sure what do you meant here. If I can use timer, the text in listbox must flicking when I add the new text.

To ameba, Richie_Simonetti:
If I have 66000 lines, it will take a lot amoiunt of time to go through each textline and find out the longest line width.
I can take out thr textwidth and try.

To olx:
I did set the listbox to invisible.
0
 
LVL 16

Expert Comment

by:Richie_Simonetti
ID: 7007197
But you could get the actual length of every line when you are populating the list in first place!
0
 
LVL 15

Expert Comment

by:ameba
ID: 7007230
Who needs precise width of the line - make it some big fixed value, or use: len() * textwidth("W")

Did you know that "On Error Resume Next" hides all errors?
Are you sure your listbox can hold more than 32768 items?
0
 
LVL 16

Expert Comment

by:Richie_Simonetti
ID: 7007234
Well, i have to put some ugly stuff to try the code :


Option Explicit
Private Type SIZE
    cx As Long
    cy As Long
End Type

Private Const LB_ADDSTRING = &H180
Private Const LB_SETHORIZONTALEXTENT = &H194
Private Const LB_GETTEXTLEN = &H18A
Private Const WM_GETFONT = &H31
Private Declare Function SendMessage Lib "user32" Alias "SendMessageA" (ByVal hwnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
Private Declare Function ReleaseDC Lib "user32" (ByVal hwnd As Long, ByVal hdc As Long) As Long

Private Declare Function SelectObject Lib "gdi32" (ByVal hdc As Long, ByVal hObject As Long) As Long
Private Declare Function GetDC Lib "user32" (ByVal hwnd As Long) As Long
Private Declare Function GetTextExtentPoint32 Lib "gdi32" Alias "GetTextExtentPoint32A" (ByVal hdc As Long, ByVal lpsz As String, ByVal cbString As Long, lpSize As SIZE) As Long

Public Sub AddToListBox(messages As String)
Dim Mess() As String, I As Long, Counter As Long
Dim lItm As Long
Dim lResult As Long, lItemmax As Long, sString As String
Dim uSize As SIZE
Dim lstDC As Long, lFont As Long, lFontOld As Long
On Error Resume Next
With lstMessages

     .Clear
     .Visible = False
     Mess = Split(messages, vbCrLf)      ' contains all lines with vbcrlf delimiter
     Counter = UBound(Mess)
     For I = 0 To Counter - 1
        lItm = SendMessage(.hwnd, LB_ADDSTRING, 0, ByVal Mess(I))
        lResult = SendMessage(.hwnd, LB_GETTEXTLEN, _
                          lItm, ByVal 0)
        If (lResult > lItemmax) Then
            sString = Mess(I)
            lItemmax = lResult
        End If
    Next
   
    lstDC = GetDC(.hwnd)
    lFont = SendMessage(.hwnd, WM_GETFONT, 0, ByVal 0)
    'Select the font in to the device context, and retain prior font
    lFontOld = SelectObject(hdc, lFont)
    lResult = GetTextExtentPoint32(hdc, sString, lResult, uSize)
    SelectObject hdc, lFontOld
    ReleaseDC .hwnd, hdc
    SendMessage .hwnd, LB_SETHORIZONTALEXTENT, uSize.cx, 0&
    .Visible = True
End With
   
End Sub

Private Sub Form_Load()
AddToListBox "n,mdnf,nds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,ttftftftfdsn,ds" & vbCrLf & _
            "n,mdnf,nds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,mdnf,ncfdedsds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,mdnf,1234543nds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,mdnf,cdxdsnds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,mdnf,nds,fnsfn,dsfn,dsn,ds" & vbCrLf & _
            "n,mdnf,nds,knmmmmmmmmmmmmjhygtffnsfn,dsfn,dsn,ds" & vbCrLf
End Sub


0
 
LVL 28

Expert Comment

by:AzraSound
ID: 7007249
>>I did try to use database but the write performance from OPC application is poor.

Did you ever consider writing to the DB instead of a log file?  An insert into the DB is quick, you would have more ability to query the log, maintain a longer history of messages logged by the OPC server, allow for simple searches, etc.
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7007456
OK, Let me explain more,
I've an OPC application that print out the system messages (error message) into the output log file. When the new log file exceed 1M size, I will delete the old one and rename the new to old log file.

I've another VB program that read both old and new ASCII file and display on the listbox. Each message will be timestamp and the user should be easy to go through the list box to check the progress of the program.

To AzraSound:
I did try to use database but the write performance from OPC application is poor. However the read is fast if I use database grid. The OPC application should be fast enough to write all log to log file without any delay.

To roverm:
I'm not sure what do you meant here. If I can use timer, the text in listbox must flicking when I add the new text.

To ameba, Richie_Simonetti:
If I have 66000 lines, it will take a lot amoiunt of time to go through each textline and find out the longest line width.
I can take out thr textwidth and try.

To olx:
I did set the listbox to invisible.
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7007463
Don't know what happen to last message?->

To ameba:
>>Who needs precise width of the line - make it some big fixed value, or use: len() * textwidth

I will get back to this and try out.

Yes it work over the magic number. I am really sure because I put the line number in front of each line.

To AzraSound:
It treat each log message is separated so that each update message will require open database, open connection, write data and close database. What do you think the speed is.

To Richie_Simonetti:
I'wll try this and let you know.

To ameba:
>>Who needs precise width of the line - make it some big fixed value, or use: len() * textwidth

I will get back to this and try out
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7007466
By the way, I use windows 2000, vb6 sp5
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008340
I try all the method and event comment out the textwidth line. The result is almost the same; just 1 sec faster.

I believe it is not textwidth to slow it down. It is additem.

Any suggestion

Should I try using datasource property?
0
 
LVL 28

Expert Comment

by:AzraSound
ID: 7008512
>>It treat each log message is separated so that each update message will require open database, open connection, write data and close database. What do you think the speed is.

It won't be so slow that you will notice a huge performance loss.  Then, you have the added bonus of quickly searching this database based on the user's preferences.  For example, instead of just coming up and showing every entry in the log, they can enter in maybe a start and end date, messages that contain a certain message, critical messages only, etc.  Plus, then you have an easy means of allowing them to delete all log entries, delete selected entries, etc (all the benefits of working with data in a database).
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008557
AzraSound,

I think of that already. The main reason we drop the database because we may need to write 20 or more messsages in a second into the log file (if required). Using database is not a proper method to do. Do you think so?

The other problem is we have to maintain how many line in the database so that I have to delete the old one before I added the new one in. To doing so the database file size will keep growing as well and I have to compact the database once a while. I don't think we have time to do that in OPC server.

Those all kinds of problems will lead us to drop database method.
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008580
rspahitz :

I tried using multi-text box. I add set my whole text string (text wiht vbcrlf delimiter) into the textbox in one short but the alignment doesn't look really good on text box

8-<
0
 
LVL 28

Expert Comment

by:AzraSound
ID: 7008596
Database speed may be quicker than you think, and if you are hitting the database that often, you could probably just leave a connection open always (and remove the performance issue of opening/closing that connection on every call).  I imagine you would use a full rich database engine like SQL Server.


>>The other problem is we have to maintain how many line in the database so that I have to delete the old one before I added the new one in.

Thats where you could use the rich features of SQL Server.  You can schedule it to run a script every so often (determined by you) that would go and remove records as needed.  This would not require your OPC server to do anything regarding that since it would be running in a separate process (SQL Server's) and executed on a schedule.
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008626
AzraSound,

For those small job, I should mention that I can only use MS access database. SQL server may be a little bit over kill. 8->
0
 
LVL 28

Expert Comment

by:AzraSound
ID: 7008680
Oh, I see...
Are you writing an OPC client or OPC server?
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008690
OPC Server
0
 
LVL 15

Accepted Solution

by:
ameba earned 200 total points
ID: 7008701
It takes 6-7 seconds on Celeron /600 MHz.
It takes longer if I have many applications running (because of swapping to disk).
Here is non-optimized code which adds 66000 items to listview:

' Form1, add listview and two buttons
Option Explicit
Private Declare Function GetTickCount Lib "kernel32" () As Long

Private Sub Form_Load()
    Command1.Caption = "create file"
    Command2.Caption = "read"
    With Me.ListView1
        .ColumnHeaders.Add , , "Message", 30000
        .View = lvwReport
    End With
End Sub

' create test file
Private Sub Command1_Click()
    Dim i As Long, ff As Integer, filename As String
    filename = "c:\test.txt"
   
    ff = FreeFile
    Open filename For Output As #ff
   
    For i = 1 To 66000
        Print #ff, i & " " & String(150, "a")
    Next
   
    Close #ff
   
    Caption = Format(filelen(filename), "#,##0")
End Sub

' read file and fill listview
Private Sub Command2_Click()
    Dim i As Long, ff As Integer, filename As String, filelen As Long
    Dim s As String, msg() As String, tim0 As Long
   
    tim0 = GetTickCount ' start timer
   
    ' read file into big string
    filename = "c:\test.txt"
    ff = FreeFile
    Open filename For Binary Access Read Lock Read Write As #ff
    s = Space$(LOF(ff))
    Get #ff, , s
    Close ff
   
    ' get array of lines
    msg = Split(s, vbCrLf)
    s = ""
   
    ' fill list
    Me.ListView1.Visible = False
    With Me.ListView1.ListItems
        .Clear
        For i = 0 To UBound(msg) - 1
            .Add , , Left(msg(i), 257)
        Next
    End With
    Me.ListView1.Visible = True
   
    Caption = (GetTickCount - tim0) / 1000 & " seconds"
End Sub
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008751
Wow, Listview is lot lot fastest than listbox.
That's great. Thanks ameba

Do you know what is the reason?

0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008752
Good thinking
0
 
LVL 15

Expert Comment

by:ameba
ID: 7008763
Thanks.
>Do you know what is the reason?
Maybe Sorted property of the listbox was set to True?
0
 
LVL 15

Expert Comment

by:ameba
ID: 7008780
' here is how to set width of the column to tmpwidth (PIXELS)
Private Const LVM_SETCOLUMNWIDTH = (LVM_FIRST + 30)
ret = SendMessage(LV.hwnd, LVM_SETCOLUMNWIDTH, 0, ByVal tmpwidth)

' there is also another msg to set column width automatically...
0
 
LVL 15

Expert Comment

by:ameba
ID: 7008794
>set column width automatically...

Private Declare Function SendMessage Lib "user32" Alias "SendMessageA" (ByVal hwnd _
   As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
Private Const LVM_SETCOLUMNWIDTH = &H1000 + 30
Private Const LVSCW_AUTOSIZE = -1
Private Const LVSCW_AUTOSIZE_USEHEADER = -2

Private Sub Form_Click()
   AutoSizeColumns Me.Listview1
End Sub

Private Sub AutoSizeColumns(Listview As Listview, Optional ByVal UseHeader As Boolean = False)
   Dim i As Integer, lParam As Long
   If UseHeader = False Then
       lParam = LVSCW_AUTOSIZE
   Else
       lParam = LVSCW_AUTOSIZE_USEHEADER
   End If
   For i = 0 To Listview.ColumnHeaders.Count - 1
       SendMessage Listview.hwnd, LVM_SETCOLUMNWIDTH, i, ByVal lParam
   Next
End Sub
0
 
LVL 26

Author Comment

by:EDDYKT
ID: 7008804
>>Maybe Sorted property of the listbox was set to True?

No , it is false  8->

Anyway thanks
0
 
LVL 5

Expert Comment

by:Kimpan
ID: 7468440
if the control is not visible during the operations, it will go much smoother.
0

Featured Post

Instantly Create Instructional Tutorials

Contextual Guidance at the moment of need helps your employees adopt to new software or processes instantly. Boost knowledge retention and employee engagement step-by-step with one easy solution.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Most everyone who has done any programming in VB6 knows that you can do something in code like Debug.Print MyVar and that when the program runs from the IDE, the value of MyVar will be displayed in the Immediate Window. Less well known is Debug.Asse…
Article by: Martin
Here are a few simple, working, games that you can use as-is or as the basis for your own games. Tic-Tac-Toe This is one of the simplest of all games.   The game allows for a choice of who goes first and keeps track of the number of wins for…
Get people started with the utilization of class modules. Class modules can be a powerful tool in Microsoft Access. They allow you to create self-contained objects that encapsulate functionality. They can easily hide the complexity of a process from…
Show developers how to use a criteria form to limit the data that appears on an Access report. It is a common requirement that users can specify the criteria for a report at runtime. The easiest way to accomplish this is using a criteria form that a…

730 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question