We value your feedback.
Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!
Course Of The Month
8 days, 17 hours left to enrollBecome a Premium Member and unlock a new, free course in leading technologies each month.
Solved
Reducing the time to populate ListBox Controls from database
Posted on 2004-09-30
Hi,
We have a Delphi 6 application that displays a DataGrid and a ListBox that are populated from a table of an SQL Server 2000 database. The DB table contains 1,55,567 rows and it takes about 4 secs to retrieve all rows from the DB.
Now, when we try to loop thru the recordset to populate the ListBox and DataGrid, it takes almost 3 minutes to get the data into the ListBox. I would like to know if there is an apporach using which this delay can be minimised as this is a client server application and the users get anxious about the delay that occurs in populating these controls.
Any help would be appreciated.
Thanx.
We have a Delphi 6 application that displays a DataGrid and a ListBox that are populated from a table of an SQL Server 2000 database. The DB table contains 1,55,567 rows and it takes about 4 secs to retrieve all rows from the DB.
Now, when we try to loop thru the recordset to populate the ListBox and DataGrid, it takes almost 3 minutes to get the data into the ListBox. I would like to know if there is an apporach using which this delay can be minimised as this is a client server application and the users get anxious about the delay that occurs in populating these controls.
Any help would be appreciated.
Thanx.
[X]
Welcome to Experts Exchange
Add your voice to the tech community where 5M+ people just like you are talking about what matters.
- Help others & share knowledge
- Earn cash & points
- Learn & ask questions
2
2- +4
12 Comments
Have you tried Datasets DisableControls, before starting to populate and EnableControls after populate?
I am assuming here that you populate Listbox programmatically.
If you loop through a large number of records you should always call DisableControls to prevent dataset for trying to update datacontrols. In my experience it should be called even if there are no datacontrols linked to a dataset.
If you loop through a large number of records you should always call DisableControls to prevent dataset for trying to update datacontrols. In my experience it should be called even if there are no datacontrols linked to a dataset.
What I do in these situations is the following: Populate a stringlist first with the items you want in the listbox. Then set the BeginUpdate property. I'm not sure of the top of my head if it is a property of tlistbox, or of the items. Then assign the stringlist to the items. Set EndUpdate to mark for completion. The BeginUpdate will block visual updates until you set EndUpdate. This will save a lot of time.
Make the listbox virtual. Make the grid virtual. Or use data-aware listboxes and grids...
Making these components virtual basically means that you just tell them how much items there must be in the grid or listbox, but you don't assign the values yet. You just use the customdraw events to check which item needs to be drawn, then look it up in your recordset and then just draw it in it's place. Don't fill your grid or listbox with real data since that would be a huge waste of memory if you have this many items.
Also, keep in mind that no sane user would ever want to see this much records on his screen. It is insanity by itself to show a user this much data since he can't do anything with it. It's like when you go to the grocery store to buy a banana and the shopkeeper tells you to go find it in this huge truck full fruits.
Lesson number one of good GUI design: NEVER show the user an enormous list of data. Offer them filters and just return e.g. the top-250 records. (And warn them if their filter resulted in more than 250 items in that case.) Or do like Google does... Get all the data and offer it to the user as multiple pages, with about 25 items per page. If the user goes to the next page, just fill the grid and listbox with the next 25 records.
But displaying hundreds of thousands of records to the user is plain stupid.
Making these components virtual basically means that you just tell them how much items there must be in the grid or listbox, but you don't assign the values yet. You just use the customdraw events to check which item needs to be drawn, then look it up in your recordset and then just draw it in it's place. Don't fill your grid or listbox with real data since that would be a huge waste of memory if you have this many items.
Also, keep in mind that no sane user would ever want to see this much records on his screen. It is insanity by itself to show a user this much data since he can't do anything with it. It's like when you go to the grocery store to buy a banana and the shopkeeper tells you to go find it in this huge truck full fruits.
Lesson number one of good GUI design: NEVER show the user an enormous list of data. Offer them filters and just return e.g. the top-250 records. (And warn them if their filter resulted in more than 250 items in that case.) Or do like Google does... Get all the data and offer it to the user as multiple pages, with about 25 items per page. If the user goes to the next page, just fill the grid and listbox with the next 25 records.
But displaying hundreds of thousands of records to the user is plain stupid.
It could be that I'm not quite getting it, but I don't think splitting information into pages is better then showing them as a whole. It still will be a lot of information. I'm with pritaeas here. if you really want to split those rows into multiple pages, a stringlist is quicker than a recordset.
In general, you should just limit the amount of data that you show to the user to something around 100 to 250 records. More than 100 is already quite a large amount, though. If there's more data to show than this, you have several options:
*) Show just a summary, grouping data and calculating totals.
*) Divide the data in multiple pages and allow users to jump to a specific page.
*) Provide the user a way to filter the data so the output is limited to what the user wants to see.
For example, say that you keep track of every phonecall made in your company. Your company has 5000 employees and half of them work for the helpdesk or are salespersons. You could imagine that this would be a lot of phonecalls. Now, after a year, you want an overview. How would you like to see it?
Would you like to see every phonecall in a grid? Let's hope not. Users are more interested in calculated results. E.g. a sum of call lengths per employee. Or the total number of calls per week. Maybe just an overview of all calls of Mr. X on date MM-DD-YYYY.
*) Show just a summary, grouping data and calculating totals.
*) Divide the data in multiple pages and allow users to jump to a specific page.
*) Provide the user a way to filter the data so the output is limited to what the user wants to see.
For example, say that you keep track of every phonecall made in your company. Your company has 5000 employees and half of them work for the helpdesk or are salespersons. You could imagine that this would be a lot of phonecalls. Now, after a year, you want an overview. How would you like to see it?
Would you like to see every phonecall in a grid? Let's hope not. Users are more interested in calculated results. E.g. a sum of call lengths per employee. Or the total number of calls per week. Maybe just an overview of all calls of Mr. X on date MM-DD-YYYY.
Comment only
Am I missing something here...
Quote:
'The DB table contains 1,55,567 rows and it takes about 4 secs to retrieve all rows from the DB.
Now, when we try to loop thru the recordset to populate the ListBox and DataGrid...it takes almost 3 minutes to get the data into the ListBox'
Why are you then putting them into another grid (if this is what you are doing)...
If you need to format them or add extra unbound columns - you should be using a datagrid and populating using unbound events...
Data grids call batches of items which is easy when you think about it (using Top and Bottom predicates).
I can't see a scenario where you would be putting very large numbers of items into a listbox as being realistic in interface terms.
Who could possibly make choices out of millions of items - it makes no sense to me - please explain further.
Voodooman
Am I missing something here...
Quote:
'The DB table contains 1,55,567 rows and it takes about 4 secs to retrieve all rows from the DB.
Now, when we try to loop thru the recordset to populate the ListBox and DataGrid...it takes almost 3 minutes to get the data into the ListBox'
Why are you then putting them into another grid (if this is what you are doing)...
If you need to format them or add extra unbound columns - you should be using a datagrid and populating using unbound events...
Data grids call batches of items which is easy when you think about it (using Top and Bottom predicates).
I can't see a scenario where you would be putting very large numbers of items into a listbox as being realistic in interface terms.
Who could possibly make choices out of millions of items - it makes no sense to me - please explain further.
Voodooman
As already stated a listbox is not suited very well for the job.
It needs to hold *all* rows even while displaying only a small amout of entries.
Using a control to handle a virtual list is the way to go.
Such a control stores only the displayed values. Mike Lischke's TVirtualTreeView is the most renowned control of that sort.
It needs to hold *all* rows even while displaying only a small amout of entries.
Using a control to handle a virtual list is the way to go.
Such a control stores only the displayed values. Mike Lischke's TVirtualTreeView is the most renowned control of that sort.
Featured Post
![[Webinar] How Hackers Steal Your Credentials](https://filedb.experts-exchange.com/files/public/2017/5/20/3315d9d8-8110-4d52-be94-a5aacdd83e4b.png)
Do You Know How Hackers Steal Your Credentials? Join us and Skyport Systems to learn how hackers steal your credentials and why Active Directory must be secure to stop them. Thursday, July 13, 2017 10:00 A.M. PDT
Question has a verified solution.
If you are experiencing a similar issue, please ask a related question
The uses clause is one of those things that just tends to grow and grow.
Most of the time this is in the main form, as it's from this form that all others are called.
If you have a big application (including many forms),
the uses clause in the in…
Introduction
I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need.
This article will address some of the concepts when dealing with a multithreaded delphi database…
In this video we outline the Physical Segments view of NetCrunch network monitor. By following this brief how-to video, you will be able to learn how NetCrunch visualizes your network, how granular is the information collected, as well as where to f…
Sometimes it takes a new vantage point, apart from our everyday security practices, to truly see our Active Directory (AD) vulnerabilities. We get used to implementing the same techniques and checking the same areas for a breach. This pattern can re…
Suggested Courses
Earn Certification
HTML5 Specialist - Certification
Free withPremium
Course of the Month8 days, 17 hours left to enroll
Earn Certification
CompTIA IT Fundamentals
$49.00$44.10
617 members asked questions and received personalized solutions in the past 7 days.
Join the community of 500,000 technology professionals and ask your questions.