[Last Call] Learn how to a build a cloud-first strategyRegister Now

x
?
Solved

In Memory Database

Posted on 2006-05-15
8
Medium Priority
?
416 Views
Last Modified: 2010-04-05
I'm looking for a light weight database component to replace a large dataset.. currently I have an array that is now growing to 1Million+ rows.. and memory usage isn't that efficient.

When I search through it I just loop throught the array.. I never expected that it would grow this large and now I need to fix the problem of high memory usage.

The data is only relavent for the session, and is retrieved from the server upon login.

Does anybody have a suggestion for a local database that I can compile in? I have found one so far: here http://www.zeitungsjunge.de/delphi/sqlite3/


0
Comment
Question by:rbohac
8 Comments
 
LVL 28

Assisted Solution

by:2266180
2266180 earned 80 total points
ID: 16682762
here are a few I found:
http://www.delphipages.com/result.cfm?ID=3567
and
http://www.delphipages.com/news/detail.cfm?ID=2259 
I haven't worked with such components so I cannot tell you which one is better.
0
 
LVL 26

Expert Comment

by:Eddie Shipman
ID: 16684760
0
 
LVL 3

Accepted Solution

by:
KyleyHarris earned 80 total points
ID: 16686621
Well. if its efficiency of datamanagement you want then you could use a MemoryDataset. but that wont take up less memory..... probably more. your array will be hard as hell to use, so I agree in changing it, but a memory dataset will have overheads. There are many vendors around that provide memory based TDatasets.
with a million rows perhaps you want to cache it to disk? not memory, by compiling in a lite DBF database or Lite SQL database that is embedded. Also look at flashfiler from TurboPower which is now opensource.

regards
  Kyley
0
VIDEO: THE CONCERTO CLOUD FOR HEALTHCARE

Modern healthcare requires a modern cloud. View this brief video to understand how the Concerto Cloud for Healthcare can help your organization.

 
LVL 17

Assisted Solution

by:TheRealLoki
TheRealLoki earned 80 total points
ID: 16687346
if you're holidng 1 million+ records, a memory dataset is not the best solution
Storing on disk is probably the way to go. (even though it is temporary), but slower.

A)
You could opt for a typed file, with an index file

// the main database file
type
    DBaseRec = Record
                 Deleted : Boolean;
                 Name: string[64];
               end;
Var
   DBase : DBaseRec;
   DBaseFile: File of DBaseRec;

The index file will hold a list of "search fields" and record numbers, so you can quickly "seek" to the record you want.
(the idea is to only have to search the index file, then use it's "record pointer" to getthe "full" data from teh actual database file)

B)
If you're talking Delphi 5, then the BDE will do a (somewhat ridiculed) decent job with very little effort.
(requires installing bde on the machine of course)

C)
If you have a later version of Delphi, I think the BDE is removed. (can probably still downlaod it from borland though)
If you want free, and a DBMS that does not require an administrator, then Firebird is a good choice
It also has an embedded install if no remote access is required
0
 
LVL 26

Assisted Solution

by:Eddie Shipman
Eddie Shipman earned 80 total points
ID: 16690350
How about Firebird embedded. This would give you total control and full SQL compliancy.
http://www.destructor.de/firebird/1.5/embedded.htm

You can store your database as a resource, save it to disk and connect, then when you are done,
delete it.
0
 
LVL 6

Author Comment

by:rbohac
ID: 16690626
wow what a great response. I'm starting to go through and look at all of these this morning.

0
 
LVL 1

Assisted Solution

by:mwbowman
mwbowman earned 80 total points
ID: 16700241

Another option is Absolute Database which supports both disk and memory based structures.

    http://www.componentace.com
0
 
LVL 6

Author Comment

by:rbohac
ID: 16752807
I actually stuck with a compressed stream type that I created, but all of the answers were very helpful. The only thing I can do that will be fair is increase the points and split them

thank you everybody
0

Featured Post

Free Tool: SSL Checker

Scans your site and returns information about your SSL implementation and certificate. Helpful for debugging and validating your SSL configuration.

One of a set of tools we are providing to everyone as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

The uses clause is one of those things that just tends to grow and grow. Most of the time this is in the main form, as it's from this form that all others are called. If you have a big application (including many forms), the uses clause in the in…
Introduction I have seen many questions in this Delphi topic area where queries in threads are needed or suggested. I know bumped into a similar need. This article will address some of the concepts when dealing with a multithreaded delphi database…
We’ve all felt that sense of false security before—locking down external access to a database or component and feeling like we’ve done all we need to do to secure company data. But that feeling is fleeting. Attacks these days can happen in many w…
Is your data getting by on basic protection measures? In today’s climate of debilitating malware and ransomware—like WannaCry—that may not be enough. You need to establish more than basics, like a recovery plan that protects both data and endpoints.…
Suggested Courses
Course of the Month17 days, 18 hours left to enroll

830 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question