• Status: Solved
  • Priority: Medium
  • Security: Public
  • Views: 213
  • Last Modified:

Using, saving and loading strings

Hi,

I wish to use some data structure to store a var called "name" that is a string, I need to store about 25.000 different names (i.e 'Joh Paul Sartre') in the memory, then save it to a file, then load it back in memory. I tried with stringlist but it consumes a lot of ram memory (also very slow to access), the same with trecord, what is the best way to do that? Code is welcome.  (I don't want to use any database componnent, I wish to alocate it in vars).

0
scoopbh
Asked:
scoopbh
3 Solutions
 
Scay7Commented:
Tried an Ini File ? or do you not want to use any files ?

Peace Scay7
0
 
Russell LibbySoftware Engineer, Advisory Commented:

Regarding the string list, how much memory do you think it is using (and moreover, how are you determining this?). If you have 25000 items, each an average of say 20 bytes, then you are you looking at roughly 700KB of memory (data and list management) for the string list. And how were you acessing the items in the list? Sequentially by index, or were you trying to perform finds on the list? If you were attempting a find, hopefully the list was sorted, else the list does sequential scans. A little more information would be helpful, but from the sounds of it, a stringlist (used correctly) should work fine for what you are attempting to do.

Regards,
Russell


0
 
LMuadDIbCommented:
you can use XML as database/storage file
example can be found here:
http://www.skch.net/columns/xml_delphi.html
0
Independent Software Vendors: We Want Your Opinion

We value your feedback.

Take our survey and automatically be enter to win anyone of the following:
Yeti Cooler, Amazon eGift Card, and Movie eGift Card!

 
TheRealLokiSenior DeveloperCommented:
more memory, or a database sound like the best solutions :-(

I don't have many problems with a TStringList.
as long as I do

SL.BeginUpdate
try
...blah blah big process eg.
    SL.LoadFromFile('c:\names.txt');
    SL.Sort;
finally
    SL.EndUpdate;
end;
it's reasonably fast.
Searching it would be ok if you "sort" it once the first time you use it, but from then on, just use an insertion sort
eg. stick the new name in the right place with 'Insert()'


Another way I've done this is with inifiles, as scay suggests. (give him the points if you decide to use inifiles)
something like :-

with TIniFile.create('names.ini') do
try
    s := ReadString('Names', 'Name 1', '');
    if s <> '' then showmessage('first name is ' + s);
    for i := 1 to 10 do
    begin
        s := ReadString('Names', 'Name 1', '');
        if s <> '' then showmessage(name ' + inttostr(i) + ' is ' + s);
    end;
finally
    free;
end;

you could even make a function such as (assuming the ini file is already created)

uses inifiles;

var
    myini:TIniFile;

function TForm1.FOrmCreate(sender: tobject);
begin
    myini := TINIFile.create('c:\names.ini');
end;

function TForm1.FormDestroy(sender: tobject);
begin
    myini.free;
end;

function TForm1.GetOnename(i: integer): string;
begin
    result := ReadString('Names', 'Name 1', '');
end;
   

1 problem with this, is if you wish to delete name "25" you can only clear it, or set it to "*DELETED*" as a flag, shuffling all the other names up by one would be very time consuming
but at least you could do a
wihle GetOneName(i) <> '' do
begin
... blah
    inc(i);
end;
to read all the names.

A file of records would be faster than a TINIFile, but I can't remember exactly how to do this anymore, so long ago...
and you say trecord was too slow for you

function TForm1.GetOneName(namenum: integer): string;
    var
        F: File of ROneNameRec;
        s: string;
        OneNameRec: ROneNameRec;
    begin
        result := '';
        assignfile(F, 'c:\temp.txt');
        filemode := 64;
        reset(F);
        if ( (namenum >=0) and (namenum < filesize(F)) ) then
        begin
            Seek(F, namenum);
            Read(F, OneNameRec);
            result := OneNameRec.name;
        end;
        closefile(F);
    end;

function TForm1.AddOneName(s: string): string;
    var
        F: File of ROneNameRec;
        OneNameRec: ROneNameRec;
    begin
        OneNameRec.Name := S;
        assignfile(F, 'c:\temp.txt');
        filemode := 64;
        reset(F);
        Seek(F, filesize(F));
        Write(F, OneNameRec);
        closefile(F);
    end;

something like that anyway...probably got the filemode wrong, and you need a check if the file does not exist yet, and you wish to create the first record..
0
 
LMuadDIbCommented:
why not a Hash TstringList?

here is some info about hashing:
http://www.mygnet.com/articulos/delphi/283/

if scoopbhs data structure isnt changed often, and is used primarily for searching then hashing is the way to go.
scoopbh hasnt said anything about adding/deleting items from the list ...

Andre N Belokon freeware hashtrie is nice, the example provided shows a dictionary text file loaded and searched (25k items) and the speed is very good. I have used the hashtrie myself for string searching / duplicate comparing-
http://delphi.icm.edu.pl/ftp/d20free/hashtrie.zip
0
 
atul_parmarCommented:
You said (I don't want to use any database componnent, I wish to alocate it in vars). Can you tell the reason why?

I recommend to go with TClientDataset.
0
 
mikelittlewoodCommented:
I suppose it would depend if he wants to ship the midas.dll with his application or not atul_parmar
0
 
David_WardCommented:
@TheRealLoki

Instead of TIniFile, try TMemIniFile ... it will be so very much faster ...
0
 
TheRealLokiSenior DeveloperCommented:
TMemIniFile reads the ini once, and bufers it all in nested stringlists. It then caches changes.
Unfortunately, scoopbh is concerned about memory usage with his big lists.
0

Featured Post

Concerto's Cloud Advisory Services

Want to avoid the missteps to gaining all the benefits of the cloud? Learn more about the different assessment options from our Cloud Advisory team.

Tackle projects and never again get stuck behind a technical roadblock.
Join Now