meech
asked on
Q: Custom File Programming
Although I have quite a bit of experience programming, all of my apps thus far have been database apps. You learn the confines and the constructs and go with it. I am currently writing an app that calls for "manual" file handling and I would like some pointers from those who have "been there, done that"
I Need:
Fixed length "Customer Information Record"
Multiple, variable length string identifiers related to a customer
The fixed length customer record sounds easy enough, random file, record structures, etc..
It is the other side that I would like some direction...
- Do I store the multiple strings in the same physical file?
- If so, how? what about updates? when the string grows?stability?
- Do I reference a customer # tag in my string structure?
- Should I implement a pointer type system to point to next/prev string? Or is the maintenance of these pointers prohibitive?
- Should I store the strings in a seperate "detail" file?
- Should I use files, or FileStreams?
- If I use FileStreams, could/should I use TStringList stream methods?
- What about performance?
Sheesh, that's a lotta questions. You'd think I was a newbie or something. Anyways...
TIA to everyone's(anyone's) help..
I Need:
Fixed length "Customer Information Record"
Multiple, variable length string identifiers related to a customer
The fixed length customer record sounds easy enough, random file, record structures, etc..
It is the other side that I would like some direction...
- Do I store the multiple strings in the same physical file?
- If so, how? what about updates? when the string grows?stability?
- Do I reference a customer # tag in my string structure?
- Should I implement a pointer type system to point to next/prev string? Or is the maintenance of these pointers prohibitive?
- Should I store the strings in a seperate "detail" file?
- Should I use files, or FileStreams?
- If I use FileStreams, could/should I use TStringList stream methods?
- What about performance?
Sheesh, that's a lotta questions. You'd think I was a newbie or something. Anyways...
TIA to everyone's(anyone's) help..
ASKER
On average the app will have under 100 master records, each with approx 20-30 (avg) string "paramaters" (detail records). Master record size under 2K. Avg detail string length -- about 200 bytes.
Performance-wise I was saying, ideally, i don't want to have to read 3000 records to get the last one. Indexing in some fashion would be preferred.
Were you thinking something along the lines of storing the master file in a record, the details in a Filestreams type, and then a third "index" file which cross references record numbers to Stream offsets or something like that?
Also, i dealing with the sizes I described above it is not inconievable that the entire database be loaded into memory at once (maybe for file compacting/restructuring/r ebuilding/ reindexing , etc.)
String searching - None. The detail file (I think of them as parameters) will need to contain the string and an identifier. No searching will be required.
It is actually a more/less traditional master/client relationship. I just would like variable text length (for file size) and Paradox memo fields scare me. I really like the Jet engine - not an option.
Performance-wise I was saying, ideally, i don't want to have to read 3000 records to get the last one. Indexing in some fashion would be preferred.
Were you thinking something along the lines of storing the master file in a record, the details in a Filestreams type, and then a third "index" file which cross references record numbers to Stream offsets or something like that?
Also, i dealing with the sizes I described above it is not inconievable that the entire database be loaded into memory at once (maybe for file compacting/restructuring/r
String searching - None. The detail file (I think of them as parameters) will need to contain the string and an identifier. No searching will be required.
It is actually a more/less traditional master/client relationship. I just would like variable text length (for file size) and Paradox memo fields scare me. I really like the Jet engine - not an option.
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
I get the idea, the only questions I have remaining are:
1) For the string file, I couldn't use record structures because records need to be a fixed length. I would have to use a TFileStream and store offsets in the index, right?
2) I thought about using the TStringList.SaveToStream/L oadFromStr eam methods. But would 1 stream accomodate multiple TStringLists?
3) For working with Streams: I know the "new" strings are null terminated unlike the old 1.0 strings. Can I just use @MyString to write the string to a stream?
4) What's your experience (if any) with TFileStreams? Do they commit automatically? Stability?
Thanks for the help....
1) For the string file, I couldn't use record structures because records need to be a fixed length. I would have to use a TFileStream and store offsets in the index, right?
2) I thought about using the TStringList.SaveToStream/L
3) For working with Streams: I know the "new" strings are null terminated unlike the old 1.0 strings. Can I just use @MyString to write the string to a stream?
4) What's your experience (if any) with TFileStreams? Do they commit automatically? Stability?
Thanks for the help....
I think you're trying to complicate things here.
Use Delphis native file handling.
Take a look at BlockRead & BlockWrite.
/// John
Use Delphis native file handling.
Take a look at BlockRead & BlockWrite.
/// John
What kind of searching do you have in mind? Substring? None?
Do you mind a separate index file?
Hugeness?
/// John