XML and file of records

Take this for what it is: my opinion.

You probably don't want to use XML as your final storage and retrieval format.  It will make the size of the files you store much larger than need be.  XML is very useful for passing data from one location to another in an independent fashion.  It's not well suited for storing a lot of data.

If you have a small amount of data that you always want to access using XML, then maybe you have an argument to go in that direction.  But every time you want to add new data, you will have to add it formatted in your XML format.

The parsing routines are for tearing the XML apart.  Given a stream, file, etc. containing XML, the parser can traverse the tree and retrieve the requested data.

I believe your best bet is to store your data in a database and either let the database return the results in XML format, if it supports that, or to have your SELECT statements return formatted XML.  This has the added benefit of only returning the data you need right then and allows you to sort the data before it is placed in the XML format.

Please follow-up with more information as to how this data is stored and used if you would like more help (assuming this was helpful :)  ).

hi edsteele...
my database is composed of files only...different files per record type. so if i have a record type 'A', and record types 'B', 'C' and 'D' which have different fields, then i'd have to store them in different files because they can't be uniformly read.  meaning, i have to program how i would like to treat the different files (opening, reading, searching, writing).  but if i use XML then i can store multiple record types within 1 XML file.  there will be no problem in opening the file since XML is text. so when i want to add a new record type which have fields in it all i have to do is add tags and its subnodes.

sample:

 dealtypeA = record
      field1 : integer;
      field2 : str4;
      field3 : str25;
 end;

 dealtypeB = record
      field1 : str1;
      field2 : str40;
      field3 : double;
      field4 : str5
 end;
 
 dealtypeA = record
      field1 : integer;
      field2 : integer;
 end;

 dealtypeA = record
      field1 : integer;
 end;

manipulate 4 record types, if u have and xml database you can define these 4 as

<dealtypes>
  <dealtypeA>
    <field1> </field1>
    .
    .
    .
  </dealtypeA>

  <dealtypeB>
    <field1> </field1>
    .
    .
    .
  </dealtypeB>

  <dealtypeC>
    <field1> </field1>
    .
    .
    .  
  </dealtypeC>

  <dealtypeD>
    <field1> </field1>
    .
    .
    .  
  </dealtypeD>

</dealypes>

also for search the database all you have to do is pass the tag name...much like a query and the parser returns the results. so it simplifies things because you have the field names specified unlike having to define 'specific' records'. so when i want to add a new field to a certain deal type, all i have to do is add a new tag under that deal type which makes all other previous data of that deal type still readable unlike when adding a new field to a record type, it will render all old record types unreadable or useless.

is what i have in mind correct? pardon me for any mistakes, i'm new to this thing and i'd like to find out if what i have in mind is applicable.
thanks a lot!

It sounds like you have the majority of what you need already straight in your mind.  What you want to do is definitely possible.  I want to get some assumptions straight.  My assumptions are:

- You have an XML parsing component and can use it.
- You are OK with the fact that your text files will be bigger in XML format.
- You want to know how to utilize these in a database-like fashion.

You can use the XML parser for searching and querying, however, you still need to write the parts that populate the file.  There are no provisions for deleting or updating without traversing the file as normal.  XML is used primarily to pass data to and from disparate systems that would otherwise not be able to do so.  Therefore, the data doesn't exists very long in that format.  If changes need to be made, the new data is recreated as XML from the original format.  Any individual "snapshot" of XML data almost never gets updated.

If you have your mind set on doing it this way, you currently have only the reading/querying piece in your hands.  You will need to write code to:

- Create new records and append them to the file.
- Delete records from the file.
- Update records in the file (This might be a delete followed by a create, to save code).

If these different record configurations are not related or are related in very predictable ways, I would still recommend an actual database with a different table per file record configuration.

I'm trying to help here, but I feel like I'm not doing you a great deal of good.  If any of my assumptions are wrong, please correct me so I can give you a better answer.

Eric

actually you're right on track, and yes i figured that to update, create and append, and delete new records i'd have to rewrite the whole thing...but since my boss won't allow us to use a database such as SQL Server (to cut the cost of the application...no license fees needed for the database)...i figured searching and querying would be faster(?) and efficient and very much flexible...but the drawback is writing the database...i guess i'll have to treat that independently...thank you very much for ur answer and ideas...if there's anything more you can add i'd greatly appreciate it! thanks again!