Solved

Storing and retrieving data from files

Posted on 2004-03-21
20
391 Views
Last Modified: 2012-05-04
I am working on a project that requires me to store large amounts of sensative  data to a file and later retrieve that data and show it in a Grid all sorted out in a nice and neat way.

I have chosen to use the (GridView) compoent by Berg Software to load the data into and my questions are.

1.) How would I go about extracting only parts of the data that is in the file (ex. a subject for an email) and displaying it as one of the cells in the grid and then later when I click on that cell extracting the body of the above email message and showing it in a new window.

2.) Since I will be working with large amounts of sometimes sensative data I would like to be able to compress and encrypt the data inside of the files so nobody can view it outside of the program so my question is obviously what is the best way to do that and ofcourse uncompress and decrypt it again so the program can use it.

I know this all might sound abit confusing so I will try my best to give you an example of the program.

Example:

A person opens up the program and writes an email to a friend with the subject of "Hey There" and then the body of the email is "Long time no see"  and then presses a button and what should happen next is the subject and the body of the email should be encrypted and stored in a local file that will then be compressed to keep it's size down.

Next when the user opens the program again the local file is automatically uncompressed and decrypted and the subject of the email (ie. Hey There) will be displayed in cell of the Grid and when the user clicks on that cell another window will open up with the body of that message (ie. Long time no see)

Anyway I hope that makes things abit clearer and to summerize again what I need help with is the best way to compress/decompress  and encrypt/decrypt the data from the file and extract only part of the data from the file when the program starts and then later extract the rest as needed.

I know this is more like 3 questions instead of 1 but they all fit together and I did not want to start a new topic for each of them and so I thought I would just ask them all here and give it a high point value.

Cheers


0
Comment
Question by:ezpete
  • 12
  • 8
20 Comments
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10643681
You could use a database. I would recommend a password protected one, and one  which offers data encryption and data compression.

I would recommend - DBISAM from http://www.elevatesoft.com/, however, it is not free.
You can also compile and embed the DBISam engine into your application, with a very small footprint.

Another off the top of my head , which is available free (OPenSOurce):

MySQL - http://www.mysql.com/

Shane

0
 

Author Comment

by:ezpete
ID: 10644255
While that might be an option it's not one that I can use because I am using D6 personal ed which does not include any db support so that will not work for me.

I have been thinking that if I could somehow compile the file like delphi does with DCU files then it would solve the encryption and compression problems in one wack but not being a delphi guru I have no idea how this could be done or even if it can be done at all.

The most important feature that I need is the ability to only extract only part of the local file when the main program starts and then extract the other parts as needed.

Anyway both questions still stand and thx for any and all replies.
0
 

Author Comment

by:ezpete
ID: 10644337
Now that I think of it I could prob use TIniFile class to store and retrieve the information the way I need to do it but again I am still stuck on the problem of the encryption and compression
0
 

Author Comment

by:ezpete
ID: 10644354
Nvm I think I have a method to do everything I need to do ... I am going to ask that this question be closed and if I need any other help I will just post a different one.
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10644411
You could use TFileStream or TMemoryStream.

This will read and right to a binary file. In addition, you could add your own Encryption and compression schemes.

I have added a STream example below - examples to store an object with an example of (integer, string, real, datetime)

There are many encryption units available on the net you could pass your info through before streaming it out.

See code below

Shane

unit UntCommon;

interface

Uses Classes, SysUtils, Contnrs, Graphics, DateUtils;

type

 TMyObject = class
 private
  fMyDateTime: TDateTime;
  fMyString: String;
  fMyInteger: Integer;
  fMyReal: Integer;
 public
  constructor Create(AMyDateTime: TDateTime;
                             AMyString: String;
                             AMyInteger: Integer
                             AMyReal: Real);
  destructor Destroy; override;
  property MyDateTime: TDateTime read fMyDateTime write fMyDateTime;
  property MySTring: String read fMySTring write fMySTring;
  property MyInteger: Integer read fMyInteger write fMyInteger;
  property MyReal: Integer read fMyReal write fMyReal;
 end;

 procedure SaveToFile(FileName: String);
 procedure LoadFromFile(FileName: String);


 procedure WriteStreamInt(Stream : TStream; Num : integer);
 procedure WriteStreamReal(Stream : TStream; Num : real);
 procedure WriteStreamStr(Stream : TStream; Str : string);
 function ReadStreamInt(Stream : TStream) : integer;
 function ReadStreamreal(Stream : TStream) : real;
 function ReadStreamStr(Stream : TStream) : string;
 procedure WriteStreamDate(Stream : TStream; ADate : TDateTime);
 function ReadStreamDate(Stream : TStream) : TDateTime;


var
 MyObjects: TObjectList;

implementation

procedure SaveToFile(FileName: String);
var
 MemStr: TMemoryStream;
 I: Integer;
begin
 MemStr:= TMemoryStream.Create;
 WriteStreamInt(MemStr, MyObjects.Count);
 for I:= 0 to MyObjects.Count - 1 do
 begin
  WriteStreamDate(MemStr, TMyObject(MyObjects[I]).MtyDateTime);
  WriteStreamStr(MemStr, TMyObject(MyObjects[I]).MyString);
  WriteStreamInt(MemStr, TMyObject(MyObjects[I]).MyInteger);
  WriteStreamReal(MemStr, TMyObject(MyObjects[I]).MyReal);
  nd;
 MemStr.SaveToFile(FileName);
 MemStr.Free;
end;

procedure LoadFromFile(FileName: String);
var
 MemStr: TMemoryStream;
 OCount: Integer;
 AMyDateTIme: TDateTime;
 AMySTring: String;
 AMyInteger: Integer;
 AMyReal: Real;
 AMyObject: TMyObject;
 I: Integer;
begin
 MemStr:= TMemoryStream.Create;
 MemStr.LoadFromFile(FileName);
 OCount:= ReadStreamInt(MemStr);
 for I:= 0 to OCount - 1 do
 begin
  AMyDateTIme:= ReadStreamDate(MemStr);
  AMySTring:= ReadStreamStr(MemStr);
  AMyInteger:= ReadStreamInt(MemStr);
  AMyReal:= ReadStreamReal(MemStr);
  AMyObject:= TMyObject.Create(AMyDateTime,  AMySTring, AMyInteger, AMyReal);
  MyObjects.Add(MyObject);
 end;
 MemStr.Free;
end;

procedure WriteStreamInt(Stream : TStream; Num : integer);
{writes an integer to the stream}
begin
  Stream.Write(Num, SizeOf(Integer));
end;

procedure WriteStreamReal(Stream : TStream; Num : real);
{writes an integer to the stream}
begin
  Stream.Write(Num, SizeOf(Real));
end;

procedure WriteStreamDate(Stream : TStream; ADate : TDateTime);
{writes an integer to the stream}
begin
  Stream.Write(ADate, SizeOf(TDateTime));
end;

procedure WriteStreamStr(Stream : TStream; Str : string);
{writes a string to the stream}
var
  StrLen : integer;
begin
  {get length of string}
  StrLen := Length(Str);
  {write length of string}
  WriteStreamInt(Stream, StrLen);
  {write characters}
  Stream.Write(Str[1], StrLen);
end;

function ReadStreamInt(Stream : TStream) : integer;
{returns an integer from stream}
begin
  if Stream.Read(Result, SizeOf(Integer)) < SizeOf(Integer) then
    Result := -1;
end;

function ReadStreamReal(Stream : TStream) : real;
{returns an integer from stream}
begin
  if Stream.Read(Result, SizeOf(real)) < SizeOf(real) then
    Result := 0.00;
end;

function ReadStreamDate(Stream : TStream) : TDateTime;
{returns an integer from stream}
begin
  if Stream.Read(Result, SizeOf(TDateTime)) < SizeOf(TDateTime) then
    Result := -1;
end;


function ReadStreamStr(Stream : TStream) : string;
{returns a string from the stream}
var
  StrLen : integer;
begin
  {get length of string}
  StrLen := ReadStreamInt(Stream);
  if StrLen > -1 then begin
    {set string to get memory}
    SetLength(Result, StrLen);
    {read characters}
    Stream.Read(Result[1], StrLen);
    end
  else
    Result := '';
  {end; if StrLen > -1 else}
end;

constructor TEvent.Create(AMyDateTime: TDateTime;
                                      AMyString: String;
                                      AMyInteger: Integer
                                      AMyReal: Real);
begin
 fMyDateTime:= AMyDateTime;
 fMyString:= AMyString;
 fMyInteger:= AMyInteger;
 fMyReal:= AMyReal;
end;

destructor TEvent.Destroy;
begin
 //
 inherited Destroy;
end;

initialization
begin
 MyObjects:= TObjectList.Create;
 MyObjects.OwnsObjects:= True;
end;

finalization
begin
 MyObjects.Free;
end;

end.
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10662435
"Now that I think of it I could prob use TIniFile class to store and retrieve the information the way I need to do it but again I am still stuck on the problem of the encryption and compression"

WHat is wrong with the solution above?

Here is a some basic Encrytion & Decyption to go with it.

Shane


Example:

var
 Password: String;

 Encrypt(Password,12345));
 Decrypt( Password, 12345)




unitCrypto;

interface

uses Forms, Windows, Classes, SysUtils;

const
  C1 = 52845;
  C2 = 22719;


function Decrypt(const S: String; Key: Word): String;
function Encrypt(const S: String; Key: Word): String;

implementation

function Encrypt(const S: String; Key: Word): String;
var
  I: byte;
begin
  SetLength(Result, Length(S));
  for I := 1 to Length(S) do begin
    Result[I] := char(byte(S[I]) xor (Key shr 8));
    Key := (byte(Result[I]) + Key) * C1 + C2;
  end;
end;

function Decrypt(const S: String; Key: Word): String;
var
  I: byte;
begin
  SetLength(Result, Length(S));
  for I := 1 to Length(S) do begin
    Result[I] := char(byte(S[I]) xor (Key shr 8));
    Key := (byte(S[I]) + Key) * C1 + C2;
  end;
end;

end.
0
 

Author Comment

by:ezpete
ID: 10682818
Prob nothing but I came up with a solution of my own before you posted the code and since I already started coding it I will just stick with it.  I will however give you the points if you can answer this question.

All I need to do with the password stuff is to protect the "preferences form" of my program so that a password will be required to change any settings and since the end user is the one who will initially sets the password I would like to know how to store that password inside the exe itself in it's encrypted form or if you can think of a better way to store the password then I am open for suggestions.

0
 
LVL 11

Accepted Solution

by:
shaneholmes earned 475 total points
ID: 10683049
Add the follwing unit to your application, and choose use the Password variable to compare

the default is admin, until you change the variable 'password', then the new one gets saved.

Shane



unit UntCommon;

interface

uses Forms, Windows, Registry, Classes, SysUtils, Dialogs, ComCtrls, RzListVw;

const
  C1 = 52845;
  C2 = 22719;


function Decrypt(const S: String; Key: Word): String;
function Encrypt(const S: String; Key: Word): String;

procedure ReadRegistry;
procedure WriteRegistry;

var
 Password: String;

implementation

function Encrypt(const S: String; Key: Word): String;
var
  I: byte;
begin
  SetLength(Result, Length(S));
  for I := 1 to Length(S) do begin
    Result[I] := char(byte(S[I]) xor (Key shr 8));
    Key := (byte(Result[I]) + Key) * C1 + C2;
  end;
end;

function Decrypt(const S: String; Key: Word): String;
var
  I: byte;
begin
  SetLength(Result, Length(S));
  for I := 1 to Length(S) do begin
    Result[I] := char(byte(S[I]) xor (Key shr 8));
    Key := (byte(S[I]) + Key) * C1 + C2;
  end;
end;

procedure WriteRegistry;
var
 Reg: TRegistry;
begin
  Reg := TRegistry.Create;
  try
    Reg.RootKey := HKEY_CURRENT_USER;
    if Reg.OpenKey('\Software\Your Company Name\Your App Name\', True) then
    begin
     Reg.WriteString('Password',Encrypt(Password,12345));
     Reg.CloseKey;
    end;
  finally
    Reg.Free;
  end;
end;

procedure ReadRegistry;
var
 Reg: TRegistry;
begin
  Reg := TRegistry.Create;
  try
    Reg.RootKey := HKEY_CURRENT_USER;
    if Reg.OpenKey('\Software\Your Company Name\Your App Name\', True) then
    begin
     if Reg.ValueExists('Password') then
     Password:= Decrypt(Reg.ReadString('Password'),12345)
    else
     Password:= 'admin';
     Reg.CloseKey;
    end;
  finally
    Reg.Free;
  end;
end;

end.
0
 

Author Comment

by:ezpete
ID: 10683922
If I understand your code correctly then there seems to be something missing from the WriteRegistry Function because when I call it the reg key is made but there is no data in it.

Also how do I retrieve the data from the registry once I have put it there. I have tried the following but it does not work.

procedure TForm2.Button1Click(Sender: TObject);
begin
 If Edit1.Text = Unit3.ReadRegistry then
 showMessage('congrats')
 else
 ShowMessage('try again');
end;

I then used the following and it worked but it does not seem this would be the correct way to use the function

procedure TForm2.Button1Click(Sender: TObject);
begin
 If Edit1.Text = Unit3.Password then
 showMessage('congrats')
 else
 ShowMessage('try again');
end;

Lastly If I understand all of this correctly all someone has to do to defeat the protection is to delete the folder in the registry that contains the key data or even the key data itself and they will have to do is enter a blank password and it will let them in and if this is correct then I think I will need something abit more secure.

Cheers,
Pete
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10683939
Hold one sec, im testing it

Shane
0
Free Trending Threat Insights Every Day

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

 
LVL 11

Expert Comment

by:shaneholmes
ID: 10683968
give me your email and ill send you a small app & explain

Shane
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10683996
ok, yes, the decryption & Excryption wouldn't be tough to break for a crypto person, but if you bury it where thay can't find it, it makes it real tough

if Reg.OpenKey('\1\eqorcx\a2\', True) then

now they have no idea where to look

Shane
0
 

Author Comment

by:ezpete
ID: 10684003
foofactor@hotmail.com
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10684012
Here is an example

Shane

procedure TForm1.btnLogOnClick(Sender: TObject);
begin
 ReadRegistry;
 if Edit1.Text = Password then
  Label1.Caption:= 'Logged On';
end;

procedure TForm1.btnChangePasswordClick(Sender: TObject);
begin
 Password:= edit2.Text;
 WriteRegistry;
end;

procedure TForm1.btnLogOffClick(Sender: TObject);
begin
  Label1.Caption:= 'Logged Off';
end;
0
 

Author Comment

by:ezpete
ID: 10684034
actually it would be quite easy because all they need to do is use a regmonitor and they could see in realtime where the information is being pulled from and they would not need to crack the encryption all they would have to do is delete the folder and the protection is bypassed altogether.
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10684050
oK, i emailed the example to you... it is very quick and crude, but should give you an idea on how to use it.

Shane
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10684063
Is your program going to be that important that all the hackers will want to tear it apart. I may want it copy - <SMILE>

Well, then you could excrypt the string and  store it in a resource file embedded in your application

Sahne

0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10684099
Remember also, nothing is 100%. If they want to break it, they will.  It all depends on how important the program is.

Shane
0
 

Author Comment

by:ezpete
ID: 10684204
The password protection feature is for the system admin of the computer the program is running on,  the program itself is actually freeware I am just trying to add all the nice lil extras that make it better.

Btw thx for the demo app and the code.

Pete
0
 
LVL 11

Expert Comment

by:shaneholmes
ID: 10684214
Your welcome

 Shane
0

Featured Post

How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

Join & Write a Comment

Suggested Solutions

Objective: - This article will help user in how to convert their numeric value become words. How to use 1. You can copy this code in your Unit as function 2. than you can perform your function by type this code The Code   (CODE) The Im…
Have you ever had your Delphi form/application just hanging while waiting for data to load? This is the article to read if you want to learn some things about adding threads for data loading in the background. First, I'll setup a general applica…
It is a freely distributed piece of software for such tasks as photo retouching, image composition and image authoring. It works on many operating systems, in many languages.
This video demonstrates how to create an example email signature rule for a department in a company using CodeTwo Exchange Rules. The signature will be inserted beneath users' latest emails in conversations and will be displayed in users' Sent Items…

757 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

18 Experts available now in Live!

Get 1:1 Help Now