Problem with rc form

Hi,
further to this
http://www.experts-exchange.com/questions/28704842/Problem-to-project.html

which static text control, should I choose below?
21
LVL 12
HuaMin ChenProblem resolverAsked:
Who is Participating?

[Product update] Infrastructure Analysis Tool is now available with Business Accounts.Learn More

x
I wear a lot of hats...

"The solutions and answers provided on Experts Exchange have been extremely helpful to me over the last few years. I wear a lot of hats - Developer, Database Administrator, Help Desk, etc., so I know a lot of things but not a lot about one thing. Experts Exchange gives me answers from people who do know a lot about one thing, in a easy to use platform." -Todd S.

sarabandeCommented:
as told, delete the File Number controls (4 controls) and remove the associated code.

then make the form wider and higher such that the Binary File Info could take the 'New Name' static text and associated edit control and button 'Update Name'. give good names for the id's as suggested multiple times. by right-click on the new controls you may add members and event handlers similar to the already existing code.

Sara
HuaMin ChenProblem resolverAuthor Commented:

as told, delete the File Number controls (4 controls) and remove the associated code.
I think I should remove Record number and Item number, right?
sarabandeCommented:
no. record number and item number were needed for the info blocks and the get record function.

Sara
Big Business Goals? Which KPIs Will Help You

The most successful MSPs rely on metrics – known as key performance indicators (KPIs) – for making informed decisions that help their businesses thrive, rather than just survive. This eBook provides an overview of the most important KPIs used by top MSPs.

HuaMin ChenProblem resolverAuthor Commented:
Many thanks Sara.

Is it only to delete the items highlighted in red below?

24c
HuaMin ChenProblem resolverAuthor Commented:
Any advice? Thanks a lot to Sara.
sarabandeCommented:
as told, only delete 4 controls regarding file number of index files. also remove functionality in dialog cpp and dialog h for these controls.

the index files with file numbers were only for temporary use. it makes less sense to evaluate them further since they were no longer used for retrieval purposes.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Very sorry Sara. Is it fine that I delete the highlighted items in above? If yes, definitely, I would further delete their related codes.
HuaMin ChenProblem resolverAuthor Commented:
Good day Sara,
Is it fine that I remove any reference to the names like
#define IDC_RAD_NAME                    1001
#define IDC_RAD_NUMBER                  1002
#define IDC_RAD_DATARECORD              1003
IDC_EDIT_FILE_PATH

Open in new window

sarabandeCommented:
no, if you still want to use `'get record' button and access records of either name index, number index, and item data.

yes, if you only want a program, to change name of an item record.

i would recommend to keep the 'get record' functionality, since it may allow you to verifiy you whether the update functionality worked correctly or not.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many thanks.
Is it fine that I remove the whole line like

EDITTEXT        IDC_EDIT_FILE_PATH, 62, 94, 199, 14, ES_AUTOHSCROLL

Open in new window


?
sarabandeCommented:
yes. you should calculate the path nevertheless if the radio box selection changed and fill the member m_strFilePath with constant strings as suggested earlier.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many thanks Sara.

It says "Unloaded" below. How to load it?
tt281.png
sarabandeCommented:
in the solution explorer tree select project binaryfileviewer (not solution), right-click and load project.

or. close the solution and reopen what should load the project as well.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Good day Sara,

to the current form below
 28
I think I really need one "console application" that is to search the previous generated list of files, are having the relevant name inside, or not. If not, within the above rc2 form, we will append both name and number to the relevant files.
HuaMin ChenProblem resolverAuthor Commented:
BTW, here is what I get, to run ReadBinaryFile
29a
if I put the same Name Key (highlighted below), I get the following
29b
is there anything wrong?
sarabandeCommented:
the item number shown in the output window surely is wrong. should be 1 to 10.

the rest looks ok. did you scroll the data record whether the searched key is at another item number?

"console application" and rc form don't fit so well together. normally you would have client-server. the server is a service, means a background program which neither has a gui nor a console. the client is either a console or a gui program and sends requests to the server which then does the updates or retrievals to the database.

note, we have some kind of database created by savebinaryfile and accessed by readbinaryfile.

the mfcapplication9 should it make possible to verify if write and read have worked correctly.

all this should work without errors before we think to add new items or to update existing items.

also note, a database with sorted index files is not so much suitable to be updated or to add keys, since this means that the index files needs to be written again.

if you want to have the database to be suitable for updates, new items, and deletions, we have to redesign the index files. they need to be organized as b-trees such that updates only require to update a few records rather than rewrite the whole index file.

Sara
HuaMin ChenProblem resolverAuthor Commented:

they need to be organized as b-trees such that updates only require to update a few records rather than rewrite the whole index file.
Many thanks Sara.

can I have more details to re-design ReadBinaryFile project?
sarabandeCommented:
do you really want to redesign all we made to a client-server architecture?
it is not only readbinaryfile but also savebinaryfile which needs to be redesigned for index files.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Yes, can we step-by-step do it? Firstly, ReadBinaryFile, and later SaveBinaryFile and other projects, if necessary?
sarabandeCommented:
yes. first i will describe the new index files which need to be a tree rather than a sorted list.

is it still a goal to have 10 million of items stored in the database?

Sara
HuaMin ChenProblem resolverAuthor Commented:
Yes, correct.

Many thanks Sara.
sarabandeCommented:
savebinaryfile currently is a 64-bit application. readbinaryfile as well but actually it could be 32 bit since it doesn't need huge containers for index since it has to read only a few records to look for a given key in binary mode. mfc applications are 32-bit windows and there are no 64-bit libraries at all since winapi is 32-bit and mfc is class wrapper for winapi. if we could make savebinaryfile - or better - the functioniality to create a huge database with 10 million items such that the final database has more than 4 gb - if we could make it 32 bit, we could put all database functions - create, initial fill, read, add, update, delete - into a dll and have one mfc application where you could invoke all these functions and show the results immediately. since the dll also must be 32-bit, you can't call 64-bit dll's from 3-bit application, we would need a different design of the index files such that they could be created by a 32-bit dll where the amount of virtual memory is limited.

however, since we already found out, that sorted index files were not suitable for add, update, delete, because for each single update, we would need to move millions of records then, we have to redesign anyhow.

the answer for both is to have the index files organized as bayer trees, b-tree.

in a b-tree index file, records are called buckets and have a size that could take a multiple of index items. the index items are organized as nodes where we could have a left 'pointer' to a bucket that contains keys which are less in order than the current node key and a right pointer to a bucket with keys that were greater than the current key.

for example if you have keys

"aa", "cs", "fe", "go",, "kl", "lh", "lm", "nb", "oo", "oy", "oz", "pa", "rr",  "sx", "ta", "xy"

they could be organized in a b-tree like

bucket 1:    [2]"kl"[3]"oo"[4]"ta"[5]
bucket 2:    [0]"aa"[0]"cs"[0]"fe"[0]"go"[0]
bucket 3:    [0]"lh"[0]"lm"[0]"nb"[0]
bucket 4:    [6]"pa"[7]"sx"[0]  
bucket 5:    [0]"xy"[0]
bucket 6:    [0]"oy"[0]"oz"[0]
bucket 7:    [0]"rr"[0]

to get the sorted list from the tree you would start with the root bucket (bucket 1), then bucket 2
because [2] is first left pointer. in bucket 2 are only leafs which don't have a pointer to left or right bucket. so we get "aa", "cs", "fe", "go" from bucket 2. back to bucket 1 we get "kl", then via right pointer to bucket 3 where we get "lh", "lm", and "nb" from. then "oo" from bucket 1, then to bucket 4 and to left child bucket 6 "oy", "oz", back to bucket 4 "pa", then bucket 7 "rr", back to bucket 4 "sx" back to bucket 1 "ta" and finally "xy" from bucket 5.

if we search for a key in the tree, we do similar to the while loop in the binary search. while in the later the next record was calculated as the mid of an interval, we now can use the bucket pointers - if not 0 - to go left, right, or up until we found the key or found another key that is less or greater in a bucket where our key should have been because of the tree order. so either we found our key or we found the bucket where it should be inserted to.

now we could make use of another benefit bayer trees have. those buckets normally are not full what means that we can "add" a new key and only have to update one single bucket. same benefit if we found a key to delete. in case our key is a leaf we only have to remove the entry from the bucket. an empty bucket we could add to list of free buckets then. if the entry is a node pointing to child buckets left or right, we would extract an entry from those buckets to replace the deleted entry. in any case we finally have a well-sorted tree again.

in our sample tree assume we would like to delete the "pa" key. we find it in bucket 4. since the entry with "pa" is a node pointing to bucket 6 at the left and bucket 7 right, we have two choices. either we use "rr" from bucket 7 to replace "pa" entry in bucket 4. then bucket 7 got empty and can be freed. in bucket 4, the "rr" would no longer point to bucket 7 but has a [0] pointer to the right. the second choice we had was to remove the "oz" from bucket 6 and replace "pa" entry by "oz" entry in bucket 4. then the replaced entry still would point to bucket 6 to the left and bucket 7 to the right. the sequence "oy", "oz", "rr" still is well sorted.

as you see from the samples we have some more things to consider than with sorted lists.

we need
- a pointer to the root bucket
- a list of free buckets where we could get a new one from or put an empty bucket back
- a method to traverse the tree such we get a sorted list again
- a method to search for a key and return a bucket position
- a method to insert a key to a bucket
- a method to divide a full bucket into two buckets
- a method to replace a deleted key by next child key
- a method to merge two buckets into one if the minimum number of entries was violated

for name keys the size of an index record is 20 for the key and 4 for the 32-bit record number of the data file. so we have 24 bytes for an entry. for pointers we need 4 bytes extra for each entry and one right pointer 4 bytes. if we would use a bucket size of 4096 we could store 146 entries in a bucket and have 8 bytes left for most right pointer and number of entries in the bucket. if we assume buckets are at least half full we would need 200,000 buckets for a maximum of 14.6 millions of keys what is about 820 million of bytes.

the free list of buckets best was made by a bitmap of a 4096 byte buffer what is 4096*8 bits == 32768 bits. so for 200,000 buckets we need about 6 buckets to handle the free bitmap. if we use 9 records the bitmap is big enough for 20 million of keys.

so for our name index files we have the following structure:

1 head record of 4k bytes taking root bucket, number of bitmap records, total number of records. length of one entry.
9 bitmap records.
200,000 index bucket records

for the number keys we have only a key length of 8 bytes. so a key entry is  12 bytes and a node is 16 bytes. so we can put 255 entries to a bucket what means we need 12,000 buckets for about 15 million of keys. so we have

1 head record of 4k bytes taking root bucket, number of bitmap records, total number of records. length of one entry.
5 bitmap records.
12,000 buckets.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many thanks. could you please update with more details to adjust the process to create files (that is SaveBinaryFile project), by using b-tree?
HuaMin ChenProblem resolverAuthor Commented:
Many thanks Sara.
Can I know more details to adjust SaveBinaryFile to represent bitmap record and one bucket?
sarabandeCommented:
as suggested we would not use 64-bit savebinaryfile but a new 32-bit dll - called database_server - which could be linked to new mfc application called database_client.

savebinaryfile then is one function of the new server (dll) that could be invoked by a button of mfc client.

we could start two ways. one is to have a new mfc application with one button 'create new database' you could enter name of the database (file), number of items per record, number of records. second we have a dll project (32-bit) with one function that is to create the database files and fill the data file and the index file properly.

second is that we have a console program like savebinaryfile which creates tree index files rather than index files with sorted lists.

the first way already is heading against the final solution towards client-server database. the second firstly tries to use as most of the existing solution as possible.

How do you want to proceed?

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many many thanks Sara. 2nd way can be fine.
sarabandeCommented:
Ok. Create a new 32 Bit win32 console project. Name it 'DataBaseServer'. No Precompiled Headers, empty project, unicode enabled.

You May copy the savebinaryfile.cpp to the New project folder, but rename main function to savebinaryfile since we will need a New main function.

The New main will have a while loop where it waits for a request of a Client program. The First request we implement is to create a New database. For that we will use Code of savebinary file beside of creation of index files.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many thanks Sara.


The New main will have a while loop where it waits for a request of a Client program.

More details to this?
sarabandeCommented:
#include <string>
#include <fstream>

class Request
{
public:
       static Request* GetRequestWithTimeout() 
       {
              return NULL;
       }
       virtual bool Execute() = 0;
};

class StopServerRequest : public Request
{
public:
      bool Execute() { return false; }
         
};

class CreateDBRequest : public Request
{
        std::string m_strDBName;
        std::string m_strFolder;
        int m_numRecords;
        int m_numItemsPerRecord;
       
public:
        CreateDBRequest(const std::string & strDBName, const std::string & strFolder, 
                                          int numRecords,  int numItemsPerRecord )
         :  m_strDBName(strDBName)
         , m_strFolder(strFolder)
         , m_numRecords(numRecords)
         , m_numItemsPerRecord(numItemsPerRecord)
        {}

        bool Execute()
        {
               return false;
        }
};

int main()
{
      while (true)
      {
             Request * pRequest = Request::GetRequestWithTimeout();
             if (pRequest != NULL)
             {
                    bool bok = pRequest->Execute();
                    if (bok == false)
                          break;
          
             }
       }
       return 0;
}

Open in new window


try to find out what happens and ask detailed questions if somewhat is not clear.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Hi Sara,
I copy SaveBinaryfile.cpp to the new project, and how to refer to this cpp file, within the new project?
HuaMin ChenProblem resolverAuthor Commented:

but rename main function to savebinaryfile

Is it case-sensitive? should I name it as "SaveBinaryFile"?

Many thanks.
HuaMin ChenProblem resolverAuthor Commented:
Is it to put your current codes into "DBServer.cpp"? If yes, how about the current codes like
int _tmain(int argc, _TCHAR* argv[])
{
	return 0;
}

Open in new window


within that?
sarabandeCommented:
the filename is not case sensitive.

if you would have created an empty project you wouldn't have any files like DBServer.cpp.

is DBServer a 32-bit win32 console project as suggested? if not, you should exit from vs and delete the project and create again. i want to use the DBServer with winapi what is difficult doing it with 64 bit.

i also suggest to switch-off unicode for the DBServer since we don't need mfc for the server part. it is in he general page of the project configuration settings where you have to choose 'multibyte character set' instead of 'unicode' for character set (both for debug and release configuration).

the DBServer.cpp would get the code i posted. if you created a new empty project you have to 'add .... new item ...' and create an empty dbserver.cpp. if dbserver already is 32-bit you could use the current cpp file but replace all contents with my post. delete stdafx.h and stdafx.cpp from project.

it might be that the code doesn't compile since i am at holiday and don't have a vs compiler. try to solve simple errors like 'undefined symbols' if i made a typo yourself. later we will create a dbserver.h and move all class declarations to the header. if you want you could try yourself. look how we did it with binaryfileheader.h.

the savebinaryfile.cpp not necessarily must compile since we only use it as a code base for to copying existing functionality. so you can but must not add it to the project by 'add ... existing item'.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Many thanks Sara. Please enjoy your holiday and when will you be back?
HuaMin ChenProblem resolverAuthor Commented:
I get this
Error	1	error C4996: 'mbstowcs': This function or variable may be unsafe. Consider using mbstowcs_s instead. To disable deprecation, use _CRT_SECURE_NO_WARNINGS. See online help for details.	c:\dbserver\dbserver\savebinaryfile.cpp	98	1	DBServer

Open in new window


when compiling the current project.

Many thanks
sarabandeCommented:
i will be back next week friday. nevertheless we can continue since most is new code anyhow.

the error you encountered actually is a warning which turned to an error if you don't decrement errorlevel 4.

the mbstowcs converts a narrow string to a wide string. you may replace it by mbstowcs_s which is a little bit safer as it requires you to pass the size of the target buffef as an additional argument. if you search for the reference docs of mbstowcs_s and compare it to those of mbstowcs you should be able to avoid the error.

note, since we try to have dbserver project as a multibyte and not a unicode project, we rarely have much need to convert from single char to wide char. so, this call in savebinaryfile.cpp probably will not be overtaken to dbserver.cpp.

Sara
sarabandeCommented:
as told the first request we try to implement is to a create a new database with 3 files, a data file and two index files. the number of records and the number of items per record are variable. so we could begin to create a small database as a starter, test all basic things and later create bigger databases.

is that right for you?

Sara
sarabandeCommented:
a client-server application contains of at least one client (program) and one server (program). those programs will communicate with methods called program-to-program or short p2p.

on a (32-bit) windows p2p could be made by

- sockets
- files
- pipes
- windows messages
- shared memory

the first could be made in wide or local network but also at on computer.

files and pipes only work on local networks. windows messages and shared memory only for the local computer. beside of the differences all the p2p methods easily can be encapsulated such that they could be exchanged with reasonable efforts if a wider scope is required.

if both the client and the server are supposed to be win32 - regardles whether mfc is involved or not, the easiest p2p is to use windows messages. that would require the server to have a (pseudo) window which is able to receive messages. the request data could be sent via message parameters which may contain handles to global memory. though this is a proved method, i would recommend to use shared memory, since it doesn't require a pseudo window and is more straight-forward. however we have keep in mind that it only works when client and server are located at the same computer.

shared memory at windows is implemented by so-called memory-mapped files. that basically is the functionality to handle files in memory (and let the os do the job to store and load data physically). since memory mapped files also could be used without a file, it is a rather simple method to reserve some memory where two programs could access same time and so pass mutual information between programs p2p.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Good day Sara,

To last line below
	...
	std::string  strtext = os.str().substr(0, maxsize);
	std::wstring wstrtext(strtext.size() + 1, L'\0');
	size_t numChars, numChars2;
	mbstowcs_s(&numChars, &wstrtext[0], numChars2, strtext.c_str(), strtext.size());

Open in new window


what should I initialize 3rd parameter to?

Please enjoy your holidays and best wishes!
sarabandeCommented:
The 1st numchar should be 0, the 3rd numchar2 should be size of buffer. If This is a wstring you have to resize it before calling mbstowcs_s.
I See that it already was resized. So use wrstrtext.length() for initialising.
Sara
HuaMin ChenProblem resolverAuthor Commented:
Good day Sara,
DBServer.cpp is now not calling events of SaveBinaryFile.cpp. what we do next?
sarabandeCommented:
both dbserver.cpp and savebinaryfile.cpp are compiling without errors?

can you post both file contents in a code window?

the Execute function of CreateDBRequest should be like

bool Execute()
{
      int ret = SaveBinaryFile(m_strDBName, m_strDBFolder, 
                                            m_numRecords, m_numItemsPerRecord);
      return (ret == 0);
}

Open in new window


the SaveBinaryFile function is main function of savebinaryfile.cpp

you need to change function prototype to accept the 4 arguments provided in the call. it are the same as in the constructor of CreateDBRequest.

also you need a new header savebinaryfile.h. look at binaryfileheader.h.

the new header should contain (at least) the declaration of the new function SaveBinaryFile. then you can include the header in DBServer.cpp and the call should be accepted both by compiler and linker.

however, it wouldn't compile since we need to move all constants like NUM_ITEMS, NUM_RECORDS to header file and make them variables. also the structures have to be moved similar to that we did in binaryfileheader.

try to use the arguments of the new function to make the SaveBinaryFile function working. of course i will help you when you post me your code. note, since savebinaryfile was a 64-bit it could consume much more memory. if we try to call savebinaryfile by a 32-bit project we firstly have to use much smaller record counts. that will be better when we change from sorted index files to index trees.

Sara
HuaMin ChenProblem resolverAuthor Commented:
Hi Sara,
Where should "CreateDBRequest" be located at? Is the Header similar to the Header appears on Mfc application?

Enjoy your holidays and Have a great weekend!
sarabandeCommented:
it should be put into dbrequest.h

the headers we are using are minimal. they don't contain more than absolutely needed.

#ifndef DBREQUEST_H
#define DBREQUEST_H

#include <string>  // needed for names and folders

class Request
{
    ....

class StopServerRequest : public Request
{
     ....
class CreateDBRequest : public Request
{
    ....


#endif 

Open in new window


if we do so, we can include such a header in an mfc and in a non-mfc project.

Sara

Experts Exchange Solution brought to you by

Your issues matter to us.

Facing a tech roadblock? Get the help and guidance you need from experienced professionals who care. Ask your question anytime, anywhere, with no hassle.

Start your 7-day free trial
sarabandeCommented:
do you proceed into turning main function of savebinaryfile project to SaveBinaryFile function being called in dbserver.cpp?

if not post where you were stuck.

Sara
It's more than this solution.Get answers and train to solve all your tech problems - anytime, anywhere.Try it for free Edge Out The Competitionfor your dream job with proven skills and certifications.Get started today Stand Outas the employee with proven skills.Start learning today for free Move Your Career Forwardwith certification training in the latest technologies.Start your trial today
C++

From novice to tech pro — start learning today.