Data Input & Manipulation using MVC++

Posted on 2009-04-21
Medium Priority
Last Modified: 2013-12-14

I am having a 'dont know where to start' issue. I am a graduate with a little C++ experience. So i may as well have none. I have been reading avidly Object Orienated programming with C++ texts and I am getting to a point where I want to start planning and drawing proposed class diagrams for  internal tool to process a plethora of different log files (.txt).

These log files are extremely large and the data in it is unformatte.I am tasked at producing a programme that will allow the user to extract all the relevant log information and format it in a way that would be suitable for statistical analysis. There are several of these files, and the ideal product of the system would be a single amalgamated storage unit that would be enable the detailed statisitcal analysis and numerical processing of the data witin it.

The next part of the system is an analysis tool that will plot charts and throw out numbers depending on user preferences. All driven from the data that has been sorted from system 1.

What I am really after is some advice with regard to the OOP design. How I might start to think of structuring the programme, the sort of classes and iinvaluable library functions about. I just need something to kick start this...

Perhaps a link, a book or some text you think could help a novice user with a small amount of experience....I really need something, I do have Microsoft Visual Studio 2008 Pro - which should speed up development.

Any suggestions would be great !

Question by:simondopickup

Accepted Solution

maartennieber earned 1000 total points
ID: 24200528

first of all, in my opinion Python is a better language for text processing than C++. If you are going to use C++ anyway, the Spirit library (from Boost C++ libraries) may be useful for parsing the text files.

To kick-start the class design, you could use the CRC-card approach.
This approach was designed to be used in a team, so it would help if you could find some co-workers (it seems like you have a big task on your hands anyways). It is a brain-storm type of method that helps you create the global outline of the class model.


Hope this helps!
LVL 40

Assisted Solution

mrjoltcola earned 1000 total points
ID: 24201066
I agree, the Boost libraries will provide Regular Expression and Parsing libraries that you may use for this, but since i don't know how complex your log files are, I can do no more than guess.

You chose C++ why? For speed, for familiarity, or is it a requirement? If C++ is not set in stone, then I would be looking at scripting languages with regular expression support like Perl or Python, but since you did ask this in the C++ zone, I assume C++ is a requirement. The downside to scripting languages is they require the runtime to be deployed or exist on the system that will run the script. If this is not an application that you will be "distributing" then scripting languages make more sense and are more productive than C++, but will not perform as well. Python and Perl will fall down when given massive data to deal with. So your term "extremely large" needs to be quantified.

If the log files are really huge (many many gigabytes) and have very complex formats, you might be looking even further than regular expressions.

But to your OOP question, it sounds like the first step is a log processor. I would not expect to spend much time on a grand OOP model for this. Its a log processor. No need for anything elegant.

The 2nd phase, the data analysis and charting is too vague in description for me to give any advice. My only overall advice is not to get caught up in object oriented analysis at the expense of getting the job done. Good luck!

Author Comment

ID: 24202015

I have chosen C++ because it is familiar and because we are developing something internally that will..someday be developed into something we can distribute. The log files we are trying to process are in the order of hundreds of megabytes each, yet there can be dozens of them to process and amalgamate into something consisting of columns of variables (output logs effectively recordings of events in time, every time something happens, the simulator platform logs the event against a time stamp).

Second phase will be loads of GUI's with radio buttons and options for the user to plot certain parameters that are read from the log processing (system1). I am not worrying about that just yet.

Thanks for your advice...I will leave this question open a little longer in case someone else has any other great advice for me. Thanks again

Featured Post

Hire Technology Freelancers with Gigs

Work with freelancers specializing in everything from database administration to programming, who have proven themselves as experts in their field. Hire the best, collaborate easily, pay securely, and get projects done right.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Introduction: Hints for the grid button.  Nested classes, templated collections.  Squash that darned bug! Continuing from the sixth article about sudoku.   Open the project in visual studio. First we will finish with the SUD_SETVALUE messa…
If you use Adobe Reader X it is possible you can't open OLE PDF documents in the standard. The reason is the 'save box mode' in adobe reader X. Many people think the protected Mode of adobe reader x is only to stop the write access. But this fe…
The viewer will learn how to use and create keystrokes in Netbeans IDE 8.0 for Windows.
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
Suggested Courses

840 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question