Data Input & Manipulation using MVC++

Posted on 2009-04-21
Last Modified: 2013-12-14

I am having a 'dont know where to start' issue. I am a graduate with a little C++ experience. So i may as well have none. I have been reading avidly Object Orienated programming with C++ texts and I am getting to a point where I want to start planning and drawing proposed class diagrams for  internal tool to process a plethora of different log files (.txt).

These log files are extremely large and the data in it is unformatte.I am tasked at producing a programme that will allow the user to extract all the relevant log information and format it in a way that would be suitable for statistical analysis. There are several of these files, and the ideal product of the system would be a single amalgamated storage unit that would be enable the detailed statisitcal analysis and numerical processing of the data witin it.

The next part of the system is an analysis tool that will plot charts and throw out numbers depending on user preferences. All driven from the data that has been sorted from system 1.

What I am really after is some advice with regard to the OOP design. How I might start to think of structuring the programme, the sort of classes and iinvaluable library functions about. I just need something to kick start this...

Perhaps a link, a book or some text you think could help a novice user with a small amount of experience....I really need something, I do have Microsoft Visual Studio 2008 Pro - which should speed up development.

Any suggestions would be great !

Question by:simondopickup
    LVL 1

    Accepted Solution


    first of all, in my opinion Python is a better language for text processing than C++. If you are going to use C++ anyway, the Spirit library (from Boost C++ libraries) may be useful for parsing the text files.

    To kick-start the class design, you could use the CRC-card approach.
    This approach was designed to be used in a team, so it would help if you could find some co-workers (it seems like you have a big task on your hands anyways). It is a brain-storm type of method that helps you create the global outline of the class model.

    Hope this helps!
    LVL 40

    Assisted Solution

    I agree, the Boost libraries will provide Regular Expression and Parsing libraries that you may use for this, but since i don't know how complex your log files are, I can do no more than guess.

    You chose C++ why? For speed, for familiarity, or is it a requirement? If C++ is not set in stone, then I would be looking at scripting languages with regular expression support like Perl or Python, but since you did ask this in the C++ zone, I assume C++ is a requirement. The downside to scripting languages is they require the runtime to be deployed or exist on the system that will run the script. If this is not an application that you will be "distributing" then scripting languages make more sense and are more productive than C++, but will not perform as well. Python and Perl will fall down when given massive data to deal with. So your term "extremely large" needs to be quantified.

    If the log files are really huge (many many gigabytes) and have very complex formats, you might be looking even further than regular expressions.

    But to your OOP question, it sounds like the first step is a log processor. I would not expect to spend much time on a grand OOP model for this. Its a log processor. No need for anything elegant.

    The 2nd phase, the data analysis and charting is too vague in description for me to give any advice. My only overall advice is not to get caught up in object oriented analysis at the expense of getting the job done. Good luck!

    Author Comment


    I have chosen C++ because it is familiar and because we are developing something internally that will..someday be developed into something we can distribute. The log files we are trying to process are in the order of hundreds of megabytes each, yet there can be dozens of them to process and amalgamate into something consisting of columns of variables (output logs effectively recordings of events in time, every time something happens, the simulator platform logs the event against a time stamp).

    Second phase will be loads of GUI's with radio buttons and options for the user to plot certain parameters that are read from the log processing (system1). I am not worrying about that just yet.

    Thanks for your advice...I will leave this question open a little longer in case someone else has any other great advice for me. Thanks again

    Featured Post

    Why You Should Analyze Threat Actor TTPs

    After years of analyzing threat actor behavior, it’s become clear that at any given time there are specific tactics, techniques, and procedures (TTPs) that are particularly prevalent. By analyzing and understanding these TTPs, you can dramatically enhance your security program.

    Join & Write a Comment

    Suggested Solutions

    Introduction: Dialogs (1) modal - maintaining the database. Continuing from the ninth article about sudoku.   You might have heard of modal and modeless dialogs.  Here with this Sudoku application will we use one of each type: a modal dialog …
    Jaspersoft Studio is a plugin for Eclipse that lets you create reports from a datasource.  In this article, we'll go over creating a report from a default template and setting up a datasource that connects to your database.
    The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
    The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.

    734 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    24 Experts available now in Live!

    Get 1:1 Help Now