C++: map and hash_map performance

Posted on 2008-11-07
Last Modified: 2013-12-14
Hi, a general question on map and hash_map:
I am doing a performance enhancement on a program; the program processes a data file every 5 minutes.

Initially the program used the STL map to store the (key, data) pair. But it seems like it slows down after a while (about one day) then picks up the speed after a period of time (after several hours).

Then the hash_map is used since the the hash_map can find the key faster. But then the same behavior happens on the new implementation of using  a hash_map.

So now the quesiton is: does this kind of behavior have somehting to do with the memory allocation? The program (map or hash_map) takes a data file, read the content to set the key and its value; but at some point of time the map or hash_map erases the pair once the key/data is no longer needed.

The only cause I can come out with is that: the allocation of memory is done for evvery new data pair, then when this data pair is erased from the map/hash_map, the memory is not released for a period of time (a lot of (key, data) pairs are added and removed from time to time) so the new memory allocation of new pairs takes a long time (to find a good memory location). At some point of time, the OS comes in to reclaim the memory (from the deleted data pairs) so the process can run fast again.

Not sure this is exactly what happened; so experts please offer your comments on how/what I should do to improve the performance.

Question by:cw1592
    LVL 21

    Expert Comment

    Use a code profiler to look at performance and easily identify exactly areas could be improved.  Red-gate ants would be an example.

    Author Comment

    BTW, the program is running on a Linux box.

    Accepted Solution

    some testing shows that the allocation of the value in the pair of (key, value) has the key impact on the performacne; using a memory/data allocation call from the Boost library can be used to improve the perfromace of the STL map.

    Write Comment

    Please enter a first name

    Please enter a last name

    We will never share this with anyone.

    Featured Post

    Highfive Gives IT Their Time Back

    Highfive is so simple that setting up every meeting room takes just minutes and every employee will be able to start or join a call from any room with ease. Never be called into a meeting just to get it started again. This is how video conferencing should work!

    The purpose of this article is to demonstrate how we can upgrade Python from version 2.7.6 to Python 2.7.10 on the Linux Mint operating system. I am using an Oracle Virtual Box where I have installed Linux Mint operating system version 17.2. Once yo…
    Exception Handling is in the core of any application that is able to dignify its name. In this article, I'll guide you through the process of writing a DRY (Don't Repeat Yourself) Exception Handling mechanism, using Aspect Oriented Programming.
    THe viewer will learn how to use NetBeans IDE 8.0 for Windows to perform CRUD operations on a MySql database.
    The viewer will learn how to use and create new code templates in NetBeans IDE 8.0 for Windows.

    759 members asked questions and received personalized solutions in the past 7 days.

    Join the community of 500,000 technology professionals and ask your questions.

    Join & Ask a Question

    Need Help in Real-Time?

    Connect with top rated Experts

    11 Experts available now in Live!

    Get 1:1 Help Now