C++ page fault tracking?

The brute force method of avoiding page faults is to have no swap file. :)

Are you experiencing any performance penalty at all? Millions per day would mean a fast spinning disk if it were really related to disk swapping. I suppose the swap isfrom cahce and not frmo disk.

I don't know of any way to break on a page fault. As to why they happen, does any of this sound like it might be a problem?

"Cache Misses and Page Faults

Missed cache hits, on both the internal and external cache, as well as page faults (going to secondary storage for program instructions and data) slow the performance of a program.

A CPU cache hit can cost your program 1020 clock cycles. An external cache hit can cost 2040 clock cycles. A page fault can cost one million clock cycles (assuming a processor that handles 500 million instructions/second and a time of 2 millisecond for a page fault). Therefore, it is in the best interest of program execution to write code that will reduce the number of missed cache hits and page faults.

One reason for slow programs is that they take more page faults or miss the cache more often than necessary. To avoid this, it's important to use data structures with good locality of reference, which means keeping related things together. Sometimes a data structure that looks great turns out to be horrible because of poor locality of reference, and sometimes the reverse is true. Here are two examples:

Dynamically allocated linked lists can reduce program performance because when you search for an item or when you traverse a list to the end, each skipped link could miss the cache or cause a page fault. A list implementation based on simple arrays might actually be much faster because of better caching and fewer page faults even  allowing for the fact that the array would be harder to grow, it still might be faster.

Hash tables that use dynamically allocated linked lists can degrade performance. By extension, hash tables that use dynamically allocated linked lists to store their contents might perform substantially worse. In fact, in the final analysis, a simple linear search through an array might actually be faster (depending on the circumstances). Array-based hash tables (so-called "closed hashing") is an often-overlooked implementation which frequently has superior performance."

You can try to programmatically guarantee your process gets more physical memory using api's such as SetProcessWorkingSetSize() and VirtualLock(), but you can adversely effect the rest of the system if you don't use them wisely.

If I had to guess, I would bet it is the SQL dlls that your code uses to do the SQL calls that is causing all the page faults, and this might be difficult for you to control.  Are you are seeing performance issues due to excessive paging?

To add to above comment:

Did you know for sure that the page faults came from your application?

If for example your disc is fragmented you naturally would get page faults for any file access. If you run your app in the debugger you necessarily increase page faults as the debugger has to load files and data which were spreaded widely somewhere on the disc. Each build will produce a lot of temporary files where each of them would arise page faults as it always is new data that couldn't be part of the cache.

In your app you can try to minimize page faults, e. g. by preferring stack before heap, i. e. by not using pointers where you allocated storage by 'new' operator. If reading files use one huge buffer and read the file with one single read. Make a modular code where you avoid thousands of code lines. Create a function if you need the same sequence of statements a multiple time. That way you increase the chance that a new part of code already could be found in cache.

interesting food for thought..  there is a lot of file reading, writing, deleting going on.. maybe it is just disc related.. I have some study to do I guess.

wayside wrote:
>>If I had to guess, I would bet it is the SQL dlls that your code
>>uses to do the SQL calls that is causing all the page faults,
>>and this might be difficult for you to control.

I restarted my agent application on a clean machine and watched it for a bit in task manager while it wasn't "doing anything" other than every 5 seconds opening a connection to the database and executing a stored procedure, one that returns no results, only causes some processing on the database server. The database was "clean" (it's a DEV environment) so there was nothing to process. No images were moving, no data was moving, only the app opening a CDatabase (MFC), issuing a call to execute a stored procedure, and then destroying the CDatabase.  These calls happen every 5 seconds.

Watching the task manager, every 5 seconds, I saw an average of 1415 new page faults. They increment at this rate even though the app is doing nothing.   I can't maintain open connections as the DB server can go down.. Is this just the nature of the beast?

I am not necessarilly opposed to "living with it" if there is no inherent problem in doing so.. I guess the page fault count is just showing a count of the number of times it happens.. Unless Windows is leaving little fragments and things behind that would eventually cause the system to come crumbling down.  

There's no inherent problem, as long as the excessive paging is not causing performance issues. At worst you get a little extra wear and tear on your disk drive.

I made a small test app that does nothing but create a new CDatabase, open the connection, execute the command, close the connection and then destroy the CDatabase instance.  This produced approximately 700 page faults per iteration.

I then did   pDB = new CDatabase() one time on app startup, and destroyed it on shutdown, so that the utility would do nothing but Open, execute and close.

This time it only produced 70 page faults per iteration, and they were all on the CDatabase::OpenEx() command. So I guess I can't reduce them beyond that point, as that's all inside MFC.

Additionally:

CDatabase::Execute() produced zero page faults, and CDatabase::Close() produced 3

The one other interesting experiment to try would be to burn one connection by keeping it open all the time, and create a 2nd CDatabase object to do the real work.

Or not - seems like you've got the pf's down to a pretty small number. :)

a second to do the real work? I'm not sure I understand what you are getting at. Simply for experimental purposes? Nothing wrong with that!

Yes, I was able to reduce the page faults from 28,000 every hour to 1400. (roughly)

much better :)

My thought was that if the OpenEx() is causing the page faults, keep a connection open all the time. You said you can't maintain an open connection all the time because the db can go down, so this connection exists purely to keep all the components loaded.

A second connection would be opened/executed/closed as needed to do the SQL.

It's a waste of a connection though, so not great for a permanent solution, but it would be interesting to see how many pf's it would eliminate.

Oh I see what you mean. Yes when I get a second I'll try that and report back the results.

>>Did you test with a CDataBase member where the OpenEx was done only once?

I have not though I will.

I first ran the tests with debugger opened but reported results as seen running release build.

It  makes sense, your comments.. I really have no choice, I cannot open and keep open a DB connection as the DB is on a different computer and I have to able to handle a loss of connectivity in near real time..  Unless perhaps there were a callback mechanism that informed me if that connection were lost. Then again perhaps I could leave it open and do a check IsOpen() prior to attempting any reads or writes.. That may limit the page faults, but would it be significantly more resource intensive to keep a connection open? Ultimately there will be potential for several clients to be connecting to my 'agent' app and I wouldn't want to create a condition that limits it's ability to respond to clients.

Good food for thought for sure.

>>My thought was that if the OpenEx() is causing the page faults, keep a connection open all the time.

I tried that - creating an always open connection, and then another to do the 'real work'

In thise case there were an average of 15 page faults per call, rather than 70.

>>>>Did you test with a CDataBase member where the OpenEx was done only once?

When OpenEx was called only once, there were zero page faults. I would still hesitate to keep a connection always open, even if I did test for IsOpen() prior to executing anything.. Or maybe that is not a rational hesitation on my part, I'm not sure. What are your thoughts?

How heavily loaded is the database server?

Database connections are likely a more limited resource than time (as measured by page faults), so I would be inclined to live with the page faults. You've already reduced the page faults by a lot, the amount you are still getting doesn't seem that bad to me.

>>How heavily loaded is the database server?

At this point in time, it is not. This product is still under development and will not grow rapidly in the short term, but  the intent is to design it to support heavy loads.

I did make a version that keeps a connection open, and I check IsOpen() prior to use, so all my server loss exception handling still functions properly, and I have reduced it from 1400 page faults an hour to less than a hundred and tick count profiling of my functions shows improved performance.

It will of course be something to watch and tune as usage increases