?
Solved

Monitoring filesystem for changes (Best practice)

Posted on 2015-02-17
4
Medium Priority
?
166 Views
Last Modified: 2015-04-04
Synopsis:
I am creating a commercial application which will monitor all system drives for changes to specific file-types. It will eventually extend to allowing monitoring of network drives, as well.

To give a little bit of background, the intention is to watch for modifications to a specific array of file types. This will inspect the files after modification to detect mass-scale changes by a malicious application. It needs to run quickly with as few resources as possible. (The more low-level, the better)

The application will start as a Windows-only development, but may eventually be ported to other platforms. Regardless, this question is for Windows practice. Ideally, it will need access to monitor non-windows filesystem types, but this could be a down-the-road thing.
I am looking into several options.

Options currently considered:

 1. Windows Audit Policies - this would monitor logs to see changes.
   
Potential drawback: I've read that you need to add lines for each root folder (cannot use a drive)
   
This feels a little "hacky" for an commercial application. I don't like the requirement to alter windows settings
   
Not sure if this would flag the process that made the changes, also (?)
2. FileSystemWatcher (https://msdn.microsoft.com/en-us/library/System.IO.FileSystemWatcher(v=vs.110).aspx)
   
I've read that this can be buggy and has certain cases in which it fails, one of which being network drives being dropped/re-added. (http://stackoverflow.com/questions/239988/filesystemwatcher-vs-polling-to-watch-for-file-changes)
   
Everything seems to be able to be mitigated, with some work; however, it's not feeling very 'enterprise', to me. (I could be wrong!)
3. Kernel Hooking
   
May not be possible
   
Drawback: Until the application is recognized and ranked on par with anti-virus applications, however, it can create some red flags, which would be a hassle in launching.

 4. WindowsAPI
   
I have worked extensively with both documented and undocumented WinAPI in the past, so I'm comfortable working in this.
   
I don't know much about the specifics on what is available. I have read a little about ReadDirectoryChangesW

     
Drawbacks: I hear that it's finicky to get configured correctly, but it is possible to use this route. There were several other issues, but I don't recall them off-hand.
   
I question whether this will:
        - A) Support all that I want to accomplish
        - B) Deliver the best speed, stability, and least tax on system resources/overall system stability
5. File System Driver
   
I don't know much about it, yet, but this sounds as though it may be the right route to go.

       
I may be missing something, and it's very possible that I might have some of what I've said above wrong. I have not done too much beyond initial research phase, so I will likely be updating this as I gain more insight. Any insight or thoughts you may have to offer will be most appreciated!

I know that there are several questions somewhat similar to this, but for this
specific application, I felt that nothing I found directly answered my question.

Finally...

A better synopsis of the application's purpose:

 
Monitor the filesystem for specific types of files being modified
(Optionally) monitor entire directories for any files being modified
Scan the file after modification to detect changes that may indicate a corruption of file type (invalid header, etc.)
If a large amount of files get modified that match our flagging, pause/freeze the application doing the writing and prompt the user for action.

Question/Request:

I am looking for thoughts and opinions on how to go about performing this large-scale monitor operation in the fastest, most efficient, enterprise-ready manner. This application will be on-par with commercial anti-virus and therefore, the best practices need to be applied, adhering to this standard!

Thanks very much, all, for any insight you might be able to offer on this!

Open in new window

0
Comment
Question by:Riktor
[X]
Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people just like you are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
  • 2
4 Comments
 
LVL 86

Accepted Solution

by:
jkr earned 2000 total points
ID: 40617032
'ReadDirectoryChangesW()' would have been the API that I'd have thought of first. There used to be a sample called 'fwatch' on MSDN (https://msdn.microsoft.com/en-us/library/aa230342%28v=vs.60%29.aspx) that illustrates it, but it seems to have been removed. But anyway, the article at http://www.codeproject.com/Articles/950/CDirectoryChangeWatcher-ReadDirectoryChangesW-all ("CDirectoryChangeWatcher - ReadDirectoryChangesW all wrapped up") uses the same approach and you can use the demo executable to see if it meets your requirements.
0
 
LVL 86

Expert Comment

by:jkr
ID: 40623765
You had quite some time to try that out now, so what are your results?
0

Featured Post

Need protection from advanced malware attacks?

Look no further than WatchGuard's Total Security Suite, providing defense in depth against today's most headlining attacks like Petya 2.0 and WannaCry. Keep your organization out of the news with protection from known and unknown threats.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

This article aims to explain the working of CircularLogArchiver. This tool was designed to solve the buildup of log file in cases where systems do not support circular logging or where circular logging is not enabled
IF you are either unfamiliar with rootkits, or want to know more about them, read on ....
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.
The viewer will learn how to clear a vector as well as how to detect empty vectors in C++.
Suggested Courses

764 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question