Welcome to Experts Exchange

Add your voice to the tech community where 5M+ people, just like you, are talking about what matters.

  • Help others & share knowledge
  • Earn cash & points
  • Learn & ask questions
Solved

Monitoring filesystem for changes (Best practice)

Posted on 2015-02-17
4
149 Views
Last Modified: 2015-04-04
Synopsis:
I am creating a commercial application which will monitor all system drives for changes to specific file-types. It will eventually extend to allowing monitoring of network drives, as well.

To give a little bit of background, the intention is to watch for modifications to a specific array of file types. This will inspect the files after modification to detect mass-scale changes by a malicious application. It needs to run quickly with as few resources as possible. (The more low-level, the better)

The application will start as a Windows-only development, but may eventually be ported to other platforms. Regardless, this question is for Windows practice. Ideally, it will need access to monitor non-windows filesystem types, but this could be a down-the-road thing.
I am looking into several options.

Options currently considered:

 1. Windows Audit Policies - this would monitor logs to see changes.
   
Potential drawback: I've read that you need to add lines for each root folder (cannot use a drive)
   
This feels a little "hacky" for an commercial application. I don't like the requirement to alter windows settings
   
Not sure if this would flag the process that made the changes, also (?)
2. FileSystemWatcher (https://msdn.microsoft.com/en-us/library/System.IO.FileSystemWatcher(v=vs.110).aspx)
   
I've read that this can be buggy and has certain cases in which it fails, one of which being network drives being dropped/re-added. (http://stackoverflow.com/questions/239988/filesystemwatcher-vs-polling-to-watch-for-file-changes)
   
Everything seems to be able to be mitigated, with some work; however, it's not feeling very 'enterprise', to me. (I could be wrong!)
3. Kernel Hooking
   
May not be possible
   
Drawback: Until the application is recognized and ranked on par with anti-virus applications, however, it can create some red flags, which would be a hassle in launching.

 4. WindowsAPI
   
I have worked extensively with both documented and undocumented WinAPI in the past, so I'm comfortable working in this.
   
I don't know much about the specifics on what is available. I have read a little about ReadDirectoryChangesW

     
Drawbacks: I hear that it's finicky to get configured correctly, but it is possible to use this route. There were several other issues, but I don't recall them off-hand.
   
I question whether this will:
        - A) Support all that I want to accomplish
        - B) Deliver the best speed, stability, and least tax on system resources/overall system stability
5. File System Driver
   
I don't know much about it, yet, but this sounds as though it may be the right route to go.

       
I may be missing something, and it's very possible that I might have some of what I've said above wrong. I have not done too much beyond initial research phase, so I will likely be updating this as I gain more insight. Any insight or thoughts you may have to offer will be most appreciated!

I know that there are several questions somewhat similar to this, but for this
specific application, I felt that nothing I found directly answered my question.

Finally...

A better synopsis of the application's purpose:

 
Monitor the filesystem for specific types of files being modified
(Optionally) monitor entire directories for any files being modified
Scan the file after modification to detect changes that may indicate a corruption of file type (invalid header, etc.)
If a large amount of files get modified that match our flagging, pause/freeze the application doing the writing and prompt the user for action.

Question/Request:

I am looking for thoughts and opinions on how to go about performing this large-scale monitor operation in the fastest, most efficient, enterprise-ready manner. This application will be on-par with commercial anti-virus and therefore, the best practices need to be applied, adhering to this standard!

Thanks very much, all, for any insight you might be able to offer on this!

Open in new window

0
Comment
Question by:Riktor
  • 2
4 Comments
 
LVL 86

Accepted Solution

by:
jkr earned 500 total points
ID: 40617032
'ReadDirectoryChangesW()' would have been the API that I'd have thought of first. There used to be a sample called 'fwatch' on MSDN (https://msdn.microsoft.com/en-us/library/aa230342%28v=vs.60%29.aspx) that illustrates it, but it seems to have been removed. But anyway, the article at http://www.codeproject.com/Articles/950/CDirectoryChangeWatcher-ReadDirectoryChangesW-all ("CDirectoryChangeWatcher - ReadDirectoryChangesW all wrapped up") uses the same approach and you can use the demo executable to see if it meets your requirements.
0
 
LVL 86

Expert Comment

by:jkr
ID: 40623765
You had quite some time to try that out now, so what are your results?
0

Featured Post

Free Tool: Path Explorer

An intuitive utility to help find the CSS path to UI elements on a webpage. These paths are used frequently in a variety of front-end development and QA automation tasks.

One of a set of tools we're offering as a way of saying thank you for being a part of the community.

Question has a verified solution.

If you are experiencing a similar issue, please ask a related question

Most PC repair technicians (if not all) always start their cleanup process by emptying the temp folders before running any removal tools. It makes sense because temp folders are common places for malware installers to lurk and removing all the junk …
Basic understanding on "OO- Object Orientation" is needed for designing a logical solution to solve a problem. Basic OOAD is a prerequisite for a coder to ensure that they follow the basic design of OO. This would help developers to understand the b…
The viewer will learn how to pass data into a function in C++. This is one step further in using functions. Instead of only printing text onto the console, the function will be able to perform calculations with argumentents given by the user.
The viewer will learn how to user default arguments when defining functions. This method of defining functions will be contrasted with the non-default-argument of defining functions.

808 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question