Link to home
Start Free TrialLog in
Avatar of infofinder
infofinderFlag for United Kingdom of Great Britain and Northern Ireland

asked on

Programmatically add app to DEP exclusion list?

Does anybody know of a way to programmatically add an app to the Data Execution Prevention exclusion list under XP SP2, Vista and so on?
Avatar of orangutang
orangutang

And how do you even get to the DEP exclusion list at all, non-programmatically?
Avatar of infofinder

ASKER

Cpl -> System ->Advanced -> Performance Settings -> Data Execution Prevention (XP SP2, running hardware with support for hardware DEP, e.g. most 64-bit processors)

The software protection wrapper I'm using is currently incompatible with full hardware DEP. Since an app can be excluded manually as above, I'm hoping there's a way to do it programmatically (e.g. at install time) as a stop-gap until the wrapper is fixed.
ASKER CERTIFIED SOLUTION
Avatar of orangutang
orangutang

Link to home
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Start Free Trial
Well that's certainly part of the solution! Adding a string value there in the form:

<full path to app> = DisableNXShowUI

does indeed add the app to the DEP exclusion list in the dialog, however it doesn't actually disable DEP for the app. It seems to be necessary for the user to actually set the checkbox directly in the dialog for it to take effect. So I guess there's another part to this, maybe another registry entry or a call to some API to make Windows acknowledge the new entry.

Any ideas?
Hmm, I don't know. I guess it goes a little deeper than simply add the value to the registry. It may be more than registry related as well. Maybe it modifies or adds some encrypted file to your HD. Anyway, I found this in a Google-translated French website:

- My program does not launch out:
That can be due to the functionality of prevention of data (EPD). Let us examine how this option functions:
1) With the right button of the mouse click on the icon Working station then on Propriétés.
2) Click on the Avancé mitre then the Paramètres button placed in the Performances heading.
3) Click on the Prévention mitre of the execution of the data.
4) Notch the button radio operator Activer the prevention of the execution of the data for all the programs and the services except those which I select:
That causes to modify a named binary value LastNoExecuteRadioButtonState and which is placed in HKEY_LOCAL_MACHINE \ SOFTWARE \ Microsoft \ Windows NT \ CurrentVersion \ AppCompatFlags \ NoExecuteState.
In this case the data of the value will be equal to 32d5 (in the contrary case: 32d4).
5) Click on the Ajouter button… then select the programs which will not be concerned with the functionality of prevention of the data.
With each program added the Windows Register will be modified in this way:
* Opening of this tree structure: HKEY_LOCAL_MACHINE \ SOFTWARE \ Microsoft \ Windows NT \ CurrentVersion \ AppCompatFlags \ Layers.
* Creation of a value chains named name and site of the achievable file. For example: C:\Program Files\Chemin\Nom _Programme.exe
* Modification of the data of the value fixed at this: DisableNXShowUI
Here now a procedure of resolution concerning a program named Cyberlink PowerDirector.
The program refuses to launch out without there being any error message.
1) Activate in this case the button radio operator n°2 then create an exception by selecting the achievable file named PowerDirector.exe.
2) Make then the opposite step by removing the programs listed in exception.
3) Reactivate then the button radio operator n°1.
4) Check that the Register was correctly updated because even if the radio operator button by defect is selected it can remain a trace of the program in HKEY_LOCAL_MACHINE \ SOFTWARE \ Microsoft \ Windows NT \ CurrentVersion \ AppCompatFlags \ Layers.
Cyberlink PowerDirector will launch out then normally.
Easy way communicated by “FXB”.
Thanks for the extra info. I'm coming to the conclusion that this just isn't going to be possible, and I guess that's not too surprising. It could be a security hole for Windows if it was allowed, given that DEP is meant to be a defense against exploits of weakly-coded software.

I'm going to accept your post as a solution, given that it's about as close to a solution as I think it's possible to get.
Yay, thank you!