How do I save a file in the RAM memory?
what is the best way to save a file in the memory RAM and to read it only while it is in the RAM. What I mean is: I need to make an application that receives a certificate from another device but I cannot store it on the local device for security reasons. So I must receive it, store it in the RAM and use it(read mode only) while it is in the RAM. If the local device crashes the file must be lost to avoid security issues .
The language I'm using is C under Linux
I think a possible solution could be to use mmap() but I'm not sure since according to what I know, mmap makes a mapping in memory of a file that exists on disk but I've never used it so I'm not sure.
Thanks in advance
The language I'm using is C under Linux
I think a possible solution could be to use mmap() but I'm not sure since according to what I know, mmap makes a mapping in memory of a file that exists on disk but I've never used it so I'm not sure.
Thanks in advance
ASKER
Thanks for answering evilrix,
but I don't get the sense of your comment. Do you mean that mmap is a good solution or not? If yes how could I use it for my problem?
but I don't get the sense of your comment. Do you mean that mmap is a good solution or not? If yes how could I use it for my problem?
What I am saying is it doesn't matter either way because you have no direct control other what and when the OS might page to the swap file.
Evilrix is saying that no matter how you store the data (in memory, or a memory mapped drive), the operating system might write the data to the harddrive at any moment. So storing data in RAM is NOT going to ensure the certificate is completely destroyed.
So perhaps a better approach would be to encrypt the data and store that on the harddrive (or encrypted in memory). But again, whenever the data is unencrypted into RAM, at any moment, that information from RAM might be paged to the harddrive.
So perhaps a better approach would be to encrypt the data and store that on the harddrive (or encrypted in memory). But again, whenever the data is unencrypted into RAM, at any moment, that information from RAM might be paged to the harddrive.
ASKER
Oh ok sorry, now I understand what you mean. Do you know if there is a way to avoid that a page is swapped on disk?
In both cases can you tell me how I should use the mmap function or another one, in order to save a file in the memory RAM and treat it like a file stored on the disk??
Sorry I'm really new to the se of the mmap function.
Thanks again
In both cases can you tell me how I should use the mmap function or another one, in order to save a file in the memory RAM and treat it like a file stored on the disk??
Sorry I'm really new to the se of the mmap function.
Thanks again
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
Do not use mmap since you state that you do not want the certificate saved anywhere but RAM.
You should create one or more shared memory regions with a container to hold the certificates if you need to keep the certificate after validating it. After validation, derive a hash key from it, compress it, and, then as HooKooDooKu suggested, encrypt it (if you need to save it), and store in shared memory. After saving it into the shared memory, immediately zero out the plain certificate, writing multiple patterns over it if extra-security is a concern.
Obviously, the shared memory region needs write permission to save the compressed, encrypted certificate. Some cpu's have a facility to modify shared memory permissions. In one family of PPC's, we were surprised to find that some PPC's would not allow us to make a shared memory region strictly read-only. So, we had to have different compilations for our evaluation board vs. the production board (which did allow change to strict read-only).
>> So storing data in RAM is NOT going to ensure the certificate is completely destroyed.
Take a look at shmctl and see if this can assure you that no swap or disk writes will occur. (Sorry, I had this requirement on a non-Linux OS, so I can only point out the man pages.)
"SHM_LOCK (Linux specific)"
"Prevent swapping of the shared memory segment. The caller must fault in any pages that are required to be present after locking is enabled. If a segment has been locked, then the (non-standard) SHM_LOCKED flag of the shm_perm.mode field in the associated data structure retrieved by IPC_STAT will be set."
http://linux.die.net/man/2/shmctl
You should create one or more shared memory regions with a container to hold the certificates if you need to keep the certificate after validating it. After validation, derive a hash key from it, compress it, and, then as HooKooDooKu suggested, encrypt it (if you need to save it), and store in shared memory. After saving it into the shared memory, immediately zero out the plain certificate, writing multiple patterns over it if extra-security is a concern.
Obviously, the shared memory region needs write permission to save the compressed, encrypted certificate. Some cpu's have a facility to modify shared memory permissions. In one family of PPC's, we were surprised to find that some PPC's would not allow us to make a shared memory region strictly read-only. So, we had to have different compilations for our evaluation board vs. the production board (which did allow change to strict read-only).
>> So storing data in RAM is NOT going to ensure the certificate is completely destroyed.
Take a look at shmctl and see if this can assure you that no swap or disk writes will occur. (Sorry, I had this requirement on a non-Linux OS, so I can only point out the man pages.)
"SHM_LOCK (Linux specific)"
"Prevent swapping of the shared memory segment. The caller must fault in any pages that are required to be present after locking is enabled. If a segment has been locked, then the (non-standard) SHM_LOCKED flag of the shm_perm.mode field in the associated data structure retrieved by IPC_STAT will be set."
http://linux.die.net/man/2/shmctl
You could use mmap with the MAP_LOCKED flag.
I'm sorry to be pessimistic but coming from a security background (and being faced with just this kind of problem plenty of times) I know that you can never be too paranoid. You need to understand one thing, if an attacker has managed to gain access to a machine that your code is running on there is absolutely nothing you can do to prevent the certificate being exposed. There are things you can do to make it harder but do not be under any illusion, there is no guaranteed way to secure against access.
Even encrypting it isn't really going to be a solution (although it might help). The problem is your application will need to know the key to decrypt so anyone at any time could attach a debugger to your application, analyse its memory and either figure out the key or just look at the certificate decrypted in memory. As long as your application needs to access the original certificate at some point it will be either in memory or on disk and if someone has access to the machine they can gain access. Put simply, you absolutely cannot prevent someone who has physical access from viewing your certificate if they really want to.
You are better off trying to come up with a way to avoid needing direct access to the certificate. For example, deal with a hash digest of the certificate rather than the certificate itself. A digest is one way, there is no way to get back to the original certificate from the digest.
http://en.wikipedia.org/wiki/Cryptographic_hash_function
Of course, digests have their own problems (such as collisions) so this may or may not be a suitable way to go but the point is there are other solutions than just dealing with sensitive data in clear text.
Even encrypting it isn't really going to be a solution (although it might help). The problem is your application will need to know the key to decrypt so anyone at any time could attach a debugger to your application, analyse its memory and either figure out the key or just look at the certificate decrypted in memory. As long as your application needs to access the original certificate at some point it will be either in memory or on disk and if someone has access to the machine they can gain access. Put simply, you absolutely cannot prevent someone who has physical access from viewing your certificate if they really want to.
You are better off trying to come up with a way to avoid needing direct access to the certificate. For example, deal with a hash digest of the certificate rather than the certificate itself. A digest is one way, there is no way to get back to the original certificate from the digest.
http://en.wikipedia.org/wiki/Cryptographic_hash_function
Of course, digests have their own problems (such as collisions) so this may or may not be a suitable way to go but the point is there are other solutions than just dealing with sensitive data in clear text.
>> anyone at any time could attach a debugger to your application
Not really if precautions are taken. They have to open the box which can trigger an intrusion state which wipes out the key (and the entire key doesn't even have to be present). But, this has been done effectively, but handling it correctly is no trivial matter - you need true security experts to architect the HW and SW to be reasonably foolproof. Without the key, do not expect the certificate or anything else to be decrypted in the next few decades (if the encryption is strong).
To accomplish this, however, does require much resources and thorough testing (every LOC must be scrutinized). You could require a boot of the box to require multiple humans with their own personal physical electronic keys and PINs. But if your application is just to protect a commercial application, then possibly you do not have the resources to be as secure as you would like. It's just a matter of effort. If it were not, then all of our various countries secrets would be easy to crack.
Not really if precautions are taken. They have to open the box which can trigger an intrusion state which wipes out the key (and the entire key doesn't even have to be present). But, this has been done effectively, but handling it correctly is no trivial matter - you need true security experts to architect the HW and SW to be reasonably foolproof. Without the key, do not expect the certificate or anything else to be decrypted in the next few decades (if the encryption is strong).
To accomplish this, however, does require much resources and thorough testing (every LOC must be scrutinized). You could require a boot of the box to require multiple humans with their own personal physical electronic keys and PINs. But if your application is just to protect a commercial application, then possibly you do not have the resources to be as secure as you would like. It's just a matter of effort. If it were not, then all of our various countries secrets would be easy to crack.
If they have enough access to be able to view the contents of files (the original premise of this question) they have enough access to attach a debugger to a running process. That being the case there is no way to decrypt without exposing the key and/or the decrypted data in memory. Rule 1 about security - as soon as an attacker has logged in access to a machine all bets are off.
But the critical files should be encrypted to prevent any analysis. Also, once tamper event occurs, the flash should have been modified to prevent a clean reboot (in fact a purge should ensue as soon as power is turned on). If they take the drive out, they will find sensitive data encrypted. Agree that the system should be designed to prevent an attacker from being able to log in. If you think they can get through the encryption scheme, then all bets are off.
>> to be able to view the contents of files
But the certificate is not going to be in a file (per the OP requirement).
But the certificate is not going to be in a file (per the OP requirement).
Since the certificate is only in RAM, then the enclosed box will have stored enough power to wipe the RAM clear if there is either a tamper event or a power shutdown. That should protect the certificate.
phoffric you are missing the point I am making....
As long as you have access to a machine you can look at the memory space of a running process. If the certificate is in memory (encrypted or not) it is accessible. The process will need the key at some point to decrypt the certificate so the key will also be in memory and accessible. Either way I will have access to the key and the encrypted cert or when the cert is decrypted I'll have access to the plain text version all in memory. Whether it is persisted to disk or not makes absolutely no difference. And, in fact, even if you do hold everything in memory there is nothing to stop the OS paging to disk unless you take very specific steps to ensure it doesn't.
Even if you NEVER store the certificate on disk but the HD is stolen we can then assume that the attacker (given enough motivation - it is not hard to access Linux as root with physical access) will be able to access the data. All the attacker has to do is put the HD in another machine, boot up and run the process and the attach a debugger. Alternatively, if they have access to the binary they can just attached a disassembler and watch what the process does whilst it runs (both of these techniques are exactly how anti-virus researchers figure out how malware works and how to neuter them).
My point about being able to view files was just to point out that IF I have enough access to do that I almost certainly have enough access to view memory (with the exception where the files would be shared on a network but without me having access to the machine but such critical files are unlikely to be shared hence I can assume the attacker does have login access).
It would be easy to give the asker some simple hints and tricks to try and avoid the worse but as experts we should actually be ensuring the asker knows all the caveats; these are nothing more than making life harder for an attacker but they are not silver bullets. With access to a machine (either login or physical) there are enough attack vectors that will compromise such tactics that we need to be realistic and ensure the asker knows this.
The simple fact is one someone has login access to a machine it is compromised and so is everything on it. There is no guaranteed way to say this isn't the case.
As long as you have access to a machine you can look at the memory space of a running process. If the certificate is in memory (encrypted or not) it is accessible. The process will need the key at some point to decrypt the certificate so the key will also be in memory and accessible. Either way I will have access to the key and the encrypted cert or when the cert is decrypted I'll have access to the plain text version all in memory. Whether it is persisted to disk or not makes absolutely no difference. And, in fact, even if you do hold everything in memory there is nothing to stop the OS paging to disk unless you take very specific steps to ensure it doesn't.
Even if you NEVER store the certificate on disk but the HD is stolen we can then assume that the attacker (given enough motivation - it is not hard to access Linux as root with physical access) will be able to access the data. All the attacker has to do is put the HD in another machine, boot up and run the process and the attach a debugger. Alternatively, if they have access to the binary they can just attached a disassembler and watch what the process does whilst it runs (both of these techniques are exactly how anti-virus researchers figure out how malware works and how to neuter them).
My point about being able to view files was just to point out that IF I have enough access to do that I almost certainly have enough access to view memory (with the exception where the files would be shared on a network but without me having access to the machine but such critical files are unlikely to be shared hence I can assume the attacker does have login access).
It would be easy to give the asker some simple hints and tricks to try and avoid the worse but as experts we should actually be ensuring the asker knows all the caveats; these are nothing more than making life harder for an attacker but they are not silver bullets. With access to a machine (either login or physical) there are enough attack vectors that will compromise such tactics that we need to be realistic and ensure the asker knows this.
The simple fact is one someone has login access to a machine it is compromised and so is everything on it. There is no guaranteed way to say this isn't the case.
>> As long as you have access to a machine you can look at the memory space of a running process.
But my point was that in order to get access to the machine, either you break the login (which can be prevented if multiple users are required to login in with electronic keys and pins), or if the box is opened, in which case tamper intrusion is detected thereby wiping out the keys and the certificates.
The actual key, can be in a secondary tamper box within the exterior box adding another layer of security. And when tamper intrusion occurs, this interior box's key is wiped out as well. Memory can get wiped out extremely fast even during a power shutdown. Likewise, but less critical, is that the program itself is zero'd out. We shouldn't rely on the program/algorithm knowledge from enabling an attacker to compromise the data. The HD doesn't seem to be part of this OP (unless the author chooses to go with mmap - which doesn't appear to be useful).
The multiple login approach is akin to enabling nuclear missiles to be launched. We don't want one login to allow control of the missiles. So, hopefully, two logins help prevent compromise of the launch. Likewise, it would take multiple persons, of which only one need be trustworthy to prevent an unauthorized login.
But my point was that in order to get access to the machine, either you break the login (which can be prevented if multiple users are required to login in with electronic keys and pins), or if the box is opened, in which case tamper intrusion is detected thereby wiping out the keys and the certificates.
The actual key, can be in a secondary tamper box within the exterior box adding another layer of security. And when tamper intrusion occurs, this interior box's key is wiped out as well. Memory can get wiped out extremely fast even during a power shutdown. Likewise, but less critical, is that the program itself is zero'd out. We shouldn't rely on the program/algorithm knowledge from enabling an attacker to compromise the data. The HD doesn't seem to be part of this OP (unless the author chooses to go with mmap - which doesn't appear to be useful).
The multiple login approach is akin to enabling nuclear missiles to be launched. We don't want one login to allow control of the missiles. So, hopefully, two logins help prevent compromise of the launch. Likewise, it would take multiple persons, of which only one need be trustworthy to prevent an unauthorized login.
The very premise of this question is how to prevent the certificate being written to disk because there is access to data on disk, which means either there is also access to memory or access to the on disk or both. Either is enough to perform an attack vector.
>> OP: but I cannot store it on the local device for security reasons
>> The very premise of this question is how to prevent the certificate being written to disk
I tried to address the issue of preventing the certificate being written to disk in http:#35703440.
>> The very premise of this question is how to prevent the certificate being written to disk
I tried to address the issue of preventing the certificate being written to disk in http:#35703440.
Understood but even if you manage to achieve that you've only made life a little hard but not impossible for an attacker. As long as they have access (either remote login or physical access - either of which is implied in the question) you cannot stop them from figuring out the data.
That really is the only point I am making BUT it is an important one that the asker needs to understand. In other words, you need to model your thread and then decide whether you are prepared to accept the risk or come up with a completely different way of doing this. Unless you fully understand a threat you cannot prevent it and it is clear to me that the asker does not fully understand the threat because the proposed solution is a naive (no offence is meant by this -- that is exactly why EE is here, to help guide those not familiar) way to handle this potential threat.
If, after modelling the threat and the proposed solution you (the asker in this case) are acceptable to the risk then that's all grand (you have to weight risk vs. effort to mitigate) but my concern is the asker doesn't understand the risk well enough to make an informed decision. It's all well and good saying do this or do that but the asker also need to know what level of risk may or may not be attached.
Unlike other questions, security questions need to be handled with care and often we need to go beyond the scope of the question to make sure the asker is fully armed with all the info they need and not just provide a direct answer that may solve what they think is their immediately problem when, in fact, all it actually does is open another can of worms (and what's worse is they may not even realise it).
That really is the only point I am making BUT it is an important one that the asker needs to understand. In other words, you need to model your thread and then decide whether you are prepared to accept the risk or come up with a completely different way of doing this. Unless you fully understand a threat you cannot prevent it and it is clear to me that the asker does not fully understand the threat because the proposed solution is a naive (no offence is meant by this -- that is exactly why EE is here, to help guide those not familiar) way to handle this potential threat.
If, after modelling the threat and the proposed solution you (the asker in this case) are acceptable to the risk then that's all grand (you have to weight risk vs. effort to mitigate) but my concern is the asker doesn't understand the risk well enough to make an informed decision. It's all well and good saying do this or do that but the asker also need to know what level of risk may or may not be attached.
Unlike other questions, security questions need to be handled with care and often we need to go beyond the scope of the question to make sure the asker is fully armed with all the info they need and not just provide a direct answer that may solve what they think is their immediately problem when, in fact, all it actually does is open another can of worms (and what's worse is they may not even realise it).
>> but not impossible for an attacker
True, the only secure message is one embedded in a ton of granite and dropped in the deepest part of the Pacific Ocean with coordinatesd destroyed.
>> you've only made life a little hard
Hate to quibble with you; but the system I'm thinking about makes life extremely hard. They can capture the box and still not make any sense out of it. But no sense belaboring this point, especially if the OP is not backed with a very large security budget.
>> but my concern is the asker doesn't understand the risk well enough to make an informed decision.
We've certainly provided some thoughtful considerations. The author needs to clearly define the sophistication of the attacker before going to extradinary expenses. If need to protect against Google or a major government that is suspected of hacking military installations, then the expense should be no object.
True, the only secure message is one embedded in a ton of granite and dropped in the deepest part of the Pacific Ocean with coordinatesd destroyed.
>> you've only made life a little hard
Hate to quibble with you; but the system I'm thinking about makes life extremely hard. They can capture the box and still not make any sense out of it. But no sense belaboring this point, especially if the OP is not backed with a very large security budget.
>> but my concern is the asker doesn't understand the risk well enough to make an informed decision.
We've certainly provided some thoughtful considerations. The author needs to clearly define the sophistication of the attacker before going to extradinary expenses. If need to protect against Google or a major government that is suspected of hacking military installations, then the expense should be no object.
>> but the system I'm thinking about makes life extremely hard
Right, but we're talking realistically about what the asker has not some hypothetical situation. :)
Thanks Paul.
Right, but we're talking realistically about what the asker has not some hypothetical situation. :)
Thanks Paul.
>> what the asker has not some hypothetical situation
It wasn't hypothetical - it was built. But, frankly, from the OP, I thought that there may be a possibility that the author actually needed a highly secure system. Will wait for his reply. Although, I don't think I can contribute much more on our architecture.
It wasn't hypothetical - it was built. But, frankly, from the OP, I thought that there may be a possibility that the author actually needed a highly secure system. Will wait for his reply. Although, I don't think I can contribute much more on our architecture.
>> It wasn't hypothetical - it was built
I mean hypothetical from the askers point of view. I am aware that such hardware exists but it's of no relevance if the asker has no access to such hardware!
The asker wanted to try and work around a possible security risk by coding an in-memory only solution any my response to that is it really makes little difference because there are so many ways to also access the data in memory or by inferring it by disassembling the binary you may as well not bother. The real solution is to try and find a way that means you don't need to old an in-memory plain-text version of the certificate.
My suggestion would be to consider working with a digest of the certificate (in the same way that passwords are always handled only by their digests, wish some added salt to make brute force much harder). At least this would mean the actual certificate is never exposed. This may not be the best way and there are probably others but encryption is NOT a solution - it is merely a deterrent.
I mean hypothetical from the askers point of view. I am aware that such hardware exists but it's of no relevance if the asker has no access to such hardware!
The asker wanted to try and work around a possible security risk by coding an in-memory only solution any my response to that is it really makes little difference because there are so many ways to also access the data in memory or by inferring it by disassembling the binary you may as well not bother. The real solution is to try and find a way that means you don't need to old an in-memory plain-text version of the certificate.
My suggestion would be to consider working with a digest of the certificate (in the same way that passwords are always handled only by their digests, wish some added salt to make brute force much harder). At least this would mean the actual certificate is never exposed. This may not be the best way and there are probably others but encryption is NOT a solution - it is merely a deterrent.
SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
>> I think that discussions about physical system security and debuggers don't seem very relevant
Of course it is relevant! The asker is trying to secure something and unless they understand just how secure (or insecure) their proposed action is they have no idea whether they have succeeded or not! With access to a debugger (something found on nearly all installs of Linux) you have access to the whole memory of a process.
>> this discussion has moved far from what the OP asked
I agree BUT the thread is all relevant. If the op asked how to cut of their hands would you just tell them without explaining the caveats and implications of doing this? Security is something that is generally not very well understood (evident just by reading this thread) and to just post a suggestion without understanding the implications to the asker or not bothering to explain them is both fool hardy and naive. As experts we have a duty of care to the asker. Their security is far more important that our expert egos or trying to win a couple of thousand points.
The very nature of this question suggests access is available to the machine it's running on. That being the case nearly all the suggestions made here will not be secure. Unless you have worked in the security industry writing industrial strength security applications the risks may be hard to grasp but I assure you they are very real and at the very least the asker need to be aware of this. They can choose to ignore them if they wish but for experts to pretend they just don't exist is not acceptable.
Of course it is relevant! The asker is trying to secure something and unless they understand just how secure (or insecure) their proposed action is they have no idea whether they have succeeded or not! With access to a debugger (something found on nearly all installs of Linux) you have access to the whole memory of a process.
>> this discussion has moved far from what the OP asked
I agree BUT the thread is all relevant. If the op asked how to cut of their hands would you just tell them without explaining the caveats and implications of doing this? Security is something that is generally not very well understood (evident just by reading this thread) and to just post a suggestion without understanding the implications to the asker or not bothering to explain them is both fool hardy and naive. As experts we have a duty of care to the asker. Their security is far more important that our expert egos or trying to win a couple of thousand points.
The very nature of this question suggests access is available to the machine it's running on. That being the case nearly all the suggestions made here will not be secure. Unless you have worked in the security industry writing industrial strength security applications the risks may be hard to grasp but I assure you they are very real and at the very least the asker need to be aware of this. They can choose to ignore them if they wish but for experts to pretend they just don't exist is not acceptable.
@evilrix: If the OP thought your discussion was relevant, he would be engaged in the conversation.
The asker may not even realise the relevance of a comment until it is explained. That is why we, the experts, have a duty of care to ensure they do understand. You are free to use the Request Attention link if you disagree. I'm sure the moderators will be more than happy to explain this to you.
Well, I've been known to give the benefit of the doubt (sometimes to a fault), and evilrix has been known to doubt the benefit (as he said, sometimes paranoia is justified, especially with security).
I'll wait for the author to provide input and clarifications before making any further comments. As far as the author not responding just yet, perhaps he is just soaking it all in and waiting for the dust to settle. That's not a bad move on his part, IMO. Keeps the confusion down.
I'll wait for the author to provide input and clarifications before making any further comments. As far as the author not responding just yet, perhaps he is just soaking it all in and waiting for the dust to settle. That's not a bad move on his part, IMO. Keeps the confusion down.
ASKER
First of all thanks to all of you for the propositions. I tried to learn something from all of you. Since I did not know many things you have talked in your propositions, I have spent some time to study some of these things.
Just to clarify my question. I'm aware that there are many levels of security as suggested by evilrix and phoffric. I'm mostly interested to avoid that once that the device I'm writing my program for, is turned off, no trace of the certificate is recorded on the flash. To be more clear, my application will receive the certificate the first time it needs it, has to save it locally(but not on the disk), and will use the certificate everytime it needs. Basically this device will receive connections from clients and has to authenticate itself(server-side authentication) to them through the certificate. This means that each time a new client will connect the application has to use the certificate).
II do not need to go so far as to prevent that a certificate is taken while it is in memory(for example through the attachment od a debugger or analysis of the memory). I just want that once my device turns off( for any reason) no trace of the certificate is in memory(if power turns off the RAM should loose all its content and on the flash there it should not be saved).
According to what my explication I'd like to ask some clarifications about some of the propositions proposed in order to decide which one is the most appropriate:
@eager: you said
>>"Do you really need to save the certificate in a file? Perhaps you could design your program so
>> that it obtains the certificate when it is needed, without a copy being made"
yes I thought to this solution but when you say "obtain" you must save it temporaly somewhere the certificate in order to use it right?? Probably the solution of the ramfs could be very useful but I've never used it. could you give me some more explications and how to create this kind of filesystem and its advantages. I will investigate on my side also but usage suggestions are very welcome
@SuperDave:
>> you suggested the same solution of eager to use the ramsfs. If you could give more details too it would be perfect.
@phoffric:
you suggested a solution through creating "one or more shared memory regions". I don't know too much aboud how to use shared memory regions. How can I do that?
@HooKooDooKu: you suggested to store it encrypted on the disk. How can I encrypt it and decrypt it? To do this I think I need a key that must be stored somewhere. How can I do that? I don't know too much how to deal with this solution. can you explain a little more?
Thanks to all for your contribution
Just to clarify my question. I'm aware that there are many levels of security as suggested by evilrix and phoffric. I'm mostly interested to avoid that once that the device I'm writing my program for, is turned off, no trace of the certificate is recorded on the flash. To be more clear, my application will receive the certificate the first time it needs it, has to save it locally(but not on the disk), and will use the certificate everytime it needs. Basically this device will receive connections from clients and has to authenticate itself(server-side authentication) to them through the certificate. This means that each time a new client will connect the application has to use the certificate).
II do not need to go so far as to prevent that a certificate is taken while it is in memory(for example through the attachment od a debugger or analysis of the memory). I just want that once my device turns off( for any reason) no trace of the certificate is in memory(if power turns off the RAM should loose all its content and on the flash there it should not be saved).
According to what my explication I'd like to ask some clarifications about some of the propositions proposed in order to decide which one is the most appropriate:
@eager: you said
>>"Do you really need to save the certificate in a file? Perhaps you could design your program so
>> that it obtains the certificate when it is needed, without a copy being made"
yes I thought to this solution but when you say "obtain" you must save it temporaly somewhere the certificate in order to use it right?? Probably the solution of the ramfs could be very useful but I've never used it. could you give me some more explications and how to create this kind of filesystem and its advantages. I will investigate on my side also but usage suggestions are very welcome
@SuperDave:
>> you suggested the same solution of eager to use the ramsfs. If you could give more details too it would be perfect.
@phoffric:
you suggested a solution through creating "one or more shared memory regions". I don't know too much aboud how to use shared memory regions. How can I do that?
@HooKooDooKu: you suggested to store it encrypted on the disk. How can I encrypt it and decrypt it? To do this I think I need a key that must be stored somewhere. How can I do that? I don't know too much how to deal with this solution. can you explain a little more?
Thanks to all for your contribution
Ramfs is created in system memory and can grow if needed. It
You can create a ramfs file system in rc.local as follows:
Create the mount point::
# mkdir -p /mnt/ram
Create a RAM file system and mount it:
# mount -t ramfs ramfs /mnt/ram
Or you can put this in /etc/fstab:
ramfs /mnt/ram ramfs noauto,rw 0 0
This RAM file system will be created when you boot your system. You can create files in on the file system (e.g. echo "certificate" >> /mnt/ram/cert) which can be read by other programs. The contents of the file system are lost on reboot.
Some more description: https://wiki.archlinux.org/index.php/Ramdisk
You can encrypt a file using crypt. Both the writer and reader need to have the key.
You can create a ramfs file system in rc.local as follows:
Create the mount point::
# mkdir -p /mnt/ram
Create a RAM file system and mount it:
# mount -t ramfs ramfs /mnt/ram
Or you can put this in /etc/fstab:
ramfs /mnt/ram ramfs noauto,rw 0 0
This RAM file system will be created when you boot your system. You can create files in on the file system (e.g. echo "certificate" >> /mnt/ram/cert) which can be read by other programs. The contents of the file system are lost on reboot.
Some more description: https://wiki.archlinux.org/index.php/Ramdisk
You can encrypt a file using crypt. Both the writer and reader need to have the key.
>> once that the device I'm writing my program for, is turned off, no trace of the certificate is recorded on the flash.
In the OP you indicated that you wanted to save the certificate in RAM. Why are you recording the certificate in the flash rather than RAM?
Is this going to be saved in flash that is built-in your system, or a removable flash drive?
If built-in, is the flash region memory mapped, or do you have an I/O driver to access the flash?
In the OP you indicated that you wanted to save the certificate in RAM. Why are you recording the certificate in the flash rather than RAM?
Is this going to be saved in flash that is built-in your system, or a removable flash drive?
If built-in, is the flash region memory mapped, or do you have an I/O driver to access the flash?
ASKER
@phoffric: sorry for the omission. the HD of the device I am writing the application is a built-in flash. this is the reason I've mentioned the flash. oh no I definitely want to save the certificate in RAM not in the flash
@eager: thanks for the explanation on the ramfs. Just another question about the the key for encrypting and decrypting the file. How do I handle the key? I mean how do I store the key in order to use it?? It's merely a question from the program architectural poin of view. Thanks
@eager: thanks for the explanation on the ramfs. Just another question about the the key for encrypting and decrypting the file. How do I handle the key? I mean how do I store the key in order to use it?? It's merely a question from the program architectural poin of view. Thanks
>> oh no I definitely want to save the certificate in RAM not in the flash
Good. Makes life easier.
I would have to research your linux version and CPU specs to see if you could make the shared memory that I proposed read-only after the certificate is loaded. Is it that you are concerned that a program may accidentally write to the certificate thereby invalidating it (or worse, intentionally changing it via attackers)?
But here is an alternative solution that is viable.
Write a simple program, CertifiateArchive, that accepts (once, if that is your requirement) a single certificate and stores it in its own address space. CertifiateArchive can than respond to a requesting program via a pipe or tcp socket. This requesting program uses it and immediately erases it.
On system shutdown, CertifiateArchive, can erase its certificate, although from your explanations, it appears that you are not concerned about left-over residual memory effect that can be recoved using electron microscopy.
Good. Makes life easier.
I would have to research your linux version and CPU specs to see if you could make the shared memory that I proposed read-only after the certificate is loaded. Is it that you are concerned that a program may accidentally write to the certificate thereby invalidating it (or worse, intentionally changing it via attackers)?
But here is an alternative solution that is viable.
Write a simple program, CertifiateArchive, that accepts (once, if that is your requirement) a single certificate and stores it in its own address space. CertifiateArchive can than respond to a requesting program via a pipe or tcp socket. This requesting program uses it and immediately erases it.
On system shutdown, CertifiateArchive, can erase its certificate, although from your explanations, it appears that you are not concerned about left-over residual memory effect that can be recoved using electron microscopy.
@phoffric -- Thanks for repeating my second suggestion about using a server running in the background. I'm not sure about your comment that in a system shutdown, a certificate can be erased. On system crash, nothing is erased. I don't think most folks worry about their secrets being revealed using electron microscopy, at least, other than folks in Langley, VA, or at the semiconductor companies.
@fmeo -- shared key or shared secret encryption (like used by crypt) means that both the reader and writer need to have the same key. This key is either coded into both programs, computed identically by both programs, or provided to each program in a secure fashion. There are some risks with shared secret keys but all media devices implement DRM using a shared key. See http://en.wikipedia.org/wiki/Symmetric-key_algorithm
@fmeo -- shared key or shared secret encryption (like used by crypt) means that both the reader and writer need to have the same key. This key is either coded into both programs, computed identically by both programs, or provided to each program in a secure fashion. There are some risks with shared secret keys but all media devices implement DRM using a shared key. See http://en.wikipedia.org/wiki/Symmetric-key_algorithm
@eager,
Actually, I missed your comment about a server running in the background - sorry about that. (This thread became rather long.) I had stopped when I saw Ramfs which I am not familiar with.
Actually, I missed your comment about a server running in the background - sorry about that. (This thread became rather long.) I had stopped when I saw Ramfs which I am not familiar with.
@eager,
>> I'm not sure about your comment that in a system shutdown, a certificate can be erased. On system crash, nothing is erased.
I was talking about a system shutdown where the power is turned off. Most RAM loses its memory within a few seconds after power is turned off.
>> I don't think most folks worry about their secrets being revealed using electron microscopy
I think there may be a lot of security minded people out there. Even the free CCleaner (http://www.piriform.com/ccleaner) has a Drive Wiper utility that allows a simple overwrite (1 pass) all the way up to something called Gutmann (35 passes) (http://en.wikipedia.org/wiki/Gutmann_method).
Although overwrites are most commonly used for disk drives, they are also used for RAM which does hold a persistent shade of what was written after the power has been turned off.
>> I'm not sure about your comment that in a system shutdown, a certificate can be erased. On system crash, nothing is erased.
I was talking about a system shutdown where the power is turned off. Most RAM loses its memory within a few seconds after power is turned off.
>> I don't think most folks worry about their secrets being revealed using electron microscopy
I think there may be a lot of security minded people out there. Even the free CCleaner (http://www.piriform.com/ccleaner) has a Drive Wiper utility that allows a simple overwrite (1 pass) all the way up to something called Gutmann (35 passes) (http://en.wikipedia.org/wiki/Gutmann_method).
Although overwrites are most commonly used for disk drives, they are also used for RAM which does hold a persistent shade of what was written after the power has been turned off.
Thinking about this some more, you will have to use mmap in some form. Even if you use ramfs files, you can't read and write them because the contents will be stored in memory buffers.
The advantage of ramfs files is that you can preserve the contents between processes. To create the file you'll have to mmap the file and receive the certificate into it's mmapped memory. Likewise you'll have to read from mmapped memory.
It should be possible, if the certificate is only used by one process, to use mmap with MAP_ANONYMOUS|MAP_LOCKED to keep it in unswapped memory.
Depending on how your kernel is set up you might need to set up a filesystem as ramfs, or you might just be able to use a directory that doesn't have anything else mounted on it. Initially, the root directory and whatever is created under it is ramfs, unless and until you use it as a mount point for something else. Sometimes the boot process does this with switch_root or pivot_root. Especially if you have some kind of specialized hardware it might not be done that way and you can create a directory under root that is part of the "rootfs" ramfs.
The advantage of ramfs files is that you can preserve the contents between processes. To create the file you'll have to mmap the file and receive the certificate into it's mmapped memory. Likewise you'll have to read from mmapped memory.
It should be possible, if the certificate is only used by one process, to use mmap with MAP_ANONYMOUS|MAP_LOCKED to keep it in unswapped memory.
Depending on how your kernel is set up you might need to set up a filesystem as ramfs, or you might just be able to use a directory that doesn't have anything else mounted on it. Initially, the root directory and whatever is created under it is ramfs, unless and until you use it as a mount point for something else. Sometimes the boot process does this with switch_root or pivot_root. Especially if you have some kind of specialized hardware it might not be done that way and you can create a directory under root that is part of the "rootfs" ramfs.
I just noticed something else (mentioned in mmap's man page about MAP_LOCKED). There is a mlock() call to keep your memory from being swapped. So using mlock you could lock all your process's memory and not have to worry about read/write buffers being in swappable memory, and use normal file operations. Or you could use that instead of mmap; it would accomplish a similar thing.
@Superdave -- Using ramfs does not require using mmap. Mmap does not create a file; that is done by open(). To create a file in ramfs, you simply open it and write to it. You can read and write files in ramfs (try it!) just like any other file, without using mmap. (My guess is that your understanding of mmap is based on reading the man page, not actually using it.)
If the OP wanted to have the certificate accessed by only single process, there would be no need to write it to a file. All the other stuff about switch_root or locking mapped memory is really irrelevant, since (a) the OP isn't booting a kernel, (b) mlock has nothing to do with files,
(c) mmap cannot do what mlock does and vice versa, and (d) internal memory buffers are free'd when the process exits.
If the OP wanted to have the certificate accessed by only single process, there would be no need to write it to a file. All the other stuff about switch_root or locking mapped memory is really irrelevant, since (a) the OP isn't booting a kernel, (b) mlock has nothing to do with files,
(c) mmap cannot do what mlock does and vice versa, and (d) internal memory buffers are free'd when the process exits.
Ok, still everyone if trying to figure out how to hold a copy of the certificate safely and still no one is realising that you just cannot do this. The solution isn't to try and make your original proposal work, the solution is to do this properly and securely.
You are just comparing certificates right? So, again, I ask why do you need to store the actual certificate? You don't need to do this; either in memory or on disk. All you need is a two step authentication process that does not require the service to know the certificate at all. I'm not 100% clear whether the client has to authenticate with the server or vice versa but the technique I am about to explain will work either way around. For the sake of explaining it I'll assume the client authenticates with the server, just swap the word client and server if it is the other way around.
First off, the service needs to have a digest of the original certificate using a strong cryptographic hash function. It will also need the public key of the client, which will have a private key. That's all the service needs, it does not need the certificate.
You then use asymmetric public/private key authentication. The client has a private key, which it uses to encrypt a strong cryptographic digest of the certificate and then it sends that to your service. The service then decrypts the digest of the certificate using the clients public key.
This is step one of your authentication. If the digest hasn't come from the client (with a secured private key) the decrypted digest will not match the severs version of it. Secondly you compare the servers digest with the clients and these should match. If they don't it's either because the client did not use the correct private key or the client did not have access to the correct certificate.
That's it. At no time is the clients private key expose nor is the certificate. Even if someone is able to figure out the digest it won't help them because they would also need the clients private key, which is secure.
Ok, you could argue that a digest can collide and that is true but the chances of a collision and the correct private/public key pair being used are about as unlikely as someone using brute force to compromise the server. Also, you can reduce the chances of a collision and a brute force attack by salting the original hash.
That's it. That is all you need to do. The certificate is secure, it cannot be exposed and there is not enough on the server side to be able to figure out how to imitate a client using anything other than brute force.
In case you are wondering, this is pretty much how digital signatures work; thus is a proven authentication technique.
You are just comparing certificates right? So, again, I ask why do you need to store the actual certificate? You don't need to do this; either in memory or on disk. All you need is a two step authentication process that does not require the service to know the certificate at all. I'm not 100% clear whether the client has to authenticate with the server or vice versa but the technique I am about to explain will work either way around. For the sake of explaining it I'll assume the client authenticates with the server, just swap the word client and server if it is the other way around.
First off, the service needs to have a digest of the original certificate using a strong cryptographic hash function. It will also need the public key of the client, which will have a private key. That's all the service needs, it does not need the certificate.
You then use asymmetric public/private key authentication. The client has a private key, which it uses to encrypt a strong cryptographic digest of the certificate and then it sends that to your service. The service then decrypts the digest of the certificate using the clients public key.
This is step one of your authentication. If the digest hasn't come from the client (with a secured private key) the decrypted digest will not match the severs version of it. Secondly you compare the servers digest with the clients and these should match. If they don't it's either because the client did not use the correct private key or the client did not have access to the correct certificate.
That's it. At no time is the clients private key expose nor is the certificate. Even if someone is able to figure out the digest it won't help them because they would also need the clients private key, which is secure.
Ok, you could argue that a digest can collide and that is true but the chances of a collision and the correct private/public key pair being used are about as unlikely as someone using brute force to compromise the server. Also, you can reduce the chances of a collision and a brute force attack by salting the original hash.
That's it. That is all you need to do. The certificate is secure, it cannot be exposed and there is not enough on the server side to be able to figure out how to imitate a client using anything other than brute force.
In case you are wondering, this is pretty much how digital signatures work; thus is a proven authentication technique.
@eager - The reason I said you can't read or write files is because the read or write system calls copy data from the device or ramfs to or from swappable RAM, which is what we are trying to avoid. However the mlock system call could allow you to do this safely.
My point about pivot_root was only that if the system didn't do that you would already have a ramfs available without needing to mount one. Otherwise you'd need to mount an extra filesystem to make it work.
My point about pivot_root was only that if the system didn't do that you would already have a ramfs available without needing to mount one. Otherwise you'd need to mount an extra filesystem to make it work.
The OP was not trying to avoid putting data in RAM. Please take a look at the Subject line.
You *can* read and write files from ramfs. Possibly the data will be copied from ram to ram, although I've not looked at the kernel code to verify this.
Mlock simply does not do what you think it does. Why don't you try it and see?
User programs do not run during boot, so pivot_root and the other boot time stuff is irrelevant.
Finally, creating and mounting a ramfs file system is trivial. See above.
You *can* read and write files from ramfs. Possibly the data will be copied from ram to ram, although I've not looked at the kernel code to verify this.
Mlock simply does not do what you think it does. Why don't you try it and see?
User programs do not run during boot, so pivot_root and the other boot time stuff is irrelevant.
Finally, creating and mounting a ramfs file system is trivial. See above.
Yes, he needs to avoid putting data in RAM if the RAM is swap-backed. I'm not sure what is a good way of "trying" things; it's hard to tell from a program's behavior what it is swapping, especially because swapping is unpredictable.
You're right that creating a ramfs is not complicated, although why do it if you don't need to.
You're right that creating a ramfs is not complicated, although why do it if you don't need to.
Superdave -- my suggestion was to try mlock and see what it does. Write a program which uses it in any way that you think is correct and see if it does what you think.
You are concerned about data possibly being written to swap. I'm not sure why. Have you ever tried to read swap space on a hard drive? Perhaps this is one of those obscure hypothetical risks, like the previous comments about electron microscopes reading residual ram charges.
As for a solution to the OP problem, what is it you suggest? Not use ramfs. Use mlock to do something that it doesn't do.
You are concerned about data possibly being written to swap. I'm not sure why. Have you ever tried to read swap space on a hard drive? Perhaps this is one of those obscure hypothetical risks, like the previous comments about electron microscopes reading residual ram charges.
As for a solution to the OP problem, what is it you suggest? Not use ramfs. Use mlock to do something that it doesn't do.
eager - Okay, I didn't actually try mlock, so I did now to see what all the excitement is about. It didn't do anything detectable, which is what I expected. So what are you saying about it?
I thought not being written to swap was one of the original concerns. That could be a bad assumption; maybe this device isn't using swap, rather than using flash for swap. If it's not swapping, then the concern is for nothing. If it is, it's up to fmeo how concerned to be about security. But it might not be that difficult to take extra precautions.
So use ramfs, with mmap if necessary, after first considering whether you can avoid the problem with cryptography as evilrix said.
I thought not being written to swap was one of the original concerns. That could be a bad assumption; maybe this device isn't using swap, rather than using flash for swap. If it's not swapping, then the concern is for nothing. If it is, it's up to fmeo how concerned to be about security. But it might not be that difficult to take extra precautions.
So use ramfs, with mmap if necessary, after first considering whether you can avoid the problem with cryptography as evilrix said.
Superdave -- mlock does exactly what it says: force pages to stay in memory. Mlock is used to improve performance in time critical applications where there is memory contention. If the application text pages are never deleted or the data pages are never swapped, you will not see any changes. There are times when mlock is the right tool -- but this is not one. As I said, mlock doesn't do anything with files or mmap.
Mmap has it's purpose: map a file into *virtual* memory (not ram), usually for execution (this is what the kernel does) or so that the file can be accessed randomly without using fseek. Mmap is not needed to address the OP's problem and likely would only add complexity without an additional benefit. I suggest that you try writing a program which mmaps a disk file and see how it works.
Mmap has it's purpose: map a file into *virtual* memory (not ram), usually for execution (this is what the kernel does) or so that the file can be accessed randomly without using fseek. Mmap is not needed to address the OP's problem and likely would only add complexity without an additional benefit. I suggest that you try writing a program which mmaps a disk file and see how it works.
ASKER
@elvirix: basically it is ecaxt what you say about the authentification process. To clarify, I need that the server authentificates itself to the client.
My question is: you say that the client just needs an hash of the certificate and the public key of the server in my case. How the client gets the public key of the server without the certificate. The public key is contained in the certificate so it needs the server's certificate. Am I wrong?
My question is: you say that the client just needs an hash of the certificate and the public key of the server in my case. How the client gets the public key of the server without the certificate. The public key is contained in the certificate so it needs the server's certificate. Am I wrong?
The key and the certificate are not related. The key is used to encrypt a hash digest of the certificate. Think of the key as a way to open a vault and inside the vault is the digest that represents your hash. Exchanging keys is always going to be the weakest part of any encryption process and this is true even with asymmetric keys.
The problem comes down to this: can you trust the key you are being given? More specifically, how do you know it has not come from someone impersonating the person you are about to trust? There are a number of solutions to this problem (for example, web browsers use certificate authorities). I'm not sure any of these are suitable for your needs so I'll explain another possible solution based on EKE or "Encrypted Key Exchange" (something I have done in the past that was more than good enough for my needs).
If you are coding both the client and the server you can give the server a private key and all the clients have the servers public key (either hard coded or [better] as part of a config that can be changed in case the keys are compromised). The clients can then encrypt their own public key using the servers public key so that only the server can decrypt it (using its own private key).
Encrypt the hash digest with the clients private key. Next encrypt the encrypted digest and the clients public key with the servers public key and send to the server. The server then decrypts the clients public key and the hash digest using its own private key. It then uses the clients public key to decrypt the hash digest again (remember it was double encrypted). You then check the hash digest to ensure it matches the servers version.
It should be noted (as with nearly every security measure there is) that this is NOT a panacea. If someone really wanted to they could compromise things but they would need access to 2 pairs of asymmetric keys, which are not stored on the same system. They would need to gain access to both the server and a client to compromise things (and if they manage to do that you probably have more serious things to worry about).
Asymmetric encryption only works if the private key is kept private so this is not a flaw in the process but a basic problem with all encryption. If you lose your house keys then your house is compromised; that's not a problem with the key, the lock or the house but a problem with key management. In all authentication systems if the keys are compromised the all bets are off!
I hope this suggestion makes sense and is helpful.
The problem comes down to this: can you trust the key you are being given? More specifically, how do you know it has not come from someone impersonating the person you are about to trust? There are a number of solutions to this problem (for example, web browsers use certificate authorities). I'm not sure any of these are suitable for your needs so I'll explain another possible solution based on EKE or "Encrypted Key Exchange" (something I have done in the past that was more than good enough for my needs).
If you are coding both the client and the server you can give the server a private key and all the clients have the servers public key (either hard coded or [better] as part of a config that can be changed in case the keys are compromised). The clients can then encrypt their own public key using the servers public key so that only the server can decrypt it (using its own private key).
Encrypt the hash digest with the clients private key. Next encrypt the encrypted digest and the clients public key with the servers public key and send to the server. The server then decrypts the clients public key and the hash digest using its own private key. It then uses the clients public key to decrypt the hash digest again (remember it was double encrypted). You then check the hash digest to ensure it matches the servers version.
It should be noted (as with nearly every security measure there is) that this is NOT a panacea. If someone really wanted to they could compromise things but they would need access to 2 pairs of asymmetric keys, which are not stored on the same system. They would need to gain access to both the server and a client to compromise things (and if they manage to do that you probably have more serious things to worry about).
Asymmetric encryption only works if the private key is kept private so this is not a flaw in the process but a basic problem with all encryption. If you lose your house keys then your house is compromised; that's not a problem with the key, the lock or the house but a problem with key management. In all authentication systems if the keys are compromised the all bets are off!
I hope this suggestion makes sense and is helpful.
ASKER
@elvirix: thanks a lot for your explanation. I had studied more or less this method at school so thanks for the refresh. Only another question to have the big picture clear.
Yous wrote that the clients have the public key of the server for example hardocoded. Am I wrong or the public key of an entity(for example a website) is stored in its certificate.
What I mean is(according to what I know) if a bank wants to be authenticated when clients connect to its website, it asks a certificate to a Certificate Authority that creates the certificate, creates a couple of keys( 1 private that only the bank knows and a public key that is stored in the certificate and is public and well known to all the clients that connect to the website of the bank and get the certificate of the bank) and then the CA signs the certificate to certify it is authentic.
In this way everyone can get the public key of the bank simpy getting the certificate of the bank(basically thi is what happens when you connect to a bank, you connect and you get its certificate). Do you confirm this or I'm doing wrong somewhere?
Yous wrote that the clients have the public key of the server for example hardocoded. Am I wrong or the public key of an entity(for example a website) is stored in its certificate.
What I mean is(according to what I know) if a bank wants to be authenticated when clients connect to its website, it asks a certificate to a Certificate Authority that creates the certificate, creates a couple of keys( 1 private that only the bank knows and a public key that is stored in the certificate and is public and well known to all the clients that connect to the website of the bank and get the certificate of the bank) and then the CA signs the certificate to certify it is authentic.
In this way everyone can get the public key of the bank simpy getting the certificate of the bank(basically thi is what happens when you connect to a bank, you connect and you get its certificate). Do you confirm this or I'm doing wrong somewhere?
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Thanks all of you guys
At any time the OS could decide to page memory to disk (this is why there is a swap file) so you can't really prevent it from potentially being persisted. Also, at any time a debugger could be attached to the process and memory examined.