How to encrypt data (flat text files) that as they are written and then stored in various folders on the server?
Question: How to encrypt data (flat text files) that as they are written and then stored in various folders on the server?
I have online web forms that users fill out. The data from some of these web forms are stored on the same server, in a plain text file. The information collected is sometimes of a personal and sensitive nature.
So, I am trying to figure out if and how I can accomplish the following.
I would like to have the text files that are dynamically created after a user fills out one of the online forms... to automatically be encrypted or password protected.
I want the information (as it is collected and stored in a folder on the server) to be encrypted (protected so no one could access the information .. immediately as it is collected and saved.
I had looked at a couple software options (from PGP and TrueCrypt) that basically created virtual drives, where the data could be saved to. But in the end, if the virtual drives were active (i.e., mounted, etc.) the data could be written to the virutal drive, but since the drive was visable .. it was also NOT protecting the data. In both cases, only if the drive was dismounted would the data be encrypted and protected. (But if the drive is dismounted, it is no longer visable and therefore could not be written to by my online forms.
I need to know how to set up "something" that would allow the files (as they are created .. as users fill out the online forms) .. to automatically be protected in a sufficient manner so that no one (another user on the network, a hacker, etc.) would be able to see the contents of the files.
I am hoping someone can help me with a suggestion that would allow me to somehow .. protect certain folders from being READ or accessed by anyone .. yet allow my scripts, etc. to write and add data into the protected folders. So, in a nut shell, data could be saved into the protected folder, but nothing could be taken out or read from the folder .. without knowing a secure password or something.
Any thoughts to get this done?
Thanks,
Gary
I have online web forms that users fill out. The data from some of these web forms are stored on the same server, in a plain text file. The information collected is sometimes of a personal and sensitive nature.
So, I am trying to figure out if and how I can accomplish the following.
I would like to have the text files that are dynamically created after a user fills out one of the online forms... to automatically be encrypted or password protected.
I want the information (as it is collected and stored in a folder on the server) to be encrypted (protected so no one could access the information .. immediately as it is collected and saved.
I had looked at a couple software options (from PGP and TrueCrypt) that basically created virtual drives, where the data could be saved to. But in the end, if the virtual drives were active (i.e., mounted, etc.) the data could be written to the virutal drive, but since the drive was visable .. it was also NOT protecting the data. In both cases, only if the drive was dismounted would the data be encrypted and protected. (But if the drive is dismounted, it is no longer visable and therefore could not be written to by my online forms.
I need to know how to set up "something" that would allow the files (as they are created .. as users fill out the online forms) .. to automatically be protected in a sufficient manner so that no one (another user on the network, a hacker, etc.) would be able to see the contents of the files.
I am hoping someone can help me with a suggestion that would allow me to somehow .. protect certain folders from being READ or accessed by anyone .. yet allow my scripts, etc. to write and add data into the protected folders. So, in a nut shell, data could be saved into the protected folder, but nothing could be taken out or read from the folder .. without knowing a secure password or something.
Any thoughts to get this done?
Thanks,
Gary
you can try to do something with gpg (http://www.gnupg.org/) but you will need to modify you application. Or may be you can apply an encryption function like 3des or aes to the sensitive fields.
ASKER
Gssc1414,
1. I am using SSL on the web pages where users fill out the online forms.
2. I read some information from:
https://www.experts-exchange.com/questions/22067774/how-to-set-up-efs-in-windows-2003-domain-environment.html
and
http://www.microsoft.com/windowsxp/using/security/expert/sharefilesefs.mspx
and
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/distrib/dsck_efs_zndl.mspx?mfr=true
but I'm still a little confused as to how to properly set up the passwords or user access.
I notice, very clearly that you say "BE CAREFUL" so I just want to make sure I do what ever I am supposed to do .. in a step by step format so I don't make a mistake.
Would you be so kind as to explain (in a step by step format) what I need to do, etc. to get this set up?
Thanks,
Gary
PS: If your 'step by step' info is good, I will gladly double my points for your effort. Thanks.
1. I am using SSL on the web pages where users fill out the online forms.
2. I read some information from:
https://www.experts-exchange.com/questions/22067774/how-to-set-up-efs-in-windows-2003-domain-environment.html
and
http://www.microsoft.com/windowsxp/using/security/expert/sharefilesefs.mspx
and
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/distrib/dsck_efs_zndl.mspx?mfr=true
but I'm still a little confused as to how to properly set up the passwords or user access.
I notice, very clearly that you say "BE CAREFUL" so I just want to make sure I do what ever I am supposed to do .. in a step by step format so I don't make a mistake.
Would you be so kind as to explain (in a step by step format) what I need to do, etc. to get this set up?
Thanks,
Gary
PS: If your 'step by step' info is good, I will gladly double my points for your effort. Thanks.
You could store text in a database that is encrypted, or you could make calls to external programs like 7zip, to not only compress the data users have, but also encrypt it. If your not providing a way for users to recover their data if they loose this password, then that is a good way to go. If you want your users to be able to recover or change their password for this secured data, you'll need a different model, and have to store the pass in plain-text, or store the passwords plain-text in a similar archive that only the you can access.
for no compression, just encryption here is a command line for 7z.
7z.exe a archive.7z -psecret -mhe -mx0 *.txt
copies *.txt files to archive.7z using password "secret". Also it encrypts archive headers (-mhe switch), so filenames will be encrypted, -mx0 means no compression, 0 is none, and 9 is the highest compression, so -mx9 would mean maximum compression.
7z.exe x archive.zip -psecret
extracts all files from archive.zip using password "secret".
-rich
for no compression, just encryption here is a command line for 7z.
7z.exe a archive.7z -psecret -mhe -mx0 *.txt
copies *.txt files to archive.7z using password "secret". Also it encrypts archive headers (-mhe switch), so filenames will be encrypted, -mx0 means no compression, 0 is none, and 9 is the highest compression, so -mx9 would mean maximum compression.
7z.exe x archive.zip -psecret
extracts all files from archive.zip using password "secret".
-rich
Absolutley, Not a problem.
Also, it's important to know that the files will not be encrypted using a standard "password" that is known. The GOOD thing about EFS is that it using Public/Private Key encryption which means that when a file is encrypted, the user that is encrypting the file encrypts it with their PUBLIC key only (unless you have specified a Recovery Agent via a Group Policy - in this case it uses the RA's public key also - allowing two PRIVATE keys to decrypt the file). This will allow only the user (that has the matching private key) that has encrypted the file access to it (also, it is required to decrypt a file in order to view it's ACL, so NTFS permissions will NOT allow or disallow acces - EFS list will be the first "check" point you could say). The reason i say BE CAREFUL is because when you encrypt a file, the public key and private key that you use to do this are assigned by one of two places 1- your own computer or 2- a CA that has been set up in your domain environment. The Certificate that includes your keys are stored in one of two places 1- your local computer, more specifically your user profile (of course, of the user that encrypted the file) or 2- the CA in your domain (which is integrated with your AD). Here is where the can of worms opens. If you use a self-signed certificate (keys assigned by your local computer and NOT a CA) and you user profile become corrupt or a password is lost, the files cannot be retrieved. Using a CA is not perfect either as user account could accidently be deleted (with digital certificates related to them) and hence losing the keys to decrypt the file.
Sorry about that rant - but I wanted to give you some background as to how it all works and why you should be careful.
In order to give you proper directions as to the way to actually set up the EFS, I need a little more information from you.
1. Is this a domain environment?
2. Do multiple people need access to these files?
Also, it's important to know that the files will not be encrypted using a standard "password" that is known. The GOOD thing about EFS is that it using Public/Private Key encryption which means that when a file is encrypted, the user that is encrypting the file encrypts it with their PUBLIC key only (unless you have specified a Recovery Agent via a Group Policy - in this case it uses the RA's public key also - allowing two PRIVATE keys to decrypt the file). This will allow only the user (that has the matching private key) that has encrypted the file access to it (also, it is required to decrypt a file in order to view it's ACL, so NTFS permissions will NOT allow or disallow acces - EFS list will be the first "check" point you could say). The reason i say BE CAREFUL is because when you encrypt a file, the public key and private key that you use to do this are assigned by one of two places 1- your own computer or 2- a CA that has been set up in your domain environment. The Certificate that includes your keys are stored in one of two places 1- your local computer, more specifically your user profile (of course, of the user that encrypted the file) or 2- the CA in your domain (which is integrated with your AD). Here is where the can of worms opens. If you use a self-signed certificate (keys assigned by your local computer and NOT a CA) and you user profile become corrupt or a password is lost, the files cannot be retrieved. Using a CA is not perfect either as user account could accidently be deleted (with digital certificates related to them) and hence losing the keys to decrypt the file.
Sorry about that rant - but I wanted to give you some background as to how it all works and why you should be careful.
In order to give you proper directions as to the way to actually set up the EFS, I need a little more information from you.
1. Is this a domain environment?
2. Do multiple people need access to these files?
ASKER
richrumble,
Although I appreciated your input .. I am just very confused as to exactly how to implement this.
I don't need compression of the data. I just need to have the files protected so no one (other than the users that I want to permit) will be able to access the contents of a folder and therefore not have access to the files and their contents.
The files are dyamaically added to the folder as users fill out the online forms. So, as they are stored on the server, I just want them to be protected .. as they are saved.
If you think 7z is still a way to go and would like to explain this further, maybe we can do this by phone (if you're in the US) or through Skype. Let me know. It just sounds more involved then I'll be able to grasp through here. I'll need a little "verbal" help. hahaha
Gary
Although I appreciated your input .. I am just very confused as to exactly how to implement this.
I don't need compression of the data. I just need to have the files protected so no one (other than the users that I want to permit) will be able to access the contents of a folder and therefore not have access to the files and their contents.
The files are dyamaically added to the folder as users fill out the online forms. So, as they are stored on the server, I just want them to be protected .. as they are saved.
If you think 7z is still a way to go and would like to explain this further, maybe we can do this by phone (if you're in the US) or through Skype. Let me know. It just sounds more involved then I'll be able to grasp through here. I'll need a little "verbal" help. hahaha
Gary
ASKER
Gssc1414
Great.
Maybe we could go over this as well by phone or using Skype so that you can verbally answer questions I might have as opposed to going back and forth, back and forth here, which will very likely confuse the heck out of me. hahaha Let me know.
Thanks,
Gary
Great.
Maybe we could go over this as well by phone or using Skype so that you can verbally answer questions I might have as opposed to going back and forth, back and forth here, which will very likely confuse the heck out of me. hahaha Let me know.
Thanks,
Gary
Also, EFS wouldn't work for passwording each users file under different passwords, unless your applicaion was going to create actual user names on the server. EFS is tied into windows authentication, so I don't think that is a viable option.
Also, TrueCrypt would be an ok option to store the files in. When files are "mounted" in TC, they are only unencrypted in memory, and never is the data on the HD decrypted, until you tell TC to do so. But when TC accesses encrypted data on the disk, it only decrypts that data to memory, but it's not all the data. I've got 4-8 gig folders I encrypt, and only 2 gig of ram, it only decrypts what is being read, not the entire archive per se. So my virus scanner can scan through a 8 gig file, or a 500gig hard-drive, and I never run out of memory, because only certain parts are being accessed, not the entire 500gigs at once.
-rich
Also, TrueCrypt would be an ok option to store the files in. When files are "mounted" in TC, they are only unencrypted in memory, and never is the data on the HD decrypted, until you tell TC to do so. But when TC accesses encrypted data on the disk, it only decrypts that data to memory, but it's not all the data. I've got 4-8 gig folders I encrypt, and only 2 gig of ram, it only decrypts what is being read, not the entire archive per se. So my virus scanner can scan through a 8 gig file, or a 500gig hard-drive, and I never run out of memory, because only certain parts are being accessed, not the entire 500gigs at once.
-rich
ASKER
Gssc1414
1. Is this a domain environment? Yes.
2. Do multiple people need access to these files? Yes (But mostly through an application that looks at the files in order to bring them into another place. It seems to do this with a specific user permissions.)
1. Is this a domain environment? Yes.
2. Do multiple people need access to these files? Yes (But mostly through an application that looks at the files in order to bring them into another place. It seems to do this with a specific user permissions.)
it's pretty easy to call 3rd party tools from .net, asp, php or other such web language. you'd store the users password input in array or variable, you can easily put that variable into the command line to call 7zip and encrypt, or even truecrypt to create a encrypted folder.
7z.exe a -tzip $username.zip -p$user_pass -mx0 $username.txt (this is a password protected zip file, which XP and beyond, would be able to decrypt natively, no compression is used)
$username and $user_pass are the variables you store the users input to. If your not sure how to create such variables or arrays, you may want to request your question be moved to a web programming forum here on EE.
-rich
7z.exe a -tzip $username.zip -p$user_pass -mx0 $username.txt (this is a password protected zip file, which XP and beyond, would be able to decrypt natively, no compression is used)
$username and $user_pass are the variables you store the users input to. If your not sure how to create such variables or arrays, you may want to request your question be moved to a web programming forum here on EE.
-rich
richrumble,
Correct me if im wrong garymgordon, but I believe he is simply looking for data encryption on a shared folder on a server. The web application will save the data to a text document on the server, which will be in a folder (encrypted by EFS). Simple as that. I think rich is getting at the point in which each individual file being accessed by a seperate user would be hard in EFS. If you were to encrypt a folder with EFS, and allow for example Bill, Mary, and John access to the files, then they will have access to ALL the files in the folder.
Does this still sound like what your looking for?
If so, sure we can talk on skype. I have never used it, but I can install it. Let me know.
Correct me if im wrong garymgordon, but I believe he is simply looking for data encryption on a shared folder on a server. The web application will save the data to a text document on the server, which will be in a folder (encrypted by EFS). Simple as that. I think rich is getting at the point in which each individual file being accessed by a seperate user would be hard in EFS. If you were to encrypt a folder with EFS, and allow for example Bill, Mary, and John access to the files, then they will have access to ALL the files in the folder.
Does this still sound like what your looking for?
If so, sure we can talk on skype. I have never used it, but I can install it. Let me know.
>I just need to have the files protected so no one (other than the users that I want to permit) will be able to access the contents of a folder and therefore not have access to the files and their contents.
That is NTFS permissions, no need to encrypt per se, you can use xcalcs.exe to allow you to add or subtract users access rights for a file or folder. If this is an intranet (internal) app, EFS could work, but users will have to use web folders: http://technet.microsoft.com/en-us/library/bb457116(TechNet.10).aspx#EHAA
I personally detest EFS, for you should also follow this laundry list of EFS best practices to even begin to think the data is secure: http://technet.microsoft.com/en-us/library/bb457065(TechNet.10).aspx
You shouldn't have to put that much effort into using an encryption system.
-rich
That is NTFS permissions, no need to encrypt per se, you can use xcalcs.exe to allow you to add or subtract users access rights for a file or folder. If this is an intranet (internal) app, EFS could work, but users will have to use web folders: http://technet.microsoft.com/en-us/library/bb457116(TechNet.10).aspx#EHAA
I personally detest EFS, for you should also follow this laundry list of EFS best practices to even begin to think the data is secure: http://technet.microsoft.com/en-us/library/bb457065(TechNet.10).aspx
You shouldn't have to put that much effort into using an encryption system.
-rich
gary,
I see where rich is coming from now (sorry, I wrote the last post before i read everything). I have never set up anything like this to work with EFS, but it might be worth a shot. I would be willing to try and help you set something up to test it and see if it functions like you want. I see exactly where rich is coming from though, now, and I wasn't taking into consideration that it's a application that your using in order to access these files that you need encrypted (sorry, it's been a long week - Finals :( ). Not all hope is lost though. If your willing to play around a little and do some testing, Let me know.
I see where rich is coming from now (sorry, I wrote the last post before i read everything). I have never set up anything like this to work with EFS, but it might be worth a shot. I would be willing to try and help you set something up to test it and see if it functions like you want. I see exactly where rich is coming from though, now, and I wasn't taking into consideration that it's a application that your using in order to access these files that you need encrypted (sorry, it's been a long week - Finals :( ). Not all hope is lost though. If your willing to play around a little and do some testing, Let me know.
ASKER
Gssc1414,
Thanks. I am open to trying anything that might work.
Please let me know how we can work on this.
Gary
Thanks. I am open to trying anything that might work.
Please let me know how we can work on this.
Gary
ASKER
Gssc1414,
I haven't heard back and was hoping you might be able to help me further on this.
Gary
I haven't heard back and was hoping you might be able to help me further on this.
Gary
ASKER
By the way .. isn't there any 3rd party software out there that can be used for this purpose that would allow me to be able to encrypt data on the server as the items are added into a particular folder. My thought is .. that the files would become encrypted as they are created or added to the folder (in any way) and then .. would require a password to view the files or .. require specific user permissions on the server to view the files? I need the files to be encrypted or protected immediately as they are added to a folder on the computer .. and then only viewable if a user is logged in .. with permissions to view the data or has a password to reveal the contents of each file.
What do you think? Does anything exist like this?
Gary
What do you think? Does anything exist like this?
Gary
A truecrypt volume, be it a partition, folder or drive, would encrypt the data stored on it as it is written to the volume. And again it's only decrypted in memory as it's accessed, but does have to be mounted for access. The data inside a TC volume is only decrypted on the disk if you go into TC and tell it to decrypt the data. You can access the data after mounting it, after it is mounted it looks like a plain old HD/Partition/Folder. You could use NTFS permissions on folders inside the TC volume to restrict access just like on a normal NTFS file or folder.
I've not seen much in the way of this type of system from 3rd parties. If I were designing such a system, I would have to lay my requirements out:
1) Text, Images and Binaries (exe's mostly) will be stored
2) All such data will need to be encrypted on the disc, but not to or from storage point
3) Allow multiple users access to data without sharing passwords
I would have an encrypted partition, and use the native harddrive ACL's to provide access restrictions
The truecrypt volume, or hardware encrypted HD (such as Seagates Momentus LT drives) would auto-mount when the PC/Server is booted. I'd use PHP or ASP to make calls to Xcalcs.exe for windows, or chmod/chgrp for Linux, and add user access rights to files and folders through a web interface. The interface would allow someone to request access, or allow preemptive access to files.
My 7zip example would need to be modified a bit, but could still work. The password for opening the archive would be stored in a DB, and the DB would also have a table of who could access the archive. It would not be easy to use on a grainular level like NTFS permissions for only certain files or folders, but could be made to work.
I however cannot do much in PHP/ASP currently, there are many talented folk here on EE that can however. I would piece the whole thing together in a few 500 pt questions from different forums.
-rich
I've not seen much in the way of this type of system from 3rd parties. If I were designing such a system, I would have to lay my requirements out:
1) Text, Images and Binaries (exe's mostly) will be stored
2) All such data will need to be encrypted on the disc, but not to or from storage point
3) Allow multiple users access to data without sharing passwords
I would have an encrypted partition, and use the native harddrive ACL's to provide access restrictions
The truecrypt volume, or hardware encrypted HD (such as Seagates Momentus LT drives) would auto-mount when the PC/Server is booted. I'd use PHP or ASP to make calls to Xcalcs.exe for windows, or chmod/chgrp for Linux, and add user access rights to files and folders through a web interface. The interface would allow someone to request access, or allow preemptive access to files.
My 7zip example would need to be modified a bit, but could still work. The password for opening the archive would be stored in a DB, and the DB would also have a table of who could access the archive. It would not be easy to use on a grainular level like NTFS permissions for only certain files or folders, but could be made to work.
I however cannot do much in PHP/ASP currently, there are many talented folk here on EE that can however. I would piece the whole thing together in a few 500 pt questions from different forums.
-rich
Microsoft's SharePoint is very similar to what you want, it allows collaboration on various M$ office documents with access rights similar to what I've outlined in my what-if above. There are also emerging open-source projects, like SVN, CVS and I think OpenOffice has something on the horizon for this...
http://o3spaces.com/Page/sp4/nctrue/index.html
It even has HTTPS connections: http://o3spaces.com/Page/sp99/ml2/Index.html
The O3Spaces Workplace Community Edition is a free-to-use and free of charge edition of the O3Spaces software. The Community Edition enables teams and workgroups of up to 10 people to collaborate and share their documents.
There is also a PRO version for varying costs: http://o3spaces.com/Page/sp104/ml2/Index.html
We are using Zimbra for a similar solution: http://www.zimbra.com/products/ Very good, and if we encrypted our HD partition we'd have all the data stored encrypted, but currently do not.
-rich
http://o3spaces.com/Page/sp4/nctrue/index.html
It even has HTTPS connections: http://o3spaces.com/Page/sp99/ml2/Index.html
The O3Spaces Workplace Community Edition is a free-to-use and free of charge edition of the O3Spaces software. The Community Edition enables teams and workgroups of up to 10 people to collaborate and share their documents.
There is also a PRO version for varying costs: http://o3spaces.com/Page/sp104/ml2/Index.html
We are using Zimbra for a similar solution: http://www.zimbra.com/products/ Very good, and if we encrypted our HD partition we'd have all the data stored encrypted, but currently do not.
-rich
ASKER
rich,
I must be missing something along the way. Just confused I guess.
I don't understand why you're recommending a "collaboration" piece of software since I'm not using it for that purpose at all.
Can I IM with you or call you so maybe you can explain this to me verbally? I'm just not understanding why you're recommending these applications. They don't (from what I am seeing) apply to my situation. But again, I am very possibly missing something along the way.
Can I IM you to chat about this?
Gary
I must be missing something along the way. Just confused I guess.
I don't understand why you're recommending a "collaboration" piece of software since I'm not using it for that purpose at all.
Can I IM with you or call you so maybe you can explain this to me verbally? I'm just not understanding why you're recommending these applications. They don't (from what I am seeing) apply to my situation. But again, I am very possibly missing something along the way.
Can I IM you to chat about this?
Gary
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
rich,
I read the prior posting but still have questions that would be best answered in an IM chat. I am still confused as to how to use your recommendation to get it to work. So, would chatting on IM be an option?
Let me know.
Gary
I read the prior posting but still have questions that would be best answered in an IM chat. I am still confused as to how to use your recommendation to get it to work. So, would chatting on IM be an option?
Let me know.
Gary
One thing to look at serioulsy here is that the EFS will only apply to the data that is stored on your server, or "in place" data. The data will be sent from the clients browser to your server in CLEAR text unless you impliment SSL to encrypt the data that is "in-transition".
With both EFS and SSL, the data will be encrypted immediatley after the user submits the data on your web form.