Solved

How to encrypt data (flat text files) that as they are written and then stored in various folders on the server?

Posted on 2008-06-17
21
408 Views
Last Modified: 2013-12-04
Question:  How to encrypt data (flat text files) that as they are written and then stored in various folders on the server?

I have online web forms that users fill out. The data from some of these web forms are stored on the same server, in a plain text file. The information collected is sometimes of a personal and sensitive nature.

So, I am trying to figure out if and how I can accomplish the following.

I would like to have the text files that are dynamically created after a user fills out one of the online forms... to automatically be encrypted or password protected.  

I want the information (as it is collected and stored in a folder on the server) to be encrypted (protected so no one could access the information .. immediately as it is collected and saved.

I had looked at a couple software options (from PGP and TrueCrypt) that basically created virtual drives, where the data could be saved to.  But in the end, if the virtual drives were active (i.e., mounted, etc.) the data could be written to the virutal drive, but since the drive was visable .. it was also NOT protecting the data. In both cases, only if the drive was dismounted would the data be encrypted and protected. (But if the drive is dismounted, it is no longer visable and therefore could not be written to by my online forms.  

I need to know how to set up "something" that would allow the files (as they are created .. as users fill out the online forms) .. to automatically be protected in a sufficient manner so that no one (another user on the network, a hacker, etc.) would be able to see the contents of the files.

I am hoping someone can help me with a suggestion that would allow me to somehow .. protect certain folders from being READ or accessed by anyone .. yet allow my scripts, etc. to write and add data into the protected folders.  So, in a nut shell, data could be saved into the protected folder, but nothing could be taken out or read from the folder .. without knowing a secure password or something.

Any thoughts to get this done?

Thanks,
Gary
0
Comment
Question by:garymgordon
  • 9
  • 7
  • 4
  • +1
21 Comments
 
LVL 3

Expert Comment

by:Gssc1414
ID: 21804738
If you know the folders that the data will be stored in, you could secure the data using the build in EFS that is included in windows. Please not that you will be encrypting the FOLDER, which will force all files saved in the folder to be encrypted as well (automatically and instantly as they are saved). Please be careful when dealing with EFS and make sure to have a good plan when going into it. You will need to look into Recovery Agents and backing up your the private keys that will be given access to the files. EFS will allow you to add multiple users to the folder to allow as many or as little peole access (as little as one account - but be CAREFUL!). I know your said the data will be stored in "various" folders, but if all the data in these folders is submitted and needs to be protected, then EFS should work just fine. I just want to urge the fact that if you do use EFS, make sure to know what your getting into and set it up properly so that you don't get caught with a bunch of encrypted data with no way to decrypt it.

One thing to look at serioulsy here is that the EFS will only apply to the data that is stored on your server, or "in place" data. The data will be sent from the clients browser to your server in CLEAR text unless you impliment SSL to encrypt the data that is "in-transition".

With both EFS and SSL, the data will be encrypted immediatley after the user submits the data on your web form.
0
 
LVL 3

Expert Comment

by:mahe2000
ID: 21804967
you can try to do something with gpg (http://www.gnupg.org/) but you will need to modify you application. Or may be you can apply an encryption function like 3des or aes to the sensitive fields.
0
 

Author Comment

by:garymgordon
ID: 21804972
Gssc1414,

1. I am using SSL on the web pages where users fill out the online forms.
2. I read some information from:
http://www.experts-exchange.com/Security/Operating_Systems_Security/Windows/Q_22067774.html
and
http://www.microsoft.com/windowsxp/using/security/expert/sharefilesefs.mspx
and
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/distrib/dsck_efs_zndl.mspx?mfr=true

but I'm still a little confused as to how to properly set up the passwords or user access.

I notice, very clearly that you say "BE CAREFUL" so I just want to make sure I do what ever I am supposed to do .. in a step by step format so I don't make a mistake.

Would you be so kind as to explain (in a step by step format) what I need to do, etc. to get this set up?

Thanks,
Gary

PS:  If your 'step by step' info is good, I will gladly double my points for your effort.  Thanks.
0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21805103
You could store text in a database that is encrypted, or you could make calls to external programs like 7zip, to not only compress the data users have, but also encrypt it. If your not providing a way for users to recover their data if they loose this password, then that is a good way to go. If you want your users to be able to recover or change their password for this secured data, you'll need a different model, and have to store the pass in plain-text, or store the passwords plain-text in a similar archive that only the you can access.

for no compression, just encryption here is a command line for 7z.
7z.exe a archive.7z -psecret -mhe -mx0 *.txt

copies *.txt files to archive.7z using password "secret". Also it encrypts archive headers (-mhe switch), so filenames will be encrypted, -mx0 means no compression, 0 is none, and 9 is the highest compression, so -mx9 would mean maximum compression.

7z.exe x archive.zip -psecret

extracts all files from archive.zip using password "secret".
-rich
0
 
LVL 3

Expert Comment

by:Gssc1414
ID: 21805163
Absolutley, Not a problem.

 Also, it's important to know that the files will not be encrypted using a standard "password" that is known. The GOOD thing about EFS is that it using Public/Private Key encryption which means that when a file is encrypted, the user that is encrypting the file encrypts it with their PUBLIC key only (unless you have specified a Recovery Agent via a Group Policy - in this case it uses the RA's public key also - allowing two PRIVATE keys to decrypt the file). This will allow only the user (that has the matching private key) that has encrypted the file access to it (also, it is required to decrypt a file in order to view it's ACL, so NTFS permissions will NOT allow or disallow acces - EFS list will be the first "check" point you could say). The reason i say BE CAREFUL is because when you encrypt a file, the public key and private key that you use to do this are assigned by one of two places 1- your own computer or 2- a CA that has been set up in your domain environment. The Certificate that includes your keys are stored in one of two places 1- your local computer, more specifically your user profile (of course, of the user that encrypted the file) or 2- the CA in your domain (which is integrated with your AD). Here is where the can of worms opens. If you use a self-signed certificate (keys assigned by your local computer and NOT a CA) and you user profile become corrupt or a password is lost, the files cannot be retrieved. Using a CA is not perfect either as user account could accidently be deleted (with digital certificates related to them) and hence losing the keys to decrypt the file.

Sorry about that rant - but I wanted to give you some background as to how it all works and why you should be careful.

In order to give you proper directions as to the way to actually set up the EFS, I need a little more information from you.

1. Is this a domain environment?
2. Do multiple people need access to these files?
0
 

Author Comment

by:garymgordon
ID: 21805183
richrumble,

Although I appreciated your input .. I am just very confused as to exactly how to implement this.

I don't need compression of the data. I just need to have the files protected so no one (other than the users that I want to permit) will be able to access the contents of a folder and therefore not have access to the files and their contents.

The files are dyamaically added to the folder as users fill out the online forms.  So, as they are stored on the server, I just want them to be protected .. as they are saved.

If you think 7z is still a way to go and would like to explain this further, maybe we can do this by phone (if you're in the US) or through Skype.  Let me know.  It just sounds more involved then I'll be able to grasp through here.  I'll need a little "verbal" help. hahaha

Gary
0
 

Author Comment

by:garymgordon
ID: 21805200
Gssc1414

Great.

Maybe we could go over this as well by phone or using Skype so that you can verbally answer questions I might have as opposed to going back and forth, back and forth here, which will very likely confuse the heck out of me. hahaha   Let me know.

Thanks,
Gary
0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21805217
Also, EFS wouldn't work for passwording each users file under different passwords, unless your applicaion was going to create actual user names on the server. EFS is tied into windows authentication, so I don't think that is a viable option.
Also, TrueCrypt would be an ok option to store the files in. When files are "mounted" in TC, they are only unencrypted in memory, and never is the data on the HD decrypted, until you tell TC to do so. But when TC accesses encrypted data on the disk, it only decrypts that data to memory, but it's not all the data. I've got 4-8 gig folders I encrypt, and only 2 gig of ram, it only decrypts what is being read, not the entire archive per se. So my virus scanner can scan through a 8 gig file, or a 500gig hard-drive, and I never run out of memory, because only certain parts are being accessed, not the entire 500gigs at once.
-rich
0
 

Author Comment

by:garymgordon
ID: 21805452
Gssc1414

1. Is this a domain environment?   Yes.
2. Do multiple people need access to these files?   Yes (But mostly through an application that looks at the files in order to bring them into another place.  It seems to do this with a specific user permissions.)
0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21805581
it's pretty easy to call 3rd party tools from .net, asp, php or other such web language. you'd store the users password input in array or variable, you can easily put that variable into the command line to call 7zip and encrypt, or even truecrypt to create a encrypted folder.

7z.exe a -tzip $username.zip -p$user_pass -mx0 $username.txt (this is a password protected zip file, which XP and beyond, would be able to decrypt natively, no compression is used)

$username and $user_pass are the variables you store the users input to. If your not sure how to create such variables or arrays, you may want to request your question be moved to a web programming forum here on EE.
-rich
0
How your wiki can always stay up-to-date

Quip doubles as a “living” wiki and a project management tool that evolves with your organization. As you finish projects in Quip, the work remains, easily accessible to all team members, new and old.
- Increase transparency
- Onboard new hires faster
- Access from mobile/offline

 
LVL 3

Expert Comment

by:Gssc1414
ID: 21805605
richrumble,

Correct me if im wrong garymgordon, but I believe he is simply looking for data encryption on a shared folder on a server. The web application will save the data to a text document on the server, which will be in a folder (encrypted by EFS). Simple as that. I think rich is getting at the point in which each individual file being accessed by a seperate user would be hard in EFS. If you were to encrypt a folder with EFS, and allow for example Bill, Mary, and John access to the files, then they will have access to ALL the files in the folder.

Does this still sound like what your looking for?

If so, sure we can talk on skype. I have never used it, but I can install it. Let me know.
0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21805647
>I just need to have the files protected so no one (other than the users that I want to permit) will be able to access the contents of a folder and therefore not have access to the files and their contents.
That is NTFS permissions, no need to encrypt per se, you can use xcalcs.exe to allow you to add or subtract users access rights for a file or folder. If this is an intranet (internal) app, EFS could work, but users will have to use web folders: http://technet.microsoft.com/en-us/library/bb457116(TechNet.10).aspx#EHAA
I personally detest EFS, for you should also follow this laundry list of EFS best practices to even begin to think the data is secure: http://technet.microsoft.com/en-us/library/bb457065(TechNet.10).aspx
You shouldn't have to put that much effort into using an encryption system.
-rich
0
 
LVL 3

Expert Comment

by:Gssc1414
ID: 21805675
gary,

I see where rich is coming from now (sorry, I wrote the last post before i read everything). I have never set up anything like this to work with EFS, but it might be worth a shot. I would be willing to try and help you set something up to test it and see if it functions like you want. I see exactly where rich is coming from though, now, and I wasn't taking into consideration that it's a application that your using in order to access these files that you need encrypted (sorry, it's been a long week - Finals :(  ). Not all hope is lost though. If your willing to play around a little and do some testing, Let me know.
0
 

Author Comment

by:garymgordon
ID: 21806544
Gssc1414,

Thanks.  I am open to trying anything that might work.

Please let me know how we can work on this.

Gary
0
 

Author Comment

by:garymgordon
ID: 21847167
Gssc1414,

I haven't heard back and was hoping you might be able to help me further on this.

Gary
0
 

Author Comment

by:garymgordon
ID: 21849013
By the way .. isn't there any 3rd party software out there that can be used for this purpose that would allow me to be able to encrypt data on the server as the items are added into a particular folder.  My thought is .. that the files would become encrypted as they are created or added to the folder (in any way) and then .. would require a password to view the files or .. require specific user permissions on the server to view the files?  I need the files to be encrypted or protected immediately as they are added to a folder on the computer .. and then only viewable if a user is logged in .. with permissions to view the data or has a password to reveal the contents of each file.

What do you think?  Does anything exist like this?

Gary
0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21849383
A truecrypt volume, be it a partition, folder or drive, would encrypt the data stored on it as it is written to the volume. And again it's only decrypted in memory as it's accessed, but does have to be mounted for access. The data inside a TC volume is only decrypted on the disk if you go into TC and tell it to decrypt the data. You can access the data after mounting it, after it is mounted it looks like a plain old HD/Partition/Folder. You could use NTFS permissions on folders inside the TC volume to restrict access just like on a normal NTFS file or folder.

I've not seen much in the way of this type of system from 3rd parties. If I were designing such a system, I would have to lay my requirements out:
1) Text, Images and Binaries (exe's mostly) will be stored
2) All such data will need to be encrypted on the disc, but not to or from storage point
3) Allow multiple users access to data without sharing passwords

I would have an encrypted partition, and use the native harddrive ACL's to provide access restrictions

The truecrypt volume, or hardware encrypted HD (such as Seagates Momentus LT drives) would auto-mount when the PC/Server is booted. I'd use PHP or ASP to make calls to Xcalcs.exe for windows, or chmod/chgrp for Linux, and add user access rights to files and folders through a web interface. The interface would allow someone to request access, or allow preemptive access to files.

My 7zip example would need to be modified a bit, but could still work. The password for opening the archive would be stored in a DB, and the DB would also have a table of who could access the archive. It would not be easy to use on a grainular level like NTFS permissions for only certain files or folders, but could be made to work.
I however cannot do much in PHP/ASP currently, there are many talented folk here on EE that can however. I would piece the whole thing together in a few 500 pt questions from different forums.
-rich

0
 
LVL 38

Expert Comment

by:Rich Rumble
ID: 21849660
Microsoft's SharePoint is very similar to what you want, it allows collaboration on various M$ office documents with access rights similar to what I've outlined in my what-if above. There are also emerging open-source projects, like SVN, CVS and I think OpenOffice has something on the horizon for this...
http://o3spaces.com/Page/sp4/nctrue/index.html
It even has HTTPS connections: http://o3spaces.com/Page/sp99/ml2/Index.html
The O3Spaces Workplace Community Edition is a free-to-use and free of charge edition of the O3Spaces software. The Community Edition enables teams and workgroups of up to 10 people to collaborate and share their documents.
There is also a PRO version for varying costs: http://o3spaces.com/Page/sp104/ml2/Index.html

We are using Zimbra for a similar solution: http://www.zimbra.com/products/ Very good, and if we encrypted our HD partition we'd have all the data stored encrypted, but currently do not.
-rich

 
0
 

Author Comment

by:garymgordon
ID: 21849999
rich,

I must be missing something along the way.  Just confused I guess.

I don't understand why you're recommending a "collaboration" piece of software since I'm not using it for that purpose at all.  

Can I IM with you or call you so maybe you can explain this to me verbally?  I'm just not understanding why you're recommending these applications.  They don't (from what I am seeing) apply to my situation.  But again, I am very possibly missing something along the way.

Can I IM you to chat about this?

Gary
0
 
LVL 38

Accepted Solution

by:
Rich Rumble earned 125 total points
ID: 21850342
My apologies, I assumed too much. The posting before that (http://www.experts-exchange.com/Security/Encryption/Q_23492090.html#a21849383) should be closer.
-rich
0
 

Author Comment

by:garymgordon
ID: 21852062
rich,
I read the prior posting but still have questions that would be best answered in an IM chat.  I am still confused as to how to use your recommendation to get it to work.  So, would chatting on IM be an option?

Let me know.
Gary
0

Featured Post

Do You Know the 4 Main Threat Actor Types?

Do you know the main threat actor types? Most attackers fall into one of four categories, each with their own favored tactics, techniques, and procedures.

Join & Write a Comment

SSL stands for “Secure Sockets Layer” and an SSL certificate is a critical component to keeping your website safe, secured, and compliant. Any ecommerce website must have an SSL certificate to ensure the safe handling of sensitive information like…
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
You have products, that come in variants and want to set different prices for them? Watch this micro tutorial that describes how to configure prices for Magento super attributes. Assigning simple products to configurable: We assigned simple products…

747 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

16 Experts available now in Live!

Get 1:1 Help Now