Where to store encryption key?

First and foremost, storing keys in the clear on a WEB server is the worst thing you can do.  The web servers are the first systems in the chain that will be compromised in an attack.  It is bad enough having keys in memory on a web server.  Never put them on disk in the clear!

I assume you are doing key exchange and encryption in your WEB application to avoid sending credit card information in the clear, right?  I also assume that the database storage issue is distinct from the WEB encryption scheme.

BTW, what encryption algorithm were you considering using?  Obviously, a symmertric key is all that is required but I am hoping you were not planning on a simple substitution or other home-brewed approach.  The guys that commit wholesale hacking and credit card fraud are very sophisticated and you would be suprised how easily anything other than professionaly built and extensively vetted cyphers are to break.  You have two particular disadvantages here as well.  First, the data is short and is of very well know structure.  Second, breaking a cypher when you have both the encrypted and clear message is many times easier.  A hacker who has gained access to your system could seed the data by making a small purchase with a credit card before he steals the whole database in order to make breaking the code eaiser.

As far as your question about best practices for storing encrypted columns and the associated keys in database, I can tell you how at least one of the big guys handles it.

Sybase ASE 15 just came out with this enhanced functionality.  It is a pretty well thought out system that you might want to adopt/adapt for PostgreSQL.

The encryption keys for each column are stored in encrypted form in a system table.  There is a system-wide master encryption key for encrypting/decrypting the column keys.  It can only set and changed (but never read)  by a user with the SSO role.  Users are then granted read/decrypt access to specific columns.  When a user goes to select a column, the database figures out if that user id has access, if so, it goes to the table with the column keys, retrieves the appropriate one, decrypts it using the system master key, then decrypts the column using the decrypted column key.

This might be overkill for what you are trying to do but there are some important ideas here.
1) Never send keys in the clear over the wire to a client process
2) Don't store the keys in clear form anywhere
3) Lock the whole thing up with a non-reversable, non-readable key

Lets take a look at how you might go about implementing a poor-man's version of the Sybase scheme.

Item 2 is simple, store the keys in encrypted form in a table.  That way even if there is a breach and access to the table is compromised, the keys don't do anyone any good.

The next nut to crack here is item 3.  While Sybase use AES (a block cipher with the same key to encrypt and decrypt) I suspect that the system-wide master key (used to encrypt the column keys) is a puplic key scheme.  Somewhere in the bowls of the database engine either hardcoded or, more likely initialized randomly at install time, is a public/private key pair.  The private key is used by the process that stores the system-wide master key and the public (using the term loosely) key is used by the query processor to decrypt that key so that it can then decrypt the column key(s) as required.  If you want to break the database security, you have to find where both keys are hidden and somehow devine exactly what encryption algorithm was used.  Not a simple thing.  Do you need that level of security?  I don't know.  You could hard-code a reversable key in a C language module to store and retrieve the master key, or, even simpler, you can simply store the master key in the clear in a table that only the Security Officer role has access to.  That leaves you vulnerable if a backup tape is lost or stolen though.  (Boy, that hasn't happenned recently ;-)  You have to decide the level of security required based on corporate compliance (e.g. Sarbanes-Oxley) as well as the potential liability of a security breach.

The really big issue and, I suspect the part of your existing plan that blows up, is item 1; never send keys in the clear over the wire.  You have a couple of ways to get around this.  First, you can do what Sybase, Oracle, DB2-UDB, etc., all do and keep all encryption/decryption on the server.  To do this you will have to build functions and/or procedures that deal with encryption, decryption, and user authorization to access a column.  The nice part about this is that you can build a generalized facility which works for any column in the database.  It's a great idea but may be beyond what you are willing to undertake.

The other approach if you cannot see your way clear to keep the encryption on the database server and have to move it out to the client side, is to retrieve the encrypted column key(s) back to the client and decrypt them there.  The issue with that scheme is trying to keep the system-wide master key secure between the server and the client(s).  Again, how far you carry this depends upon the security level you need to enforce.  If you could run the monthly re-bill as a batch job right on the database server, you would have fewer concerns.  I am guessing that that is not practical because you are using a WEB server-based credit card authorization facility forcing you to implemented on the IIS and/or Apache server.  Obviously, you only need to implement the re-bill on one of those.

If your re-bill application is the only thing that needs to decrypt columns, you could publish a one-time public key from the client and have the database server use that to send the system master key over encrypted with it.  The client can then decrypt it and use it to decrypt column key(s) as needed.  All of this gets kind of clumsy and still requires that you develop some encryption capability at the server end.

One final thought is that this work has been done before so there is no need to reinvent the wheel.  I would take a look at RSA's BSAFE developers kit.  I have no idea what the pricing structure is but everything you would need to do this right is in there.  I am sure there are other commercial products out there as well.

Sorry there isn't a simple answer, much less solution, to your question, however; when you can pick up the paper just about any day of the week and read about another high profile data theft, you kind of have to be a bit careful about how you handle financial data.

Best of luck,
Bill