Shortest key possible to encrypt data at 128, 192, 256 bits of encryption

this must be some very very very sensitive information for you to need to encrypt each user's information with a different key.  that is very unusual for normal websites.  SHA-1 requires a minimum key length of 20 bytes and that is the size of the samples you are providing.....

however, i guess you could write a little encode/decode function....because the keys are in hex they 'appear' to be a long char string....if you wanted to encode the 20byte hex value to a char value you could do that, and you could lower the amount of characters the user will have to type in as the "key" (the encoded key, which would need to be decoded to its hex value).

but(!) as long as you do that encoding you're gonna eliminate a lot of characters from the key making its effectiveness equal to that of a shorter key. (try to envision a brute force cracker here - a lot less to try).

why don't you go with files being used as keys? with a bit of a salt in the file and added password that in turn can be far simpler to remember it would make the life of your users easier. to use the code written with a similar scheme you can check out the truecrypt project (http://www.truecrypt.org). then of course you'll have to conform to the license they use. a version of GPL probably.

bear in mind that the worst thing you can do is use obscurity in your security scheme. that will definitely scare off some of the more knowledgeable customers who in turn won't advertise your service with word of mouth, quite the opposite instead - so you'll have to come up with something better than the encoding of your keys.

you can encode 3 hex digits in 2 characters from a set of 64

Thanks for the suggestions everyone!

The data is very very sensitive, hence the added layer of security using a separate key to encrypt each user's information.

ryandale56 and ozo - do either of you know of a good non-"obscure" way of encoding hex values, so that we dont "scare off some of the more knowledgeable customers" as jakopriit said?

jakopriit - obscurity is something we are absolutely trying to avoid, hence the use of AES and SHA.  I'm not really sure how using a file and an added salt makes a key more secure.  Say we generated a key for a user: a5bH2S@adg.  How do we then use that short of a key to provide very strong encryption using the file/salt technique?  (I can start a new question if that is too specific for this topic)

the password then is essentially only a small part of a key. without the file part, the password itself is useless and the other way around.

say you write a java application to upload the file/key and the password. the encrypted upload communication there has to be "salted" there with seemingly random amount of random data helping to fend the replay attacks and such, much like the SSL/TLS handshakes. In case of the "very, very sensitive" data you are obliged to think of all the possible and improbable attack types your site can suffer.

and yes, if you feel like more questions are storming your fingertips, eager to reach the keyboard, you can start new questions any time you like ;)

btw: ozo probably referred to BASE64 (or the reverse of it) : http://tools.ietf.org/html/rfc4648

We can't use their username or password as part of the encryption key as we have a design constraint that can't allow this as others get a different username and password but share the same key. What we really want to ask is there any other alternative other than using an accepted method like Sha1 (actually we are going to use sha224 as sha1 was compromised) generated key acceptable to the security professionals out there. We are concerned about the substantial key length. One concern is for our users, the other is for security. Obviously, the shorter the key, the easier it will be to brute force it.

jakopriit , when you say "why don't you go with files being used as keys", could you elaborate a bit here. Thanks!

you've lost me there.. why would I want to share the key with Joe Average?

files as keys -- well, did you look at the excellent documentation the truecrypt project provides? http://www.truecrypt.org/docs/

Why user must remember your keys. Why you not send the user a card or USB Stick with key, try to decript info with some hardware.

I'm not so sure about effectiveness of SHA algorithm as a password generator. It's intend for quite different use - secure hash digest generation namely. So your password strength actually depends on the nature and quality of the data SHA operates on. More so if this data is directly/indirectly based on user input.

Ideally you must have true random data as a key/password. Straight way to have N-bit strong password constrained to set of X different characters is to generate true random string with at least N*log(2)/log(X) characters. This means 32 character string for 128-bit strong password when using [0-9A-F] hex characters. When utilizing both lower- and upper-case versions of all printable standard english keyboard keys one can have set of 94 different characters to operate on. In this case random string must be at least 20 character long for 128-bit randomness.

As for tools to ease the life of users to remember and enter such long passwords it all depends on the budget and intended audience. You can go with hardware tokens and appropriate PKI roll-out. Or as alternative quick-kill solution - recommend your audience the use of password management tools (along with providing detailed step-by-step usage instructions in your case). For one I use KeePass (http://keepass.sourceforge.net) as an excellent FOSS solution. It can even integrate with various browsers through plug-ins or one can utilize it's secure copy-paste operation.

As for final word, in provided scenario user data stored on your site is unaccessible for everyone except the holder of the key/password. Are you sure this fits well with your intentions and design goals? You won't be able to display/process this information without user participation.

Cheers!

In over 12 years of being a security expert, I've NEVER come across anyone who has had to address a problem in this way.  I recommend taking a few steps back and looking at exactly what you want to secure....
What's wrong with using PKI and client certificates?  That way, users will know the username/password, and will be able to use their installed client certs to access encrypted data.  If information really is that sensitive, you should also look at 2-factor authentication (ie SecurID tokens).
Why do you perceive that using 2 passwords (ie username/password and SHA key) is giving you a greater depth of security?  If a hacker has gone to the lengths of sniffing a username/password, then he would also have sniffed the SHA key that goes with it, so it's not actually giving you defense in depth at all?

As I said - you need to seriously reconsider generation of passwords using SHA algorithm. Best option is to random-generate appropriate password strings as you will change user passwords anyway. Technically you may solve original problem by this little Python script:

import sys

# Global parameters
allHexChars = sorted("0123456789ABCDEF")
allEnglishKeyboard = sorted("`1234567890-=qwertyuiop[]\\asdfghjkl;'zxcvbnm,./~!@#$%^&*()_+QWERTYUIOP{}|ASDFGHJKL:\"ZXCVBNM<>?")

def passwordRecode(sourceString, sourceAllowedChars, targetAllowedChars):
    """Convert password string into different set of allowed characters
       Parameters:
           sourceString       - source password as a string
           sourceAllowedChars - list with all allowed characters for source
           targetAllowedChars - list with all allowed characters for target"""
       
    # We need to be sure about allowedChars sets
    def checkAllowedChars(chars):
          "Checks allowed chars for consistency (no duplicate characters)"
          sortedChars = sorted(chars)
          for idx in range(len(sortedChars) - 1):
                if sortedChars[idx] == sortedChars[idx + 1]:
                      return False
          return True

    assert checkAllowedChars(sourceAllowedChars)
    assert checkAllowedChars(targetAllowedChars)

    sourceBase = len(sourceAllowedChars)
    targetBase = len(targetAllowedChars)

    # Convert source string into large integer
    sourceDict = {}
    for idx in range(sourceBase):
          sourceDict[sourceAllowedChars[idx]] = idx

    # Long integers in Python have unlimited precision
    key = 0L
    base = 1L
    for char in sourceString:
          key += base * sourceDict[char]
          base *= sourceBase

    # Compute target
    target = []
    while key > 0:
          target.append(targetAllowedChars[key % targetBase])
          key = long(key / targetBase)
    targetString = "".join(target)

    return targetString

# Default action is to use first command line argument as
# a source and print result on standard output
print passwordRecode(sys.argv[1], allHexChars, allEnglishKeyboard)

Security isn't a program, it is a process. If your concerned about the DB falling into the wrong hands, or hypothetical hands, then encrypt the DB itself, as well as the users data. The users data is encrypted with their own individual password, and your DB is encrypted with another. Security is about trade-offs, if it weren't, then I'd be out of a job :) The idea of using a file for the key is a good one, I thought and thought about how you would do that, and I think applying it in the reverse of how it was suggested (jakopriit) would be best. And by that I mean:
Encrypt a partition or the folder that houses the DB with TrueCrypt. Use keyfiles for that partition/folder(s) encryption. The users password will be their encryption key.
Users can make good and long passwords, and you can make it fun for them, I give funny examples typically. There is practically no substitute for a long password, even a simple one. I use every BF app ever known (well lot's of them) and they all have an "incremental" mode after or before the dictionary attack. Some do try to guess what would be typed rather than just trying every possible combination like this: aa, aaa, aab, aac, aad, aae, ... ass ... azz, baa (etc..)

Ih8MYB0$$&hizporche!! (I hate my boss and his porche!)
:),itIZfryday2day! (Smile, it is friday today!)
:-0eyeStillcan'tbeilieveit'sNOTBUTTER! (Oh, I still can't believe it's not butter!)
passWORDZr_4suckers  (passwords are for suckers)
1WISHuweren'tHEAR!)  (I wish you weren't here wink/grin)
Ydidn'tIthink0Fthese?   (why didn't I think of these?)
7.00$4sigarettes!OMG  (seven dollars for cigarettes! Oh my god)
Misspellings, abbreviations, character substitutions and special character usage, as well as passwords over 10 chars long. If the user doesn't make a strong password, they weren't that concerned with keeping this data safe. So protect them from themselves by encrypting your DB also. TrueCrypt has great entropy and is a very good program, you should read up on it's many benefits, like being able to hide so that a would be attacker might not even know the encrypted data is there: http://www.truecrypt.org/docs/?s=plausible-deniability http://www.truecrypt.org/docs/keyfiles.php http://www.truecrypt.org/faq.php

The DB encryption will be the first hurdle for a would be attacker, and the users passwords would be next. We've discussed entropy previously, and the length that an attack on modern cryptosystems would/will take for even the NSA/GOV. I think having the users supply a file for the password is a good idea, but I couldn't imagine it being any more secure than what I outlined above, as well as simpler for all... I hope.
-rich

Just a quick note - encrypting the database files with Truecrypt, or even native SQL 2005 / Oracle 8+ encryption can really slow things down on databases with high rates of access requests.

Thanks for all the responses everyone!  We are still internally debating the whole key idea and looking into your responses.  

I know this is an off topic, but this topic is so interesting.

My Q to you is, why you want that encryption if your users will give their pass to the first one who ask for it?

The important is not the password is the person yuo give it.

Thanks for the help, sorry forgot to award points earlier.

Just a little more reading ;)
http://www.schneier.com/blog/archives/2007/01/choosing_secure.html
http://www.wired.com/news/columns/0,72300-0.html
-rich