Solved

Storing Passwords Versus Hashes in Challenge-Response Authentication

Posted on 2014-04-21
20
553 Views
Last Modified: 2014-05-19
So the gauntlet was thrown in this question:

http://www.experts-exchange.com/Security/Encryption/Q_28416241.html

...and I thought I'd set the stage here for the battle. The question is:

When performing challenge-response authentication, is it *ever* necessary to store the clear-text password?

The rules:

Cite your references--don't just assert without evidence to support
For God's sakes, keep it civil

Let the games begin!
0
Comment
Question by:käµfm³d   👽
  • 7
  • 7
  • 3
  • +2
20 Comments
 
LVL 38

Accepted Solution

by:
Rich Rumble earned 216 total points
Comment Utility
The ultimate take-away from that question, which will be very simple to solve (I almost figured it out in 10 minutes) is that you should not write you own encryption.
The linked Q is a hash, not encryption. It's a fixed length (27 chars max, or 9 three character positions), it's not even a true hash because this one is reversible, the output can be reversed to give you the input. Secure hashes cannot do that, you have to hash more input and then compare to the output to see if it matches, if not, hash another and compare. In the linked Q it's easy to see how reversible it is.
I'm only going to cite one person:
https://www.schneier.com/blog/archives/2011/04/schneiers_law.html

As far as the challenge-response, no it makes no sense to have a password in plain-text be part of the C-R, you want the hash of the plain-text, and then you want to use the challenge to re-hash the hash, and then see if that matches up. You do need the Plain-Text to create the hash, and then the challenge to hash the hash of the PT, so technically you NEED the plain-text first, but don't use the challenge on the plain-text only, you need to at least hash the PT first, then apply the challenge. The cost for doing so is nothing for the authorized, but much effort must double for the hacker.
http://www.experts-exchange.com/Security/Misc/A_12386-How-secure-are-passwords.html I am citing myself.
-rich
0
 
LVL 38

Expert Comment

by:Rich Rumble
Comment Utility
I'll add that no-one should store the PT, for a system to work storing the hash is better than the PT obviously. My article explains why and how some hashes are more expensive for an attacker, and even are setup to combat specialized hardware, the PT makes no sense to STORE, it should only be input when needed, and only to memory, never written to disk or stored. Accept PT as input, then hash it, then use Challenge-Response, against another hash that is stored.
-rich
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
Coincidentally enough, I just found out who Schneier is last night while surfing YouTube. What are the odds?

P.S.

I briefly scanned through your article just now. I think it will make for an interesting read later  = )
0
 
LVL 32

Expert Comment

by:Stefan Hoffmann
Comment Utility
Did I hit someone with my gauntlet?

@richrumble, well phrased answer, also your linked article.
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
Did I hit someone with my gauntlet?
Ha! No, I think it was skullnobrains. My understanding of such authentication was along the same lines:  you store the hash, not the true password. So my first impression of skullnobrain's comment was that it was incorrect. But to give the benefit of the doubt, I wanted to open a thread dedicated solely to discussing that topic (since the original question wasn't concerned with this issue). I wanted to have a friendly discourse on the topic in the hopes that I might learn something myself  = )
0
 
LVL 38

Expert Comment

by:Rich Rumble
Comment Utility
If you know one or the other, the hash and or the plain-text you can still authenticate against challenge response. This is how "pass-the-hash" works, an attacker extracts the hash of the password, and when challenge response prompts, the challenge is put with the hash, and the response is sent, and will be accepted. So even if you don't know the PT pass, if you have the hash, you can still authenticate.
With windows, you pass the pass (see mimikatz, or wce), or use pass-the-hash.
http://en.wikipedia.org/wiki/Pass_the_hash
Both require Admin, well technically you can get hashes by reading HDD's when the OS isn't booted, but ordinarily you use admin/system priv's to get the hashes and or the PT password from memory.
2-factor authentication can thwart both attacks, but in the windows world you don't get to use 2fa when \\ip.ip.ip.ip or using WMI/PowerShell or SMB/CIFS in general.
(See my other articles :)
-rich
0
 
LVL 73

Assisted Solution

by:sdstuber
sdstuber earned 142 total points
Comment Utility
I'm not sure what the question is really asking here.

Taking the bold text verbatim the answer is quite simple: "No".    The previous question and wikipedia links provide example.  

So, it's obviously not "necessary" .  So the more interesting question then is:  "Is it a good idea to do it anyway?"

That seems to be only marginally debatable though.
Given a good hash (say  SHA3-512) it's essentially impossible to extract the password from it.  (by current stats from http://en.wikipedia.org/wiki/SHA-3, still no means of attacking a big SHA2, let a lone SHA3.)

But, encrypting it means you "simply" have to crack the encryption (technically or socially) and you get the password.

So, obviously hashing is better; because it should make the password irretrievable.


But... this question isn't just about hashing vs encryption, it's also about CR.  Which then introduces a wrinkle.

The hash, no matter how good, effectively becomes the new plain-text password.

By this I mean, lets say the host has a hashed version of the password stored.
For simplicity I'll say it hashes to ABCDEF, but in reality it would be a gigantic hex string.

Host:  I challenge you with string "xyz"

Client: ok, I accept,  I hash your "xyz" along with my "ABCDEF" and generate a new hash.  "01234ABC"  (again, the real hash will be much larger) and send that as my response.

Host: Hashes "xyz" along with "ABCDEF" , generates the same value "01234ABC" to confirm the response.


Note - nowhere does the client or the host ever use the password itself.

As alluded to in http:#a40012719 , the shared secret isn't the password, it's the hash.
The original password is no longer relevant.

In fact, you could even view the original password as a virtual attack vector.  Since it allows you to generate the hash, which, within the CR framework,  has become the new de facto password.

So, now, if I can see the hash, it's easy to argue you're back to being in plain-text again.  Since you can easily read the only pertinent piece of information needed to log in.

Thus, you'd probably want to encrypt the hash and store that, but then I'm kind of dodging the question.

Hence my confusion.  What is really being asked?  Is there actually anything to debate here?
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
@sdstuber

OK, point taken. I have modified the question.
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
Gentlemen, unless I'm misunderstanding something, you never "pass the hash". You simply store the hash. End-users always remember their passwords, not the hashes. The user sends his password, and the system generates a hash against the supplied data. If it matches what was previously hashed (when the user originally registered), BINGO!; otherwise, "unauthorized."

Storing the hash allows you to effectively keep the password in your database (I use the term generically) without actually having the password itself. The DBA cannot simply "SELECT password..." nor can an attacker who compromises the system do the same. Further, if a user were to pass the hashed value, then that value would be hashed, which would have an astronomical chance (a snowball's chance in...if you prefer) of hashing to its own value.

This of course assumes 1-way hashing. I believe the game changes when you start talking about 2-way (reversible?) hashes.

The only reason I can think of why you'd possibly want to encrypt the hashes is that on the off-chance that your system is compromised, an attacker would have a list of hashes that they could then spend time and resources on trying to reverse engineer--no small feat, as I understand.
0
 
LVL 38

Assisted Solution

by:Rich Rumble
Rich Rumble earned 216 total points
Comment Utility
Pass the hash is an attack on CR systems, by knowing the "answer" all you have to do is nod your head when the challenge comes in.
Pass= 123456
hash= asdfghjkl
Challenge= zzzz
I know the hash, and so the CR comes, I add zzz to the hash, and authentication happens. If I know the pass, I hash the pass, the CR comes, I add zzz to the hash, and I authenticate. The Hashes are stored in the windows SAM (and in memory), so they can be retrieved with admin access or offline access for that matter.
End users have to remember the pass, so the system can hash it and pass it on, but a hacker doesn't have to know the pass, they only have to know the Hash.
2-way hashing, that isn't a thing :) That is encryption. Asymmetric encryption uses two different keys, on for encryption the other for decryption, the two keys are interchangeable. Symmetric encryption uses the same password to encrypt as it does to decrypt.
Hashing is a mathematical operation that ideally has a very low collision rate, that way every unique input results in unique output. Salted hashing says even the same input should result in different output to a certain degree.

Encryption makes something unreadable, A hash creates a new output from the input, in such a way that can't be reversed, but can be double checked by repeating the operation. Decryption makes the unreadable readable again, in a reversed process.

Take Full-disk-encryption. Drive looks like gibberish when the OS isn't running. Turn the OS on, input the right password, and while it's running the OS looks just like it did before encryption. The key (password) is able to decrypt each section of the HDD as it is accessed. You can't do that with hashing. Same for a DB, encrypt the DB, store anything in it, hashes or what have you. When the DB is closed, looks like jibberish, when it is opened, looks like a DB. My articles all say this better than I am here :)
-rich
0
How to run any project with ease

Manage projects of all sizes how you want. Great for personal to-do lists, project milestones, team priorities and launch plans.
- Combine task lists, docs, spreadsheets, and chat in one
- View and edit from mobile/offline
- Cut down on emails

 
LVL 38

Assisted Solution

by:Rich Rumble
Rich Rumble earned 216 total points
Comment Utility
Here too we have to be careful not to confuse authentication itself with challenge response. While CR is often used as authentication, for SSL it's used for setting up communications. With SSL this is often easier, since the Asymmetric keys are already on your OS or in your browser.
So to authenticate to Facebook, let's say they salt the users passwords:
123456 - ff945ad232b01721becfa5163b53b722$l"m
123456 - 41bd13869aa5a301b9d6420328669769$v9w
123456789 - 6b42a7a5675893363bdeda1ee60407bb$BUN
123456789 - fd51f57e1febe9fde002829c71eb8337$]rq
Same passwords, different salts. FB has to establish an SSL connection so it can get the user's PT pass, add the salt to it, then see if that matches.
You can't authenticate against a salted DB unless you know the PT, but to do it securely you have to establish a secure connection first, and part of that secure connection can be a CR. So maybe that is what skullwhatsitname was talking about.
The other way is to have the client do the hashing after the server sends the salt over, but that is not very secure and gives away the salt which should remain on the server and never be sent.
*edited to add*
Storing the PT pass is not necessary, sending it however might be necessary.
-rich
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
OK, I was looking at "pass the hash" from a web perspective, not from a Windows perspective. The above description puts it into a bit better context. I'll follow up with the articles after work tonight.
0
 
LVL 26

Assisted Solution

by:skullnobrains
skullnobrains earned 142 total points
Comment Utility
hi guys. just to make a couple of things clear, i had no intention of "throwing a gauntlet" at anyone, and i'd actually encryt passwords in many cases.

this thread is getting very long, and you discussed quite thoroughly many of the relevant points.

i have no intention to say that not encrypting passwords is "better", but i'd like to add a few notes : feel free to discuss them


---



regarding the fact that an admin can run SQL queries against a database,

if you're looking at user's security, it clearly represents an additional threat because passwords may be reused for other things.

if you're looking at the application's security, even that will not make a difference. the admin like the sniffer will only need to know the hash to authenticate. once you store the hash, the server 'in fine' can only know about the hash so the pt becomes irrelevant


using encrypted password storages might be more relevant than encrypting each and evry password.


--

there is only a number of possible hashes. although this number is HUGE, the complexity is still less than what long passphrases would produce...


this might be one case where using a reversible (preferrably hard to reverse and using a complicated salt) encryption algorythm might be as good as a hash.

--

although there are good implementations, many of them are quite naive : as an example, using the hex representation of md5s for storage will only make the attacker's task easier in many cases.

let's say you can sniff the network and manage to retrieve a series of challenges and responses, knowing that the password's length and restricting the characters to 16 makes it overall easier to brute force. (not speaking about how easier it might be to break some of them knowing the input chars)

obviously this is easy to overcome  by using binary values rather than hex representations and possibly apply an extra variable-length algorithm.


--

one last note, ragarding custom algorithms. a letter to letter algorythm such as vigenere's is indeed pretty useful if used before applying another, because it makes combinated brute force + dictionary attacks very complicated (ie attacks where you use brute force and check a glossary to see if a word matches the password in order to find a salt, for example)
0
 
LVL 38

Expert Comment

by:Rich Rumble
Comment Utility
@skullnobrains, some things are right, others are wrong. Have a look at my articles regarding hashes and encryption, hex/binary storage|representation of a hash makes no difference in strength/time to crack a hash. Substitution ciphers and codes while difficult to decipher, and not typically cracked using popular software, are often much weaker encryption schemes than what is used today. Some still elude us to be sure (panel 4 of the NSA's sculpture), however they aren't typically useful for computers to use.
Vigenere/Cesar ciphers and others are quite weak and very susceptible to frequency analysis attacks.
If anyone has questions about hashes vs encryption vs ciphers vs codes let me know in another Q and I'll follow up.
To ultimately answer this question:
You need the PT pass to be sent when using a Salted hash, you do not need to store it beyond that point, you can send the hash of a password if the hash is being checked against and unsalted hash like those found in the windows SAM or NTDS.dit files, other hashes too can be unsalted, most are. Another good read for everyone:
http://arstechnica.com/security/2013/05/how-crackers-make-minced-meat-out-of-your-passwords/
http://arstechnica.com/security/2012/08/passwords-under-assault/
http://arstechnica.com/security/2013/03/how-i-became-a-password-cracker/

-rich
0
 
LVL 26

Assisted Solution

by:skullnobrains
skullnobrains earned 142 total points
Comment Utility
hex/binary storage|representation of a hash makes no difference in strength/time to crack a hash

obviously, yes, but you missed the point. trying to crack a secret is one thing. trying to crack a secret that is know to be a 32 digits hexadecimal number (or worse a portion of it) is much easier

Vigenere/Cesar ciphers and others are quite weak and very susceptible to frequency analysis attacks.

obviously as well... which is why they are only useful when applied before a hashing algorythm. if for example you store a number of salted md5, (salted by concatenating the salt with the secret beforehand), applying a vigenere to the secret is useful : if an attacker gains knowlege of a few md5s, it will be much more complicated to brute-force the salt by trying to find sequences that contain a well known password. btw, there are dedicated rainbow tables for such usage.

You need the PT pass to be sent when using a Salted hash

i do not understand the point. what are you talking about ? password authentication in a system ?
0
 
LVL 38

Expert Comment

by:Rich Rumble
Comment Utility
>obviously as well... which is why they are only useful when applied before a hashing algorithm. if for example you store a number of salted md5, (salted by concatenating the salt with the secret beforehand)
Weak is weak, rot-13'ing, shifting, obfuscating again is trivial for a machine to unravel, before after during, security by obscurity we all know doesn't work. I digress.
The point of this thread is however why a PT is needed, specifically to be stored, we've all I think agreed that is not so, the PT does not need to be stored in a CR system. The only time a PT is needed is to check against a hash that is salted on the auth server.
If it's unsalted, then effectively the hash of the PT is the password, and a CR system can re-hash the password hash using the challenge, and compare the result.
With a salted hash, you cannot do a CR against the hash, because the user only knows the PT. The user doesn't get the salt, and shouldn't, so the client can't hash the PT of the user and then apply a challenge because it would be missing the salt. So a salted hash, the client has to send the PT, so the server can apply the salt, and then check, there is no CR checking against a salted hash like there is with non-salted ones.
That does not mean that a CR isn't taking place, but it's not taking place using the password, like SSH in a linux system, the passwords are hashed and salted, ssh first creates encrypted tunnel, and then the PT of the pass is sent to the server for it to add the salt to and then check. The client can't hash it first, because the salt is applied to the plain-text and not the hash. If the salt is applied after the PT is hashed, then you get no benefit from the salt, the salt has to be applied to the PT not a hash of the PT. If you know the hash and you send that to the server, then it applies the salt and the results work, the salt did nothing to shore up the password, it added one operation, and not a very expensive one at that.
</pwd_101>
-rich
0
 
LVL 26

Expert Comment

by:skullnobrains
Comment Utility
Weak is weak, rot-13'ing, shifting, obfuscating again is trivial for a machine to unravel, before after during, security by obscurity we all know doesn't work. I digress.

you are still missing the point, and this is absolutely not related to obfuscation in any way. in the case i mentioned above, knowing that a vigenere is applied will not help the attacker in any way unless if he knows the key. rot13 would be quite foolish indeed. anyway, this is totally beyond the scope of this thread.

The point of this thread is however why a PT is needed, specifically to be stored, we've all I think agreed that is not so, the PT does not need to be stored in a CR system.

agreed, it is not needed to have the PT. it just makes no difference unless the client and server agreed on a shared secret beforehand, since the hash effectively becomes the password.

If it's unsalted, then effectively the hash of the PT is the password, and a CR system can re-hash the password hash using the challenge, and compare the result.

i think i finally get your point. by "salted" you mean that the authentication mechanism will always apply a non-reproductible transformation to the password before checking it against what is stored in the db.

agreed : by definition, this process does not allow challenge-response authentication. we're still stating the obvious

this is precisely why i do not consider that storing "unsalted" hashes rather than passwords makes much of a difference (if at all).

the only way it could not be the case, is if both the client and server side agreed on a shared secret (which you'd probably like to call "salt") beforehand, which is quite beyond the scope of this thread but does allow cr with salted hashes. in that case and that case only is the security improved by storing hashes.

in a linux system, the passwords are hashed and salted

you probably refer to distributions that use shadow-utils (most of them). true but the salts are stored together with the passwords


If the salt is applied after the PT is hashed, then you get no benefit from the salt, the salt has to be applied to the PT not a hash of the PT. If you know the hash and you send that to the server, then it applies the salt and the results work

whatever this means, i do not get it. applying a salt to a hash does not really ring a bell.

btw, ssh does support challengeresponse and multiple other authentication mechanism



it's high time for me to leave this thread

best regards to all
0
 
LVL 74

Author Comment

by:käµfm³d 👽
Comment Utility
All,

I'll close this out this evening when I have more time to review all of the responses.
0
 
LVL 73

Assisted Solution

by:sdstuber
sdstuber earned 142 total points
Comment Utility
Storing the hash allows you to effectively keep the password in your database (I use the term generically) without actually having the password itself. The DBA cannot simply "SELECT password..." nor can an attacker who compromises the system do the same.

With CR, then the DBA CAN simply select the password because the hash has become the password, and it's plain text at that (unless you encrypt the hash.)

At the risk of repeating, I'll try to reexplain my example above.

If I challenge with you with xyz,  and then you can either respond with a hash (xyz + hash(password))

or, if you're the malicious DBA or attacker above, you don't need to know the original password, you can simply respond with  hash(xyz + stolenhash)


Note, neither I nor the dba knows the password, nor is the hash of the password ever sent between us.
The CR communication consists only of the challenge and then a hash of a hash response.
That's the "wrinkle" that CR introduces.

Since the response is a hash that contains the shared secret.  The "secret" must be kept intact on the host.  The fact that the secret was intended to be hash of an unknown password is irrelevant.  Once created, the hash IS the secret.  If the hash is stolen, the thief doesn't ever need to know where it came from.
0
 
LVL 74

Author Closing Comment

by:käµfm³d 👽
Comment Utility
Thank you all for your participation. This was very difficult for me to grade, so I spread the love. I appreciate everyone's comments.
0

Featured Post

What Security Threats Are You Missing?

Enhance your security with threat intelligence from the web. Get trending threat insights on hackers, exploits, and suspicious IP addresses delivered to your inbox with our free Cyber Daily.

Join & Write a Comment

Phishing is at the top of most security top 10 efforts you should be pursuing in 2016 and beyond. If you don't have phishing incorporated into your Security Awareness Program yet, now is the time. Phishers, and the scams they use, are only going to …
Password hashing is better than message digests or encryption, and you should be using it instead of message digests or encryption.  Find out why and how in this article, which supplements the original article on PHP Client Registration, Login, Logo…
Sending a Secure fax is easy with eFax Corporate (http://www.enterprise.efax.com). First, Just open a new email message.  In the To field, type your recipient's fax number @efaxsend.com. You can even send a secure international fax — just include t…
Here's a very brief overview of the methods PRTG Network Monitor (https://www.paessler.com/prtg) offers for monitoring bandwidth, to help you decide which methods you´d like to investigate in more detail.  The methods are covered in more detail in o…

771 members asked questions and received personalized solutions in the past 7 days.

Join the community of 500,000 technology professionals and ask your questions.

Join & Ask a Question

Need Help in Real-Time?

Connect with top rated Experts

15 Experts available now in Live!

Get 1:1 Help Now