The art of storing passwords

Published by Evil Bee Friday, 04 July 2008 22:55

Almost every website nowadays needs to maintain a list of users and passwords. Many multi-user applications require a way to authenticate users, and passwords seem like a natural.

You don’t necessarily have to provide your own username/password authentication solution, but if you want to understand the logic behind storing passwords and understand how to correctly implement a password management scheme, this article is for you.

This article assumes a web application – slightly different rules apply to a distributed multi-user application.

Simplest way - store raw passwords

The simplest approach to manage user names and passwords is to store everything in plaintext (no encryption or scrambling) in a file or database. The result would be something like this:

User id	User name	Password
51	banchi	wendolin
52	barza	anita28
53	amaury	amaury
54	teddy	221084

Advantages of storing raw passwords

Authenticating (checking that the username and password pair matches the pair in the table) is very simple – just compare the strings!
Forgotten passwords can be retrieved – the password is easily accessible, given the user name.

Disadvantages of storing passwords ‘in the clear’

First, anyone with access to the file (or able to SELECT from the table) gains immediate access to all passwords! An employee with legitimate access to the file might print the file or email out the information, and Voila! all the passwords are compromised.
The second problem is that during the authentication exchange, the password is visible on the network. Unless secure communication is used throughout, the password can be seen while traveling on the network. For example, even if the web application uses SSL to submit the password, the password is still visible when the web application server SELECTs the information from the remote database. The results of the query are transferred unencrypted over the network.

Is storing password ‘in the clear’ an acceptable solution?

If your data store is encrypted (using SQL Server 2005, for example),
and your internal communication network is highly secure (uses only IP-SEC or a VPN tunnel for communication between servers),
and you only use secure communication between the application client and the application server,
and you trust all employees with database access to never make mistakes (such as printing the password information or storing to file),
and you are sure nobody else has physical access to any of the servers used.

Then, yes, storing passwords ‘in the clear’ is fine.

Encrypting passwords – a little more secure

A better approach for storing passwords (and the only viable alternative if users need to be able to recover passwords) is scrambling the passwords before storing them.

This approach relies on having a secret. The secret is either the scrambling algorithm, or the key used in conjunction with a modern encryption algorithm.

Scrambling (or encrypting) passwords is a reversible operation. The secret is used to garble the password, and the same secret can be used to retrieve the original password. When a user supplies a password, the stored password is de-scrambled using the secret, and the passwords are compared. An alternative approach is to scramble the provided password using the secret and compare the two garbled versions – a match indicates the provided password was good.

If a user needs to retrieve a password, the stored password is de-scrambled and provided to the user (usually via email).

User id	User name	Password
51	banchi	k463dD8F
52	barza	56lkV#p7
53	amaury	8Fk4lVQ0
54	teddy	k468dD8F

Advantages of scrambling using a secret

A forgotten or lost password can be retrieved.
Only a single secret needs to be stored securely (either an algorithm or a key).
For multi-user distributed applications, when using encryption, either the clear text password needs to be communicated (to be checked), or the secret must be communicated to perform the authentication on the front end.

Disadvantages of scrambling passwords

If the secret is compromised, all passwords are compromised. If somebody has access to the secret and to the password store, all passwords can be unscrambled!
Access to the password store alone is sufficient to provide information about the passwords. Since all passwords are scrambled using the same algorithm, if two users share the same scrambled password, they must have the same password as well. Crafty hackers with access to the password store can create users with known passwords and check for other users with the same password. This type of attack is a variation of the ‘known plaintext’ attack. Attacks such as this can be stopped using a SALT (see below).
When using a block cipher, the password length must be stored as part of the scrambled password. The length must be stored because block ciphers always produce a fixed-size block of scrambled text. If the password length is not scrambled (for example, if stored as a column in a table), the information is very useful to password crackers. Knowing the exact length of the password makes it much easier to try to guess a password.

Is this an acceptable password storing solution?

If lost password retrieval is a must, then yes, this is the only acceptable solution. A few guidelines though:

Store your secret in a secure place. Hard-coding the secret in the application code is not a good solution. In my opinion, storing the secret in a file (even if it is the web.config file) is a terrible idea. If you must use a secret, store it in a database (limiting access by requiring an authenticated connection) or in an access-restricted registry key.
Use a good cryptographic algorithm (examples below), don’t create your own or use a trivial scrambling algorithm. Unless you know exactly what you are doing, your own algorithm might be very easy to crack.
Use a SALT (see below) to prevent two users with the same password from having the same scrambled password.
Encryption output is binary, and must be encoded if stored as text. If storing binary data is OK, no encoding is necessary, but if the encrypted passwords are to be stored as text, consider using RADIX64 to convert binary information to text. RADIX64 uses a character for every 6 bits, expanding the output by 33%.

Storing hashed passwords – the one-way solution

A cryptographic hash is also known as a ‘one-way function’. A hash function takes input of any length, and produces a unique output of constant length. For example, if we hash a password (of any length) with the MD5 cryptographic hash, the result would be a 128 bit number that uniquely corresponds to the password. Cryptographic hashes work on more than passwords – if the cryptographic hash of two files is identical, then the two files are identical as well.

In recent years, as computing power increased, some cryptographic hash functions are no longer recommended for use (MD4, MD5, SHA1). In my opinion, if used just for hashing passwords, you are probably OK. As for me, I modified my code to use SHA2.

When storing hashed passwords, the password is hashed (run through the hashing algorithm) and the resulting hash is stored instead of the password. To compare passwords, just hash the given password using the same hash function and compare the results. If the hashes are the same, the passwords match.

The beauty of a one way function is that there is no way to compute the password based on the hash. Hashed passwords are not immune to brute force attacks – given a dictionary and the password hash, a hacker can compute the hashes of all the words in the dictionary, compare the words with the password hash, and discover the password. This is where strong passwords (containing letters, numbers, and special characters) help us defend against brute-force attacks.

Advantages of storing hashed passwords

The original ‘clear text’ password is never stored. Even if the password store is compromised, only the hashes become public.
The password length is not stored and cannot be estimated, making password cracking that much harder.
There is no need for a secret as none is used to hash the password.
For multi-user distributed applications, the password hash can be used for authentication. When using encryption, either the clear text password needs to be communicated (to be checked), or the secret must be communicated to perform the authentication on the front end.

Disadvantages of storing hashed passwords

Lost passwords cannot be recovered (except using brute-force methods). A new password must be created and communicated to the user.
Like encrypted passwords, if a SALT (see below) is not used, users with the same password will have the same password hash.

Is this an acceptable password storing solution?

Yes, but please follow these guidelines:

Use a good cryptographic hash. MD5 is no longer recommended, and neither is SHA1. SHA2 is the current favorite. If you must use MD5 or SHA1, use a stronger salting algorithm.
Use a SALT (see below) to prevent two users with the same password from having the same scrambled password.
Hash output is binary, and must be encoded if stored as text. If storing binary data is OK, no encoding is necessary, but if the hashed passwords are to be stored as text, consider using RADIX64 to convert binary information to text. RADIX64 uses a character for every 6 bits, expanding the output by 33%.

Salting – everything tastes better with a little salt

Hashing (or encrypting) the same password for two users results in the same output. The repeated information can be used maliciously to obtain passwords. If I am a user with access to the password store, I can set my own password to a dictionary word, and then scan the password store for somebody else with the same hashed or encrypted password. If I find a match, I cracked that user’s password.

To prevent the above problem, we can inject some variation into the hashing (or encrypting scheme). For example: if we prefix the password with the user name, the hash or encrypted output will no longer match.

Using a random salt significantly improves the strength of encrypting passwords, and makes brute-force cracking much more costly.

To use a random salt, compute a random number, and use the random number as a component when calculating the hash or when encrypting. Store the random number in a database column so the number can be available later when checking the password.

Conclusion

The simple guidelines are:

If you need to retrieve passwords, use encryption.
If you do not need to retrieve passwords, use hashes (more secure).
Whatever you do, salt the passwords.

articles

whoisonline