Secure API Design Case Study: Have I been Pawned

Secure API design is a critical aspect of security design reviews. However, finding good examples or case studies to demonstrate this in practice can be challenging. Recently, I came across the "Have I Been Pwned" (HIBP) API and realized it would make an excellent candidate for a secure API design case study.

The Have I been Pawned (HIBP) website and API allow users to check whether their passwords have been involved in previous data breaches. The API is frequently used by password managers to check if a password has been compromised

Insecure Design

The simplest but least secure and privacy-focused way to design this API call is to send the parameter directly as a query in a GET request. However, as you can imagine, this approach would expose your password to a third-party website.

Another insecure yet easy-to-implement solution is to send the first few characters of the password as a query parameter in a GET request. However, similar to the example above, this approach would expose parts of the password to a third-party website and significantly reduce the password's entropy.

The HIBP website allows users to enter their passwords in the UI to check whether they have been compromised. However, the API incorporates several interesting mitigations that significantly enhance its security and privacy.

Secure API Design

Hashing

As outlined in the HIBP documentation, you can send the first five characters of the SHA-1 or NTLM hash of a password to the API. The API then returns all matching password hashes that begin with those five characters. On average, the response includes approximately 800 password hashes for such a query.

The SHA-1 hashing algorithm generates a 20-byte hash value, which is typically represented as 40 hexadecimal digits. By disclosing the first five characters of the hash, there will be a reduction in the entropy. However, the remaining entropy is still 16 ^ 35 (where 16 represents the characters 0–9 and A–F in hexadecimal, and 35 is the length of the remaining hash). The amount of entropy here is significantly higher than disclosing five characters from a typical 10-character password.

This design assumes that with first 5 characters of the SHA-1 hash the maintainer or an attacker can’t predict the rest of 35 characters of the password. But the main mitigation is the pre-image resistance (one way hash function ) property of the SHA-1 hash function as described in this Stack Overflow post.

Note

SHA-1 is not a recommended cryptographic hash function in 2024. We will discuss this more in detail in next section.

Padding

An active attacker with the ability to monitor encrypted traffic blobs could analyze the response length to make educated guesses about the first five characters of the password hash. To mitigate this risk, the API offers an option to enable padding. This padding ensures that an attacker monitoring response sizes cannot infer the input, as there is no observable consistency in the responses. However, as expected, enabling padding significantly increases the size of the API response.

Security Concerns with this design

Although the design utilized by HIBP improves the security and privacy of user data significantly, there are couple of minor concerns with the design.

Use of SHA-1 to hash passwords

As we know, SHA-1 is considered insecure for many purposes, particularly for storing user passwords. However, the threat model in this case is quite different. Users expect that their actual passwords will never be stored or logged within the HIBP infrastructure. Additionally, the author has written an excellent blog post explaining his reasoning for using the SHA-1 hash function.

Passing sensitive parameters through a query string

As we know, passing sensitive information through a query string is not recommended. The HIBP API sends the first five characters of the password hash as a query string in the URL. However, an attacker with this level of access could likely perform other, potentially more sensitive actions. To mitigate this risk, an organization can download the password data and host it under their own control, which could help address the issue.

Conclusion

The HIBP API improves the security and privacy of the user password significantly by sending the first few characters of the password hash.