challenge-response authentication

In information security, challenge-response authentication is a type of authentication protocol where one entity presents a challenge or question, and another entity provides a valid response to be authenticated. Challenge-response authentication is a type of authentication method used to prove the identity of a user or other entity  requesting access to a computer, network or other network resource. This method uses previously answered challenge questions to authenticate the user; simple user ID and password authentication is the most commonly implemented type of challenge-response system.

There are different approaches to authentication using challenge-response systems, but modern challenge-response authentication methods typically incorporate one or more cryptographic protocols to prove the user being authenticated knows a password without the need to share the password itself.

In challenge-response authentication, the client application initially obtains a random challenge -- usually data of some type -- from the server. For password-based challenge-response systems, the client system computes a response by applying a cryptographic hash function to the challenge from the server combined with the user's password. Then, the application sends the response, as well as the original challenge, back to the server.

When the server receives the response, it applies the same hashing function to the challenge data combined with its own copy of the user's password. If the resulting value and the response sent by the application match, there is a very high degree of probability the user has submitted the correct password.

While challenge-response authentication systems generally rely on cryptography to securely authenticate users, CAPTCHA (the Completely Automated Public Turing Test to Tell Computers and Humans Apart) challenge-response system is an example of a noncryptographic challenge-response protocol designed to differentiate humans from robotic software programs. CAPTCHA is used to prevent spam and auto-registration of new accounts for email or websites.

How challenge-response authentication works

When a user attempts to log onto  a system or network resource, the challenge-response system server generates a challenge, usually a random number, that is then sent to the client machine.

The client software uses a secret key, or a key based on its password, to encrypt the challenge data using an encryption algorithm or one-way hash function. It then sends the result back to the network server.

The challenge-response authentication system performs the same cryptographic process on the challenge, comparing its result to the response from the client. If the two values match, the authentication system is able to authenticate the client.

There are two types of challenge questions: static and dynamic.

Static questions enable the user to choose from a bank of predefined questions or allow the user to create custom challenge questions. The user then provides the answers to the challenge questions he has selected. For example, a static challenge might be to provide the name of the user's first pet, first car or first-grade teacher -- the correct values will not change over time, and the user can specify the correct values as part of their account setup.

Dynamic questions are created by extracting public data about the user that the individual should know, such as a previous street address or the make and model of a previous vehicle. The system presents the user with random questions and answers that utilize this data from which the user must select the correct answer.

Challenge-response authentication can defend against session replay attacks, in which an attacker listens to previous messages and resends them later to get the same credentials as the original message. Challenge-response systems defend against replay attacks, because each challenge and response is unique. An attacker monitoring credentials exchanges and then attempting to reuse credentials will not succeed in gaining access.

Some types of challenge-response systems can help defend against man-in-the-middle attacks, particularly when the challenge and response requires some knowledge to which the attacker does not have access. For example, challenge and response values that are digitally signed by an endpoint using a private key, or that depend on any other data that has not been compromised by an attacker, should protect the endpoint from a man-in-the-middle attack.

Uses for challenge-response authentication

Organizations typically use challenge-response authentication systems to enable users to reset their own passwords, as well as for emergency access, enabling users to resolve their problems and get to work more quickly, reducing the burden on their help desks and saving money.

Learn how the next generation of authentication may kill the password.

Challenge-response authentication helps when users forget or lock out their password and can't log into the network. With a challenge-response system in place, users in that situation can identify themselves to the system by providing a valid username or email, authenticate themselves by answering a series of personal or challenge questions and then select a new password.

Challenge-response questions can also be used as part of a risk-based authentication mechanism, where users must answer challenge questions in the event a risk threshold is met and additional verification is needed. Organizations such as card issuers, retailers and order processers, as well as financial services providers, can customize their exposure to risk -- and enhance customer satisfaction -- by requiring challenge-response authentication only for their consumers who are most at risk.

Best practices for challenge-response authentication

When implementing challenge-response authentication systems, an organization should follow some best practices to use them securely. Some guidelines include the following:

  • Implement challenge-response authentication systems with an automated enrollment system to invite users to complete their challenge-response profiles.
  • Avoid questions that have answers that can be easily researched or guessed.
  • Try to use questions that have answers a user can easily remember.
  • Use standardized and user-selected questions, because user-generated questions are difficult to guess, so hackers will be less likely to discover the answers.

When designing and deploying an interactive challenge-response authentication system, best practices include guidelines on how challenges should be issued and how the system should respond. Some practices include the following:

  • Ask standard questions first, and then ask user-generated questions only after the standard ones have been answered correctly.
  • Avoid asking the user to respond to more than six questions during the authentication process to avoid alienating users who may become impatient and fail to enroll in the system.
  • Do not display answers to challenge questions, as doing so can expose them to a malicious eavesdropper.
  • Encrypt all data that may be used for authentication, particularly a user's answers to personal challenge questions.
  • Lock user accounts after a specified number of failed authentication attempts.

Examples of challenge-response authentication systems

In addition to ordinary user ID and password systems and CAPTCHA systems used for challenge-response authentication, challenge-response authentication systems can be classified by the cryptographic algorithms and techniques they use for securing the authentication process.

Some examples of challenge-response authentication systems include the following:

  • The secure shell protocol includes a challenge-response mechanism that uses a separate public key pair for each direction of a communication session. A local client authenticates itself to a remote server by sending the server a value signed with the server's public key. The remote server authenticates itself to the client with a value signed with the client's public key.
  • Zero-knowledge password proof systems depend on cryptographic methods that enable both parties -- client and server -- to authenticate to each other. These systems allow each party to confirm to each other that they have a correct password, but without the need to share that password with each other.
  • Challenge-Handshake Authentication Protocol (CHAP), specified in IETF RFC 1994, defines an authentication protocol that uses a three-way handshake. The authenticating system sends a challenge message to the system being authenticated. The next step is for that system to perform a one-way hash on the challenge and return the hash value to the authenticating system. The authenticating system can then allow the session to continue if the hashed values match, or terminate the connection if they don't.
  • The OATH Challenge-Response Algorithm (OCRA) is another challenge-response authentication method, defined in IETF RFC 6287. It uses a challenge-response algorithm developed by the Initiative for Open Authentication (OATH) for a cryptographically strong challenge-response authentication.
This was last updated in October 2018

Continue Reading About challenge-response authentication

Dig Deeper on Security Resources