Rawpixel - Fotolia

What are the most important email security protocols?

Internet email was designed independent of security considerations, but these are the top email security protocols that add mechanisms to keep messaging safe from threats.

Not long after the first internet Email protocols were developed, computer scientist Andrew S. Tanenbaum wrote, "The nice thing about standards is that you have so many to choose from."

He wasn't wrong. Although the original internet application protocols rarely addressed security, there are now many choices for email security protocols.

Basic, unsecure email depends on just a handful of protocols:

  • The Simple Mail Transfer Protocol (SMTP) specifies how messages are transmitted.
  • The Internet Message Format, or Request For Comments 5322, and Multipurpose Internet Mail Extension (MIME) specifications determine how messages are to be formatted.
  • The Internet Message Access Protocol, or IMAP4, and the Post Office Protocol, or POP3, specify how email clients retrieve messages from SMTP servers.

With the dominance of webmail, however, email exchanges between servers and users are now almost universally accomplished using web browsers. Most of those exchanges can be encrypted while they are in transit using the HTTPS protocol.

Securing email requires more than just encrypting data in motion, however. Protocols for encrypting email messages are just the beginning. Email security encompasses the exchange of messages and other data between email servers, as well as providing mechanisms for preventing email domain spoofing, authenticating that messages have been sent from valid domains and encrypting transactions between email servers that are forwarding email for delivery.

Message encryption

Security protocols defeat spoofing
Worried about email spoofing? There are email security protocols for that.

Encrypting transmissions between clients and servers provides some degree of privacy as the message is in transit. However, the only way to be sure that the body of an email message remains private until it is decrypted by its intended recipient is to encrypt it. Encryption protocols include the following:

  • HTTPS uses Transport Layer Security (TLS) to encrypt streams of network traffic between clients and servers. It is not invoked directly in email, but is used for web traffic and thus is used to encrypt webmail messages. SMTP Secure (SMTPS) works like HTTPS for SMTP, and uses TLS to encrypt message exchanges between clients and servers. However, encrypted TLS traffic is decrypted at its destination, so cleartext messages may be accessible on email servers as messages are routed unless some other encryption protocol like STARTTLS is in use.
  • STARTTLS is a service extension for SMTP that supports opportunistic encryption between mail servers and clients. When the STARTTLS extension is in use, communicating mail systems negotiate the use of encryption and authentication algorithms to protect exchanges. All message content, as well as message metadata, can be encrypted. However, once the transmissions are received, the data will be decrypted.
  • S/MIME (Secure/MIME) is the standard that defines how to encrypt and authenticate MIME-formatted data. While S/MIME content can be encrypted, the email headers are not, so an attacker would be able to see who is sending the message and who the intended recipient is.
  • OpenPGP is a standard for encryption and authentication of data, including email messages, based on the Pretty Good Privacy framework. OpenPGP is interoperable with S/MIME, and while data can be protected, the metadata around encrypted messages is not.

The primary use of these email security protocols is to encrypt messages and prevent attackers from gaining access to the information in the messages, but there is more to email security than encryption.

Email infrastructure security protocols

One of the most important email security challenges is keeping spam, phishing attempts and other malicious email out of users' inboxes. These email security protocols support efforts to reduce spam by using domain authentication:

  • The SMTP Mail Transfer Agent Strict Transport Security protocol helps secure the email environment by enabling SMTP servers to add encryption via TLS. It also gives enterprises a mechanism to enable servers to refuse to connect with other servers that do not offer TLS connections with a trusted certificate. By requiring a trusted certificate and rejecting connections from unauthenticated servers, email providers can prevent attackers from using fraudulent domains to send phishing or spam email.
  • The Sender Policy Framework (SPF) provides a protocol that enables domain owners to identify which hosts are authorized to use their domain names when sending email and defines how that authorization can be verified. It provides a way for domain owners to announce which IP addresses are authorized to send email on behalf of the domain. It also reduces the likelihood that spam or phishing emails can be sent with that domain spoofed as the source of the messages, though SPF is usually enabled with additional email security protocols that provide stronger assurances that email originated from the proper domain.
  • DomainKeys Identified Mail (DKIM) builds on the SPF and enables the entity that owns the signing domain to link itself with a digital signature that authenticates that entity.
  • Domain-Based Message Authentication, Reporting & Conformance (DMARC) provides mechanisms for notification and mandating actions when messages fail authentication under SPF and DKIM. While SPF and DKIM can flag messages as being spoofed, DMARC enables domain owners to advertise what actions should be taken when spoofed addresses are detected and for recipients to determine the appropriate response action.

There have been many other email security protocols proposed over the years. For example, the DNS-Based Authentication of Named Entities (DANE) protocol was intended to enable authentication of domains by applications like email. However, the internet at large has been very slow to deploy DANE, along with the DNS Security Extensions -- a protocol for signing DNS records -- that DANE relies on.

Commercial or proprietary approaches may address some email security challenges. In fact, much of the DKIM protocol originated at Yahoo Mail as DomainKeys just as TLS is based on the proprietary SSL protocol developed by Netscape to enable encrypted e-commerce. History shows that when a good proprietary security protocol gains traction in the market, it will soon become an open standard.

This was last published in June 2019

Dig Deeper on Security audit, compliance and standards