MTA-STS and TLS-RPT: Forcing Encrypted Email Delivery

When your mail server sends a message to another domain, it looks up the recipient's MX record, opens a TCP connection on port 25, and starts speaking SMTP. If both sides advertise the STARTTLS extension, the connection is upgraded to TLS before the message body is transferred. If either side doesn't, the message is delivered in cleartext over the public internet.

That "if both sides advertise it" clause is the problem. STARTTLS is what RFC parlance calls opportunistic encryption: the sender will use TLS if available, but accepts cleartext as a fallback. An attacker who can modify SMTP traffic — a hostile ISP, a state-level adversary, a misconfigured network appliance — can strip the STARTTLS capability from the server's banner. The sender, seeing no offer of encryption, sends the message in the clear, and neither end knows anything went wrong.

The Downgrade Attack, Concretely

Here's what an active stripping attack looks like:

1. Your mail server connects to mx.example.com on port 25.
2. The recipient's server normally responds with a banner including 250-STARTTLS.
3. An attacker on the path rewrites that line to 250-XSTART (any unrecognized keyword).
4. Your mail server, seeing no STARTTLS capability, proceeds in cleartext.
5. The attacker passes the cleartext SMTP session through, reading every byte.

Research from Google's transparency team documented this attack happening in the wild as early as 2014, particularly affecting outbound mail from countries like Tunisia and Iraq, where ISPs were observed performing exactly this rewrite. The attack requires no key compromise, no cryptographic break — only the ability to modify TCP-level traffic on the SMTP path.

What MTA-STS Adds

MTA-STS (RFC 8461) is a policy mechanism that lets a receiving domain declare: any sender talking to my MX servers must use TLS, must validate my certificate, and must refuse to send if either fails.

The mechanism has three pieces:

1. A DNS TXT record at _mta-sts.example.com that contains a policy version ID.
2. A policy file served over HTTPS at https://mta-sts.example.com/.well-known/mta-sts.txt.
3. A sender that supports MTA-STS, which fetches and caches the policy.

The policy file looks like this:

version: STSv1
mode: enforce
mx: mx1.example.com
mx: mx2.example.com
max_age: 604800

Mode is the lever. testing means "log violations but don't refuse delivery"; enforce means "if TLS or certificate validation fails, refuse to deliver and queue for retry"; none means "policy is currently disabled" (used during decommissioning).

Once an MTA-STS-aware sender has fetched and cached your policy, it will not deliver to your domain over cleartext SMTP — even if a network attacker strips STARTTLS, because the sender knows from cache that your policy requires TLS.

TLS-RPT: The Reporting Channel

MTA-STS tells senders what to require. TLS-RPT (RFC 8460) gives you visibility into what actually happened. It's a sibling spec, structurally similar to DMARC reporting.

You publish a DNS TXT record at _smtp._tls.example.com:

v=TLSRPTv1; rua=mailto:tls-reports@example.com

Senders that support TLS-RPT (Google, Microsoft, and several large MTAs do) will email you a daily JSON report summarizing TLS connection outcomes to your domain — successful handshakes, certificate validation failures, STARTTLS downgrades, MTA-STS policy fetch failures, the works. Without TLS-RPT, the first you'll typically know about a misconfigured certificate is a user complaint two weeks later.

How These Compare to Existing Email Auth

Mechanism	What it protects	Threat addressed
SPF	Sending IP authorization	Forged envelope sender
DKIM	Message integrity (signed)	Tampering in transit
DMARC	SPF/DKIM alignment	Header-from spoofing
MTA-STS	Transport encryption requirement	STARTTLS stripping
TLS-RPT	Visibility into TLS failures	Silent downgrade detection
DANE/TLSA	Cert binding via DNSSEC	CA compromise, STARTTLS strip

MTA-STS and DANE are alternative ways to solve the same downgrade problem. DANE binds the certificate hash directly in DNS using DNSSEC; MTA-STS uses a separate HTTPS-fetched policy. They can coexist. In practice MTA-STS has seen wider adoption because it doesn't require DNSSEC on the recipient domain — and global DNSSEC deployment has stalled below 30% for years.

What This Doesn't Do

MTA-STS protects only inbound transport to your domain. It doesn't:

• Encrypt message content end-to-end — for that you need PGP or similar
• Protect outbound mail from your users to others (that depends on the recipient's policy)
• Help non-supporting senders — Gmail, Outlook, and Yahoo support it; many small servers don't
• Stop a sufficiently motivated attacker who can also poison your HTTPS policy fetch (mitigated but not eliminated by caching)

It's a transport hardening layer. It closes one specific, real attack — silent STARTTLS stripping — that authentication mechanisms (SPF, DKIM, DMARC) don't address at all.

Should You Deploy It?

If you operate an email-receiving domain and care about senders not being silently downgraded when delivering to you: yes. Deployment is mechanical:

1. Issue a valid TLS certificate covering all your MX hostnames.
2. Stand up mta-sts.example.com as an HTTPS endpoint serving your policy file.
3. Add the DNS TXT records for both MTA-STS and TLS-RPT.
4. Start in mode: testing for two weeks, monitor TLS-RPT reports for unexpected failures, then flip to mode: enforce.

Tools like checktls.com and internet.nl can validate the deployment from the outside. Microsoft Defender for Office 365 and Google Workspace both honor MTA-STS policies as senders, so a correctly deployed policy meaningfully changes the security profile of inbound mail from those providers' users — which is most of the internet's mail volume.

For a broader view of email auth as a stack, see our overview of SPF, DKIM, and DMARC.