Encryption Protocols

How Group Encrypted Messaging Actually Works

May 3, 2026 10 min read Haven Team

Adding a third person to an encrypted conversation seems like it should be simple. It isn't. The cryptographic properties that make 1:1 messaging secure — forward secrecy, post-compromise security, deniability — become significantly harder to preserve as group size grows. This is why group messaging security deserves its own analysis.


When Signal introduced group chats, they faced a problem that doesn't exist in 1:1 messaging: how do you efficiently encrypt a single message for many recipients while preserving strong security guarantees? The naive answer — encrypt the message separately for each recipient — works but scales poorly. A group with 1,000 members would require 1,000 separate encryptions for every message, burning bandwidth and computation.

The more interesting answers involve shared group keys, and that's where the tradeoffs get complicated.

The Naive Approach: Pairwise Encryption

The simplest group messaging implementation is pairwise encryption: the sender establishes a separate secure channel with each group member and sends the message individually to each one. This provides full forward secrecy because each channel uses the Signal Protocol (Double Ratchet), which rotates keys on every message.

Signal actually used this approach for groups in its earlier implementations. The security properties are strong — each pairwise channel is as secure as a 1:1 conversation. But the cost is linear scaling: message overhead grows proportionally with group size. For a group of 100 people, sending one message requires 100 individual encryptions, 100 individual network deliveries, and 100 individual ratchet advances. This is fine for small groups; it becomes impractical at scale.

Sender Keys: Signal's Group Optimization

Signal's current group implementation uses Sender Keys to solve the scaling problem. Each group member generates a Sender Key — a chain key used to derive a sequence of encryption keys for that member's outgoing messages. The Sender Key is distributed to each group member individually (using their established pairwise channels), but after that initial distribution, the sender encrypts each message once with the current key from their chain, and that single ciphertext is delivered to all group members.

This reduces message overhead from O(N) to O(1) for subsequent messages after the key distribution step. The tradeoff is in forward secrecy characteristics: Sender Keys advance forward through a hash chain (like the Symmetric Ratchet in the Signal Protocol), providing forward secrecy against passive observers who later compromise a key. But they don't provide the full Double Ratchet's break-in recovery — if a member's current state is compromised, the attacker can decrypt subsequent messages until the chain is reset.

Key insight

Sender Keys provide forward secrecy (past messages remain protected after key compromise) but weaker post-compromise security (recovery after a breach requires explicit key rotation). This is an explicit tradeoff for group scalability.

The Member Addition and Removal Problem

The hardest problem in secure group messaging isn't encrypting messages — it's managing membership changes securely.

When a new member joins a group, they should not be able to decrypt messages sent before they joined. This property is called forward secrecy for new members (or sometimes "post-join secrecy"). If group messages are encrypted with a shared key that the new member is given upon joining, and that key was used for all past messages, they can decrypt everything. Preventing this requires generating a new group key when a member is added, and ensuring past ciphertexts were encrypted with a different key.

When a member leaves (or is removed), they should not be able to decrypt messages sent after their departure. This property is called post-remove secrecy. Achieving it requires rotating the group key after removal — which is straightforward in principle but expensive in practice, because the new key must be distributed to all remaining members through their individual secure channels. For a group of 1,000 members, removing one member triggers 999 individual key distributions.

In Signal's Sender Keys implementation, removing a member from a group requires each remaining member to distribute a new Sender Key. This is handled by the server delivering the removal event and the clients generating fresh sender key distributions, but the communication overhead grows linearly with group size.

MLS: A Cryptographically Sound Group Protocol

Messaging Layer Security (MLS, RFC 9420) was designed from first principles to solve group messaging security at scale. Rather than Sender Keys' O(N) key distribution on membership change, MLS uses a binary tree structure where group members occupy the leaves and internal nodes hold intermediate keys derived from their children.

This tree structure — called a "ratchet tree" in the MLS specification — allows key updates to propagate efficiently. When a member updates their key material (a routine operation called a "commit"), only the nodes on the path from their leaf to the root need to be updated. In a balanced tree with N members, that's O(log N) operations rather than O(N). For a group of 1,000 members, that's roughly 10 operations rather than 999.

Protocol Message Encryption Cost Member Add/Remove Cost Forward Secrecy Post-Compromise Security
Pairwise (Signal v1) O(N) — one per member O(1) add / O(N) remove Strong Strong
Sender Keys (Signal current) O(1) — shared key O(N) — re-distribute sender keys Forward Weaker
MLS (RFC 9420) O(1) — shared epoch key O(log N) — tree path update Strong Strong

Epochs and Ordering in MLS

MLS introduces the concept of an epoch — a version of the group state. Every time the group membership changes or key material is updated, the group advances to a new epoch. Messages are encrypted under the epoch key valid at the time of sending.

This epoch structure solves several problems simultaneously: it provides a clean boundary for forward secrecy (past epoch keys are deleted after transition), it allows the group to recover from device compromise by rotating to a new epoch, and it provides a cryptographic audit trail of who made what changes to the group at what time.

The challenge with epochs is ordering. In a distributed system with multiple active participants, two members might independently try to advance the epoch at the same time — a conflict called a "concurrent commit." MLS has a defined resolution mechanism for this, but it requires careful implementation and adds complexity that pairwise or Sender Key approaches avoid.

What Applications Are Deploying

As of 2026, MLS adoption in production messaging apps is growing but still limited. WhatsApp announced MLS adoption with its partnership with the IETF standardization process. Apple Messages began using MLS for group chats. Matrix has an MLS implementation in development for its federated protocol.

Haven uses MLS for its encrypted group chat, specifically because of the log(N) membership change overhead and the strong post-compromise security guarantees. For the user base Haven targets — security-conscious individuals who care about forward secrecy and recovery from device compromise — the stronger security properties justify the additional implementation complexity relative to Sender Keys.

Signal continues using its Sender Keys approach for large groups, with the rationale that the engineering complexity of MLS is not yet justified by the security improvement for Signal's threat model and scale. Both positions are defensible; the right choice depends on group size distributions, deployment constraints, and the adversary models the service is designed to resist.

What This Means for Users

The practical takeaway isn't that one app is universally better — it's that group messaging security has real complexity under the hood, and "end-to-end encrypted" doesn't tell you which group protocol was used or what its security properties actually are.

Questions worth asking about any secure group messaging app:

These questions have clear technical answers in well-designed protocols. If an app can't answer them, that's a signal worth heeding. Forward secrecy in 1:1 contexts is well-understood; in group contexts, it requires explicit design choices that vary significantly between implementations.

Try Haven free for 15 days

Encrypted email and chat in one app. No credit card required.

Get Started →