Part of Stem. This page is the high-level description of how peers synchronise resources: what a conversation is about, who may talk to whom, what moves, in what order, and what is written down about it. The exact messages are in the Sync RPC specification, and the shipping protocol it augments is described in Network.
Principles
Sync is selective and governed by explicit policy. Users and applications choose which resources to follow, which peers to exchange data with, what to fetch on demand, and what to keep available offline. Permission to read something does not by itself trigger replication, and sharing one resource does not require sharing a whole space. Policies are inspectable, with clear scope and predictable behaviour.
From that commitment the protocol takes six rules.
Reading and syncing are one permission until there is encryption. A peer receives exactly the blobs whose readers include a principal it has proven to hold, plus everything readable by everyone. The sync access level exists so a future encrypted layer can let a site hold what it cannot read; today it is granted only to declared sites.
Peers pull. A push is an offer, and an offer carries proof. Nothing enters a peer's store because a stranger sent it. A peer accepts an Offer only for a scope its policy wants, from a sender that can show authority for that scope, and then it fetches and validates each blob itself.
No fan-out to strangers. A peer talks to the authority peers of a space and to its trusted peers, and to peers its policy names. It never samples random peers for content, and it never sends content to a peer that did not ask for the scope.
Authority arrives first. Blobs are served so that the Grants and Groups that authorise a blob arrive before the blob, the Node before its state, and a dependency before what depends on it. A receiving peer can therefore validate as it goes instead of stashing most of what it receives.
Uniform denial. A blob a peer may not read is reported exactly as a blob the server does not hold, on every surface. Fingerprints are computed over the filtered set, so a hidden blob never changes a fingerprint the requester sees.
Everything is logged. Every blob served is a disclosure with a basis. Every batch received is a transfer with a claimed scope and per-blob verdicts. This is the privacy bookkeeping described in Privacy.
Peers and identity
A peer is a daemon reachable over libp2p. Its connection is secured by a device key, which never signs content. An account is an Ed25519 key that signs blobs (Identity). The two are bound only for the life of a connection, by Authenticate: the caller signs the tuple of its account, its own peer id, the server's peer id and a fresh timestamp. A connection may be bound to several accounts. After that, every method on the connection is evaluated against the union of those accounts' grants. An unauthenticated connection is evaluated as the audience everyone.
Three sets of peers matter for a space.
Authority peers hold write, admin or sync on the space root in the authority graph, and are reachable: the owner's devices, collaborators' devices, and the declared site. They are where a peer syncs a space from first and where it offers new blobs.
Trusted peers are the peers of accounts the local account has a relationship with: accounts it has joined (a contact with join), and accounts that hold a grant from it. They may be asked for scopes that the authority peers did not fully satisfy.
The site is a peer that publishes the space on the web. In Stem a site is a signed relationship, defined in Site: the space root's attributes name site{url, peer}, and the owner has issued a sync grant to that peer's account. A URL alone grants nothing. A site peer authenticates like any other peer and receives private content on the basis site.
Scopes and scope sets
Every conversation is about a scope: a space, a node in it (the space root when omitted), a depth (exact, children or subtree) and a set of facets. The facets are state (Node, Change and Snapshot blobs of the resources in scope and their dependencies), files (media the state embeds), comments (comments targeting the resources in scope), authority (Grants, Revocations and Groups that bear on them) and profile (the space root's state only). Omitting facets means everything except profile.
The scope set is the set of blobs a peer holds for a scope, filtered to what the requesting connection may read. Both peers derive it from their own index with the same rule, so they can compare it without listing it. Because the rule is shared, a change to it is a protocol change and takes a new protocol id.
Policy
Standing interest is data, not a hidden scheduler table. Each account has a policy resource in its own space whose Snapshot value is a list of policy rules. Each rule names a scope and one of four modes.
mode | the peer does |
|---|---|
| reconciles the scope periodically, watches it while connected, keeps what arrives, and may garbage-collect history the kind does not require |
| follows, and never garbage-collects anything in the scope; the offline guarantee |
| fetches only when a client asks for the scope; may garbage-collect it afterwards |
| never fetches the scope and refuses offers for it, whatever else would apply |
A rule may name extra peers to exchange the scope with and an interval. Rules are evaluated most specific scope first, so ignore on one child can sit under follow on its parent.
Defaults, written into the policy when an account is created so they are visible and editable: every space the account owns is pin at its root, subtree; every space the account has joined is follow at its root, subtree; everything else is on-demand. A client that shows a resource calls Sync, which brings it up to date now and keeps it hot for a short while, without touching the policy.
The policy resource has access own and one grant: audience key of the account itself. Its readers are therefore the account's own devices, and they receive each other's policy edits through ordinary sync of the account's space. SetPolicy publishes a new Snapshot; GetPolicy reads the merged policy in force.
A sync run
A run brings one scope up to date. Runs are started by the policy's intervals, by Watch notifications, and by clients calling Sync. One run proceeds in seven steps.
Choose peers. The authority peers of the scope's space that are reachable, then the trusted peers, then any peers the matching policy rule names. A peer that failed to dial recently is skipped for a backoff period.
Authenticate. For each peer, bind the connection to every local account that has any grant on the space. Connections to a peer are reused across runs.
Reconcile. Run Reconcile rounds until the ranges agree. The result is the list of CIDs the peer holds for the scope, readable by us, that we lack.
Fetch, authority first. Call Fetch for those CIDs. The server returns Groups, Grants and Revocations, then Nodes, then Snapshots and Changes with dependencies before dependents, then files.
Validate and index. Every blob goes through the runtime model: hash check, signature, kind rules, authority against the claimed scope. A blob whose dependency or authority has still not arrived is stashed, as today, and applied when it does.
Record the transfer. One transfer row per peer per run: who, under which scope, how many received, indexed, stashed, rejected.
Watch while hot. While a client is looking at the scope, or the rule says follow and the peer is connected, keep a Watch open so new blobs arrive without another reconciliation.
sequenceDiagram
participant A as Peer A (follower)
participant B as Peer B (authority)
A->>B: Authenticate(account, ts, sig)
B-->>A: expires
loop until ranges agree
A->>B: Reconcile(scope, ranges)
B-->>A: ranges (skip / fingerprint / list)
end
A->>B: Fetch(scope, cids)
B-->>A: blobs in authority-first order, missing
Note over A: validate, index, stash what still lacks deps; record transfer
Note over B: record one disclosure per blob served
A->>B: Watch(scope)
B-->>A: notification(cids) on every indexing commitOffers
Publishing is local first. A client, the CLI or the SDK calls Publish on its own daemon with the signed blobs and the scope they belong to. The daemon runs them through the same pipeline as a transfer, then sends an Offer to the scope's authority peers, naming the scope, the CIDs and a proof grant when the signer is not the space owner. The receiving peer applies one rule.
An offer is accepted when the receiver's policy wants the scope (its mode is not ignore, and for a peer that is not an authority of the space, is follow or pin), and the sender either is the space owner or presents a proof that gives it write on the scope's node. The receiver answers with the CIDs it wants, then fetches them from the sender and validates each one into the claimed scope: a blob that does not belong to the scope, or whose signer lacks authority there, is rejected and counted in the transfer. A CID the receiver already holds is left out of wanted without comment, so an offer cannot be used to learn what a peer has. A site peer, as an authority of every space it serves, accepts offers for those spaces under this rule and nothing else.
Live updates
A Watch is a server stream on a connection. The server sends one notification per indexing commit that added blobs to the watched scope set that the watcher may read. The watcher fetches them with Fetch. Because an authority peer indexes a new comment the moment its author's daemon offers it, and every follower holds a watch on that peer, a comment reaches followers within the roadmap's target of three seconds, and a document change within ten, without polling. A watch ends with the connection or with Goodbye; on reconnect the watcher passes since and receives what it missed.
Retention and garbage collection
What a peer keeps is decided by policy mode, by the kind, and by link kind.
A pin scope is never collected.
A follow scope keeps every resource's current state and the history the kind requires: history for Change graphs by default, latest for Snapshot chains. Superseded Snapshots may be dropped once nothing depends on them.
An on-demand scope may be collected when no client has asked for it recently and no retained blob depends on it.
Within a kept resource, the link kinds dep, proof, parent, schema and file are retained with the source. embed, link, mention and target are not: an embedded document is fetched when shown, under its own scope and its own readers.
Collection never removes a blob that another peer has been served in a still-live disclosure within the retention window, so that a Fetch for it can be honoured; past that window it may go. Open: the retention window and whether a site should refuse to collect anything it ever served.
Private content
Private content is content whose readers are not everyone. Over the network it is protected at four points.
The scope set is filtered to the connection's accounts before any fingerprint is computed. When a hidden blob exists in a range, the fingerprint is folded over the filtered items, so it is identical to what a peer holding only the readable items would compute.
Fetch reports an unreadable CID in missing, exactly like an unknown one.
Offer never confirms what the receiver already holds.
Every served private blob is a disclosure with basis grant, site or offer, naming the account and the grant that allowed it.
A bearer audience (anyone with the link) is honoured only by a site over HTTPS, where the secret can be presented in a request; peers do not present bearer secrets to each other. Comments on a private document have access target, so their readers are the document's readers, and they travel in the comments facet of the document's scope only to those readers.
Abuse and limits
A Reconcile or Fetch for a scope creates no durable state on the server unless the server's own policy already materialises that scope; arbitrary scopes from strangers cost one query and leave no row.
An Offer is bounded (ten thousand CIDs) and must pass the acceptance rule before any fetch begins; an unwanted scope is refused before the server does any work.
A Fetch is bounded (one thousand CIDs) and blobs are at most 2 MiB, as today.
Inbound reconciliation is limited per connection and in total, with a short wait before resource-exhausted; Watch streams are limited per connection.
No method stores an unvalidated blob. Nothing like Bitswap's acceptance of arbitrary data exists.
A peer that fails authentication or presents an invalid proof is backed off like a failed dial.
Today (HM24)
HM24 | Stem |
|---|---|
Range-based set reconciliation over | Kept unchanged in |
Bitswap fetch with a per-(peer, CID) filter, render-priority order |
|
|
|
|
|
Site identified by | Site is a signed relationship: root attributes plus a |
Up to 20 sampled peers per tier, plus gateways | Authority peers, trusted peers, and peers named by policy |
A WRITER capability at any path unlocks the whole space's private blobs | A connection receives exactly the blobs whose readers include its accounts |
Private items folded out of fingerprints, blockstore fails closed, Bitswap fails open | One evaluator for every surface; absence and denial are indistinguishable |
Nothing recorded about what was served or received | Disclosure and transfer ledgers |
Open questions
Whether a nested, content-addressed tree (a Prolly tree) per scope should replace the flat per-scope set, so that re-homing a resource into a narrower audience is a subtree re-key. The worked cases record what each case needs.
Routing without a distributed hash table: a fresh install reaches only authority peers it can resolve from a space root and peers it was told about. A delegated router or a relay directory is the likely next step.
Encryption, which would separate sync from read and let sites hold what they cannot read; the sync level is reserved for it.
Whether a device should be able to prove an account binding once and reuse it across connections.
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime