The “compute haves” work is mainly here:
Main path: discovery local store
File:
backend/hmnet/syncing/discovery.goFunction:
func (s *Service) DiscoverObjectWithProgress(...)Around the middle of that function, it builds the local RBSR store once:
store := newAuthorizedStore()
err := s.db.WithSave(ctx, func(conn *sqlite.Conn) error {
return loadRBSRStore(conn, dkeys, store)
})
store.Seal()
That is the local “what do we have?” dataset for this discovery.
Then the same store is passed to all peer syncs:
res := s.syncWithManyPeers(ctxLocalPeers, subsMap, store, prog, auth, blobTypes)The expensive DB part
Same file:
func loadRBSRStore(conn *sqlite.Conn, dkeys map[DiscoveryKey]struct{}, store *authorizedStore) errorThis calls:
collectBlobs(conn, dkeys, false)Then runs a big select over rbsr_blobs, blobs, structural_blobs, blob_visibility, ordered by timestamp/hash, and inserts each item into the in-memory RBSR store:
store.Insert(ts, cidBytes)So the DB-heavy “compute local haves” part is:
DiscoverObjectWithProgress
-> loadRBSRStore
-> collectBlobs
-> fillTables
-> SELECT ... FROM rbsr_blobs ...
-> store.Insert(...)
-> store.Seal()Per-peer reconciliation haves
File:
backend/hmnet/syncing/syncing.goFunctions:
syncWithManyPeers(...)
syncWithPeer(...)
syncResources(...)Each peer gets the same store, filtered by authorization:
filteredStore := store.WithFilter(authorizedSpaces)Then syncResources creates an RBSR session:
ne, err := rbsr.NewSession(store, rbsrMsgSizeBytes)And each reconcile round computes haves/wants here:
msg, err = ne.ReconcileWithIDs(msg, &haves, &wants)RBSR algorithm code
File:
backend/hmnet/syncing/rbsr/rbsr.go
Function:
func (n *Session) reconcile(...)
In ListMode, it compares our store range with the peer’s range:
n.store.ForEach(lower, upper, func(_ int, item Item) bool {
have := item.Value
if _, exists := theirElems[string(have)]; !exists {
*haveIDs = append(*haveIDs, have)
}
...
})That is literally where “haves” are appended.
Important nuance
Current code already builds loadRBSRStore once per DiscoverObjectWithProgress call, then reuses it across peers in syncWithManyPeers.
We compute haves once per peer:
the RBSR range comparison work in
ReconcileWithIDs, which does happen per peer; ormultiple frontend-triggered discoveries for the same document, each rebuilding its own store; or
server-side
ReconcileBlobs, where each inbound peer request builds a store separately.
Recap
The main haves code is:
DB/local dataset build:
backend/hmnet/syncing/discovery.go→loadRBSRStoreper-peer haves/wants computation:
backend/hmnet/syncing/syncing.go→syncResources→ReconcileWithIDsactual haves append:
backend/hmnet/syncing/rbsr/rbsr.go→Session.reconcile
Current discovery code appears to build the expensive DB-backed local store once per discovery, not once per peer.
Do you like what you are reading? Subscribe to receive updates.
Unsubscribe anytime