Observability, audit and recovery

Five records are kept apart, because they answer different questions. Merging them would make each one worse at its own job.

The five records

• Technical observability — logs, metrics and traces. Answers "is the system healthy, and where is it slow".
• Operation timeline — the durable state and the steps an operation went through. Answers "what happened to this specific transfer".
• Audit trail — administrative and signing actions. Answers "who authorized this, and with which key".
• On-chain evidence — receipts, transaction identifiers, confirmations, logs and UTxOs. Answers "what does the network say actually happened".
• Ledger — the financial impact. Answers "what does this mean for the balances".

A log line is not evidence of settlement, and a receipt is not an accounting entry. Keeping them separate is what lets each one be trusted for what it is.

Every identifier the execution produced is readable

A record only answers its question if what it holds can be matched against the other four, and that matching is done by identifier. The operation therefore publishes each identifier the execution actually produced, rather than the one that happens to be most common:

• transactionHash, transactionHashes and transactions — the transaction, or transactions, it settled in, each named with its network.
• userOpHash — the UserOperation it travelled as, when it took the ERC-4337 path. On the operation, and on each of its steps and its legs. An operation that executed some other way carries no such field rather than an empty one.

Both are needed together: a bundler and an eth_getUserOperationReceipt call know an operation by its userOpHash, while a block explorer and a receipt know it by its transactionHash. Publishing only the second is what used to leave an incident that starts at the bundler with no way back to the operation. See Operations, quotes and states.

What never appears in a log

Structured logs never contain master keys, private keys, access tokens, seeds, passphrases or identity documents.

Operational recovery

Recovery covers:

• backups, and restore drills that actually exercise them;
• rehydrating workers from durable state after a restart;
• safe replay of events, which is possible because consumers deduplicate;
• rebuilding projections from the durable source;
• reconciling from the blockchain when the internal state is incomplete.

The last one is the important one. Because the chain holds facts the platform did not produce, a damaged internal state can be rebuilt from what actually settled, instead of from what the platform believed.