Offline-First Sync Engine

A production-oriented reference implementation for durable local writes, incremental synchronization, idempotent retries, tombstones, deterministic conflict resolution, and recovery from unreliable networks.

The repository is intentionally focused on one hard problem: making user data safe when requests fail, responses disappear, devices edit concurrently, and processes restart.

What it demonstrates

• Swift 6.2 strict concurrency
• Actor-isolated local storage and sync coordination
• Atomic local note plus operation-log persistence
• Durable retry queue with attempt tracking
• Batch upload and incremental pull checkpoints
• Idempotency keys for at-least-once delivery
• Tombstones for deletion propagation
• Deterministic logical-clock conflict resolution
• Exponential backoff and rate-limit handling
• Real URLSession transport
• Dependency-free Python reference server
• Deterministic two-device failure simulation
• Unit, integration, backend, and live HTTP tests

Architecture

flowchart LR
    UI[Mobile feature] --> Engine[OfflineFirstSyncEngine actor]
    Engine --> Store[LocalStore actor]
    Engine --> Resolver[Conflict resolver]
    Engine --> Transport[SyncTransport]
    Transport --> HTTP[HTTPSyncTransport]
    HTTP --> Server[Reference sync server]
    Store --> Snapshot[(Notes + operation log + checkpoint)]

Loading

A user action writes the updated note and its sync operation to local storage before any network request starts. The UI reads the local state immediately. The sync loop uploads queued operations, removes only acknowledged operations, then pulls changes after the last confirmed pull checkpoint. The pull response advances that checkpoint.

See docs/architecture.md and docs/diagrams.md.

Important guarantees

1. A local edit survives process termination before upload.
2. A lost server acknowledgement does not create a duplicate semantic write.
3. Applying the same remote operation repeatedly does not advance server state twice.
4. Deletes remain as tombstones until every client can observe them.
5. Identical histories produce the same conflict winner.
6. Only one sync loop runs for an engine instance.
7. A failed sync leaves pending operations intact.

Run

Requirements:

• Swift 6.2+
• Python 3.11+

swift test
swift run sync-demo

Run the real HTTP path:

./scripts/e2e.sh

Run every check:

./scripts/verify.sh

Package layout

Sources/
  SyncDomain/       Models, clocks, protocol DTOs, conflict policy
  SyncStorage/      In-memory and durable JSON stores
  SyncEngine/       Local mutations, queue processing, retry, merge
  SyncHTTP/         URLSession transport
  SyncSimulation/   Fault-injecting server and convergence scenario
  SyncCLI/          Deterministic offline demo
  SyncLiveCLI/      Live HTTP end-to-end demo
backend/            Reference sync server and backend tests
Tests/              Domain, storage, engine, and simulation tests
docs/               Protocol, architecture, diagrams, decisions
scripts/            Verification, coverage, metrics, boundary checks

Conflict policy

This implementation uses a logical clock and device ID tie-breaker. Wall-clock time is secondary. It avoids relying on synchronized device clocks and makes conflict outcomes deterministic.

This is not a universal merge strategy. Last-writer-wins is suitable for this reference note model, but financial records, collaborative text, inventory, and workflow state often require domain-specific merges, append-only events, or CRDTs.

Delivery semantics

The protocol assumes at-least-once request delivery. Each operation has a stable OperationID. The server records processed IDs and can acknowledge a retried request without committing it twice.

Exactly-once network delivery is not claimed.

Try the failure scenario

swift run sync-demo performs this sequence:

1. Device A creates a note offline.
2. Device B pulls it.
3. Both devices edit it while disconnected.
4. Device A uploads, but its response is dropped after the server commits.
5. Device A retries the same operation.
6. Device B uploads its competing edit.
7. Both devices pull and converge.

Deliberate limits

• JSON file storage keeps the sample dependency-free. A production app would normally use SQLite, Core Data, Room, Realm, or another transactional database.
• Tombstone compaction requires server knowledge of client progress and is not implemented.
• Authentication refresh belongs outside the engine, behind SyncTransport.
• The reference protocol syncs complete note snapshots rather than field-level patches.

License

MIT