Client API

This is what an app imports to keep its data in Loess: @loess/sdk and a platform entry, @loess/sdk/browser in a browser or @loess/sdk/memory in a test. The sdk signs, seals, sends, retries, caches, and runs each circle's program on the device. The app writes facts, reads them with SQL or Rows, and shows what Device.status reports.

The API uses Effect. A call fails with LoessError, and a sign-in can also fail with SignInError. Each has a reason a screen can show (see Errors). A rejected write doesn't fail: it shows in Device.status (see Status).

Open and sign in

import { Loess } from '@loess/sdk';
import { layerBrowser } from '@loess/sdk/browser';
 
const program = Effect.gen(function* () {
  const loess = yield* Loess.open({ storage: 'https://host.example' });
  const state = yield* Stream.runHead(loess.state);
  // SignedOut, SignedIn with the device, or Failed with why a kept sign-in didn't open
});
 
Effect.runFork(
  program.pipe(
    Effect.scoped,
    Effect.provide(layerBrowser({ storeName: 'moments', passkeyName: 'Moments' })),
  ),
);

• Loess.open takes the device store's Web Lock, so a second tab waits, and opens the device that signed in there before, with no prompt and no storage read. storage is the URL where a new identity and its circles start. program is the program object new circles get.
• loess.state is a stream: SignedOut, SignedIn with the Device, or Failed with a LoessError when the kept sign-in can't open, such as when its keys are gone. Show sign-up only for SignedOut, so a person whose device failed never makes a second identity.
• layerBrowser({ storeName, passkeyName, domain }) keeps the device store and the device's keys in IndexedDB and uses WebAuthn for passkeys. It reaches storage over HTTP and follows the relay over a WebSocket. Two apps on one origin with the same storeName are one device.

A new device signs in once:

const { device, phrase } = yield * loess.signUp; // show the phrase once
const device = yield * loess.signIn.passkey; // two prompts
const device = yield * loess.signIn.phrase('abandon ability able ...'); // no prompt
yield * loess.signOut; // sign out and keep this device's keys
yield * loess.forget; // sign out and drop this device's keys

• signUp makes an identity on the app's storage whose owners are a recovery phrase and, when the platform has one, a passkey, and adds this device. It needs storage. The person writes the phrase down.
• signIn.passkey prompts twice: to pick the passkey, and to sign the record that adds the device. The device signs the head itself, because that record lists it as a committer. The passkey's user handle holds the identity's id, and the device finds the identity on the storage the app passes.
• signIn.phrase builds the identity's first record from the phrase to find its id, and adds the device with no prompt. If storage holds no identity for the phrase, sign-in fails with unknown.
• signOut keeps the device's keys, so the same identity signs it back in as the same device. forget drops them, so the next sign-in makes it a new device.

Circles

const notes = yield * device.circle('notes'); // the identity's circle with this label, or a new one
const all = device.circles; // a stream of the identity's circles, from any of its devices

• device.circle(label) finds the identity's circle with that label, or creates one with the identity as its only owner. A new circle opens at once from what the device made, offline too, and its outbox sends the circle's first commit to storage before any write to it.
• device.circles gives the list each time it changes, also when another device of the identity creates or joins a circle. The identity's facts list each circle as circle:<id> with circle/label.
• A Circle has its id and the label the app gave it. leave drops it from the identity's circles on every device.

Write

const write =
  yield *
  notes.write([
    { id: 'note:1', field: 'note/title', value: 'Groceries' },
    { id: 'note:1', field: 'note/done', value: null },
  ]);

• A write shows in every query and live query of this device before the call returns, offline too. It is in the device store first, so a closed tab or a crash loses nothing. The outbox sends it as soon as storage answers.
• The changes of one write apply together or not at all, and the write gives its WriteId.
• The circle's program runs on the write first. If it rejects the write, or a replaces no longer holds on this device, the write is never sent, and Device.status shows it as rejected at once, as it shows a committer's rejection. write still gives its WriteId.
• A change sets the fact of an id and field. A value is text, a number, bytes, a reference { id }, or null, which deletes the fact. With no replaces, the last write in storage's order is kept. With replaces, the hash of the value it expects, a committer rejects the write if the fact holds another value now.

Read

import { sql } from '@loess/sdk';
 
const Title = Schema.Struct({ id: Schema.String, value: Schema.String });
const titles = sql(
  Title,
)`SELECT id, value FROM stored_facts WHERE field = ${'note/title'} AND current ORDER BY id`;
 
const now = yield * notes.query(titles);
yield * Stream.runForEach(notes.live(titles), render);

• sql(row) builds a read-only statement over the stored_facts view, which holds each fact with its writer and committed_at. Each interpolation is a bound parameter, never SQL text, and each row is decoded with the schema.
• query runs a statement once. live runs it now and again after each change to the circle's database, from this device or another.
• A cold circle reads only the pages the statement touches. A query that reads more than 128 pages fails, so a scan fails in a test with realistic data.

Rows

Rows maps a typed record to facts. A row's facts share its id, and each field of the row is one fact field, {name}/{field}, so other apps read the same data as plain facts.

import { Rows } from '@loess/sdk';
 
const Post = Schema.Struct({ caption: Schema.String, at: Schema.Number });
const posts = Rows.make('post', Post);
 
const id = yield * posts.id({ author: identity });
yield * posts.insert(home, id, { caption: 'dawn', at: Date.now() });
const dawn = { circle: home, id };
yield * posts.update(dawn, { caption: 'dusk' }, { expected: { caption: 'dawn' } });
yield * posts.delete(dawn);
 
const feed = posts.live(friendCircles, { order: { field: 'at', direction: 'desc' }, limit: 50 });

• Each write is one Circle.write. update and delete take a RowReference, which holds the circle and the id. A Row is one too, and also holds the author of its newest value.
• id() makes {name}: followed by the time and 4 random bytes, so ids sort in the order they were made. id({ tag }) makes {name}:{tag}, for a row with a fixed id, such as person:me. id({ author }) puts the author last, {name}:{tag}~{author}, and the author program lets only that member change a row with that id.
• update with expected changes a field only if it still holds that value, and otherwise the write is rejected with the value found.
• live(circles, query) runs one statement in each circle and merges the rows. circles is a list or a stream, and the rows follow the stream. where keeps rows whose fields, id, or author hold these values, and order walks the index of one field and stops at limit. Without order, it walks a field every row holds in the order of the rows' ids, and stops at limit. id() without a tag makes ids in the order they were made, so the rows of every member interleave in that order.
• A row whose facts don't decode with the schema is left out, so a new optional field still reads old rows.

Blobs

const reference = yield * home.putBlob(photoBytes); // "blob:9f2c…"
yield * home.write([{ id: 'post:1', field: 'post/photo', value: { id: reference } }]);
const bytes = yield * home.readBlob(reference); // undefined until storage holds the bytes

• putBlob seals the bytes with the circle's data key and gives a reference: blob: followed by the hex of the sealed blob's hash. This device shows the write that references it at once, and the outbox keeps the write unsent until storage stores the blob, then sends it. Storage passes the write to the relay as it arrives, so other open devices show it once the upload ends. A later write that changes or references the same ids or blobs waits behind it, and any other write is sent at once. The outbox orders writes by id only, so a write that depends on a held write through a string field, such as a like's post, can commit first. If storage refuses the upload, the write is rejected. If storage lacks a write's blob, a committer rejects the write with blob.
• readBlob gives undefined for text that isn't a reference, or for a blob storage doesn't hold yet, so read it again later.

Status

yield * Stream.runForEach(device.status, (writes) => show(writes));
yield * device.dismiss(write);

device.status lists this device's writes that aren't settled, and the rejected ones, each time one changes. Each has its WriteId and a status:

• unchecked: the program needs pages this device doesn't hold, so no view shows the write yet.
• unsent: checked, and kept on the device until storage accepts it. Chat shows "waiting to send" if a write stays here for 2 seconds.
• sent: in the circle's inbox, so a committer settles it even if this device never opens again.
• rejected: by a committer, or by the program on this device, either at once or when an unchecked write's pages arrive. It holds the write's changes, so a screen can say which change it was after a reload too, and a Rejection with the reason and, for a conflict, the fact's current value. It stays until the app passes it to dismiss.

A committed write leaves the list.

Members and invites

const link = yield * friends.invite('write', { count: 5 });
const joined = yield * device.join(link, 'friends of ada'); // on the other person's device
const members = yield * friends.members;
yield * friends.remove(member);

• invite(role, { count }), as an owner, commits a record that lists the link's key with role and count, the number of people it admits, 1 by default, and gives the link's text. Whoever holds the text reads the circle until an owner removes the link.
• device.join(link, label) opens the circle with the link's keys and returns once storage answers the join request, which the circle's outbox sends again after a network failure or a reload. A committer admits the identity with no owner online. It lists the circle in the identity under label, so the person's other devices open it too.
• members gives the members of the newest head this device checked, each with a role: read, write, or own.
• remove(member), as an owner, starts a new key version that the member never gets.

Signals

yield * notes.signal(new TextEncoder().encode('typing'));
yield * Stream.runForEach(notes.signals, ({ member, bytes }) => showPresence(member, bytes));

signal signs bytes, such as presence or a typing mark, with the sending member's key, seals them with the member's id, and sends them through the relay to the devices that have the circle open now. A browser sends them on its follow's WebSocket, and the relay doesn't send them back to it. The relay stores nothing and can't read the bytes or the author, so a device that isn't open never gets them. The signature covers the circle, the member, and the bytes, and a receiving device checks it against the sender's key: a key member's own key, or an identity's write key from its head. The device checks against the write key it last read for that identity, and reads it again when a signature fails, so it accepts a signal signed with a replaced key until it has read the new one. signals drops a signal that fails the check, or whose sender isn't a member now, so a member can't claim another's id or pass a signal from another circle. The first signal from another identity waits for up to 3 storage reads, and later ones read nothing until a signature fails. A member can still resend another member's signal in the same circle, because a signal holds no time.

Devices and owners

const owner = yield * device.unlock.passkey; // or device.unlock.phrase(words)
yield * owner.removeDevice(lost.member);

• device.devices lists the identity's devices, each with its member key and isThisDevice.
• device.unlock gives an Owner. A phrase needs no prompt, and a passkey prompts once to unlock and once for each record it signs.
• removeDevice commits the identity's next record without the device, with new keys, and then new keys for each circle the identity owns that is open on this device. Each open circle then takes the new keys, so a Circle from before keeps reading, writing, and streaming.

Errors

Each type's doc says what each reason means. The entry also exports the types an app needs in its own code, such as WriteStatus, Change, MemberId, Role, and Statement.

Testing an app

memoryNetwork from @loess/sdk/memory runs storage, the relay, and, with hasCommitterService, a committer service in the test's process. Each device it makes has its own device store, keys, and passkeys, counts its storage requests, and has setOffline. @loess/test-utils gives eventually, which retries a check until it passes, and firstWhere, which waits for the first value of a stream that passes. An app's tests run each person on a device of their own.

Not built

• Sign-up offline.
• Owner actions other than removeDevice, such as adding a passkey or a device link.
• A list of the open invite links, and moving a circle to other storage.