Skip to main content

The model

Everything this engine delivers news about is a slot: an identity that something may refer to, that may hold content, and that a frame may deliver news about. An entity is a slot. One column of an entity is a slot. A bucket is a slot. A derived value is a slot. A service is not a slot, and it holds no state: it is the code that files the slots of one kind under their keys, in the store each of its store slots carries.

Glossary​

slotTOIMSlot
A plain object that is the identity of one key in one collection, or of one single value. It carries its owner, its holdCount and the effects filed on it.
entity slotTOIMEntitySlot
A slot whose content is one entity, read as slot.entity.
column slotTOIMColumnSlot
A slot hanging off a carrier slot that stands for one column, a group of columns or any column. An effect on it is woken only when a write moves that column.
bucketTOIMBucketSlot
A slot whose content is other slots.
computedTOIMComputedSlot
A slot whose value is derived from other slots and is refreshed by the read that asks for it.
effectTOIMEffect
A body filed on slots by OIMEffectService.INSTANCE.createEffect. A frame runs it when one of its sources moved.
channel slotTOIMChannelSlot
What every write goes through. It carries the clock, the effects scheduled by the slots that moved and the flush state; OIMChannelService.INSTANCE is the code that runs it.
frame
The delivery of the writes made since the previous frame: each effect those writes scheduled runs once, then the after-each-frame handlers run.
flush
One call of db.flush(). It runs onBeforeFlush and a frame when anything is owed, then the unused-slot collector gives back its holds and the after-each-flush handlers run.
hold
A counted reference on a slot (holdCount), taken by owner.holdSlot or a collection's holdSlotByKey and given back by owner.unholdSlot.
store slotTOIMStoreSlot
One collection: the state of one instance — the store its slots are filed in, its channelSlot, its listeners and column state. createStoreSlot(db) makes one; every door that works on a whole collection takes it first.
mint
Create the slot of a key that has none and file it in the store of its store slot.
release
Unfile a slot that nothing holds or refers to, once its release policy allows it. The slot's RELEASED flag is then set (isOIMSlotReleased(slot)).

Three sentences carry the rest of this section:

  1. Four doors mint an identity: writing a value, subscribing, taking a hold, and getSlotByKey, whose mint the unused-slot collector holds for the turn. peekSlotByKey and hasSlotContentByKey look a key up and mint nothing.
  2. A slot goes when nothing refers to it, and a reference is given back by the thing that took it.
  3. Where a trade-off has two defensible strategies, both ship and you pick at construction.

A key is resolved to its slot once, and from then on every reader, writer and subscriber works through that object: an effect is filed on the slot itself, a bucket holds the slot itself, a column is a slot hanging off its carrier. Nothing on the delivery path looks a key up again, and a subscriber who arrived before the value did is subscribed to the same object the value later lands on.

The three kinds of slot​

There are three, and no others:

KindWhat its content isType
entitya value, whose columns can be watched one by oneTOIMEntitySlot
bucketother slotsTOIMBucketSlot
computeda value derived from other slotsTOIMComputedSlot

Every slot, whatever its kind, has the shape TOIMSlot. Its fields are readonly in the type you are handed: you read them, and ask slot.owner for holds, subscriptions and releases. slot.storeSlot is the store slot the slot is filed in, which is why a door handed a slot needs no store slot of its own.

Each kind comes in two shapes: one of it, with no key; or many of it, filed by key. The store service decides which: a single is built over OIMSingleSlotStoreService.INSTANCE, which files at most one slot in a TOIMSingleSlotStore ({ slot }), and its reads are OIMSingleSlotService's.

kindsinglecollection
entitycreateOIMSingleSlotService(): getSingleSlot, holdSingleSlot, peekSingleSlotcreateOIMSlotCollectionService({ selectKey }): getSlotByKey, holdSlotByKey, peekSlotByKey
bucketcreateOIMSingleSlotService()createOIMSlotCollectionService(), or an index
computedcreateOIMSingleComputedService({ … }), read through the service itself (getSingleSlot, holdSingleSlot)createOIMComputedService({ … })

createStoreSlot mints nothing. A single's slot is minted on the first getSingleSlot(storeSlot) or holdSingleSlot(storeSlot), exactly as a collection's slot is minted on the first read of its key, and peekSingleSlot(storeSlot) answers it without minting. It is released at its release policy like any slot of that type, and the next get or hold mints a fresh one. A single invents no behaviour of its own. Use the single shape for one value the whole program shares (settings, a session, one selection); see Standalone records. Use a collection as soon as the values are told apart by a key.

Both computed shapes own TOIMComputedSlot, the plain data type minted from createSources: a computed slot dies when nothing holds it, the store slot outlives it and mints a fresh one on the next read or hold. Its value is read as slot.value, a getter over the value service the slot carries, which re-derives a stale value before answering. See The compute graph.

Every collection is a content service, a collection service and a store slot​

A collection type is two stateless objects:

  • the content service — what one slot holds and how it is written: OIMEntityContentService (built by createOIMEntityContentService()), OIMPositionalArrayBucketContentService / OIMSetBucketContentService / OIMSortedArrayBucketContentService, or a computed's OIMComputedValueService. It is the strategy, and it owns no slot owner;
  • the collection service over it — OIMSlotCollectionService (or OIMSingleSlotService for a single), the read doors by key and the column doors. It builds the slot owner, an OIMSlotOwnerService, that holds, subscribes and releases the slots of every store slot of that type. Its slotContentService is the content service.

A store slot (TOIMStoreSlot) is one collection, the state the two services work on. It is not a slot: it has no owner, no hold count and no effects, so it is never held or subscribed.

Setup shared by the examples on this page
import { OIMDB, OIMEffectService, OIMSlotAtUnholdReleasePolicy, OIMSlotNeverReleasePolicy, OIMUnusedSlotCollectorService, createOIMComputedService, createOIMReplacingEntityWriter, type TOIMEntitySlot, type TOIMStoreKey, createOIMEntityContentService, createOIMSlotCollectionService, OIMPositionalArrayBucketContentService, OIMBucketCommands, OIMDefaults } from '@oimdb/core';

type Card = { id: string; deck: string; title: string };

const db = new OIMDB();
const effectService = OIMEffectService.INSTANCE;
const cardService = createOIMSlotCollectionService({ slotContentService: createOIMEntityContentService<Card>() });
const cards = cardService.createStoreSlot(db);
const byDeckService = createOIMSlotCollectionService({ slotContentService: new OIMPositionalArrayBucketContentService<TOIMEntitySlot<Card>>({ bucketCommands: new OIMBucketCommands({ maxCommandsPerSlot: undefined }) }) });
const byDeck = byDeckService.createStoreSlot(db);
const rankService = createOIMComputedService<TOIMStoreKey, number>({});
const ranks = rankService.createStoreSlot(
db,
(computedSlot) => cardService.peekSlotByKey(cards, computedSlot.key)?.entity?.title.length ?? 0,
({ key }) => ({ sourceSlots: [cardService.getSlotByKey(cards, key)] }),
);

declare const pk: TOIMStoreKey;
declare const key: TOIMStoreKey;
declare const entity: Card;
declare const slot: TOIMEntitySlot<Card, TOIMStoreKey>;
declare const body: () => void;
cardService.getSlotByKey(cards, pk) // an entity collection
byDeckService.holdSlotByKey(byDeck, key) // a bucket collection
rankService.getSlotByKey(ranks, key) // a computed collection

What a content service adds is its own vocabulary for its own content: writeSlotContentByKey, updateEntity and clearSlotContent on an entity content service; addBucketMemberSlots, removeBucketMemberSlots and getBucketMemberSlots on a bucket content service; getValueForSlot on a computed value service.

The root: OIMDB​

OIMDB is what every create… factory that builds a collection type takes, and what every createStoreSlot(db) takes. The computed factories (createOIMComputedService, createOIMSortedComputedService, createOIMBucketMemoComputedService) take no db: a computed store slot gets its channel slot from createStoreSlot(db, …) or from the store slot it reads. The root holds what every collection in one program must share, and the defaults you want to state once:

PropertyWhat it is
defaultChannelSlotthe TOIMChannelSlot every store slot made with createStoreSlot(db) delivers through
clockthe revision and epoch every write is stamped with
collectorSlotthe collector slots: the unused-slot collector that holds what getters mint, and the three release collectors
schemaswhat the createSchemas option returned, called once at the end of the constructor; undefined without it (see A database with schemas)

Its constructor takes the channel's options (channelScheduler, errorPolicy, onBeforeFlush), the clock and timer (getNowMs, releaseTimer), and createSchemas in one object. The strategies a factory falls back on are not the root's: they are an OIMDefaults passed as defaults. db.flush() flushes its default channel slot, and db.createEffect(body, sourceSlots?, { lazy? }) creates an effect on it and returns the disposer. db.dispose() drops the database: no later flush delivers, its effects stop and its release wheels are cancelled.

const appDb = new OIMDB({
channelScheduler: (flush) => { void Promise.resolve().then(flush); },
});
const appDefaults = new OIMDefaults({
entityWriter: createOIMReplacingEntityWriter(),
slotReleasePolicy: OIMSlotAtUnholdReleasePolicy.INSTANCE,
});

const noteService = createOIMSlotCollectionService({ defaults: appDefaults, slotContentService: createOIMEntityContentService<Card>({ defaults: appDefaults }) });
const notes = noteService.createStoreSlot(appDb);
const archiveService = createOIMSlotCollectionService({ defaults: appDefaults, slotContentService: createOIMEntityContentService<Card>({ defaults: appDefaults }), slotReleasePolicy: OIMSlotNeverReleasePolicy.INSTANCE });
const archive = archiveService.createStoreSlot(appDb);

A factory option outranks defaults, and defaults outranks the built-in default. createOIMComputedService reads only its own slotReleasePolicy, so a defaults release policy does not reach a computed. The policy belongs to the collection type, so every store slot of one type shares it.

An effect over slots of two collections runs in one frame only if both store slots report to the same channel slot. The same holds for the collector: a mint is given back by the flush of the channel slot that owns that collector. OIMDB is the place where those are made once and shared.

OIMDefaults is an instance, not a process-wide setting: nothing is set once for everyone, and two parts of an app can build their services with different defaults. The package still refuses to load twice in one process — two copies would each have their own classes and built-in defaults.

The four doors​

The four doors are writing a value, subscribing, taking a hold, and reading a key that has none. What admits each of them is that the act leaves a reference behind on what it produced — a value, an effect, a count, or a tracked mint — so nothing can unfile the identity under the caller who just obtained it.

cardService.writeSlotContentByKey(cards, entity.id, entity); // door 1: write. The value is the reference.
cardService.writeSlotContentByKey(cards, key, entity); // the same door, stated by key.
effectService.createEffect(db, body, [slot]); // door 2: subscribe. The effect is the reference.
const held = cardService.holdSlotByKey(cards, key); // door 3: hold. A counted reference and a give-back.
slot.holdSlot(); // the same door, on a slot you were handed.
const read = cardService.getSlotByKey(cards, pk); // door 4: read. The collector's hold is the reference.

The same four doors exist on every collection. A single uses getSingleSlot(storeSlot) and holdSingleSlot(storeSlot); on a column, they are the collection service's getColumnSlot(slot, columnName) and holdColumnSlot(slot, columnName).

A read by getSlotByKey mints, and the collector takes the mint back. It returns TSlot and never undefined: on a miss it mints the identity, files it, and hands it to the db's unused-slot collector, which holds it until the next flush(). A read is admitted as a door only because of that hold — without it the read would hand back an identity the next release unfiles under its caller, after which the next resolution mints a second one and the caller reads empty for ever. A caller who then holds or subscribes keeps the identity; a caller who walks away costs one slot until the end of the turn.

Which reads mint:

MintsDoes not mint
getSlotByKey, getSingleSlot, getColumnSlot, getAnyColumnSlot, getEntitiesByKey on an index servicepeekSlotByKey, peekSingleSlot, hasSlotContentByKey, hasSingleSlotContent, isSameKey, getSlots, getSlotCount, getBucketMemberSlots

peekSlotByKey(storeSlot, key)?.entity answers the entity or undefined, and hasSlotContentByKey(storeSlot, key) answers whether a slot is filed under the key with content; both look the key up and file nothing. The one by-key write is the whole-value writeSlotContentByKey; any other write by key resolves the slot first (getSlotByKey(storeSlot, key).updateEntity(patch)), and that read is the mint. The complete list is in Reading and subscribing.

A read can be scoped past a flush. OIMUnusedSlotCollectorService.INSTANCE.open(collectorSlot) and close(collectorSlot) — or run(collectorSlot, body), which brackets body with both — defer the give-back: while a scope is open the collector's flush returns having done nothing, so a frame inside the scope does not take a mint back, and the outermost close gives everything back. Scopes nest; a close with no open throws.

const unusedSlotCollectorSlot = db.collectorSlot.unusedSlotCollectorSlot;
const sameSlot = OIMUnusedSlotCollectorService.INSTANCE.run(unusedSlotCollectorSlot, () => {
const first = cardService.getSlotByKey(cards, 'c1');
db.flush();
return first === cardService.getSlotByKey(cards, 'c1');
});

The write door always hands back the identity it resolved or minted, whatever the write changed. A door that mints must leave a reference behind, and that is the same test the four doors are admitted on. What the write changed goes into the optional changed array instead — see Writing.

A refusal and an unchanged write are different things. A refusal means no content ever arrived: the write did not happen and that identity is unwound, so a refused key is not filed. An unchanged write did happen and the content was already there: the door completed and the caller is handed the identity, and whether an identity nothing keeps is worth keeping is the owner's question, asked through releaseSlotIfUnheld.

A slot goes when nothing refers to it​

The whole life of a slot, from mint to release, is covered in Slot lifecycle.

There is no release API, no sweep a caller starts, and no owner field for a caller to set. The owner is asked — slot.unholdSlot() by whoever gives a hold back, and, at a deadline, slot.owner.releaseSlotIfUnheld(slot) by the release queue, which needs no deadline question answered first — and it re-reads every condition, so asking twice is harmless.

holdCount has one writer, the owner. A command that released a slot on request would let one caller unfile an identity another caller is still subscribed to, and the second caller would keep reading an object no lookup returns any more. Counting references and letting the owner decide is what makes "nothing refers to it" checkable in one place.

releaseSlotIfUnheld(slot) is public on IOIMSlotOwner, because the release queue reaches it through slot.owner. It is not a release command: it is a guard that returns having done nothing unless the slot is unheld, carries no column slots, still owned by the object asked and allowed to go by the release policy. It returns silently on a slot something else already released, where every door that tries to work with a released slot throws.

The terms, and who gives each back:

TermGiven back by
holdCountslot.unholdSlot(); the unused-slot collector's flush after a getSlotByKey; a slot leaving a bucket; effectService.disposeEffect(effect), for the hold the effect took
effectseffectService.disposeEffect(effect)
a column slot on the carrierunholdColumnSlot(columnSlot) and the last effect on that column being disposed; the carrier is asked again when its last column goes
the valueslot.clearSlotContent() or bucket.clearSlotContent() — under the default policy; see the release policies below
a composite-key edgethe tree pruning that composite key part

What the value counts for is the release policy's answer. Every collection type is built with one slotReleasePolicy, and the policy is asked only once nothing holds the slot. The table of policies, the default of each kind and the time policies with their markers are in Slot lifecycle. With the default, an entity you wrote and nobody holds stays in memory until it is deleted, which is what a store is for; a computed nobody holds is dropped, because it can be derived again.

A slot in a bucket is a counted reference, a hold. A slot you put in a bucket and never took out is a slot nothing reclaims: a bucket's slots are the caller's, so the give-back is the caller's too.

Subscribing takes a hold too. createEffect files the effect in slot.effects and calls holdSlot on each root the caller named (both through owner.subscribeToSlot); disposeEffect(effect) gives both back. A subscribed slot is therefore a held slot, and the release clocks need not count subscribers.

A collection given an idle window (OIMSlotAfterIdleReleasePolicy) opens that window at exactly two moments: when a slot first gets content while nothing subscribes to it, and when the last subscriber leaves (unsubscribeFromSlot takes the last effect off the slot). A give-back (unholdSlot) opens no window and stamps nothing, but it asks the owner whether the slot may go, so a slot whose window ran out while it was held is released at the give-back. A held slot is not released while it is held. A collection given no time policy runs no clock, and that is the default. The windows are described in Slot lifecycle.

slot.clearSlotContent() takes the value away and leaves the identity standing for as long as anything still wants it. So clearSlotContent then writeSlotContentByKey under one key is one identity, and whoever was subscribed is still subscribed to the thing they were given.

Ending the whole channel is a different level and has its own call. Every term above is per identity, which suits one subscription ending but not everything ending: walking every bucket and every slot to unfile each one is bookkeeping for a future that never arrives, because the moment you drop the channel all of it is garbage together. db.dispose() ends it without walking your graph, on your word that nothing is subscribed to anything on that channel slot — the engine does not check that, because checking means asking every slot for its effects list, which is the walk the call exists to skip. The one list it does walk is the unused-slot collector's, which holds what this frame's getters minted and which nothing else would ever give back once the channel is dropped. It still does not let anything ask for one slot to go. See dropping the whole channel.

The frame​

A frame is the delivery of every write since the previous frame, as a batch: each effect those writes scheduled runs once, however many writes touched its sources, and then the after-each-frame handlers run. A flush is one db.flush(): it runs onBeforeFlush and a frame when anything is owed, then the unused-slot collector gives back its holds and the after-each-flush handlers run.

A write marks the slot and schedules the effects filed on it; it runs nothing. Ten writes to one slot before a flush cost ten marks and one delivery, and an effect over several slots that all moved runs once. When the flush happens is the channelScheduler — a function handed the flush to call later — and with none, nothing is ever delivered until you call flush() yourself.

  • An effect hears only what happens after it. effectService.createEffect(...) is stamped with the channel slot's clock at birth and its body first runs in a frame where one of its sources moved. effectService.createAndRunEffect(...) also runs the body once, now.
  • Writes are refused for the whole of a flush, not just the frame body. A subscriber is told during the frame, on a half-built world — the graph has not advanced, the delivery list has already been taken — so a write from there would land in a frame that is no longer collecting. onBeforeFlush and the after-each-frame handlers are inside the same flush and refused too. It throws by name. Every content service asks the channel slot before it writes, on a collection and on a single alike.
  • flush() inside a frame is refused for the same reason: a frame inside a frame is not nested, it is the outer one taken apart. flush() from the tail of a flush, onBeforeFlush or an after-each-frame handler, is not refused: it returns having done nothing.
  • No door writes from inside a flush. A write in response to a change happens after flush() has returned: a handler hands the work to a microtask or a later callback rather than writing itself. Handlers given to OIMChannelService.INSTANCE.subscribeAfterEachFlush(channelSlot, handler) run after the flush has lowered its flag and given the collector back, so a write from one is accepted and asks for the next frame.
  • A derivation is asked, never woken. Nothing schedules a computed slot and no frame walks to one; it settles when a reader asks for its value. A derivation nobody reads costs nothing per frame.
  • One throwing handler does not abort the frame by default. The throw goes to the error policy: OIMLogAndContinueErrorPolicy (the default) logs it, hands it to its onError, and the frame finishes; OIMThrowAtOnceErrorPolicy throws it out of flush() and leaves the effects not yet reached for the next flush.

The full rules, with batches and the after-frame hooks, are in The channel.

Forks are strategies chosen at construction​

Where a trade-off has two strategies that are each right for some consumer, this package ships both behind one abstraction and you pick at construction — once, statically, injected. The choice then costs nothing at run time: no branch, no flag, no check on the hot path, because the object already is the answer.

The forks a consumer meets first:

ForkStrategiesChosen at
how a write landscreateOIMInPlaceEntityWriter() / createOIMReplacingEntityWriter()the entityWriter option of the entity content service, or of OIMDB
whether the first value handed in is copiedinsertOIMEntityWithoutCopy (default) / insertOIMEntityWithCopy / insertOIMEntityWithSanitizedCopythe entityInserter option of either writer factory
how a bucket holds slotsOIMSetBucketContentService / OIMPositionalArrayBucketContentService (the positions are the caller's; such a bucket never sorts) / OIMSortedArrayBucketContentServicethe slotContentService option of the bucket or index factory
what naming a slot recordsOIMPositionalArrayBucketContentService (per mention) / OIMBucketUniqueMemberSlotsArrayService (per slot)the slotContentService option of the bucket factory
who decides a bucket's slotsa bucket collection (the caller) / an index (the entity)which factory you call
whether and when a slot is releasedthe release policiesslotReleasePolicy on the factory's options, or on OIMDB's
when a flush runsmanual flush() / a channelSchedulerOIMDB's options
what a throwing handler doesOIMLogAndContinueErrorPolicy / OIMThrowAtOnceErrorPolicyerrorPolicy on OIMDB's options

The whole list, keys and stores included, is in Architecture.

Every collection has a channel​

Every store slot is created with a channel slot, and every write to it, collection or single, is reported there. createStoreSlot(db) takes db.defaultChannelSlot; an index or a computed store slot over another store slot takes that one's. What createStoreSlot takes is an IOIMChannelMap, an object with a defaultChannelSlot, so a second channel is a second OIMDB or any such object over a createOIMChannelSlot(…). See Writing.

Where to next​