Skip to main content

Buckets

Setup shared by the examples on this page
import { OIMBucketCommands, OIMBucketCommandsCursor, OIMDB, OIMEffectService, OIMPositionalArrayBucketContentService, OIMBucketUniqueMemberSlotsArrayService, OIMSetBucketContentService, getOIMOrderedListCommands, type TOIMBucketSlotsArrayStore, type TOIMBucketSlotsSetStore, type TOIMEntitySlot, type TOIMOrderedListCommand, type TOIMStoreKey, createOIMEntityContentService, createOIMSlotCollectionService, createOIMSingleSlotService } from '@oimdb/core';

type Card = { id: string; deck: string; rank: number };

declare function render(value?: unknown): void;
declare function rebuildFrom(slots: Iterable<TOIMEntitySlot<Card>>): void;
declare function applyToTheRenderedList(commands: readonly TOIMOrderedListCommand[]): void;

const db = new OIMDB();
const effectService = OIMEffectService.INSTANCE;
const cardService = createOIMSlotCollectionService({ slotContentService: createOIMEntityContentService<Card>() });
const cards = cardService.createStoreSlot(db);

const orderedDeckService = createOIMSlotCollectionService({ slotContentService: new OIMPositionalArrayBucketContentService<TOIMEntitySlot<Card>>({ bucketCommands: new OIMBucketCommands({ maxCommandsPerSlot: undefined }) }) });
const orderedDecks = orderedDeckService.createStoreSlot(db);

A bucket is a slot whose content is other slots. A bucket collection files buckets by key, in the store of each bucket store slot. Together they answer "which entities belong to this thing".

Which one you need​

Your situationTakeSection
The members come from outside: a server page, a search result, a selection, a visible windowcreateOIMSlotCollectionService({ slotContentService: new OIMSetBucketContentService() })A bucket's slots are the caller's
Membership is a function of the entity (card.deck)an index, createOIMSlotCollectionIndexServiceIndexes
A list sorted by columnscreateOIMSortedComputedService over an index serviceOrdering
A list whose order the user decides (drag, reorder)createOIMSlotCollectionService() — array-driven by defaultPositions the caller decides
...and a renderer that patches instead of rebuildingthe same: its default content service records OIMBucketCommandsThe commands, on the bucket slot
One list under no keycreateOIMSingleSlotService()A single bucket
A diff between two lists you hold yourselfgetOIMOrderedListCommandsRendering an ordered bucket

A bucket's slots are the caller's​

A bucket collection is manual. Which slots a bucket holds is a fact about the response a caller is composing: a search result, a page of server ids, a visible window, a selection. No column can express absence. There is no value an entity can carry that means "and these three entities that the server did not return are not here". So only the code that composed the response can state which slots the bucket holds, and the engine does not guess.

The other strategy ships too: an index, for when belonging really is a function of the entity. Which one you get depends on which factory you call.

Building one​

Setup shared by the examples on this page
const byDeckService = createOIMSlotCollectionService({ slotContentService: new OIMSetBucketContentService<TOIMEntitySlot<Card>>() });
const byDeck = byDeckService.createStoreSlot(db);
const deckContentService = byDeckService.slotContentService;

A bucket holds bucket member slots of any slot type — entity slots, bucket slots, computeds — and the bucket's type parameter is the type of its bucket member slots, TBucketMemberSlot extends TOIMSlot (default TOIMEntitySlot).

A bucket collection is two stateless objects, like an entity collection:

  1. How a bucket holds its slots, and in what order — the bucket content service, the strategy: OIMPositionalArrayBucketContentService (the default), OIMSetBucketContentService, or the sorted one an index builds for itself. It has every write and member read of a bucket.
  2. Where its buckets live — the collection service over it, OIMSlotCollectionService, with an ordinary slot store service (a Map store unless you pass bucketSlotStoreService). A store is chosen by how something is addressed and knows nothing about what it holds.

The bucket store slot carries the channel slot, so who is told is decided at createStoreSlot(db).

The options of createOIMSlotCollectionService(options?):

OptionDefaultWhat it decides
slotContentServicenew OIMPositionalArrayBucketContentService({ bucketCommands: new OIMBucketCommands({ maxCommandsPerSlot }) })how a bucket holds its slots
maxCommandsPerSlotOIM_MAX_COMMANDS_PER_SLOTthe recorder of the default content service — see the commands
bucketSlotStoreServicea Map store; required when TKey is an array (an OIMSlotTreeStoreService), because a Map store compares arrays by referencewhere buckets are filed
slotReleasePolicydefaults.slotReleasePolicy, then OIMSlotAtEmptyReleasePolicy.INSTANCE — an unheld bucket stays while it has members and goes once it is emptywhether and when an unheld bucket goes
columnSlotStoreService / columnNamesa shared map-driven storewhere bucket column slots live

createStoreSlot(db) creates one bucket store slot: one keyed store of buckets. A door that works by key or over the whole store takes that store slot first and is the collection service's; a door that works on one bucket you hold takes only the bucket slot and is the content service's.

The content service is typed by the bucket member slot type it holds, so pairing it with a bucket of another bucket member slot type is a compile error rather than a silent misread.

One content service instance may serve several bucket collections. It holds no per-bucket state — the bucket does.

The buckets are the collection's​

Asks for one take the bucket store slot first:

declare const key: TOIMStoreKey;

byDeckService.holdSlotByKey(byDeck, key); // the bucket; holdCount up by one
byDeckService.getSlotByKey(byDeck, key); // TOIMBucketSlot, never undefined; mints on a miss and the
// unused-slot collector holds it for this turn
byDeckService.peekSlotByKey(byDeck, key); // the bucket or undefined; never mints
byDeckService.hasSlotContentByKey(byDeck, key); // a bucket is filed there and holds at least one slot
byDeckService.getSlots(byDeck);
byDeckService.getSlotCount(byDeck);

Hold a bucket when you will keep using it across frames (a subscription, a component); getSlotByKey is for a read inside this turn.

Writing a bucket's slots​

const red = byDeckService.holdSlotByKey(byDeck, 'red');
const cardSlot1 = cardService.getSlotByKey(cards, 'c1');
const cardSlot2 = cardService.getSlotByKey(cards, 'c2');
const cardSlot3 = cardService.getSlotByKey(cards, 'c3');

red.writeSlotContent([cardSlot1, cardSlot2]); // the whole list, stated
red.addBucketMemberSlots([cardSlot3]); // this one joins, nothing else moves
red.removeBucketMemberSlots([cardSlot1]); // this one leaves, nothing else moves
red.clearSlotContent(); // everyone out

red.unholdSlot();

addBucketMemberSlot / removeBucketMemberSlot are the one-slot forms. Apart from the whole-list writeSlotContentByKey, a write by key resolves the bucket first (getSlotByKey mints it, peekSlotByKey does not). A change that matters only to a bucket somebody already asked for goes through peekSlotByKey and does nothing on undefined.

Which door: you hold the answer, or you hold the delta​

writeSlotContent is for a caller who has the list that should be there and does not know what changed: a search result, a page from a server, a filter re-run. One call, and the engine works out the arrivals and the departures for you.

Its cost is proportional to what the bucket holds, not to the change. The diff builds a set over the whole incoming list and walks the standing one, so a batch of ten into a bucket of two hundred pays for two hundred and ten.

addBucketMemberSlots and removeBucketMemberSlots are for a caller who knows the delta — an entity arrived over a socket, a card was dragged in, an infinite list appended a page, an entity was deleted. The standing slots keep the holds they already have. Both wake subscribers once, and neither wakes anybody when nothing actually joined or left.

Adding costs what you name. Removing from a set-driven bucket costs what you name; removing k slots from an ordered bucket of n is one pass over the list, n + k. Removing them one call at a time instead is up to k·n. The remove commands it records replay in order to the same list.

addBucketMemberSlots removes nothing. If the authoritative list shrank and you call addBucketMemberSlots, the slots that left stay in the bucket, held. Not knowing what left is exactly when writeSlotContent is the right door; its cost is the price of not knowing.

writeSlotContent(bucketSlot, slots) returns true when anything arrived or departed. It:

  • is idempotent. Re-sending an unchanged list evicts nothing and wakes nobody.
  • follows the strategy. A set-driven bucket takes the set difference. An array-driven bucket compares the sequence: the same slots in the same order change nothing, and any other sequence, a reordered one included, is written whole, because the sequence is what that bucket holds.
  • keeps the slots both lists name. A slot in the old and the new list keeps its identity and its subscribers through the write.

clearSlotContent on an empty bucket records nothing and wakes nobody.

A slot in a bucket is a counted reference, a hold. Joining a bucket takes a hold on the slot; leaving gives it back. So a bucket you never emptied keeps its slots alive, and slot.clearSlotContent() does not take the slot out of your buckets. It takes the value away and leaves a valueless slot in the bucket, just like the valueless slots a bucket holds when it was resolved before its entities arrived.

Reading a bucket's slots​

const bucket = byDeckService.holdSlotByKey(byDeck, 'red');

for (const cardSlot of bucket.getBucketMemberSlots()) { render(cardSlot.entity); }
bucket.getBucketMemberSlotCount(); // read without the walk
bucket.hasBucketMemberSlot(cardService.getSlotByKey(cards, 'c1'));

A write followed by a read with no flush between them hands back this frame's slots, in the order the caller wrote them. A bucket never sorts.

The slots come back live: the Set itself, or the list itself. See the rule that comes with "live". Iterate the slots when you render each one through its own subscription; copy them with Array.from for a one-off read. The entities of an index bucket by key are indexService.getEntitiesByKey.

Subscribing to a bucket is an ordinary effect:

const bucket = byDeckService.holdSlotByKey(byDeck, 'red');

effectService.createAndRunEffect(db, () => { render(bucket.getBucketMemberSlots()); }, [bucket]);

The bucket slot is written when its membership or its order changes, and only then.

How a bucket holds its slots: the content services​

Content serviceStorageWhat one add recordsTake it when
OIMSetBucketContentService({ bucketCommands? })Setone entry per slot; a slot named twice is oneorder does not matter, or a sorted view will impose one
OIMPositionalArrayBucketContentService({ bucketCommands? })listone per mention; a slot named twice is twothe caller's order is part of the answer, and the list may repeat a slot (a playlist) or the caller never repeats one
OIMBucketUniqueMemberSlotsArrayService({ bucketCommands? })list + Setone per slot, order keptthe caller's order is part of the answer, and a position must name exactly one entity; a repeated add or insert is refused

The array-driven service is the default of createOIMSlotCollectionService (with a recorder) and of createOIMSlotCollectionIndexService (without one). The set-driven service's membership test and removal are a hash lookup. It keeps insertion order for iteration, but has no positions: it has no insertBucketMemberSlotAt, removeBucketMemberSlotAt, removeBucketMemberSlotRange or moves, and no getBucketMemberSlotAt / getBucketMemberSlotPosition either — asking a set for a place does not compile, so nothing walks the set to answer it.

The array-driven service is what makes positions the caller's: nothing re-decides the order on a read, and the position doors are the whole of how the list moves. Its membership test and a removal of one named slot scan the list.

And what naming a slot records: one per mention or one per slot​

new OIMPositionalArrayBucketContentService<TOIMEntitySlot<Card>>({
bucketMemberSlotsArrayService: new OIMBucketUniqueMemberSlotsArrayService<TOIMEntitySlot<Card>>(),
});

The member slots service is the only writer of an array bucket's list and the Set beside it, so the two never disagree. With the unique one the membership test is a Set lookup; with the default it scans the list.

The bucket's own type says which storage it carries, so the pairing is checked:

TOIMBucketSlot<TOIMBucketSlotsSetStore<TOIMEntitySlot<Card>>> // OIMSetBucketContentService
TOIMBucketSlot<TOIMBucketSlotsArrayStore<TOIMEntitySlot<Card>>> // OIMPositionalArrayBucketContentService

Handing a set-driven bucket to the array-driven service is a compile error. The content service you do not import stays out of your bundle.

Positions the caller decides​

Take this when the order is the user's: a kanban column, a playlist, a priority queue someone reorders by hand. createOIMSlotCollectionService() builds an array-driven bucket collection with a recorder, and you decide the positions:

Setup shared by the examples on this page
const pageService = createOIMSlotCollectionService({ slotContentService: new OIMPositionalArrayBucketContentService<TOIMEntitySlot<Card>>({ bucketCommands: new OIMBucketCommands({ maxCommandsPerSlot: undefined }) }) });
const page = pageService.createStoreSlot(db);
const pageContentService = pageService.slotContentService;

The doors​

An index is only valid until the next write; a slot is stable across frames. Reach for a position when the position is what you already have (a drag handle reports one), and for addBucketMemberSlots / removeBucketMemberSlots / writeSlotContent otherwise. The two reads are on the array-based content services (array-driven and sorted), the five writes on the array-driven one only; a set-driven bucket has no positions. Each takes the bucket slot and no store slot.

DoorWhat it does
getBucketMemberSlotAt(bucketSlot, index)who sits there, undefined past the end
getBucketMemberSlotPosition(bucketSlot, slot)where it sits, -1 when it does not
insertBucketMemberSlotAt(bucketSlot, at, slot)puts it at at, clamped to the ends of the list
removeBucketMemberSlotAt(bucketSlot, at)takes it out and hands it back, undefined past the end
removeBucketMemberSlotRange(bucketSlot, at, count)takes a run out and hands it back; a run past the end stops at the end
moveBucketMemberSlotToPosition(bucketSlot, from, to)moves one slot and hands it back; refuses from === to and either end out of range
moveBucketMemberSlotRangeToPosition(bucketSlot, from, count, to)moves a run, to a landing read in the list without the run in it, clamped

Every position and count must be a whole number: NaN or a fraction throws before anything moves. ±Infinity passes that check and is treated as any other out-of-range position by the door it reaches: clamped by an insert and by a range move's landing, refused by removeBucketMemberSlotAt and moveBucketMemberSlotToPosition.

The array-driven service records one place per mention, so insertBucketMemberSlotAt of a slot the bucket already holds gives it a second position. Take OIMBucketUniqueMemberSlotsArrayService when a position must name exactly one entity.

The recorded insert names the index the slot landed on (at clamped to the ends of the list); a repeat on OIMBucketUniqueMemberSlotsArrayService inserts and records nothing (OIM_NOTHING_WAS_INSERTED, -1). On the array-driven service the landing can differ from getBucketMemberSlotPosition, which names a repeated slot's first position.

Per strategy​

A kanban column. One bucket per column; a card dragged between columns:

const fromColumn = pageService.holdSlotByKey(page, 'todo');
const toColumn = pageService.holdSlotByKey(page, 'done');
const card = cardService.getSlotByKey(cards, 'c1');
const cardItWasDroppedOn = cardService.getSlotByKey(cards, 'c2');

pageContentService.removeBucketMemberSlotAt(fromColumn, pageContentService.getBucketMemberSlotPosition(fromColumn, card));
pageContentService.insertBucketMemberSlotAt(
toColumn, pageContentService.getBucketMemberSlotPosition(toColumn, cardItWasDroppedOn) + 1, card,
);

Dragged inside one column it is one moveBucketMemberSlotToPosition.

A priority queue the user reorders. Raising to the front is a literal position; nudging is relative to where the entity says it is:

const bucket = pageService.holdSlotByKey(page, 'queue');
const job = cardService.getSlotByKey(cards, 'c1');

pageContentService.moveBucketMemberSlotToPosition(bucket, pageContentService.getBucketMemberSlotPosition(bucket, job), 0);
pageContentService.moveBucketMemberSlotToPosition(
bucket, pageContentService.getBucketMemberSlotPosition(bucket, job),
pageContentService.getBucketMemberSlotPosition(bucket, job) + 2,
);

A whole list handed in at once — a server response, a restored draft. writeSlotContent replaces the lot:

declare const slotsInTheOrderTheServerSent: readonly TOIMEntitySlot<Card>[];

const bucket = pageService.holdSlotByKey(page, 'results');
bucket.writeSlotContent(slotsInTheOrderTheServerSent);

A reorder a renderer has to animate. Read the commands, not the list — below.

The commands, on the bucket slot​

OIMBucketCommands records onto the bucket slot what this frame's writes DID. A frontend rendering a reorder needs exactly that: the reorder commands, not "something changed", because a rebuild destroys the node identity that an animation, a focus and a scroll anchor are attached to. Take it for a positional list a renderer patches; leave it out for a bucket that is only iterated.

The recorder is chosen when the content service is built, as bucketCommands in its options (new OIMBucketCommands({ maxCommandsPerSlot })). It records for the buckets of every store slot that content service serves. A content service built with new and no bucketCommands records nothing, so a bucket nobody renders positionally allocates no command. Nothing has to be disposed.

const bucket = pageService.holdSlotByKey(page, 'results');

effectService.createEffect(db, () => {
if (bucket.isWholeListRecorded) rebuildFrom(bucket.getBucketMemberSlots());
else applyToTheRenderedList(bucket.commands ?? []);
}, [bucket]);

The content service already writes the bucket slot on every mutation, so a consumer subscribes to the bucket. TOIMBucketSlot carries two fields for it, and nothing else a consumer reads:

FieldWhat it says
commandsreadonly TOIMOrderedListCommand[], or undefined before the first write — what this frame's writes did, in order
isWholeListRecordedthe script was given up on; read the bucket's slots and rebuild

Read both inside the frame. They are reset once the flush has fully ended, so they exist only in an effect body or a subscribeAfterEachFrame handler; after db.flush() returns, commands is empty (or undefined) and isWholeListRecorded is false. There is no "commands since frame N": a consumer that did not read this frame's script reads the list instead.

OIMBucketCommandsCursor does that bookkeeping for one view of one bucket. takeBucketCommands() returns what the view has not applied yet: one reset with every member on the first take, after a whole-list record and on every wake of a bucket that records no commands; otherwise this frame's commands, and only their tail when the view already took part of the same script earlier in the frame.

const resultsBucket = pageService.holdSlotByKey(page, 'results');
const bucketCommandsCursor = new OIMBucketCommandsCursor({ bucketSlot: resultsBucket });
const watchDisposer = resultsBucket.watch(() => { applyToTheRenderedList(bucketCommandsCursor.takeBucketCommands()); });

A positional write is always recorded as a position, and addBucketMemberSlots as an insert at the end. isWholeListRecorded is set by a write of the whole bucket — writeSlotContent, clearSlotContent, moveBucketMemberSlotRangeToPosition — by a removal from a bucket whose content service keeps no positions, which has no index to patch, and by the script outgrowing the list it describes. maxCommandsPerSlot is the commands-per-slot ratio above which that happens; it defaults to OIM_MAX_COMMANDS_PER_SLOT, which is 1, because replaying a script longer than the list it describes costs more than replacing the list. A consumer animating moves cannot use a whole-list answer at all, since every move it was going to animate is gone with it. So it passes Infinity and is handed every command, however long the script runs.

A single bucket​

One bucket under no address: the current search result, the visible window, a selection. Take it for a hand-curated list that nothing looks up by key; take a keyed bucket collection as soon as there is more than one such list and a key tells them apart.

A single bucket is a store slot of createOIMSingleSlotService(options?), an OIMSingleSlotService over a bucket content service with OIMSingleSlotStoreService.INSTANCE as its store. Its options are those of createOIMSlotCollectionService without a store. createStoreSlot(db) mints nothing; its one bucket slot is read through the single doors: getSingleSlot(storeSlot) hands it out (minting it on the first read, or a fresh one if the last was released, and the unused-slot collector holds that mint until the next flush), holdSingleSlot(storeSlot) hands it out held, and peekSingleSlot(storeSlot) answers it without minting.

const searchResultService = createOIMSingleSlotService({ slotContentService: new OIMPositionalArrayBucketContentService<TOIMEntitySlot<Card>>({ bucketCommands: new OIMBucketCommands({ maxCommandsPerSlot: undefined }) }) });
const searchResult = searchResultService.createStoreSlot(db);
const searchResultSlot = searchResultService.holdSingleSlot(searchResult);

searchResultSlot.writeSlotContent([cardService.getSlotByKey(cards, 'c1')]);
effectService.createEffect(db, () => {
render(Array.from(searchResultSlot.getBucketMemberSlots()));
}, [searchResultSlot]);

Every door of the content service takes the single bucket slot as it takes a keyed one, and so does the util markOIMBucketColumnsChanged(bucketSlot, patch, changedColumns). Nothing marks a single bucket's columns for you: whoever changes its members calls markOIMBucketColumnsChanged, as the index does for its buckets.

The single bucket slot is what an effect subscribes to. It lives under the release policy like any bucket: while an effect or a hold keeps it, it stays; once it is unheld and empty under the default OIMSlotAtEmptyReleasePolicy it is released, and the next getSingleSlot or holdSingleSlot mints a fresh, empty one. Pass OIMSlotNeverReleasePolicy.INSTANCE as slotReleasePolicy for a bucket slot that is never released.

Rendering an ordered bucket​

getOIMOrderedListCommands(prev, next, maxCommandsPerSlot?) turns "what I last rendered" into "what the bucket holds now" as a command list, so a consumer patches instead of rebuilding. Take it when the list was not written through a content service with a recorder — a sorted computed, an index bucket, a list you built yourself — or when the consumer was not there for the frames in between; a positional bucket with a recorder already hands you the script.

declare const whatIRendered: readonly TOIMEntitySlot<Card>[];

const bucket = orderedDeckService.holdSlotByKey(orderedDecks, 'red');
const now = Array.from(bucket.getBucketMemberSlots());

for (const command of getOIMOrderedListCommands(whatIRendered, now)) {
switch (command.type) {
case 'insert': /* command.index, command.slot */ break;
case 'remove': /* command.index */ break;
case 'move': /* command.from, command.to */ break;
case 'reset': /* command.slots */ break;
}
}

whatIRendered must be a snapshot you took where you rendered, never the value getBucketMemberSlots handed you. getBucketMemberSlots returns the bucket's live list, which is why it is typed Iterable and not an array. A prev kept live is the same array as next on the following frame, so the diff comes back empty, no reorder is ever drawn, and nothing throws. Keep the Array.from(...) copy, or subscribe to createOIMBucketMemoComputedService, whose value is a fresh array on every write that changed the list and the same array while it did not.

A list may name one slot twice — the array-driven content service records every mention. The diff matches the mentions of one identity in the order they appear, and the script reproduces the new list mention for mention. Its minimal move count is claimed for lists whose identities are distinct.

Its third argument, maxCommandsPerSlot, is the commands-per-slot-of-next ratio above which it returns one reset instead of the script — the same ratio OIMBucketCommands takes as maxCommandsPerSlot. Left out it never returns a reset at all, which is the arm an animating consumer wants.

It is the diff itself and does not subscribe for you: read the two lists inside your own effect and call this.

Giving a bucket store slot back​

A bucket collection has no handle to give back, and neither has a bucket store slot. Each bucket is held by whoever subscribed to it or put slots in it, and it goes when the last of those lets go. A caller who means to empty a bucket store slot clears each bucket in it:

for (const bucket of Array.from(byDeckService.getSlots(byDeck))) bucket.clearSlotContent();

Where to next​