datasourcex-util

Utility classes and functions commonly used in DataSource integrations, such as efficiently observable Maps, and stream transformations.

Key components

Observable maps and shared flows

ComponentDescription
FlowMap / MutableFlowMapA Map that is also observable via asFlow, asFlowWithState, and per-key valueFlow. Built with mutableFlowMapOf / toMutableFlowMap, or collected from an event stream with flowMapIn.
CompletingSharedFlow / MutableCompletingSharedFlowA SharedFlow variant that also propagates completion and error events to subscribers. Built with shareInCompleting.
SharedFlowCacheKeyed cache of SharedFlows, sharing one upstream collection per key and evicting a key once its upstream ends so the cache can't grow unbounded. Built with sharedFlowCache(...), or completingSharedFlowCache(...) for a CompletingSharedFlowCache that propagates completion/errors.
CompletingSharedFlowCache / LoadingCompletingSharedFlowCacheKeyed cache of CompletingSharedFlows, sharing one underlying collection per key.

Stores

ComponentDescription
FlowStore / MutableFlowStoreA store-backed map exposing a delta-only stream plus a read-through, Caffeine-bounded cache. Built with flowStore / flowStoreIn / mutableFlowStore.
AsyncFlowStore / AsyncMutableFlowStoreSuspending views of the stores whose reads/writes dispatch the store I/O themselves.
Store / StoreReader / StoreWriterThe SPI you implement to back a MutableFlowStore; mutations enlist on the caller's transaction and publish on commit.
TxContext / VersionedThe transaction handle a write enlists on, and a value paired with the store-assigned version.

Flow operators

OperatorDescription
bufferingDebounceBuffers elements until a quiet period elapses, then emits them as a List.
throttleLatestEmits at most once per interval, keeping the latest value and dropping older ones.
flatMapFirstApplies a function to the first element together with the entire upstream flow.
demultiplexByGroups elements by a key selector and processes each group's sub-flow.
retryWithExponentialBackoffRetries the upstream on error with an exponential delay between attempts.
castCasts a Flow<*> to a Flow<T>.
timeoutFirst / timeoutFirstOrNull / timeoutFirstOrDefaultErrors / emits null / emits a default if no first element arrives within a timeout.
materialize / dematerialize (and *Unboxed)Convert between flow values and ValueOrCompletion events.

Event models and folds

ComponentDescription
MapEvent / SimpleMapEvent / SetEvent / VersionedMapEventSealed event types describing map and set mutations (with or without old values / versions).
FlowMapStreamEvent / ValueOrCompletionMaterialised stream events: initial-state-plus-deltas, and value-or-terminal-signal.
runningFoldToMap* / runningFoldToSetFold a delta stream into a live Flow of map / set snapshots.
conflateKeys / toEventsCollapse per-key updates, and expand a collection stream into entry events.

Serialization

ComponentDescription
registerDataSourceSerializers / registerPersistentCollectionSerializersRegister Fory serializers for the event types and persistent collections.
DataSourceModule, registerDataSourceModule / addDataSourceModuleJackson 2 / Jackson 3 modules that serialize the event types without annotations.
Jackson2JsonHandler / Jackson3JsonHandlerJsonHandler implementations backed by a Jackson ObjectMapper.

DataSource and general utilities

ComponentDescription
AntPatternNamespaceA Namespace matching subjects by Ant-style patterns, with path-variable extraction.
SimpleDataSourceFactory / SimpleDataSourceConfigBuild a DataSource from a simplified config for tests and examples.
KLoggerAn slf4j Logger wrapper with lazily-evaluated message lambdas.
ReadWriteLockA non-reentrant suspending read/write lock.
withTimeoutA coroutine timeout that throws TimeoutException rather than a CancellationException.

See also

Participating in an application-owned jOOQ transaction:

Samples

val store =
    mutableFlowStore(
        JooqAccountStore(rootDsl),
        Caffeine.newBuilder().maximumSize(10_000),
        txContext = Configuration::asTxContext,
    )

// Install the publishing listener once on the DSLContext; transactions opened from it run the
// store's buffered commit/rollback actions in their commit/rollback callbacks.
val dsl =
    rootDsl
        .configuration()
        .derive(DefaultTransactionListenerProvider(FlowStorePublishingListener))
        .dsl()

withContext(Dispatchers.IO) {
  dsl.transaction { config ->
    val alice = store.get("alice", config) ?: Account("alice", 0)
    val bob = store.get("bob", config) ?: Account("bob", 0)
    store.put("alice", alice.copy(balance = alice.balance - 10), config)
    store.put("bob", bob.copy(balance = bob.balance + 10), config)
  }
}

Packages