Persistence

Persistence

DataSources provide basic volatile persistence by default. The last 120k events will be stored in memory (~5MB of RAM) and will be queryable.

If you don't know what a DataSource is, read this guide first

MessageDataSources can have their default transport-manager side storage modified by setting the setPersistenceEngineFactory method on the QueryableMessageIDProvider.

Twoslash failure

Errors were thrown in the sample, but not included in an errors tag

These errors were not marked as being expected: 2554 2345 2339.
Expected: // @errors: 2554 2345 2339

Compiler Errors:

template/src/transport-manager/index.tsx
[2554] 8 : Expected 1-2 arguments, but got 0.
[2345] 11 : Argument of type 'ElectronIPCRemoteQueryExecutor<{}>' is not assignable to parameter of type 'MultiWritable<{}>'.
[2339] 20 : Property 'setPersistenceEngineFactory' does not exist on type 'QueryableMessageIDProvider<{}>'.

Raising Code:

## Code

'''ts
0 // @filename: template/src/transport-manager/index.tsx
1 import {
2   Event,
3   QueryableMessageIDProvider,
4   ElectronIPCRemoteQueryExecutor,
5 } from '@electricui/core-timeseries'
6 import { Message } from '@electricui/core'
7 import { timing } from '@electricui/timing'
8 import { deviceManager } from './config'
9 
10 const remoteQueryExecutor = new ElectronIPCRemoteQueryExecutor()
11 const queryableMessageIDProvider = new QueryableMessageIDProvider(
12   deviceManager,
13   remoteQueryExecutor,
14 )
15 // ---cut---
16 // template/src/transport-manager/index.tsx
17 import {
18   DataSource,
19   PersistenceEngineMemory,
20 } from '@electricui/core-timeseries'
21 
22 queryableMessageIDProvider.setPersistenceEngineFactory(
23   'speed',
24   (dataSource: DataSource) => {
25     // Use the in-memory persistence engine with a ceiling of 20k points
26     return new PersistenceEngineMemory(20_000)
27   },
28 )
'''

Note

As the PersistenceEngineMemory allocates storage for the message based on the ceiling value per-device, applications which handle frequently changing devices or large meshes should be aware of the memory impact of deep storage pools.

Allocated space = eventSize * maxItems * number of unique devices that have connected

If no persistence is desired, the PersistenceEnginePassthrough engine can be used which does no buffering.

Twoslash failure

Errors were thrown in the sample, but not included in an errors tag

These errors were not marked as being expected: 2554 2345 2339.
Expected: // @errors: 2554 2345 2339

Compiler Errors:

template/src/transport-manager/index.tsx
[2554] 8 : Expected 1-2 arguments, but got 0.
[2345] 11 : Argument of type 'ElectronIPCRemoteQueryExecutor<{}>' is not assignable to parameter of type 'MultiWritable<{}>'.
[2339] 20 : Property 'setPersistenceEngineFactory' does not exist on type 'QueryableMessageIDProvider<{}>'.

Raising Code:

## Code

'''ts
0 // @filename: template/src/transport-manager/index.tsx
1 import {
2   Event,
3   QueryableMessageIDProvider,
4   ElectronIPCRemoteQueryExecutor,
5 } from '@electricui/core-timeseries'
6 import { Message } from '@electricui/core'
7 import { timing } from '@electricui/timing'
8 import { deviceManager } from './config'
9 
10 const remoteQueryExecutor = new ElectronIPCRemoteQueryExecutor()
11 const queryableMessageIDProvider = new QueryableMessageIDProvider(
12   deviceManager,
13   remoteQueryExecutor,
14 )
15 // ---cut---
16 import {
17   DataSource,
18   PersistenceEnginePassthrough,
19 } from '@electricui/core-timeseries'
20 // template/src/transport-manager/index.tsx
21 
22 queryableMessageIDProvider.setPersistenceEngineFactory(
23   'speed',
24   (dataSource: DataSource) => {
25     // No storage, events are streamed through the entire system
26     return new PersistenceEnginePassthrough()
27   },
28 )
'''

Custom DataSources

DataSources can have their default storage modified by setting the setPersistenceEngineFactory method on the dataSource itself.

import {
DataSource,
PersistenceEngineMemory,
} from '@electricui/core-timeseries'
 
const customDataSource = new DataSource()
customDataSource.setPersistenceEngineFactory(
dataSource => new PersistenceEngineMemory(20_000),
)