Handling Structs, Bitfields and Arrays

Structures underpin the representation of custom data types and provide logical grouping and syntactical sugar to developers. Structures or custom types allow developers to pass more complex data between functions and semantically improve variable access with dot notation. Use of these language features is a vital part of serious software projects.

If you haven't added some structured data to your firmware yet, follow through the hardware focussed part of this guide:

User Interface Codecs

With our structure setup on the embedded side, we need to teach the UI how to read inbound packets and correctly form data into an outbound packet.

Codecs consist of a filter to catch relevant packets, a decoder to convert the inbound bytestream into a Typescript object, and an encoder to convert a Typescript object back into a buffer of bytes for hardware.

The default template generated by arc has a demonstration codec setup in the src/transport-manager/config/codec.tsx file.

Setting up a Codec

Defining a Custom Type

The guide follows best-practices and defines a LEDSettings type in the Typed State file to arrange the UI side data. Strictly typed objects allow Electric UI to generate 'compile time errors' if misused in the frontend code.

The typedState.ts type is just a pair of number values (the order doesn't matter):

tsx
export type LEDSettings = {
enable: number
glowTime: number
}

Make sure you import the LEDSettings type in the codecs.tsx file!

Codec Structure

tsx
import { Codec } from '@electricui/core'
import { smart-buffer } from 'SmartBuffer'
import { LEDSettings } from '../../application/typedState'
tsx
export class LEDCodec extends Codec<LEDSettings> {
filter(message: Message): boolean {
return message.messageID === 'led'
}
encode(payload: LEDSettings): Buffer {
// ...
}
decode(payload: Buffer): LEDSettings {
// ...
}
}
// Create the instances of the codecs
export const customCodecs = [new LEDCodec()]

Encoders and decoders are pure functions that convert an object to and from a Buffer.

It's important that you remember to add a new instance of the codec to the customCodecs export at the bottom of the file. This is used to streamline the inclusion of your codec with Electric UI's data/communications handling pipeline.

Codecs are added as part of the communications pipeline for each transport in use. In serial.ts, you'll see a block of code which creates the pipeline, adds the default codecs (used for integer/float/callback etc), and then adds the custom codecs with .addCodecs()as shown:

tsx
import { customCodecs } from './codecs'
tsx
// Halfway down, inside the serialTransportFactory
const codecPipeline = new CodecDuplexPipeline()
codecPipeline.addCodecs(customCodecs)
codecPipeline.addCodecs(defaultCodecList)

This approach can be modified to add codecs spanning other files, i.e. create a complexCodec.tsx file, then import it and add it to the pipeline of your various transports (serial, websockets, bluetooth all have their own pipelines).

Error Handling

A codec needs both encode() and decode() methods. In an example situation where a microcontroller provides read-only sensor data, the encoder should throw an error when attempting to encode an outbound packet.

tsx
export class ReadOnlyCodec extends Codec<LEDSettings> {
filter(message: Message): boolean {
return message.messageID === 'led'
}
encode(payload: LEDSettings): never {
throw new Error('The led messageID is read only.')
}
decode(payload: Buffer): LEDSettings {
throw new Error('The led messageID decoder is unimplemented.')
}
}

Decoding

The decode function's payload argument is a NodeJS Buffer. The buffer can be read directly, but we recommend a helper library called SmartBuffer.

SmartBuffer provides a simple but powerful approach to manipulating buffers of bytes, as it automatically tracks the 'cursor' position as the raw bytes are read or written. The SmartBuffer methods follow the stdtype.h set used in electricui-embedded.

tsx
reader.readUInt8()
reader.readInt8()
reader.readUInt16LE()
reader.readFloatLE()

Little Endian and Big Endian functions are available for n-byte types (LE is most typical with microcontrollers).

These functions accept an optional offset argument which moves the start address of the decode in the buffer, useful for jumping over padding bytes.

In action, the decoder would be written like this:

tsx
export class LEDCodec extends Codec<LEDSettings> {
decode(payload: Buffer): LEDSettings {
const reader = SmartBuffer.fromBuffer(payload)
// Read out the enable boolean
const enable = reader.readUInt8()
// 'read' the padding byte to increment the cursor
reader.readUInt8()
// Read out the duration
const glowTime = reader.readUInt16LE()
// Build our object
const settings: LEDSettings = {
enable: enable,
glow_time: glowTime,
}
// Return the completed object, the type checking will
// validate that it matches the type of LEDSettings
return settings
}
}

For most structured data, a more concise approach performs the buffer read while creating the settings object:

tsx
const reader = SmartBuffer.fromBuffer(payload)
const settings: LEDSettings = {
enable: reader.readUInt8(),
glowTime: reader.readUInt16LE(2), // the offset must be passed as an argument
}
return settings

Encoding

The encoder works in reverse to the decoder implementation. SmartBuffer again has utility in automatically tracking the cursor. Defining the size during the SmartBuffer instantiation is optional.

tsx
export class LEDCodec extends Codec<LEDSettings> {
// ... filter
encode(payload: LEDSettings): Buffer {
// SmartBuffers automatically keeps track of read and write offsets / cursors.
const packet = new SmartBuffer({ size: 4 })
packet.writeUInt8(payload.enable)
packet.writeUInt8(0x00) // Write a padding byte.
packet.writeUInt16LE(payload.glow_time)
// Push the buffer up the pipeline
return packet.toBuffer()
}
// ... decoder
}

The null case is handled automatically:

  • Outgoing messages with a payload of null will have a zero length payload on the hardware side.
  • Incoming messages with a zero length payload will have a null payload on the UI side.

Parsing raw payload buffers

If you want to access data in the inbound buffer manually, just ignore SmartBuffer and access the payload argument directly. This can be desirable when decoding a packed bitfield.

Note the methods available behave slightly differently, no support for reading strings and no automatic reading cursor.

Below is a reimplementation of the decoder described above, note the lack of the reader.

tsx
export class LEDCodec extends Codec<LEDSettings> {
// ... filter, encoder
decode(payload: Buffer): LEDSettings {
const settings: LEDSettings = {
enable: payload.readUInt8(0), // the offset must be passed as an argument
glowTime: payload.readUInt16LE(2), // incremented offset to match the buffer position
}
return settings
}
}

Using structure elements with UI components

When using a structured variable with components we use standard accessor syntax.

tsx
// You can still poll the with normal messageID string syntax
<IntervalRequester variables={['led_state', 'led']} interval={250} />
tsx
<Printer accessor="lit_time" />
// becomes
<Printer accessor={ state => state.led.glow_time } />
tsx
<Switch
unchecked={0}
checked={1}
accessor="led_blink"
writer={(state, value) => (state.led_blink = value)}
/>
// uses the same syntax as any component using writer props,
<Switch
unchecked={0}
checked={1}
accessor={state => state.led.enable}
writer={(state, value) => (state.led.enable = value)}
/>

A slider has a little more syntax, as a slider supports multiple handles to mutate different structure members.

tsx
<Slider>
<Slider.Handle accessor="lit_time" />
</Slider>
// becomes
<Slider
writer={(state, values) => {
state.led.glow_time = values.duration
}}
>
<Slider.Handle name="duration" accessor={state => state.led.glow_time} />
</Slider>

Other Codec Tips

With the bulk of codecs explained, here are a few additional features and examples you might run into when developing your application!

Inline integer-enum translation

Often firmware developers will express operating states as an enum (which is an integer in memory).

Typescript enums allow similar functionality, referencing an integer state by an enum. As a simple demonstration of syntax:

tsx
export enum TEMPERATURE_ERROR {
NONE = 0,
NOT_CONNECTED,
LOW,
HIGH,
GENERIC,
}
// Usage
const none = TEMPERATURE_ERROR.NONE // 0
const high = TEMPERATURE_ERROR.HIGH // 3

The codec can simply pass the integer through to the UI element. The redux devtools and internal representation will be the integer, however the source code can reference the named member.

Note that this approach requires the integer value on the embedded side to match the integer value of the enum on the UI side.

tsx
export interface SensorMessage {
sensorError: TEMPERATURE_ERROR
temperature: number
}
export class SensorCodec extends Codec<SensorMessage> {
// ... filter
encode(payload: SensorMessage): Buffer {
const packet = new SmartBuffer()
packet.writeUInt8(message.payload.sensor_error)
packet.writeUInt8(message.payload.temperature)
return packet.toBuffer()
}
decode(payload: Buffer): SensorMessage {
const reader = SmartBuffer.fromBuffer(message.payload)
return {
sensorError: reader.readUInt8(),
temperature: reader.readuint8(),
}
}
}

When interacting with the enum in the user interface code, import the enum definition, and use it in the same manner.

tsx
import { TEMPERATURE_ERROR } from './../../transport-manager/config/codecs'
// Function which returns nicely formatted text based on sensor state
const PrettyTempErrorPrinter = () => {
const temp_state = useHardwareState(state => state.temp.sensorError)
let error_text: string = 'Invalid'
if (temp_state === TEMPERATURE_ERROR.NONE) {
error_text = 'Everything is OK!'
} else if (temp_state === TEMPERATURE_ERROR.NOT_CONNECTED) {
error_text = 'Please connect the temperature probe'
} else {
error_text = 'Warning. Temperature reading error.'
}
return <span>{error_text}</span>
}

Manipulating data in codecs

In special situations where data being passed to or from the UI needs additional conversions (for human readability, or otherwise).

While these transforms can be done in the UI's frontend code, its also possible to manipulate the data at the codec side. This makes the usage of the value more consistent across your UI components.

This approach can be abused to minimise data transfer, useful in larger arrays of data or in constrained bandwidth applications.

In this toy example, we'll convert a temperature value which is being transmitted as a single unsigned byte. In this case, we know the temperature conversion range falls within a -20°C to +60°C range.

( temp°C + 10 ) * 4

The general flow of data might look like this:

Value
Temperature Sensor2.60V
ADC Sample768 counts
Steinhart temperature conversion48.75°C
Integer value sent to UI48
A transformed value sent to UI235

By offsetting the temperature value and multiplying, we use more of the byte's usable range compared to using a simple uint8 converted temperature value. There is nearly zero cost to performing this kind of transform, but results in higher resolution data on the UI side, drastically improving the appearance of a graph for the same bandwidth.

Integer Value of 48Transformed Value of 235
Display Resolution1°C steps0.25°C steps
Display RangePositive temperatures onlySupports subzero
Why?Doesn't use most of the 8-bit rangeMaximises usage of the 8-bit range
Byte ReadabilityHuman readableNeeds transformation before display

In the decoder, just perform mathematical operations as you would normally.

tsx
export interface TemperatureSensors {
micro: number
motor: number
external: number
}
export class TempCodec extends Codec<TemperatureSensors> {
// ... filter, encoder
decode(payload: Buffer): TemperatureSensors {
const reader = SmartBuffer.fromBuffer(message.payload)
return {
micro: (reader.readUInt8() / 4) - 10,
motor: (reader.readUInt8() / 4) - 10,
external: (reader.readUInt8() / 4) - 10,
}
}

This same style of transform can occur in an encoder to transform a user configurable value back into the desired form for the microcontroller.

While this example is simplified, this kind of transform can be used to convert a RGB colour structure into HSV, transform a 8-bit PWM value to a 0-100% range duty cycle, or create an new watts member from incoming volts and current structure elements.

Bitfields

Bitfields are commonly used on embedded platforms to pack multiple settings into a single byte, saving space, or to load into a peripheral's register.

Below is how you have a bitfield of booleans.

tsx
import { Codec } from '@electricui/core'
import { Pack, Unpack, BitFieldObject } from '@electricui/utility-bitfields'
tsx
export interface BitFieldSettings extends BitFieldObject {
bitA: boolean
bitB: boolean
bitC: boolean
bitD: boolean
bitE: boolean
bitF: boolean
bitG: boolean
bitH: boolean
}
const bitFieldOrder = [
'bitA', // 0bX0000000
'bitB', // 0b0X000000
'bitC', // 0b00X00000
'bitD', // 0b000X0000
'bitE', // 0b0000X000
'bitF', // 0b00000X00
'bitG', // 0b000000X0
'bitH', // 0b0000000X
]
export class BitfieldCodec extends Codec<BitFieldSettings> {
filter(message: Message): boolean {
return message.messageID === 'bitfield'
}
encode(payload: BitFieldSettings): Buffer {
return Pack(payload, bitFieldOrder)
}
decode(payload: Buffer): BitFieldSettings {
// otherwise it's just a string, pass it on as is.
return Unpack(payload, bitFieldOrder)
}
}
export const customCodecs = [
// ...
new BitfieldCodec(),
]

Arrays

Arrays are easy to work with!

On the embedded side (assuming electricui-embedded) the size of the data structure is silently grabbed as part of the macros used in the eui_message_t array.

This means for any default typed array, the UI automatically handles the sizing and dissects the variable into elements.

For custom types, we recommend using SmartBuffer in a loop to collect each items of the array.

To use a specific element from the array in the UI

typescript
<Printer accessor={state => state.leds[3].glow_time} />