Handling Structs, Bitfields and Arrays

Structures underpin the representation of custom data types and provide logical grouping and syntactical sugar to developers. Structures or custom types allow developers to pass more complex data between functions and semantically improve variable access with dot notation. Use of these language features is a vital part of serious software projects.

If you haven't added some structured data to your firmware yet, follow through the hardware focussed part of this guide:

User Interface Codecs

With our structure setup on the embedded side, we need to teach the UI how to read inbound packets and correctly form data into an outbound packet.

Codecs consist of a filter to catch relevant packets, a decoder to convert the inbound bytestream into a Typescript object, and an encoder to convert a Typescript object back into a buffer of bytes for hardware.

The default template generated by arc has a demonstration codec setup in the src/transport-manager/config/codec.tsx file.

Setting up a Codec

Defining a Custom Type

The guide follows best-practices and defines a LEDSettings type in the Typed State file to arrange the UI side data. Strictly typed objects allow Electric UI to generate 'compile time errors' if misused in the frontend code.

The typedState.ts type is just a pair of number values (the order doesn't matter):

export type LEDSettings = {
  enable: number
  glowTime: number
}

Make sure you import the LEDSettings type in the codecs.tsx file!

Codec Structure

import { Codec } from '@electricui/core'
import { smart-buffer } from 'SmartBuffer'
import { LEDSettings } from '../../application/typedState'

export class LEDCodec extends Codec<LEDSettings> {
  filter(message: Message): boolean {
    return message.messageID === 'led'
  }
  encode(payload: LEDSettings): Buffer {
    // ...
  }
  decode(payload: Buffer): LEDSettings {
    // ...
  }
}

Encoders and decoders are pure functions that convert an object to and from a Buffer.

Codecs are added as part of the communications pipeline for each transport in use. In serial.ts, you'll see a block of code which creates the pipeline, then adds the custom codecs with .addCodecs() as shown:

import { LEDCodec } from './codecs'

// Halfway down, inside the serialTransportFactory
const codecPipeline = new CodecDuplexPipelineWithDefaults()
// Create the instances of the codecs
const customCodecs = [
  new LEDCodec()
]
codecPipeline.addCodecs(customCodecs)

This approach can be modified to add codecs spanning other files, i.e. create a complexCodec.tsx file, then import it and add it to the pipeline of your various transports (serial, websockets, bluetooth all have their own pipelines).

Codec instances are created per transport so they can maintain state per connection (and thus per device).

Error Handling

A codec needs both encode() and decode() methods. In an example situation where a microcontroller provides read-only sensor data, the encoder should throw an error when attempting to encode an outbound packet.

export class ReadOnlyCodec extends Codec<LEDSettings> {
  filter(message: Message): boolean {
    return message.messageID === 'led'
  }
  encode(payload: LEDSettings): never {
    throw new Error('The led messageID is read only.')
  }
  decode(payload: Buffer): LEDSettings {
    throw new Error('The led messageID decoder is unimplemented.')
  }
}

Decoding

The decode function's payload argument is a NodeJS Buffer. The buffer can be read directly, but we recommend a helper library called SmartBuffer.

SmartBuffer provides a simple but powerful approach to manipulating buffers of bytes, as it automatically tracks the 'cursor' position as the raw bytes are read or written. The SmartBuffer methods follow the stdtype.h set used in electricui-embedded.

reader.readUInt8()
reader.readInt8()
reader.readUInt16LE()
reader.readFloatLE()

Little Endian and Big Endian functions are available for n-byte types (LE is most typical with microcontrollers).

These functions accept an optional offset argument which moves the start address of the decode in the buffer, useful for jumping over padding bytes.

In action, the decoder would be written like this:

export class LEDCodec extends Codec<LEDSettings> {
  decode(payload: Buffer): LEDSettings {
    const reader = SmartBuffer.fromBuffer(payload)
    // Read out the enable boolean
    const enable = reader.readUInt8()
    // 'read' the padding byte to increment the cursor
    reader.readUInt8()
    // Read out the duration
    const glowTime = reader.readUInt16LE()
    // Build our object
    const settings: LEDSettings = {
      enable: enable,
      glow_time: glowTime,
    }
    // Return the completed object, the type checking will
    // validate that it matches the type of LEDSettings
    return settings
  }
}

For most structured data, a more concise approach performs the buffer read while creating the settings object:

const reader = SmartBuffer.fromBuffer(payload)
const settings: LEDSettings = {
  enable: reader.readUInt8(),
  glowTime: reader.readUInt16LE(2), // the offset must be passed as an argument
}
return settings

Encoding

The encoder works in reverse to the decoder implementation. SmartBuffer again has utility in automatically tracking the cursor. Defining the size during the SmartBuffer instantiation is optional.

export class LEDCodec extends Codec<LEDSettings> {
  // ... filter
  encode(payload: LEDSettings): Buffer {
    // SmartBuffers automatically keeps track of read and write offsets / cursors.
    const packet = new SmartBuffer({ size: 4 })
    packet.writeUInt8(payload.enable)
    packet.writeUInt8(0x00) // Write a padding byte.
    packet.writeUInt16LE(payload.glow_time)
    // Push the buffer up the pipeline
    return packet.toBuffer()
  }
  // ... decoder
}

The null case is handled automatically:

• Outgoing messages with a payload of null will have a zero length payload on the hardware side.
• Incoming messages with a zero length payload will have a null payload on the UI side.

Parsing raw payload buffers

If you want to access data in the inbound buffer manually, just ignore SmartBuffer and access the payload argument directly. This can be desirable when decoding a packed bitfield.

Note the methods available behave slightly differently, no support for reading strings and no automatic reading cursor.

Below is a reimplementation of the decoder described above, note the lack of the reader.

export class LEDCodec extends Codec<LEDSettings> {
  // ... filter, encoder
  decode(payload: Buffer): LEDSettings {
    const settings: LEDSettings = {
      enable: payload.readUInt8(0), // the offset must be passed as an argument
      glowTime: payload.readUInt16LE(2), // incremented offset to match the buffer position
    }
    return settings
  }
}

Using structure elements with UI components

When using a structured variable with components we use standard accessor syntax.

// You can still poll the with normal messageID string syntax
<IntervalRequester variables={['led_state', 'led']} interval={250} />

<Printer accessor="lit_time" />
// becomes
<Printer accessor={ state => state.led.glow_time } />

<Switch
  unchecked={0}
  checked={1}
  accessor="led_blink"
  writer={(state, value) => (state.led_blink = value)}
/>
// uses the same syntax as any component using writer props,
<Switch
  unchecked={0}
  checked={1}
  accessor={state => state.led.enable}
  writer={(state, value) => (state.led.enable = value)}
/>

A slider has a little more syntax, as a slider supports multiple handles to mutate different structure members.

<Slider>
  <Slider.Handle accessor="lit_time" />
</Slider>
// becomes
<Slider
  writer={(state, values) => {
    state.led.glow_time = values.duration
  }}
>
  <Slider.Handle name="duration" accessor={state => state.led.glow_time} />
</Slider>

Other Codec Tips

With the bulk of codecs explained, here are a few additional features and examples you might run into when developing your application!

Inline integer-enum translation

Often firmware developers will express operating states as an enum (which is an integer in memory).

Typescript enums allow similar functionality, referencing an integer state by an enum. As a simple demonstration of syntax:

export enum TEMPERATURE_ERROR {
  NONE = 0,
  NOT_CONNECTED,
  LOW,
  HIGH,
  GENERIC,
}
// Usage
const none = TEMPERATURE_ERROR.NONE // 0
const high = TEMPERATURE_ERROR.HIGH // 3

The codec can simply pass the integer through to the UI element. The Device State and internal representations will be the integers, however the source code can reference the named member.

Note that this approach requires the integer value on the embedded side to match the integer value of the enum on the UI side.

export interface SensorMessage {
  sensorError: TEMPERATURE_ERROR
  temperature: number
}
export class SensorCodec extends Codec<SensorMessage> {
  // ... filter
  encode(payload: SensorMessage): Buffer {
    const packet = new SmartBuffer()
    packet.writeUInt8(message.payload.sensor_error)
    packet.writeUInt8(message.payload.temperature)
    return packet.toBuffer()
  }
  decode(payload: Buffer): SensorMessage {
    const reader = SmartBuffer.fromBuffer(message.payload)
    return {
      sensorError: reader.readUInt8(),
      temperature: reader.readuint8(),
    }
  }
}

When interacting with the enum in the user interface code, import the enum definition, and use it in the same manner.

import { TEMPERATURE_ERROR } from './../../transport-manager/config/codecs'
// Function which returns nicely formatted text based on sensor state
const PrettyTempErrorPrinter = () => {
  const temp_state = useHardwareState(state => state.temp.sensorError)
  let error_text: string = 'Invalid'
  if (temp_state === TEMPERATURE_ERROR.NONE) {
    error_text = 'Everything is OK!'
  } else if (temp_state === TEMPERATURE_ERROR.NOT_CONNECTED) {
    error_text = 'Please connect the temperature probe'
  } else {
    error_text = 'Warning. Temperature reading error.'
  }
  return <span>{error_text}</span>
}

Manipulating data in codecs

In special situations where data being passed to or from the UI needs additional conversions (for human readability, or otherwise).

While these transforms can be done in the UI's frontend code, its also possible to manipulate the data at the codec side. This makes the usage of the value more consistent across your UI components.

This approach can be abused to minimise data transfer, useful in larger arrays of data or in constrained bandwidth applications.

In this toy example, we'll convert a temperature value which is being transmitted as a single unsigned byte. In this case, we know the temperature conversion range falls within a -20°C to +60°C range.

( temp°C + 10 ) * 4

The general flow of data might look like this:

By offsetting the temperature value and multiplying, we use more of the byte's usable range compared to using a simple uint8 converted temperature value. There is nearly zero cost to performing this kind of transform, but results in higher resolution data on the UI side, drastically improving the appearance of a graph for the same bandwidth.

In the decoder, just perform mathematical operations as you would normally.

export interface TemperatureSensors {
  micro: number
  motor: number
  external: number
}
export class TempCodec extends Codec<TemperatureSensors> {
  // ... filter, encoder
  decode(payload: Buffer): TemperatureSensors {
    const reader = SmartBuffer.fromBuffer(message.payload)
    return {
        micro:    (reader.readUInt8() / 4) - 10,
        motor:    (reader.readUInt8() / 4) - 10,
        external: (reader.readUInt8() / 4) - 10,
    }
}

This same style of transform can occur in an encoder to transform a user configurable value back into the desired form for the microcontroller.

While this example is simplified, this kind of transform can be used to convert a RGB colour structure into HSV, transform a 8-bit PWM value to a 0-100% range duty cycle, or create an new watts member from incoming volts and current structure elements.

Bitfields

Bitfields are commonly used on embedded platforms to pack multiple settings into a single byte, saving space, or to load into a peripheral's register.

Below is how you have a bitfield of booleans.

import { Codec } from '@electricui/core'
import { pack, unpack, BitFieldObject } from '@electricui/utility-bitfields'

export interface BitFieldSettings extends BitFieldObject {
  bitA: boolean
  bitB: boolean
  bitC: boolean
  bitD: boolean
  bitE: boolean
  bitF: boolean
  bitG: boolean
  bitH: boolean
}
const bitFieldOrder = [
  'bitA', // 0bX0000000
  'bitB', // 0b0X000000
  'bitC', // 0b00X00000
  'bitD', // 0b000X0000
  'bitE', // 0b0000X000
  'bitF', // 0b00000X00
  'bitG', // 0b000000X0
  'bitH', // 0b0000000X
]
export class BitfieldCodec extends Codec<BitFieldSettings> {
  filter(message: Message): boolean {
    return message.messageID === 'bitfield'
  }
  encode(payload: BitFieldSettings): Buffer {
    return pack(payload, bitFieldOrder)
  }
  decode(payload: Buffer): BitFieldSettings {
    // otherwise it's just a string, pass it on as is.
    return unpack(payload, bitFieldOrder)
  }
}
export const customCodecs = [
  // ...
  new BitfieldCodec(),
]

Arrays

Arrays are easy to work with!

On the embedded side (assuming electricui-embedded) the size of the data structure is silently grabbed as part of the macros used in the eui_message_t array.

This means for any default typed array, the UI automatically handles the sizing and dissects the variable into elements.

For custom types, we recommend using SmartBuffer in a loop to collect each items of the array.

To use a specific element from the array in the UI

<Printer accessor={state => state.leds[3].glow_time} />

[^1]: Strict alignment architectures like SPARC don't allow it at all [^2]: ARM M0 cores are stricter about alignment than more recent ARM variants, which have less or no impact with poorly aligned data access.