2.2.5 Embedded SDK

Sensor Transforms (se_tr_xxx.c)

• Takes one frame of sample(s) at a time and a pointer to the end of the FrameIDX array

• Performs an operation, such as magnitude, and stores it into the FrameIDX array

• Returns the number of points added to FrameIDX array

Sensor Filter (se_filter_xxx.c)

• Takes one frame of sample(s) at a time and a pointer to the FrameIDX array

• Performs an operation on the incoming sample to modify it in some way, i.e., moving average filter

• If it needs to, stores information in an internal ring buffer about other data it needs to use

• Always returns time synchronized data

• Returns -1 while it waits for more data; otherwise, it returns 1 when it has enough data to perform the filter operation

Segmentation Algorithm (sg_xxx.c)

• Must have either window size or max buffer length defined as part of the algorithm. This is used to define the buffer size of the circular buffer.

• Must make use of the ring buffer without modification to input data or data inside of the ring buffer

• All parameters must be passed through a struct seg_params. All user-facing parameters must be defined in the Python input contract with a corresponding c_param:index.

• All flags must be part of seg_params as multiple segmenters can use the same segmenter

• Must have an init function that describes resetting the ring buffer after a segment is found

• Returns 1 if a segment is found; otherwise, returns 0

Segment Filter (sg_filter_xxx.c)

• Returns ‘1’ or ‘0’. ‘1’ if the pipeline continues; ‘0’ if the pipeline should be terminated for this segment.

Segment Transforms (sg_tr_xxx)

• Operates a segment of data

• Can modify data in the ring buffer but is not allowed to add new data to the ring buffer or add data to an additional ring buffer

  Note: This can potentially cause issues. A method to address ring buffer issues is currently unavailable.

• Example is tr_strip, which demeans the data in the ring buffer

Feature Generators (fg_xxx.c)

• Create a single or multiple features from a segment of data

• Adds a feature as a float to the feature vector stored in kb_model

• Returns the number of features added to the feature vector

Feature Transforms (fg_tr_xxx.c)

• Takes a pointer to the feature generator and feature parameters

• Operates on the feature vectors in place

Recognition

• Takes a feature vector as input

• Returns a classification result

Knowledge Pack Overview

The core of the Embedded SDK is the kb_model_t structure that stores information for each model. Thekb_model_t defines the overall state machine. The pointer to this object passes through each function then acts on it to change/advance its state. The definition for this function is shown below, along with definitions for each of the parameters.

typedef struct{
    int framelen;            // Number of columns in the ring buffer this pipeline is associated with
    int sframelen;            // Number of columns in the streaming data ring buffer this pipeline is associated with
    int sg_length;           // This is the length of an identified segment of data.
    int sg_index;            // This is the position of the start of the index in the ring buffer
    int last_read_idx;       // This is the index of the last read value from the rb
    uint8_t parent;              // index of the parent model
    uint8_t feature_bank_index;   // index of the head of the feature bank
    uint16_t feature_bank_size;    // number of features in a bank
    uint8_t feature_bank_number;  // number of feature banks
    uint16_t feature_vector_size;  // The size of the feature vector
    uint8_t *feature_vector;  // pointer to the current feature vector
    uint8_t classifier_type; // This is the classifier type that the model uses
    uint8_t classifier_id;   // context to use when submitting this models feature vector
    seg_params *psegParams;  // Pointer to the segmenter data structure
    FVCOMP_T *pFeatures;     // pointer to the feature vector array
    ring_buffer_t *pringb;   // pointer to the ring buffer
    ring_buffer_t *psringb;  // Pointer to the streaming data buffer
    SENSOR_DATA_T *frameData; // pointer to array containing the most recently added data sample (placed here then into the ring buffer)
                                // (Note: we could probably just add directly to the ring buffer instead of doing this copy in the future)
    model_feature_gen_f feature_gen; // Pointer to the feature generator function
    model_data_streaming_f data_streaming; // Pointer to the streaming data function
    model_data_segmentation_f segmentation; // Pointer to the data segmentation function
    model_recognize_vector_f recognize_vector; // Pointer to the feature vector classifier function
    model_feature_transform_f feature_transform; // Pointer to the feature transform function
    uint8_t *model_uuid; //Contains the 16-bytes used to identify the model in the cloud.
    int total_classifications; //Contains the total classifications run on the system per power cycle.
    kb_profile_t m_profile; //model profiler
} kb_model_t;

The following describes how kb_models are actually used. There are two main types of APIs for data running pipelines:

1. Passing a single sample at a time using a run model

2. Passing a buffer segment, then calling run segment

These functions return -1 when there is no classification yet (could be waiting for more data or something else). Other error codes can be returned as other negative numbers. Set up a specific ENUM for these to be defined.

These return 0 for unknown and 1, 2, 3… for the class. By using a combination of the model_index and the class, a user is able to map back to what the actual class represents.

This starts with the kb_run_model API, which is the simplest entry point into running a model. This takes a piece of sensor data and the model_index to associate that data with.

int kb_run_model(SENSOR_DATA_T *pSample, int nsensors, int model_index)
{
    int ret = -1;
    if (kb_data_streaming(pSample, nsensors, model_index))
    {
        if (kb_segmentation(model_index) == 1)
        {
            kb_feature_generation_reset(model_index);
            if (kb_feature_generation(model_index) == 1)
            {
                ret = kb_generate_classification(model_index);
                kb_feature_generation_increment(model_index);
                return ret;
            }
            else
            {
                kb_reset_model(model_index);
                return -2;
            }
        }
    }
    return -1;
}

The model then goes through four main stages:

• kb_data_streaming: Handles streaming data that comes in. Returns -1 or 1

• kb_segmentation: Looks for segments of data and returns 1 or -1

• kb_feature_generation: Generates features for a specified model

• kb_generate_classification: Recognizes the feature vector stored in the model

All of these functions are defined in kb.c and follow a similar pattern. One is described in more detail here to help you understand the basic design philosophy. You can look at others in the kb.c file for your own understanding.

The function kb_recognize takes the model_index. Then, call the kb_feature_transform function, which takes the generated feature vector and prepares it to be passed to the trained classifier algorithm. When callingkb_recognize, the prepared feature vector is fed to the classifier algorithm. This returns an int value. 65535 is checked for, which is a reserved pattern for Unknown.

int kb_generate_classification(int model_index)
{
    int ret = -1;
    if (kb_feature_transform(model_index) == 1)
    {
        ret = kb_recognize(model_index);
        if (ret == 65535)
        {
            ret = 0;
        }
    }

    return ret;
}