2.4.3 Sensor Filters

Modifies a sensor in place and performs some sort of filtering (ie. moving average). This acts on a Sensor source that was either input or created using a sensor transform. This does not create a new source.

Moving Average Sensor Transform

Performs a symmetric moving average filter on the input column, creates a new column with the filtered data.

Parameters

  • input_data – Dataframe containing the time series data.

  • group_columns – columns to group data by before processing.

  • input_column – sensor stream to apply moving average filter on.

  • order (filter) – the number of samples to average to the left and right.

Returns

input data after having been passed through symmetric moving average filter

Streaming Downsample

Downsample the entire dataframe into a dataframe of size filter_length by taking the average over the samples within the filter length.

Parameters

  • input_data – dataframe

  • group_columns (a list) – List of columns on which grouping is to be done. Each group will go through downsampling one at a time

  • input_columns – List of columns to be downsampled

  • filter_length – Number of samples in each new filter length

Returns

The downsampled dataframe.

Return type

DataFrame

Streaming Downsample by Decimation

Decrease the sample rate by a factor of filter_length, this will only keep one samples for every length of the filter.

Parameters

  • input_data – dataframe

  • columns – List of columns to be downsampled

  • group_columns (a list) – List of columns on which grouping is to be done. Each group will go through downsampling one at a time

  • filter_length – integer; Number of samples in each new filter length

Returns

The downsampled dataframe.

Return type

DataFrame

Examples

>>> from pandas import DataFrame
>>> df = DataFrame([[3, 3], [4, 5], [5, 7], [4, 6], [3, 1],
                    [3, 1], [4, 3], [5, 5], [4, 7], [3, 6]],
                    columns=['accelx', 'accely'])
>>> df
Out:
       accelx  accely
    0       3       3
    1       4       5
    2       5       7
    3       4       6
    4       3       1
    5       3       1
    6       4       3
    7       5       5
    8       4       7
    9       3       6
>>> client.pipeline.reset()
>>> client.pipeline.set_input_data('test_data', df, force = True)
>>> client.pipeline.add_transform('Streaming Downsample by Decimation', params={'group_columns':[],                                                            'columns' : ['accelx', 'accely'],                                                            'filter_length' : 5 })
>>> results, stats = client.pipeline.execute()
>>> print results
    Out:
            accelx  accely
        0     3       3
        1     3       1
        2     3       6
High Pass Filter

This is a simple IIR Filter that is useful for removing drift from sensors by subtracting the attentuated running average.

\[y_{i}= y_{i-1} + \alpha (x_{i} - y_{i-1})\]
Parameters

  • input_data – Dataframe containing the time series data

  • input_columns – sensor streams to apply the filter to

  • alpha – attenuation coefficient

Returns

input data after having been passed through the IIR filter