9.1.2.2 Log File: Feather Format

Feather (Apache Arrow) is a format available for storing logged data into a file.

Feather data is ZSTD compressed and is directly consumable by several external tools and scripting languages such as Python (see example below.) For more information see arrow.apache.org/ and, for example, arrow.apache.org/cookbook/py/io.html#reading-a-feather-file for Python examples.

Example 2: Dump of a Feather file using feather-check.py.

c:\temp>python feather-check.py DvrtProtocol-1.feather
Length: 2
Reading DvrtProtocol-1.feather as pyarrow table
pyarrow.Table
timestamp: double not null
demo_uint8: uint8 not null
demo_float: float not null
----
timestamp: [[4458.8244819,4458.9324957,4459.0405149,4459.1484745,4459.256488,...,4482.0444293,4482.1524266,4482.260458,4482.3684368,4482.4764697]]
demo_uint8: [[151,161,172,182,192,...,9,19,29,39,49]]
demo_float: [[1617665.8,1617678.2,1617692,1617704.5,1617717,...,1620368.2,1620380.8,1620393.2,1620405.8,1620418.2]]
Total number of rows: 219

feather-check.py

import sys
import pyarrow.feather as pf

print("Length: " + str(len(sys.argv)))

if (len(sys.argv) < 2):
    print("Usage: feather-check.py <file_name>")
    sys.exit(1)

fileName = sys.argv[1]
print("Reading " + fileName + " as pyarrow table")

table = pf.read_table(fileName)

print(table)
print("Total number of rows: " + str(len(table)))

Timestamps describe the time since the data visualizer backend process was started, in seconds. They are represented as 64-bit floating point data in the Feather format.

"data-visualizer": {"encoding": {"version": 1, "type": "basic"}, "time-origin-ms-since-epoch": 1731113302508, "timestamp": {"format": "integer", "units": "seconds", "encoding": "relative", "resolution": 1e-06}} 

"data-visualizer": {"timestamp": { "format": "integer", "units": "seconds", "encoding": "relative", "resolution": 1e-06}}

"data-visualizer": {"units": "V" }