Sensor Recognition¶

The Reef learns to tell the state of a machine or environment from a sensor signal - whether a motor is unbalanced, or whether a person is walking or has fallen. Record each state, train on the computer and deploy to the board; the board's NPU then decides four times a second. It all happens on IPRO AI Studio's Sensor page. The firmware has five simulated sensors, so before your own sensor is wired in, one Reef and one USB-C cable take you through the whole flow.

Prerequisites¶

• An IPRO7AI Reef EVK flashed with the Edge AI firmware; see Flash the Edge AI Firmware. Studio deploys the sensor model; nothing more to flash.
• A USB-C data cable from the Reef's USB-C to the computer.
• IPRO AI Studio installed; see Install IPRO AI Studio.
• No other hardware for the simulated sensors or the board's microphone. Your own sensor needs a data source added to the firmware first; see Add your own sensor.

How it works¶

• Samples from the data source (a simulated sensor, the microphone or your sensor) go into a 1 MB ring buffer on the board and stream unchanged to Studio over USB-C.
• Studio turns each piece of signal (a "window") into a 64×64×8 feature map, trains a whole small CNN on the computer, and compiles it into an NPU model.
• On a deploy, the model goes into the sensor model slot together with its pre-processing parameters, class names and data source, and the board restarts by itself.
• The board pre-processes exactly as Studio does, runs the NPU, sends four results a second to Studio, and shows the current class as the RGB LED's color.

Connect the board¶

The Sensor page¶

Choose a data source¶

Choose the data source with Source at the top right of the Signal area. When you change the source, Studio fills in the suggested window length and view.

A simulated sensor draws a new speed, level and noise for every recording, and its level also drifts slowly every 10 seconds, so two recordings of one state differ the way two runs of the same machine do.

For a simulated sensor, two rows of controls sit under the waveform:

• State buttons: for example Simulated vib: off normal imbalance misalign bearing looseness; click one to switch the simulated state. Key SW4 on the Reef also steps through the states.
• Parameters: set them and press Apply.

Record¶

Tips for recording:

• With a simulated sensor, 10 to 30 seconds per class is enough.
• Hold each time for more than twice the window length. The last quarter of each recording is held out for scoring; a class recorded too briefly cannot be scored. Shorter than one window shows too short: hold for at least one window.
• If the data stream skips, the page says the stream skipped: kept as <N> takes and keeps the recording as N pieces.
• Record slow signals (such as temperature) longer, at least 15 windows.

Import CSV: add data recorded elsewhere to the selected class with Import CSV….

• One column per channel, one row per sample, comma-separated; a header row is skipped.
• The sample rate must match the board's current data source, and the number of columns its channel count; otherwise the page shows an <N> columns error.
• Use the raw counts the board streams, not volts or g: the spectrum's floor is one count, so data converted to units looks different from the live signal.

Train¶

If the page says <k> of <n> classes have no held-out data, some classes were recorded too briefly each time; record them again for more than twice the window length. Training time depends on the number of classes, recording length, model size and whether UNKNOWN is on. INTELPRO measured from about 4 s (2 classes of 15 s each) to about 90 s (3 classes of 15 s each, Medium, UNKNOWN on).

Teach the model to say UNKNOWN: with Say UNKNOWN for what it was not taught ticked, Studio makes many variations of your recordings, learns them as an extra UNKNOWN class, and sets a score floor for each class. For a state nobody recorded, the board answers UNKNOWN instead of forcing it into a known class. With UNKNOWN on you can teach up to 7 classes (UNKNOWN takes the eighth).

Record only the normal state: anomaly detection: when exactly one class has recordings (usually "normal"), you can tick Anomaly detection: learn the one class as normal. Studio adds man-made anomalies to the normal data - new tones, knocks, noise bursts, level jumps or drift, stuck readings, large speed changes, a dead channel, clipping, stronger peaks or harmonics, a channel gain change and slow beating - so the model learns "normal" against "everything else", and faults never recorded come out as ANOMALY. Studio sets the decision threshold so that about 1% of the varied normal data is called anomalous. For faults you can record, classification (one class per fault) is usually more reliable than anomaly detection.

Deploy¶

A deploy usually takes a few seconds; larger models take longer to send and write. The sensor model has a single slot: if writing is cut off, there is no model; deploy again.

Live results¶

After the deploy, the On the board area updates four times a second:

• Large type: the current decision - a class name, UNKNOWN (a state it was not taught) or ANOMALY (an anomaly in anomaly-detection mode).
• Bars: each class's probability.
• The Reef's RGB LED shows the current class as a color too, so you can read the result without the computer.

With a simulated sensor, switch to a state you did not teach to check that UNKNOWN works; set ramp to tens of seconds to watch when the decision changes as a fault grows. Right at a state change one window can hold both states, so an occasional UNKNOWN is normal; judge from several results in a row.

Console commands in sensor mode:

Verify¶

1. After the deploy, On the board updates four times a second, and the large type follows the simulated state buttons or the machine's real state.
2. For every state you taught, the large type shows that class's name with the highest probability.
3. With UNKNOWN on, switching to a state you did not teach shows UNKNOWN.
4. The RGB LED color changes with the class; after a restart the model is still there (the boot log shows SENM: model …).

Save and retrain¶

• Save…: writes all recordings to a folder: one .npz file per class plus sensor_teach.json with the sample rate and class order.
• Open…: opens a folder saved with Save… to record more or retrain.

• The same folder can be trained from the command line, without the window, into a model package and deployed (--source must be the source you recorded from):

  ipro-ai train sensor rec/ --source sim_vib --out vib.ipkg
  ipro-ai deploy vib.ipkg --switch
  

  The defaults are the Sensor page's (Medium, Strong, at most 60 epochs, stop after 8 without improvement); --size, --window, --mode spectrogram|wave, --unknown, --anomaly and other options match the page's settings.

Add your own sensor¶

The Edge AI firmware as shipped has only the five simulated sensors and the board's microphone as data sources; it has no driver for any external sensor. There are two ways to use your own:

Option 1: add a data source to the firmware (recommended)

A firmware engineer adds an entry to the source table in the SDK's apps/ai/ipro_reef_teach/sensor/sensor_src.c: an I2C or SPI part registers with ipro_ai_source_sampled_register() (the platform calls your read function on a timer), a stream with ipro_ai_source_ring_register() (data can be pushed from an interrupt). Both are declared in components/ai/source/include/ipro_ai_source.h. The rules:

• Samples are int16, channels interleaved, one sample rate for all channels.
• Up to 8 channels; any sample rate, as long as one window fits in the 1 MB ring buffer. A polled source called on the platform's timer reaches at most 1 kHz (the system tick).
• The source reports its name, sample rate, channel count and unit.

Pins 3 and 5 of the Reef's 40-pin header J_IO are I2C (SDA GPIO19, SCL GPIO18, 4.7 kΩ pull-ups on the board), at 3.3 V; see 40-pin header J_IO. Build and flash as in Flash the Edge AI Firmware, leaving out --mfg to update the firmware only.

Studio needs no change: it reads the source's name, sample rate and channel count from the board, and the new source appears in the Source menu. The model records the data source it was trained on, and the board runs it only on that source.

Option 2: import CSV

Data recorded elsewhere with the same sensor, sample rate and channel count can be added to a class with Import CSV…. The deployed model still needs a matching data source on the board to run, so this only adds training data; it does not replace option 1.

Specifications¶

Tips¶

• Simulated sensors are for learning the flow. With a real sensor, collect new data, train again and validate on site.
• Anomaly detection only knows the "normal" you recorded. Record normal data at every normal load and speed of the machine; otherwise a heavier normal load may read as an anomaly while a real overload may pass as normal.
• Faults with a clear frequency signature (bearing faults, looseness, a lost phase) are the easiest to catch; faults that only raise the amplitude (overload, unbalanced phases) are harder for anomaly detection - use classification for them.
• Use the waveform view for slow signals (temperature, pressure), at least 15 windows per recording; record plenty of normal data for anomaly detection.
• The spectrum view needs at least 128 samples per window; at very low sample rates use the waveform view or a longer window.
• A fault whose frequencies differ by less than sample rate / 128 shows only as slow beating over time; it needs a window of 2 s or more and remains the hardest case.
• A model runs only on the data source (name, sample rate, channels) it was trained on; change the source or its parameters and you train and deploy again.

Troubleshooting¶

For anything else, see Troubleshooting.

Next steps¶

• Flash the Edge AI Firmware: update only the firmware after adding your data source.
• IPRO7AI Edge AI Programming Guide: the full interfaces of data sources, pre-processing and the application framework, available from INTELPRO; see Documents.