8 Firmware and OTA Updates

This section documents how to enable firmware updates in the design, whether it is serial or Over The Air (OTA)

8.1 Device Firmware Upgrade Over BLE

Introduction

One of the highly important features of wirelessly-connected devices is the capability of Over-The-Air device firmware update (OTA DFU). The increasing demand by end-users for this functionality is:

  • To address issues and security vulnerabilities.

  • To ship products to market faster and have the option of delaying lower priority features and being able to roll them out to devices in the field.

PIC32CX-BZ6 and PIC32WM-BZ6/PIC32-BZ6 family of devices support Over-The-Air upgrade through BLE.

Microchip defined OTA profile and service enables firmware upgrade over the BLE link using Generic Attribute Profile (GATT). The BLE OTA protocol defines the communication between the OTAU target and OTAU manager. The OTAU manager can be a mobile device (iOS/Android) or any BLE device that implements the OTA GATT client protocol that transfers the upgrade firmware to the OTAU target. The OTAU target implements the OTA GATT server protocol to receive the new firmware image.
Note:
  1. The OTA DFU will update program flash memory but not include Device Configuration Register. User can check Device Configuration Register in “system” of MCC project graph.
  2. After OTA DFU and use IDE or IPE to program the devices, the programmed firmware cannot run. It is a known issue. Please program the firmware once again (totally twice) to fix the issue.

Secure BLE OTA DFU Process

With the increasing popularity of OTA DFU capability in IoT devices, devices are getting exposed to vulnerabilities and security threats. So, it is important to make sure that the device’s OTA DFU process is secure and reliable. One of the way of sending upgradable image reliably and securely is to sign and encrypt the image.

Here are the basics of how an Over-The-Air Device Firmware Update (OTA DFU) process works over BLE link:

Figure 8-1. BLE OTA DFU Process
  1. BLE OTA DFU file (Firmware Image encrypted, signed) is uploaded to the OTAU Manager. OTAU Manager can be a smart phone or any BLE device that supports OTA Client.
  2. OTAU Target ( PIC32CX-BZ6 and PIC32WM-BZ6/PIC32-BZ6 device) queries the OTAU Manager and fetches new firmware image.
  3. The image is decrypted, validated and applied.

8.1.1 BLE OTA DFU Firmware Architecture

The following figure illustrates the BLE OTA DFU Firmware Architecture of PIC32CXBZ6 device.

Figure 8-2. BLE OTA DFU Firmware Architecture
  • MCHP OTA Profile and Service Middleware:

    • Defines and handles the MCHP defined BLE GATT Server and Profile for getting the upgradable image over BLE link.

    • Implements OTA state management and error handling.

    • Implements defragmentation and decryption of received new image.

  • DFU Middleware:

    • DFU APIs for writing the OTA image into Embedded Flash.

    • This module provides Device Firmware Update (DFU) capabilities common across the different supported transport mediums like UART, BLE

  • OTA Event Handling:

    • OTA events from “MCHP OTA Profile and Service” middleware is sent to application for DFU permission.

  • OTA DFU Encryption Key management:

    • Trigger new Flash Image authentication by calling Bootloader API.

    • Timeout handling in case of failure/error while receiving the image.

  • Bootloader:

    • Bootloader is responsible for authentication and activation of new firmware, booting into the application, optionally entering into Serial DFU mode to get new image over serial interface.
      Note: Bootloader code is running from boot ROM inPIC32CX-BZ6 and PIC32WM-BZ6 family of devices.
Figure 8-3. BLE OTA DFU Firmware Architecture - H3 Code.

8.1.2 Memory Management

PIC32CX-BZ6 and PIC32WM-BZ6/PIC32-BZ6 family of devices has enough Embedded Flash memory to hold the new upgradable OTA image until new image is authenticated. The image is received in blocks over BLE link. If the image is encrypted, it will be decrypted and stored in Embedded Flash Slot1. Once the complete image is received, image is validated for signature. Then the device reset will trigger the bootloader in Boot Flash region to load the image from Slot1 to Slot0. Now the new firmware starts executing.

8.1.3 Microchip Defined BLE OTA DFU Profile

The BLE OTA DFU profile is a GATT based profile. It is designed to perform device firmware update over the air. In general mobile will perform the OTAU client role and the BLE device will be in server role. OTAU service is a Microchip’s propriety service with a 16-byte service UUID. There shall be only one instance of the OTAS in a device. The OTAS shall be instantiated as a «Primary Service». The service Universally Unique Identifier (UUID) value shall be set to «4D434850-253D-46B3-9923-E61B8E8215D7».

There are 3 characteristics defined under the service:

Characteristic Name

Universally Unique Identifier (UUID)

Properties

Permissions

OTA Feature

4D434850-22E4-4246-AF03-0C4A2F906358

Read

Encryption required

OTA Data

4D434850-34D9-40A6-BA7E-56F57C8CD478

WriteWithoutResponse, Notify

Encryption required

OTA Control Point

4D434850-9327-45DE-8882-C97F39028A76

Write

Encryption required

Further details on OTAU service and profile can be found in stack documents:

  • BLE OTA Profile - See MPLAB® Harmony Wireless BLE in Reference Documentation from Related Links.

  • BLE OTA Service - See MPLAB® Harmony Wireless BLE in Reference Documentation from Related Links.

8.1.4 BLE OTA DFU Image File Definition

BLE OTA DFU File contains the OTAU header and Upgradable Flash Image as shown in below figure. This file is loaded into OTAU Manager (Smart phone/ GATT client) to send to OTAU Target over BLE link.

Figure 8-4. BLE OTA DFU Image File Definition
  • Flash Image: Meta-data Header + Executable Firmware. This is full image content which is programmed in the device Flash Slot1.
  • Meta-data Header: Flash Image will have a metadata header, metadata payload and metadata footer that will give the Bootloader firmware information about where the firmware image is located, security decryption information, signature, sequence number etc. Digital signatures ensure the authenticity of the image and integrity of the data in the image. A digital signature also ensures that the data within the image has not been modified (preserving integrity) and is intact as it was generated at the source. Refer to meta data header format from BLE Stack Documenation.
  • OTAU File Encryption: The executable firmware can be encrypted (This is configurable). Encrypting the image ensures the confidentiality of the data. This makes that no unauthorized parties are able to peek at the contents of the image. Only the end-device should be able to decrypt the image. AES128-CBC method of encryption is used. Only the firmware image is encrypted and OTAU Header is not encrypted.
  • BLE OTAU Header: This holds the OTAU File information for BLE OTA DFU Client (Ex: mobile App) to perform OTA DFU procedures. This header is not being sent over the air to OTAU Target.
Figure 8-5. BLE OTAU Header

◦ Total Header length: 16 bytes

◦ BLE OTAU Header Version (HEADER_VER): 1 byte

  • 0x01: PIC32CX-BZ6 and PIC32WM-BZ6/PIC32-BZ6

  • Others: RFU.

◦ Flash Image is encrypted or not (FLASH_IMG_ENC): 1 byte

  • 0x00: Firmware Image is not encrypted.

  • 0x01: Firmware Image is encrypted by AES-CBC method.

  • Others: RFU.

◦ Checksum : 2 bytes

  • Checksum value of full OTAU file.

◦ Flash Image ID (FLASH_IMG_ID): 4 byte

  • Identity number of Flash Image.

◦ Flash Image Revision/version (FLASH_IMG_REV): 4 byte

◦ OTAU File Type (FILE_TYPE): 1 Byte

  • 0x01: BLE OTAU File

  • 0x02: BLE+Zigbee Combo OTAU File

  • Others: RFU

◦ Reserved: 3 byte

OTA file is .bin file which can be generated from MPLABX Tools environment as illustrated below. The detail steps for image generation is explained later.
  • BLE OTA Header and Encryption Key Configuration:
Figure 8-6. BLE OTA Header and Encryption Key Configuration in MPLABX
  • Meta-data Header Configuration:
Figure 8-7. Meta-data Header Configuration in MPLABX

8.1.5 BLE OTA DFU Image Distribution Procedure

The following steps explain the BLE OTA DFU Image Distribution Procedure:
  1. Encrypted and Signed BLE OTAU bin file generation using signature and OTA configurator from MPLABX Tools device properties. Place the OTAU file in OTAU manager.
  2. Transfer the OTAU Flash image over BLE OTA profile/service to OTAU Target
  3. Image is decrypted using AES Key in Target and stored in Embedded Flash Slot1
  4. Once the complete image is received, the device is reset. Bootloader will copy new image from Slot1 to Slot0 after image authentication is done.
Figure 8-8. BLE OTA DFU Image Distribution Procedure

8.1.6 BLE OTA DFU Implementation

The following steps explain how to add the BLE OTA DFU functionality in Microchip BLE application.
  1. The first 5 sections will explain how to enable the OTA DFU features in a general BLE application that is intended to be the OTAU Target.

  2. The last two sections will explain how to make the BLE OTA DFU file and to how to use MBD as an OTAU Manager to upgrade the OTAU Target with the BLE OTA DFU file.

8.1.6.1 MCC Component and Code Generation

Prerequisites
  1. Open any BLE application based on MCC (Example project/ Custom created project).

  2. Launch MPLABx Code Configurator (MCC).

Figure 8-9. MPLABx Code Configurator
Pull-in MCC Components

The following steps explain the components/configurations needed for OTA functionality alone.

Click “+” symbol “BLE OTA APP SERVICE” component from “Device Resources” of MCC to project graph. When the component is added, the dependent components also getting added automatically. Accept Dependencies or satisfiers, select “Yes” in order to add the dependent components.
  1. Configure “BLE OTA APP SERVICE” component as the following, based on the need:

    1. Disable Flash Image ID - This option checks for Image ID. If this option is enabled the upgradable image must have the same ID of current image, only then OTA upgrade procedure will happen. Within the project properties, set the “Flash Image ID” prefix to 0x9F and the remaining numbers should be above 0, for example, use 2 as shown below.
    2. Enable Image Decryption - If the upgradable image is secured (encrypted using AES128 CBC), this configuration has to be enabled to do decryption of the received image. The AES Key must match with the Key in BLE OTA DFU bin file generation configuration as shown below. Do not accept any dependencies after the “Enable Image Decryption” is enabled. Select “No” to any dependency popups at this point.
Note: Do not accept any Crypto related dependencies after the “Enable Image Decryption” is enabled. Select “No” to these dependency popups if they are displayed.
Verifying Configurations
All the required configurations are automatically done when the “BLE OTA APP SERVICE” component is added. Verify if the Project Graph window has all the expected MCC configurations.
Note: Depending on the application you are adding OTA DFU features to, your project graph may include other components, but the OTA DFU specific components and their configurations are listed below.
  1. BLE OTA profile and OTA Service Components - Enable Server Role. This module generates the code needed for MCHP defined BLE OTA profile and service.

  2. BLE_Stack component - DFU Module enable - This module generates the code needed for device firmware upgrade middleware for writing the OTA image into Embedded Flash.
  3. BLE_Stack component - Advertising and Scan data payload is configured for manufacturer specific service UUID and local name. BLE OTA Microchip Data (MBD) mobile app expects service UUID as “0xDAFE” to filter the PIC32-BZ6.
  4. FreeRTOS stack size - Set the size to be 5120 bytes.
  5. App Timer Service component - The BLE OTA procedure requires error handling and initializing the state in case of failure in receiving the image. There are 2 software timers based on FreeRTOS systick is used. APP_TIMER_OTA_TIMEOUT for error handling. APP_TIMER_OTA_REBOOT for resetting the device after successful image reception.
  6. RCON component - As mentioned above APP_TIMER_OTA_REBOOT timer is used for resetting the device after successful image reception. Device reset is performed using SW reset in RCON module.
  7. DIS BLE Service - Device Info Service is BT SIG defined service. This service can be used to hold the firmware version. When upgrading the Image, OTAU manager gets the current running image version. OTA server in device can use firmware version from DIS and send to OTAU manager. Using the FW from DIS is just optional. The “Firmware Revision” number will be seen in the MBD app but it must match the “FW_IMG_REV” in the project properties in the “Header” section.
    If DIS is not needed for customer application, DIS component can be removed and the below code for sending firmware version must be modified.
  8. FreeRTOS Total Heap Size - Set the “Total Heap Size” to be 75,812.
Generating a Code

For more details on code generation, refer to MPLAB Code Configurator (MCC) Code Generation from Related Links.

Files and Routines Automatically Generated by MCC

After generating the code from MCC tool by clicking Generate button. The project folder structure is as follows:

8.1.6.2 OTA Application Development

  1. Compile MCC auto generated project:

    • Compile the MCC auto generated project as the following.
    • Addressing the mandatory error (if not already done): User action required in app_user_edits.c. Follow the steps mentioned in the note and do the necessary changes. Then comment the #error message as below.

  2. Call BLE OTA Init function, DIS service Init function, and start advertising in app.c and add the include files.

    #include "app_ota/app_ota_handler.h"
    #include "ble_dis/ble_dis.h"
    
    APP_OTA_HDL_Init();
    BLE_DIS_Add();
    BLE_GAP_SetAdvEnable(0x01,0x00);
  3. Call BLE OTA event handler function in "app_ble/app_otaps_handler.c".

    APP_OTA_EvtHandler(p_event);
    #include "../app_ota/app_ota_handler.h"
  4. Uncomment timer message ID's in "app_timer/app_timer.c" needed for OTA error handling and reboot timer. When the timer is fired, the related message is posted in freeRTOS application task queue.

  5. Define the timer message ID's in APP_MsgId_T structure in app.h.

    APP_TIMER_OTA_TIMEOUT_MSG,
    APP_TIMER_OTA_REBOOT_MSG,
  6. Call OTA timer handlers in APP_Tasks() in "app.c".

    else if(p_appMsg->msgId == APP_TIMER_OTA_TIMEOUT_MSG)
    {
    APP_OTA_Timeout_Handler();
    }
    else if(p_appMsg->msgId== APP_TIMER_OTA_REBOOT_MSG)
    {
    APP_OTA_Reboot_Handler();
    }
  7. Call BLE gap connected/disconnected event handler in "app_ble/app_ble_handler.c" . app_ble_conn_handler.c handles the events and also restarts the Advertising when disconnected.

    APP_BleGapConnEvtHandler(p_event);
    #include "../app_ble_conn_handler.h"
  8. Change "BLE_GAP_EVT_ENCRYPT_STATUS" to "GAP_STATUS_SUCCESS" in “app_ble_conn_handler.c”.
  9. Compile the project for no errors.
Note: Within the Project Properties, the .bin file can be generated by clicking the “Create OTA File” button.
The generated file will be located in the “[project_name].X” folder where the project was created.

API documentation reference:

8.1.6.3 Project Properties Configuration for Current/Running Firmware

MPLABX Tools environment allows to configure and append the Metadata header into application image. The needed script and sample private key .pem files are added into project folder (.X) when the OTA code was generated from MCC. For example, the BLE sensor application contains the java_friendly_private_key.pem for BLE OTA header authentication

Attention:
Note: Before proceeding to below steps, close the OTA application project and reopen the project. This step is mandatory for "SignFirmware" and "OTA" bin generation MPLABX script to be loaded into project environment.
  1. Right click on the project and select Properties.

  2. Adds Metadata header into application image. Refer to Meta-data Header Configuration for more details on header format. There are configurable parameters like Firmware Rev (version), authentication method and manufacturer ID. Configure as per requirement.
    • Step1: “FW_IMG_REV” – Firmware version of current running firmware. This must be same as version in DIS service.

    • Step2: “Auth Method” - There are 3 Supported authenticated methods
      • Signature validation (ECDSA256-SHA256) – most secure, and data integrity check. Signature is encrypted using ECDSA private key. 256-bit ECDSA Public key to be programmed in device to do signature decryption. openssl (example commands: https://techdocs.akamai.com/iot-token-access-control/docs/generate-jwt-ecdsa-keys) is one of the open tools to generate ECDSA public/private key pair. Sample private key is available in project folder .X. Public key is input in Bootloader project.

      • Hash validation (SHA-256) – less security, and data integrity check.

      • No validation (None) – no security, no integrity check.

    Warning:

    Ensure to keep "Sequence number" as non-zero value and not 0xFFFFFFFF.

  3. After completing the above steps, click Apply, then OK to take the changes into effect.

8.1.6.4 Unified Image Generation

  1. Compile the project for no errors.

  2. MPLABX generated hex image will be available in project production folder dist\default\production. There are 3 new images created as an outcome of "SignFirmware" MPLABX script. These images are the valid images which has the metadata header appended.

    1. signed.unified.hex - This is signed unified image along with bootloader. This is the current image to be programmed on the device.

    2. signed.hex - This image is signed image without bootloader. This does not work if bootloader is not present on the device.

8.1.6.5 Programming the Current Application Image using MPLABX IDE

  • Build the project after completing all the above changes and program on Curiosity board:

  • Now the device (OTAU Target) is running the application with OTA capability, and able to be upgraded using OTAU manager like mobile phone.

8.1.6.6 Upgradable Image .bin File Generation

This section explains how to generate the .bin file for a general BLE application. This section assumes that the ble_sensor application is the OTAU Target and that the new upgradable .bin file will also come from the ble_sensor application.

  1. Assume that ble_sensor is the OTAU Target and we wish to upgrade the firmware with a new ble_sensor application.
    1. Follow step 1 in Project Properties Configuration for Current/Running Firmware to open the “Header” section for the new project firmware.
    2. Change the AUTH_MTHD to “ECDSA p256+SHA256”.
    3. Change the FW_IMG_REV to the new firmware version. In this example, we will upgrade from 1.0.0.0 to 1.0.0.1.
    4. Select the Private Key File that was generated by the OTAU Target project as explained in “Project Properties Configuration for Current/Running Firmware”. For more details refer to the Project Properties Configuration for Current/Running Firmware from Related Links.
    5. Click Apply, then OK to apply the changes to the project properties and to close the window.
    6. Launch MCC on the new firmware project.
    7. Open the “Device Information Service” Configuration Options and change the Firmware Revision to the same version that you put for FW_IMG_REV in the project properties.
    8. Generate the code.
    9. Perform any application firmware updates for the new firmware revision. In the case of ble_sensor, we will change the BLE_SENSOR_VERSION in app.c to the same version number as the DIS and FW_IMG_REV.
    10. Clean and Build the project.
  2. Reopen the project properties and open the OTA window.
    1. Set the AES key, Init Vector, and Flash Image ID as the same as configured in step 1 of subsection “Pull-in MCC Components” in the section “MCC Component and Code Generation”. Rename the OTA file.
    2. Select “Create OTA File” and the file will be generate in the .X folder:

8.1.7 BLE OTA DFU Demo

The following BLE OTA demo steps are common for any application implementing OTA functionality.

BLE sensor application is the sample project which implements OTA functionality as explained in the above steps. The current image with OTA capability and new upgradable .bin (ble_sensor_OTA.bin) is also available in <Harmony Content Path>\apps\ble\advanced_applications\ble_sensor\precompiled_hex for your reference. This section assumes that the user has the ble_sensor application programmed to a PIC32-BZ6 Curiosity Board as the OTAU Target and are upgrading the firmware with a new firmware version of ble_sensor which was generated in “Upgradable image .bin file generation”.

  • Hardware Requirement:

Tool

Qty

PIC32-BZ6 Curiosity Board

1

Micro USB cable

1

Android/iOS Mobile

1

  • Smartphone App: Microchip Bluetooth Data (MBD) iOS/Android app available in stores

  • Demo steps:

    1. To perform the OTAU, the upgradable firmware bin file created in Upgradable Image .bin File Generation must be stored in the MBD app. Send the upgradable bin file to the mail. Refer to the following screenshots to store the image received on mail into MBD app.
      • iOS Mobile:
      • Android Mobile:

    2. Turn on mobile Bluetooth and select “OTA DFU" tab of Microchip Bluetooth Data App on the mobile device.
    3. Power On the OTA capable target device ( PIC32CX-BZ6 and PIC32WM-BZ6/PIC32-BZ6 ) which was programmed with ble_sensor.X.production.signed.hex from \apps\ble\advanced_applications\ble_sensor\precompiled_hex. Device will be advertising. Click on "SCAN" button in mobile app and the device be visible in scanned list.
    4. Click on intended device from the scanned list. The mobile app will request to "Pair" with the device. Select "Pair". Mobile app will now get connected with OTA Target device. "Connected" message will appear for a short while.
      Note: After pairing with the device, you must forget the device in the Bluetooth settings of the mobile phone if you attempt to reconnect to the device using MBD on the same mobile device.
    5. Click the ‘Select Image’ option to choose the available firmware file.
    6. The OTAU firmware image file copied in the step 1 will be visible on screen. Click on that image.
    7. After confirming the FW version press OK button.
    8. Firmware update will be initiated and the progress will be shown in mobile app. The process would take few seconds.
    9. Once the complete image is transferred, "OTA Update successfully" status will be shown in mobile app. Click on "OK" and then device will perform authentication of the image. If validation is successful, then system reboot automatically.
    10. On bootup, device does the authentication of new image in Slot1 and then copies the new image to Slot0. Now the new image will be running from Slot0.
    11. To verify whether the running image on the target is newly upgraded image, follow steps 2 to 7 and check the current version. Additionally, if you are testing ble_sensor application and followed the steps in "Upgradable image .bin file generation" you will be able to see the firmware update in a terminal emulator like TeraTerm when the NMCLR button is pressed after the OTA DFU completion.