Software update controlled by an application¶

Overview¶

In this tutorial, we'll write a small application that runs on a device, downloads new packages and installs them.

What you'll learn:

How to create a Yocto layer on top of Welma and integrate your app
How to use Welma's software update D-Bus API
How to manage installation and roll back automatically

What you'll need:

A Raspberry Pi 4B device
A host computer with a Linux-based operating system;
The device is reachable from the host through an IP network by SSH, using the name device-under-test.

Step 1: initial setup¶

Create your working directories on your workstation:

ROOT=...               # your root working directory
mkdir $ROOT/build      # your Yocto build directory
mkdir $ROOT/sources    # your Yocto layers
mkdir $ROOT/www        # your packages served through HTTP

Download Welma into $ROOT/sources:

cd $ROOT/sources

MACHINE=raspberrypi4-64-welma
WELMA_REF=scarthgap-next
WELMA_GIT_NAMESPACE=witekio/rnd/theembeddedkit/welma

git clone git@gitlab.com:$WELMA_GIT_NAMESPACE/welma-manifest.git
welma-manifest/setup-download welma-manifest/$WELMA_REF/manifest-$MACHINE.txt

Set up your Yocto build directory in $ROOT/build.

Terminal-1: Yocto build environment

cd $ROOT
source sources/meta-welma/setup/setup-build-env \
       sources/meta-welma-raspberrypi/conf/templates/$MACHINE

If you did not fuse the PBKH on your device, do not try to do it now, but be sure to add WELMA_SECURE_BOOT="0" in your conf/local.conf.

Build the "dev" image:

Terminal-1: Yocto build environment

bitbake welma-image-minimal-dev

Install the image on you device:

Insert the SD card in your workstation.
Copy the Welma image to the SD card (be careful about the destination /dev/mmcblk0, your milage may vary):

Terminal-1: Yocto build environment

gunzip --stdout \
    tmp/deploy/images/$MACHINE/welma-image-minimal-dev-$MACHINE.wic.gz |
    sudo dd of="/dev/mmcblk0" bs=10M
sync

Remove the SD card from your workstation, and insert it into your device. Do the cabling so as to get console access and SSH access to your device. Then start your device.

You should now see the device booting on the console, something like this:

Console: /dev/ttyUSB0

  0.51 RPi: BOOTSYS release VERSION:69471177 DATE: 2025/05/08 TIME: 16:21:35
...
  2.53 RPi: BOOTLOADER release VERSION:69471177 DATE: 2025/05/08 TIME: 16:21:35
...
U-Boot 2024.01 (Jan 08 2024 - 15:37:48 +0000)
...
Starting kernel ...
...
[    0.000000] Linux version 6.6.63-v8 (oe-user@oe-host) (aarch64-poky-linux-gcc (GCC) 13.4.0, GNU ld (GNU Binutils) 2.42.0.20240723) #1 SMP PREEMPT Fri Dec  6 10:10:05 UTC 2024
...
Welma by Witekio 1.6.1 raspberrypi4-64-welma ttyS0

raspberrypi4-64-welma login:

Step 2: create a Yocto layer¶

Create a dedicated directory $ROOT/sources/meta-app, in which you create the following:

meta-app/conf/layer.conf

# Add layer to BBPATH
BBPATH .= ":${LAYERDIR}"

# Add recipes to BBFILES
BBFILES += "${LAYERDIR}/recipes-*/*/*.bb \
            ${LAYERDIR}/recipes-*/*/*.bbappend"

BBFILE_COLLECTIONS += "app-layer"
BBFILE_PATTERN_app-layer = "^${LAYERDIR}/"
BBFILE_PRIORITY_app-layer = "1000"

LAYERVERSION_app-layer = "1"
LAYERSERIES_COMPAT_app-layer = "scarthgap"

# Enable gobject introspection, needed by pydbus
DISTRO_FEATURES:append = " gobject-introspection-data"
IMAGE_INSTALL:append = " autoupdate systemd-user-appconf"

This file conf/layer.conf is mandatory for Yocto layers. It collects all recipes and bbappends (BBFILES). It names the layer (BBFILE_COLLECTIONS). We set a high priority (BBFILE_PRIORITY_app-layer), as we want this project specific layer to take precedence over all other generic layers. We also integrate our autoupdate app in the default image (IMAGE_INSTALL).

Step 3: add your application¶

We're now creating a minimal application, called autoupdate, that is in charge of periodically polling an HTTP server, fetching new versions of appro and installing them.

This application is made of a simple program in Python, located in /app/bin/autoupdate (in the appro partition).

In meta-app, create the following files:

recipes-app/autoupdate/autoupdate.bb
recipes-app/autoupdate/files/autoupdate.service
recipes-app/autoupdate/files/autoupdate
recipes-core/systemd/systemd-user-appconf.bb
recipes-core/systemd/files/01-unit-reboot-on-exit.conf

Write the program itself, as follows:

meta-app/recipes-app/autoupdate/files/autoupdate

#!/usr/bin/env python3

import os
import pydbus
import sys
import time
import urllib.request

DBUS_NAME = 'com.witekio.update1'
DBUS_OBJECT = '/com/witekio/update1'
DBUS_INTERFACE = 'com.witekio.update1.Manager'

AUTOUPDATE_INSTALLED = "/var/lib/autoupdate/installed"  # used to keep track of which version got installed
AUTOUPDATE_PENDING = "/var/lib/autoupdate/pending"      # will be moved to _INSTALLED after confirmation
AUTOUPDATE_ACTIVE_PARTITIONS = "/var/lib/autoupdate/latest-active-partitions" # used to detect rollback after reboot


def dbus_init():
    bus = pydbus.SystemBus()
    obj = bus.get(DBUS_NAME, DBUS_OBJECT)
    itf = obj[DBUS_INTERFACE]
    return obj, itf


def dbus_install(filename):
    DBUS_INTERFACE.BeginUpdate()
    try:
        DBUS_INTERFACE.InstallLocalFile(filename)
        DBUS_INTERFACE.SubmitUpdate()
    except Exception:
        DBUS_INTERFACE.AbortUpdate()


def wait_for_network(url):
    while True:
        try:
            print(f"wait_for_network: GET {url}")
            urllib.request.urlopen(f"{url}").read()  # read() leads to closing the socket
            break
        except Exception:
            # Retry in 5 s
            time.sleep(5)


def get_active_partitions():
    """Get a determinist footprint of active partitions"""
    dir_active = "/dev/disk/partitions/active"
    partitions = [os.readlink(f"{dir_active}/{f}") for f in os.listdir(dir_active)]
    partitions.sort()
    return ' '.join(partitions)


def complete_pending_install(url):
    """Complete a pending installation and report to 'url'"""
    if os.path.exists(AUTOUPDATE_PENDING):
        # Let's find out if the update was successful or if we rolled back
        with open(AUTOUPDATE_ACTIVE_PARTITIONS) as fd:
            if get_active_partitions().strip() == fd.read().strip():
                # No change. That's a rollback.
                urllib.request.urlopen(f"{url}/update/rollback", data=b'')  # data= implies method POST
                os.unlink(AUTOUPDATE_PENDING)
            else:
                # Partitions changed. That means a successful update.
                urllib.request.urlopen(f"{url}/update/ok", data=b'')  # data= implies method POST
                os.rename(AUTOUPDATE_PENDING, AUTOUPDATE_INSTALLED)


# Main

host = sys.argv[1]
port = sys.argv[2]

URL_BASE = f"http://{host}:{port}"

DBUS_OBJECT, DBUS_INTERFACE = dbus_init()

wait_for_network(URL_BASE)

if DBUS_OBJECT.State == 'under-test':
    DBUS_INTERFACE.ConfirmUpdate()

complete_pending_install(URL_BASE)

while True:
    try:
        # Get latest revision to be installed
        print(f"GET {URL_BASE}/applicable")
        applicable = urllib.request.urlopen(f"{URL_BASE}/applicable").read()
        applicable = applicable.decode('utf-8').strip()
        try:
            # Get revision of locally installed package
            with open(AUTOUPDATE_INSTALLED) as fd:
                installed = fd.read().strip()
        except Exception:
            installed = None

        print(f"installed={installed}, applicable={applicable}")
        if installed is None or installed != applicable:
            # A new package is available. Download and install it.
            print(f"GET {URL_BASE}/{applicable}")
            urllib.request.urlretrieve(f"{URL_BASE}/{applicable}", "/tmp/autoupdate.pkg")
            # Call install via d-bus
            dbus_install("/tmp/autoupdate.pkg")
            # Save active partitions
            with open(AUTOUPDATE_ACTIVE_PARTITIONS, 'w') as fd:
                fd.write(get_active_partitions()+"\n")
            # Save the revision that we installed
            with open(AUTOUPDATE_PENDING, 'w') as fd:
                fd.write(f"{applicable}\n")
            # Exit normally. The systemd config should trigger a reboot.
            break
    except Exception as e:
        print(f"Failed: {e}")

    time.sleep(20)

This autoupdate program, written in Python implements a minimalist algorithm:

When it starts, it waits until it can reach a remote HTTP server (wait_for_network());
Then it retrieves the applicable revision of package that should get installed (/applicable);
If it differs from what is installed, then it initiates an installation and exit. Systemd is configured to reboot (see autoupdate.service and 01-unit-reboot-on-exit.conf);
After reboot, it waits until it can reach a remote HTTP server and confirms the update procedure.

Notes:

We're using here a very simple scheme for the revision of a package: the revision is the filename.
If the update procedure is not confirmed after 2 minutes, Welma triggers a reboot and rolls back the system.
The autoupdate program keeps track of active partitions before and after reboot in order to detect if a rollback occurred and report accordingly to the HTTP server.
The autoupdate program keeps track of which package got installed (in /var/lib/autoupdate/installed), in order not to reinstall always the same package in an infinite loop.
It relies on the pydbus library for calling D-Bus services.
It is a good practice not to use any .py extension for the filename, as it can happen that we later want to code this program in another language without changing its name (and without having a filename extension that is not consistent with is contents).

We now need a Yocto recipe to have the program installed in our image:

meta-app/recipes-app/autoupdate/autoupdate.bb

SUMMARY = "A minimal autoupdater"
LICENSE = "CLOSED"

SRC_URI = "\
    file://autoupdate \
    file://autoupdate.service \
"

inherit systemd-user useradd

SYSTEMD_USER_AUTO_ENABLE = "enable"
SYSTEMD_USER_SERVICE:${PN} = "autoupdate.service"
SYSTEMD_USER_PACKAGES = "${PN}"
SYSTEMD_USERS = "user"

USERADD_PACKAGES = "${PN}"
USERADD_PARAM:${PN} = "user"

prefix = "/app"

do_install() {
    install -D -m 0644 ${WORKDIR}/autoupdate.service ${D}/${systemd_user_unitdir}/autoupdate.service
    install -D -m 0755 ${WORKDIR}/autoupdate ${D}/app/bin/autoupdate
    install -d --owner user --group user ${D}/var/lib/autoupdate
}

FILES:${PN} += "/app"

RDEPENDS:${PN} = "welma-dm python3-pydbus"

This recipe autoupdate.bb installs the autoupdate program and has it started automatically at boot (autoupdate.service), as a systemd user (unprivileged) service. For that, it leverages Welma's bbclass systemd-user (configured through SYSTEMD_USER_ variables). It also creates /var/lib/autoupdate, which will be used by the autoupdate program to store its files.

Let's write a systemd service description to have the app automatically started at boot time:

meta-app/recipes-app/autoupdate/files/autoupdate.service

[Unit]
Description=Auto-update app

# When the unit stops and enters a failed state or inactive state,
# have the manager exit
SuccessAction=exit

[Service]
ExecStart=/app/bin/autoupdate 192.168.1.35 8011

Environment=PYTHONUNBUFFERED=non-empty

StandardInput=null
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=default.target

This file autoupdate.service starts the autoupdate program at boot time. You'll need to figure out the IP address of your workstation and replace 192.168.1.35 by yours. The parameter SuccessAction=exit is a way to have the systemd user instance stop when the program stops, and in conjunction with 01-unit-reboot-on-exit.conf below, it leads to a whole system reboot.

The parameter PYTHONUNBUFFERED= is set to prevent Python from buffering what gets printed by the program autoupdate, so that we can see the messages in the journal immediately when they are printed.

We need a little more systemd configuration to have the system reboot when autoupdate exits:

meta-app/recipes-core/systemd/systemd-user-appconf.bb

SUMMARY = "systemd user configuration"
DESCRIPTION = "Reboot system when systemd user instance exits"
LICENSE = "CLOSED"

SRC_URI = "file://01-unit-reboot-on-exit.conf"

FILES:${PN} = "${systemd_unitdir}"

do_install:append() {
    install -D -m 0644 ${WORKDIR}/01-unit-reboot-on-exit.conf \
                       ${D}/${systemd_unitdir}/system/user@2000.service.d/01-unit-reboot-on-exit.conf
}

This systemd-user-appconf.bb recipe places a systemd configuration drop-in for the systemd user instance of user 2000. That is the numeric identifier of the user named user that is running our autoupdate program.

meta-app/recipes-core/systemd/files/01-unit-reboot-on-exit.conf

[Unit]
# When the unit stops and enters a failed state or inactive state, reboot
# following the normal shutdown procedure
FailureAction=reboot
SuccessAction=reboot

This 01-unit-reboot-on-exit.conf drop-in will have the systemd system instance reboot when the systemd user instance exits.

Add meta-app in bblayers and build:

Terminal-1: Yocto build environment

$ cat << EOF >> conf/bblayers.conf
BBLAYERS =+ "$ROOT/sources/meta-app"
EOF

$ bitbake welma-image-minimal-dev

Install the resulting image tmp/deploy/images/$MACHINE/welma-image-minimal-dev-$MACHINE.wic.gz onto your device, boot it, then connect via SSH and read the journal:

# journalctl -f
Jan 29 17:07:52 raspberrypi4-64-welma autoupdate[729]: GET http://192.168.1.35:8011/applicable
Jan 29 17:07:52 raspberrypi4-64-welma autoupdate[729]: Failed: <urlopen error [Errno 111] Connection refused>
Jan 29 17:08:12 raspberrypi4-64-welma autoupdate[729]: GET http://192.168.1.35:8011/applicable
Jan 29 17:08:12 raspberrypi4-64-welma autoupdate[729]: Failed: <urlopen error [Errno 111] Connection refused>
...

We get an error, and that is normal as we did not set up any HTTP server to connect to yet.

Step 4: serve your package¶

We'll host our packages in directory $ROOT/www.

Create a very simple server in Python, that serves files of the current directory and reports POST requests. It is not secure and should not be used for production.

$ROOT/www/http-server

#!/usr/bin/env python3
#
# Usage: python3 server.py PORT

import http.server
import socketserver
import sys


class CustomHandler(http.server.SimpleHTTPRequestHandler):
    def do_GET(self):
        super().do_GET()

    def do_POST(self):
        self.send_response(http.HTTPStatus.OK)
        self.end_headers()


socketserver.TCPServer.allow_reuse_address = True
TCP_PORT = int(sys.argv[1])

with socketserver.TCPServer(("", TCP_PORT), CustomHandler) as httpd:
    print(f"Serving HTTP on port {TCP_PORT} ...")
    httpd.serve_forever()

Then copy your appro package to www:

Terminal-1: Yocto build environment

$ cp build/tmp/deploy/images/raspberrypi4-64-welma/welma-image-minimal-dev-raspberrypi4-64-welma.appro.swu \
     $ROOT/www/appro-1.1.swu
$ echo appro-1.1.swu > $ROOT/www/applicable

Finally, start serving the files:

Terminal-2

$ cd $ROOT/www && python3 http-server 8011
Serving HTTP on port 8011 ...

Here, be sure to use the same TCP port as specified in autoupdate.service above. You should finally see:

Terminal-2

192.168.1.25 - - [29/Jan/2026 17:49:59] "GET /applicable HTTP/1.1" 200 -
192.168.1.25 - - [29/Jan/2026 17:49:59] "GET /appro-1.1.swu HTTP/1.1" 200 -
192.168.1.25 - - [29/Jan/2026 17:52:12] "GET / HTTP/1.1" 200 -
192.168.1.25 - - [29/Jan/2026 17:52:12] "POST /update/ok HTTP/1.1" 200 -
192.168.1.25 - - [29/Jan/2026 17:52:12] "GET /applicable HTTP/1.1" 200 -
192.168.1.25 - - [29/Jan/2026 17:52:32] "GET /applicable HTTP/1.1" 200 -
...

The IP address 192.168.1.25 is the one of the device and may vary.

Let's dissect these messages. All are requests or notifications coming from autoupdate running on the device:

GET /applicable: this is the request to get which the applicable version is.
GET /appro-1.1.swu: this is the request to download the package appro-1.1.swu.
GET /: this occurs more than 1 min after the previous one. The device is checking if the HTTP server is reachable (remember wait_for_network()).
POST /update/ok: this is the report that the installation procedure is complete and sucessful.
GET /applicable: These are the periodic polling to check if a new version is available.

Summary¶

You learned how to create a simple Yocto layer for your application, and how to use Welma's software update mechanism.

You can further play and test installing a package from the previous tutorial, that does not contain the autoupdate program, leading to the fact that after reboot the installation procedure will not be confirmed and a rollback will be triggered.

Links to the mentioned files: