[Ubuntu News] Never lose robot data again: automatic ROS 2 bag recording and upload

[Ubuntu News] Never lose robot data again: automatic ROS 2 bag recording and upload - Printable Version

+- Sick Gaming (https://sickgaming.net)
+-- Forum: Computers (https://sickgaming.net/forum-86.html)
+--- Forum: Linux, BSD & Unix (https://sickgaming.net/forum-88.html)
+--- Thread: [Ubuntu News] Never lose robot data again: automatic ROS 2 bag recording and upload (/thread-114438.html)

[Ubuntu News] Never lose robot data again: automatic ROS 2 bag recording and upload - xSicKxBot - 11-03-2026

If you work with robots, perhaps you’ve come up against a common, but annoying situation: the one field run that mattered is the one nobody recorded. Or it was recorded but the ROS 2 bag (a recording of ROS 2 topics) is sitting in the robot’s storage, at a customer site, three time zones away.

This issue is commonly caused by manually starting ros2 bag record and using

Code:
scp

to copy gigabytes off the robot after every test, which just does not scale. It can also lead to missing bag synchronizations or accidental overwrites.

There is a better way: let every robot record itself continuously and upload its bags to your server automatically. That is exactly what the ros2-exporter-agent snap does.

What can the

Code:
ros2-exporter-agent

do?

The snap runs four small cooperating services on the robot:

recorder: records your ROS 2 topics to MCAP bags, splitting them by time or size
synchronization: every two minutes, this service attempts to upload bag files with rclone
daily-rotation: restarts recording at midnight so bags are grouped in clean, timestamped sets
auto-clean: deletes local files older than 10 minutes, so the disk does not fill up with old bags. Bags live locally only as a short staging buffer

The design was built with safety in mind: recording only starts once auto-clean is running, and the sync service retries automatically. If the server is briefly unreachable, the robot retries the upload automatically.

A server setup for every team

The server itself can live wherever it makes sense for you: a machine on the local network, right where the robots operate, a self-hosted server back at your office, or simply a cloud server.

The upload side is plain rclone, which supports more than 40 storage backends. The snap always uploads to an rclone remote called bagstore independently of the configure backend. Switching from a lab server to AWS S3 is a configuration change away.

You can find below two examples of rclone configurations for the ros2-exporter-agent.

A simple SSH-accessible server (SFTP backend):

Code:
[fileserver]
type = sftp
host = bags.robot-company.com
user = robot
port = 22
key_file = /root/snap/ros2-exporter-agent/common/device_rsa_key

[bagstore]
type = alias
remote = fileserver:/srv/bags/robot-01

An S3-compatible bucket (AWS, Ceph, MinIO…):

Code:
[s3store]
type = s3
provider = Ceph
access_key_id = <access-key>
secret_access_key = <secret-key>
endpoint = https://s3.robot-company.com

[bagstore]
type = alias

Notice the pattern: the first section is backend-specific, but the

Code:
bagstore

alias never changes. Whether you manage one robot or a hundred with SSH or S3, the snap logic stays identical.

Set up a robot in a few minutes

Getting started takes one command and two configuration files:

Code:
sudo snap install ros2-exporter-agent --channel latest/candidate

On installation, the snap creates two templates in

Code:
/var/snap/ros2-exporter-agent/common/local-configuration/

.

Rename them to remove the .template suffix, then edit them:

Code:
rclone.conf
describes where bags go using any of the backends above
Code:
rosbag2-recorder.yaml
describes what to record, which topics to include or exclude, and how often to split bag files

The configuration section of the snap’s README documents both formats with examples.

Once both files are in place, the services pick them up and the robot starts recording and uploading on its own.

Central configuration for a fleet

Editing files on each robot works up to a certain point. However, the snap also offers options for managing fleets. In this case, the snap also reads its configuration through a content interface. A separate configuration snap holds

Code:
rclone.conf

and

Code:
rosbag2-recorder.yaml

, and any robot that connects to it gets the same setup. This automates away much of the manual editing.

The rob-cos-demo-configuration repository is a ready-made example, with a ‘

Code:
basic

‘ branch (SFTP file server) and an ‘

Code:
advanced

‘ branch (S3-compatible storage).

The configuration snap also integrates some automations, such as generating a UID per device and applying the UID to the configuration. Such automations could inspire you for your own fleet.

Fork it, adapt the configuration to your own server details, build your own configuration snap, and every robot that plugs in is configured identically. By updating the configuration once, the whole fleet will receive the update.

Ideas to take it further

Once you start saving bags from your fleet, you might want to take it further with additional integrations:

Pull a bag to your laptop: use a small wrapper around
Code:
rclone
copy (or a shared alias/script) so any engineer can grab yesterday’s bag from
Code:
robot-07
with one command.
Browse bags in a web viewer: place a lightweight file browser in front of the storage, so anyone can find and download bags from a browser.
Open bags straight from storage: tools like Foxglove can load a remote MCAP file by URL, so you can start inspecting a bag without downloading it first.