+- Sick Gaming (https://www.sickgaming.net)
+-- Forum: Computers (https://www.sickgaming.net/forum-86.html)
+--- Forum: Linux, FreeBSD, and Unix types (https://www.sickgaming.net/forum-88.html)
+--- Thread: Fedora - Backup on Fedora Silverblue with Borg (/thread-89454.html)
Fedora - Backup on Fedora Silverblue with Borg - xSicKxBot - 03-25-2019
Backup on Fedora Silverblue with Borg
<div style="margin: 5px 5% 10px 5%;"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/backup-on-fedora-silverblue-with-borg.png" width="1024" height="285" title="" alt="" /></div><div><p>When it comes to backing up a Fedora Silverblue system, some of the traditional tools may not function as expected. BorgBackup (Borg) is an alternative available that can provide backup capability for your Silverblue based systems. This how-to explains the steps for using BorgBackup 1.1.8 as a layered package to back up Fedora Silverblue 29 system.</p>
<p> <span id="more-25674"></span> </p>
<p>On a normal Fedora Workstation system, <em>dnf</em> is used to install a package. However, on Fedora Silverblue, <em>rpm-ostree install</em> is used to install new software. This is termed layering on the Silverblue system, since the core ostree is an immutable image and the rpm package is layered onto the core system during the install process resulting in a new local image with the layered package. </p>
</p>
<blockquote class="wp-block-quote">
<p>“BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption.” </p>
<p><cite>From the Borg website</cite></p></blockquote>
<p>Additionally, the main way to interact with Borg is via the command line. Reading the Quick Start guide it becomes apparent that Borg is well suited to scripting. In fact, it is pretty much necessary to use some form of shell script when performing repeated thorough backup’s of a system. A basic script is provided in the <a href="https://borgbackup.readthedocs.io/en/stable/quickstart.html">Borg Quick Start guide</a> , as a point to get started.</p>
<h2>Installing Borg</h2>
<p>In a terminal, type the following command to install BorgBackup as a layered package: </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$rpm-ostree install borgbackup</div>
</div>
<p>This installs BorgBackup to the Fedora Silverblue system. To use it, reboot into the new ostree with: </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$systemctl reboot</div>
</div>
<p>Now Borg is installed, and ready to use. </p>
<h2>Some notes about Silverblue and it’s file system, layered packages and flatpaks</h2>
<h4>The file system</h4>
<p>Silverblue is an immutable operating system based on ostree, with support for layering rpm’s through the use of rpm-ostree. At the user level, this means the path that appears as <em>/home </em>in a flatpak, will actually be <em>/var/home</em> to the system. For programs like Borg, and other backup tools this is important to remember since they often require the actual path, so in this example that would be <em>/var/home</em> instead of just <em>/home</em>.</p>
<p>Before starting a backup it’s a good idea to understand where potential data could be stored, and then if that data should be backed up. Silverblue’s file system layout is very specific with respect to what is writable and what is not. On Silverblue <em>/etc</em> and <em>/var</em> are the only places that are not immutable, therefore writable. On a single user system, typically the user home directory would be a likely choice for data backup. Normally excluding Downloads, but including Documents and more. Also, <em>/etc</em> is a logical choice for some configuration options you don’t want to go through again. Take notes of what to exclude from your home directory and from <em>/etc</em>. Some files and subdirectories of /etc you need root or sudo privileges to access.</p>
<h4>Flatpaks</h4>
<p>Flatpak applications store data in your home directory under <em>$HOME/.var/app/flatpakapp</em>, regardless of whether they were installed as user or system. If installed at a user level, there is also data found in <em>$HOME/.local/share/flatpak/app/</em>, or if installed at a system level it will be found in<em> /var/lib/flatpak/app</em> For the purposes of this article, it was enough to list the flatpak’s installed and redirect the output to a file for backing up. Reasoning that if there is a need to reinstall them (flatpaks) the list file could be used to do it from. For a more robust approach, examining the flatpak file system layouts can be done <a href="https://github.com/flatpak/flatpak/wiki/Filesystem">here.</a></p>
<h4>Layering and rpm-ostree </h4>
<p>There is no easy way for a user to retrieve the layered package information aside from the </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$rpm-ostree status</div>
</div>
<p> command. Which shows the current and previous ostree commit’s layered packages, and if any commits are pinned they would be listed too. Below is the output on my system, note the LayeredPackages label at the end of each commit listing.</p>
<figure class="wp-block-image"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/backup-on-fedora-silverblue-with-borg.png" alt="Terminal output of rpm-ostree status command." class="wp-image-25692" /></figure>
<p>The command </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$ostree log</div>
</div>
<p> is useful to retrieve a history of commits for the system. Type it in your terminal to see the output. </p>
<h2>Preparing the backup repo</h2>
<p>In order to use Borg to back up a system, you need to first initialize a Borg repo. Before initializing, the decision must be made to use encryption (or not) and if so, what mode. </p>
<p>With Borg the data can be protected using 256-bit AES encryption. The integrity and authenticity of the data, which is encrypted on the clientside, is verified using HMAC-SHA256. The encryption modes are listed below.</p>
<h4>Encryption modes</h4>
<table class="wp-block-table is-style-stripes">
<tbody>
<tr>
<td>Hash/MAC</td>
<td>Not encrypted<br />
no auth</td>
<td>Not encrypted,<br />
but authenticated</td>
<td>Encrypted (AEAD w/ AES)<br />
and authenticated</td>
</tr>
<tr>
<td>SHA-256</td>
<td>none</td>
<td>authenticated</td>
<td>repokey<br />
keyfile</td>
</tr>
<tr>
<td>BLAKE2b</td>
<td>n/a</td>
<td>authenticated-blake2</td>
<td>repokey-blake2<br />
keyfile-blake2</td>
</tr>
</tbody>
</table>
<p>The encryption mode decided on was keyfile-blake2, which requires a passphrase to be entered as well as the keyfile being needed. </p>
<p>Borg can use the following compression types which you can specify at backup creation time.</p>
<ul>
<li> lz4 (super fast, low compression) </li>
<li>zstd (wide range from high speed and low compression to high compression and lower speed) </li>
<li>zlib (medium speed and compression) </li>
<li>lzma (low speed, high compression) </li>
</ul>
<p>For compression lzma was chosen at setting 6, the highest sensible compression level. The initial backup took 4 minutes 59.98 seconds to complete, while subsequent ones have taken less than 20 seconds as a rule. </p>
<h4>Borg init</h4>
<p>To be able to perform backups with Borg, first, create a directory for your Borg repo:</p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$mkdir borg_testdir</div>
</div>
<p>and then change to it. </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$cd borg_testdir</div>
</div>
<p>Next, initialize the Borg repo with the borg init command: </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$borg init -e=keyfile-blake2 .</div>
</div>
<p>Borg will prompt for your passphrase, which is case sensitive, and at creation must be entered twice. A suitable passphrase of alpha-numeric characters and symbols, and of a reasonable length should be created. It can be changed later on if needed without affecting the keyfile, or your encrypted data. The keyfile can be exported and should be for backup purposes, along with the passphrase, and stored somewhere secure.</p>
<h4>Creating a backup</h4>
<p>Next, create a test backup of the Documents directory, remember on Silverblue the actual path to the user Documents directory is <em>/var/home/username/Documents</em>. In practice on Silverblue, it is suitable to use <em>~/</em> or <em>$HOME </em>to indicate your home directory. The distinction between the actual path and environment variables being the real path does not change whereas the environment variable can be changed. From within the Borg repo, type the following command </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$borg create .::borgtest /var/home/username/Documents</div>
</div>
<p>and that will create a backup of the Documents directory named <strong>borgtest</strong>. To break down the command a bit; <strong>create</strong> requires a <strong>repo location</strong>, in this case <strong>. </strong>since we are in the <strong>top level</strong> of the <strong>repo</strong>. That makes the path <strong>.::borgtest</strong> for the backup name. Finally <strong>/var/home/username/Documents</strong> is the location of the data we are backing up. </p>
<p>The following command </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$borg list</div>
</div>
<p>returns a listing of your backups, after a few days it look similar to this:</p>
<figure class="wp-block-image"><img src="http://www.sickgaming.net/blog/wp-content/uploads/2019/03/backup-on-fedora-silverblue-with-borg-1.png" alt="" class="wp-image-25709" /><figcaption>Output of borg list command in my backup repo.</figcaption></figure>
<p> To delete the test backup, type the following in the terminal </p>
<div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px">
<div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">$borg delete .::borgtest</div>
</div>
<p>at this time Borg will prompt for the encryption passphrase in order to delete the backup. </p>
<h2>Pulling it together into a shell script</h2>
<p>As mentioned Borg is an eminently script friendly tool. The Borg documentation links provided are great places to find out more about BorgBackup, and there is more. The example script provided by Borg was modified to suit this article. Below is a version with the basic parts that others could use as a starting point if desired. It tries to capture the three information pieces of the system and apps mentioned earlier. The output of <em>flatpak list</em>, <em>rpm-ostree status</em>, and <em>ostree log</em> as human readable files given the same names each time so overwritten each time. The repo setup had to be changed since the original example is for a remote server login with ssh, and this was intended to be used locally. The other changes mostly involved correcting directory paths, tailoring the excluded content to suit this systems home directory, and choosing the compression. </p>
<pre class="wp-block-code"> <div class="codecolorer-container text default" style="overflow:auto;border:1px solid #9F9F9F;width:435px;height:300px"><div class="text codecolorer" style="padding:5px;font:normal 12px/1.4em Monaco, Lucida Console, monospace">#!/bin/sh<br />
<br />
# This gets the ostree commit data, this file is overwritten each time<br />
sudo ostree log fedora-workstation:fedora/29/x86_64/silverblue > ostree.log<br />
<br />
rpm-ostree status > rpm-ostree-status.lst<br />
<br />
# Flatpaks get listed too<br />
flatpak list > flatpak.lst<br />
<br />
# Setting this, so the repo does not need to be given on the commandline:<br />
export BORG_REPO=/var/home/usernamehere/borg_testdir<br />
<br />
# Setting this, so you won't be asked for your repository passphrase:(Caution advised!)<br />
export BORG_PASSPHRASE='usercomplexpassphrasehere'<br />
<br />
# some helpers and error handling:<br />
info() { printf "\n%s %s\n\n" "$( date )" "$*" >&2; }<br />
trap 'echo $( date ) Backup interrupted >&2; exit 2' INT TERM<br />
<br />
info "Starting backup"<br />
<br />
# Backup the most important directories into an archive named after<br />
# the machine this script is currently running on:<br />
borg create \<br />
--verbose \<br />
--filter AME \<br />
--list \<br />
--stats \<br />
--show-rc \<br />
--compression auto,lzma,6 \<br />
--exclude-caches \<br />
--exclude '/var/home/*/borg_testdir'\<br />
--exclude '/var/home/*/Downloads/'\<br />
--exclude '/var/home/*/.var/' \<br />
--exclude '/var/home/*/Desktop/'\<br />
--exclude '/var/home/*/bin/' \<br />
\<br />
::'{hostname}-{now}' \<br />
/etc \<br />
/var/home/ssnow \<br />
<br />
backup_exit=$?<br />
<br />
info "Pruning repository"<br />
<br />
# Use the `prune` subcommand to maintain 7 daily, 4 weekly and 6 monthly<br />
# archives of THIS machine. The '{hostname}-' prefix is very important to<br />
# limit prune's operation to this machine's archives and not apply to<br />
# other machines' archives also:<br />
<br />
borg prune \<br />
--list \<br />
--prefix '{hostname}-' \<br />
--show-rc \<br />
--keep-daily 7 \<br />
--keep-weekly 4 \<br />
--keep-monthly 6 \<br />
<br />
prune_exit=$?<br />
<br />
# use highest exit code as global exit code<br />
global_exit=$(( backup_exit > prune_exit ? backup_exit : prune_exit ))<br />
<br />
if [ ${global_exit} -eq 0 ]; then<br />
info "Backup and Prune finished successfully"<br />
elif [ ${global_exit} -eq 1 ]; then<br />
info "Backup and/or Prune finished with warnings"<br />
else<br />
info "Backup and/or Prune finished with errors"<br />
fi<br />
<br />
exit ${global_exit}</div></div> </pre>
<p>This listing is missing some more excludes that were specific to the test system setup and backup intentions, and is very basic with room for customization and improvement. For this test to write an article it wasn’t a problem having the passphrase inside of a shell script file. Under normal use it is better to enter the passphrase each time when performing the backup.</p>
</div>