If you’re like me, you may find yourself running Windows for a variety of reasons from work to gaming. Sure you could run Fedora in a virtual machine or as a container, but those don’t blend into a common windows experience as easily as the Windows Subsystem for Linux (WSL). Using Fedora via WSL will let you blend the two environments together for a fantastic development environment.
Prerequisites
There are a few basics you’ll need in order to make this all work. You should be running Windows 10, and have WSL2 installed already. If not, check out the Microsoft documentation for instructions, and come back here when you’re finished. Microsoft recommends setting wsl2 as the distro default for simplicity. This guide assumes you’ve done that.
Next, you’re going to need some means of unpacking xz compressed files. You can do this with another WSL-based distribution, or use 7zip.
Download a Fedora 33 rootfs
Since Fedora doesn’t ship an actual rootfs archive, we’re going to abuse the one used to generate the container image for dockerhub. You will want to download the tar.xz file from the fedora-cloud GitHub repository. Once you have the tar.xz, uncompress it, but don’t unpack it. You want to end up with something like fedora-33-datestamp.tar. Once you have that, you’re ready to build the image.
Composing the WSL Fedora build
I prefer to use c:\distros, but you can choose nearly whatever location you want. Whatever you choose, make sure the top level path exists before you import the build. Now open a cmd or powershell prompt, because it’s time to import:
PS C:\Users\jperrin> wsl.exe -l -v
NAME STATE VERSION
Fedora-33 Stopped 2
From here, you can start to play around with Fedora in wsl, but we have a few things we need to do to make it actually useful as a wsl distro.
wsl -d Fedora-33
This will launch Fedora’s wsl instance as the root user. From here, you’re going to install a few core packages and set a new default user. You’re also going to need to configure sudo, otherwise you won’t be able to easily elevate privileges if you need to install something else later.
wslutilites uses curl and wget for things like VS Code integration, so they’re useful to have around. Since you need to use a Copr repo for this, you want the added dnf functionality.
Add your user
Now it’s time to add your user, and set it as the default.
useradd -G wheel yourusername
passwd yourusername
Now that you’ve created your username and added a password, make sure they work. Exit the wsl instance, and launch it again, this time specifying the username. You’re also going to test sudo, and check your uid.
Assuming everything worked fine, you’re now ready to set the default user for your Fedora setup in Windows. To do this, exit the wsl instance and get back into Powershell. This Powershell one-liner configures your user properly:
Now you should be able to launch WSL again without specifying a user, and be yourself instead of root.
Customize!
From here, you’re done getting the basic Fedora 33 setup running in wsl, but it doesn’t have the Windows integration piece yet. If this is something you want, there’s a Copr repo to enable. If you choose to add this piece, you’ll be able to run Windows apps directly from inside your shell, as well as integrate your Linux environment easily with VS Code. Note that Copr is not officially supported by Fedora infrastructure. Use packages at your own risk
dnf copr enable trustywolf/wslu
Now you can go configure your terminal, setup a Python development environment, or however else you want to use Fedora 33. Enjoy!
Stratis is described on its official website as an “easy to use local storage management for Linux.” See this short video for a quick demonstration of the basics. The video was recorded on a Red Hat Enterprise Linux 8 system. The concepts shown in the video also apply to Stratis in Fedora.
Stratis version 2.1 introduces support for encryption. Continue reading to learn how to get started with encryption in Stratis.
Prerequisites
Encryption requires Stratis version 2.1 or greater. The examples in this post use a pre-release of Fedora 33. Stratis 2.1 will be available in the final release of Fedora 33.
You’ll also need at least one available block device to create an encrypted pool. The examples shown below were done on a KVM virtual machine with a 5 GB virtual disk drive (/dev/vdb).
Create a key in the kernel keyring
The Linux kernel keyring is used to store the encryption key. For more information on the kernel keyring, refer to the keyrings manual page (man keyrings).
Use the stratis key set command to set up the key within the kernel keyring. You must specify where the key should be read from. To read the key from standard input, use the –capture-key option. To retrieve the key from a file, use the –keyfile-path <file> option. The last parameter is a key description. It will be used later when you create the encrypted Stratis pool.
For example, to create a key with the description pool1key, and to read the key from standard input, you would enter:
# stratis key set --capture-key pool1key
Enter desired key data followed by the return key:
The command prompts us to type the key data / passphrase, and the key is then created within the kernel keyring.
To verify that the key was created, run stratis key list:
# stratis key list
Key Description
pool1key
This verifies that the pool1key was created. Note that these keys are not persistent. If the host is rebooted, the key will need to be provided again before the encrypted Stratis pool can be accessed (this process is covered later).
If you have multiple encrypted pools, they can have a separate keys, or they can share the same key.
The keys can also be viewed using the following keyctl commands:
Now that a key has been created for Stratis, the next step is to create the encrypted Stratis pool. Encrypting a pool can only be done at pool creation. It isn’t currently possible to encrypt an existing pool.
Use the stratis pool create command to create a pool. Add –key-desc and the key description that you provided in the previous step (pool1key). This will signal to Stratis that the pool should be encrypted using the provided key. The below example creates the Stratis pool on /dev/vdb, and names it pool1. Be sure to specify an empty/available device on your system.
# stratis pool create --key-desc pool1key pool1 /dev/vdb
You can verify that the pool has been created with the stratis pool list command:
# stratis pool list
Name Total Physical Properties
pool1 4.98 GiB / 37.63 MiB / 4.95 GiB ~Ca, Cr
In the sample output shown above, ~Ca indicates that caching is disabled (the tilde negates the property). Cr indicates that encryption is enabled. Note that caching and encryption are mutually exclusive. Both features cannot be simultaneously enabled.
Next, create a filesystem. The below example, demonstrates creating a filesystem named filesystem1, mounting it at the /filesystem1 mountpoint, and creating a test file in the new filesystem:
# stratis filesystem create pool1 filesystem1
# mkdir /filesystem1
# mount /stratis/pool1/filesystem1 /filesystem1
# cd /filesystem1
# echo "this is a test file" > testfile
Access the encrypted pool after a reboot
When you reboot you’ll notice that Stratis no longer shows your encrypted pool or its block device:
# stratis pool list
Name Total Physical Properties
# stratis blockdev list
Pool Name Device Node Physical Size Tier
To access the encrypted pool, first re-create the key with the same key description and key data / passphrase that you used previously:
# stratis key set --capture-key pool1key
Enter desired key data followed by the return key:
Next, run the stratis pool unlock command, and verify that you can now see the pool and its block device:
# stratis pool unlock
# stratis pool list
Name Total Physical Properties
pool1 4.98 GiB / 583.65 MiB / 4.41 GiB ~Ca, Cr
# stratis blockdev list
Pool Name Device Node Physical Size Tier
pool1 /dev/dm-2 4.98 GiB Data
Next, mount the filesystem and verify that you can access the test file you created previously:
# mount /stratis/pool1/filesystem1 /filesystem1/
# cat /filesystem1/testfile
this is a test file
Use a systemd unit file to automatically unlock a Stratis pool at boot
It is possible to automatically unlock your Stratis pool at boot without manual intervention. However, a file containing the key must be available. Storing the key in a file might be a security concern in some environments.
The systemd unit file shown below provides a simple method to unlock a Stratis pool at boot and mount the filesystem. Feedback on a better/alternative methods is welcome. You can provide suggestions in the comment section at the end of this article.
Start by creating your key file with the following command. Be sure to substitute passphrase with the same key data / passphrase you entered previously.
LVM is a tool for logical volume management which includes allocating disks, striping, mirroring and resizing logical volumes. It is commonly used on Fedora installations (prior to BTRFS as default it was LVM+Ext4). But have you ever started up your system to find a message like the image above, after you logged in? Uh oh, Gnome just said the home volume is almost out of space! Luckily, there is likely some space sitting around in another volume, unused and ready to re-alocate. Here’s how to reclaim hard-drive space with LVM.
The key to easily re-alocate space between volumes is the Logical Volume Manager (LVM). Fedora 32 and before use LVM to divide disk space by default. This technology is similar to standard hard-drive partitions, but LVM is a lot more flexible. LVM enables not only flexible volume size management, but also advanced capabilities such as read-write snapshots, striping or mirroring data across multiple drives, using a high-speed drive as a cache for a slower drive, and much more. All of these advanced options can get a bit overwhelming, but resizing a volume is straight-forward.
LVM basics
The volume group serves as the main container in the LVM system. By default Fedora only defines a single volume group, but there can be as many as needed. Actual hard-drive and hard-drive partitions are added to the volume group as physical volumes. Physical volumes add available free space to the volume group. A typical Fedora install has one formatted boot partition, and the rest of the drive is a partition configured as an LVM physical volume.
Out of this pool of available space, the volume group allocates one or more logical volumes. These volumes are similar to hard-drive partitions, but without the limitation of contiguous space on the disk. LVM logical volumes can even span multiple devices! Just like hard-drive partitions, logical volumes have a defined size and can contain any filesystem which can then be mounted to specific directories.
What’s needed
Confirm the system uses LVM with the gnome-disks application, and make sure there is free space available in some other volume. Without space to reclaim from another volume, this guide isn’t useful. A Fedora live CD/USB is also needed. Any file system that needs to shrink must be unmounted. Running from a live image allows all the volumes on the hard-disk to remain unmounted, even important directories like / and /home.
Use gnome-disks to verify free space
A word of warning
No data should be lost by following this guide, but it does muck around with some very low-level and powerful commands. One mistake could destroy all data on the hard-drive. So backup all the data on the disk first!
Resizing LVM volumes
To begin, boot the Fedora live image and select Try Fedora at the dialog. Next, use the Run Command to launch the blivet-gui application (accessible by pressing Alt-F2, typing blivet-gui, then pressing enter). Select the volume group on the left under LVM. The logical volumes are on the right.
Explore logical volumes in blivet-gui
The logical volume labels consist of both the volume group name and the logical volume name. In the example, the volume group is “fedora_localhost-live” and there are “home”, “root”, and “swap” logical volumes allocated. To find the full volume, select each one, click on the gear icon, and choose resize. The slider in the resize dialog indicates the allowable sizes for the volume. The minimum value on the left is the space already in use within the file system, so this is the minimum possible volume size (without deleting data). The maximum value on the right is the greatest size the volume can have based on available free space in the volume group.
Resize dialog in blivet-gui
A grayed out resize option means the volume is full and there is no free space in the volume group. It’s time to change that! Look through all of the volumes to find one with plenty of extra space, like in the screenshot above. Move the slider to the left to set the new size. Free up enough space to be useful for the full volume, but still leave plenty of space for future data growth. Otherwise, this volume will be the next to fill up.
Click resize and note that a new item appears in the volume listing: free space. Now select the full volume that started this whole endeavor, and move the slider all the way to the right. Press resize and marvel at the new improved volume layout. However, nothing has changed on the hard drive yet. Click on the check-mark to commit the changes to disk.
Review changes in blivet-gui
Review the summary of the changes, and if everything looks right, click Ok to proceed. Wait for blivet-gui to finish. Now reboot back into the main Fedora install and enjoy all the new space in the previously full volume.
Planning for the future
It is challenging to know how much space any particular volume will need in the future. Instead of immediately allocating all available free space, consider leaving it free in the volume group. In fact, Fedora Server reserves space in the volume group by default. Extending a volume is possible while it is online and in use. No live image or reboot needed. When a volume is almost full, easily extend the volume using part of the available free space and keep working. Unfortunately the default disk manager, gnome-disks, does not support LVM volume resizing, so install blivet-gui for a graphical management tool. Alternately, there is a simple terminal command to extend a volume:
Reclaiming hard-drive space with LVM just scratches the surface of LVM capabilities. Most people, especially on the desktop, probably don’t need the more advanced features. However, LVM is there when the need arises, though it can get a bit complex to implement. BTRFS is the default filesystem, without LVM, starting with Fedora 33. BTRFS can be easier to manage while still flexible enough for most common usages. Check out the recent Fedora Magazine articles on BTRFS to learn more.
COPR is a collection of personal repositories for software that isn’t carried in Fedora. Some software doesn’t conform to standards that allow easy packaging. Or it may not meet other Fedora standards, despite being free and open-source. COPR can offer these projects outside the Fedora set of packages. Software in COPR isn’t supported by Fedora infrastructure or signed by the project. However, it can be a neat way to try new or experimental software.
This article presents a few new and interesting projects in COPR. If you’re new to using COPR, see the COPR User Documentation for how to get started.
Dialect
Dialect translates text to foreign languages using Google Translate. It remembers your translation history and supports features such as automatic language detection and text to speech. The user interface is minimalistic and mimics the Google Translate tool itself, so it is really easy to use.
Installation instructions
The repo currently provides Dialect for Fedora 33 and Fedora Rawhide. To install it, use these commands:
gh is an official GitHub command-line client. It provides fast access and full control over your project issues, pull requests, and releases, right in the terminal. Issues (and everything else) can also be easily viewed in the web browser for a more standard user interface or sharing with others.
Installation instructions
The repo currently provides gh for Fedora 33 and Fedora Rawhide. To install it, use these commands:
Glide is a minimalistic media player based on GStreamer. It can play both local and remote files in any multimedia format supported by GStreamer itself. If you are in need of a multi-platform media player with a simple user interface, you might want to give Glide a try.
Installation instructions
The repo currently provides Glide for Fedora 32, 33, and Rawhide. To install it, use these commands:
ALE is a plugin for Vim text editor, providing syntax and semantic error checking. It also brings support for fixing code and many other IDE-like features such as TAB-completion, jumping to definitions, finding references, viewing documentation, etc.
Installation instructions
The repo currently provides vim-ale for Fedora 31, 32, 33, and Rawhide, as well as for EPEL8. To install it, use these commands:
Silverblue is an operating system for your desktop built on Fedora. It’s excellent for daily use, development, and container-based workflows. It offers numerous advantages such as being able to roll back in case of any problems. If you want to update to Fedora 33 on your Silverblue system, this article tells you how. It not only shows you what to do, but also how to revert things if something unforeseen happens.
Prior to actually doing the rebase to Fedora 33, you should apply any pending updates. Enter the following in the terminal:
$ rpm-ostree update
or install updates through GNOME Software and reboot.
Rebasing using GNOME Software
The GNOME Software shows you that there is new version of Fedora available on the Updates screen.
Fedora 33 is available
First thing you need to do is to download the new image, so click on the Download button. This will take some time and after it’s done you will see that the update is ready to install.
Fedora 33 is ready for installation
Click on the Install button. This step will take only a few moments and then you will be prompted to restart your computer.
Restart is needed to rebase to Fedora 33 Silverblue
Click on Restart button and you are done. After restart you will end up in new and shiny release of Fedora 33. Easy, isn’t it?
Rebasing using terminal
If you prefer to do everything in a terminal, than this next guide is for you.
Rebasing to Fedora 33 using terminal is easy. First, check if the 33 branch is available:
$ ostree remote refs fedora
You should see the following in the output:
fedora:fedora/33/x86_64/silverblue
Next, rebase your system to the Fedora 33 branch.
$ rpm-ostree rebase fedora:fedora/33/x86_64/silverblue
Finally, the last thing to do is restart your computer and boot to Fedora 33.
How to roll back
If anything bad happens—for instance, if you can’t boot to Fedora 33 at all—it’s easy to go back. Pick the previous entry in the GRUB menu at boot, and your system will start in its previous state before switching to Fedora 33. To make this change permanent, use the following command:
$ rpm-ostree rollback
That’s it. Now you know how to rebase Silverblue to Fedora 33 and roll back. So why not do it today?
Fedora 33 Workstation is the latest release of our free, leading-edge operating system. You can download it from the official website here right now. There are several new and noteworthy changes in Fedora 33 Workstation. Read more details below.
GNOME 3.38
Fedora 33 Workstation includes the latest release of GNOME Desktop Environment for users of all types. GNOME 3.38 in Fedora 33 Workstation includes many updates and improvements, including:
A new GNOME Tour app
New users are now greeted by “a new Tour application, highlighting the main functionality of the desktop and providing first time users a nice welcome to GNOME.”
The new GNOME Tour application in Fedora 33
Drag to reorder apps
GNOME 3.38 replaces the previously split Frequent and All apps views with a single customizable and consistent view that allows you to reorder apps and organize them into custom folders. Simply click and drag to move apps around.
GNOME 3.38 Drag to Reorder
Improved screen recording
The screen recording infrastructure in GNOME Shell has been improved to take advantage of PipeWire and kernel APIs. This will help reduce resource consumption and improve responsiveness.
GNOME 3.38 also provides many additional features and enhancements. Check out the GNOME 3.38 Release Notes for further information.
B-tree file system
As announced previously, new installations of Fedora 33 will default to using Btrfs. Features and enhancements are added to Btrfs with each new kernel release. The change log has a complete summary of the features that each new kernel version brings to Btrfs.
Swap on ZRAM
Anaconda and Fedora IoT have been using swap-on-zram by default for years. With Fedora 33, swap-on-zram will be enabled by default instead of a swap partition. Check out the Fedora wiki page for more details about swap-on-zram.
Nano by default
Fresh Fedora 33 installations will set the EDITOR environment variable to nano by default. This change affects several command line tools that spawn a text editor when they require user input. With earlier releases, this environment variable default was unspecified, leaving it up to the individual application to pick a default editor. Typically, applications would use vi as their default editor due to it being a small application that is traditionally available on the base installation of most Unix/Linux operating systems. Since Fedora 33 includes nano in its base installation, and since nano is more intuitive for a beginning user to use, Fedora 33 will use nano by default. Users who want vi can, of course, override the value of the EDITOR variable in their own environment. See the Fedora change request for more details.
Today, I’m excited to share the results of the hard work of thousands of contributors to the Fedora Project: our latest release, Fedora 33, is here! This is a big release with a lot of change, but I believe all that work will also make it a comfortable one, fulfilling our goal of bringing you the latest stable, powerful, and robust free and open source software in many easy to use offerings.
If you just want to get to the bits without delay, head over to https://getfedora.org/ right now. For details, read on!
Find the Fedora flavor that’s right for you!
Fedora Editions are targeted outputs geared toward specific “showcase” uses on the desktop, in server and cloud environments—and now for Internet of Things as well.
Fedora Workstation focuses on the desktop, and in particular, it’s geared toward software developers who want a “just works” Linux operating system experience. This release features GNOME 3.38, which has plenty of great improvements as usual. The addition of the Tour application helps new users learn their way around. And like all of our other desktop-oriented variants, Fedora Workstation now uses BTRFS as the default filesystem. This advanced filesystem lays the foundation for bringing a lot of great enhancements in upcoming releases. For your visual enjoyment, Fedora 33 Workstation now features an animated background (based on time of day) by default.
Fedora CoreOS is an emerging Fedora Edition. It’s an automatically-updating, minimal operating system for running containerized workloads securely and at scale. It offers several update streams that can be followed for automatic updates that occur roughly every two weeks. Currently the next stream is based on Fedora 33, with the testing and stable streams to follow. You can find information about released artifacts that follow the next stream from the download page and information about how to use those artifacts in the Fedora CoreOS Documentation.
Fedora IoT, newly promoted to Edition status, provides a strong foundation for IoT ecosystems and edge computing use cases. Among many other features, Fedora 33 IoT introduces the Platform AbstRaction for SECurity (PARSEC), an open-source initiative to provide a common API to hardware security and cryptographic services in a platform-agnostic way.
Of course, we produce more than just the Editions. Fedora Spins and Labs target a variety of audiences and use cases, including Fedora CompNeuro, which brings a plethora of open source computational modelling tools for neuroscience, and desktop environments like KDE Plasma and Xfce.
And, don’t forget our alternate architectures: ARM AArch64, Power, and S390x. New in Fedora 33, AArch64 users can use the .NET Core language for cross-platform development. We have improved support for Pine64 devices, NVidia Jetson 64 bit platforms, and the Rockchip system-on-a-chip devices including the Rock960, RockPro64, and Rock64. (However, a late-breaking note: there may be problems booting on some of these devices. Upgrading from existing Fedora 32 will be fine. More info will be on the Common Bugs page as we have it.)
We’re also excited to announce that the Fedora Cloud Base Image and Fedora CoreOS will be available in Amazon’s AWS Marketplace for the first time with Fedora 33. Fedora cloud images have been available in the Amazon cloud for over a decade, and you can launch our official images by AMI ID or with a click. The Marketplace provides an alternate way to get the same thing, with significantly wider visibility for Fedora. This will also make our cloud images available in new AWS regions more quickly. Thank you especially to David Duncan for making this happen!
General improvements
No matter what variant of Fedora you use, you’re getting the latest the open source world has to offer. Following our “First” foundation, we’ve updated key programming language and system library packages, including Python 3.9, Ruby on Rails 6.0, and Perl 5.32. In Fedora KDE, we’ve followed the work in Fedora 32 Workstation and enabled the EarlyOOM service by default to improve the user experience in low-memory situations.
To make the default Fedora experience better, we’ve set nano as the default editor. nano is a friendly editor for new users. Those of you who want the power of editors like vi can, of course, set your own default.
We’re excited for you to try out the new release! Go to https://getfedora.org/ and download it now. Or if you’re already running a Fedora operating system, follow the easy upgrade instructions. For more information on the new features in Fedora 33, see the release notes.
A note on Secure Boot
Secure Boot is a security standard which ensures that only officially-signed operating system software can load on your computer. This is important for preventing persistent malware which could hide itself in your computer’s firmware and survive even an operating system reinstallation. However, in the wake of the Boot Hole vulnerability, the cryptographic certificate used to sign Fedora bootloader software will be revoked and replaced with a new one. Because this will have a broad impact, revocation should not happen widely until the second quarter of 2021 or later.
However, some users may have received this revocation from other operating systems or firmware updates already. In that case, Fedora installations will not boot with Secure Boot enabled. To be clear, this will not affect most users. If it does affect you, you can boot with Secure Boot disabled for the time being. We will release an update signed with the new certificate to be available on all supported releases well before broad-scale certificate revocation takes place, and at that point Secure Boot should be reenabled.
In the unlikely event of a problem….
If you run into a problem, check out the Fedora 33 Common Bugs page, and if you have questions, visit our Ask Fedora user-support platform.
Thank you everyone
Thanks to the thousands of people who contributed to the Fedora Project in this release cycle, and especially to those of you who worked extra hard to make this another on-time release during a pandemic. Fedora is a community, and it’s great to see how much we’ve supported each other.
The kernel team is working on final integration for kernel 5.9. This version was just recently released, and will arrive soon in Fedora. As a result, the Fedora kernel and QA teams have organized a test week from Monday, October 26, 2020 through Monday, November 02, 2020. Refer to the wiki page for links to the test images you’ll need to participate. Read below for details.
How does a test week work?
A test week is an event where anyone can help make sure changes in Fedora work well in an upcoming release. Fedora community members often participate, and the public is welcome at these events. If you’ve never contributed before, this is a perfect way to get started.
To contribute, you only need to be able to do the following things:
Download test materials, which include some large files
Read and follow directions step by step
The wiki page for the kernel test day has a lot of good information on what and how to test. After you’ve done some testing, you can log your results in the test day web application. If you’re available on or around the day of the event, please do some testing and report your results. We have a document which provides all the steps written.
Happy testing, and we hope to see you on test day.
Many computers use the Network Time Protocol (NTP) to synchronize their system clocks over the internet. NTP is one of the few unsecured internet protocols still in common use. An attacker that can observe network traffic between a client and server can feed the client with bogus data and, depending on the client’s implementation and configuration, force it to set its system clock to any time and date. Some programs and services might not work if the client’s system clock is not accurate. For example, a web browser will not work correctly if the web servers’ certificates appear to be expired according to the client’s system clock. Use Network Time Security (NTS) to secure NTP.
Fedora 331 is the first Fedora release to support NTS. NTS is a new authentication mechanism for NTP. It enables clients to verify that the packets they receive from the server have not been modified while in transit. The only thing an attacker can do when NTS is enabled is drop or delay packets. See RFC8915 for further details about NTS.
NTP can be secured well with symmetric keys. Unfortunately, the server has to have a different key for each client and the keys have to be securely distributed. That might be practical with a private server on a local network, but it does not scale to a public server with millions of clients.
NTS includes a Key Establishment (NTS-KE) protocol that automatically creates the encryption keys used between the server and its clients. It uses Transport Layer Security (TLS) on TCP port 4460. It is designed to scale to very large numbers of clients with a minimal impact on accuracy. The server does not need to keep any client-specific state. It provides clients with cookies, which are encrypted and contain the keys needed to authenticate the NTP packets. Privacy is one of the goals of NTS. The client gets a new cookie with each server response, so it doesn’t have to reuse cookies. This prevents passive observers from tracking clients migrating between networks.
The default NTP client in Fedora is chrony. Chrony added NTS support in version 4.0. The default configuration hasn’t changed. Chrony still uses public servers from the pool.ntp.org project and NTS is not enabled by default.
Currently, there are very few public NTP servers that support NTS. The two major providers are Cloudflare and Netnod. The Cloudflare servers are in various places around the world. They use anycast addresses that should allow most clients to reach a close server. The Netnod servers are located in Sweden. In the future we will probably see more public NTP servers with NTS support.
A general recommendation for configuring NTP clients for best reliability is to have at least three working servers. For best accuracy, it is recommended to select close servers to minimize network latency and asymmetry caused by asymmetric network routing. If you are not concerned about fine-grained accuracy, you can ignore this recommendation and use any NTS servers you trust, no matter where they are located.
If you do want high accuracy, but you don’t have a close NTS server, you can mix distant NTS servers with closer non-NTS servers. However, such a configuration is less secure than a configuration using NTS servers only. The attackers still cannot force the client to accept arbitrary time, but they do have a greater control over the client’s clock and its estimate of accuracy, which may be unacceptable in some environments.
Enable client NTS in the installer
When installing Fedora 33, you can enable NTS in the Time & Date dialog in the Network Time configuration. Enter the name of the server and check the NTS support before clicking the + (Add) button. You can add one or more servers or pools with NTS. To remove the default pool of servers (2.fedora.pool.ntp.org), uncheck the corresponding mark in the Use column.
Network Time configuration in
Fedora installer
Enable client NTS in the configuration file
If you upgraded from a previous Fedora release, or you didn’t enable NTS in the installer, you can enable NTS directly in /etc/chrony.conf. Specify the server with the nts option in addition to the recommended iburst option. For example:
server time.cloudflare.com iburst nts
server nts.sth1.ntp.se iburst nts
server nts.sth2.ntp.se iburst nts
You should also allow the client to save the NTS keys and cookies to disk,
so it doesn’t have to repeat the NTS-KE session on each start. Add the
following line to chrony.conf, if it is not already present:
ntsdumpdir /var/lib/chrony
If you don’t want NTP servers provided by DHCP to be mixed with the servers you
have specified, remove or comment out the following line in chrony.conf:
sourcedir /run/chrony-dhcp
After you have finished editing chrony.conf, save your changes and restart the chronyd service:
systemctl restart chronyd
Check client status
Run the following command under the root user to check whether the NTS key
establishment was successful:
The KeyID, Type, and KLen columns should have non-zero values. If they are zero, check the system log for error messages from chronyd. One possible cause of failure is a firewall is blocking the client’s connection to the server’s TCP port ( port 4460).
Another possible cause of failure is a certificate that is failing to verify because the client’s clock is wrong. This is a chicken-or-the-egg type problem with NTS. You may need to manually correct the date or temporarily disable NTS in order to get NTS working. If your computer has a real-time clock, as almost all computers do, and it’s backed up by a good battery, this operation should be needed only once.
If the computer doesn’t have a real-time clock or battery, as is common with
some small ARM computers like the Raspberry Pi, you can add the -s
option to /etc/sysconfig/chronyd to restore time saved on the last
shutdown or reboot. The clock will be behind the true time, but if the
computer wasn’t shut down for too long and the server’s certificates were not
renewed too close to their expiration, it should be sufficient for the time
checks to succeed. As a last resort, you can disable the time checks with the nocerttimecheck directive. See the chrony.conf(5) man page
for details.
Run the following command to confirm that the client is making NTP
measurements:
The Reach column should have a non-zero value; ideally 377. The value 377 shown above is an octal number. It indicates that the last eight requests all had a valid response. The validation check will include NTS authentication if enabled. If the value only rarely or never gets to 377, it indicates that NTP requests or responses are getting lost in the network. Some major network operators are known to have middleboxes that block or limit rate of large NTP packets as a mitigation for amplification attacks that exploit the monitoring protocol of ntpd. Unfortunately, this impacts NTS-protected NTP packets, even though they don’t cause any amplification. The NTP working group is considering an alternative port for NTP as a workaround for this issue.
Enable NTS on the server
If you have your own NTP server running chronyd, you can enable server NTS support to allow its clients to be synchronized securely. If the server is a client of other servers, it should use NTS or a symmetric key for its own synchronization. The clients assume the synchronization chain is secured between all servers up to the primary time servers.
Enabling server NTS is similar to enabling HTTPS on a web server. You just need a private key and certificate. The certificate could be signed by the Let’s Encrypt authority using the certbot tool, for example. When you have the key and certificate file (including intermediate certificates), specify them in chrony.conf with the following directives:
Make sure the ntsdumpdir directive mentioned previously in the
client configuration is present in chrony.conf. It allows the server
to save its keys to disk, so the clients of the server don’t have to get new
keys and cookies when the server is restarted.
Restart the chronyd service:
systemctl restart chronyd
If there are no error messages in the system log from chronyd, it should be
accepting client connections. If the server has a firewall, it needs to allow
both the UDP 123 and TCP 4460 ports for NTP and NTS-KE respectively.
You can perform a quick test from a client machine with the following command:
$ chronyd -Q -t 3 'server foo.example.net iburst nts maxsamples 1'
2020-10-13T12:00:52Z chronyd version 4.0 starting (+CMDMON +NTP +REFCLOCK +RTC +PRIVDROP +SCFILTER +SIGND +ASYNCDNS +NTS +SECHASH +IPV6 +DEBUG)
2020-10-13T12:00:52Z Disabled control of system clock
2020-10-13T12:00:55Z System clock wrong by -0.001032 seconds (ignored)
2020-10-13T12:00:55Z chronyd exiting
If you see a System clock wrong message, it’s working
correctly.
On the server, you can use the following command to check how many NTS-KE
connections and authenticated NTP packets it has handled:
If you see non-zero NTS-KE connections accepted and Authenticated
NTP packets, it means at least some clients were able to connect to the
NTS-KE port and send an authenticated NTP request.
— Cover photo by Louis. K on Unsplash —
1. The Fedora 33 Beta installer contains an older chrony prerelease which doesn’t work with current NTS servers because the NTS-KE port has changed. Consequently, in the Network Time configuration in the installer, the servers will always appear as not working. After installation, the chrony package needs to be updated before it will work with current servers.
This article explains how to make incremental or differential backups, with a catalog available to restore (or export) at the point you want, with Butterfly Backup.
Requirements
Butterfly Backup is a simple wrapper of rsync written in python; the first requirement is python3.3 or higher (plus module cryptography for init action). Other requirements are openssh and rsync (version 2.5 or higher). Ok, let’s go!
[Editors note: rsync version 3.2.3 is already installed on Fedora 33 systems]
After that, installing Butterfly Backup is very simple by using the following commands to clone the repository locally, and set up Butterfly Backup for use:
$ git clone https://github.com/MatteoGuadrini/Butterfly-Backup.git
$ cd Butterfly-Backup
$ sudo python3 setup.py
$ bb --help
$ man bb
To upgrade, you would use the same commands too.
Example
Butterfly Backup is a server to client tool and is installed on a server (or workstation). The restore process restores the files into the specified client. This process shares some of the options available to the backup process.
Backups are organized accord to precise catalog; this is an example:
$ tree destination/of/backup
.
├── destination
│ ├── hostname or ip of the PC under backup
│ │ ├── timestamp folder
│ │ │ ├── backup folders
│ │ │ ├── backup.log
│ │ │ └── restore.log
│ │ ├─── general.log
│ │ └─── symlink of last backup
│
├── export.log
├── backup.list
└── .catalog.cfg
Butterfly Backup has six main operations, referred to as actions, you can get information about them with the –help command.
$ bb --help
usage: bb [-h] [--verbose] [--log] [--dry-run] [--version] {config,backup,restore,archive,list,export} ... Butterfly Backup optional arguments: -h, --help show this help message and exit --verbose, -v Enable verbosity --log, -l Create a log --dry-run, -N Dry run mode --version, -V Print version action: Valid action {config,backup,restore,archive,list,export} Available actions config Configuration options backup Backup options restore Restore options archive Archive options list List options export Export options
Configuration
Configuration mode is straight forward; If you’re already familiar with the exchange keys and OpenSSH, you probably won’t need it. First, you must create a configuration (rsa keys), for instance:
$ bb config --new
SUCCESS: New configuration successfully created!
After creating the configuration, the keys will be installed (copied) on the hosts you want to backup:
$ bb config --deploy host1
Copying configuration to host1; write the password:
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/home/arthur/.ssh/id_rsa.pub"
/usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
/usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
arthur@host1's password: Number of key(s) added: 1 Now try logging into the machine, with: "ssh 'arthur@host1'"
and check to make sure that only the key(s) you wanted were added. SUCCESS: Configuration copied successfully on host1!
Backup
There are two backup modes: single and bulk. The most relevant features of the two backup modes are the parallelism and retention of old backups. See the two parameters –parallel and –retention in the documentation.
Single backup
The backup of a single machine consists in taking the files and folders indicated in the command line, and putting them into the cataloging structure indicated above. In other words, copy all file and folders of a machine into a path.
Above all, bulk mode backups share the same options as single mode, with the difference that they accept a file containing a list of hostnames or ips. In this mode backups will performed in parallel (by default 5 machines at a time). Above all, if you want to run fewer or more machines in parallel, specify the –parallel parameter.
Incremental of the previous backup, for instance:
$ cat /home/arthur/pclist.txt
host1
host2
host3
$ bb backup --list /home/arthur/pclist.txt --destination /mnt/backup --data User Config --type Unix
ERROR: The port 22 on host2 is closed!
ERROR: The port 22 on host3 is closed!
Start backup on host1
SUCCESS: Command rsync -ahu --no-links --link-dest=/mnt/backup/host1/2020_09_19__10_28 arthur@host1:/home :/etc /mnt/backup/host1/2020_09_19__10_50
There are four backup modes, which you specify with the –mode flag: Full (backup all files) , Mirror (backup all files in mirror mode), Differential (is based on the latest Full backup) and Incremental (is based on the latest backup). The default mode is Incremental; Full mode is set by default when the flag is not specified.
Listing catalog
The first time you run backup commands, the catalog is created. The catalog is used for future backups and all the restores that are made through Butterfly Backup. To query this catalog use the list command. First, let’s query the catalog in our example:
$ bb list --catalog /mnt/backup BUTTERFLY BACKUP CATALOG Backup id: aba860b0-9944-11e8-a93f-005056a664e0
Hostname or ip: host1
Timestamp: 2020-09-19 10:28:12 Backup id: dd6de2f2-9a1e-11e8-82b0-005056a664e0
Hostname or ip: host1
Timestamp: 2020-09-19 10:50:59
To export the catalog list use it with an external tool like cat, include the ––log flag:
$ bb list --catalog /mnt/backup --log
$ cat /mnt/backup/backup.list
Restore
The restore process is the exact opposite of the backup process. It takes the files from a specific backup and push it to the destination computer. This command perform a restore on the same machine of the backup, for instance:
$ bb restore --catalog /mnt/backup --backup-id dd6de2f2-9a1e-11e8-82b0-005056a664e0 --computer host1 --log
Want to do restore path /mnt/backup/host1/2020_09_19__10_50/etc? To continue [Y/N]? y
Want to do restore path /mnt/backup/host1/2020_09_19__10_50/home? To continue [Y/N]? y
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2020_09_19__10_50/restore.log /mnt/backup/host1/2020_09_19__10_50/etc arthur@host1:/restore_2020_09_19__10_50
SUCCESS: Command rsync -ahu -vP --log-file=/mnt/backup/host1/2020_09_19__10_50/restore.log /mnt/backup/host1/2020_09_19__10_50/home/* arthur@host1:/home
Without specifying the “type” flag that indicates the operating system on which the data are being retrieved, Butterfly Backup will select it directly from the catalog via the backup-id.
Archive old backup
Archive operations are used to store backups by saving disk space.
Butterfly Backup was born from a very complex need; this tool provides superpowers to rsync, automates the backup and restore process. In addition, the catalog allows you to have a system similar to a “time machine”.
In conclusion, Butterfly Backup is a lightweight, versatile, simple and scriptable backup tool.
