Posted on Leave a comment

Fedora Job Opening: Community Action and Impact Coordinator (FCAIC)

It is bittersweet to announce that I have decided to move on from my role as the Fedora Community Action and Impact Coordinator (FCAIC). For me, this role has been full of growth, unexpected challenges, and so much joy. It has been a privilege to help guide our wonderful community through challenges of the last three years. I’m excited to see what the next FCAIC can do for Fedora. If you’re interested in applying, see the FCAIC job posting on Red Hat Jobs and read more about the role below. 

Adapting to Uncertain Times

When I applied back in 2019, a big part of the job description was to travel the globe, connecting with and supporting Fedora communities worldwide. As we all know, that wasn’t possible with the onset of COVID-19 and everything that comes with a pandemic. 

Instead, I learned how to create virtual experiences for Fedora, connect with people solely in a virtual environment, and support contributors from afar. Virtual events have been a HUGE success for Fedora. The community has shown up for those events in such a wonderful way. We have almost tripled our participation in our virtual events since the first Release Party in 2020. We have more than doubled the number of respondents to the Annual Contributor Survey over last year’s turnout. I am proud of the work I have accomplished and even more so how much the community has grown and adapted to a very challenging couple of years.

What’s next for me

As some of you may know, I picked up the Code of Conduct (CoC) work that my predecessor Brian Exelbierd (Bex) started for Fedora. After the Fedora Council approved the new CoC, I then got started on additional pieces of related work: Supplemental Documentation and Moderation Guidelines. I am also working on expanding the small Code of Conduct Committee(CoCC) to include more community members. As a part of the current CoCC, I have helped to deal with the majority of the incidents throughout my time as FCAIC. 

Because of my experience with all this CoC work, I will be moving into a new role inside of Red Hat’s OSPO: Code of Conduct Specialist. I will be assisting other Community Architects (like the FCAIC role) to help roll out CoC’s and governance around them, as well as collaborating with other communities to develop a Community of Practice around this work. I am excited and determined to take on this new challenge and very proud to be a part of an organization that values work that prioritizes safety and inclusion. 

What’s next for Fedora

This is an amazing opportunity for the Fedora community to grow in new and exciting ways. Every FCAIC brings their own approach to this role as well as their own ideas, strengths, and energy. I will be working with Matthew Miller, Ben Cotton, and Red Hat to help hire and onboard the new Fedora Community Action and Impact Coordinator. I will continue as FCAIC until we hire someone new, and will help transition them into the role. Additionally, I will offer support, advice, and guidance as others who have moved on have done for me. I am eager to see who comes next and how I can help them become a success. And, as I have for years prior to my tenure as FCAIC, I will continue to participate in the community, albeit in different ways. 

This means we are looking for a new FCAIC! Do you love Fedora? Do you want to help support and grow the community full time? This is the core of what the FCAIC does. The job description has a list of the primary job responsibilities and required skills- but that is just a taste of what is required and what it is to support the Fedora community full time. Day-to-day work includes working with the Mindshare Committee, managing the Fedora budget, and being a part of many other teams and in many places. You should be ready and excited to write about Fedora’s achievements, policies, as well as generate strategies to help the community succeed. And, of course, there is event planning and support (Flock, Nest, Hatch, Release Parties, etc). It can be tough work, but it is a lot of fun and wonderfully rewarding to help Fedora thrive. 

How to apply

Do you enjoy working with people all over the world, with a variety of skills and interests? Are you good at setting long term goals and seeing them through to completion? Can you set priorities, follow through, and know when to say “no” in order to focus on the most important tasks for success? Are you excited about building not only a successful Linux distribution, but also a healthy project? Is Fedora’s mission deeply important to you? If you said “yes” to these questions, you might be a great candidate for the FCAIC role. If you think you’re a great fit, please apply online, or contact Marie Nordin, or Jason Brooks.

Posted on Leave a comment

Using Linux System Roles to implement Clevis and Tang for automated LUKS volume unlocking

One of the key aspects of system security is encrypting storage at rest. Without encrypted storage, any time a storage device leaves your presence it can be at risk. The most obvious scenario where this can happen is if a storage device (either just the storage device or the entire system, server, or laptop) is lost or stolen.

However, there are other scenarios that are a concern as well: perhaps you have a storage device fail, and it is replaced under warranty — many times the vendor will ask you to return the original device. If the device was encrypted, it is much less of a concern to return it back to the hardware vendor.

Another concern is anytime your storage device is out of sight there is a risk that the data is copied or cloned off of the device without you even being aware. Again, if the device is encrypted, this is much less of a concern.

Fedora (and other Linux distributions) include the Linux Unified Key Setup (LUKS) functionality to support disk encryption. LUKS is easy to use, and is even integrated as an option in the Fedora Anaconda installer.

However there is one challenge that frequently prevents people from implementing LUKS on a large scale, especially for the root filesystem: every single time you reboot the host you generally have to manually access the console and type in the LUKS passphrase so the system can boot up.

If you are running Fedora on a single laptop, this might not be a problem, after all, you probably are sitting in front of your laptop any time you reboot it. However, if you have a large number of Fedora instances, this quickly becomes impractical to deal with.

If you have hundreds of systems, it is impractical to manually type the LUKS passphrase on each system on every reboot

You might be managing Fedora systems that are at remote locations, and you might not even have good or reliable ways to access a console on them. In this case, rebooting the hosts could result in them not coming up until you or someone else travels to their location to type in the LUKS passphrase.

This article will cover how to implement a solution to enable automated LUKS volume unlocking (and the process to implement these features will be done using automation as well!)

Overview of Clevis and Tang

Clevis and Tang are an innovative solution that can help with the challenge of having systems with encrypted storage boot up without manual user intervention on every boot. At a high level, Clevis, which is installed on the client systems, can enable LUKS volumes to be unlocked without user intervention as long as the client system has network access to a configurable number of Tang servers.

The basic premise is that the Tang server(s) are on an internal/private or otherwise secured network, and if the storage devices are lost, stolen, or otherwise removed from the environment, that they would no longer have network access to the Tang server(s), and thus no longer automatically unlock automatically at boot.

Tang is stateless and doesn’t require authentication or even TLS, which means it is very lightweight and easy to configure, and can run from a container. In this article, I’m only setting up a single Tang server, however it is also possible to have multiple Tang servers in an environment, and to configure the number Tang servers the Clevis clients must connect to in order to unlock the encrypted volume. For example, you could have three Tang servers, and require the Clevis clients to be able to connect to at least two of the three Tang servers.

For more information on how Tang and Clevis work, refer to the GitHub pages: Clevis and Tang, or for an overview of the inner workings of Tang and Clevis, refer to the Securing Automated Decryption New Cryptography and Techniques FOSDEM talk.

Overview of Linux System Roles

Linux System Roles is a set of Ansible Roles/Collections that can help automate the configuration and management of many aspects of Fedora, CentOS Stream, RHEL, and RHEL derivatives. Linux System Roles is packaged in Fedora as an RPM (linux-system-roles) and is also available on Ansible Galaxy (as both roles and as a collection). For more information on Linux System Roles, and to see a list of included roles, refer to the Linux System Roles project page.

Included in the list of Linux System Roles are the nbde_client, nbde_server, and firewall roles that will be used in this article. The nbde_client and nbde_server roles are focused on automating the implementation of Clevis and Tang, respectively. The “nbde” in the role names stands for network bound disk encryption, which is another term to refer to using Clevis and Tang for automated unlocking of LUKS encrypted volumes. The firewall role can automate the implementation of firewall settings, and will be used to open a port in the firewall on the Tang server.

Demo environment overview

In my environment, I have a Raspberry Pi, running Fedora 36 that I will install Linux System Roles on and use as my Ansible control node. In addition, I’ll use this same Raspberry Pi as my Tang server. This device is configured with the pi.example.com hostname.

In addition, I have four other systems in my environment: two Fedora 36 systems, and two CentOS Stream 9 systems, named fedora-server1.example.com, fedora-server2.example.com, c9s-server1.example.com, and c9s-server2.example.com. Each of these four systems has a LUKS encrypted root filesystem and currently the LUKS passphrase must be manually typed in each time the systems boot up.

I’ll use the nbde_server and firewall roles to install and configure Tang on my pi.example.com system, and use the nbde_client role to install and configure Clevis on my four other systems, enabling them to automatically unlock their encrypted root filesystem if they can connect to the pi.example.com Tang system.

Installing Linux System Roles and Ansible on the Raspberry Pi

I’ll start by installing the linux-system-roles package on the pi.example.com host, which will act as my Ansible control node. This will also install ansible-core and several other packages as dependencies. These packages do not need to be installed on the other four systems in my environment (which are referred to as managed nodes).

$ sudo dnf install linux-system-roles

SSH keys and sudo configuration need to be configured so that the control node host can connect to each of the managed nodes in the environment and escalate to root privileges.

Defining the Ansible inventory file

Still on the pi.example.com host, I’ll create an Ansible inventory file to group the five systems in my environment into two Ansible inventory groups. The nbde_servers group will contain a list of hosts that I would like to configure as Tang servers (which in this example is only the pi.example.com host), and the nbde_clients group will contain a list of hosts that I would like to configure as Clevis clients. I’ll name this inventory file inventory.yml and it contains the following content:

all: children: nbde_servers: hosts: pi.example.com: nbde_clients: hosts: fedora35-server1.example.com: fedora35-server2.example.com: c9s-server1.example.com: c9s-server2.example.com:

Creating Ansible Group variable files

Ansible variables are set to specify what configuration should be implemented by the Linux System Roles. Each role has a README.md file that contains important information on how to use each role, including a list of available role variables. The README.md files for the nbde_server, nbde_client, and firewall roles are available in the following locations, respectively:

  • /usr/share/doc/linux-system-roles/nbde_server/README.md
  • /usr/share/doc/linux-system-roles/nbde_client/README.md
  • /usr/share/doc/linux-system-roles/firewall/README.md

I’ll create a group_vars directory with the mkdir group_vars command. Within this directory, I’ll create a nbde_servers.yml file and nbde_clients.yml file, which will define, respectively, the variables that should be set for systems listed in the nbde_servers inventory group and the nbde_clients inventory group.

The nbde_servers.yml file contains the following content, which will instruct the firewall role to open TCP port 80, which is the default port used by Tang:

firewall: - port: ['80/tcp'] state: enabled

The nbde_clients.yml file contains the following content:

nbde_client_bindings: - device: /dev/vda2 encryption_password: !vault | $ANSIBLE_VAULT;1.1;AES256 62666465373138636165326639633... servers: - http://pi.example.com

Under nbde_client_bindings, device specifies the backing device of the encrypted root filesystem on the four managed nodes. The encryption_password specifies a current LUKS passphrase that is required to configure Clevis. In this example, I’ve used ansible-vault to encrypt the string rather than place the LUKS passphrase in clear text. And finally, under servers, a list of Tang servers that Clevis should bind to are specified. In this example, the Clevis clients will be configured to bind to the pi.example.com Tang server.

Creating the playbook

I’ll create a simple Ansible playbook, named nbde.yml that will call the firewall and nbde_server roles for systems in the nbde_servers inventory group, and call the nbde_client role for systems in the nbde_clients group:

- name: Open firewall for Tang hosts: nbde_servers roles: - linux-system-roles.firewall - name: Deploy NBDE Tang server hosts: nbde_servers roles: - linux-system-roles.nbde_server - name: Deploy NBDE Clevis clients hosts: nbde_clients roles: - linux-system-roles.nbde_client

At this point, I have the following files and directories created:

  • inventory.yml
  • nbde.yml
  • group_vars/nbde_clients.yml
  • group_vars/nbde_servers.yml

Running the playbook

The nbde.yml playbook can be run with the following command:

$ ansible-playbook nbde.yml -i inventory.yml --ask-vault-pass -b

The -i flag specifies which inventory file should be used, the –ask-vault-pass flag will prompt for the Ansible Vault password to decrypt the encryption_password variable, and the -b flag specifies that Ansible should escalate to root privileges.

play recap output from ansible-playbook command showing playbook successfully completed

Validating the configuration

To validate the configuration, I rebooted each of my four managed nodes that were configured as Clevis clients of the Raspberry Pi Tang server. Each of the four managed nodes boots up and briefly pauses on the LUKS passphrase prompt:

Systems boot up to LUKS passphrase prompt, and automatically continue booting after a brief pause

However, after the brief delay, each of the four systems continued booting up without requiring me to enter the LUKS passphrase.

Conclusion

If you would like to secure your data at rest with LUKS encryption, but need a solution that enables systems to boot up without intervention, consider implementing Clevis and Tang. Linux System Roles can help you implement Clevis and Tang, as well as a number of other aspects of your system, in an automated manner.

Posted on Leave a comment

Fedora Workstation’s State of Gaming – A Case Study of Far Cry 5 (2018)

First-person shooter video games are a great proving ground for strategies that make you finish on the top, reflexes that help you to shoot before getting shot and agility that adjusts you to whatever a situation throws at you. Add the open-ended nature brought in by large intricately-designed worlds into the mix, and it dials the player experience to eleven and, with that, it also becomes great evidence of what a platform is capable of. Needless to say, I have been a great fan of open-world first-person shooter games. And Ubisoft’s Far Cry series happens to be the one which remains closest to my heart. So I tried the (second) most recent release in the long-running series, Far Cry 5 which came out in 2018, on Fedora Workstation 35 to see how it performs.

Just like in my previous case study, the testing hardware has an AMD RDNA2-based GPU, where the video game was configured to the highest possible graphical preset to stress the hardware into performing as much as its limiting factor. To ensure that we have a fair comparison, I set up two environments – one with Windows 10 Pro 21H2 and one with Fedora Workstation 35, both having up-to-date drivers and support software such as MSI Afterburner or MangoHUD for monitoring, Steam or Lutris for video game management and OBS Studio for footage recording. Adding to that, the benchmarks were ensured to be both representatives of a common gameplay scenario and variable enough to address resolution scaling and HD textures.

Cover art for “Far Cry 5”, Ubisoft, fair use, via Wikimedia Commons

Before we get into some actual performance testing and comparison results, I would like to go into detail about the video game that is at the centre of this case study. Far Cry 5 is a first-person action-adventure video game developed by Ubisoft Montreal and Ubisoft Toronto. The player takes the role of an unnamed junior deputy sheriff who is trapped in Hope County, a fictional region based in Montana and has to fight against a doomsday cult to take back the county from the grasp of its charismatic and powerful leader. The video game has been well received for the inclusion of branching storylines, role-playing elements and side quests, and is optimized enough to be a defining showcase of what the underlying hardware and platform are capable of.

Preliminary

Framerate

The first test that was performed had a direct implication on how smooth the playing experience would be across different platforms but on the same hardware configuration.

Without HD textures

On a default Far Cry 5 installation, I followed the configuration stated above but opted out of the HD textures pack to warm up the platforms with a comparatively easier test. Following are the results.

  1. On average, the video game had around a whopping 59.25% more framerate on Fedora Workstation 35 than on Windows 10 Pro 21H2.
  2. To ensure an overall consistent performance, both the minimum and maximum framerates were also noted to monitor dips and rises.
  3. The minimum framerates on Fedora Workstation 35 were ahead by a big 49.10% margin as compared to those on Windows 10 Pro 21H2.
  4. The maximum framerates on Fedora Workstation 35 were ahead by a big 62.52% margin as compared to those on Windows 10 Pro 21H2.
  5. The X11 display server had roughly 0.52% more minimum framerate as compared to Wayland, which can be taken as a margin of error.
  6. The Wayland display server had roughly 3.87% more maximum framerate as compared to X11, which can be taken as a margin of error.

With HD textures

On a default Far Cry 5 installation, I followed the configuration stated above, but this time I enabled the HD textures pack to stress the platforms with a comparatively harder test. Following are the results.

  1. On average, the video game had around a whopping 65.63% more framerate on Fedora Workstation 35 than on Windows 10 Pro 21H2.
  2. To ensure an overall consistent performance, both the minimum and maximum framerates were also noted to monitor dips and rises.
  3. The minimum framerates on Fedora Workstation 35 were ahead by a big 59.11% margin as compared to those on Windows 10 Pro 21H2.
  4. The maximum framerates on Fedora Workstation 35 were ahead by a big 64.21% margin as compared to those on Windows 10 Pro 21H2.
  5. The X11 display server had roughly 9.77% more minimum framerate as compared to Wayland, which is big enough to be considered.
  6. The Wayland display server had roughly 1.12% more maximum framerate as compared to X11, which can be taken as a margin of error.

Video memory usage

The second test that was performed had less to do with the playing experience and more with the efficiency of graphical resource usage. Following are the results.

Without HD textures

On a default Far Cry 5 installation, I followed the configuration stated above but opted out of the HD textures pack to use comparatively lesser video memory across the platforms. Following are the results.

  1. On average, Fedora Workstation 35 uses around 31.94% lesser video memory as compared to Windows 10 Pro 21H2.
  2. The Wayland display server uses roughly 1.78% more video memory as compared to X11, which can be taken as a margin of error.
  3. The video game usage estimated is closer to the actual readings on Fedora Workstation 35 than they are those on Windows 10 Pro 21H2.
  4. Adding this to the previous results speaks about how Fedora Workstation 35 performs better while using fewer resources.

With HD textures

On a default Far Cry 5 installation, I followed the configuration stated above but this time I enabled the HD textures pack to stress the platforms by occupying more video memory. Following are the results.

  1. On average, Fedora Workstation 35 uses around 22.79% lesser video memory as compared to Windows 10 Pro 21H2.
  2. The Wayland display server uses roughly 2.73% more video memory as compared to X11, which can be taken as a margin of error.
  3. The video game usage estimated is closer to the actual readings on Fedora Workstation 35 than they are those on Windows 10 Pro 21H2.
  4. Adding this to the previous results speaks about how Fedora Workstation 35 performs better while using fewer resources.

System memory usage

The third test that was performed had less to do with the playing experience and more with how other applications can fit in the available memory while the video game is running. Following are the results.

Without HD textures

On a default Far Cry 5 installation, I followed the configuration stated above but opted out of the HD textures pack to warm up the platforms with a comparatively easier test. Following are the results.

  1. On average, Fedora Workstation 35 uses around 38.10% lesser system memory as compared to Windows 10 Pro 21H2.
  2. The Wayland display server uses roughly 4.17% more system memory as compared to X11, which can be taken as a margin of error.
  3. Adding this to the previous results speaks about how Fedora Workstation 35 performs better while using fewer resources.
  4. Lesser memory usage by the video game leaves out extra headroom for other applications to run simultaneously with no compromises.

With HD textures

On a default Far Cry 5 installation, I followed the configuration stated above, but this time I enabled the HD textures pack to stress the platforms with a comparatively harder test. Following are the results.

  1. On average, Fedora Workstation 35 uses around 33.58% lesser system memory as compared to Windows 10 Pro 21H2.
  2. The Wayland display server uses roughly 7.28% more system memory as compared to X11, which is big enough to be considered.
  3. Adding this to the previous results speaks about how Fedora Workstation 35 performs better while using fewer resources.
  4. Lesser memory usage by the video game leaves out extra headroom for other applications to run simultaneously with no compromises.

Advanced

Without HD textures

On a default Far Cry 5 installation, I followed the previously stated configuration without the HD textures pack and ran the tests with varied resolution multipliers. Following are the results.

Minimum framerates recorded

  1. A great deal of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.1x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers do not seem to have a great effect on the framerate on Windows 10 Pro 21H2 as much as on Fedora Workstation 35.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 2.0x resolution multiplier appear to be marginally better than those on Fedora Workstation 35.

Maximum framerates recorded

  1. A small amount of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.1x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers change starts noticeably affecting performance on Windows 10 Pro 21H2 on a 1.6x scale, beyond which it falls greatly.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 1.6x resolution multiplier and beyond appear to be better than those on Fedora Workstation 35.

Average framerates recorded

  1. A minor amount of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.1x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers change starts noticeably affecting performance on Windows 10 Pro 21H2 on a 1.6x scale, beyond which it falls greatly.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 1.9x resolution multiplier and beyond appear to be better than those on Fedora Workstation 35.

With HD textures

On a default Far Cry 5 installation, I followed the previously stated configuration with the HD textures pack and ran the tests with varied resolution multipliers. Following are the results.

Minimum framerates recorded

  1. A great deal of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.5x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers do not seem to have a great effect on the framerate on Windows 10 Pro 21H2 as much as on Fedora Workstation 35.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 2.0x resolution multiplier appear to be marginally better than those on Fedora Workstation 35.

Maximum framerates recorded

  1. A great deal of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.0x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers change starts noticeably affecting performance on Windows 10 Pro 21H2 on a 1.6x scale, beyond which it falls greatly.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 1.6x resolution multiplier and beyond appear to be better than those on Fedora Workstation 35.

Average framerates recorded

  1. A minor amount of inconsistent performance is visible on Fedora Workstation 35 with both display servers in lower resolution scales.
  2. The inconsistencies seem to normalize for the resolution multipliers on and beyond the 1.1x resolution scale for Fedora Workstation 35.
  3. Resolution multipliers change starts noticeably affecting performance on Windows 10 Pro 21H2 on a 1.6x scale, beyond which it falls greatly.
  4. Although Windows 10 Pro 21H2 misses out on potential performance advantages in lower resolution multipliers, it has been consistent.
  5. Records on Windows 10 Pro 21H2 in the 1.9x resolution multiplier and beyond appear to be better than those on Fedora Workstation 35.

Inferences

If the test results and observations baffle you, please allow me to tell you that you are not the only one who feels like that. For a video game that was created to run on Windows, it is hard to imagine how it ends up performing way better on Fedora Workstation 35, all while using a much lesser amount of system resources at all times. Special attention has been given to noting down the highest highs and lowest lows of framerates to ensure that consistent performance is made available.

But wait a minute – how is it that Fedora Workstation 35 manages to make this possible? Well, while I do not have a clear idea of what exactly goes on behind the scenes, I do have a certain number of assumptions that I suspect might be the reasons attributing to such brilliant visuals, great framerates and efficient resource usage. These can potentially act as starting points for us to understand the features of Fedora Workstation 35 for compatibility layers to make use of.

  1. Effective caching of graphical elements and texture assets in the video memory allows for keeping only those data in the memory which are either actively made use of or regularly referenced. The open-source AMD drivers help Fedora Workstation 35 make efficient use of the available frame buffer.
  2. Quick and frequent cycling of data elements from the video memory helps to bring down total occupancy per application at any point in time. The memory clocks and shader clocks are left at the application’s disposal by the open-source AMD drivers, and firmware bandwidth limits are all but absent.
  3. With AMD Smart Access Memory (SAM) enabled, the CPU is no longer restricted to using only 256MiB of the video memory at a time. A combination of leading-edge kernel and up-to-date drivers makes it available on Fedora Workstation 35 and capable of harnessing the technology to its limits.
  4. Extremely low system resource usage by supporting software and background services leaves out a huge majority of them to be used by the applications which need it the most. Fedora Workstation 35 is a lightweight distribution, which does not get in your way and puts the resources on what’s important.
  5. Faster loading of data elements to and from the physical storage devices to the system memory is greatly enhanced with the use of high-capacity modern copy-on-write file systems like BTRFS and journaling file systems like EXT4, which happens to be the suggested file system for Fedora Workstation 35.

Performance improvements like these only make me want to indulge more in testing and finding out what else Fedora Workstation is capable of. Do let me know what you think in the comments section below.

Posted on Leave a comment

Contribute at the Fedora Linux 37 Test Week for Kernel 5.18 

The kernel team is working on final integration for Linux kernel 5.18. This version was just recently released, and will arrive soon in Fedora. As a result, the Fedora kernel and QA teams have organized a test week now through Sunday, June 12, 2022. Refer to the wiki page for links to the test images you’ll need to participate. Read below for details.

How does a test week work?

A test week is an event where anyone can help make sure changes in Fedora work well in an upcoming release. Fedora community members often participate, and the public is welcome at these events. If you’ve never contributed before, this is a perfect way to get started.

To contribute, you only need to be able to do the following things:

  • Download test materials, which include some large files
  • Read and follow directions step by step

The wiki page for the kernel test day has a lot of good information on what and how to test. After you’ve done some testing, you can log your results in the test day web application. If you’re available on or around the day of the event, please do some testing and report your results. We have a document which provides all the steps written.

Happy testing, and we hope to see you on test day.

Posted on Leave a comment

4 cool new projects to try in Copr for May 2022

Copr is a build system for anyone in the Fedora community. It hosts thousands of projects for various purposes and audiences. Some of them should never be installed by anyone, some are already being transitioned to the official Fedora Linux repositories, and the rest are somewhere in between. Copr gives you the opportunity to install third-party software that is not available in Fedora Linux repositories, try nightly versions of your dependencies, use patched builds of your favorite tools to support some non-standard use cases, and just experiment freely.

If you don’t know how to enable a repository or if you are concerned about whether it is safe to use Copr, please consult the project documentation.

This article takes a closer look at interesting projects that recently landed in Copr.

Python-QT6

Do you miss QT6 Python bindings for Fedora Linux? Here they are. https://copr.fedorainfracloud.org/coprs/g/kdesig/python-qt6/

KDE SIG owns this project. Therefore, it should be a quality one. And one day, it may land in Fedora Linux. 

Example of usage:

$ python Python 3.10.4 (main, Mar 25 2022, 00:00:00) [GCC 12.0.1 20220308 (Red Hat 12.0.1-0)] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import PyQt6 >>> from PyQt6.QtWidgets import QApplication, QWidget >>> import sys >>> app = QApplication(sys.argv) >>> window = QWidget() >>> window.show() >>> app.exec() 0 

More documentation can be found at 

https://www.pythonguis.com/tutorials/pyqt6-creating-your-first-window/.

Installation instructions

This package is available for Fedora Linux 36 and Rawhide. To install it, enter these commands:

sudo dnf copr enable @kdesig/python-qt6
sudo dnf install python3-qt6

Cloud-Native Utilities

A collection of cloud-native development tools.

These packages do not follow Fedora packaging guidelines, are statically built, and opt to bundle all dependencies.

Currently available packages:

  • Terraform – terraform
  • Packer – packer
  • Helm – helm
  • Tekton CLI – tektoncd-cli tektoncd-cli-doc
  • Knative CLI – knative-client knative-client-doc
  • Buildpack CLI – pack

All build recipes can be viewed in dist-git or from Pagure: https://pagure.io/mroche/cloud-utilities

Installation instructions

These packages are available for Fedora 36 Linux and Rawhide. To install them, enter this command:

sudo dnf copr enable mroche/cloud-native-utilities

DNF 5

You may be aware the DNF team is working on DNF5. There is a change proposal for Fedora Linux 38. The benefit is that every package management software — including PackageKit, and DNFDragora — should use a common libdnf library. If you have an application that handles RPM packages, you should definitely check out this project.

https://copr.fedorainfracloud.org/coprs/rpmsoftwaremanagement/dnf5-unstable/

Another similar project from the DNF team is 

https://copr.fedorainfracloud.org/coprs/jmracek/dnf5-alternatives/.

Installation instructions

These packages are available for Fedora Linux 35, 36 and Rawhide. To install them, enter these commands:

sudo dnf copr enable  rpmsoftwaremanagement/dnf5-unstable
sudo dnf install dnf5
sudo dnf copr enable jmracek/dnf5-alternatives
sudo dnf install microdnf-deprecated 

Hare

Hare is a systems programming language designed to be simple, stable and robust. Hare uses a static type system, manual memory management, and a minimal runtime. It is well suited to writing operating systems, system tools, compilers, networking software, and other low-level, high-performance tasks. A detailed overview can be found in these slides.

My summary is: Hare is simpler than C. It can be easy. But if you insist on shooting in your legs, Hare will allow you to do it.

Copr project.

Installation Instructions

These packages are available for Fedora Linux 35, 36 and Rawhide. They are also available for OpenSUSE Leap and Tumbleweed. To install them, enter these commands:

sudo dnf copr enable sentry/qbe
sudo dnf copr enable sentry/hare
sudo dnf install hare harec qbe 
Posted on Leave a comment

What’s new in Fedora Workstation 36

The latest release of Fedora Workstation 36 continues the Fedora Project’s ongoing commitment to delivering the latest innovations in the open source world. This article describes some of the notable user-facing changes that appear in this version.

GNOME 42

Fedora Workstation 36 includes the latest version of the GNOME desktop environment. GNOME 42 includes many improvements and new features. Just some of the improvements include:

  • Significantly improved input handling, resulting in lower input latency and improved responsiveness when the system is under load. This is particularly beneficial for games and graphics applications.
  • The Wayland session is now the default for those who use Nvidia’s proprietary graphics driver.
  • A universal dark mode is now available.
  • A new interface has been added for taking screenshots and screen video recordings.

In addition, many of the core apps have been ported to GTK 4, and the shell features a number of subtle refinements.

Refreshed look and feel

GNOME 42 as featured in Fedora Workstation 36

GNOME Shell features a refreshed look and feel, with rounder and more clearly separated elements throughout. All the symbolic icons have been updated and the top bar is no longer rounded.

Universal dark mode option

In Settings > Appearance, you can now choose a dark mode option which applies a dark theme to all supported applications. In addition, the pre-installed wallpapers now include dark mode variants. Dark themes can help reduce eye-strain when there is low ambient light, can help conserve battery life on devices with OLED displays, and can reduce the risk of burn-in on OLED displays. Plus, it looks cool!

New screenshot interface

Taking screenshots and screen video recordings is now easier than ever

Previously, pressing the Print Screen key simply took a screenshot of the entire screen and saved it to the Pictures folder. If you wanted to customize your screenshots, you had to remember a keyboard shortcut, or manually open the Screenshots app and use that to take the screenshot you wanted. This was inconvenient.

Now, pressing Print Screen presents you with an all-new user interface that allows you to take a screenshot of either your entire screen, just one window, or a rectangular selection. You can also choose whether to hide or show the mouse pointer, and you can also now take a screen video recording from within the new interface.

Core applications

Apps made in GTK 4 + libadwaita feature a distinct visual style

GNOME’s core applications have seen a number of improvements. A number of them have been ported to GTK 4 and use libadwaita, a new widget library that implements GNOME’s Human Interface Guidelines.

  • Files now includes the ability to sort files by creation date, and includes some visual refinements, such as a tweaked headerbar design and file renaming interface.
  • The Software app now includes a more informative update interface, and more prominently features GNOME Circle apps.
  • The Settings app now has a more visually appealing interface matching the visual tweaks present throughout GNOME Shell.
  • Text Editor replaces Gedit by default. Text Editor is an all-new app built in GTK 4 and libadwaita. You can always reinstall Gedit by searching for it in the Software app.

Wayland support on Nvidia’s proprietary graphics driver

In previous versions, Fedora Workstation defaulted to the X display server when using Nvidia’s proprietary graphics driver – now, Fedora Workstation 36 uses the Wayland session by default when using Nvidia’s proprietary graphics driver.

If you experience issues with the Wayland session, you can always switch back to the Xorg session by clicking the gear icon at the bottom-right corner of the login screen and choosing “GNOME on Xorg”.

Under-the-hood changes throughout Fedora Linux 36

  • When installing or upgrading packages with DNF or PackageKit, weak dependencies that have been manually removed will no longer be reinstalled. That is to say: if foo is installed and it has bar as a weak dependency, and bar is then removed, bar will not be reinstalled when foo is updated.
  • The Noto fonts are now used by default for many languages. This provides greater coverage for different character sets. For users who write in the Malayalam script, the new Meera and RIT Rachana fonts are now the default.
  • systemd messages now include unit names by default rather than just the description, making troubleshooting easier.
systemd messages shows unit names by default

Upgrade now!

You can upgrade your system through GNOME Software, via dnf system-upgrade in the terminal, or download the live ISO image from the official website.

Also check out…

There are always cool things happening in the Fedora Project!

Posted on Leave a comment

From ifcfg to keyfiles: modernizing NetworkManager configuration in Fedora Linux 36

One of the changes in Fedora Linux 36 is that new installations will no longer support the ifcfg files to configure networking. What are those and what replaces them?

A bit of history

In the good old days, connecting a Linux box to a network was easy. For each of the interface cards connected to a network, the system administrator would drop a configuration file into the /etc directory. That configuration file would describe the addressing configuration for a particular network. On Fedora Linux, the configuration file would actually be a shell script snippet like this:

$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
DEVICE=eth0
BOOTPROTO=dhcp

A shell script executed on startup would read the file and apply the configuration. Simple.

Towards the end of 2004, however, a change was in the air. Quite literally — the Wi-Fi has become ubiquitous. The portable computers of the day could rapidly connect to new networks and the USB bus allowed even the wired network adapters to come and go while the system was up and running. The network configuration became more dynamic than ever before, rendering the existing network configuration tooling impractical. To the rescue came NetworkManager. On a Fedora Linux system, NetworkManager uses configuration like this:

$ cat /etc/sysconfig/network-scripts/ifcfg-eth0
TYPE=Ethernet
DEVICE=eth0
BOOTPROTO=dhcp

Looks familiar? It should. From the beginning, NetworkManager was intended to work with the existing configuration formats. In fact, it ended up with plugins which would seamlessly convert between NetworkManager’s internal configuration model and the distribution’s native format. On Fedora, it would be the aforementioned ifcfg files.

Let’s take a closer look at them.

Ifcfg files

The legacy network service, now part of the network-scripts package, originally defined the ifcfg file format. Along with the package comes a file called sysconfig.txt that, quite helpfully, documents the format.

As NetworkManager gained traction it often found itself in need of expressing a configuration that was not supported by the old fashioned network service. Given the nature of configuring things with shell scripts, adding new settings is no big deal. The unknown ones are generally just silently ignored. The NetworkManager’s idea of what ifcfg files should look like is described in the nm-settings-ifcfg-rh(5) manual.

In general, NetworkManager tries hard to write ifcfg files that work well with the legacy network service. Nevertheless, sometimes it is just not possible. These days, the number of network connection types that NetworkManager supports vastly outnumber what the legacy network service can configure. . A new format is now used to express what the legacy format can not. This includes VPN connections, broadband modems and more.

Keyfiles

The new format closely resembled the NetworkManager’s native configuration model:

$ cat /etc/NetworkManager/system-connections/VPN.ovpn
[connection]
id=My VPN
uuid=c85a7cdb-973b-491f-998d-b09a590af10e
type=vpn [vpn]
ca=/etc/pki/tls/certs/vpn-ca.pem
connection-type=password
remote=vpn.example.com
username=lkundrak
service-type=org.freedesktop.NetworkManager.openvpn [ipv6]
method=auto
never-default=true

The actual format should be instantly familiar to everyone familiar with Linux systems. It’s the “ini file” or “keyfile” — a bunch of plain text key-value pairs, much like the ifcfg files use, grouped into sections. The nm-settings-ifcfg-keyfile(5) manual documents the format thoroughly.

The main advantage of using this format is that it closely resembles NetworkManager’s idea of how to express network configuration, used both internally and on the D-Bus API. It’s easier to extend without taking into consideration the quirks of the mechanism that was designed in without the benefit of foresight back when the world was young. This means less code, less surprises and less bugs.

In fact there’s nothing the keyfile format can’t express that ifcfg files can. It can express the simple wired connections just as well as the VPNs or modems.

Migrating to keyfiles

The legacy network service served us well for many years, but its days are now long over. Fedora Linux dropped it many releases ago and without it there is seemingly little reason to use the ifcfg files. That is, for new configurations. While Fedora Linux still supports the ifcfg files, it has defaulted to writing keyfiles for quite some time.

Starting with Fedora Linux 36, the ifcfg support will no longer be present in new installations. If you’re still using ifcfg files, do not worry — the existing systems will keep it on upgrades. Nevertheless, you can still decide to uninstall it and carry your configuration over to keyfiles. Keep on reading to learn how.

If you’re like me, you installed your system years ago and you have a mixture of keyfiles and ifcfg files. Here’s how can you check:

$ nmcli -f TYPE,FILENAME,NAME conn
TYPE FILENAME NAME
ethernet /etc/sysconfig/network-scripts/ifcfg-eth0 eth0
wifi /etc/sysconfig/network-scripts/ifcfg-Guest Guest
wifi /etc/NetworkManager/system-connections/Base48 Base48
vpn /etc/NetworkManager/system-connections/VPN.ovpn My VPN

This example shows a VPN connection that must have always used a keyfile and a Wi-Fi connection presumably created after Fedora Linux switched to writing keyfiles by default. There’s also an Ethernet connection and Wi-Fi one from back in the day that use the ifcfg plugin. Let’s see how we can convert those to keyfiles.

The NetworkManager’s command line utility, nmcli(1), acquired a new connection migrate command, that can change the configuration backend used by a connection profile.

It’s a good idea to make a backup of /etc/sysconfig/network-scripts/ifcfg-* files, in case anything goes wrong. Once you have the backup you can try migrating a single connection to a different configuration backend (keyfile by default):

$ nmcli connection migrate eth0
Connection 'eth0' (336aba93-1cd7-4cf4-8e90-e2009db3d4d0) successfully migrated.

Did it work?

$ nmcli -f TYPE,FILENAME,NAME conn
TYPE FILENAME NAME
ethernet /etc/NetworkManager/system-connections/eth0.nmc eth0
wifi /etc/sysconfig/network-scripts/ifcfg-Guest Guest
wifi /etc/NetworkManager/system-connections/Base48 Base48
vpn /etc/NetworkManager/system-connections/VPN.ovpn My VPN

Cool. Can I migrate it back, for no good reason?

$ nmcli conn migrate --plugin ifcfg-rh eth0
Connection 'eth0' (336aba93-1cd7-4cf4-8e90-e2009db3d4d0) successfully migrated.

Excellent. Without specifying more options, the “connection migrate” command ensures all connections use the keyfile backend:

$ nmcli conn migrate
Connection '336aba93-1cd7-4cf4-8e90-e2009db3d4d0' (eth0) successfully migrated.
Connection '3802a9bc-6ca5-4a17-9d0b-346f7212f2d3' (Red Hat Guest) successfully migrated.
Connection 'a082d5a0-5e29-4c67-8b6b-09af1b8d55a0' (Base48) successfully migrated.
Connection 'c85a7cdb-973b-491f-998d-b09a590af10e' (Oh My VPN) successfully migrated.

And that’s all. Now that your system has no ifcfg files, the configuration backend that supports them is of no use and you can remove it:

# dnf remove NetworkManager-initscripts-ifcfg-rh

Your system now works the same as it did before, but you can rejoice, for it is now modern.

Posted on Leave a comment

Updating Edge Devices with OSTree and Pulp

Connecting industrial machinery to the internet has given birth to infinite opportunities that range from performance improvements and predictive maintenance to data modelling that can lead to novel solutions and use cases. The possibilities are endless. Connecting machinery on such a scale can test the limits of cloud connectivity, depending on your location and network limitations.

An edge device is any piece of hardware that sits at the boundary between two networks. When initial computation happens on servers at the edge, it speeds up user’s interactions with the cloud. Therefore, adding edge devices provides opportunities to optimize performance, shorten the journey, and lighten the load on your cloud connection.

As amazing as it sounds, managing all of this functionality demands continuous attention from administrators. Having a reliable solution to distribute, deploy, and update systems for edge devices from the outset will help you spend time on things that matter.

In this article, we look at how OSTree is well-positioned for upgrading and updating edge devices with versioned updates of Linux-based operating systems. Furthermore, we’ll explore how Pulp facilitates managing and preparing updates of the OSTree content, as well as making it available to edge devices. Together, they provide a powerful free and open-source solution for administering edge devices.

How does OSTree help manage Edge devices?

If you need to deploy hundreds of operating systems to edge devices, safe in the knowledge that you can easily manage future updates and maintenance, an OSTree’s immutable and image-based operating system is ready for the task.

OSTree functions like git, but for operating system binaries. It has git-like content-addressed repositories. The ability to commit and branch entire root filesystem trees resembles the way you submit changes in git. With OSTree, you build an operating system with pre-installed packages, known as an operating system image. After you build the operating system image, it is possible to track it, sign it, test it, and deploy it. These images function as immutable file system trees. When the time comes to change or update, you simply build a new image and deploy it. By atomically switching between different versions of images, you are completely replacing filesystem trees.

OSTree also has a simple CLI that you can use for managing simple workflows, for example, for switching between different versions of images/filesystem trees.

Where do Fedora-IoT Images feature?

As a standalone tool, the base OSTree CLI is not the most feature-rich utility for managing repository content. To make life easier, in the following demo, we will use rpm-ostree. rpm-ostree is a hybrid image/package system that combines the standard OSTree technology as a base image format and accepts RPM on both the client and server-side.

rpm-ostree integrates with Fedora IoT. In comparison to other ecosystems, instead of installing packages via DNF, you install packages with rpm-ostree. After rebooting all changes are applied to a new version of the image.

You can also upgrade or install a new Fedora IoT image with the rpm-ostree utility.

Where and how does Pulp come into this?

Pulp is a platform that handles content management workflows. Using Pulp, you can sync packages from remote repositories such as an RPM server, PyPI, Docker Hub, Ansible Galaxy, and many more. You can host and modify synced packages in repositories inside the Pulp server. You can publish repositories that contain packages available for deployment to production environments.

In our scenario, Pulp provides a platform for storing particular versions of OSTree content, promoting approved content through the content management lifecycle, for example from dev to test, and from test to prod. Pulp also provides a method for publishing content that is consumed by edge devices. Using Pulp, you can pull the latest packages, test, and publish only when safe to do so. Pulp ensures the safety, security, and repeatability of your content supply chain.

The following diagram provides a simplified overview of Pulp. On the left are shown different content types that are mirrored into Pulp from remote sources. These repositories are then served, for instance, to different CI/CD or production environments.

A simplified overview of Pulp. The content is mirrored from remote repositories and made available to different types of environments.

Pulp creates a new repository version automatically when updating or removing packages in a repository. You can distribute each repository version independently.

Pulp has a plugin-based architecture, which means that you must add a plugin for each content type you want to use. For managing OSTree content, you need the OSTree plugin. You can then mirror content from a remote repository, import content from a local tarball, and modify content within a Pulp repository while preserving the integrity of the original content. You can move commits and refs from one repository to another or delete them. Pulp ensures that you are safe to experiment while your production environment remains pinned to a particular version.

Putting it all together

In this section, let’s look at how to build an image with an OSTree commit.

Building a Customized Fedora-IoT Image

We start by booting a new virtual machine (VM) that will have an installed Fedora-IoT OS. For the purposes of this example, it is best to have the same version of the OS installed as the running edge devices have.

All commands in this section are executed on the main admin VM (Fedora IoT 35 OS). On this admin VM, we will build the images that we will then distribute to the edge devices.

Before you begin:

  • First, ensure that the VM is accessible via SSH. To test, enter the following command from within the target OS:
$ systemctl is-active sshd
  • Next, ensure that the following tools for composing operating system images are installed: 
$ sudo rpm-ostree install osbuild-composer composer-cli
$ sudo systemctl enable --now osbuild-composer.socket
  • Now, apply the installed packages by rebooting the system.

In this example a nano editor package is installed on all edge devices. We need to build an image containing a commit with the package.

Create a blueprint file that describes what changes you want to make to the image as shown here:

$ cat install-nano.toml name = "nano-commit"
description = "Installing nano"
version = "0.0.1" [[packages]]
name = "nano"
version = "*"

Push this blueprint to the os build composer utility, which is a tool for composing operating system images. composer-cli communicates with osbuild composer through the CLI:

$ composer-cli blueprints push install-nano.toml

Build a new image:

$ composer-cli compose start-ostree nano-commit fedora-iot-commit --ref fedora/stable/x86_64/iot

The composer will use resources available in your current OS (such as a default operating system version).

Regularly check the status of the build:

$ composer-cli compose status

When the build finishes, download the image:

$ composer-cli compose image ${IMAGE_UUID}

The downloaded image is basically an OSTree repository packed into a tarball. When you extract the archived content, you will notice that one ref is referencing the checksum of a commit. You can find it inside the refs/heads/ directory.

Publishing the Customized Image with Pulp

All commands shown in this section are executed on the main admin VM (Fedora IoT 35 OS).

Before you begin:

  • Ensure that you have installed Pulp and the Pulp CLI for managing OSTree repositories:
$ python3 -m venv venv && source venv/bin/activate
$ pip install pulp-cli-ostree
  • Then configure the reference to the Pulp server:
$ pulp config create && pulp status

Now configure a proxy server or SSH port forwarding to enable network communication between the VM and Pulp. Ensure that you can ping the Pulp server from the VM.


First, create a new OSTree repository:

$ pulp ostree repository create --name fedora-iot

The following command will import the tarball created in the previous section into Pulp:

$ pulp ostree repository import-commits --name fedora-iot --file ${IMAGE_TARBALL_C1} --repository_name repo

Publish the parsed commit as a remote OSTree repository hosted by Pulp:

$ pulp ostree distribution create --name fedora-iot --base-path fedora-iot --repository fedora-iot

Try to fetch the commit checksum from the ref:

$ curl http://${PULP_BASE_ADDR}/pulp/content/pulp-fedora-iot/refs/heads/fedora/stable/x86_64/iot

Distributing the Customized Image to an Edge Device

The Edge device can be another VM or a real device running Fedora IoT.

All commands shown in this section are executed on an Edge device (Fedora IoT 35 OS).

Before you begin:

  • Configure a proxy server or SSH port forwarding to enable network communication between an Edge device and Pulp. Ensure that you can ping the Pulp server from the Edge device. 
  • Ensure that the Edge device is accessible with SSH:
$ systemctl is-active sshd

The nano package should NOT come pre-installed with the official bare Fedora IoT 35 image. Verify that by attempting to run nano inside your terminal.

In Fedora IoT, updates are retrieved from the URL defined in /etc/ostree/remotes.d/fedora-iot.conf. This file can be modified manually or by adding a new remote repository. Learn more at Adding and Removing Remote Repositories.

You can automate the upgrade procedure with an upgrade policy that will be configured at the beginning of deployment. This is done by writing a kickstart file that will boot up an edge device into a headless state. However, for demonstrative purposes, let’s act like a villain and update the aforementioned configuration file manually to have the following content:

[remote "fedora-iot"]
url=http://${PULP_BASE_ADDR}/pulp/content/pulp-fedora-iot/refs/heads/fedora/stable/x86_64/iot
gpg-verify=false
ref=fedora/stable/x86_64/iot

Do not forget to replace the variable ${PULP_BASE_ADDR} with a valid base path to the pulp server.

The following command shows you that some packages are going to be installed:

$ rpm-ostree upgrade

Reboot the edge device:

$ systemctl reboot

…rebooting…

Log in to the edge VM via ssh, and check the presence of the nano package that comes from Pulp:

$ nano

Done! You have successfully distributed a customized Fedora IoT image via Pulp!

In case of any questions, do not hesitate to reach out to us at https://pulpproject.org/help.

Posted on Leave a comment

3-2-1 Backup plan with Fedora ARM server

Fedora Server Edition works on Single Board Computers (SBC) like Raspberry Pi. This article is aimed at data backup and restoration of personal data for users who want to take advantage of solid server systems and built-in tools like Cockpit. It describes 3 levels of backup.

Pre-requisites

To use this guide, all you need is a working Fedora Linux workstation and the following items.

  • You should read, understand, and practice the requirements as documented in the Fedora Docs for server installation and administration
  • An SBC (Single Board Computer), tested for Fedora Linux. Check hardware status here.
  • Fedora ARM server raw image & ARM image installer
  • A choice of microSD Card (64 GB / Class 10) and SSD device
  • Ethernet cable / DHCP reserved IP or static IP
  • A Linux client workstation with ssh keys prepared
  • Make a choice of cloud storage services
  • Have an additional Linux workstation available

With this setup, I opted for Raspberry Pi 3B+/4B+ (one for hot-swap) because of the price and availability at the time of writing this article. While the Pi server is remotely connected using Cockpit, you can position the Pi near the router for a neat set-up.

Harden server security

After following through with server installation and administration on the SBC, it is a good practice to harden the server security with firewalld.

You must configure the firewall as soon as the server is online before connecting the storage device to the server. Firewalld is a zone-based firewall. It creates one pre-defined zone ‘FedoraServer’ after following through with the installation and administration guide in the Fedora Docs.

Rich rules in firewalld

Rich rules are used to block or allow a particular IP address or address range. The following rule accepts SSH connections only from the host with the registered IP (of client workstation) and drops other connections. Run the commands in Cockpit Terminal or terminal in client workstation connect to the server via ssh.

firewall-cmd --add-rich-rule='rule family=ipv4 source address=<registered_ip_address>/24 service name=ssh log prefix="SSH Logs" level="notice" accept'

Reject ping requests from all hosts

Use this command to set the icmp reject and disallow ping requests

firewall-cmd --add-rich-rule='rule protocol value=icmp reject'

To carry out additional firewall controls, such as managing ports and zones, please refer to the link below. Please be aware that misconfiguring the firewall may make it vulnerable to security breaches.

Managing firewall in Cockpit
firewalld rules

Configure storage for file server

The next step is to connect a storage device to the SBC and partition a newly attached storage device using Cockpit. With Cockpit’s graphical server management interface, managing a home lab (whether a single server or several servers) is much simpler than before. Fedora Linux server offers Cockpit as standard.

In this setup, an SSD device, powered by the USB port of the SBC, is placed in service without the need for an additional power supply.

  • Connect the storage device to a USB port of the SBC
  • After Cockpit is running (as set up in the pre-requisites), visit ip-address-of-machine:9090 in the web browser of your client workstation
  • After logging into Cockpit, click ‘Turn on administrative access’ at the top of the Cockpit page
  • Click the “Storage” on the left pane
  • Select the device under “Drives” section to format and partition a blank storage device
Cockpit Storage management
  • On the screen of the selected storage device create a new partition table or format and create new partitions. When prompted to initialize disk, in the “Partitioning” type, select GPT partition
  • For a file system type from the drop-down list (XFS and ext4), choose ext4. This is suitable for an SBC with limited I/O capability (like USB 2.0 port) and limited bandwidth (less than 200MB/s)
Create a partition in Cockpit
  • To create a single partition taking up all the storage space on the device, specify its mount point, such as “/media” and click “Ok”
  • Click “Create partition”, which creates a new partition mounted at “/media”.

Create backups and restore from backups

Backups are rarely one-size-fits-all. There are a few choices to make such as where the data is backed up, the steps you take to backup data, identify any automation, and determine how to restore backed-up data.

Backup workflow – version 1.0

Backup 1. rsync from client to file server (Raspberry Pi)

The command used for this transfer was:

rsync -azP ~/source syncuser@host1:/destination
Options:
-a, --archive
-z, --compress
-P, --progress

To run rsync with additional options, set the following flags:

Update destination files in-place

--inplace

Append data onto shorter files

--append

Source-side deduplication combined with compression is the most effective way to reduce the size of data to be backed up before it goes to backup storage.

I run this manually at the end of the day. Automation scripts are advantageous once I settled in with the cloud backup workflow.

For details on rsync, please visit the Fedora magazine article here.

Backup 2. rsync from file server to primary cloud storage

Factors to consider when selecting cloud storage are;

  • Cost: Upload, storage, and download fee
  • rsync, sftp supported
  • Data redundancy (RAID 10 or data center redundancy plan in place)
  • Snapshots

One of the cloud storage fitting these criteria is Hetzner’s hosted Nextcloud – Storage Box. You are not tied to a supplier and are free to switch without an exit penalty.

Generate SSH keys and create authorized key files in the file server

Use ssh-keygen to generate a new pair of SSH keys for the file server and cloud storage.

ssh-keygen Generating public/private rsa key pair.
Enter file in which to save the key . . . 

Insert the required public SSH keys into a new local authorized_keys file.

cat .ssh/id_rsa.pub >> storagebox_authorized_keys

Transfer keys to cloud storage

The next step is to upload the generated authorized_keys file to the Storage Box. To do this, create the directory .ssh with permission 700 and create the file authorized_keys with the public SSH keys and permission 600. Run the following command.

echo -e "mkdir .ssh \n chmod 700 .ssh \n put storagebox_authorized_keys .ssh/authorized_keys \n chmod 600 .ssh/authorized_keys" | sftp <username>@<username>.your-storagebox.de

Use rsync over ssh

Use rsync to synchronize the current state of your file directories to Storage Box.

rsync --progress -e 'ssh -p23' --recursive <local_directory> <username>@<username>.your-storagebox.de:<target_directory>

This process is called a push operation because it “pushes” a directory from the local system to a remote system.

Restore a directory from cloud storage

To restore a directory from the Storage Box, swap the directories:

rsync --progress -e 'ssh -p23' --recursive <username>@<username>.your-storagebox.de:<remote_directory> <local_directory>

Backup 3. Client backup to secondary cloud storage

Deja Dup is in the Fedora software repo, making it a quick backup solution for Fedora Workstation. It handles the GPG encryption, scheduling, and file inclusion (which directories to back up).

Backing up to the secondary cloud
Restoring files from cloud storage

Archive personal data

Not every data needs a 3-2-1 backup strategy. That is personal data share. I repurposed a hand-me-down laptop with a 1TB HDD as an archive of personal data (family photos).

Go to “Sharing” in settings (in my case, the GNOME file manager) and toggle the slider to enable sharing.

Turn on “file sharing”, “Networks” and “Required password”, which allows you to share your public folders with other workstations on your local network using WebDAV.

Prepare fallback options

Untested backups are no better than no backups at all. I take the ‘hot swap’ approach in a home lab environment where disruptions like frequent power outages or liquid damages do happen. However, my recommendations are far from disaster recovery plans or automatic failover in corporate IT.

  • Dry run restoration of files on a regular basis
  • Backup ssh/GPG keys onto an external storage device
  • Copy a raw image of the Fedora ARM server onto an SD card
  • Keep snapshots of full backups at primary cloud storage
  • Automate backup process to minimize human error or oversight

Track activity and troubleshoot with Cockpit

As your project grows, so does the number of servers you manage. Activity and alert tracking in Cockpit ease your administrative burden. You can achieve this in three ways using Cockpit’s graphical interface.

SELinux menu

How to diagnose network issues, find logs and troubleshoot in Cockpit

  • Go to SELinux to check logs
  • Check “solution details”
  • Select “Apply this solution” when necessary
  • View automation script and run it if necessary
SELinux logs

Network or storage logs

Server logs track detailed metrics that correlate CPU load, memory usage, network activity, and storage performance with the system’s journal. Logs are organized under the network or storage dashboard.

Storage logs in Cockpit

Software updates

Cockpit helps security updates on preset time and frequency. You can run all updates when you need them.

Software updates

Congratulations on setting up a file/backup server with the Fedora ARM server edition.

Posted on Leave a comment

Contribute at the Fedora Kernel 5.17, CoreOS, Cloud, IoT, and Audio test days

Fedora Linux test days are events where anyone can help make sure changes in Fedora work well in an upcoming release. Fedora community members often participate, and the public is welcome at these events. If you’ve never contributed to Fedora before, this is a perfect way to get started.

There are six upcoming test events in the next two weeks.

  • Sunday April 03 through April 10, is to test the Kernel 5.17 changes in Fedora.
  • Monday April 04 through April 11, this test week is focusing on testing Fedora CoreOS.
  • Wednesday April 06 , is to test the Fedora IoT Edition.
  • Friday April 08, is to test Fedora 36 Cloud Base Images.
  • Wednesday April 13, is to test Audio.
  • Thursday April 14, is to test Upgrade Path from Fedora 34 and 35 to Fedora 36.

Come and test with us to make the upcoming Fedora 36 even better. Read more below on how to do it.

Kernel test week

The kernel team is working on the final integration for kernel 5.17. This version was just recently released and will arrive soon in Fedora.

The Fedora kernel and QA teams have organized a test week for Sunday April 03 through April 10. Refer to the wiki page for links to the test images you’ll need to participate. This document clearly outlines the steps.

Fedora CoreOS test week

The Fedora CoreOS team released the first Fedora CoreOS next stream based on Fedora 36. They expect to promote this to the testing stream in two weeks, on the usual schedule.

The Fedora CoreOS and QA teams have organized a test week. It begins Monday, April 04 and runs through the end of the week. Refer to the wiki page for links to the test cases and materials you’ll need to participate.

Fedora IoT Edition test day

Fedora Internet of Things is a variant of Fedora focused on IoT ecosystems. Whether you work on a home assistant, industrial gateways, or data storage and analytics, Fedora IoT provides a trusted open source platform to build on. Fedora IoT produces a monthly rolling release to help you keep your ecosystem up-to-date.

The IoT and QA teams will have their test day on Wednesday, April 06. Refer to the wiki page for links and resources to test the IoT Edition.

Fedora Cloud test day

Now that the Fedora Linux 36 is coming close to the release date, the Fedora Cloud SIG would like to get the community together to find and squash some bugs.

The test day is organized for Friday April 08. This event will test Fedora Cloud Base content. See the wiki page for links to the Beta Cloud Base Images. We have qcow, AMI, and ISO images ready for testing.

Audio test day

As part of a recent proposal, Fedora replaced the PulseAudio daemon with a functionally compatible implementation based on PipeWire. This means that all existing clients using the PulseAudio client library will continue to work as before, as well as applications shipped as Flatpak. The last few releases noted significant issues in the community and hence the origin of this regression test day.

See this wiki page for information on testing that everything works as expected. This will occur on Wednesday, April 13.

Upgrade test day

As we come closer to Fedora Linux 36 release dates, it’s time to test upgrades. This release has a lot of changes and it becomes essential that we test the graphical upgrade methods as well as the command line methods.

As a part of this test day, we will test upgrading from a full updated F35 and F34 to F36 for all architectures (x86_64, ARM, aarch64) and variants (WS, cloud, server, silverblue, IoT). See this wiki page for information and details. This test day will happen on Thursday, April 14.