09-09-2019, 06:19 PM
How RPM packages are made: the source RPM
<div><p>In a <a href="https://fedoramagazine.org/rpm-packages-explained/">previous post, we looked at what RPM packages are</a>. They are archives that contain files and metadata. This metadata tells RPM where to create or remove files from when an RPM is installed or uninstalled. The metadata also contains information on “dependencies”, which you will remember from the previous post, can either be “runtime” or “build time”. </p>
<p>As an example, we will look at <em>fpaste</em>. You can download the RPM using <em>dnf</em>. This will download the latest version of <em>fpaste</em> that is available in the Fedora repositories. On Fedora 30, this is currently 0.3.9.2:</p>
<pre class="wp-block-preformatted">$ dnf download fpaste ...
fpaste-0.3.9.2-2.fc30.noarch.rpm</pre>
<p>Since this is the built RPM, it contains only files needed to use <em>fpaste</em>:</p>
<pre class="wp-block-preformatted">$ rpm -qpl ./fpaste-0.3.9.2-2.fc30.noarch.rpm
/usr/bin/fpaste
/usr/share/doc/fpaste
/usr/share/doc/fpaste/README.rst
/usr/share/doc/fpaste/TODO
/usr/share/licenses/fpaste
/usr/share/licenses/fpaste/COPYING
/usr/share/man/man1/fpaste.1.gz
</pre>
<h2>Source RPMs</h2>
<p>The next link in the chain is the source RPM. All software in Fedora must be built from its source code. We do not include pre-built binaries. So, for an RPM file to be made, RPM (the tool) needs to be:</p>
<ul>
<li>given the files that have to be installed,</li>
<li>told how to generate these files, if they are to be compiled, for example,</li>
<li>told where these files must be installed,</li>
<li>what other dependencies this particular software needs to work properly.</li>
</ul>
<p>The source RPM holds all of this information. Source RPMs are similar archives to RPM, but as the name suggests, instead of holding the built binary files, they contain the source files for a piece of software. Let’s download the source RPM for <em>fpaste</em>:</p>
<pre class="wp-block-preformatted">$ dnf download fpaste --source
...
fpaste-0.3.9.2-2.fc30.src.rpm</pre>
<p>Notice how the file ends with “src.rpm”. All RPMs are built from source RPMs. You can easily check what source RPM a “binary” RPM comes from using dnf too:</p>
<pre class="wp-block-preformatted">$ dnf repoquery --qf "%{SOURCERPM}" fpaste
fpaste-0.3.9.2-2.fc30.src.rpm</pre>
<p>Also, since this is the source RPM, it does not contain built files. Instead, it contains the sources and instructions on how to build the RPM from them:</p>
<pre class="wp-block-preformatted">$ rpm -qpl ./fpaste-0.3.9.2-2.fc30.src.rpm
fpaste-0.3.9.2.tar.gz
fpaste.spec</pre>
<p>Here, the first file is simply the source code for <em>fpaste</em>. The second is the “spec” file. The spec file is the recipe that tells RPM (the tool) how to create the RPM (the archive) using the sources contained in the source RPM—all the information that RPM (the tool) needs to build RPMs (the archives) are contained in spec files. When we package maintainers add software to Fedora, most of our time is spent writing and perfecting the individual spec files. When a software package needs an update, we go back and tweak the spec file. You can see the spec files for ALL packages in Fedora at our source repository at <a href="https://src.fedoraproject.org/browse/projects/">https://src.fedoraproject.org/browse/projects/</a></p>
<p>Note that one source RPM may contain the instructions to build multiple RPMs. <em>fpaste</em> is a very simple piece of software, where one source RPM generates one “binary” RPM. Python, on the other hand is more complex. While there is only one source RPM, it generates multiple binary RPMs:</p>
<pre class="wp-block-preformatted">$ sudo dnf repoquery --qf "%{SOURCERPM}" python3
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-devel
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-libs
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-idle
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-tkinter
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm
</pre>
<p>In RPM jargon, “python3” is the “main package”, and so the spec file will be called “python3.spec”. All the other packages are “sub-packages”. You can download the source RPM for python3 and see what’s in it too. (Hint: patches are also part of the source code):</p>
<pre class="wp-block-preformatted">$ dnf download --source python3
python3-3.7.4-1.fc30.src.rpm $ rpm -qpl ./python3-3.7.4-1.fc30.src.rpm
00001-rpath.patch
00102-lib64.patch
00111-no-static-lib.patch
00155-avoid-ctypes-thunks.patch
00170-gc-assertions.patch
00178-dont-duplicate-flags-in-sysconfig.patch
00189-use-rpm-wheels.patch
00205-make-libpl-respect-lib64.patch
00251-change-user-install-location.patch
00274-fix-arch-names.patch
00316-mark-bdist_wininst-unsupported.patch
Python-3.7.4.tar.xz
check-pyc-timestamps.py
idle3.appdata.xml
idle3.desktop
python3.spec</pre>
<h2>Building an RPM from a source RPM</h2>
<p>Now that we have the source RPM, and know what’s in it, we can rebuild our RPM from it. Before we do so, though, we should set our system up to build RPMs. First, we install the required tools:</p>
<pre class="wp-block-preformatted">$ sudo dnf install fedora-packager</pre>
<p>This will install the rpmbuild tool. rpmbuild requires a default layout so that it knows where each required component of the source rpm is. Let’s see what they are:</p>
<pre class="wp-block-preformatted"># Where should the spec file go?
$ rpm -E %{_specdir}
/home/asinha/rpmbuild/SPECS # Where should the sources go?
$ rpm -E %{_sourcedir}
/home/asinha/rpmbuild/SOURCES # Where is temporary build directory?
$ rpm -E %{_builddir}
/home/asinha/rpmbuild/BUILD # Where is the buildroot?
$ rpm -E %{_buildrootdir}
/home/asinha/rpmbuild/BUILDROOT # Where will the source rpms be?
$ rpm -E %{_srcrpmdir}
/home/asinha/rpmbuild/SRPMS # Where will the built rpms be?
$ rpm -E %{_rpmdir}
/home/asinha/rpmbuild/RPMS</pre>
<p>I have all of this set up on my system already:</p>
<pre class="wp-block-preformatted">$ cd
$ tree -L 1 rpmbuild/
rpmbuild/
├── BUILD
├── BUILDROOT
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS 6 directories, 0 files
</pre>
<p>RPM provides a tool that sets it all up for you too:</p>
<pre class="wp-block-preformatted">$ rpmdev-setuptree</pre>
<p>Then we ensure that we have all the build dependencies for <em>fpaste</em> installed:</p>
<pre class="wp-block-preformatted">sudo dnf builddep fpaste-0.3.9.2-3.fc30.src.rpm</pre>
<p>For <em>fpaste</em> you only need Python and that must already be installed on your system (dnf uses Python too). The builddep command can also be given a spec file instead of an source RPM. Read more in the man page:</p>
<pre class="wp-block-preformatted">$ man dnf.plugin.builddep</pre>
<p>Now that we have all that we need, building an RPM from a source RPM is as simple as:</p>
<pre class="wp-block-preformatted">$ rpmbuild --rebuild fpaste-0.3.9.2-3.fc30.src.rpm
..
.. $ tree ~/rpmbuild/RPMS/noarch/
/home/asinha/rpmbuild/RPMS/noarch/
└── fpaste-0.3.9.2-3.fc30.noarch.rpm 0 directories, 1 file</pre>
<p>rpmbuild will install the source RPM and build your RPM from it. You can now install the RPM to use it as you do–using dnf. Of course, as said before, if you want to change anything in the RPM, you must modify the spec file—we’ll cover spec files in next post.</p>
<h2>Summary</h2>
<p>To summarise this post in two short points:</p>
<ul>
<li>the RPMs we generally install to use software are “binary” RPMs that contain built versions of the software</li>
<li>these are built from source RPMs that include the source code and the spec file that are needed to generate the binary RPMs.</li>
</ul>
<p>If you’d like to get started with building RPMs, and help the Fedora community maintain the massive amount of software we provide, you can start here: <a href="https://fedoraproject.org/wiki/Join_the_package_collection_maintainers">https://fedoraproject.org/wiki/Join_the_package_collection_maintainers</a></p>
<p>For any queries, post to the <a href="https://lists.fedoraproject.org/archives/list/[email protected]/">Fedora developers mailing list</a>—we’re always happy to help!</p>
</div>
https://www.sickgaming.net/blog/2019/08/...ource-rpm/
<div><p>In a <a href="https://fedoramagazine.org/rpm-packages-explained/">previous post, we looked at what RPM packages are</a>. They are archives that contain files and metadata. This metadata tells RPM where to create or remove files from when an RPM is installed or uninstalled. The metadata also contains information on “dependencies”, which you will remember from the previous post, can either be “runtime” or “build time”. </p>
<p>As an example, we will look at <em>fpaste</em>. You can download the RPM using <em>dnf</em>. This will download the latest version of <em>fpaste</em> that is available in the Fedora repositories. On Fedora 30, this is currently 0.3.9.2:</p>
<pre class="wp-block-preformatted">$ dnf download fpaste ...
fpaste-0.3.9.2-2.fc30.noarch.rpm</pre>
<p>Since this is the built RPM, it contains only files needed to use <em>fpaste</em>:</p>
<pre class="wp-block-preformatted">$ rpm -qpl ./fpaste-0.3.9.2-2.fc30.noarch.rpm
/usr/bin/fpaste
/usr/share/doc/fpaste
/usr/share/doc/fpaste/README.rst
/usr/share/doc/fpaste/TODO
/usr/share/licenses/fpaste
/usr/share/licenses/fpaste/COPYING
/usr/share/man/man1/fpaste.1.gz
</pre>
<h2>Source RPMs</h2>
<p>The next link in the chain is the source RPM. All software in Fedora must be built from its source code. We do not include pre-built binaries. So, for an RPM file to be made, RPM (the tool) needs to be:</p>
<ul>
<li>given the files that have to be installed,</li>
<li>told how to generate these files, if they are to be compiled, for example,</li>
<li>told where these files must be installed,</li>
<li>what other dependencies this particular software needs to work properly.</li>
</ul>
<p>The source RPM holds all of this information. Source RPMs are similar archives to RPM, but as the name suggests, instead of holding the built binary files, they contain the source files for a piece of software. Let’s download the source RPM for <em>fpaste</em>:</p>
<pre class="wp-block-preformatted">$ dnf download fpaste --source
...
fpaste-0.3.9.2-2.fc30.src.rpm</pre>
<p>Notice how the file ends with “src.rpm”. All RPMs are built from source RPMs. You can easily check what source RPM a “binary” RPM comes from using dnf too:</p>
<pre class="wp-block-preformatted">$ dnf repoquery --qf "%{SOURCERPM}" fpaste
fpaste-0.3.9.2-2.fc30.src.rpm</pre>
<p>Also, since this is the source RPM, it does not contain built files. Instead, it contains the sources and instructions on how to build the RPM from them:</p>
<pre class="wp-block-preformatted">$ rpm -qpl ./fpaste-0.3.9.2-2.fc30.src.rpm
fpaste-0.3.9.2.tar.gz
fpaste.spec</pre>
<p>Here, the first file is simply the source code for <em>fpaste</em>. The second is the “spec” file. The spec file is the recipe that tells RPM (the tool) how to create the RPM (the archive) using the sources contained in the source RPM—all the information that RPM (the tool) needs to build RPMs (the archives) are contained in spec files. When we package maintainers add software to Fedora, most of our time is spent writing and perfecting the individual spec files. When a software package needs an update, we go back and tweak the spec file. You can see the spec files for ALL packages in Fedora at our source repository at <a href="https://src.fedoraproject.org/browse/projects/">https://src.fedoraproject.org/browse/projects/</a></p>
<p>Note that one source RPM may contain the instructions to build multiple RPMs. <em>fpaste</em> is a very simple piece of software, where one source RPM generates one “binary” RPM. Python, on the other hand is more complex. While there is only one source RPM, it generates multiple binary RPMs:</p>
<pre class="wp-block-preformatted">$ sudo dnf repoquery --qf "%{SOURCERPM}" python3
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-devel
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-libs
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-idle
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm $ sudo dnf repoquery --qf "%{SOURCERPM}" python3-tkinter
python3-3.7.3-1.fc30.src.rpm
python3-3.7.4-1.fc30.src.rpm
</pre>
<p>In RPM jargon, “python3” is the “main package”, and so the spec file will be called “python3.spec”. All the other packages are “sub-packages”. You can download the source RPM for python3 and see what’s in it too. (Hint: patches are also part of the source code):</p>
<pre class="wp-block-preformatted">$ dnf download --source python3
python3-3.7.4-1.fc30.src.rpm $ rpm -qpl ./python3-3.7.4-1.fc30.src.rpm
00001-rpath.patch
00102-lib64.patch
00111-no-static-lib.patch
00155-avoid-ctypes-thunks.patch
00170-gc-assertions.patch
00178-dont-duplicate-flags-in-sysconfig.patch
00189-use-rpm-wheels.patch
00205-make-libpl-respect-lib64.patch
00251-change-user-install-location.patch
00274-fix-arch-names.patch
00316-mark-bdist_wininst-unsupported.patch
Python-3.7.4.tar.xz
check-pyc-timestamps.py
idle3.appdata.xml
idle3.desktop
python3.spec</pre>
<h2>Building an RPM from a source RPM</h2>
<p>Now that we have the source RPM, and know what’s in it, we can rebuild our RPM from it. Before we do so, though, we should set our system up to build RPMs. First, we install the required tools:</p>
<pre class="wp-block-preformatted">$ sudo dnf install fedora-packager</pre>
<p>This will install the rpmbuild tool. rpmbuild requires a default layout so that it knows where each required component of the source rpm is. Let’s see what they are:</p>
<pre class="wp-block-preformatted"># Where should the spec file go?
$ rpm -E %{_specdir}
/home/asinha/rpmbuild/SPECS # Where should the sources go?
$ rpm -E %{_sourcedir}
/home/asinha/rpmbuild/SOURCES # Where is temporary build directory?
$ rpm -E %{_builddir}
/home/asinha/rpmbuild/BUILD # Where is the buildroot?
$ rpm -E %{_buildrootdir}
/home/asinha/rpmbuild/BUILDROOT # Where will the source rpms be?
$ rpm -E %{_srcrpmdir}
/home/asinha/rpmbuild/SRPMS # Where will the built rpms be?
$ rpm -E %{_rpmdir}
/home/asinha/rpmbuild/RPMS</pre>
<p>I have all of this set up on my system already:</p>
<pre class="wp-block-preformatted">$ cd
$ tree -L 1 rpmbuild/
rpmbuild/
├── BUILD
├── BUILDROOT
├── RPMS
├── SOURCES
├── SPECS
└── SRPMS 6 directories, 0 files
</pre>
<p>RPM provides a tool that sets it all up for you too:</p>
<pre class="wp-block-preformatted">$ rpmdev-setuptree</pre>
<p>Then we ensure that we have all the build dependencies for <em>fpaste</em> installed:</p>
<pre class="wp-block-preformatted">sudo dnf builddep fpaste-0.3.9.2-3.fc30.src.rpm</pre>
<p>For <em>fpaste</em> you only need Python and that must already be installed on your system (dnf uses Python too). The builddep command can also be given a spec file instead of an source RPM. Read more in the man page:</p>
<pre class="wp-block-preformatted">$ man dnf.plugin.builddep</pre>
<p>Now that we have all that we need, building an RPM from a source RPM is as simple as:</p>
<pre class="wp-block-preformatted">$ rpmbuild --rebuild fpaste-0.3.9.2-3.fc30.src.rpm
..
.. $ tree ~/rpmbuild/RPMS/noarch/
/home/asinha/rpmbuild/RPMS/noarch/
└── fpaste-0.3.9.2-3.fc30.noarch.rpm 0 directories, 1 file</pre>
<p>rpmbuild will install the source RPM and build your RPM from it. You can now install the RPM to use it as you do–using dnf. Of course, as said before, if you want to change anything in the RPM, you must modify the spec file—we’ll cover spec files in next post.</p>
<h2>Summary</h2>
<p>To summarise this post in two short points:</p>
<ul>
<li>the RPMs we generally install to use software are “binary” RPMs that contain built versions of the software</li>
<li>these are built from source RPMs that include the source code and the spec file that are needed to generate the binary RPMs.</li>
</ul>
<p>If you’d like to get started with building RPMs, and help the Fedora community maintain the massive amount of software we provide, you can start here: <a href="https://fedoraproject.org/wiki/Join_the_package_collection_maintainers">https://fedoraproject.org/wiki/Join_the_package_collection_maintainers</a></p>
<p>For any queries, post to the <a href="https://lists.fedoraproject.org/archives/list/[email protected]/">Fedora developers mailing list</a>—we’re always happy to help!</p>
</div>
https://www.sickgaming.net/blog/2019/08/...ource-rpm/