RPM repositories and yum provide ready made and available in default Red Hat (and derivatives)
distribution infrastructure for distribution of custom patches and configuration files. The only
problem is to learn how to use them. here this page might help. Many people are afraid
of even touching this area because it looks like building your own RPMs is very complex. It is not.
It is actually less complex that it sound.
To distribute a set of patches to all your servers you need to create a custom RPM with those
patches.
The simplest way to create your own RPM is to start with somebody else RPM and modify it for your
needs. Source ROMs are better deal.
For testing you can install an rpm to an arbitrary directory (e.g. /my/targetdir) placing the
package files under targetdir and the rpm database in the targetdir/var/rpm directory with something
like:
If you're maintaining multiple Red Hat Enterprise Linux systems (or equivalent offsets like CentOS
or Scientific Linux) your administration work with the particular hosts will gain in a routine. Because
even the best administrator might forget something it would be advantageously to have a central software
and configuration management solution. Chef and Puppet are two very mighty and popular management
tools for this application. Depending on your system landscape and needs these tools might also be
oversized though Red Hat Package Manager (RPM) can emerge as a functional alternative in this case.
It is often forgotten that RPM can be used for sharing own software and configurations as well.
If you're not managing huge system landscapes with uncontrolled growth of software and want to have
a easy-to-use solution, you might want to have a look at RPM.
I'm myself using RPM to maintain my whole Red Hat Enterprise Linux system landscape this article
will show you how easy RPM can be used to simplify system management.
Platform
The core of this scenario is a web-service which can be implemented on a dedicated host or on
a pre-existing server (e.g. web server) as well. This web-service offers RPM packages for downloading
and doesn't even need to be a RHEL or CentOS system because the server only serves the RPM files
(and ideally doesn't create them).
RPM packages are created and replicated to the web server using SSH and Rsync on dedicated RHEL
or CentOS systems depending on the distribution releases you want to maintain (e.g. RHEL 5 and 6).
A YUM repository is created of the RPM packages collection preliminary using createrepo. The YUM
repository can be used by additional servers and other clients and consists of multiple sub-directories
that are named after the supported processor architectures. After the repository had been configured
on the client it can be used for downloading and installing additional software packages. If you
only have to maintain RPM packages for one distribution release you can scale down your test environment.
In this case a YUM repository for the Red Hat Enterprise Linux releases 5 and 6 is created.
Web server file structure
The YUM repository directory (myrepo in this case) consists of multiple sub-directories containing
the software packages per supported distribution release and processor architecture. The names of
these folders are very important the name has to be the same like the value of the appropriate
YUM variable $releasever (discussed later!).
A table of popular RPM-based Linux distributions:
| $releasever |
Explanation |
| 5Server |
RHEL 5 |
| 6Server |
RHEL 6 |
| 5Workstation |
RHED 5 |
| 6Workstation |
RHED 6 |
| 5 / 5.1 / 5.2 /
|
CentOS / Scientific Linux 5 |
| 6 / 6.1 / 5.2 /
|
CentOS / Scientific Linux 6 |
| 17 / 18 /
|
Fedora 17 / 18 /
|
Example: If you want to serve software packages for the Red Hat Enterprise Linux releases 5 and
6 you'll have to create two sub-directories: 5Server and 6Server.
This implies the following directory structure:
myrepo/
/5Server
.../repodata
.../noarch
...
.../i686
...
.../x86_64
...
/6Server
(see above)
...
The appropriate main directories are created on the web server the sub-directories and further
contents are copied to the machine using SSH / Rsync later:
# mkdir -p /var/www/html/myrepo/{5,6}Server
Preparation
Before RPM packages can be created and served to other hosts using a YUM repository several development
tools need to be installed:
# yum install rpm-build createrepo rpmdevtools
I suggest to create the RPM packages on dedicated hosts or virtual machines and copy the packages
to the web server using SSH and Rsync afterwards. The web server should never be used as development
environment additionally, due to security reasons. Especially if you want to serve packages for multiple
distribution releases (RHEL5, RHEL6) you will definitely need appropriate test environments.
The web server host needs to be prepared for serving the data (if not done yet) e.g. for a EL
system:
# yum install httpd
# chkconfig httpd on
# system-config-firewall-tui
# service httpd start
Due to security reasons, a dedicated service user for creating the packages is created on the
development machines. RPM packages should never be created under the user context of root! Afterwards
the needed directory structures are created using rpmdev-setuptree (this command doesn't exist under
EL5):
# useradd su-rpmdev
# passwd su-rpmdev
# su - su-rpmdev
$ rpmdev-setuptree
$ ln -s /usr/src/redhat ~su-rpmdev/rpmbuild #symb. link under EL5
If you're using EL5 you'll find the needed structures below /usr/src/redhat the directory permissions
need to be set for the created user su-rpmdev. In the new created folder rpmbuild (respectively below
/usr/src/redhat) consists of the following sub-directories:
- BUILD folder in which source code archives are extracted and compiled.
- RPMS created RPM packages are stored here in appropriate sub-directories per supported processor
architecture (x86_64, i686, noarch,
).
- SOURCES contains source code archives that are needed for creating RPM binary packages;
e.g. software source codes or archives containing configuration files (see example below).
- SPECS contains structure files (.spec files) that define the content and structure of RPM
packages; this file is needed to create the RPM file.
- SRPMS source code RPM packages; are created on demand. Enables clients to also download
source codes and compile packages manually using yumdownloader.
SOURCES and SPECS are the most important directories most of the time the RPM farmer is moving
around there.
Example 1: configuration files
The most administrators will customize the standard configuration of a Enterprise Linux systems
to fit their needs. Some popular examples of customizable configuration files are:
- NTP
- sudo
- environment profiles (below /etc/profile.d)
- WINS and DNS configuration
As a matter of principle, creating RPM packages is a complex topic which is only raised roughly
in this article. If you want to go deeper with this topic you might want to have a look at the Fedora
wiki and documentation there are plenty useful information:
In thie case a NTP configuration shall be packaged in a RPM file and rolled out on all hosts in
the company. First of all, a specfile is created for the future RPM package:
$ cd ~su-rpmdev/rpmbuild/SPECS
$ rpmdev-newspec mycompany-ntp
Skeleton specfile (minimal) has been created to "mycompany-ntp.spec".
All rpmdev tools doesn't exist under EL5 so that you'll have to create the specfile on your own!
If you're on EL6, the previous command created a skeleton of a RPM specfile this file is edited
using an editor now:
$ vim mycompany-ntp.spec
Name: mycompany-ntp
Version:
Release: 1%{?dist}
Summary:
Group:
License:
URL:
Source0:
BuildRequires:
Requires:
%description
%prep
%setup -q
%build
%configure
make %{?_smp_mflags}
%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT
%clean
rm -rf $RPM_BUILD_ROOT
%files
%defattr(-,root,root,-)
%doc
%changelog
Beside meta information about the application, additional scripts for compiling and creating the
package are included. The particular meta variables are largely self-explanatory some explanations:
- Name name of the software package (mycompany-ntp-1.0-1)
- Version version of the software package (my-company-ntp-1.0-1)
- Release release of the software package (my-company-ntp-1.0-1)
- Summary short package description
- Group software group the package belongs to. A full listing of possible groups can be found
below: /usr/share/doc/rpm-*/GROUPS
- License assigned license (GPL, AGPL, MIT,
)
- URL URL of the project website
- Source0 relative path starting in SOURCES to the source code
- BuildRequires dependencies that are needed to compile the source code (e.g. header files
or library source codes)
- Requires dependencies that are needed to run the application (in this case: the NTP daemon)
- Conflicts defines third-party software packages and version that are incompatible with the
customized software packages (missing in template but very important)
- %description long description of the software package
- %changelog changelog, applied changes, authors, and so on
Some of the additional script or macro segments:
- %prep preparation before compiling the source code; e.g. applying patches
- %build compiling the source code
- %install creating the RPM package; copying of created binary files and directory structures
to the RPM package
To be honest it's not hard to lose track in the beginning. I suggest to have a look at finished
RPM specfiles often you'll learn some "tricks" by having a look at other's work.
There are two ways to have a look at finished RPM specfiles. Some additional repositories are
serving their specfiles over SVN or GIT. As an example, Repoforge has a public GIT mirror for this:
[click me!]
Another possibility is to include the optional source code channels of additional repositories
like EPEL and download the source code packages:
# vi /etc/yum.repos.d/epel.repo
...
[epel-source]
...
enabled=1
ESC ZZ
# yum install yum-utils
# yumdownloader --source nrpe
The RPM packages can be adapted using cpio another more comfortable way is to use Midnight Commander
to examine the package. The RPM package includes a CPIO archive named CONTENTS.cpio the specfile
is stored there:
To build a preferably "clean" package it is important to create a source code archive even if
you only want to share configuration files. It is also possible to create those files directly in
the install macro of the specfile but especially if you want to share multiple or long configuration
files you'll lose track . In this case an archive containing the NTP configuration is created:
$ mkdir ~/rpmbuild/SOURCES/mycompany-ntp-1.0
$ cd ~/rpmbuild/SOURCES/mycompany-ntp-1.0
$ vi ntp.conf
driftfile /var/lib/ntp/drift
server localserver.loc
ESC ZZ
$ cd ..
$ tar cf mycompany-ntp-1.0.tar.gz mycompany-ntp-1.0/*
Afterwards the RPM specfile is modified my version looks like this:
$ cd ../SPECS
$ cat mycompany-ntp.spec
Name: mycompany-ntp
Version: 1.0
Release: 1%{?dist}
Summary: MyCompany customized NTP configuration
Group: System Environment/Daemons
License: GPL
Source0: %name-%version.tar.gz
Requires: ntp
%description
This package includes MyCompany customized NTP configuration files.
Feel free to delete this package if received outside the MyCompany network.
%prep
%setup -q
%build
%install
rm -rf $RPM_BUILD_ROOT
install -m 0755 -d %{buildroot}%{_sysconfdir}/mycompany
install -m 0644 ntp.conf %{buildroot}%{_sysconfdir}/mycompany/ntp.conf
%clean
rm -rf $RPM_BUILD_ROOT
%files
%config(noreplace) %{_sysconfdir}/mycompany/ntp.conf
I'm sure you notices that the modified NTP configuration file isn't stored at its accurate place
(/etc/ntp.conf) it is rather saved in an alternative directory (/etc/mycompany/ntp.conf). The reason
for this is that the former configuration file (which is provided by the ntp package) can be replaced
because of the noreplace flag:
#code quote of the ntp RPM specfile
%files
%config(noreplace) %{_sysconfdir}/ntp.conf
This package stores its configuration file in an alternative directory which can't be overwritten
by other RPM packages.
You'll have to help yourself by using a "trigger trick" that saves the former configuration and
creates a symbolic link to the new configuration after the installation. After the uninstallation
of the package this step is rolled back. For implement this you'll have to add the following macros
to your RPM specfile:
%triggerin -- mycompany-ntp
if [ ! -h /etc/ntp.conf -o ! "`readlink /etc/ntp.conf`" = "/etc/mycompany/ntp.conf" ] ; then
if [ -e /etc/ntp.conf ] ; then
mv -f /etc/ntp.conf /etc/ntp.conf.orig
fi
ln -s /etc/mycompany/ntp.conf /etc/ntp.conf
fi
%triggerun -- mycompany-ntp
if [ $1 -eq 0 -a $2 -gt 0 -a -e /etc/ntp.conf.orig ] ; then
mv -f /etc/ntp.conf.orig /etc/ntp.conf
fi
%triggerpostun -- mycompany-ntp
if [ $2 -eq 0 ]; then
rm -f /etc/ntp.conf.rpmsave /etc/ntp.conf.orig
fi
if [ -e /etc/ntp.conf.rpmnew ] ; then
mv /etc/ntp.conf.rpnnew /etc/ntp.conf.orig
fi
%postun
if [ -e /etc/ntp.conf.orig -a -h /etc/ntp.conf -a ! -e "`readlink /etc/ntp.conf`" ] ; then
mv -f /etc/ntp.conf.orig /etc/ntp.conf
fi
Simplified summarization of the triggers functions:
- Installation: if the former configuration file exists ".orig" is appended to the file
name and a symbolic link to the new configuration file is created
- Deinstallation of the customized NTP configuration: if the former configuration file still
exists the file name is reset
- After uninstalling the customized NTP configuration: remaining additional or newly added NTP
configuration files are deleted
- After uninstalling: resetting the file name of the former NTP configuration file
I abdicated the URL and BuildRequires tags in my specfile because there are no websites or special
compiling dependencies for a customized NTP configuration.
Example 2: Meta packages
There are plenty of applications and configuration that are part of a senseful customized system
installation to name some practical examples: sudo configuration, GNU Screen (of course!), and
customized application profiles.
To avoid doing the application installation manually everytime, meta packages can be built to
simplify the process. These packages don't have their own files they only have dependencies to
other packages.
Because of this, RPM meta packages aren't assigned to special processor architectures (x86_64,
i686, s390,
) an additional specification "BuildArch noarch" is added in the specfile.
Another practical example: a meta package for installing NTP including the customized configuration
and telnet for checking the daemon's function:
$ cd SOURCES
$ mkdir mycompany-ntp-full-1.0
$ tar cvfz mycompany-ntp-full-1.0.tar.gz mycompany-ntp-full-1.0
$ cd ../SPECS
$ cat mycompany-ntp-full.spec
Name: mycompany-ntp-full
Version: 1.0
Release: 1%{?dist}
Summary: MyCompany customized NTP configuration and netstat utility
Group: System Environment/Daemons
License: GPL
Source0: %name-%version.tar.gz
BuildArch: noarch
Requires: ntp mycompany-ntp net-tools
%description
This package includes MyCompany customized NTP configuration files.
Feel free to delete this package if received outside the MyCompany network.
%prep
%setup -q
%build
%install
%clean
rm -rf $RPM_BUILD_ROOT
%files
Using rpm you can list the package's dependencies:
$ rpmbuild -bb mycompany-ntp-full.spec
$ rpm --query -Rp ../RPMS/noarch/mycompany-ntp-full-1.0-1.el6.noarch.rpm
ntp
mycompany-ntp
net-tools
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(CompressedFileNames) <= 3.0.4-1
Package dependencies can very depending on the distribution release. In this example between RHEL5
and 6 the packages providing the telnet command are different here:
rhel5 # rpm -qf $(which telnet)
krb5-workstation-1.6.1-70.el5_9.2
rhel6 # rpm-qf $(which telnet)
net-tools-1.60-110.el6_2.x86_64
You might want to consider this in the specfile:
Requires: ntp mycompany-ntp
%{?el5:Requires: krb5-workstation}
%{?el6:Requires: net-tools}
The first dependeny line is significant for all releases, the following lines are considered under
RHEL5 or RHEL6.
You can also define particular versions in combination with the Requires and Conflicts tags
for example, if you want to reference a myapp package which is at least version 1.1. One of the following
lines can be used:
Requires: myapp >= 1.1
Conflicts: myapp < 1.1
If you want to reference a special version of a package there are also two possibilities choose
one:
Requires: myapp = 1.1
Conflicts: myapp < 1.1 myapp > 1.1
It's a kind of philosophical question which of the two possibilites is used like the question
if the glass is half-full or half-empty. Either a package is excluded or referenced explicitely.
Further information regarding RPM dependencies can be found on the official RPM website: [click
me!]
Let's go back to the former motivation of this meta package: as an alternative you can also define
package groups in your own YUM repository. If you have already worked with the YUM commands grouplist,
groupinstall and groupremove, you might know this logical grouping of software packages. You can
find an interesting article about this in the YUM wiki: [click me!]
Package and repository creation
Okay we have a RPM specfile now, what's next? RPM packages are creating using the rpmbuild utility.
This tool has a plenty of swithces and arguments as an example you can also create source code
packages or packages for additional processor architectures.
Important parameters:
| Parameter |
Explanation |
| -ba |
Creates a binary and source code package |
| -bb |
Creates a binary package |
| -bp |
Extracting and patching (if necessary) of the source code |
| -bs |
Creates a source code package |
| target=noarch |
Creates a platform independent package |
| target=i686 |
a 32-bit package |
| target=x86_64 |
a 64-bit package |
Some examples:
# Creates a binary and source code package of myapp
$ rpmbuild -ba myapp.spec
# Creates a 32-bit binary package of myapp
$ rpmbuild -bb --target=i686 myapp.spec
Afterwards you'll find a RPM package below RPMS depending on your processor architecture (noarch,
i686 or x86_64). If you're a proud owner of a IBM System Z machine you might want to have a look
below s390 or s390z.
$ ls RPMS/*/*.rpm
RPMS/i386/mycompany-ntp-1.0-1.el6.i386.rpm
This RPM package could now be installed using YUM:
# yum localinstall --nogpgcheck mycompany-ntp-1.0-1.el6.i386.rpm
Synchronization and automation
After creating the RPM packages on the particular test machines (e.g. RHEL6 and RHEL5) the packages
need to be copied to the web server. I suggest using SSH and Rsync for a synchronization between
the test machines, the main test machine and the web server.
If you don't want to do this manually every time you can automate this using a small script:
1.Synchronization between the EL5 machine and the main test machine:
$ ln -s /usr/src/redhat /home/su-rpmdev/rpmbuild
$ vi /home/su-rpmdev/export_repo.sh
#!/bin/sh
rsync -avz --delete /home/su-rpmdev/rpmbuild/RPMS/ /opt/myrepo
createrepo --database /opt/myrepo/
rsync -avz --delete -e ssh /opt/myrepo/ su-rpmdev@MAIN:/opt/myrepo/5Server
ESC ZZ
$ chmod +x /home/su-rpmdev/export_repo.sh
$ ./home/su-rpmdev/export_repo.sh
The first rsync command copied all RPM packages of all processor architectures below /opt/myrepo
if a RPM package is deleted in the source, it is also deleted below /opt/myrepo. createrepo created
a SQLite database for the YUM repository (myrepo) below /opt/myrepo. The second rsync command copies
the local YUM repository to the main test machine (MAIN).
2.Synchronization between the main test machine (EL6) and the web server:
$ vi /home/su-rpmdev/export_repo.sh
#!/bin/sh
rsync -avz --delete /home/su-rpmdev/rpmbuild/RPMS/ /opt/myrepo/6Server
createrepo --database /opt/myrepo/6Server
rsync -avz --delete -e ssh /opt/myrepo/ su-rpmdev@WEB:/var/www/html/myrepo
ESC ZZ
$ chmod +x /home/su-rpmdev/export_repo.sh
$ ./home/su-rpmdev/export_repo.sh
The first rsync command copies all (EL6) RPM packages below /opt/myrepo/6Server. After that createrepo
creates a SQLite-Datenbank for the EL6 repository. The second rsync command copies the whole repository
(including the other EL5 repository) to the web server (WEB).
Usage and test
To use the new YUM repository on other hosts, an appropriate YUM configuration needs to be created.
In this file the repository URL and other parameters like package signing are defined. The syntax
looks a bit like good-old Windows .ini files:
# vi /etc/yum.repos.d/myrepo.repo
[myrepo]
name=mycompany packages for EL
baseurl=http://server01/$releasever
enabled=1
gppgcheck=0
ESC ZZ
You might see the variable $releasever it was mentioned in the table above. This variable is
replaced by another value depending on your distribution release in this case by 5Server or 6Server.
These directories had been filled with the appropriate RPM packages from the test machines.
After that, all available packages of the repository can be listed:
# yum --disablerepo='*' --enablerepo='myrepo' makecache
# yum --disablerepo='*' --enablerepo='myrepo' list available
...
mycompany-ntp 1.0-1 myrepo
If you have multiple repository web servers (e.g. because of big setups for processing the amount
of requests or compensate failures) you can assign a mirror list:
# vi /etc/yum.repos.d/myrepo.repo
[myrepo]
...
#baseurl=...
mirrorlist=myrepo.mirror
ESC ZZ
# vi /etc/yum.repos.d/myrepo.mirror
http://server01/$releasever
http://server02/$releasever
http://server03/$releasever
For every download YUM uses one of these servers beginning with the first one. If this server
fails or doesn't have the file YUM will select the next server.
By the way these configuration files could be shared using a RPM package, too. If this is done,
you only need to install a RPM package to access the YUM repository zugreifen. This is how access
to Fedora's Extra Packages For Enterprise Linux (EPEL) is provided: [click me!]
This package could look like this:
# vi SPECS/myrepo-release.spec
Name: myrepo-release
Version: 1.0
Release: 1%{?dist}
Summary: mycompany Packages for Enterprise Linux repository configuration
Group: System Environment/Base
BuildArch: noarch
License: GPL
URL: http://www.mycompany.com
Source0: myrepo-release-%{version}.tar.gz
BuildRoot: %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)
%description
This package contains the mycompany customized Packages for Enterprise Linux repository.
%prep
%setup -q
%build
%install
install -m 0755 -d %{buildroot}%{_sysconfdir}/yum.repos.d/
install -m 0755 myrepo.repo %{buildroot}%{_sysconfdir}/yum.repos.d/myrepo.repo
install -m 0755 myrepo.mirror %{buildroot}%{_sysconfdir}/yum.repos.d/myrepo.mirror
%clean
rm -rf $RPM_BUILD_ROOT
%files
%config(noreplace) %{_sysconfdir}/yum.repos.d/myrepo.repo
%config(noreplace) %{_sysconfdir}/yum.repos.d/myrepo.mirror
%changelog
* Sat Jun 29 2013 FirstName LastName - 1.0-1
- initial release
ESC ZZ
# tar tvfz SOURCES/myrepo-release-1.0.tar.gz
-rw-r--r-- su-rpmdev/su-rpmdev 94 2013-06-28 18:12 myrepo-release-1.0/myrepo.mirror
-rw-r--r-- su-rpmdev/su-rpmdev 189 2013-06-28 18:09 myrepo-release-1.0/myrepo.repo
# tar xvfz SOURCES/myrepo-release-1.0.tar.gz -C SOURCES/
# cat SOURCES/myrepo-release-1.0/myrepo.mirror
http://server01/$releasever
http://server02/$releasever
http://server03/$releasever
# cat SOURCES/myrepo-release-1.0/myrepo.repo
[myrepo]
name=mycompany packages for EL
#baseurl=http://server01/$releasever
mirrorlist=/etc/yum.repos.d/myrepo.mirror
enabled=1
gppgcheck=0
After the RPM specfile had been created the package can be created and distributed easily. Finally,
you only have to install the RPM package to use the YUM repository:
# rpmbuild -bb SPECS/myrepo-release.spec
# scp RPMS/noarch/myrepo-release-1.0-1.rpm root@host:/root
root@host # yum localinstall --nogpgcheck myrepo-release*.rpm
root@host # yum repolist
repoid reponame
...
myrepo mycompany packages for EL
Et voilΰ!
Perspective / Additional ideas
Of course there is a lot that can be optimized. To name some of these things:
- SSH public-key authentification while synchronizing the RPM packages between the test machines
and the web server the password of the service user has to be entered twice. This can be avoided
by implementing SSH public-key authentification.
- Theme for viewing the repository in web browsers when the repository is accessed using a
normal web browser the directory contents are displayed. Depending on your web server this might
look ugly a header including your company's logo and usage information could be used in addition.
- Version control using SVN / GIT especially if you're working in a team on your repository
it might be very handy to use a version control system like SVN or GIT.
Integration in Red Hat Network Satellite, Spacewalk or SuSE Manager if you're maintaining a
big amount of hosts you (hopefully) have the management suite of your preferred distributor. You
can include your repository to simply share your packages.
- Package signing To avoid manipulated packages, packages can be signed using a key which
has to be copied to every client. Using this mechanism, clients can verify that the downloaded
packages are authentic.
To manage a plenty of systems you don't need a "egg-laying, milk-bearing woolly sow" RPM is
a mighty tool which can also be used for software and configuration management. If you're engaged
with Red Hat Enterprise Linux, you will often into a situation where you have to automate something
quickly. And that's where RPM can also help you to reach your goal quickly.
This work is licensed under a
Creative Commons Attribution-NonCommercial-NoDerivs
3.0 Unported License
