GUEST POST: Software and configuration management made easy with RPM Posted on: September 3, 2013

Jerry Sullivan by Christian Stankowic

If you're maintaining multiple Red Hat Enterprise Linux systems (or equivalent offsets like CentOS or Scientific Linux) your administration work with the particular hosts will gain in a routine. Because even the best administrator might forget something it would be advantageously to have a central software and configuration management solution. Chef and Puppet are two very mighty and popular mangement tools for this application. Depending on your system landscape and needs these tools might also be oversized though – Red Hat Package Manager (RPM) can emerge as a functional alternative in this case.

It is often forgotten that RPM can be used for sharing own software and configurations as well. If you're not managing huge system landscapes with uncontrolled growth of software and want to have a easy-to-use solution, you might want to have a look at RPM.

I'm myself using RPM to maintain my whole Red Hat Enterprise Linux system landscape – this article will show you how easy RPM can be used to simplify system management.


Platform

The core of this scenario is a web-service which can be implemented on a dedicated host or on a pre-existing server (e.g. web server) as well. This web-service offers RPM packages for downloading and doesn't even need to be a RHEL or CentOS system because the server only serves the RPM files (and ideally doesn't create them).

RPM packages are created and replicated to the web server using SSH and Rsync on dedicated RHEL or CentOS systems depending on the distribution releases you want to maintain (e.g. RHEL 5 and 6). A YUM repository is created of the RPM packages collection preliminary using createrepo. The YUM repository can be used by additional servers and other clients and consists of multiple sub-directories that are named after the supported processor architectures. After the repository had been configured on the client it can be used for downloading and installing additional software packages. If you only have to maintain RPM packages for one distribution release you can scale down your test environment.

In this case a YUM repository for the Red Hat Enterprise Linux releases 5 and 6 is created.

Web server file structure

The YUM repository directory (myrepo in this case) consists of multiple sub-directories containing the software packages per supported distribution release and processor architecture. The names of these folders are very important – the name has to be the same like the value of the appropriate YUM variable $releasever (discussed later!).

A table of popular RPM-based Linux distributions:

$releasever Explanation
5Server RHEL 5
6Server RHEL 6
5Workstation RHED 5
6Workstation RHED 6
5 / 5.1 / 5.2 / CentOS / Scientific Linux 5
6 / 6.1 / 5.2 / CentOS / Scientific Linux 6
17 / 18 / Fedora 17 / 18 /

Example: If you want to serve software packages for the Red Hat Enterprise Linux releases 5 and 6 you'll have to create two sub-directories: 5Server and 6Server.

This implies the following directory structure:

myrepo/
       /5Server
            .../repodata
            .../noarch
                      ...
            .../i686
                    ...
            .../x86_64
                      ...
       /6Server
               (see above)
               ...

The appropriate main directories are created on the web server – the sub-directories and further contents are copied to the machine using SSH / Rsync later:

# mkdir -p /var/www/html/myrepo/{5,6}Server
Preparation

Before RPM packages can be created and served to other hosts using a YUM repository several development tools need to be installed:

# yum install rpm-build createrepo rpmdevtools

I suggest to create the RPM packages on dedicated hosts or virtual machines and copy the packages to the web server using SSH and Rsync afterwards. The web server should never be used as development environment additionally, due to security reasons. Especially if you want to serve packages for multiple distribution releases (RHEL5, RHEL6) you will definitely need appropriate test environments.

The web server host needs to be prepared for serving the data (if not done yet) – e.g. for a EL system:

# yum install httpd
# chkconfig httpd on
# system-config-firewall-tui
# service httpd start

Due to security reasons, a dedicated service user for creating the packages is created on the development machines. RPM packages should never be created under the user context of root! Afterwards the needed directory structures are created using rpmdev-setuptree (this command doesn't exist under EL5):

# useradd su-rpmdev
# passwd su-rpmdev
# su - su-rpmdev
$ rpmdev-setuptree
$ ln -s /usr/src/redhat ~su-rpmdev/rpmbuild    #symb. link under EL5

If you're using EL5 you'll find the needed structures below /usr/src/redhat – the directory permissions need to be set for the created user su-rpmdev. In the new created folder rpmbuild (respectively below /usr/src/redhat) consists of the following sub-directories:

  • BUILD – folder in which source code archives are extracted and compiled.
  • RPMS – created RPM packages are stored here in appropriate sub-directories per supported processor architecture (x86_64, i686, noarch, ).
  • SOURCES – contains source code archives that are needed for creating RPM binary packages; e.g. software source codes or archives containing configuration files (see example below).
  • SPECS – contains structure files (.spec files) that define the content and structure of RPM packages; this file is needed to create the RPM file.
  • SRPMS – source code RPM packages; are created on demand. Enables clients to also download source codes and compile packages manually using yumdownloader.

SOURCES and SPECS are the most important directories – most of the time the RPM farmer is moving around there. 😉

Example 1: configuration files

The most administrators will customize the standard configuration of a Enterprise Linux systems to fit their needs. Some popular examples of customizable configuration files are:

  • NTP
  • sudo
  • environment profiles (below /etc/profile.d)
  • WINS and DNS configuration

As a matter of principle, creating RPM packages is a complex topic which is only raised roughly in this article. If you want to go deeper with this topic you might want to have a look at the Fedora wiki and documentation – there are plenty useful information:

In thie case a NTP configuration shall be packaged in a RPM file and rolled out on all hosts in the company. First of all, a specfile is created for the future RPM package:

$ cd ~su-rpmdev/rpmbuild/SPECS
$ rpmdev-newspec mycompany-ntp
Skeleton specfile (minimal) has been created to "mycompany-ntp.spec".

All rpmdev tools doesn't exist under EL5 so that you'll have to create the specfile on your own!

If you're on EL6, the previous command created a skeleton of a RPM specfile – this file is edited using an editor now:

$ vim mycompany-ntp.spec
Name:           mycompany-ntp
Version:
Release:        1%{?dist}
Summary:

Group:
License:
URL:
Source0:

BuildRequires:
Requires:

%description

%prep
%setup -q

%build
%configure
make %{?_smp_mflags}

%install
rm -rf $RPM_BUILD_ROOT
make install DESTDIR=$RPM_BUILD_ROOT

%clean
rm -rf $RPM_BUILD_ROOT

%files
%defattr(-,root,root,-)
%doc

%changelog

Beside meta information about the application, additional scripts for compiling and creating the package are included. The particular meta variables are largely self-explanatory – some explanations:

  • Name – name of the software package (mycompany-ntp-1.0-1)
  • Version – version of the software package (my-company-ntp-1.0-1)
  • Release – release of the software package (my-company-ntp-1.0-1)
  • Summary – short package description
  • Group – software group the package belongs to. A full listing of possible groups can be found below: /usr/share/doc/rpm-*/GROUPS
  • License – assigned license (GPL, AGPL, MIT, )
  • URL – URL of the project website
  • Source0 – relative path starting in SOURCES to the source code
  • BuildRequires – dependencies that are needed to compile the source code (e.g. header files or library source codes)
  • Requires – dependencies that are needed to run the application (in this case: the NTP daemon)
  • Conflicts – defines third-party software packages and version that are incompatible with the customized software packages (missing in template – but very important)
  • %description – long description of the software package
  • %changelog – changelog, applied changes, authors, and so on

Some of the additional script or macro segments:

  • %prep – preparation before compiling the source code; e.g. applying patches
  • %build – compiling the source code
  • %install – creating the RPM package; copying of created binary files and directory structures to the RPM package

To be honest – it's not hard to lose track in the beginning. I suggest to have a look at finished RPM specfiles – often you'll learn some "tricks" by having a look at other's work.

There are two ways to have a look at finished RPM specfiles. Some additional repositories are serving their specfiles over SVN or GIT. As an example, Repoforge has a public GIT mirror for this: [click me!]

Another possibility is to include the optional source code channels of additional repositories – like EPEL – and download the source code packages:

# vi /etc/yum.repos.d/epel.repo
...
[epel-source]
...
enabled=1

ESC ZZ
# yum install yum-utils
# yumdownloader --source nrpe

The RPM packages can be adapted using cpio – another more comfortable way is to use Midnight Commander to examine the package. The RPM package includes a CPIO archive named CONTENTS.cpio – the specfile is stored there:

To build a preferably "clean" package it is important to create a source code archive – even if you only want to share configuration files. It is also possible to create those files directly in the install macro of the specfile – but especially if you want to share multiple or long configuration files you'll lose track . In this case an archive containing the NTP configuration is created:

$ mkdir ~/rpmbuild/SOURCES/mycompany-ntp-1.0
$ cd ~/rpmbuild/SOURCES/mycompany-ntp-1.0
$ vi ntp.conf
driftfile /var/lib/ntp/drift
server localserver.loc

ESC ZZ
$ cd ..
$ tar cf mycompany-ntp-1.0.tar.gz mycompany-ntp-1.0/*

Afterwards the RPM specfile is modified – my version looks like this:

$ cd ../SPECS
$ cat mycompany-ntp.spec
Name:           mycompany-ntp
Version:        1.0
Release:        1%{?dist}
Summary:        MyCompany customized NTP configuration

Group:          System Environment/Daemons
License:        GPL
Source0:        %name-%version.tar.gz

Requires:       ntp

%description
This package includes MyCompany customized NTP configuration files.
Feel free to delete this package if received outside the MyCompany network.

%prep
%setup -q

%build

%install
rm -rf $RPM_BUILD_ROOT
install -m 0755 -d %{buildroot}%{_sysconfdir}/mycompany
install -m 0644 ntp.conf %{buildroot}%{_sysconfdir}/mycompany/ntp.conf

%clean
rm -rf $RPM_BUILD_ROOT

%files
%config(noreplace) %{_sysconfdir}/mycompany/ntp.conf

I'm sure you notices that the modified NTP configuration file isn't stored at its accurate place (/etc/ntp.conf) – it is rather saved in an alternative directory (/etc/mycompany/ntp.conf). The reason for this is that the former configuration file (which is provided by the ntp package) can be replaced because of the noreplace flag:

#code quote of the ntp RPM specfile
%files
%config(noreplace) %{_sysconfdir}/ntp.conf

This package stores its configuration file in an alternative directory which can't be overwritten by other RPM packages.

You'll have to help yourself by using a "trigger trick" that saves the former configuration and creates a symbolic link to the new configuration after the installation. After the uninstallation of the package this step is rolled back. For implement this you'll have to add the following macros to your RPM specfile:

%triggerin -- mycompany-ntp
if [ ! -h /etc/ntp.conf -o ! "`readlink /etc/ntp.conf`" = "/etc/mycompany/ntp.conf" ] ; then
        if [ -e /etc/ntp.conf ] ; then
                mv -f /etc/ntp.conf /etc/ntp.conf.orig
        fi
        ln -s /etc/mycompany/ntp.conf /etc/ntp.conf
fi

%triggerun -- mycompany-ntp
if [ $1 -eq 0 -a $2 -gt 0 -a -e /etc/ntp.conf.orig ] ; then
        mv -f /etc/ntp.conf.orig /etc/ntp.conf
fi

%triggerpostun -- mycompany-ntp
if [ $2 -eq 0 ]; then
        rm -f /etc/ntp.conf.rpmsave /etc/ntp.conf.orig
fi
if [ -e /etc/ntp.conf.rpmnew ] ; then
        mv /etc/ntp.conf.rpnnew /etc/ntp.conf.orig
fi

%postun
if [ -e /etc/ntp.conf.orig -a -h /etc/ntp.conf -a ! -e "`readlink /etc/ntp.conf`" ] ; then
        mv -f /etc/ntp.conf.orig /etc/ntp.conf
fi

Simplified summarization of the triggers functions:

  1. Installation: if the former configuration file exists ".orig" is appended to the file name and a symbolic link to the new configuration file is created
  2. Deinstallation of the customized NTP configuration: if the former configuration file still exists the file name is reset
  3. After uninstalling the customized NTP configuration: remaining additional or newly added NTP configuration files are deleted
  4. After uninstalling: resetting the file name of the former NTP configuration file

I abdicated the URL and BuildRequires tags in my specfile because there are no websites or special compiling dependencies for a customized NTP configuration. 😉

Example 2: Meta packages

There are plenty of applications and configuration that are part of a senseful customized system installation – to name some practical examples: sudo configuration, GNU Screen (of course!), and customized application profiles.

To avoid doing the application installation manually everytime, meta packages can be built to simplify the process. These packages don't have their own files – they only have dependencies to other packages.

Because of this, RPM meta packages aren't assigned to special processor architectures (x86_64, i686, s390, ) – an additional specification "BuildArch noarch" is added in the specfile.

Another practical example: a meta package for installing NTP including the customized configuration and telnet for checking the daemon's function:

$ cd SOURCES
$ mkdir mycompany-ntp-full-1.0
$ tar cvfz mycompany-ntp-full-1.0.tar.gz mycompany-ntp-full-1.0
$ cd ../SPECS
$ cat mycompany-ntp-full.spec
Name:           mycompany-ntp-full
Version:        1.0
Release:        1%{?dist}
Summary:        MyCompany customized NTP configuration and netstat utility

Group:          System Environment/Daemons
License:        GPL
Source0:        %name-%version.tar.gz

BuildArch:      noarch
Requires:       ntp mycompany-ntp net-tools

%description
This package includes MyCompany customized NTP configuration files.
Feel free to delete this package if received outside the MyCompany network.

%prep
%setup -q

%build

%install

%clean
rm -rf $RPM_BUILD_ROOT

%files

Using rpm you can list the package's dependencies:

$ rpmbuild -bb mycompany-ntp-full.spec
$ rpm --query -Rp ../RPMS/noarch/mycompany-ntp-full-1.0-1.el6.noarch.rpm
ntp
mycompany-ntp
net-tools
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rpmlib(CompressedFileNames) <= 3.0.4-1

Package dependencies can very depending on the distribution release. In this example between RHEL5 and 6 – the packages providing the telnet command are different here:

rhel5 # rpm -qf $(which telnet)
krb5-workstation-1.6.1-70.el5_9.2
rhel6 # rpm-qf $(which telnet)
net-tools-1.60-110.el6_2.x86_64

You might want to consider this in the specfile:

Requires:       ntp mycompany-ntp
%{?el5:Requires: krb5-workstation}
%{?el6:Requires: net-tools}

The first dependeny line is significant for all releases, the following lines are considered under RHEL5 or RHEL6.

You can also define particular versions in combination with the Requires and Conflicts tags – for example, if you want to reference a myapp package which is at least version 1.1. One of the following lines can be used:

Requires:        myapp >= 1.1
Conflicts:       myapp < 1.1

If you want to reference a special version of a package there are also two possibilities – choose one:

Requires:        myapp = 1.1
Conflicts:       myapp < 1.1 myapp > 1.1

It's a kind of philosophical question which of the two possibilites is used – like the question if the glass is half-full or half-empty. Either a package is excluded or referenced explicitely. 😉

Further information regarding RPM dependencies can be found on the official RPM website: [click me!]

Let's go back to the former motivation of this meta package: as an alternative you can also define package groups in your own YUM repository. If you have already worked with the YUM commands grouplist, groupinstall and groupremove, you might know this logical grouping of software packages. You can find an interesting article about this in the YUM wiki: [click me!]

Package and repository creation

Okay – we have a RPM specfile now, what's next? RPM packages are creating using the rpmbuild utility. This tool has a plenty of swithces and arguments – as an example you can also create source code packages or packages for additional processor architectures.

Important parameters:

Parameter Explanation
-ba Creates a binary and source code package
-bb Creates a binary package
-bp Extracting and patching (if necessary) of the source code
-bs Creates a source code package
–target=noarch Creates a platform independent package
–target=i686 a 32-bit package
–target=x86_64 a 64-bit package

Some examples:

# Creates a binary and source code package of myapp
$ rpmbuild -ba myapp.spec
# Creates a 32-bit binary package of myapp
$ rpmbuild -bb --target=i686 myapp.spec

Afterwards you'll find a RPM package below RPMS depending on your processor architecture (noarch, i686 or x86_64). If you're a proud owner of a IBM System Z machine you might want to have a look below s390 or s390z. 😉

$ ls RPMS/*/*.rpm
RPMS/i386/mycompany-ntp-1.0-1.el6.i386.rpm

This RPM package could now be installed using YUM:

# yum localinstall --nogpgcheck mycompany-ntp-1.0-1.el6.i386.rpm
Synchronization and automation

After creating the RPM packages on the particular test machines (e.g. RHEL6 and RHEL5) the packages need to be copied to the web server. I suggest using SSH and Rsync for a synchronization between the test machines, the main test machine and the web server.

If you don't want to do this manually every time you can automate this using a small script:

1.Synchronization between the EL5 machine and the main test machine:

$ ln -s /usr/src/redhat /home/su-rpmdev/rpmbuild
$ vi /home/su-rpmdev/export_repo.sh
#!/bin/sh
rsync -avz --delete /home/su-rpmdev/rpmbuild/RPMS/ /opt/myrepo
createrepo --database /opt/myrepo/
rsync -avz --delete -e ssh /opt/myrepo/ su-rpmdev@MAIN:/opt/myrepo/5Server

ESC ZZ
$ chmod +x /home/su-rpmdev/export_repo.sh
$ ./home/su-rpmdev/export_repo.sh

The first rsync command copied all RPM packages of all processor architectures below /opt/myrepo – if a RPM package is deleted in the source, it is also deleted below /opt/myrepo. createrepo created a SQLite database for the YUM repository (myrepo) below /opt/myrepo. The second rsync command copies the local YUM repository to the main test machine (MAIN).

2.Synchronization between the main test machine (EL6) and the web server:

$ vi /home/su-rpmdev/export_repo.sh
#!/bin/sh
rsync -avz --delete /home/su-rpmdev/rpmbuild/RPMS/ /opt/myrepo/6Server
createrepo --database /opt/myrepo/6Server
rsync -avz --delete -e ssh /opt/myrepo/ su-rpmdev@WEB:/var/www/html/myrepo

ESC ZZ
$ chmod +x /home/su-rpmdev/export_repo.sh
$ ./home/su-rpmdev/export_repo.sh

The first rsync command copies all (EL6) RPM packages below /opt/myrepo/6Server. After that createrepo creates a SQLite-Datenbank for the EL6 repository. The second rsync command copies the whole repository (including the other EL5 repository) to the web server (WEB).

Usage and test

To use the new YUM repository on other hosts, an appropriate YUM configuration needs to be created. In this file the repository URL and other parameters like package signing are defined. The syntax looks a bit like good-old Windows .ini files:

# vi /etc/yum.repos.d/myrepo.repo
[myrepo]
name=mycompany packages for EL
baseurl=http://server01/$releasever
enabled=1
gppgcheck=0

ESC ZZ

You might see the variable $releasever – it was mentioned in the table above. This variable is replaced by another value depending on your distribution release – in this case by 5Server or 6Server. These directories had been filled with the appropriate RPM packages from the test machines.

After that, all available packages of the repository can be listed:

# yum --disablerepo='*' --enablerepo='myrepo' makecache
# yum --disablerepo='*' --enablerepo='myrepo' list available
...
mycompany-ntp                      1.0-1                      myrepo

If you have multiple repository web servers (e.g. because of big setups for processing the amount of requests or compensate failures) you can assign a mirror list:

# vi /etc/yum.repos.d/myrepo.repo
[myrepo]
...
#baseurl=...
mirrorlist=myrepo.mirror

ESC ZZ
# vi /etc/yum.repos.d/myrepo.mirror
http://server01/$releasever
http://server02/$releasever
http://server03/$releasever

For every download YUM uses one of these servers – beginning with the first one. If this server fails or doesn't have the file YUM will select the next server.

By the way – these configuration files could be shared using a RPM package, too. If this is done, you only need to install a RPM package to access the YUM repository zugreifen. This is how access to Fedora's Extra Packages For Enterprise Linux (EPEL) is provided: [click me!]

This package could look like this:

# vi SPECS/myrepo-release.spec
Name:           myrepo-release
Version:        1.0
Release:        1%{?dist}
Summary:        mycompany Packages for Enterprise Linux repository configuration

Group:          System Environment/Base
BuildArch:      noarch
License:        GPL
URL:            http://www.mycompany.com
Source0:        myrepo-release-%{version}.tar.gz
BuildRoot:      %{_tmppath}/%{name}-%{version}-%{release}-root-%(%{__id_u} -n)

%description
This package contains the mycompany customized Packages for Enterprise Linux repository.

%prep
%setup -q

%build

%install
install -m 0755 -d %{buildroot}%{_sysconfdir}/yum.repos.d/
install -m 0755 myrepo.repo %{buildroot}%{_sysconfdir}/yum.repos.d/myrepo.repo
install -m 0755 myrepo.mirror %{buildroot}%{_sysconfdir}/yum.repos.d/myrepo.mirror

%clean
rm -rf $RPM_BUILD_ROOT

%files
%config(noreplace) %{_sysconfdir}/yum.repos.d/myrepo.repo
%config(noreplace) %{_sysconfdir}/yum.repos.d/myrepo.mirror

%changelog
* Sat Jun 29 2013 FirstName LastName  - 1.0-1
- initial release

ESC ZZ
# tar tvfz SOURCES/myrepo-release-1.0.tar.gz
-rw-r--r-- su-rpmdev/su-rpmdev 94 2013-06-28 18:12 myrepo-release-1.0/myrepo.mirror
-rw-r--r-- su-rpmdev/su-rpmdev 189 2013-06-28 18:09 myrepo-release-1.0/myrepo.repo

# tar xvfz SOURCES/myrepo-release-1.0.tar.gz -C SOURCES/
# cat SOURCES/myrepo-release-1.0/myrepo.mirror
http://server01/$releasever
http://server02/$releasever
http://server03/$releasever
# cat SOURCES/myrepo-release-1.0/myrepo.repo
[myrepo]
name=mycompany packages for EL
#baseurl=http://server01/$releasever
mirrorlist=/etc/yum.repos.d/myrepo.mirror
enabled=1
gppgcheck=0

After the RPM specfile had been created the package can be created and distributed easily. Finally, you only have to install the RPM package to use the YUM repository:

# rpmbuild -bb SPECS/myrepo-release.spec
# scp RPMS/noarch/myrepo-release-1.0-1.rpm root@host:/root
root@host # yum localinstall --nogpgcheck myrepo-release*.rpm
root@host # yum repolist
repoid            reponame
...
myrepo            mycompany packages for EL

Et voilà! 🙂

Perspective / Additional ideas

Of course there is a lot that can be optimized. To name some of these things:

  • SSH public-key authentification – while synchronizing the RPM packages between the test machines and the web server the password of the service user has to be entered twice. This can be avoided by implementing SSH public-key authentification.
  • Theme for viewing the repository in web browsers – when the repository is accessed using a normal web browser the directory contents are displayed. Depending on your web server this might look ugly – a header including your company's logo and usage information could be used in addition.
  • Version control using SVN / GIT – especially if you're working in a team on your repository it might be very handy to use a version control system like SVN or GIT.
    Integration in Red Hat Network Satellite, Spacewalk or SuSE Manager – if you're maintaining a big amount of hosts you (hopefully) have the management suite of your preferred distributor. You can include your repository to simply share your packages.
  • Package signing – To avoid manipulated packages, packages can be signed using a key which has to be copied to every client. Using this mechanism, clients can verify that the downloaded packages are authentic.

To manage a plenty of systems you don't need a "egg-laying, milk-bearing woolly sow" – RPM is a mighty tool which can also be used for software and configuration management. If you're engaged with Red Hat Enterprise Linux, you will often into a situation where you have to automate something quickly. And that's where RPM can also help you to reach your goal quickly. 😉

Creative Commons License
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 3.0 Unported License .