Installation of open source version of PBSpro

The choice between the commercial and open source versions of PBS Pro is entirely up to you and your organization. If your organization does not require commercial support of all software, you may want to start with the open source version.

You may decide to purchase commercial support in the future. The commercial and open source versions of the product are very similar in terms of functionality, but the commercial version receives additional QA and validation.

You may download pre-built packages of the open source release from http://pbspro.org/4 or download the source code from GitHub. If you download the source code, please do so from the latest release branch https://github.com/PBSPro/pbspro/tree/release_14_1_branch5

Instructions to build from source may be found in the INSTALL file in the source package.

Looks like you need to built RPMs after you install the server and install client using them.  Some information about how to  generate RPMs for distribution on multiple systems can be found at: https://pbspro.atlassian.net/wiki/display/PBSPro/Building+PBS+Pro+Using+rpmbuild7

Unlike torque, pbspro uses ProstresSQL to store information about nodes, queues, jobs etc.  It should be installed as a prerequisite on the headnode.  That means that most information about Torque installation is not applicable. Configuration is also different.  In other words this is a different product.

For example, in order to add nodes you need to execute the command qmgr -c "create node $NODE_HOSTNAME" for each such node. You can do multiple notes in a loop:

while read line; do

[ -n "$line" ] && qmgr -c "create node $line";

done < ./nodefile

where ./nodefile contains the list of nodes, one per line.

You can find the administration and other guides here: http://www.pbsworks.com/SupportGT.aspx?d=PBS-Professional,-Documentation113 As usual they are not very useful. Useful posts are mostly scattered in various discussion forums. 

In recent version the commercial version of the Installation Guide no longer applies to the OSS release because the INSTALL script is no longer needed, but you still may find some (minimal) information in it useful.

PBS Pro does not require a shared filesystem to function unless you are employing the server failover feature. In this case data store should be on the shared filesystem which is mounted only on the active headnode. 

In general, you the RPMs can be used as follows (supplied RPMs can be used probably only on RHEL/CentOS 7.1; I have had problems with trying to install them on RHEL 7.3):

1. pbspro-server: This should be installed only on the headnode (or two if you employ failover) in your cluster where the server and scheduler will run. Most admins prefer not to run jobs on this node, but it is certainly possible. This RPM also installs everything you would get with the execution and client packages.
2. pbspro-execution: This package should be installed on all nodes where you want your jobs to run. It provides the pbs_mom daemon, but not the server or scheduler. Admins generally restrict login access to these nodes to prevent users from logging in and impacting running jobs. This RPM also includes everything you would get with the client package.
3. pbspro-client: Allows users to submit and control their jobs.  This is connected with the concept of login nodes: this package should be installed on nodes where users are allowed login access, like their workstations. It does not contain any daemons, but allows users to submit and control their jobs.

There are also two optional packages:

• pbspro-debuginfo: Only required when trying to debug an issue via gdb. You won't need this unless you manage to crash something in PBS Pro that creates a core dump.
• pbspro-*-src: This is the source code for PBS Pro, also used for debugging.

Generally you should not mix different versions of PBS and use the same version of bothe server and the MOM. All daemons and commands must be the same version of PBS Professional.

Firewalls

PBS needs to be able to use any port for outgoing connections, but only specific ports for incoming connections. If you have firewalls running on the server or execution hosts (bad idea, but sometimes you do not control the environment ans security department attempt to wave dead chickens  can't be overruled) , be sure to allow incoming connections on the appropriate ports for each host. By default, the PBS server and MoM daemons use ports 15001 through 15004 for incoming connections, and any port below 1024 for outgoing connections. See section 3.3.4, “Network Addresses and Ports”, on page 36 for a list of ports.

Ports Used by PBS Daemons in TPP Mode

• pbs_server 15001 TPP (TCP) All communication to server
• pbs_mom 15002 TPP (TCP) All communication to MoM
• pbs_resmon 15003 TPP (TCP) Scheduler-MoM resource requests (pbs_resmon listens on this port)
• pbs_sched 15004 TPP (TCP) All communication to scheduler
• pbs_datastore 15007 proprietary PBS information storage and retrieval
• License server 6200 proprietary All communication to license server
• pbs_comm 17001 TPP (TCP) All communication to pbs_comm

Firewall-based issues are often associated with server-MoM communication failures and messages such as 'premature end of message' in the log files. Inital installation should be done with the firewall down. Later you can enable it and troubleshoot the issues one by one.  Same is true about SElinux -- permissive mode is recommended for the installation.

Required Name Resolution

This might be a reason you newly installed PBS does not work. PBS is pretty picky in this regard and diagnostic messages are not very illuminating.  Make sure that the following are true:

• Use only one canonical name per host. The canonical name must be unambiguous.
• On the server/scheduler host, the short name must resolve to the correct IP address.
• On the server/scheduler host, the IP address must reverse resolve to the canonical name.
• Make sure that different resolvers cannot disagree when resolving the server host, whether you are using /etc/hosts, DNS, LDAP, NIS, or something else.
• Every MoM must resolve each MoM to the same IP address that the server recognizes for that MoM. So if the server recognizes MoM A at IP address w.x.y.z, all other MoMs must resolve MoM A to w.x.y.z.
• Make sure that the IP address of each machine in the complex resolves to the fully qualified domain name for that machine, and vice versa. Forward and reverse hostname resolution must work consistently between all machines.
• The server must be able to look up the IP addresses for any execution host, any client host, and itself.
• Make sure that forward and reverse name lookup operate according to the IETF standard. The network on which you will be deploying PBS must be configured according to IETF standards.

You can take care of some of the name resolution testing before you install PBS. However, you must do some testing using the pbs_hostn command, after you install PBS. The "Initial Configuration" chapter follows the "Installation" chapter, and includes steps to test name resolution. We include an overview of the whole process here for clarity:

1. Disable firewall
2. Set up and verify name resolution
   •  Test name resolution by using ping command; if necessary, fix & re-test
3. Try to install  PBS
   • Re-test name resolution by using pbs_hostn command.
4. If name resolution does not work correctly:
   • Uninstall PBS
   • Fix name resolution
   • go to p.2

System Clocks in Sync

Clocks on all participating systems should be in sync. NTP should be running.  Difference in clocks can be the reason why clints can't proparly communicate with the headnode.

When configuring /etc/hosts, do the following

When configuring /etc/hosts, do the following

• Use the server’s FQDN as the first item on the first line on the PBS-to-PBS interface
• Use different FQDNs as the first item on other lines
• Use a name on only one line

NOTES:

• If you want redundancy in your network interface, consider using bonding. Aside from presenting a transparent interface, this can allow you to load-balance network traffic across different networks.
• If name resolution is a problem in a network that should be working, tell nscd not to cache the host name of the machine with the problem.
• If you are using nscd and you change an IP address or hostname, restart nscd on all hosts.

User Accounts

Users who will submit jobs must have accounts at the server and at each execution host.

SSH

Typically PBS is configured to use the Secure Copy (scp) for file transfers. The administrator need to set-up the SSH keys as described in section 13.8, "Enabling Passwordless Authentication" on page 870 in the PBS Professional Administrator’s Guide.

See also section 8.5 "Delivery of Output Files" on page 198 in the PBS Professional User’s Guide (pbsworks.com)

Contacting the Server

Use the PBS_SERVER_HOST_NAME parameter in pbs.conf on each host in the complex to specify the FQDN of the server, under these circumstances:

• The host on which the PBS server runs has multiple interfaces and some of these interfaces are limited to a private network that might not be addressable outside of the immediate complex
• The server name to be used in Job IDs needs to be different from the PBS_SERVER parameter. It might become impossible for a client to contact the server where this option is not used or is misconfigured. Take extreme care when using PBS_SERVER_HOST_NAME for this reason.

You can specify the server name with the following order of precedence, highest first:

• Specifying server name at the client
• Specifying server name at the command line, e.g. pbsnodes -s <server name>
• Setting the PBS_PRIMARY and PBS_SECONDARY environment variables
• Setting the PBS_SERVER_HOST_NAME environment variable
• Setting the PBS_SERVER environment variable
• Setting PBS_PRIMARY and PBS_SECONDARY in pbs.conf
• Setting PBS_SERVER_HOST_NAME in pbs.conf
• Setting PBS_SERVER in pbs.conf

Validate the Installation

•Check files and directories: To validate the installation of PBS Professional, at any time, run the pbs_probe command. It will review the installation (installed files, directory and file permissions, etc) and report any problems found. For details, see “pbs_probe” on page 60 of the PBS Professional Reference Guide.

•Check PBS version. Use the qstat command to find out what version of PBS Professional you have:
qstat -fB

•Check hostname resolution:

• At the server, use the pbs_hostn command with the name of each host in the complex. This should complain if hostname resolution is not working correctly. See “pbs_hostn” on page 40 of the PBS Professional Reference Guide.

• Make sure that rcp and/or scp work correctly. They must work outside of PBS before PBS can use them Run rcp and/or scp between machines in the complex to make sure they work. If there are problems, see section 2.1.3, “Name Resolution and Network Configuration”, on page 8.

Installation from source of free version of PBS Pro using the configure script

Adapted from PostgreSQL Linux downloads (Debian) in June 2017:

1. Install the prerequisite packages for building PBS Pro. For CentOS systems you should run the following command as root:

   yum install -y gcc make rpm-build libtool hwloc-devel \ 
       libX11-devel libXt-devel libedit-devel libical-devel \ 
       ncurses-devel perl postgresql-devel python-devel tcl-devel \ 
      tk-devel swig expat-devel openssl-devel libXext libXft \ 
      autoconf automake 

   For openSUSE systems you should run the following command as root:

   zypper install gcc make rpm-build libtool hwloc-devel \ 
      libX11-devel libXt-devel libedit-devel libical-devel \ 
      ncurses-devel perl postgresql-devel python-devel tcl-devel \
      tk-devel swig libexpat-devel libopenssl-devel libXext-devel \ 
      libXft-devel fontconfig autoconf automake 

   For Debian systems you should run the following command as root:

   sudo apt-get install gcc make libtool libhwloc-dev libx11-dev \ 
      libxt-dev libedit-dev libical-dev ncurses-dev perl \ 
      python-dev tcl-dev tk-dev swig \ 
      libexpat-dev libssl-dev libxext-dev libxft-dev autoconf \ 
     automake 

   sudo apt-get install postgresql-server-dev-all

   Note: it is recommended to install PostgressSQL separately on Debian. See instructions at PostgreSQL Linux downloads (Debian)
    

2. Install the prerequisite packages for running PBS Pro.

   In addition to the commands below, you should also install a text editor of your choosing (vim, emacs, gedit, etc.).

   For CentOS systems you should run the following command as root:

   yum install -y expat libedit postgresql-server python sendmail sudo tcl tk libical 

   For openSUSE systems you should run the following command as root:

   zypper install expat libedit postgresql-server python sendmail sudo tcl tk libical1

   For Debian systems you should run the following command as root (postgress server should be installed sep[aratly, may requre reboot):

   apt-get install expat libedit2  python sendmail-bin sudo tcl tk libical1a 

3. Open a terminal as a normal (non-root) user, unpack the PBS Pro tarball, and cd to the package directory.

   tar -xpvf pbspro-14.0.1.tar.gz 
   cd pbspro-14.0.1 

4. Generate the configure script and Makefiles. (See note 1 below)

   ./autogen.sh 

5. Display the available build parameters.

   ./configure --help 

6. Configure the build for your environment. You may utilize the 66 parameters displayed in the previous step. (See note 2 below)

   For CentOS and Debian systems you should run the following command:

   ./configure --prefix=/opt/pbs 

   For openSUSE systems (see note 3 below) you should run the following command:

   ./configure --prefix=/opt/pbs --libexecdir=/opt/pbs/libexec 

7. Build PBS Pro by running "make". (See note 4 below)

   make

8. Configure sudo to allow your user account to run commands as root. Refer to the online manual pages for sudo, sudoers, and visudo.

9. Install PBS Pro. Use sudo to run the command as root.

   sudo make install 

10. Configure PBS Pro by executing the post-install script.

    sudo /opt/pbs/libexec/pbs_postinstall 

11. Edit /etc/pbs.conf to configure the PBS Pro services that should be started. If you are installing PBS Pro on only 96 one system, you should change the value of PBS_START_MOM from zero to one.

    If you use vi as your editor, you would run:

    sudo vi /etc/pbs.conf 

12. Some file permissions must be modified to add SUID privilege. sudo chmod 4755 /opt/pbs/sbin/pbs_iff /opt/pbs/sbin/pbs_rcp
13. Start the PBS Pro services

    sudo /etc/init.d/pbs start 

14. All configured PBS services should now be running. Update your PATH and MANPATH variables by sourcing the appropriate PBS Pro profile or logging out and back in.

    For Bourne shell (or similar) run the following:

    . /etc/profile.d/pbs.sh

15. You should now be able to run PBS Pro commands to submit and query jobs. Some examples follow.

    bash$ qstat -B 
    Server Max Tot Que Run Hld Wat Trn Ext Status 
    ---- ----- ----- ----- ----- ----- ----- ----- ----- ----------- 
    host1 0 0 0 0 0 0 0 0 Active 
    

    bash$ pbsnodes -a 
    host1 <
    Mom = host1  
    ntype = PBS  
    state = free  
    pcpus = 2  
    resources_available.arch = linux  
    resources_available.host = host1  
    resources_available.mem = 2049248kb  
    resources_available.ncpus = 2  
    resources_available.vnode = host1  
    resources_assigned.accelerator_memory = 0kb  
    resources_assigned.mem = 0kb  
    resources_assigned.naccelerators = 0  
    resources_assigned.ncpus = 0  
    resources_assigned.vmem = 0kb  
    resv_enable = True  
    sharing = default_shared  
    license = l  
    

    Now you can run a test

    bash$ echo "sleep 60" | qsub  
    0.host1  
    
    bash$ qstat -a  
    
    host1:  
    Req'd Req'd Elap  
    Job ID Username Queue Jobname SessID NDS TSK Memory Time S Time  
    --------------- -------- -------- ---------- ------ --- --- ------ ----- - -----  
    0.host1 mike workq STDIN 2122 1 1 -- -- R 00:00  
    --------------------------------------------------------------------  
    

1. If you modify configure.ac or adjust timestamps on any files that are automatically generated, you will need to regenerate them by re-running autogen.sh.
2. It is advisable to create a simple shell script that calls configure with the appropriate options for your environment. This ensures configure will be called with the same arguments during subsequent invocations. If you have already run configure you can regenerate all of the Makefiles by running "./config.status". The first few lines of config.status will reveal the options that were specified when configure was run. If you set environment variables such as CFLAGS it is best to do so as an argument to configure (e.g. ./configure CFLAGS="-O0 -g" --prefix=/opt/pbs). This will ensure consistency when config.status regenerates the Makefiles.
3. The openSUSE rpm package expands %_libexecdir to /opt/pbs/lib rather than /opt/pbs/libexec which causes problems for the post- install scripts. Providing the --libexecdir value to configure overrides this behavior.
4. You need to use a POSIX (or nearly POSIX) make. GNU make works quite well in this regard; BSD make does not. If you are having any sort of build problems, your make should be a prime suspect. Tremendous effort has been expended to provide proper dependency generation and makefiles without relying on any non-POSIX features. The build should work fine with a simple call to make, however, complicating things by using various make flags is not guaranteed to work. Don't be surprised if the first thing that make does is call configure again.

NEWS CONTENTS

• 20180621 : How to install pbs on compute node and configure the server and compute node - Users-Site Administrators - PBS Professional Op ( Jun 21, 2018 , community.pbspro.org )
• 20180621 : Building PBS Pro Using rpmbuild - PBS Pro - PBS Pro Confluence ( Building PBS Pro Using rpmbuild - PBS Pro - PBS Pro Confluence, )

Old News ;-)

[Jun 21, 2018] How to install pbs on compute node and configure the server and compute node - Users-Site Administrators - PBS Professional Op

Jun 21, 2018 | community.pbspro.org

How to install pbs on compute node and configure the server and compute node? Users/Site Administrators You have selected 0 posts.

select all

cancel selecting Jun 2016 1 / 9 Jul 2016 Apr 2017 Joey Jun '16 Hi guys,
I am new to HPC and PBS or torque. I am able to install PBS pro from source code on my head node . But not sure how to install the compute node and cconfigure it. I didn't see any documentation in the github either. Can anyone give me some help? Thanks

• created Jun '16
• last reply Apr '17

buchmann Jun '16 Install is pretty similar on the compute nodes - however, you do not need the "server" parts.
There are OK docs on the Altair "pro" site, see answer to previous question "documentation-is-missing/81".

In short, you the Altair docs for v13, and/or the INSTALL file procedure. (Or install from pre-build binaries).
Actual method will depend on your system type etc.

I prefer to install using pre-compiled RPMs (CentOS72 systems), which presently means that I will compile these from tarball+spec-file (slightly modified spec-file).

Hope this helps.
/Bjarne subhasisb Jun '16 @Joey thanks for joining the pbspro forum.

You can find the documentation about pbspro here: https://pbspro.atlassian.net/wiki/display/PBSPro/User+Documentation 730

Kindly do not hesitate to post questions about any specific issues you are facing.

Thanks,
Subhasis Joey Jun '16 1 Thanks for your reply.

I rebuild the CentOS72 rpm with the src from Centos7.zip
installed pbspro-server-14.1.0-13.1.x86_64.rpm on mye headnode
installed pbspro-execution-14.1.0-13.1.x86_64.rpm on my compute node.
On the head node
create /var/spool/pbs/server_priv/nodes with following:

computenode1 np=1

/etc/pbs.conf:
PBS_SERVER=headnode
PBS_START_SERVER=1
PBS_START_SCHED=1
PBS_START_COMM=1
PBS_START_MOM=0
PBS_EXEC=/opt/pbs
PBS_HOME=/var/spool/pbs
PBS_CORE_LIMIT=unlimited
PBS_SCP=/bin/scp

on the compute node

/var/spool/pbs/mom_priv/config as following

$logevent 0x1ff
$clienthost headnode
$restrict_user_maxsysid 999

/etc/pbs.conf
PBS_SERVER=headnode
PBS_START_SERVER=0
PBS_START_SCHED=0
PBS_START_COMM=0
PBS_START_MOM=1
PBS_EXEC=/opt/pbs
PBS_HOME=/var/spool/pbs
PBS_CORE_LIMIT=unlimited
PBS_SCP=/bin/scp

after that I start the pbs on headnode and compute node without error:
#/etc/init.d/pbs start
But when I try to run pbsnodes -a, it tells me:
pbsnodes: Server has no node list
If I run a script it will just Queue there.

Both server firewalld are turned off and pingable.

Can anyone give me some help? Thanks

subhasisb Jul '16 Hi @Joey ,

Unlike torque, pbspro uses a real relational database underneath to store information about nodes, queues, jobs etc. Thus creating a nodes file is not supported under pbspro.

To add a node to pbs cluster, use the qmgr command as follows:

qmgr -c "create node hostname"

HTH
regards,
Subhasis

Joey Jul '16 Thanks for your reply. I thought PBS and torque are the same except one is open source and one is commerical. subhasisb Jul '16 Hi @Joey

They might feel similar since Torque was based on the OpenPBS codebase. OpenPBS was a version of PBS released as opensource many years back.

Post that, Altair engineering has put in a huge amount of effort towards PBS Professional and added tons of features and improvements in terms of scalability, robustness and ease of use over decades which resulted in it becoming the number one work load manager in the HPC world. Altair has now open-sourced PBS Professional.

So, pbspro is actually very different from torque in terms of capability and performance, and is actually a completely different product.

Let us know if you need further information in switching to pbspro.

Thanks and Regards,
Subhasis

10 months later sxy Apr '17 Hi Subhasis,

To add a node to pbs cluster, use the qmgr command as follows:

qmgr -c "create node hostname"

if a site has a few hundreds of compute nodes, the above method is very tedious.
would there be any easy/quick ways to register computer nodes with pbs server like the nodes file in torque?

Thanks,

Sue

mkaro Apr '17 This is one way to accomplish it

while read line; do [ -n "$line" ] && qmgr -c "create node $line"; done <nodefile

where nodefile contains the list of nodes, one per line.

Building PBS Pro Using rpmbuild - PBS Pro - PBS Pro Confluence

Building PBS Pro Using rpmbuild

Skip to end of metadata

• Created by Anne Urban, last modified on Aug 10, 2016

Go to start of metadata

A. Create rpm Development Tree (if it does not exist)

Create the tree by hand, or use "rpmdev-setuptree" to create necessary directories. To create by hand:

1. Go to your home directory; you don't need to be root

2. mkdir rpmbuild

3. cd rpmbuild

4. mkdir BUILD BUILDROOT RPMS SOURCES SPECS SRPMS

---

B. Install Dependencies

Below is the list of prerequisite packages. (These are also specified in the "pbspro.spec" file):

• gcc
• autoconf
• automake
• hwloc-devel
• libX11-devel
• libXt-devel
• libedit-devel
• libical-devel
• ncurses-devel
• perl
• postgresql-devel
• python-devel >= 2.6
• python-devel < 3.0
• tcl-devel
• tk-devel
• swig
• expat-devel (if suse then libexpat-devel)
• openssl-devel (if suse then libopenssl-devel)

1. Install dependencies for building PBS Pro:

For CentOS systems, run the following command as root:

yum install -y gcc make rpm-build libtool hwloc-devel libX11-devel libXt-devel libedit-devel libical-devel ncurses-devel perl postgresql-devel python-devel tcl-devel tk-devel swig expat-devel openssl-devel libXext libXft

For openSUSE systems, run the following command as root:

zypper install gcc make rpm-build libtool hwloc-devel libX11-devel libXt-devel libedit-devel libical-devel ncurses-devel perl postgresql-devel python-devel tcl-devel tk-devel swig libexpat-devel libopenssl-devel libXext-devel libXft-devel fontconfig

2. Install prerequisite packages for running PBS Pro:

In addition to the commands below, you should also install a text editor of your choosing (vim, emacs, gedit, etc.).

For CentOS systems, run the following command as root:

yum install -y expat libedit postgresql-server python sendmail sudo tcl tk libical

For openSUSE systems, run the following command as root:

zypper install expat libedit postgresql-server python sendmail sudo tcl tk libical1

3. Install prerequisite packages for testing PBS Pro:

Make sure you have the following on your local system:

the pip command

the sudo command

the which command

the net-tools package

---

C. Build the rpm Package for PBS Pro <pbs_version>

You can either build from source cloned from GitHub, or build from source rpm (srpm).

To Build from source cloned from GitHub:

Note: <pbs_version> specified at pbspro/src/pbspro.spec

1. Go to pbspro directory which you have cloned (git clone https://github.com/PBSPro/pbspro)
2. Run the autogen.sh script to generate the configure script and Makefile.in files
./autogen.sh
(generates configure script and Makefile.in templates)
3. Run ./configure
4. Generate the source tar that will use to build the rpm
make dist
make dist generates .tar.gz (pbspro-<pbs_version>.tar.gz) file in the same directory.
5. Move the pbspro-<pbs_version>.tar.gz file to ~/rpmbuild/SOURCES
6. Move pbspro/pbspro.spec file to ~/rpmbuild/SPECS
7. Change directory to ~/rpmbuild/SPECS
8. Run the below command
rpmbuild -ba pbspro.spec (This command will generate the below rpm packages in ~/rpmbuild/RPMS dirctory)

a) pbspro-client-<pbs_version>-0.x86_64.rpm

b) pbspro-debuginfo-<pbs_version>-0.x86_64.rpm

c) pbspro-execution-<pbs_version>-0.x86_64.rpm

d) pbspro-server-<pbs_version>-0.x86_64.rpm

---

To Build the rpm Package from pbspro-<pbs_version>.src.rpm:

1. Download from http://download.opensuse.org/repositories/home:/pbsproci/

2. You can either use the –rebuild option, or you can build the rpm yourself.

To use the --rebuild option:

Do an rpmbuild –rebuild pbspro-<pbs_version>.src.rpm

To build the rpm yourself:

Install the pbspro-<pbs_version>.src.rpm

rpm -i pbspro-<pbs_version>.src.rpm

This installs pbspro-<pbs_version>.tar.gz and pbspro.spec in their respective directories under the rpmbuild dev tree.

Change directory to ~/rpmbuild/SPECS and run the below command

rpmbuild -ba pbspro.spec (This command generates the below rpm packages in ~/rpmbuild/RPMS dirctory)

a) pbspro-client-<pbs_version>-0.x86_64.rpm

b) pbspro-debuginfo-<pbs_version>-0.x86_64.rpm

c) pbspro-execution-<pbs_version>-0.x86_64.rpm

d) pbspro-server-<pbs_version>-0.x86_64.rpm

---

D. Install and Test Your rpm Package Locally

Installing and Testing Locally on CentOS7:

1. Log in as root
   
2. Change directory to ~/rpmbuild/RPMS
   
3. Install the package: yum -y install /root/rpmbuild/RPMS/x86_64/pbspro-server-<pbs_version>.x86_64.rpm
4. Enable MoM via "PBS_START_MOM=1" in /etc/pbs.conf file
5. Start the PBS Pro daemons using /etc/init.d/pbs start
6. From a non-root account, make sure that PBS works by running a job
7. Continue as root
8. cd test/fw
9. Install PTL and dependencies: pip install -r requirements.txt .
10. Create user accounts and groups required by PTL: pbs_config --make-ug
11. cd ../tests
12. Run PTL tests: pbs_benchpress -l INFOCLI2 -o ptl.txt
13. Once pbs_benchpress completes, you can find the PTL log in ptl.txt

---

Installing and Testing Locally on openSUSE 13.2:

1. Log in as root
   
2. Change directory to ~/rpmbuild/RPMS
3. Install the package: zypper -n install /root/rpmbuild/RPMS/x86_64/pbspro-server-<pbs_version>.x86_64.rpm
4. Enable MoM via "PBS_START_MOM=1" in /etc/pbs.conf file
   
5. Start the PBS Pro daemons using /etc/init.d/pbs start
6. From a non-root account, make sure that PBS works by running a job
7. Continue as root
8. cd test/fw
9. Install PTL and dependencies: pip install -r requirements.txt .
10. Create user accounts and groups required by PTL: pbs_config --make-ug
11. cd ../tests
12. Run PTL tests: pbs_benchpress -l INFOCLI2 -o ptl.txt
13. Once pbs_benchpress completes, you can find the PTL log in ptl.txt

---

Site Map

1. Bjarne Büchmann

   Nice description, thanks.

   It warrants a single comment, though, based on my experience building on CentOS 7.2: CentOS 7 uses postfix rather than sendmail.

   > yum install -y ... sendmail ...

   For build on CentOS, I do not recommend installing sendmail unless you really want it. Sendmail is considered deprecated on CentOS7, and the default mail transport agent (MTA) is postfix. To have the build work with any of the MTAs, then modify the spec file slightly to require MTA rather than sendmail, like eg:

   perl -pi -e 's/^Requires: sendmail$/Requires: MTA/' pbspro.spec

   If no MTA is already installed, then I would opt for postfix rather than sendmail, ie. to do

   > yum install -y ... postfix ...

   See also http://community.pbspro.org/t/centos7-install-issues/83/4

   /Bjarne

1. Bjarne Büchmann

   In order to do the test part (actually get any job to run), I had to define the server as a node (in qmgr). Just starting the MOM was not enough, as the server does not want to talk to the mom (I guess).

   So, likely(?) something like this is missing in the steps (Section D after step 5 or so):

   sudo qmgr -c "create node $(hostname -s)"

   Then, probably we also need to restart both server and mom to get the config right, ie. do something like

   sudo /etc/init.d/pbs restart

   Can somebody confirm this?

   /Bjarne

   PS: I use systemctl rather than the init scripts directly, but that is a different story.

1. Bjarne Büchmann

   Further comments on the test part (this is for centos72, but could have impact on other flavours as well):

   A: The use of pip to install python-packages system-wide is likely not a good choice, as pip will modify the package-maintained (yum/rpm) python installation. If you later upgrade the underlying python libs (with yum), then lots of stuff could go wrong(?)

   The three packages in requirements.txt could be installed from standard RPMs with something like:

   sudo yum install -y python-nose python-beautifulsoup4 pexpect

   (This of course assumes that the standard packages are "new enough" for the tests to run.)

   B: I cant find pbs_config as part of my installation/compiled rpm. I do not see it in the pre-complied rpm either (pbspro-server-14.1.0-13.1.x86_64.rpm). Did I do something wrong, or??

   C: What exactly is "pbs_config --make-ug" supposed to do? I prefer not to have it modify (automagically) say, my users and groups. I'd much rather do that manually.
   As part of other mentioned pre-prequisites (eg. section 3.5.2 in the installation and upgrade guide for pbspro 13.1) I created a user and group named pbsdata. Has it anything to do with that?

   I much prefer to have identical UIDs and GIDs across machines, which is why I try to always define the necessary users and groups prior to the package installation. Comments?

   D: Would it be possible to run these tests as a standard user? Or at least define some pbs manager/operator, such that they do not have to run as root? Running test/build scripts as root is not a secure way to do stuff.

   Thanks - I hope someone can follow up on these issues.

   /Bjarne

   • Permalink
   • Aug 04, 2016

   1. 

      Hiren Vadalia

      A)

      I agree with you that use of pip to install system wide is not good choice. But here while installing PTL you can pass either '-–user' or '–-prefix' option to pip which will not install PTL system wide.
      or you can manually install those dependencies using yum install and then pip will not install those dependencies.
      NOTE: if you install ptl in user specified directory then please update PYTHONPATH and PATH variable with user specified directory.

      B)
      since pbs_config is part of PTL, you will get that command only after you install PTL. This is because not all PTL commands (except pbs_loganalyzer and pbs_stat) is included in PBS rpm package. So, what you are seeing is correct and you are not doing anything wrong.

      C)
      "pbs_config --make-ug" creates users and groups required for PTL as per pre-defined (as written below) configuration.
      However it is not mandatory to run "pbs_config --make-ug" before running tests, if you have created users and groups manually then you can ignore this step.
      Pre-defined configuration for users and groups:
      Groups:
      group_name=tstgrp00 gid=1900
      group_name=tstgrp01 gid=1901
      group_name=tstgrp02 gid=1902
      group_name=tstgrp03 gid=1903
      group_name=tstgrp04 gid=1904
      group_name=tstgrp05 gid=1905
      group_name=tstgrp06 gid=1906
      group_name=tstgrp07 gid=1907
      group_name=pbs gid=901
      Users:
      username=pbstest uid=4355 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbstest shell=/bin/bash
      username=pbsoper uid=4356 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbsoper shell=/bin/bash
      username=pbsadmin uid=4357 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbsadmin shell=/bin/bash
      username=pbsroot uid=4371 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbsroot shell=/bin/bash
      username=pbsmgr uid=4367 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbsmgr shell=/bin/bash
      username=pbsother uid=4358 gid=tstgrp00 groups=tstgrp02,pbs home_dir=/home/pbsother shell=/bin/bash
      username=pbsuser uid=4359 gid=tstgrp00 home_dir=/home/pbsuser shell=/bin/bash
      username=pbsuser1 uid=4361 gid=tstgrp00 groups=tstgrp01,tstgrp02 home_dir=/home/pbsuser1 shell=/bin/bash
      username=pbsuser2 uid=4362 gid=tstgrp00 groups=tstgrp01,tstgrp03 home_dir=/home/pbsuser2 shell=/bin/bash
      username=pbsuser3 uid=4363 gid=tstgrp00 groups=tstgrp01,tstgrp04 home_dir=/home/pbsuser3 shell=/bin/bash
      username=pbsuser4 uid=4364 gid=tstgrp01 groups=tstgrp04,tstgrp05 home_dir=/home/pbsuser4 shell=/bin/bash
      username=pbsuser5 uid=4365 gid=tstgrp02 groups=tstgrp04,tstgrp06 home_dir=/home/pbsuser5 shell=/bin/bash
      username=pbsuser6 uid=4366 gid=tstgrp03 groups=tstgrp03,tstgrp07 home_dir=/home/pbsuser6 shell=/bin/bash
      username=tstusr00 uid=11000 gid=tstgrp00 home_dir=/home/tstusr00 shell=/bin/bash
      username=tstusr01 uid=11001 gid=tstgrp00 home_dir=/home/tstusr01 shell=/bin/bash

      D)
      Yes, you can run tests as a standard user but for that you have to setup sudoers file such a way that standard user can do operation without password as another user or as root using sudo command.
      However we can't totally eliminate use of root as there are some PBS operation which only root can do like:
      reading and parsing accounting logs
      changing PBS configuration files
      running qmgr command with operation like create node/queue
      setting queue/node/server attributes
      start/stop/restart/status of PBS daemons
      etc...

      • Permalink
      • Aug 05, 2016

      1. 

         Bjarne Büchmann

         Thank you, Hiren, for the in-depth explanations. I do have more questions and comments (yes, I know - I query a lot).

         Regarding pnt A:

         The "pip-instruction" reads (for centos):

         > Install PTL and dependencies: pip install -r requirements.txt.

         Hiren adds:

         > But here while installing PTL you can pass either '-–user' or '–-prefix' option to pip which will not install PTL system wide

         In fact, I am still unsure about what "PTL" (likely a pbs/python/something lib - Pbs Test Lib?). I guess the pip line will install it based on what is in the actual test/fw dir (and particularly the ptl subdir), but I am still not in a position to actually want to try this command on my system. Too much unknown stuff going around. (Also see below).

         Regarding pnts C+D:

         Adding 15 new users and 7 new groups just to run a test suite seems excessive. Also, adding these to wheel/sudoers is not really different from running under root, so it is not an option for me.

         >However we can't totally eliminate use of root as there are some PBS operation which only root can do like:
         >reading and parsing accounting logs
         >running qmgr command with operation like create node/queue
         >setting queue/node/server attributes

         All the above may be done by a pbs manager, right? No sudo/root access required. So, presumably adding a few of the new users as pbs managers and/or operators should do the trick - at least for the above commands.

         >start/stop/restart/status of PBS daemons

         Obviously, this will need some OS access. But if we know that this is all we need, then we can limit it to a few explicit commands, which will be OK for a system like ours. It's the granting full access to everything, which disturbs me.

         >changing PBS configuration files

         Which files need to be changed? Stuff /etc/pbs.conf or stuff under PBSHOME? Is there a way to do what you need without full root access? Could the files be owned by a "pbs admin" group (eg pbsdata?) and then modified by pbs users, which is member of that group?

         > etc...

         Really, there is more? (Yes, presumably adding jobs as different users, and such I guess).

         Full root/OS access for a test suite is still no-go in my view. (Probably a lot of other sys admins would feel the same).

         I'd love to install PTL and run the test suite, but I cannot in good conscience follow the instructions as they are laid out.

         If some of these issues are ironed out, then presumably the test suite could be added as an RPM - depending on the requirements (as RPMs) as well as on the particular pbs RPM version as well. It still should be done in a way such that the test suite can run without sudo access, though. These are just random thoughts, and I don't know it it will happen.

         Best

         Bjarne

         • Permalink
         • Aug 05, 2016

         1. 

            Subhasis Bhattacharya

            Hi Bjarne,

            You have brought forth a great point, and this had been something many of us had been thinking on for a while now. PTL (PBS Test lab) really needs to be made to execute as much as possible without needing root (sure allowing sudo for everything is not an option). In fact we had been discussing the same even about the product itself, ie, PBSPro ceasing to need to be root. (The fact that PTL currently needs root or a very permissive sudoers file is already a problem in executing this inhouse for us)

            Now, since the entire PTL sources are open, as community members, we can contribute towards eliminating these restrictions from PTL (and even PBS) step by step. Currently, I think, there are a few ways to trust the PTL code and execute it:

            a) The PTL sources are open, so you can figure out exactly what it would do if you executed it. (still too much hassle I guess)

            b) Create a container with PTL and use that to run tests instead of doing it on your host (we, the community could even create a container image with PTL and all the users etc preconfigured in it for others to safely use)

            c) Exercise PTL via our automatic CI test runs on the Travis cloud (via github): When you submit a PBS code change (or even submit a new PTL test) Travis will execute those tests automatically on a cloud instance.

            Thanks and Regards,

            Subhasis

            • Permalink
            • Aug 05, 2016

            1. 

               Bjarne Büchmann

               If I understand correctly, the PBS Test Lab would be something to be run on a dedicated test machine - especially if one is to modify the PBS code base, and want to make sure that the whole shebang still works. As I have installed just the vanilla 14.1.0, I (likely?) do not need to run those tests at all - for sure not on our production systems.

               I thought that the tests were more in line of checking if we got the installations right. Executing in a container - or on an external host system - will for sure not help me in that regard,

               Anyway, for now I will skip these tests.

               I would suggest to move the sections regarding tests to a completely different page / howto - dedicated to developers, who really need to modify code etc. However, maybe a simpler test suite could be set up - to just test simple functionality of a created system after install...?

               Best,

               Bjarne

               • Permalink
               • Aug 05, 2016

               1. 

                  Subhasis Bhattacharya

                  You are right. The PTL tests are to test the functionality of the software. This is not an installation check. You would only need to run PTL if you changed code and wanted to check whether you might have broken some existing functionality (ie caused regressions).

                  So, for your case, you can just go ahead with 14.1.0 downloaded version and run it.

                  Your suggestion about testing the installation itself sounds like great advise - we do have a tool called pbs_probe that we are coming up with which will check that required files are installed and permissions are all in place. Maybe we can extend the functionality of that tool to be more useful.

                  Also, I do agree we need to move the pages to reflect the content better towards the target user/reader. We hope to make it better organized soon (Anne Urban - lets chat about this shortly)

                  Thanks for all the inputs.

                  Regards,

                  Subhasis

Installation of the client

Building RPM packages from PBSpro source

Installing PBSpro on RHEL or CentOS 6

Issues with PostgresSQL

Sendmail instead of postfix issue

Recommended Links

Softpanorama Recommended

Top articles

Sites

Top articles

Sites

pbspro-INSTALL at master · PBSPro-pbspro · GitHub

Building PBS Pro Using rpmbuild - PBS Pro - PBS Pro Confluence

Building PBSpro on Debian

PBS Professional 12.0 Installation and Upgrade Guide

Etc

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotes :  Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce :  Bernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS :  Programming Languages History : PL/1 : Simula 67 : C : History of GCC development :  Scripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month :  How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the Softpanorama society. We do not warrant the correctness of the information provided or its fitness for any purpose. The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Last modified: March, 12, 2019