Software Distribution

News	Enterprise Unix System Administration	Recommended Links	Unix System Monitoring	Job schedulers	Unix Configuration Management Tools	Perl Admin Tools and Scripts
SSH Usage in Pipes	Password-less SSH login	scp	sftp	Tips	History	Humor	Etc

There are many tools for delivering software to multiple servers. Some good, some not so good. The most basic tools are based on ssh (or rsh). I have tried to provide a few details about some of the in the previous lecture. See also discussion ] Slashdot Automating Unix and Linux Administration (Dec 17, 2003)

Learn to script (Score:4, Interesting)
by holden_t (444907) <holden_t@NOspAm.yahoo.com> on Thursday October 09, @03:09PM (#7175570) Certainly I haven't read the book but it looks as if Kirk is offering examples of how to write scripts to handle everyday gruntwork. Good idea.
But I say to those that call themselves sys.admins, Learn how to script!!!
I work at a large bankrupt telcom :) and it's amazing the amount of admins that don't have the slightest idea how to write the simplest loop. Or use ksh, bash, or csh's cmd history. Or vi.
Maybe this is just a corporate thing. They were raised, in a sense, in a setting where all they had to do was add users and replace disks. Maybe they never learned how to do anything else.
Back in '83 I took manuals home and poured over every page, every weekend for months. That didn't make me a good admin but it gave me a good foundation. From there I had to just halfway use my head (imagination?) and start writing scripts. Ugly? Sure. Did they get better? Of course!
Now I play admin on 110+ machines, and I stay bored. Why? Because I've written a response engine in Expect that handles most of my everyday problems. I call it AGE, Automated Gruntwork Eliminator.
There's no way I could have done this if I had just sat back and floated, not put in a bit of effort to learn new things.
Multiple Machines (Score:5, Interesting)

by BrookHarty (9119) on Thursday October 09, @01:48PM (#7175005)
(http://www.ironwolve.com/)

One of the problems we have, is when you have clusters with 100+ machines, and need to push configs, or gather stats off each box.
On solaris, we run a script called "shout" that does a for/next loop that ssh's into each box and runs a command for us. We also have one called "Scream" which does some root privilege ssh enabled commands.
Nortel has a nice program called CLIManager (use to be called CLImax), that allows you telnet into multiple passports and run commands. Same idea, but the program formats data to display. Say you wanted to display "ipconfig" on 50 machines, this would format it, so you have columns of data, easy to read and put in reports.
Also, has a "Watch" command that will repeat a command, and format the data. Say you want to display counters.
I have not seen an opensource program that does the same as "CliManager" but its has to be one of the best idea's that should be implemented in opensource. Basically, it logs into multiple machines, parses and displays data, and outputs all errors on another window to keep your main screen clean.
Think of logging into 10 machines, and doing a tail -f on an active log file. Then the program would parse the data, display it in a table, and all updates would be highlighted.
I havnt spoken to the author of CliManager, but I guess he also hated logging into multiple machines, and running the same command. This program has been updated over the years, and is now the standard interface to the nodes. It just uses telnet and a command line, but you can log into 100's of nodes at once.
Wish I could post pics and the tgz file, maybe someone from Nortel can comment. (Runs on Solaris, NT and linux) Re:Multiple Machines (Score:2)
by Xzzy (111297) <sether@ t r u 7 h.org> on Thursday October 09, @04:21PM (#7176481)
(http://tru7h.org) > Nortel has a nice program called CLIManager (use
> to be called CLImax), that allows you telnet into
> multiple passports and run commands.
Fermilab has available a tool called rgang that does (minus the output formatting) something like this:
http://fermitools.fnal.gov/abstracts/rgang/abstrac t.html
We use it regularily on a cluster of 176 machines. It's biggest flaw is it tends to hang when one of the machines it encounters is down.
But it is free so I won't complain. :) Multiple Machines in Parallel (Score:1)
by cquark (246669) on Thursday October 09, @04:29PM (#7176572)

One of the problems we have, is when you have clusters with 100+ machines, and need to push configs, or gather stats off each box. On solaris, we run a script called "shout" that does a for/next loop that ssh's into each box and runs a command for us. We also have one called "Scream" which does some root privilege ssh enabled commands.
While the serial approach of looping through machines is a huge improvement over making changes by hand, for large scale environments, you need to use a parallel approach, with 16 processes or so contacting machines in parallel. I wrote my own script, but these days the Parallel::ForkManager [cpan.org] module for perl does the process management part for you. Re:Multiple Machines (Score:2)
by Sevn (12012) on Thursday October 09, @04:57PM (#7176807)
(http://www.dangpow.com/~sevn | Last Journal: Tuesday April 01, @07:18PM) I do pretty much the same thing this way:
Generate ssh key file.
Put pub key file in $HOME/.ssh/authorized_keys2 on the remote machines.
Have a text file with a list of all the names the machines resolve to.
for i in `cat machinelist.txt`; do echo "running blah on $i"; ssh user@$i 'some command I want to run on all machines'; echo " "; done
It comes in handy for stuff like checking the mail queues or doing a tail -50 on a log file. Mundane stuff like that. Everyone once in a while I'll do basically the same thing with scp instead. It can get as complicated as you want. I used a for loop like this to remount 150 /tmp dirs noexec and make the edits to fstab. Re:Multiple Machines (Score:2)
by drinkypoo (153816) <drink@hypeDEBIANrlogos.org minus distro> on Thursday October 09, @10:00PM (#7179637)
(http://slashdot.org/ | Last Journal: Friday November 21, @04:31PM) IBM also owns Tivoli Systems, which made something called TME10, the current name of which escapes me at the moment. TME10 uses CORBA (their ORB is now Java, but it used to be basically ANSI C plus classes, compiled with the microsoft compiler on windows and gcc on most other platforms. Lots of it was perl, some of it was shell, plenty of it was C. Methods called Perl scripts pretty damn frequently. The interface was completely configurable and not only could you customize them without purchasing any additional products (if you felt froggy) but they also sold products to make this easier to do.
Last I checked this package ran with varying degrees of ability (but most operating systems were very well suppored) on all major Commercial Unices, BSDi, Linux, OS/2, NT, Novell, and a bunch of random Unices that most people have never heard of, and never had to. It was sometimes problematic but the fact is that it was incredibly cross-platform.

It was a neat way to do system monitoring. It would be nice to develop something open source like that. I think that today it would not be all that difficult a task. I'd like to see all communications be encrypted, with arbitrary shapes allowed in the network in terms of who talks to who, and who has control over who, to reflect the realities of organizations.
Re:Multiple Machines (Score:0)
by Anonymous Coward on Thursday October 09, @04:14PM (#7176396) IBM has two solutions depending on the environment. PSSP under AIX will allow you to run distrbuted command across nodes with either a correct RSH config or SSH Keys with no passphrase. PSSP, also, allow for parrallel copy. Under Linux( and AIX actually) there is CSM which also allows for DSH with the same config requirements. You can do Parallel copy under CSM, but you have to be tricky with something like, "dsh headnode:/file /file" . Re:Learn to script (Score:2)
by Wolfrider (856) <kingneutron@yahoTOKYOo.com minus city> on Friday October 10, @08:10PM (#7187085)
(http://wolfrdr.tripod.com/linuxtips.html) O'Reilly's book helped me quite a bit.
http://www.oreilly.com/catalog/bash2/
In addition, Debian has a new package called abs-guide that I haven't checked out yet.
http://packages.debian.org/unstable/doc/abs-guide. html
--I've written a bunch of helpful bash scripts to help me with everyday stuff, as well as aliases and functions. If you want, email me - kingneutron at yahoo NOSPAM dot com and put "Request for bash scripts" in the subject line, and I'll send you a tarball. Might be useful... (Score:2)
by Vrallis (33290) on Friday October 10, @12:22AM (#7180451)
(http://krynn.penguinpowered.com) This might very well be a book I'll pick up sometime. I'm always looking for more ideas.
I maintain about ~170 remote Linux boxes (in our company's retail stores and warehouses), as well as our ~30 or so inhouse servers.

I went through a lot of work to enable our rollout and conversion to go more smoothly. The network and methodology for users, printers, etc. is extremely simplified and patterened.

For each of the 3 'models' of PCs we use, I have a master system that I produced. I used Mondo Rescue [mondorescue.com] to produce CD backups of these systems. These systems act as serial terminal controllers, print spoolers, routers, desktop system usage (OpenOffice, Mozilla, Kmail under KDE), and other functions as needed.

When we need to replace a system, or rollout a new location, we grab a system, pop in the Mondo CD, and do a nuke restore. When done, we have a standard configuration user that we log in as. It runs a quick implementation script where you answer anywhere from 3-8 questions (depending on the system type and options), and it configures everything. All networking, users, sets up Kmail, configures all printers and terminals (we use Comtrol Rocketport serial boards), and so on.

If the system is physically ready, we can have it ready software-wise in about 20 minutes (2 CDs to restore).

Updates are done via a couple different methods. I use SSH (over our internal VPN, using key authentication) in scripts to do most updates. If I need to do anything major, such as recently updating Mozilla, we do a CD distribution. The users have a simple menu to take care of running the update for them, even with autorun under KDE. Just pop in the CD, and it automatically takes them into the menu they need.

All logs are duplicated across the network to a central server, but intrusion is less likely as these systems sit on a private frame network. They do, however, have fully secured network setups, as we use cheap dial-up internet access as a backup in case the frame circuit goes down.

I can't help but feel every day like this is just one big hack/kludge, but it works, works damned well, and was about half the cost of any other solution (i.e. higher end Cisco routers to handle various functions, and using Equinox ELS-IIs or the like...those pieces of crap never would work right, we finally pulled only 2 we had in use, and they are currently collecting dust in a storage cabinet).

Needless to say, I am *always* looking for ideas to improve upon this.

Dr. Nikolai Bezroukov

Notes:

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Some amount of grammar and spelling errors should be expected.
The site contain some broken links as it develops like a living tree... Please try to use Google, Open directory, etc. to find a replacement link (see HOWTO search the WEB for details). We would appreciate if you can mail us a correct link.

Google Search
Open directory	Research Index

Old News ;-)

[May 16, 2009] makeself

makeself is a small shell script that generates a self-extractable compressed TAR archive from a directory. The resulting file appears as a shell script, and can be launched as is. The archive will then uncompress itself to a temporary directory and an arbitrary command will be executed (for example, an installation script).

This is pretty similar to archives generated with WinZip Self-Extractor in the Windows world.

[Apr 2, 2009] Spacewalk

freshmeat.net

Spacewalk is a Linux and Solaris systems management solution. It allows you to inventory your systems (hardware and software information), install and update software on your systems, collect and distribute your custom software packages into manageable groups, provision (Kickstart) your systems, manage and deploy configuration files to your systems, monitor your systems, provision virtual guests, and start/stop/configure virtual guests.

[Mar 10, 2009] Cluster SSH

Perl-based

freshmeat.net

Cluster SSH opens terminal windows with connections to specified hosts and an administration console. Any text typed into the administration console is replicated to all other connected and active windows. This tool is intended for, but not limited to, cluster administration where the same configuration or commands must be run on each node within the cluster. Performing these commands all at once via this tool ensures all nodes are kept in sync.

[Dec 14, 2008] Cool Solutions Using ClusterSSH to Perform Tasks on Multiple Servers Simultaneously By Martijn Pepping

Problem:
As an administrator of SLES/OES Linux clusters or multiple SUSE Linux servers you are probably familiar with that fact that you have to make an identical change on more than one server. Those can be things like editing files, execute commands, collect data or some other administrative task.

There are a couple of way to do this. You can write a script that performs the change for you, or you can SSH into a server, make the change and repeat that task manually for every server.

Now both ways can cost an extended amount of time. Writing and testing a shell script takes some time and performing the task by hand on lets say five or more servers also costs time.

Now, wouldn't it be a real timesaver when you have only one console in which you can perform tasks on multiple servers simultaneously? This solution can be found in ClusterSSH.

Solution:

With ClusterSSH it is possible to make a SSH connection to multiple servers and perform tasks from one single command window, without any scripting. The 'cssh' command lets you connect to any server specified as a command line argument, or to groups of servers (or cluster nodes) defined in a configuration file.

The 'cssh' command opens a terminal window to every server which can be used to review the output sent from the cssh-console, or to edit a single host directly. Commands given in to the cssh-console are executed on every connected host. When you start typing in the cssh-console you'll see that the same command also show up on the commandline of the connected systems.

The state of connected systems can be toggled from the cssh-console. So if you want to exclude certain hosts temporarily from specific command, you can do this with a single mouseclick. Also, hosts can be added on the fly and open terminal windows can automatically be rearranged.

One caveat to be aware of is when editing files. Never assume that file is identical on all systems. For example, lines in a file you are editing may be in a different order. Don't just go to a certain line in a file and start editing. Instead search for the text you want to exit, just to be sure the correct text is edited on all connected systems.

Example:

Configuration files section from the man-page:
/etc/clusters
This file contains a list of tags to server names mappings. When any name is used on the command line it is checked to see if it is a tag in /etc/clusters (or the .csshrc file, or any additional cluster file specified by -c). If it is a tag, then the tag is replaced with the list of servers from the file. The file is formatted as follows:
<tag> [user@]<server> [user@]<server> [...]
i.e.
# List of servers in live

live admin1@server1 admin2@server2 server3 server4
Clusters may also be specified within the users .csshrc file, as documented below.
/etc/csshrc & $HOME/.csshrc
This file contains configuration overrides - the defaults are as marked. Default options are overwritten first by the global file, and then by the user file.

Environment:

ClusterSSH can be used to any system running the SSH daemon.

ClusterSSH RPM for openSUSE 10.2/10.3 and SLE10 are available through the openSUSE Build-service: http://download.opensuse.org/repositories/home:/martijn/

The ClusterSSH sourcecode can be downloaded from: http://sourceforge.net/project/showfiles.php?group_id=89139

[Aug 25, 2008] pssh 1.4.0 by Brent N. Chun -

About: pssh provides parallel versions of the OpenSSH tools that are useful for controlling large numbers of machines simultaneously. It includes parallel versions of ssh, scp, and rsync, as well as a parallel kill command.

Changes: A 64-bit bug was fixed: select now uses None when there is no timeout rather than sys.maxint. EINTR is caught on select, read, and write calls. Longopts were fixed for pnuke, prsync, pscp, pslurp, and pssh. Missing environment variables options support was added.

alphaWorks Remote System Management Tool Overview

Remote Server Management Tool is an Eclipse plug-in that provides an integrated graphical user interface (GUI) environment and enables testers to manage multiple remote servers simultaneously. The tool is designed as a management tool for those who would otherwise telnet to more than one server to manage the servers and who must look at different docs and man pages to find commands for different platforms in order to create or manage users and groups and to initiate and monitor processes. This tool handles these operations on remote servers by using a user-friendly GUI; in addition, it displays configuration of the test server (number of processors, RAM, etc.). The activities that can be managed by this tool on the remote and local server are divided as follows:

Process Management: This utility lists the process running on UNIX and Windows® servers. One can start and stop processes. Along with process listing, the utility also provides details of the resources used by the process.

User Management: This utility facilitates creation of users and groups on UNIX servers; it also provides options for listing, creating, deleting, and modifying the attributes of users and groups.

File Management: This utility acts as a windows explorer for any selected server, irrespective of its operating system. One can create, edit, delete, and copy files and directories on local or remote servers. Testers can tail the remote files.

How does it work?
This Eclipse plug-in was written with the Standard Widget Toolkit (SWT). The tool has a perspective named Remote System Management; the perspective consists of test servers and a console view. The remote test servers are mounted in the Test Servers view for management of their resources (process, file system, and users or groups).
At the back end, this Eclipse plug-in uses the Software Test Automation Framework (STAF). STAF is an open-source framework that masks the operating system-specific details and provides common services and APIs in order to manage system resources. The APIs are provided for a majority of the languages. Along with the built-in services, STAF also supports external services. The Remote Server Management Tool comes with two STAF external services: one for user management and another for proving system details.

radmind

radmind is a suite of Unix command-line tools and a server designed to remotely administer the file systems of multiple Unix machines. At its core, radmind operates as a tripwire. It is able to detect changes to any managed filesystem object, e.g. files, directories, links, etc. However, radmind goes further than just integrity checking: once a change is detected, radmind can optionally reverse the change. Each managed machine may have its own loadset composed of multiple, layered overloads. This allows, for example, the operating system to be described separately from applications. Loadsets are stored on a remote server. By updating a loadset on the server, changes can be pushed to managed machines.

spill

spill manages symbolic links under one tree which point to matching filenames in another. When individual projects are configured with project/version-specific -- prefix= settings, to keep their installations segregated, spill can make them appear to be installed in a common place, e.g. under /usr/local. It can also delete the links associated with a particular program. It is similar in concept to various other programs such as stow, depot, and relink. However, it's written in C, so it isn't reliant on an interpreter being available. It also doesn't assume complete control of the directory tree where the symbolic links are created. It can create both absolute or relative symbolic links, the latter being more convenient in some setups.

RDist

Rdist is a program to maintain identical copies of files over multiple hosts. It preserves the owner, group, mode, and mtime of files if possible and can update programs that are executing. relink
relink is a package management tool for the organization and management of software packages. It should run on any Unix platform that runs Perl. It is similar to tools such as RPM (Red Hat/Mandrake), pkgadd (Slackware/Sun), stow(GNU), and depot(CMU).

[Nov 5, 2006] Shell Scripts for execution on multiple servers

Fanout Runs commands on remote machines via ssh.

Fanout allows you to run non-interactive commands on remote machines simultaneously, collecting the output in an organized fashion.
AScopy A site maintenance utility.

AScopy is a site maintenance utility. Its main purpose is to be portable and usable across any platform that has a working Bash shell and SSH.

[Nov 5, 2006] Perl Scripts for execution on multiple servers

Cluster SSH A cluster administration tool.
cssh Perl tool to run multiple commands to multiple hosts from a central shell over SSH.
mass.pl A program for remote administration of many machines.
ghosts Parallel remote host shell tool written in Perl

[Nov 5, 2006] Python Scripts for execution on multiple servers

The Distribulator Python-based, cluster-aware, SSH-based command execution and file transfer utility.
OctopuSsh An application for cluster administration via SSH.
PuSSH A python wrapper for the SSH client that issues commands to multiple machines.
Pydsh Python-based remote administration tool for managing multiple servers.
pssh Parallel versions of the OpenSSH tools.