Softpanorama

May the source be with you, but remember the KISS principle ;-)
Contents Bulletin Scripting in shell and Perl Network troubleshooting History Humor

PSSH

News Parallel command execution Recommended Links Unix Configuration Management Tools pssh Slurping Perl Admin Tools and Scripts parallel
Job schedulers Unix System Monitoring            
rsync Cluster SSH   Mussh dsh SSH Power Tool Tentakel Multi Remote Tools
SSH Usage in Pipes Password-less SSH login scp sftp Tips History Humor Etc
PSSH provides parallel versions of OpenSSH and related tools. Included are pssh, pscp, prsync, pnuke, and pslurp. The project includes psshlib which can be used within custom applications. The source code is written in Python and can be cloned from:

git clone http://code.google.com/p/parallel-ssh/

PSSH is supported on Python 2.4 and greater (including Python 3.1 and greater). It was originally written and maintained by Brent N. Chun. Due to his busy schedule, Brent handed over maintenance to Andrew McNabb in October 2009.

Linux.com Parallel SSH execution and a single shell to control them all

Many people use SSH to log in to remote machines, copy files around, and perform general system administration. If you want to increase your productivity with SSH, you can try a tool that lets you run commands on more than one remote machine at the same time. Parallel ssh, Cluster SSH, and ClusterIt let you specify commands in a single terminal window and send them to a collection of remote machines where they can be executed.

Why you would need a utility like this when, using openSSH, you can create a file containing your commands and use a bash for loop to run it on a list of remote hosts, one at a time? One advantage of a parallel SSH utility is that commands can be run on several hosts at the same time. For a short-running task this might not matter much, but if a task needs an hour to complete and you need to run it on 20 hosts, parallel execution beats serial by a mile. Also, if you want to interactively edit the same file on multiple machines, it might be quicker to use a parallel SSH utility and edit the file on all nodes with vi rather than concoct a script to do the same edit.

Many of these parallel SSH tools include support for copying to many hosts at once (a parallel version of scp) or using rsync on a collection of hosts at once. Because the parallel SSH implementations know about all the hosts in a group, some of them also offer the ability to execute a command "on one host" and will work out which host to pick using load balancing. Finally, some parallel SSH projects let you use barriers so that you can execute a collection of commands and explicitly have each node in the group wait until all the nodes have completed a stage before moving on to the next stage of processing.

Parallel ssh (pssh)

The Parallel ssh project includes parallel versions of the shell (pssh), scp (pscp), rsync (prsync), and kill (pnuke).

pssh is packaged for openSUSE as a 1-Click install, is available in Ubuntu Hardy Universe and the Fedora 9 repositories. I used the 64-bit package from the Fedora 9 repositories.

All of the Parallel ssh commands have the form command -h hosts-file options, where the hosts-file contains a list of all the hosts that you want to have the command executed on. For example, the first pssh command below will execute the date command on p1 and p2 as the ben user. The optional -l argument specifies the username that should be used to log in to the remote machines.

 
# cat hosts-file p1 p2 # pssh -h hosts-file -l ben date [1] 21:12:55 [SUCCESS] p2 22 [2] 21:12:55 [SUCCESS] p1 22 # pssh -h hosts-file -l ben -P date p2: Thu Oct 16 21:14:02 EST 2008 p2: [1] 21:13:00 [SUCCESS] p2 22 p1: Thu Sep 25 15:44:36 EST 2008 p1: [2] 21:13:00 [SUCCESS] p1 22
 

Normally the standard output from the remote hosts is not shown to you. The -P option in the last invocation displays the output from both remote hosts as well as the exit status. If you are running more complex commands you might like to use -i instead to see each remote host's output grouped nicely under its hostname rather than mixed up as the output comes in from the hosts. You can also use the --outdir pssh option to specify the path of a directory that should be used to save the output from each remote host. The output for each host is saved in separate file named with the remote machine's hostname.

You can use the --timeout option to specify how long a command can take. It defaults to 60 seconds. This means that if your command fails to complete within 60 seconds on a host, pssh will consider it an error and report it as such, as shown below. You can increase the timeout to something well above what might be acceptable (for example to 24 hours) to avoid this problem.

 
# pssh -h hosts-file -l ben -i "sleep 65; date" [1] 21:19:26 [FAILURE] p1 22 Timeout [2] 21:19:26 [FAILURE] p2 22 (4, 'Interrupted system call')
 

The pscp command takes the same -h, -l, and --timeout options and includes a --recursive option to enable deep copying from the local host. At the end of the command you supply the local and remote paths you would like to copy. The first pscp command in the example below copies a single file to two remote hosts in parallel. The following ssh command checks that the file exists on the p1 machine. The second pscp command fails in a verbose manner without really telling you the simple reason why. Knowing that I was trying to copy a directory over, I added the --recursive option to the command and it executed perfectly. The final ssh command verifies that the directory now exists on the p1 remote host.

 
$ mkdir example-tree $ date > example-tree/df1.txt $ date > example-tree/df2.txt $ mkdir example-tree/subdir1 $ date > example-tree/subdir1/df3.txt $ pscp -h hosts-file -l ben example-tree/df1.txt /tmp/df1.txt [1] 21:28:36 [SUCCESS] p1 22 [2] 21:28:36 [SUCCESS] p2 22 $ ssh p1 "cat /tmp/df1.txt" Thu Oct 16 21:27:25 EST 2008 $ pscp -h hosts-file -l ben example-tree /tmp/example-tree ... python: Python/ceval.c:2918: set_exc_info: Assertion `frame != ((void *)0)' failed. Aborted $ pscp -h hosts-file -l ben --recursive example-tree /tmp/example-tree [1] 21:29:57 [SUCCESS] p1 22 [2] 21:29:57 [SUCCESS] p2 22 $ ssh p1 "ls -l /tmp/example-tree" total 24 -rw-r--r-- 1 ben ben 29 2008-09-25 16:01 df1.txt -rw-r--r-- 1 ben ben 29 2008-09-25 16:01 df2.txt drwxr-xr-x 2 ben ben 4096 2008-09-25 16:01 subdir1
 

The prsync command uses only a handful of the command-line options from rsync. In particular, you cannot use the verbose or dry-run options to get details or see what would have been done. The command shown below will rsync the example-tree into /tmp/example-tree on the remote hosts in a manner similar to the final command in the pscp example.

 
$ prsync -h hosts-file -l ben -a --recursive example-tree /tmp
 

The main gain of the prsync command over using the normal rsync command with pssh is that prsync gives a simpler command line and lets you sync from the local machine to the remote hosts directly. Using pssh and rsync, you are running the rsync command on each remote machine, so the remote machine will need to connect back to the local machine in order to sync.

The pslurp command is sort of the opposite to the pscp in that it grabs a file or directory off all the remote machines and copies it to the local machine. The below command grabs the example-tree directory from both p1 and p2 and stores them into /tmp/outdir. The -r option is shorthand for --recursive. As you can see, for each remote host a new directory is created with the name of the host, and inside that directory a copy of example-tree is made using the local directory name supplied as the last argument to pslurp.

 
# mkdir /tmp/outdir # pslurp -h hosts-file -L /tmp/outdir -l ben -r /tmp/example-tree example-tree # l /tmp/outdir drwxr-xr-x 3 root root 4.0K 2008-10-16 21:47 p1/ drwxr-xr-x 3 root root 4.0K 2008-10-16 21:47 p2/ # l /tmp/outdir/p2 drwxr-xr-x 3 root root 4.0K 2008-10-16 21:47 example-tree/ # l /tmp/outdir/p2/example-tree/ -rw-r--r-- 1 root root 29 2008-10-16 21:47 df10.txt -rw-r--r-- 1 root root 29 2008-10-16 21:47 df1.txt ... drwxr-xr-x 2 root root 4.0K 2008-10-16 21:47 subdir1/
 

You can use environment variables to make things easier with Parallel ssh. You can use the PSSH_HOSTS variable to name the hosts file instead of using the -h option. Likewise, the PSSH_USER environment variable lets you set the username to log in as, like the -l pssh command line option.

 

pssh provides a number of commands for executing against a group of computers, using SSH. It’s most useful for operating on clusters of homogenously-configured hosts.

The package contains:

These tools are good for controlling large collections of nodes, where faster alternatives

pssh Syntax

pssh command -h hosts-file options

-h –hosts   hosts file (each line “host[:port] [user]“)
-l –user    username (OPTIONAL)
-p –par     max number of parallel threads (OPTIONAL)
-o –outdir  output directory for stdout files (OPTIONAL)
-t –timeout timeout in seconds to do ssh to a host (OPTIONAL)
-v –verbose turn on warning and diagnostic messages (OPTIONAL)
-O –options SSH options (OPTIONAL)

where the hosts-file contains a list of all the hosts that you want to have the command executed on.

pssh Examples

The following example runs hostname on three machines (IPs or hostnames) specified in the file ips.txt using login irb2 and saves the output in /tmp/foo.

sudo cat ips.txt
128.112.152.122
18.31.0.190
128.232.103.201

sudo pssh -h ips.txt -l irb2 -o /tmp/foo hostname

Success on 128.112.152.122:22
Success on 18.31.0.190:22
Success on 128.232.103.201:22

sudo ls /tmp/foo

128.112.152.122  128.232.103.201  18.31.0.190

sudo cat /tmp/foo/*

planetlab-1.cs.princeton.edu
planetlab1.xeno.cl.cam.ac.uk
planetlab1.lcs.mit.edu

By default, pssh uses at most 32 ssh processes in parallel to ssh to the various nodes. (This is somewhat important if you’re controlling hundreds or thousands of machines.) By default, it also uses a timeout of one minute to ssh to a node and obtain a result. For ssh commands that take longer than this (e.g., sleep 61), the -t option can be used. Note that pssh and pnuke have a default timeout of one minute. pscp and prsync have no default timeout, but one can be specified using the -t option.

pscp

Here’s an example of using pscp to copy files in parallel to a set of machines.

sudo  pscp -h ips.txt -l irb2 /etc/hosts /tmp/hosts

Success on 128.112.152.122:22
Success on 18.31.0.190:22
Success on 128.232.103.201:22

Using the -r option will perform a recursive copy for copying entire directories.


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

[Dec 18, 2014] Execute commands simultaneously on multiple servers Using PSSH/Cluster SSH/Multixterm

[Jul 06, 2010] What is a good modern parallel SSH tool - Server Fault

Q. I have heard that pssh and clusterssh are two popular ones, but I thought I would open it to discussion here and see what the community's experiences with these tools were? What are the gotchas? Any decent hacks or use cases?

A: I have used pssh and it's easy and works quite well. It's really great for quick queries.

If you find yourself managing servers I'd suggest something more robust and in a slightly different realm (configuration management) such as Puppet or CFEngine.

There is also dsh for parallel ssh runs.

Mussh is a good alternative, it is already included in many Linux distros.

Mussh is a shell script that allows you to execute a command or script over ssh on multiple hosts with one command. When possible mussh will use ssh-agent and RSA/DSA keys to minimize the need to enter your password more than once.

The SSH Power Tool (sshpt) was designed for parallel SSH without requiring that the user setup pre-shared SSH keys. It is superior to pssh and clusterssh in that it supports executions via sudo and can also copy files and execute them afterwards (optionally, via sudo as well). By default it outputs results in CSV format but sshpt.py doubles as an importable Python module so you can use it in your own programs (I used to use it as a back-end behind a a custom-built web-based reporting tool at my former employer).

[Jul 05, 2010] parallel-ssh - Project Hosting on Google Code

Python-based. More sophisticated then cluster ssh which is Perl-based...

PSSH provides parallel versions of OpenSSH and related tools. Included are pssh, pscp, prsync, pnuke, and pslurp. The project includes psshlib which can be used within custom applications. The source code is written in Python and can be cloned from:

git clone git://aml.cs.byu.edu/pssh.git

PSSH was originally written and maintained by Brent N. Chun. Due to his busy schedule, Brent handed over maintenance to Andrew McNabb in October 2009.

Save time managing multiple systems with Parallel SSH Linux and Open Source

TechRepublic.com

OpenSSH is perhaps one of the most powerful and versatile tools available to any Linux user. It allows you to securely connect to a remote system via a shell or encrypted FTP and also allows you to copy files securely to and from remote systems.

For a user caring for multiple systems, OpenSSH is extremely useful, but being able to execute OpenSSH commands in parallel is even more so. This is where Parallel SSH, or pssh, comes in. Pssh provides parallel versions of the OpenSSH tools, meaning you can execute commands on various hosts in parallel, copy files in parallel, and so forth. Pssh is essentially a frontend to OpenSSH written in Python. It includes pssh, pscp, and prsync, as well as pslurp (the opposite of pscp in that it downloads rather than uploads) and pnuke (a frontend to the kill command).

Using pssh is extremely easy. There are no manpages, but calling the command with no arguments will bring up the help, which describes each option.

Every command uses a plaintext “hosts” file that is a simple text file containing the hosts to act upon, one per line. As an example, assume you wanted to make sure that the time on each server was identical. This could be done using the date command, but to do it with regular SSH, you would have to execute the command at the same time on each host using screen or multiple terminals. With pssh, this is one simple command:

$ pssh -h hosts -P date
hades: Wed Nov 12 10:21:11 MST 2008
hades: [1] 10:21:11 [SUCCESS] hades 22
odin: Wed Nov 12 10:21:11 MST 2008
odin: [2] 10:21:11 [SUCCESS] odin 22
$ cat hosts
hades
odin

Contrast that to using ssh directly:

$ for host in hades odin; do ssh ${host} "date"; done
Wed Nov 12 10:24:02 MST 2008
Wed Nov 12 10:24:02 MST 2008

Recommended Links



Etc

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available in our efforts to advance understanding of environmental, political, human rights, economic, democracy, scientific, and social justice issues, etc. We believe this constitutes a 'fair use' of any such copyrighted material as provided for in section 107 of the US Copyright Law. In accordance with Title 17 U.S.C. Section 107, the material on this site is distributed without profit exclusivly for research and educational purposes.   If you wish to use copyrighted material from this site for purposes of your own that go beyond 'fair use', you must obtain permission from the copyright owner. 

ABUSE: IPs or network segments from which we detect a stream of probes might be blocked for no less then 90 days. Multiple types of probes increase this period.  

Society

Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers :   Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism  : The Iron Law of Oligarchy : Libertarian Philosophy

Quotes

War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda  : SE quotes : Language Design and Programming Quotes : Random IT-related quotesSomerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose BierceBernard Shaw : Mark Twain Quotes

Bulletin:

Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 :  Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method  : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law

History:

Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds  : Larry Wall  : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOSProgramming Languages History : PL/1 : Simula 67 : C : History of GCC developmentScripting Languages : Perl history   : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history

Classic books:

The Peter Principle : Parkinson Law : 1984 : The Mythical Man-MonthHow to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite

Most popular humor pages:

Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor

The Last but not Least


Copyright © 1996-2016 by Dr. Nikolai Bezroukov. www.softpanorama.org was created as a service to the UN Sustainable Development Networking Programme (SDNP) in the author free time. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License.

The site uses AdSense so you need to be aware of Google privacy policy. You you do not want to be tracked by Google please disable Javascript for this site. This site is perfectly usable without Javascript.

Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.

FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.

This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...

You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info

Disclaimer:

The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.

Last modified: February 16, 2017