Softpanorama

Home Switchboard Unix Administration Red Hat TCP/IP Networks Neoliberalism Toxic Managers
May the source be with you, but remember the KISS principle ;-)

Slightly Skeptical View on Enterprise Unix Administration

News Webliography of problems with "pure" cloud environment Recommended Books Recommended Links Recommended Tools to Enhance Command Line Usage in Windows Programmable Keyboards Microsoft IntelliType Macros
Unix Configuration Management Tools Job schedulers Unix System Monitoring Over 50 and unemployed Corporate bullshit as a communication method Diplomatic Communication Bosos or Empty Suits (Aggressive Incompetent Managers)
ILO command line interface Using HP ILO virtual CDROM iDRAC7 goes unresponsive - can't connect to iDRAC7 Resetting frozen iDRAC without unplugging the server Troubleshooting HPOM agents Webliography of problems with "pure" cloud environment The tar pit of Red Hat overcomplexity
Bare metal recovery of Linux systems Shadow IT Is DevOps a yet another "for profit" technocult Carpal tunnel syndrome Sysadmin Horror Stories Humor Etc


The KISS rule can be expanded as: Keep It Simple, Sysadmin ;-)

This page is written as a pretest against overcomplexity and bizarre data center atmosphere dominant in "semi-outsourced" or fully outsourced datacenters ;-). Unix/Linux sysadmins are being killed with overcomplexity of the environment. As Charlie Schluting noted in 2010: (Enterprise Networking Plane, April 7, 2010)

What happened to the old "sysadmin" of just a few years ago? We've split what used to be the sysadmin into application teams, server teams, storage teams, and network teams. There were often at least a few people, the holders of knowledge, who knew how everything worked, and I mean everything. Every application, every piece of network gear, and how every server was configured -- these people could save a business in times of disaster.

Now look at what we've done. Knowledge is so decentralized we must invent new roles to act as liaisons between all the IT groups.

Architects now hold much of the high-level "how it works" knowledge, but without knowing how any one piece actually does work.

In organizations with more than a few hundred IT staff and developers, it becomes nearly impossible for one person to do and know everything. This movement toward specializing in individual areas seems almost natural. That, however, does not provide a free ticket for people to turn a blind eye.

Specialization

You know the story: Company installs new application, nobody understands it yet, so an expert is hired. Often, the person with a certification in using the new application only really knows how to run that application. Perhaps they aren't interested in learning anything else, because their skill is in high demand right now. And besides, everything else in the infrastructure is run by people who specialize in those elements. Everything is taken care of.

Except, how do these teams communicate when changes need to take place? Are the storage administrators teaching the Windows administrators about storage multipathing; or worse logging in and setting it up because it's faster for the storage gurus to do it themselves? A fundamental level of knowledge is often lacking, which makes it very difficult for teams to brainstorm about new ways evolve IT services. The business environment has made it OK for IT staffers to specialize and only learn one thing.

If you hire someone certified in the application, operating system, or network vendor you use, that is precisely what you get. Certifications may be a nice filter to quickly identify who has direct knowledge in the area you're hiring for, but often they indicate specialization or compensation for lack of experience.

Resource Competition

Does your IT department function as a unit? Even 20-person IT shops have turf wars, so the answer is very likely, "no." As teams are split into more and more distinct operating units, grouping occurs. One IT budget gets split between all these groups. Often each group will have a manager who pitches his needs to upper management in hopes they will realize how important the team is.

The "us vs. them" mentality manifests itself at all levels, and it's reinforced by management having to define each team's worth in the form of a budget. One strategy is to illustrate a doomsday scenario. If you paint a bleak enough picture, you may get more funding. Only if you are careful enough to illustrate the failings are due to lack of capital resources, not management or people. A manager of another group may explain that they are not receiving the correct level of service, so they need to duplicate the efforts of another group and just implement something themselves. On and on, the arguments continue.

Most often, I've seen competition between server groups result in horribly inefficient uses of hardware. For example, what happens in your organization when one team needs more server hardware? Assume that another team has five unused servers sitting in a blade chassis. Does the answer change? No, it does not. Even in test environments, sharing doesn't often happen between IT groups.

With virtualization, some aspects of resource competition get better and some remain the same. When first implemented, most groups will be running their own type of virtualization for their platform. The next step, I've most often seen, is for test servers to get virtualized. If a new group is formed to manage the virtualization infrastructure, virtual machines can be allocated to various application and server teams from a central pool and everyone is now sharing. Or, they begin sharing and then demand their own physical hardware to be isolated from others' resource hungry utilization. This is nonetheless a step in the right direction. Auto migration and guaranteed resource policies can go a long way toward making shared infrastructure, even between competing groups, a viable option.

Blamestorming

The most damaging side effect of splitting into too many distinct IT groups is the reinforcement of an "us versus them" mentality. Aside from the notion that specialization creates a lack of knowledge, blamestorming is what this article is really about. When a project is delayed, it is all too easy to blame another group. The SAN people didn't allocate storage on time, so another team was delayed. That is the timeline of the project, so all work halted until that hiccup was restored. Having someone else to blame when things get delayed makes it all too easy to simply stop working for a while.

More related to the initial points at the beginning of this article, perhaps, is the blamestorm that happens after a system outage.

Say an ERP system becomes unresponsive a few times throughout the day. The application team says it's just slowing down, and they don't know why. The network team says everything is fine. The server team says the application is "blocking on IO," which means it's a SAN issue. The SAN team say there is nothing wrong, and other applications on the same devices are fine. You've ran through nearly every team, but without an answer still. The SAN people don't have access to the application servers to help diagnose the problem. The server team doesn't even know how the application runs.

See the problem? Specialized teams are distinct and by nature adversarial. Specialized staffers often relegate themselves into a niche knowing that as long as they continue working at large enough companies, "someone else" will take care of all the other pieces.

I unfortunately don't have an answer to this problem. Maybe rotating employees between departments will help. They gain knowledge and also get to know other people, which should lessen the propensity to view them as outsiders

Additional useful material on the topic can also be found in my older article Solaris vs Linux:

Abstract

Introduction

Nine factors framework for comparison of two flavors of Unix in a large enterprise environment

Four major areas of Linux and Solaris deployment

Comparison of internal architecture and key subsystems

Security

Hardware: SPARC vs. X86

Development environment

Solaris as a cultural phenomenon

Using Solaris-Linux enterprise mix as the least toxic Unix mix available

Conclusions

Acknowledgements

Webliography

Here are my notes/reflection of sysadmin problem in strange (and typically pretty toxic) IT departments of large corporations:

 


Top Visited
Switchboard
Latest
Past week
Past month

NEWS CONTENTS

Old News ;-)

2018 2017 2016 2015 2014 2013 2012 2011 2010 2009
2008 2007 2006 2005 2004 2003 2002 2001 2000 1999

"I appreciate Woody Allen's humor because one of my safety valves is an appreciation for life's absurdities. His message is that life isn't a funeral march to the grave. It's a polka."

-- Dennis Kusinich

[Apr 22, 2018] Happy Sysadmin Appreciation Day 2016 Opensource.com

Apr 22, 2018 | opensource.com

Necessity is frequently the mother of invention. I knew very little about BASH scripting but that was about to change rapidly. Working with the existing script and using online help forums, search engines, and some printed documentation, I setup Linux network attached storage computer running on Fedora Core. I learned how to create an SSH keypair and configure that along with rsync to move the backup file from the email server to the storage server. That worked well for a few days until I noticed that the storage servers disk space was rapidly disappearing. What was I going to do?

That's when I learned more about Bash scripting. I modified my rsync command to delete backed up files older than ten days. In both cases I learned that a little knowledge can be a dangerous thing but in each case my experience and confidence as Linux user and system administrator grew and due to that I functioned as a resource for other. On the plus side, we soon realized that the disk to disk backup system was superior to tape when it came to restoring email files. In the long run it was a win but there was a lot of uncertainty and anxiety along the way.

[Apr 22, 2018] Unix Horror story script question Unix Linux Forums Shell Programming and Scripting

Apr 22, 2018 | www.unix.com

scottsiddharth Registered User

Unix Horror story script question


This text and script is borrowed from the "Unix Horror Stories" document.

It states as follows

"""""Management told us to email a security notice to every user on the our system (at that time, around 3000 users). A certain novice administrator on our system wanted to do it, so I instructed them to extract a list of users from /etc/passwd, write a simple shell loop to do the job, and throw it in the background.
Here's what they wrote (bourne shell)...

for USER in `cat user.list`;
do mail $USER <message.text &
done

Have you ever seen a load average of over 300 ??? """" END

My question is this- What is wrong with the script above? Why did it find a place in the Horror stories? It worked well when I tried it.

Maybe he intended to throw the whole script in the background and not just the Mail part. But even so it works just as well... So?

Thunderbolt

RE:Unix Horror story script question
I think, it does well deserve to be placed Horror stories.

Consider the given server for with or without SMTP service role, this script tries to process 3000 mail commands in parallel to send the text to it's 3000 repective receipents.

Have you ever tried with valid 3000 e-mail IDs, you can feel the heat of CPU (sar 1 100)

P.S.: I did not tested it but theoretically affirmed.

Best Regards.

Thunderbolt, View Public Profile 3 11-24-2008 - Original Discussion by scottsiddharth

Quote:

Originally Posted by scottsiddharth

Thank you for the reply. But isn't that exactly what the real admin asked the novice admin to do.

Is there a better script or solution ?

Well, Let me try to make it sequential to reduce the CPU load, but it will take no. of users*SLP_INT(default=1) seconds to execute....

#Interval between concurrent mail commands excution in seconds, minimum 1 second.

      SLP_INT=1
      for USER in `cat user.list`;
      do; 
         mail $USER <message.text; [ -z "${SLP_INT}" ] && sleep 1 || sleep ${SLP_INT}" ;
      done

[Apr 22, 2018] THE classic Unix horror story programming

Looks like not much changed since 1986. I amazed how little changed with Unix over the years. RM remains a danger although zsh and -I option on gnu rm are improvement. . I think every sysadmin wiped out important data with rm at least once in his career. So more work on this problem is needed.
Notable quotes:
"... Because we are creatures of habit. If you ALWAYS have to type 'yes' for every single deletion, it will become habitual, and you will start doing it without conscious thought. ..."
"... Amazing what kind of damage you can recover from given enough motivation. ..."
"... " in "rm -rf ~/ ..."
Apr 22, 2008 | www.reddit.com

probablycorey 10 years ago (35 children)

A little trick I use to ensure I never delete the root or home dir... Put a file called -i in / and ~

If you ever call rm -rf *, -i (the request confirmation option) will be the first path expanded. So your command becomes...

rm -rf -i

Catastrophe Averted!

mshade 10 years ago (0 children)
That's a pretty good trick! Unfortunately it doesn't work if you specify the path of course, but will keep you from doing it with a PWD of ~ or /.

Thanks!

aythun 10 years ago (2 children)
Or just use zsh. It's awesome in every possible way.
brian@xenon:~/tmp/test% rm -rf *
zsh: sure you want to delete all the files in /home/brian/tmp/test [yn]?
rex5249 10 years ago (1 child)
I keep an daily clone of my laptop and I usually do some backups in the middle of the day, so if I lose a disk it isn't a big deal other than the time wasted copying files.
MyrddinE 10 years ago (1 child)
Because we are creatures of habit. If you ALWAYS have to type 'yes' for every single deletion, it will become habitual, and you will start doing it without conscious thought.

Warnings must only pop up when there is actual danger, or you will become acclimated to, and cease to see, the warning.

This is exactly the problem with Windows Vista, and why so many people harbor such ill-will towards its 'security' system.

zakk 10 years ago (3 children)
and if I want to delete that file?!? ;-)
alanpost 10 years ago (0 children)
I use the same trick, so either of:

$ rm -- -i

or

$ rm ./-i

will work.

emag 10 years ago (0 children)
rm /-i ~/-i
nasorenga 10 years ago * (2 children)
The part that made me the most nostalgic was his email address: mcvax!ukc!man.cs.ux!miw

Gee whiz, those were the days... (Edit: typo)

floweryleatherboy 10 years ago (6 children)
One of my engineering managers wiped out an important server with rm -rf. Later it turned out he had a giant stock of kiddy porn on company servers.
monstermunch 10 years ago (16 children)
Whenever I use rm -rf, I always make sure to type the full path name in (never just use *) and put the -rf at the end, not after the rm. This means you don't have to worry about hitting "enter" in the middle of typing the path name (it won't delete the directory because the -rf is at the end) and you don't have to worry as much about data deletion from accidentally copy/pasting the command somewhere with middle click or if you redo the command while looking in your bash history.

Hmm, couldn't you alias "rm -rf" to mv the directory/files to a temp directory to be on the safe side?

branston 10 years ago (8 children)
Aliasing 'rm' is fairly common practice in some circles. It can have its own set of problems however (filling up partitions, running out of inodes...)
amnezia 10 years ago (5 children)
you could alias it with a script that prevents rm -rf * being run in certain directories.
jemminger 10 years ago (4 children)
you could also alias it to 'ls' :)
derefr 10 years ago * (1 child)
One could write a daemon that lets the oldest files in that directory be "garbage collected" when those conditions are approaching. I think this is, in a roundabout way, how Windows' shadow copy works.
branston 10 years ago (0 children)
Could do. Think we might be walking into the over-complexity trap however. The only time I've ever had an rm related disaster was when accidentally repeating an rm that was already in my command buffer. I looked at trying to exclude patterns from the command history but csh doesn't seem to support doing that so I gave up.

A decent solution just occurred to me when the underlying file system supports snapshots (UFS2 for example). Just snap the fs on which the to-be-deleted items are on prior to the delete. That needs barely any IO to do and you can set the snapshots to expire after 10 minutes.

Hmm... Might look at implementing that..

mbm 10 years ago (0 children)
Most of the original UNIX tools took the arguments in strict order, requiring that the options came first; you can even see this on some modern *BSD systems.
shadowsurge 10 years ago (1 child)
I just always format the command with ls first just to make sure everything is in working order. Then my neurosis kicks in and I do it again... and a couple more times just to make sure nothing bad happens.
Jonathan_the_Nerd 10 years ago (0 children)
If you're unsure about your wildcards, you can use echo to see exactly how the shell will expand your arguments.
splidge 10 years ago (0 children)
A better trick IMO is to use ls on the directory first.. then when you are sure that's what you meant type rm -rf !$ to delete it.
earthboundkid 10 years ago * (0 children)
Ever since I got burned by letting my pinky slip on the enter key years ago, I've been typing echo path first, then going back and adding the rm after the fact.
zerokey 10 years ago * (2 children)
Great story. Halfway through reading, I had a major wtf moment. I wasn't surprised by the use of a VAX, as my old department just retired their last VAX a year ago. The whole time, I'm thinking, "hello..mount the tape hardware on another system and, worst case scenario, boot from a live cd!"

Then I got to, "The next idea was to write a program to make a device descriptor for the tape deck" and looked back at the title and realized that it was from 1986 and realized, "oh..oh yeah...that's pretty fucked."

iluvatar 10 years ago (0 children)

Great story

Yeah, but really, he had way too much of a working system to qualify for true geek godhood. That title belongs to Al Viro . Even though I've read it several times, I'm still in awe every time I see that story...

cdesignproponentsist 10 years ago (0 children)
FreeBSD has backup statically-linked copies of essential system recovery tools in /rescue, just in case you toast /bin, /sbin, /lib, ld-elf.so.1, etc.

It won't protect against a rm -rf / though (and is not intended to), although you could chflags -R schg /rescue to make them immune to rm -rf.

clytle374 10 years ago * (9 children)
It happens, I tried a few months back to rm -rf bin to delete a directory and did a rm -rf /bin instead.

First thought: That took a long time.

Second thought: What do you mean ls not found.

I was amazed that the desktop survived for nearly an hour before crashing.

earthboundkid 10 years ago (8 children)
This really is a situation where GUIs are better than CLIs. There's nothing like the visual confirmation of seeing what you're obliterating to set your heart into the pit of your stomach.
jib 10 years ago (0 children)
If you're using a GUI, you probably already have that. If you're using a command line, use mv instead of rm.

In general, if you want the computer to do something, tell it what you want it to do, rather than telling it to do something you don't want and then complaining when it does what you say.

earthboundkid 10 years ago (3 children)
Yes, but trash cans aren't manly enough for vi and emacs users to take seriously. If it made sense and kept you from shooting yourself in the foot, it wouldn't be in the Unix tradition.
earthboundkid 10 years ago (1 child)
  1. Are you so low on disk space that it's important for your trash can to be empty at all times?
  2. Why should we humans have to adapt our directory names to route around the busted-ass-ness of our tools? The tools should be made to work with capital letters and spaces. Or better, use a GUI for deleting so that you don't have to worry about OMG, I forgot to put a slash in front of my space!

Seriously, I use the command line multiple times every day, but there are some tasks for which it is just not well suited compared to a GUI, and (bizarrely considering it's one thing the CLI is most used for) one of them is moving around and deleting files.

easytiger 10 years ago (0 children)
Thats a very simple bash/ksh/python/etc script.
  1. script a move op to a hidden dir on the /current/ partition.
  2. alias this to rm
  3. wrap rm as an alias to delete the contents of the hidden folder with confirmation
mattucf 10 years ago (3 children)
I'd like to think that most systems these days don't have / set as root's home directory, but I've seen a few that do. :/
dsfox 10 years ago (0 children)
This is a good approach in 1986. Today I would just pop in a bootable CDROM.
fjhqjv 10 years ago * (5 children)
That's why I always keep stringent file permissions and never act as the root user.

I'd have to try to rm -rf, get a permission denied error, then retype sudo rm -rf and then type in my password to ever have a mistake like that happen.

But I'm not a systems administrator, so maybe it's not the same thing.

toast_and_oj 10 years ago (2 children)
I aliased "rm -rf" to "omnomnom" and got myself into the habit of using that. I can barely type "omnomnom" when I really want to, let alone when I'm not really paying attention. It's saved one of my projects once already.
shen 10 years ago (0 children)
I've aliased "rm -rf" to "rmrf". Maybe I'm just a sucker for punishment.

I haven't been bit by it yet, the defining word being yet.

robreim 10 years ago (0 children)
I would have thought tab completion would have made omnomnom potentially easier to type than rm -rf (since the -rf part needs to be typed explicitly)
immure 10 years ago (0 children)
It's not.
lespea 10 years ago (0 children)
before I ever do something like that I make sure I don't have permissions so I get an error, then I press up, home, and type sudo <space> <enter> and it works as expected :)
kirun 10 years ago (0 children)
And I was pleased the other day how easy it was to fix the system after I accidentally removed kdm, konqueror and kdesktop... but these guys are hardcore.
austin_k 10 years ago (0 children)
I actually started to feel sick reading that. I've been in a IT disaster before where we almost lost a huge database. Ugh.. I still have nightmares.
umilmi81 10 years ago (4 children)
Task number 1 with a UNIX system. Alias rm to rm -i. Call the explicit path when you want to avoid the -i (ie: /bin/rm -f). Nobody is too cool to skip this basic protection.
flinchn 10 years ago (0 children)
i did an application install at an LE agency last fall - stupid me mv ./etc ./etcbk <> mv /etc /etcbk

ahh that damned period

DrunkenAsshole 10 years ago (0 children)
Were the "*"s really needed for a story that has plagued, at one point or another, all OS users?
xelfer 10 years ago (0 children)
Is the home directory for root / for some unix systems? i thought 'cd' then 'rm -rf *' would have deleted whatever's in his home directory (or whatever $HOME points to)
srparish 10 years ago (0 children)
Couldn't he just have used the editor to create the etc files he wanted, and used cpio as root to copy that over as an /etc?

sRp

stox 10 years ago (1 child)
Been there, done that. Have the soiled underwear to prove it. Amazing what kind of damage you can recover from given enough motivation.
sheepskin 10 years ago * (0 children)
I had a customer do this, he killed it about the same time. I told him he was screwed and I'd charge him a bunch of money to take down his server, rebuild it from a working one and put it back up. But the customer happened to have a root ftp session up, and was able to upload what he needed to bring the system back. by the time he was done I rebooted it to make sure it was cool and it booted all the way back up.

Of course I've also had a lot of customer that have done it, and they where screwed, and I got to charge them a bunch of money.

jemminger 10 years ago (0 children)
pfft. that's why lusers don't get root access.
supersan 10 years ago (2 children)
i had the same thing happened to me once.. my c:\ drive was running ntfs and i accidently deleted the "ntldr" system file in the c:\ root (because the name didn't figure much).. then later, i couldn't even boot in the safe mode! and my bootable disk didn't recognize the c:\ drive because it was ntfs!! so sadly, i had to reinstall everything :( wasted a whole day over it..
b100dian 10 years ago (0 children)
Yes, but that's a single file. I suppose anyone can write hex into mbr to copy ntldr from a samba share!
bobcat 10 years ago (0 children)
http://en.wikipedia.org/wiki/Emergency_Repair_Disk
boredzo 10 years ago (0 children)
Neither one is the original source. The original source is Usenet, and I can't find it with Google Groups. So either of these webpages is as good as the other.
docgnome 10 years ago (0 children)
In 1986? On a VAX?
MarlonBain 10 years ago (0 children)

This classic article from Mario Wolczko first appeared on Usenet in 1986 .

amoore 10 years ago (0 children)
I got sidetracked trying to figure out why the fictional antagonist would type the extra "/ " in "rm -rf ~/ ".
Zombine 10 years ago (2 children)

...it's amazing how much of the system you can delete without it falling apart completely. Apart from the fact that nobody could login (/bin/login?), and most of the useful commands had gone, everything else seemed normal.

Yeah. So apart from the fact that no one could get any work done or really do anything, things were working great!

I think a more rational reaction would be "Why on Earth is this big, important system on which many people rely designed in such a way that a simple easy-to-make human error can screw it up so comprehensively?" or perhaps "Why on Earth don't we have a proper backup system?"

daniels220 10 years ago (1 child)
The problem wasn't the backup system, it was the restore system, which relied on the machine having a "copy" command. Perfectly reasonable assumption that happened not to be true.
Zombine 10 years ago * (0 children)
Neither backup nor restoration serves any purpose in isolation. Most people would group those operations together under the heading "backup;" certainly you win only a semantic victory by doing otherwise. Their fail-safe data-protection system, call it what you will, turned out not to work, and had to be re-engineered on-the-fly.

I generally figure that the assumptions I make that turn out to be entirely wrong were not "perfectly reasonable" assumptions in the first place. Call me a traditionalist.

[Apr 22, 2018] rm and Its Dangers (Unix Power Tools, 3rd Edition)

Apr 22, 2018 | docstore.mik.ua
14.3. rm and Its Dangers

Under Unix, you use the rm command to delete files. The command is simple enough; you just type rm followed by a list of files. If anything, rm is too simple. It's easy to delete more than you want, and once something is gone, it's permanently gone. There are a few hacks that make rm somewhat safer, and we'll get to those momentarily. But first, here's a quick look at some of the dangers.

To understand why it's impossible to reclaim deleted files, you need to know a bit about how the Unix filesystem works. The system contains a "free list," which is a list of disk blocks that aren't used. When you delete a file, its directory entry (which gives it its name) is removed. If there are no more links ( Section 10.3 ) to the file (i.e., if the file only had one name), its inode ( Section 14.2 ) is added to the list of free inodes, and its datablocks are added to the free list.

Well, why can't you get the file back from the free list? After all, there are DOS utilities that can reclaim deleted files by doing something similar. Remember, though, Unix is a multitasking operating system. Even if you think your system is a single-user system, there are a lot of things going on "behind your back": daemons are writing to log files, handling network connections, processing electronic mail, and so on. You could theoretically reclaim a file if you could "freeze" the filesystem the instant your file was deleted -- but that's not possible. With Unix, everything is always active. By the time you realize you made a mistake, your file's data blocks may well have been reused for something else.

When you're deleting files, it's important to use wildcards carefully. Simple typing errors can have disastrous consequences. Let's say you want to delete all your object ( .o ) files. You want to type:

% rm *.o

But because of a nervous twitch, you add an extra space and type:

% rm * .o

It looks right, and you might not even notice the error. But before you know it, all the files in the current directory will be gone, irretrievably.

If you don't think this can happen to you, here's something that actually did happen to me. At one point, when I was a relatively new Unix user, I was working on my company's business plan. The executives thought, so as to be "secure," that they'd set a business plan's permissions so you had to be root ( Section 1.18 ) to modify it. (A mistake in its own right, but that's another story.) I was using a terminal I wasn't familiar with and accidentally created a bunch of files with four control characters at the beginning of their name. To get rid of these, I typed (as root ):

# rm ????*

This command took a long time to execute. When about two-thirds of the directory was gone, I realized (with horror) what was happening: I was deleting all files with four or more characters in the filename.

The story got worse. They hadn't made a backup in about five months. (By the way, this article should give you plenty of reasons for making regular backups ( Section 38.3 ).) By the time I had restored the files I had deleted (a several-hour process in itself; this was on an ancient version of Unix with a horrible backup utility) and checked (by hand) all the files against our printed copy of the business plan, I had resolved to be very careful with my rm commands.

[Some shells have safeguards that work against Mike's first disastrous example -- but not the second one. Automatic safeguards like these can become a crutch, though . . . when you use another shell temporarily and don't have them, or when you type an expression like Mike's very destructive second example. I agree with his simple advice: check your rm commands carefully! -- JP ]

-- ML

[Apr 22, 2018] How to prevent a mistaken rm -rf for specific folders?

Notable quotes:
"... There's nothing more on a traditional Linux, but you can set Apparmor/SELinux/ rules that prevent rm from accessing certain directories. ..."
"... Probably your best bet with it would be to alias rm -ri into something memorable like kill_it_with_fire . This way whenever you feel like removing something, go ahead and kill it with fire. ..."
Jan 20, 2013 | unix.stackexchange.com

I think pretty much people here mistakenly 'rm -rf'ed the wrong directory, and hopefully it did not cause a huge damage.. Is there any way to prevent users from doing a similar unix horror story?? Someone mentioned (in the comments section of the previous link) that

... I am pretty sure now every unix course or company using unix sets rm -fr to disable accounts of people trying to run it or stop them from running it ...

Is there any implementation of that in any current Unix or Linux distro? And what is the common practice to prevent that error even from a sysadmin (with root access)?

It seems that there was some protection for the root directory (/) in Solaris (since 2005) and GNU (since 2006). Is there anyway to implement the same protection way to some other folders as well??

To give it more clarity, I was not asking about general advice about rm usage (and I've updated the title to indicate that more), I want something more like the root folder protection: in order to rm -rf / you have to pass a specific parameter: rm -rf --no-preserve-root /.. Is there similar implementations for customized set of directories? Or can I specify files in addition to / to be protected by the preserve-root option?


amyassin, Jan 20, 2013 at 17:26

I think pretty much people here mistakenly ' rm -rf 'ed the wrong directory, and hopefully it did not cause a huge damage.. Is there any way to prevent users from doing a similar unix horror story ?? Someone mentioned (in the comments section of the previous link ) that

... I am pretty sure now every unix course or company using unix sets rm -fr to disable accounts of people trying to run it or stop them from running it ...

Is there any implementation of that in any current Unix or Linux distro? And what is the common practice to prevent that error even from a sysadmin (with root access)?

It seems that there was some protection for the root directory ( / ) in Solaris (since 2005) and GNU (since 2006). Is there anyway to implement the same protection way to some other folders as well??

To give it more clarity, I was not asking about general advice about rm usage (and I've updated the title to indicate that more), I want something more like the root folder protection: in order to rm -rf / you have to pass a specific parameter: rm -rf --no-preserve-root / .. Is there similar implementations for customized set of directories? Or can I specify files in addition to / to be protected by the preserve-root option?

mattdm, Jan 20, 2013 at 17:33

1) Change management 2) Backups. – mattdm Jan 20 '13 at 17:33

Keith, Jan 20, 2013 at 17:40

probably the only way would be to replace the rm command with one that doesn't have that feature. – Keith Jan 20 '13 at 17:40

sr_, Jan 20, 2013 at 18:28

safe-rm maybe – sr_ Jan 20 '13 at 18:28

Bananguin, Jan 20, 2013 at 21:07

most distros do `alias rm='rm -i' which makes rm ask you if you are sure.

Besides that: know what you are doing. only become root if necessary. for any user with root privileges security of any kind must be implemented in and by the user. hire somebody if you can't do it yourself.over time any countermeasure becomes equivalaent to the alias line above if you cant wrap your own head around the problem. – Bananguin Jan 20 '13 at 21:07

midnightsteel, Jan 22, 2013 at 14:21

@amyassin using rm -rf can be a resume generating event. Check and triple check before executing it – midnightsteel Jan 22 '13 at 14:21

Gilles, Jan 22, 2013 at 0:18

To avoid a mistaken rm -rf, do not type rm -rf .

If you need to delete a directory tree, I recommend the following workflow:

Never call rm -rf with an argument other than DELETE . Doing the deletion in several stages gives you an opportunity to verify that you aren't deleting the wrong thing, either because of a typo (as in rm -rf /foo /bar instead of rm -rf /foo/bar ) or because of a braino (oops, no, I meant to delete foo.old and keep foo.new ).

If your problem is that you can't trust others not to type rm -rf, consider removing their admin privileges. There's a lot more that can go wrong than rm .


Always make backups .

Periodically verify that your backups are working and up-to-date.

Keep everything that can't be easily downloaded from somewhere under version control.


With a basic unix system, if you really want to make some directories undeletable by rm, replace (or better shadow) rm by a custom script that rejects certain arguments. Or by hg rm .

Some unix variants offer more possibilities.

amyassin, Jan 22, 2013 at 9:41

Yeah backing up is the most amazing solution, but I was thinking of something like the --no-preserve-root option, for other important folder.. And that apparently does not exist even as a practice... – amyassin Jan 22 '13 at 9:41

Gilles, Jan 22, 2013 at 20:32

@amyassin I'm afraid there's nothing more (at least not on Linux). rm -rf already means "delete this, yes I'm sure I know what I'm doing". If you want more, replace rm by a script that refuses to delete certain directories. – Gilles Jan 22 '13 at 20:32

Gilles, Jan 22, 2013 at 22:17

@amyassin Actually, I take this back. There's nothing more on a traditional Linux, but you can set Apparmor/SELinux/ rules that prevent rm from accessing certain directories. Also, since your question isn't only about Linux, I should have mentioned OSX, which has something a bit like what you want. – Gilles Jan 22 '13 at 22:17

qbi, Jan 22, 2013 at 21:29

If you are using rm * and the zsh, you can set the option rmstarwait :
setopt rmstarwait

Now the shell warns when you're using the * :

> zsh -f
> setopt rmstarwait
> touch a b c
> rm *
zsh: sure you want to delete all the files in /home/unixuser [yn]? _

When you reject it ( n ), nothing happens. Otherwise all files will be deleted.

Drake Clarris, Jan 22, 2013 at 14:11

EDIT as suggested by comment:

You can change the attribute of to immutable the file or directory and then it cannot be deleted even by root until the attribute is removed.

chattr +i /some/important/file

This also means that the file cannot be written to or changed in anyway, even by root . Another attribute apparently available that I haven't used myself is the append attribute ( chattr +a /some/important/file . Then the file can only be opened in append mode, meaning no deletion as well, but you can add to it (say a log file). This means you won't be able to edit it in vim for example, but you can do echo 'this adds a line' >> /some/important/file . Using > instead of >> will fail.

These attributes can be unset using a minus sign, i.e. chattr -i file

Otherwise, if this is not suitable, one thing I practice is to always ls /some/dir first, and then instead of retyping the command, press up arrow CTL-A, then delete the ls and type in my rm -rf if I need it. Not perfect, but by looking at the results of ls, you know before hand if it is what you wanted.

NlightNFotis, Jan 22, 2013 at 8:27

One possible choice is to stop using rm -rf and start using rm -ri . The extra i parameter there is to make sure that it asks if you are sure you want to delete the file.

Probably your best bet with it would be to alias rm -ri into something memorable like kill_it_with_fire . This way whenever you feel like removing something, go ahead and kill it with fire.

amyassin, Jan 22, 2013 at 14:24

I like the name, but isn't f is the exact opposite of i option?? I tried it and worked though... – amyassin Jan 22 '13 at 14:24

NlightNFotis, Jan 22, 2013 at 16:09

@amyassin Yes it is. For some strange kind of fashion, I thought I only had r in there. Just fixed it. – NlightNFotis Jan 22 '13 at 16:09

Silverrocker, Jan 22, 2013 at 14:46

To protect against an accidental rm -rf * in a directory, create a file called "-i" (you can do this with emacs or some other program) in that directory. The shell will try to interpret -i and will cause it to go into interactive mode.

For example: You have a directory called rmtest with the file named -i inside. If you try to rm everything inside the directory, rm will first get -i passed to it and will go into interactive mode. If you put such a file inside the directories you would like to have some protection on, it might help.

Note that this is ineffective against rm -rf rmtest .

ValeriRangelov, Dec 21, 2014 at 3:03

If you understand C programming language, I think it is possible to rewrite the rm source code and make a little patch for kernel. I saw this on one server and it was impossible to delete some important directories and when you type 'rm -rf /direcotyr' it send email to sysadmin.

[Apr 22, 2018] Unix-Linux Horror Stories Unix Horror Stories The good thing about Unix, is when it screws up, it does so very quickly

Notable quotes:
"... And then I realized I had thrashed the server. Completely. ..."
"... There must be a way to fix this , I thought. HP-UX has a package installer like any modern Linux/Unix distribution, that is swinstall . That utility has a repair command, swrepair . ..."
"... you probably don't want that user owning /bin/nologin. ..."
Aug 04, 2011 | unixhorrorstories.blogspot.com

Unix Horror Stories: The good thing about Unix, is when it screws up, it does so very quickly The project to deploy a new, multi-million-dollar commercial system on two big, brand-new HP-UX servers at a brewing company that shall not be named, had been running on time and within budgets for several months. Just a few steps remained, among them, the migration of users from the old servers to the new ones.

The task was going to be simple: just copy the home directories of each user from the old server to the new ones, and a simple script to change the owner so as to make sure that each home directory was owned by the correct user. The script went something like this:

#!/bin/bash

cat /etc/passwd|while read line
      do
         USER=$(echo $line|cut -d: -f1)
         HOME=$(echo $line|cut -d: -f6)
         chown -R $USER $HOME
      done

[NOTE: the script does not filter out system ids from userids and that's a grave mistake. also it was run before it was tested ; -) -- NNB]

As you see, this script is pretty simple: obtain the user and the home directory from the password file, and then execute the chown command recursively on the home directory. I copied the files, executed the script, and thought, great, just 10 minutes and all is done.

That's when the calls started.

It turns out that while I was executing those seemingly harmless commands, the server was under acceptance test. You see, we were just one week away from going live and the final touches were everything that was required. So the users in the brewing company started testing if everything they needed was working like in the old servers. And suddenly, the users noticed that their system was malfunctioning and started making furious phone calls to my boss and then my boss started to call me.

And then I realized I had thrashed the server. Completely. My console was still open and I could see that the processes started failing, one by one, reporting very strange messages to the console, that didn't look any good. I started to panic. My workmate Ayelen and I (who just copied my script and executed it in the mirror server) realized only too late that the home directory of the root user was / -the root filesystem- so we changed the owner of every single file in the filesystem to root!!! That's what I love about Unix: when it screws up, it does so very quickly, and thoroughly.

There must be a way to fix this , I thought. HP-UX has a package installer like any modern Linux/Unix distribution, that is swinstall . That utility has a repair command, swrepair . So the following command put the system files back in order, needing a few permission changes on the application directories that weren't installed with the package manager:

swrepair -F

But the story doesn't end here. The next week, we were going live, and I knew that the migration of the users would be for real this time, not just a test. My boss and I were going to the brewing company, and he receives a phone call. Then he turns to me and asks me, "What was the command that you used last week?". I told him and I noticed that he was dictating it very carefully. When we arrived, we saw why: before the final deployment, a Unix administrator from the company did the same mistake I did, but this time, people from the whole country were connecting to the system, and he received phone calls from a lot of angry users. Luckily, the mistake could be fixed, and we all, young and old, went back to reading the HP-UX manual. Those things can come handy sometimes!

Morale of this story: before doing something on the users directories, take the time to see which is the User ID of actual users - which start usually in 500 but it's configuration-dependent - because system users's IDs are lower than that.

Send in your Unix horror story, and it will be featured here in the blog!

Greetings,
Agustin

Colin McD, 16 de marzo de 2017, 15:02

This script is so dangerous. You are giving home directories to say the apache user and you probably don't want that user owning /bin/nologin.

[Apr 20, 2018] GitHub - teejee2008-timeshift System restore tool for Linux. Creates filesystem snapshots using rsync+hardlinks, or BTRFS snap

Notable quotes:
"... System Restore ..."
Apr 20, 2018 | github.com

Timeshift

Timeshift for Linux is an application that provides functionality similar to the System Restore feature in Windows and the Time Machine tool in Mac OS. Timeshift protects your system by taking incremental snapshots of the file system at regular intervals. These snapshots can be restored at a later date to undo all changes to the system.

In RSYNC mode, snapshots are taken using rsync and hard-links . Common files are shared between snapshots which saves disk space. Each snapshot is a full system backup that can be browsed with a file manager.

In BTRFS mode, snapshots are taken using the in-built features of the BTRFS filesystem. BTRFS snapshots are supported only on BTRFS systems having an Ubuntu-type subvolume layout (with @ and @home subvolumes).

Timeshift is similar to applications like rsnapshot , BackInTime and TimeVault but with different goals. It is designed to protect only system files and settings. User files such as documents, pictures and music are excluded. This ensures that your files remains unchanged when you restore your system to an earlier date. If you need a tool to backup your documents and files please take a look at the excellent BackInTime application which is more configurable and provides options for saving user files.

[Apr 02, 2018] How Many Opioid Overdoses Are Suicides

Notable quotes:
"... By Martha Bebinger of WBUR. Originally published at Kaiser Health News ..."
"... The National Suicide Prevention Lifeline is 800-273-8255. ..."
"... This story is part of a partnership that includes WBUR , NPR and Kaiser Health News. ..."
Apr 02, 2018 | www.nakedcapitalism.com

Posted on March 30, 2018 by Yves Smith Yves here. See also this related Kaiser Health News story: Omissions On Death Certificates Lead To Undercounting Of Opioid Overdoses .

It takes a lot of courage for an addict to recover and stay clean. And it is sadly not news that drug addiction and high levels of prescription drug use are signs that something is deeply broken in our society. There are always some people afflicted with deep personal pain but our system is doing a very good job of generating unnecessary pain and desperation.

By Martha Bebinger of WBUR. Originally published at Kaiser Health News

Mady Ohlman was 22 on the evening some years ago when she stood in a friend's bathroom looking down at the sink.

"I had set up a bunch of needles filled with heroin because I wanted to just do them back-to-back-to-back," Ohlman recalled. She doesn't remember how many she injected before collapsing, or how long she lay drugged-out on the floor.

"But I remember being pissed because I could still get up, you know?"

She wanted to be dead, she said, glancing down, a wisp of straight brown hair slipping from behind an ear across her thin face.

At that point, said Ohlman, she'd been addicted to opioids -- controlled by the drugs -- for more than three years.

"And doing all these things you don't want to do that are horrible -- you know, selling my body, stealing from my mom, sleeping in my car," Ohlman said. "How could I not be suicidal?"

For this young woman, whose weight had dropped to about 90 pounds, who was shooting heroin just to avoid feeling violently ill, suicide seemed a painless way out.

"You realize getting clean would be a lot of work," Ohlman said, her voice rising. "And you realize dying would be a lot less painful. You also feel like you'll be doing everyone else a favor if you die."

Ohlman, who has now been sober for more than four years, said many drug users hit the same point, when the disease and the pursuit of illegal drugs crushes their will to live. Ohlman is among at least 40 percent of active drug users who wrestle with depression, anxiety or another mental health issue that increases the risk of suicide.

Measuring Suicide Among Patients Addicted To Opioids

Massachusetts, where Ohlman lives, began formally recognizing in May 2017 that some opioid overdose deaths are suicides. The state confirmed only about 2 percent of all overdose deaths as suicides, but Dr. Monica Bhare l, head of the Massachusetts Department of Public Health, said it's difficult to determine a person's true intent.

"For one thing, medical examiners use different criteria for whether suicide was involved or not," Bharel said, and the "tremendous amount of stigma surrounding both overdose deaths and suicide sometimes makes it extremely challenging to piece everything together and figure out unintentional and intentional."

Research on drug addiction and suicide suggests much higher numbers.

"[Based on the literature that's available], it looks like it's anywhere between 25 and 45 percent of deaths by overdose that may be actual suicides," said Dr. Maria Oquendo , immediate past president of the American Psychiatric Association.

Oquendo pointed to one study of overdoses from prescription opioids that found nearly 54 percent were unintentional. The rest were either suicide attempts or undetermined.

Several large studies show an increased risk of suicide among drug users addicted to opioids, especially women. In a study of about 5 million veterans, women were eight times as likely as others to be at risk for suicide, while men faced a twofold risk.

The opioid epidemic is occurring at the same time suicides have hit a 30-year high , but Oquendo said few doctors look for a connection.

"They are not monitoring it," said Oquendo, who chairs the department of psychiatry at the University of Pennsylvania. "They are probably not assessing it in the kinds of depths they would need to prevent some of the deaths."

That's starting to change. A few hospitals in Boston, for example, aim to ask every patient admitted about substance use, as well as about whether they've considered hurting themselves.

"No one has answered the chicken and egg [problem]," said Dr. Kiame Mahaniah , a family physician who runs the Lynn Community Health Center in Lynn, Mass. Is it that patients "have mental health issues that lead to addiction, or did a life of addiction then trigger mental health problems?"

With so little data to go on, "it's so important to provide treatment that covers all those bases," Mahaniah said.

'Deaths Of Despair'

When doctors do look deeper into the reasons patients addicted to opioids become suicidal, some economists predict they'll find deep reservoirs of depression and pain.

In a seminal paper published in 2015, Princeton economists Angus Deaton and Anne Case tracked falling marriage rates, the loss of stable middle-class jobs and rising rates of self-reported pain. The authors say opioid overdoses, suicides and diseases related to alcoholism are all often "deaths of despair."

"We think of opioids as something that's thrown petrol on the flames and made things infinitely worse," Deaton said, "but the underlying deep malaise would be there even without the opioids."

Many economists agree on remedies for that deep malaise. Harvard economics professor David Cutle r said solutions include a good education, a steady job that pays a decent wage, secure housing, food and health care.

"And also thinking about a sense of purpose in life," Cutler said. "That is, even if one is doing well financially, is there a sense that one is contributing in a meaningful way?"

Tackling Despair In The Addiction Community

"I know firsthand the sense of hopelessness that people can feel in the throes of addiction," said Michael Botticelli , executive director of the Grayken Center for Addiction at Boston Medical Center; he is in recovery for an addiction to alcohol.

Botticelli said recovery programs must help patients come out of isolation and create or recreate bonds with family and friends.

"The vast majority of people I know who are in recovery often talk about this profound sense of re-establishing -- and sometimes establishing for the first time -- a connection to a much larger community," Botticelli said.

Ohlman said she isn't sure why her attempted suicide, with multiple injections of heroin, didn't work.

"I just got really lucky," Ohlman said. "I don't know how."

A big part of her recovery strategy involves building a supportive community, she said.

"Meetings; 12-step; sponsorship and networking; being involved with people doing what I'm doing," said Ohlman, ticking through a list of her priorities.

There's a fatal overdose at least once a week within her Cape Cod community, she said. Some are accidental, others not. Ohlman said she's convinced that telling her story, of losing and then finding hope, will help bring those numbers down.

The National Suicide Prevention Lifeline is 800-273-8255.

This story is part of a partnership that includes WBUR , NPR and Kaiser Health News.

[Mar 28, 2018] Sysadmin wiped two servers, left the country to escape the shame by Simon Sharwood

Mar 26, 2018 | theregister.co.uk
"This revolutionary product allowed you to basically 'mirror' two file servers," Graham told The Register . "It was clever stuff back then with a high speed 100mb FDDI link doing the mirroring and the 10Mb LAN doing business as usual."

Graham was called upon to install said software at a British insurance company, which involved a 300km trip on Britain's famously brilliant motorways with a pair of servers in the back of a company car.

Maybe that drive was why Graham made a mistake after the first part of the job: getting the servers set up and talking.

"Sadly the software didn't make identifying the location of each disk easy," Graham told us. "And – ummm - I mirrored it the wrong way."

"The net result was two empty but beautifully-mirrored servers."

Oops.

Graham tried to find someone to blame, but as he was the only one on the job that wouldn't work.

His next instinct was to run, but as the site had a stack of Quarter Inch Cartridge backup tapes, he quickly learned that "incremental back-ups are the work of the devil."

Happily, all was well in the end.

[Mar 27, 2018] Cutting 'Old Heads' at IBM

Mar 27, 2018 | news.slashdot.org

(propublica.org) As the world's dominant technology firm, payrolls at International Business Machines swelled to nearly a quarter-million U.S. white-collar workers in the 1980s. Its profits helped underwrite a broad agenda of racial equality, equal pay for women and an unbeatable offer of great wages and something close to lifetime employment, all in return for unswerving loyalty. But when high tech suddenly started shifting and companies went global, IBM faced the changing landscape with a distinction most of its fiercest competitors didn't have: a large number of experienced and aging U.S. employees .

The company reacted with a strategy that, in the words of one confidential planning document, would "correct seniority mix." It slashed IBM's U.S. workforce by as much as three-quarters from its 1980s peak, replacing a substantial share with younger, less-experienced and lower-paid workers and sending many positions overseas. ProPublica estimates that in the past five years alone, IBM has eliminated more than 20,000 American employees ages 40 and over, about 60 percent of its estimated total U.S. job cuts during those years. In making these cuts, IBM has flouted or outflanked U.S. laws and regulations intended to protect later-career workers from age discrimination, according to a ProPublica review of internal company documents, legal filings and public records, as well as information provided via interviews and questionnaires filled out by more than 1,000 former IBM employees.

[Mar 19, 2018] Gogo - Create Shortcuts to Long and Complicated Paths in Linux

I do not see any bright ideas here. Using aliases is pretty much equivalent to this.
Mar 19, 2018 | www.tecmint.com

For example, if you have a directory ~/Documents/Phone-Backup/Linux-Docs/Ubuntu/ , using gogo , you can create an alias (a shortcut name), for instance Ubuntu to access it without typing the whole path anymore. No matter your current working directory, you can move into ~/cd Documents/Phone-Backup/Linux-Docs/Ubuntu/ by simply using the alias Ubuntu .

Read Also : bd – Quickly Go Back to a Parent Directory Instead of Typing "cd ../../.." Redundantly

In addition, it also allows you to create aliases for connecting directly into directories on remote Linux servers.

How to Install Gogo in Linux Systems

To install Gogo , first clone the gogo repository from Github and then copy the gogo.py to any directory in your PATH environmental variable (if you already have the ~/bin/ directory, you can place it here, otherwise create it).

$ git clone https://github.com/mgoral/gogo.git
$ cd gogo/
$ mkdir -p ~/bin        #run this if you do not have ~/bin directory
$ cp gogo.py ~/bin/

... ... ...

To start using gogo , you need to logout and login back to use it. Gogo stores its configuration in ~/.config/gogo/gogo.conf file (which should be auto created if it doesn't exist) and has the following syntax.
# Comments are lines that start from '#' character.
default = ~/something
alias = /desired/path
alias2 = /desired/path with space
alias3 = "/this/also/works"
zażółć = "unicode/is/also/supported/zażółć gęślą jaźń"

If you run gogo run without any arguments, it will go to the directory specified in default; this alias is always available, even if it's not in the configuration file, and points to $HOME directory.

To display the current aliases, use the -l switch

[Mar 13, 2018] GitHub - intoli-exodus Painless relocation of Linux binaries and all of their dependencies without containers.

Mar 13, 2018 | github.com

Painless relocation of Linux binaries–and all of their dependencies–without containers.

The Problem Being Solved

If you simply copy an executable file from one system to another, then you're very likely going to run into problems. Most binaries available on Linux are dynamically linked and depend on a number of external library files. You'll get an error like this when running a relocated binary when it has a missing dependency.

aria2c: error while loading shared libraries: libgnutls.so.30: cannot open shared object file: No such file or directory

You can try to install these libraries manually, or to relocate them and set LD_LIBRARY_PATH to wherever you put them, but it turns out that the locations of the ld-linux linker and the glibc libraries are hardcoded. Things can very quickly turn into a mess of relocation errors,

aria2c: relocation error: /lib/libpthread.so.0: symbol __getrlimit, version
GLIBC_PRIVATE not defined in file libc.so.6 with link time reference

segmentation faults,

Segmentation fault (core dumped)

or, if you're really unlucky, this very confusing symptom of a missing linker.

$ ./aria2c
bash: ./aria2c: No such file or directory
$ ls -lha ./aria2c
-rwxr-xr-x 1 sangaline sangaline 2.8M Jan 30 21:18 ./aria2c

Exodus works around these issues by compiling a small statically linked launcher binary that invokes the relocated linker directly with any hardcoded RPATH library paths overridden. The relocated binary will run with the exact same linker and libraries that it ran with on its origin machine.

[Mar 13, 2018] How To's

Mar 13, 2018 | linuxtechlab.com

Tips & Tricks How to restore deleted files in Linux with Foremost

by Shusain · March 2, 2018

It might have happened to you at one point or another that you deleted a file or an image by mistake & than regretted it immediately. So can we restore such a deleted file/image on Linux machine. In this tutorial, we are going to discuss just that i.e. how to restore a deleted file on Linux machine.

To restore a deleted file on Linux machine, we will be using an application called 'Foremost' . Foremost is a Linux based program data for recovering deleted files. The program uses a configuration file to specify headers and footers to search for. Intended to be run on disk images, foremost can search through most any kind of data without worrying about the format.

Note:- We can only restore deleted files in Linux as long as those sectors have not been overwritten on the hard disk.

We will now discuss how to recover the data with foremost. Let's start tutorial by installation of Foremost on CentOS & Ubuntu systems.

( Recommended Read: Complete guide for creating Vagrant boxes with VirtualBox )

(Also Read: Checking website statistics using Webalizer )

Install Foremost

To install Foremost on CentOS, we will download & install the foremost rpm from official webpage. Open terminal & execute the following command,

$ sudo yum install https://forensics.cert.org/centos/cert/7/x86_64//foremost-1.5.7-13.1.el7.x86_64.rpm –y

With Ubuntu, the foremost package is available with default repository. To install foremost on Ubuntu, run the following command from terminal,

$ sudo apt-get install foremost

Restore deleted files in Linux

For this scenario, we have kept an image named 'dan.jpg ' on our system. We will now delete it from the system with the following command,

$ sudo rm –rf dan.jpg

Now we will use the foremost utility to restore the image, run the following command to restore the file,

$ foremost –t jpeg –I /dev/sda1

Here, with option 't' , we have defined the type of file that needs to be restored,

-I , tells the foremost to look for the file in partition ' /dev/sda1' . We can check the partition with 'mount' command.

Upon successful execution of the command, the file will be restored in current folder. We can also add option to restore the file in a particular folder with option 'o'

$ foremost –t jpeg –I /dev/sda1 –o /root/test_folder

Note:- The restored file will not have the same file name of the original file as the filename is not stored with file itself. So file name will be different but the data should all be there.

With this we now end our tutorial on how to restore deleted files in Linux machine using Foremost. Please feel free to send in any questions or suggestion using the comment box below.

[Jan 29, 2018] How Much Swap Should You Use in Linux by Abhishek Prakash

Red Hat recommends a swap size of 20% of RAM for modern systems (i.e. 4GB or higher RAM).
Notable quotes:
"... So many people (including this article) are misinformed about the Linux swap algorithm. It doesn't just check if your RAM reaches a certain usage point. It's incredibly complicated. Linux will swap even if you are using only 20-50% of your RAM. Inactive processes are often swapped and swapping inactive processes makes more room for buffer and cache. Even if you have 16GB of RAM, having a swap partition can be beneficial ..."
Jan 25, 2018 | itsfoss.com

27 Comments

How much should be the swap size? Should the swap be double of the RAM size or should it be half of the RAM size? Do I need swap at all if my system has got several GBs of RAM? Perhaps these are the most common asked questions about choosing swap size while installing Linux. It's nothing new. There has always been a lot of confusion around swap size.

For a long time, the recommended swap size was double of the RAM size but that golden rule is not applicable to modern computers anymore. We have systems with RAM sizes up to 128 GB, many old computers don't even have this much of hard disk.

... ... ...

Swap acts as a breather to your system when the RAM is exhausted. What happens here is that when the RAM is exhausted, your Linux system uses part of the hard disk memory and allocates it to the running application.

That sounds cool. This means if you allocate like 50GB of swap size, your system can run hundreds or perhaps thousands of applications at the same time? WRONG!

You see, the speed matters here. RAM access data in the order of nanoseconds. An SSD access data in microseconds while as a normal hard disk accesses the data in milliseconds. This means that RAM is 1000 times faster than SSD and 100,000 times faster than the usual HDD.

If an application relies too much on the swap, its performance will degrade as it cannot access the data at the same speed as it would have in RAM. So instead of taking 1 second for a task, it may take several minutes to complete the same task. It will leave the application almost useless. This is known as thrashing in computing terms.

In other words, a little swap is helpful. A lot of it will be of no good use.

Why is swap needed?

There are several reasons why you would need swap.

... ... ...

Can you use Linux without swap?

Yes, you can, especially if your system has plenty of RAM. But as explained in the previous section, a little bit of swap is always advisable.

How much should be the swap size?

... ... ...

If you go by Red Hat's suggestion , they recommend a swap size of 20% of RAM for modern systems (i.e. 4GB or higher RAM).

CentOS has a different recommendation for the swap partition size . It suggests swap size to be:

Ubuntu has an entirely different perspective on the swap size as it takes hibernation into consideration. If you need hibernation, a swap of the size of RAM becomes necessary for Ubuntu. Otherwise, it recommends:

... ... ...

Jaden

So many people (including this article) are misinformed about the Linux swap algorithm. It doesn't just check if your RAM reaches a certain usage point. It's incredibly complicated. Linux will swap even if you are using only 20-50% of your RAM. Inactive processes are often swapped and swapping inactive processes makes more room for buffer and cache. Even if you have 16GB of RAM, having a swap partition can be beneficial (especially if hibernating)

kaylee

I have 4 gigs of ram on old laptop running cinnamon going by this, it is set at 60 ( what does 60 mean, and would 10 be better ) i do a little work with blender ( VSE and just starting to mess with 3d text ) should i change to 10
thanks

a. First check your current swappiness value. Type in the terminal (use copy/paste):

cat /proc/sys/vm/swappiness

Press Enter.

The result will probably be 60.

b. To change the swappiness into a more sensible setting, type in the terminal (use copy/paste to avoid typo's):

gksudo xed /etc/sysctl.conf

Press Enter.

Now a text file opens. Scroll to the bottom of that text file and add your swappiness parameter to override the default. Copy/paste the following two green lines:

# Decrease swap usage to a more reasonable level
vm.swappiness=10

c. Save and close the text file. Then reboot your computer.

DannyB

I have 32 GB of memory. Since I use SSD and no actual hard drive, having Swap would add wear to my SSD. For more than two years now I have used Linux Mint with NO SWAP.

Rationale: a small 2 GB of extra "cushion" doesn't really matter. If programs that misbehave use up 32 GB, then they'll use up 34 GB. If I had 32 GB of SWAP for a LOT of cushion, then a misbehaving program is likely to use it all up anyway.

In practice I have NEVER had a problem with 32 GB with no swap at all. At install time I made the decision to try this (which has been great in hindsight) knowing that if I really did need swap later, I could always configure a swap FILE instead of a swap PARTITION.

But I've never needed to and have never looked back at that decision to use no swap. I would recommend it.

John_Betong

No swap on Ubuntu 17.04 I am pleased to say http://www.omgubuntu.co.uk/2016/12/ubuntu-17-04-drops-swaps-swap-partitions-swap-files

Yerry Sherry

A very good explanation why you SHOULD use swap: https://chrisdown.name/2018/01/02/in-defence-of-swap.htm

[Jan 14, 2018] How to remount filesystem in read write mode under Linux

Jan 14, 2018 | kerneltalks.com

Most of the time on newly created file systems of NFS filesystems we see error like below :

1 2 3 4 root @ kerneltalks # touch file1 touch : cannot touch ' file1 ' : Read - only file system

This is because file system is mounted as read only. In such scenario you have to mount it in read-write mode. Before that we will see how to check if file system is mounted in read only mode and then we will get to how to re mount it as a read write filesystem.


How to check if file system is read only

To confirm file system is mounted in read only mode use below command –

1 2 3 4 # cat /proc/mounts | grep datastore / dev / xvdf / datastore ext3 ro , seclabel , relatime , data = ordered 0 0

Grep your mount point in cat /proc/mounts and observer third column which shows all options which are used in mounted file system. Here ro denotes file system is mounted read-only.

You can also get these details using mount -v command

1 2 3 4 root @ kerneltalks # mount -v |grep datastore / dev / xvdf on / datastore type ext3 ( ro , relatime , seclabel , data = ordered )

In this output. file system options are listed in braces at last column.


Re-mount file system in read-write mode

To remount file system in read-write mode use below command –

1 2 3 4 5 6 root @ kerneltalks # mount -o remount,rw /datastore root @ kerneltalks # mount -v |grep datastore / dev / xvdf on / datastore type ext3 ( rw , relatime , seclabel , data = ordered )

Observe after re-mounting option ro changed to rw . Now, file system is mounted as read write and now you can write files in it.

Note : It is recommended to fsck file system before re mounting it.

You can check file system by running fsck on its volume.

1 2 3 4 5 6 7 8 9 10 root @ kerneltalks # df -h /datastore Filesystem Size Used Avail Use % Mounted on / dev / xvda2 10G 881M 9.2G 9 % / root @ kerneltalks # fsck /dev/xvdf fsck from util - linux 2.23.2 e2fsck 1.42.9 ( 28 - Dec - 2013 ) / dev / xvdf : clean , 12 / 655360 files , 79696 / 2621440 blocks

Sometimes there are some corrections needs to be made on file system which needs reboot to make sure there are no processes are accessing file system.

[Jan 14, 2018] How to Install Snipe-IT Asset Management Software on Debian 9

Jan 14, 2018 | www.howtoforge.com

Snipe-IT is a free and open source IT assets management web application that can be used for tracking licenses, accessories, consumables, and components. It is written in PHP language and uses MySQL to store its data. In this tutorial, we will learn how to install Snipe-IT on Debian 9 server.

[Jan 14, 2018] Linux yes Command Tutorial for Beginners (with Examples)

Jan 14, 2018 | www.howtoforge.com

You can see that user has to type 'y' for each query. It's in situation like these where yes can help. For the above scenario specifically, you can use yes in the following way:

yes | rm -ri test Q3. Is there any use of yes when it's used alone?

Yes, there's at-least one use: to tell how well a computer system handles high amount of loads. Reason being, the tool utilizes 100% processor for systems that have a single processor. In case you want to apply this test on a system with multiple processors, you need to run a yes process for each processor.

[Jan 14, 2018] Working with Vim Editor Advanced concepts

Jan 14, 2018 | linuxtechlab.com

Opening multiple files with VI/VIM editor

To open multiple files, command would be same as is for a single file; we just add the file name for second file as well.

$ vi file1 file2 file 3

Now to browse to next file, we can use

$ :n

or we can also use

$ :e filename

Run external commands inside the editor

We can run external Linux/Unix commands from inside the vi editor, i.e. without exiting the editor. To issue a command from editor, go back to Command Mode if in Insert mode & we use the BANG i.e. '!' followed by the command that needs to be used. Syntax for running a command is,

$ :! command

An example for this would be

$ :! df -H

Searching for a pattern

To search for a word or pattern in the text file, we use following two commands in command mode,

Both of these commands are used for same purpose, only difference being the direction they search in. An example would be,

$ :/ search pattern (If at beginning of the file)

$ :/ search pattern (If at the end of the file)

Searching & replacing a pattern

We might be required to search & replace a word or a pattern from our text files. So rather than finding the occurrence of word from whole text file & replace it, we can issue a command from the command mode to replace the word automatically. Syntax for using search & replacement is,

$ :s/pattern_to_be_found/New_pattern/g

Suppose we want to find word "alpha" & replace it with word "beta", the command would be

$ :s/alpha/beta/g

If we want to only replace the first occurrence of word "alpha", then the command would be

$ :s/alpha/beta/

Using Set commands

We can also customize the behaviour, the and feel of the vi/vim editor by using the set command. Here is a list of some options that can be use set command to modify the behaviour of vi/vim editor,

$ :set ic ignores cases while searching

$ :set smartcase enforce case sensitive search

$ :set nu display line number at the begining of the line

$ :set hlsearch highlights the matching words

$ : set ro change the file type to read only

$ : set term prints the terminal type

$ : set ai sets auto-indent

$ :set noai unsets the auto-indent

Some other commands to modify vi editors are,

$ :colorscheme its used to change the color scheme for the editor. (for VIM editor only)

$ :syntax on will turn on the color syntax for .xml, .html files etc. (for VIM editor only)

This complete our tutorial, do mention your queries/questions or suggestions in the comment box below.

[Jan 14, 2018] Learn to use Wget command with 12 examples

Jan 14, 2018 | linuxtechlab.com

Downloading file & storing with a different name

If we want to save the downloaded file with a different name than its default name, we can use '-O' parameter with wget command to do so,

$ wget -O nagios_latest https://downloads.sourceforge.net/project/nagios/nagios-4.x/nagios-4.3.1/nagios-4.3.1.tar.gz?r=&ts=1489637334&use_mirror=excellmedia

Replicate whole website

If you need to download all contents of a website, you can do so by using '--mirror' parameter,

$ wget --mirror -p --convert-links -P /home/dan xyz.com

Here, wget – mirror is command to download the website,

-p, will download all files necessary to display HTML files properly,

--convert-links, will convert links in documents for viewing,

-P /home/dan, will save the file in /home/dan directory.

Download only a certain type of files

To download only a file with certain format type, use '-r -A' parameters,

$ wget -r -A.txt Website_url

Exclude a certain file type

While downloading a website, if you don't want to download a certain file type you can do so by using '- – reject' parameter,

$ wget --reject=png Website_url

[Jan 14, 2018] Sysadmin Tips on Preparing for Vacation by Kyle Rankin

Notable quotes:
"... If you can, freeze changes in the weeks leading up to your vacation. Try to encourage other teams to push off any major changes until after you get back. ..."
"... Check for any systems about to hit a disk warning threshold and clear out space. ..."
"... Make sure all of your backup scripts are working and all of your backups are up to date. ..."
Jan 11, 2018 | www.linuxjournal.com

... ... ...

If you do need to take your computer, I highly recommend making a full backup before the trip. Your computer is more likely to be lost, stolen or broken while traveling than when sitting safely at the office, so I always take a backup of my work machine before a trip. Even better than taking a backup, leave your expensive work computer behind and use a cheaper more disposable machine for travel and just restore your important files and settings for work on it before you leave and wipe it when you return. If you decide to go the disposable computer route, I recommend working one or two full work days on this computer before the vacation to make sure all of your files and settings are in place.

Documentation

Good documentation is the best way to reduce or eliminate how much you have to step in when you aren't on call, whether you're on vacation or not. Everything from routine procedures to emergency response should be documented and kept up to date. Honestly, this falls under standard best practices as a sysadmin, so it's something you should have whether or not you are about to go on vacation.

One saying about documentation is that if something is documented in two places, one of them will be out of date. Even if you document something only in one place, there's a good chance it is out of date unless you perform routine maintenance. It's a good practice to review your documentation from time to time and update it where necessary and before a vacation is a particularly good time to do it. If you are the only person that knows about the new way to perform a procedure, you should make sure your documentation covers it.

Finally, have your team maintain a page to capture anything that happens while you are gone that they want to tell you about when you get back. If you are the main maintainer of a particular system, but they had to perform some emergency maintenance of it while you were gone, that's the kind of thing you'd like to know about when you get back. If there's a central place for the team to capture these notes, they will be more likely to write things down as they happen and less likely to forget about things when you get back.

Stable State

The more stable your infrastructure is before you leave and the more stable it stays while you are gone, the less likely you'll be disturbed on your vacation. Right before a vacation is a terrible time to make a major change to critical systems. If you can, freeze changes in the weeks leading up to your vacation. Try to encourage other teams to push off any major changes until after you get back.

Before a vacation is also a great time to perform any preventative maintenance on your systems. Check for any systems about to hit a disk warning threshold and clear out space. In general, if you collect trending data, skim through it for any resources that are trending upward that might go past thresholds while you are gone. If you have any tasks that might add extra load to your systems while you are gone, pause or postpone them if you can. Make sure all of your backup scripts are working and all of your backups are up to date.

Emergency Contact Methods

Although it would be great to unplug completely while on vacation, there's a chance that someone from work might want to reach you in an emergency. Depending on where you plan to travel, some contact options may work better than others. For instance, some cell-phone plans that work while traveling might charge high rates for calls, but text messages and data bill at the same rates as at home.

... ... ... Kyle Rankin is senior security and infrastructure architect, the author of many books including Linux Hardening in Hostile Networks, DevOps Troubleshooting and The Official Ubuntu Server Book, and a columnist for Linux Journal. Follow him @kylerankin

[Jan 14, 2018] Linux Filesystem Events with inotify by Charles Fisher

Notable quotes:
"... Lukas Jelinek is the author of the incron package that allows users to specify tables of inotify events that are executed by the master incrond process. Despite the reference to "cron", the package does not schedule events at regular intervals -- it is a tool for filesystem events, and the cron reference is slightly misleading. ..."
"... The incron package is available from EPEL ..."
Jan 08, 2018 | www.linuxjournal.com

Triggering scripts with incron and systemd.

It is, at times, important to know when things change in the Linux OS. The uses to which systems are placed often include high-priority data that must be processed as soon as it is seen. The conventional method of finding and processing new file data is to poll for it, usually with cron. This is inefficient, and it can tax performance unreasonably if too many polling events are forked too often.

Linux has an efficient method for alerting user-space processes to changes impacting files of interest. The inotify Linux system calls were first discussed here in Linux Journal in a 2005 article by Robert Love who primarily addressed the behavior of the new features from the perspective of C.

However, there also are stable shell-level utilities and new classes of monitoring dæmons for registering filesystem watches and reporting events. Linux installations using systemd also can access basic inotify functionality with path units. The inotify interface does have limitations -- it can't monitor remote, network-mounted filesystems (that is, NFS); it does not report the userid involved in the event; it does not work with /proc or other pseudo-filesystems; and mmap() operations do not trigger it, among other concerns. Even with these limitations, it is a tremendously useful feature.

This article completes the work begun by Love and gives everyone who can write a Bourne shell script or set a crontab the ability to react to filesystem changes.

The inotifywait Utility

Working under Oracle Linux 7 (or similar versions of Red Hat/CentOS/Scientific Linux), the inotify shell tools are not installed by default, but you can load them with yum:

 # yum install inotify-tools
Loaded plugins: langpacks, ulninfo
ol7_UEKR4                                      | 1.2 kB   00:00
ol7_latest                                     | 1.4 kB   00:00
Resolving Dependencies
--> Running transaction check
---> Package inotify-tools.x86_64 0:3.14-8.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

==============================================================
Package         Arch       Version        Repository     Size
==============================================================
Installing:
inotify-tools   x86_64     3.14-8.el7     ol7_latest     50 k

Transaction Summary
==============================================================
Install  1 Package

Total download size: 50 k
Installed size: 111 k
Is this ok [y/d/N]: y
Downloading packages:
inotify-tools-3.14-8.el7.x86_64.rpm               |  50 kB   00:00
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
Warning: RPMDB altered outside of yum.
  Installing : inotify-tools-3.14-8.el7.x86_64                 1/1
  Verifying  : inotify-tools-3.14-8.el7.x86_64                 1/1

Installed:
  inotify-tools.x86_64 0:3.14-8.el7

Complete!

The package will include two utilities (inotifywait and inotifywatch), documentation and a number of libraries. The inotifywait program is of primary interest.

Some derivatives of Red Hat 7 may not include inotify in their base repositories. If you find it missing, you can obtain it from Fedora's EPEL repository , either by downloading the inotify RPM for manual installation or adding the EPEL repository to yum.

Any user on the system who can launch a shell may register watches -- no special privileges are required to use the interface. This example watches the /tmp directory:

$ inotifywait -m /tmp
Setting up watches.
Watches established.

If another session on the system performs a few operations on the files in /tmp:

$ touch /tmp/hello
$ cp /etc/passwd /tmp
$ rm /tmp/passwd
$ touch /tmp/goodbye
$ rm /tmp/hello /tmp/goodbye

those changes are immediately visible to the user running inotifywait:

/tmp/ CREATE hello
/tmp/ OPEN hello
/tmp/ ATTRIB hello
/tmp/ CLOSE_WRITE,CLOSE hello
/tmp/ CREATE passwd
/tmp/ OPEN passwd
/tmp/ MODIFY passwd
/tmp/ CLOSE_WRITE,CLOSE passwd
/tmp/ DELETE passwd
/tmp/ CREATE goodbye
/tmp/ OPEN goodbye
/tmp/ ATTRIB goodbye
/tmp/ CLOSE_WRITE,CLOSE goodbye
/tmp/ DELETE hello
/tmp/ DELETE goodbye

A few relevant sections of the manual page explain what is happening:

$ man inotifywait | col -b | sed -n '/diagnostic/,/helpful/p'
  inotifywait will output diagnostic information on standard error and
  event information on standard output. The event output can be config-
  ured, but by default it consists of lines of the following form:

  watched_filename EVENT_NAMES event_filename


  watched_filename
    is the name of the file on which the event occurred. If the
    file is a directory, a trailing slash is output.

  EVENT_NAMES
    are the names of the inotify events which occurred, separated by
    commas.

  event_filename
    is output only when the event occurred on a directory, and in
    this case the name of the file within the directory which caused
    this event is output.

    By default, any special characters in filenames are not escaped
    in any way. This can make the output of inotifywait difficult
    to parse in awk scripts or similar. The --csv and --format
    options will be helpful in this case.

It also is possible to filter the output by registering particular events of interest with the -e option, the list of which is shown here:

access create move_self
attrib delete moved_to
close_write delete_self moved_from
close_nowrite modify open
close move unmount

A common application is testing for the arrival of new files. Since inotify must be given the name of an existing filesystem object to watch, the directory containing the new files is provided. A trigger of interest is also easy to provide -- new files should be complete and ready for processing when the close_write trigger fires. Below is an example script to watch for these events:

#!/bin/sh
unset IFS                                 # default of space, tab and nl
                                          # Wait for filesystem events
inotifywait -m -e close_write \
   /tmp /var/tmp /home/oracle/arch-orcl/ |
while read dir op file
do [[ "${dir}" == '/tmp/' && "${file}" == *.txt ]] &&
      echo "Import job should start on $file ($dir $op)."

   [[ "${dir}" == '/var/tmp/' && "${file}" == CLOSE_WEEK*.txt ]] &&
      echo Weekly backup is ready.

   [[ "${dir}" == '/home/oracle/arch-orcl/' && "${file}" == *.ARC ]]
&&
      su - oracle -c 'ORACLE_SID=orcl ~oracle/bin/log_shipper' &

   [[ "${dir}" == '/tmp/' && "${file}" == SHUT ]] && break

   ((step+=1))
done

echo We processed $step events.

There are a few problems with the script as presented -- of all the available shells on Linux, only ksh93 (that is, the AT&T Korn shell) will report the "step" variable correctly at the end of the script. All the other shells will report this variable as null.

The reason for this behavior can be found in a brief explanation on the manual page for Bash: "Each command in a pipeline is executed as a separate process (i.e., in a subshell)." The MirBSD clone of the Korn shell has a slightly longer explanation:

# man mksh | col -b | sed -n '/The parts/,/do so/p'
  The parts of a pipeline, like below, are executed in subshells. Thus,
  variable assignments inside them fail. Use co-processes instead.

  foo | bar | read baz          # will not change $baz
  foo | bar |& read -p baz      # will, however, do so

And, the pdksh documentation in Oracle Linux 5 (from which MirBSD mksh emerged) has several more mentions of the subject:

General features of at&t ksh88 that are not (yet) in pdksh:
  - the last command of a pipeline is not run in the parent shell
  - `echo foo | read bar; echo $bar' prints foo in at&t ksh, nothing
    in pdksh (ie, the read is done in a separate process in pdksh).
  - in pdksh, if the last command of a pipeline is a shell builtin, it
    is not executed in the parent shell, so "echo a b | read foo bar"
    does not set foo and bar in the parent shell (at&t ksh will).
    This may get fixed in the future, but it may take a while.

$ man pdksh | col -b | sed -n '/BTW, the/,/aware/p'
  BTW, the most frequently reported bug is
    echo hi | read a; echo $a   # Does not print hi
  I'm aware of this and there is no need to report it.

This behavior is easy enough to demonstrate -- running the script above with the default bash shell and providing a sequence of example events:

$ cp /etc/passwd /tmp/newdata.txt
$ cp /etc/group /var/tmp/CLOSE_WEEK20170407.txt
$ cp /etc/passwd /tmp/SHUT

gives the following script output:

# ./inotify.sh
Setting up watches.
Watches established.
Import job should start on newdata.txt (/tmp/ CLOSE_WRITE,CLOSE).
Weekly backup is ready.
We processed events.

Examining the process list while the script is running, you'll also see two shells, one forked for the control structure:

$ function pps { typeset a IFS=\| ; ps ax | while read a
do case $a in *$1*|+([!0-9])) echo $a;; esac; done }


$ pps inot
  PID TTY      STAT   TIME COMMAND
 3394 pts/1    S+     0:00 /bin/sh ./inotify.sh
 3395 pts/1    S+     0:00 inotifywait -m -e close_write /tmp /var/tmp
 3396 pts/1    S+     0:00 /bin/sh ./inotify.sh

As it was manipulated in a subshell, the "step" variable above was null when control flow reached the echo. Switching this from #/bin/sh to #/bin/ksh93 will correct the problem, and only one shell process will be seen:

# ./inotify.ksh93
Setting up watches.
Watches established.
Import job should start on newdata.txt (/tmp/ CLOSE_WRITE,CLOSE).
Weekly backup is ready.
We processed 2 events.


$ pps inot
  PID TTY      STAT   TIME COMMAND
 3583 pts/1    S+     0:00 /bin/ksh93 ./inotify.sh
 3584 pts/1    S+     0:00 inotifywait -m -e close_write /tmp /var/tmp

Although ksh93 behaves properly and in general handles scripts far more gracefully than all of the other Linux shells, it is rather large:

$ ll /bin/[bkm]+([aksh93]) /etc/alternatives/ksh
-rwxr-xr-x. 1 root root  960456 Dec  6 11:11 /bin/bash
lrwxrwxrwx. 1 root root      21 Apr  3 21:01 /bin/ksh ->
                                               /etc/alternatives/ksh
-rwxr-xr-x. 1 root root 1518944 Aug 31  2016 /bin/ksh93
-rwxr-xr-x. 1 root root  296208 May  3  2014 /bin/mksh
lrwxrwxrwx. 1 root root      10 Apr  3 21:01 /etc/alternatives/ksh ->
                                                    /bin/ksh93

The mksh binary is the smallest of the Bourne implementations above (some of these shells may be missing on your system, but you can install them with yum). For a long-term monitoring process, mksh is likely the best choice for reducing both processing and memory footprint, and it does not launch multiple copies of itself when idle assuming that a coprocess is used. Converting the script to use a Korn coprocess that is friendly to mksh is not difficult:

#!/bin/mksh
unset IFS                              # default of space, tab and nl
                                       # Wait for filesystem events
inotifywait -m -e close_write \
   /tmp/ /var/tmp/ /home/oracle/arch-orcl/ \
   2</dev/null |&                      # Launch as Korn coprocess

while read -p dir op file              # Read from Korn coprocess
do [[ "${dir}" == '/tmp/' && "${file}" == *.txt ]] &&
      print "Import job should start on $file ($dir $op)."

   [[ "${dir}" == '/var/tmp/' && "${file}" == CLOSE_WEEK*.txt ]] &&
      print Weekly backup is ready.

   [[ "${dir}" == '/home/oracle/arch-orcl/' && "${file}" == *.ARC ]]
&&
      su - oracle -c 'ORACLE_SID=orcl ~oracle/bin/log_shipper' &

   [[ "${dir}" == '/tmp/' && "${file}" == SHUT ]] && break

   ((step+=1))
done

echo We processed $step events.

Note that the Korn and Bolsky reference on the Korn shell outlines the following requirements in a program operating as a coprocess:

Caution: The co-process must:

An fflush(NULL) is found in the main processing loop of the inotifywait source, and these requirements appear to be met.

The mksh version of the script is the most reasonable compromise for efficient use and correct behavior, and I have explained it at some length here to save readers trouble and frustration -- it is important to avoid control structures executing in subshells in most of the Borne family. However, hopefully all of these ersatz shells someday fix this basic flaw and implement the Korn behavior correctly.

A Practical Application -- Oracle Log Shipping

Oracle databases that are configured for hot backups produce a stream of "archived redo log files" that are used for database recovery. These are the most critical backup files that are produced in an Oracle database.

These files are numbered sequentially and are written to a log directory configured by the DBA. An inotifywatch can trigger activities to compress, encrypt and/or distribute the archived logs to backup and disaster recovery servers for safekeeping. You can configure Oracle RMAN to do most of these functions, but the OS tools are more capable, flexible and simpler to use.

There are a number of important design parameters for a script handling archived logs:

Given these design parameters, this is an implementation:

# cat ~oracle/archutils/process_logs

#!/bin/ksh93

set -euo pipefail
IFS=$'\n\t'  # http://redsymbol.net/articles/unofficial-bash-strict-mode/

(
 flock -n 9 || exit 1          # Critical section-allow only one process.

 ARCHDIR=~oracle/arch-${ORACLE_SID}

 APREFIX=${ORACLE_SID}_1_

 ASUFFIX=.ARC

 CURLOG=$(<~oracle/.curlog-$ORACLE_SID)

 File="${ARCHDIR}/${APREFIX}${CURLOG}${ASUFFIX}"

 [[ ! -f "$File" ]] && exit

 while [[ -f "$File" ]]
 do ((NEXTCURLOG=CURLOG+1))

    NextFile="${ARCHDIR}/${APREFIX}${NEXTCURLOG}${ASUFFIX}"

    [[ ! -f "$NextFile" ]] && sleep 60  # Ensure ARCH has finished

    nice /usr/local/bin/lzip -9q "$File"

    until scp "${File}.lz" "yourcompany.com:~oracle/arch-$ORACLE_SID"
    do sleep 5
    done

    CURLOG=$NEXTCURLOG

    File="$NextFile"
 done

 echo $CURLOG > ~oracle/.curlog-$ORACLE_SID

) 9>~oracle/.processing_logs-$ORACLE_SID

The above script can be executed manually for testing even while the inotify handler is running, as the flock protects it.

A standby server, or a DataGuard server in primitive standby mode, can apply the archived logs at regular intervals. The script below forces a 12-hour delay in log application for the recovery of dropped or damaged objects, so inotify cannot be easily used in this case -- cron is a more reasonable approach for delayed file processing, and a run every 20 minutes will keep the standby at the desired recovery point:

# cat ~oracle/archutils/delay-lock.sh

#!/bin/ksh93

(
 flock -n 9 || exit 1              # Critical section-only one process.

 WINDOW=43200                      # 12 hours

 LOG_DEST=~oracle/arch-$ORACLE_SID

 OLDLOG_DEST=$LOG_DEST-applied

 function fage { print $(( $(date +%s) - $(stat -c %Y "$1") ))
  } # File age in seconds - Requires GNU extended date & stat

 cd $LOG_DEST

 of=$(ls -t | tail -1)             # Oldest file in directory

 [[ -z "$of" || $(fage "$of") -lt $WINDOW ]] && exit

 for x in $(ls -rt)                    # Order by ascending file mtime
 do if [[ $(fage "$x") -ge $WINDOW ]]
    then y=$(basename $x .lz)          # lzip compression is optional

         [[ "$y" != "$x" ]] && /usr/local/bin/lzip -dkq "$x"

         $ORACLE_HOME/bin/sqlplus '/ as sysdba' > /dev/null 2>&1 <<-EOF
                recover standby database;
                $LOG_DEST/$y
                cancel
                quit
                EOF

         [[ "$y" != "$x" ]] && rm "$y"

         mv "$x" $OLDLOG_DEST
    fi
              

 done
) 9> ~oracle/.recovering-$ORACLE_SID

I've covered these specific examples here because they introduce tools to control concurrency, which is a common issue when using inotify, and they advance a few features that increase reliability and minimize storage requirements. Hopefully enthusiastic readers will introduce many improvements to these approaches.

The incron System

Lukas Jelinek is the author of the incron package that allows users to specify tables of inotify events that are executed by the master incrond process. Despite the reference to "cron", the package does not schedule events at regular intervals -- it is a tool for filesystem events, and the cron reference is slightly misleading.

The incron package is available from EPEL . If you have installed the repository, you can load it with yum:

# yum install incron
Loaded plugins: langpacks, ulninfo
Resolving Dependencies
--> Running transaction check
---> Package incron.x86_64 0:0.5.10-8.el7 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

=================================================================
 Package       Arch       Version           Repository    Size
=================================================================
Installing:
 incron        x86_64     0.5.10-8.el7      epel          92 k

Transaction Summary
==================================================================
Install  1 Package

Total download size: 92 k
Installed size: 249 k
Is this ok [y/d/N]: y
Downloading packages:
incron-0.5.10-8.el7.x86_64.rpm                      |  92 kB   00:01
Running transaction check
Running transaction test
Transaction test succeeded
Running transaction
  Installing : incron-0.5.10-8.el7.x86_64                          1/1
  Verifying  : incron-0.5.10-8.el7.x86_64                          1/1

Installed:
  incron.x86_64 0:0.5.10-8.el7

Complete!

On a systemd distribution with the appropriate service units, you can start and enable incron at boot with the following commands:

# systemctl start incrond
# systemctl enable incrond
Created symlink from
   /etc/systemd/system/multi-user.target.wants/incrond.service
to /usr/lib/systemd/system/incrond.service.

In the default configuration, any user can establish incron schedules. The incrontab format uses three fields:

<path> <mask> <command>

Below is an example entry that was set with the -e option:

$ incrontab -e        #vi session follows

$ incrontab -l
/tmp/ IN_ALL_EVENTS /home/luser/myincron.sh $@ $% $#

You can record a simple script and mark it with execute permission:

$ cat myincron.sh
#!/bin/sh

echo -e "path: $1 op: $2 \t file: $3" >> ~/op

$ chmod 755 myincron.sh

Then, if you repeat the original /tmp file manipulations at the start of this article, the script will record the following output:

$ cat ~/op

path: /tmp/ op: IN_ATTRIB        file: hello
path: /tmp/ op: IN_CREATE        file: hello
path: /tmp/ op: IN_OPEN          file: hello
path: /tmp/ op: IN_CLOSE_WRITE   file: hello
path: /tmp/ op: IN_OPEN          file: passwd
path: /tmp/ op: IN_CLOSE_WRITE   file: passwd
path: /tmp/ op: IN_MODIFY        file: passwd
path: /tmp/ op: IN_CREATE        file: passwd
path: /tmp/ op: IN_DELETE        file: passwd
path: /tmp/ op: IN_CREATE        file: goodbye
path: /tmp/ op: IN_ATTRIB        file: goodbye
path: /tmp/ op: IN_OPEN          file: goodbye
path: /tmp/ op: IN_CLOSE_WRITE   file: goodbye
path: /tmp/ op: IN_DELETE        file: hello
path: /tmp/ op: IN_DELETE        file: goodbye

While the IN_CLOSE_WRITE event on a directory object is usually of greatest interest, most of the standard inotify events are available within incron, which also offers several unique amalgams:

$ man 5 incrontab | col -b | sed -n '/EVENT SYMBOLS/,/child process/p'

EVENT SYMBOLS

These basic event mask symbols are defined:

IN_ACCESS          File was accessed (read) (*)
IN_ATTRIB          Metadata changed (permissions, timestamps, extended
                   attributes, etc.) (*)
IN_CLOSE_WRITE     File opened for writing was closed (*)
IN_CLOSE_NOWRITE   File not opened for writing was closed (*)
IN_CREATE          File/directory created in watched directory (*)
IN_DELETE          File/directory deleted from watched directory (*)
IN_DELETE_SELF     Watched file/directory was itself deleted
IN_MODIFY          File was modified (*)
IN_MOVE_SELF       Watched file/directory was itself moved
IN_MOVED_FROM      File moved out of watched directory (*)
IN_MOVED_TO        File moved into watched directory (*)
IN_OPEN            File was opened (*)

When monitoring a directory, the events marked with an asterisk (*)
above can occur for files in the directory, in which case the name
field in the returned event data identifies the name of the file within
the directory.

The IN_ALL_EVENTS symbol is defined as a bit mask of all of the above
events. Two additional convenience symbols are IN_MOVE, which is a com-
bination of IN_MOVED_FROM and IN_MOVED_TO, and IN_CLOSE, which combines
IN_CLOSE_WRITE and IN_CLOSE_NOWRITE.

The following further symbols can be specified in the mask:

IN_DONT_FOLLOW     Don't dereference pathname if it is a symbolic link
IN_ONESHOT         Monitor pathname for only one event
IN_ONLYDIR         Only watch pathname if it is a directory

Additionally, there is a symbol which doesn't appear in the inotify sym-
bol set. It is IN_NO_LOOP. This symbol disables monitoring events until
the current one is completely handled (until its child process exits).

The incron system likely presents the most comprehensive interface to inotify of all the tools researched and listed here. Additional configuration options can be set in /etc/incron.conf to tweak incron's behavior for those that require a non-standard configuration.

Path Units under systemd

When your Linux installation is running systemd as PID 1, limited inotify functionality is available through "path units" as is discussed in a lighthearted article by Paul Brown at OCS-Mag .

The relevant manual page has useful information on the subject:

$ man systemd.path | col -b | sed -n '/Internally,/,/systems./p'

Internally, path units use the inotify(7) API to monitor file systems.
Due to that, it suffers by the same limitations as inotify, and for
example cannot be used to monitor files or directories changed by other
machines on remote NFS file systems.

Note that when a systemd path unit spawns a shell script, the $HOME and tilde ( ~ ) operator for the owner's home directory may not be defined. Using the tilde operator to reference another user's home directory (for example, ~nobody/) does work, even when applied to the self-same user running the script. The Oracle script above was explicit and did not reference ~ without specifying the target user, so I'm using it as an example here.

Using inotify triggers with systemd path units requires two files. The first file specifies the filesystem location of interest:

$ cat /etc/systemd/system/oralog.path

[Unit]
Description=Oracle Archivelog Monitoring
Documentation=http://docs.yourserver.com

[Path]
PathChanged=/home/oracle/arch-orcl/

[Install]
WantedBy=multi-user.target

The PathChanged parameter above roughly corresponds to the close-write event used in my previous direct inotify calls. The full collection of inotify events is not (currently) supported by systemd -- it is limited to PathExists , PathChanged and PathModified , which are described in man systemd.path .

The second file is a service unit describing a program to be executed. It must have the same name, but a different extension, as the path unit:

$ cat /etc/systemd/system/oralog.service

[Unit]
Description=Oracle Archivelog Monitoring
Documentation=http://docs.yourserver.com

[Service]
Type=oneshot
Environment=ORACLE_SID=orcl
ExecStart=/bin/sh -c '/root/process_logs >> /tmp/plog.txt 2>&1'

The oneshot parameter above alerts systemd that the program that it forks is expected to exit and should not be respawned automatically -- the restarts are limited to triggers from the path unit. The above service configuration will provide the best options for logging -- divert them to /dev/null if they are not needed.

Use systemctl start on the path unit to begin monitoring -- a common error is using it on the service unit, which will directly run the handler only once. Enable the path unit if the monitoring should survive a reboot.

Although this limited functionality may be enough for some casual uses of inotify, it is a shame that the full functionality of inotifywait and incron are not represented here. Perhaps it will come in time.

Conclusion

Although the inotify tools are powerful, they do have limitations. To repeat them, inotify cannot monitor remote (NFS) filesystems; it cannot report the userid involved in a triggering event; it does not work with /proc or other pseudo-filesystems; mmap() operations do not trigger it; and the inotify queue can overflow resulting in lost events, among other concerns.

Even with these weaknesses, the efficiency of inotify is superior to most other approaches for immediate notifications of filesystem activity. It also is quite flexible, and although the close-write directory trigger should suffice for most usage, it has ample tools for covering special use cases.

In any event, it is productive to replace polling activity with inotify watches, and system administrators should be liberal in educating the user community that the classic crontab is not an appropriate place to check for new files. Recalcitrant users should be confined to Ultrix on a VAX until they develop sufficient appreciation for modern tools and approaches, which should result in more efficient Linux systems and happier administrators.

Sidenote: Archiving /etc/passwd

Tracking changes to the password file involves many different types of inotify triggering events. The vipw utility commonly will make changes to a temporary file, then clobber the original with it. This can be seen when the inode number changes:

# ll -i /etc/passwd
199720973 -rw-r--r-- 1 root root 3928 Jul  7 12:24 /etc/passwd

# vipw
[ make changes ]
You are using shadow passwords on this system.
Would you like to edit /etc/shadow now [y/n]? n

# ll -i /etc/passwd
203784208 -rw-r--r-- 1 root root 3956 Jul  7 12:24 /etc/passwd

The destruction and replacement of /etc/passwd even occurs with setuid binaries called by unprivileged users:

$ ll -i /etc/passwd
203784196 -rw-r--r-- 1 root root 3928 Jun 29 14:55 /etc/passwd

$ chsh
Changing shell for fishecj.
Password:
New shell [/bin/bash]: /bin/csh
Shell changed.

$ ll -i /etc/passwd
199720970 -rw-r--r-- 1 root root 3927 Jul  7 12:23 /etc/passwd

For this reason, all inotify triggering events should be considered when tracking this file. If there is concern with an inotify queue overflow (in which events are lost), then the OPEN , ACCESS and CLOSE_NOWRITE,CLOSE triggers likely can be immediately ignored.

All other inotify events on /etc/passwd might run the following script to version the changes into an RCS archive and mail them to an administrator:

#!/bin/sh

# This script tracks changes to the /etc/passwd file from inotify.
# Uses RCS for archiving. Watch for UID zero.

PWMAILS=Charlie.Root@openbsd.org

TPDIR=~/track_passwd

cd $TPDIR

if diff -q /etc/passwd $TPDIR/passwd
then exit                                         # they are the same
else sleep 5                                      # let passwd settle
     diff /etc/passwd $TPDIR/passwd 2>&1 |        # they are DIFFERENT
     mail -s "/etc/passwd changes $(hostname -s)" "$PWMAILS"
     cp -f /etc/passwd $TPDIR                     # copy for checkin

#    "SCCS, the source motel! Programs check in and never check out!"
#     -- Ken Thompson

     rcs -q -l passwd                            # lock the archive
     ci -q -m_ passwd                            # check in new ver
     co -q passwd                                # drop the new copy
fi > /dev/null 2>&1

Here is an example email from the script for the above chfn operation:

-----Original Message-----
From: root [mailto:root@myhost.com]
Sent: Thursday, July 06, 2017 2:35 PM
To: Fisher, Charles J. <Charles.Fisher@myhost.com>;
Subject: /etc/passwd changes myhost

57c57
< fishecj:x:123:456:Fisher, Charles J.:/home/fishecj:/bin/bash
---
> fishecj:x:123:456:Fisher, Charles J.:/home/fishecj:/bin/csh

Further processing on the third column of /etc/passwd might detect UID zero (a root user) or other important user classes for emergency action. This might include a rollback of the file from RCS to /etc and/or SMS messages to security contacts. ______________________

Charles Fisher has an electrical engineering degree from the University of Iowa and works as a systems and database administrator for a Fortune 500 mining and manufacturing corporation.

[Dec 25, 2017] American Carnage by Brad Griffin

Notable quotes:
"... It tells me that the bottom line is that Christmas has become a harder season for White families. We are worse off because of BOTH social and economic liberalism which has only benefited an elite few. The bottom half of the White population is now in total disarray – drug addiction, demoralization, divorce, suicide, abortion, atomization, stagnant wages, declining household income and investments – and this dysfunction is creeping up the social ladder. The worst thing we can do is step on the accelerator. ..."
Dec 24, 2017 | www.unz.com

As we move into 2018, I am swinging away from the Republicans. I don't support the Paul Ryan "Better Way" agenda. I don't support neoliberal economics. I think we have been going in the wrong direction since the 1970s and don't want to continue going down this road.

  1. Opioid Deaths: As we all know, the opioid epidemic has become a national crisis and the White working class has been hit the hardest by it. It is a "sea of despair" out there.
  2. White Mortality: As the family crumbles, religion recedes in his life, and his job prospects dwindle, the middle aged White working class man is turning to drugs, alcohol and suicide: The White suicide rate has soared since 2000:
  3. Median Household Income: The average household in the United States is poorer in 2017 than it was in 1997:
  4. Real GDP: Since the late 1990s, real GDP and real median household income have parted ways:
  5. Productivity and Real Wages: Since the 1970s, the minimum wage has parted ways with productivity gains in the US economy:
  6. Stock Market: Since 2000, the stock market has soared, but 10% of Americans own 80% of stocks. The top 1% owns 38% of stocks. In 2007, 3/4th of middle class households were invested in the stock market, but now only 50% are investors. Overall, 52% of Americans now own stocks, which is down from 65%. The average American has less than $1,000 in their combined checking and savings accounts.

Do you know what this tells me?

It tells me that the bottom line is that Christmas has become a harder season for White families. We are worse off because of BOTH social and economic liberalism which has only benefited an elite few. The bottom half of the White population is now in total disarray – drug addiction, demoralization, divorce, suicide, abortion, atomization, stagnant wages, declining household income and investments – and this dysfunction is creeping up the social ladder. The worst thing we can do is step on the accelerator.

Paul Ryan and his fellow conservatives look at this and conclude we need MORE freedom. We need lower taxes, more free trade, more deregulation, weaker unions, more immigration and less social safety net spending. He wants to follow up tax reform with entitlement reform in 2018. I can't but see how this is going to make an already bad situation for the White working class even worse.

I'm not rightwing in the sense that these people are. I think their policies are harmful to the nation. I don't think they feel any sense of duty and obligation to the working class like we do. They believe in liberal abstractions and make an Ayn Rand fetish out of freedom whereas we feel a sense of solidarity with them grounded in race, ethnicity and culture which tempers class division. We recoil at the evisceration of the social fabric whereas conservatives celebrate this blind march toward plutocracy.

Do the wealthy need to own a greater share of the stock market? Do they need to own a greater share of our national wealth? Do we need to loosen up morals and the labor market? Do we need more White children growing up in financially stressed, broken homes on Christmas? Is the greatest problem facing the nation spending on anti-poverty programs? Paul Ryan and the True Cons think so.

Yeah, I don't think so. I also think it is a good thing right now that we aren't associated with the mainstream Right. In the long run, I bet this will pay off for us. I predict this platform they have been standing on for decades now, which they call the conservative base, is going to implode on them. Donald Trump was only the first sign that Atlas is about to shrug.

(Republished from Occidental Dissent by permission of author or representative)

[Dec 15, 2017] The Crisis Ahead The U.S. Is No Country for Older Men and Women

Notable quotes:
"... The U.S. has a retirement crisis on its hands, and with the far right controlling the executive branch and both houses of Congress, as well as dozens of state governments, things promise to grow immeasurably worse. ..."
"... It wasn't supposed to be this way. Past progressive presidents, notably Franklin D. Roosevelt and Lyndon B. Johnson, took important steps to make life more comfortable for aging Americans. FDR signed the Social Security Act of 1935 into law as part of his New Deal, and when LBJ passed Medicare in 1965, he established a universal health care program for those 65 and older. But the country has embraced a neoliberal economic model since the election of Ronald Reagan, and all too often, older Americans have been quick to vote for far-right Republicans antagonistic to the social safety net. ..."
"... Since then, Ryan has doubled down on his delusion that the banking sector can manage Social Security and Medicare more effectively than the federal government. Republican attacks on Medicare have become a growing concern: according to EBRI, only 38 percent of workers are confident the program will continue to provide the level of benefits it currently does. ..."
"... As 2017 winds down, Americans with health problems are still in the GOP's crosshairs -- this time because of so-called tax reform. The Tax Cuts and Jobs Act (both the House and Senate versions) includes provisions that would undermine Obamacare and cause higher health insurance premiums for older Americans. According to AARP, "Older adults ages 50-64 would be at particularly high risk under the proposal, facing average premium increases of up to $1,500 in 2019 as a result of the bill." ..."
"... Countless Americans who are unable to afford those steep premiums would lose their insurance. The CBO estimates that the Tax Cuts and Jobs Act would cause the number of uninsured under 65 to increase 4 million by 2019 and 13 million by 2027. The bill would also imperil Americans 65 and over by cutting $25 billion from Medicare . ..."
"... Analyzing W2 tax records in 2012, U.S. Census Bureau researchers Michael Gideon and Joshua Mitchell found that only 14 percent of private-sector employers in the U.S. were offering a 401(k) or similar retirement packages to their workers. That figure was thought to be closer to 40 percent, but Gideon and Mitchell discovered the actual number was considerably lower when smaller businesses were carefully analyzed, and that larger companies were more likely to offer 401(k) plans than smaller ones. ..."
"... Today, millions of Americans work in the gig economy who don't have full-time jobs or receive W2s, but instead receive 1099s for freelance work. ..."
"... The combination of stagnant wages and an increasingly high cost of living have been especially hellish for Americans who are trying to save for retirement. The United States' national minimum wage, a mere $7.25 per hour, doesn't begin to cover the cost of housing at a time when rents have soared nationwide. Never mind the astronomical prices in New York City, San Francisco or Washington, D.C. Median rents for one-bedroom apartments are as high as $1,010 per month in Atlanta, $960 per month in Baltimore, $860 per month in Jacksonville and $750 per month in Omaha, according to ApartmentList.com. ..."
"... yeah, Canada has a neoliberal infestation that is somewhere between the US and the UK. France has got one too, but it is less advanced. I'll enjoy my great healthcare, public transportation, and generous paid time off while I can. ..."
"... Europeans may scratch their heads, but they should recall their own histories and the long struggle to the universal benefits now enjoyed. Americans are far too complacent. This mildness is viewed by predators as weakness and the attacks will continue. ..."
"... Not sure if many of the readers here watch non-cable national broadcast news, but Pete Peterson and his foundation are as everpresent an advertiser as the pharma industry. Peterson is the strongest, best organized advocate for gutting social services, social security, and sending every last penny out of the tax-mule consumer's pocket toward wall street. The guy needs an equivalent counterpoint enemy. ..."
"... The social advantages that we still enjoy were fought in the streets, and on the "bricks" flowing with the participants blood. 8 hr. day; women's right to vote; ability and right for groups of laborers to organize; worker safety laws ..and so many others. There is no historical memory on how those rights were achieved. We are slowly slipping into an oligarchy greased by the idea that the physical possession of material things is all that matters. Sheeple, yes. ..."
"... Mmm, I think American voters get what they want in the end. They want their politicians because they believe the lies. 19% of Americans believe they are in the top 1% of wealth. A huge percentage of poor people believe they or their kids will (not can, but will) become wealthy. Most Americans can't find France on a map. ..."
"... I may have been gone for about thirty years, but that has only sharpened my insights into America. It's very hard to see just how flawed America is from the inside but when you step outside and have some perspective, it's frightening. ..."
"... Our government, beginning with Reagan, turned its back on promoting the general welfare. The wealthy soon learned that their best return on investment was the "purchase" of politicians willing to pass the legislation they put in their hands. Much of their investment included creating the right wing media apparatus. ..."
"... The Class War is real. It has been going on for 40 years, with the Conservative army facing virtually no resistance. Conservatives welcome Russia's help. Conservatives welcome barriers to people voting. Conservatives welcome a populace that believes lies that benefit them. Conservatives welcome the social and financial decline of the entire middle class and poor as long as it profits the rich financially, and by extension enhances their power politically. ..."
"... "Single acts of tyranny may be ascribed to the accidental opinion of the day, but a series of oppressions, begun at a distinguished period and pursued unalterably through every change of ministers, too plainly prove a deliberate, systematic plan of reducing [a people] to slavery" Thomas Jefferson. Rights of British America, 1774 ME 1:193, Papers 1:125 ..."
"... yes, my problem with the post as well, completely ignores democrat complicity the part where someone with a 26k salary will pay 16k in insurance? No they won't, the system would collapse in that case which will be fine with me. ..."
"... As your quote appears to imply, it's not a problem that can be solved by voting which, let's not forget, is nothing more than expressing an opinion. I am not sticking around just to find out if economically-crushed, opiod-, entertainment-, social media-addled Americans are actually capable of rolling out tumbrils for trips to the guillotines in the city squares. I strongly suspect not. ..."
"... This is the country where, after the banks crushed the economy in 2008, caused tens of thousands to lose their jobs, and then got huge bailouts, the people couldn't even be bothered to take their money out of the big banks and put it elsewhere. Because, you know, convenience! Expressing an opinion, or mobilizing others to express an opinion, or educating or proselytizing others about what opinion to have, is about the limit of what they are willing, or know how to do. ..."
Dec 14, 2017 | www.nakedcapitalism.com

Yves here. I imagine many readers are acutely aware of the problems outlined in this article, if not beset by them already. By any rational standard, I should move now to a much cheaper country that will have me. I know individuals who live most of the year in third-world and near-third world countries, but they have very cheap ways of still having a toehold in the US and not (yet or maybe ever) getting a long-term residence visa. Ecuador is very accommodating regarding retirement visas, and a Social Security level income goes far there, but yours truly isn't retiring any time soon. And another barrier to an international move (which recall I did once, so I have some appreciation for what it takes), is that one ought to check out possible destinations but if you are already time and money and energy stressed, how do you muster the resources to do that at all, let alone properly?

Aside from the potential to greatly reduce fixed costs, a second impetus for me is Medicare. I know for most people, getting on Medicare is a big plus. I have a very rare good, very old insurance policy. When you include the cost of drug plans, Medicare is no cheaper than what I have now, and considerably narrows my network. Moreover, I expect it to be thoroughly crapified by ten years from now (when I am 70), which argues for getting out of Dodge sooner rather than later.

And that's before you get to another wee problem Lambert points out that I would probably not be happy in a third world or high end second world country. But the only bargain "world city" I know of is Montreal. I'm not sure it would represent enough of an all-in cost saving to justify the hassle of an international move and the attendant tax compliance burdens .and that charitably assumes I could even find a way to get permanent residence. Ugh.

By Alex Henderson, who has written for the L.A. Weekly, Billboard, Spin, Creem, the Pasadena Weekly and many other publications. Follow him on Twitter @alexvhenderson. Originally published at Alternet

Millions can no longer afford to retire, and may never be able when the GOP passes its tax bill.

The news is not good for millions of aging Baby Boomers and Gen Xers in the United States who are moving closer to retirement age. According to the Employee Benefit Research Institute's annual report on retirement preparedness for 2017, only 18 percent of U.S.-based workers feel "very confident" about their ability to retire comfortably ; Craig Copeland, senior research associate for EBRI and the report's co-author, cited "debt, lack of a retirement plan at work, and low savings" as "key factors" in workers' retirement-related anxiety. The Insured Retirement Institute finds a mere 23 percent of Baby Boomers and 24 percent of Gen Xers are confident that their savings will last in retirement. To make matters worse, more than 40 percent of Boomers and over 30 percent of Gen Xers report having no retirement savings whatsoever .

The U.S. has a retirement crisis on its hands, and with the far right controlling the executive branch and both houses of Congress, as well as dozens of state governments, things promise to grow immeasurably worse.

It wasn't supposed to be this way. Past progressive presidents, notably Franklin D. Roosevelt and Lyndon B. Johnson, took important steps to make life more comfortable for aging Americans. FDR signed the Social Security Act of 1935 into law as part of his New Deal, and when LBJ passed Medicare in 1965, he established a universal health care program for those 65 and older. But the country has embraced a neoliberal economic model since the election of Ronald Reagan, and all too often, older Americans have been quick to vote for far-right Republicans antagonistic to the social safety net.

In the 2016 presidential election, 55 percent of voters 50 and older cast their ballots for Donald Trump against just 44 percent for Hillary Clinton. (This was especially true of older white voters; 90 percent of black voters 45 and older, as well as 67 percent of Latino voters in the same age range voted Democratic.)

Sen. Bernie Sanders' (I-VT) economic proposals may have been wildly popular with millennials, but no demographic has a greater incentive to vote progressive than Americans facing retirement. According to research conducted by the American Association of Retired Persons, the three greatest concerns of Americans 50 and older are Social Security, health care costs and caregiving for loved ones -- all areas that have been targeted by Republicans.

House of Representatives Speaker Paul Ryan, a devotee of social Darwinist Ayn Rand , has made no secret of his desire to privatize Social Security and replace traditional Medicare with a voucher program. Had George W. Bush had his way and turned Social Security over to Wall Street, the economic crash of September 2008 might have left millions of senior citizens homeless.

Since then, Ryan has doubled down on his delusion that the banking sector can manage Social Security and Medicare more effectively than the federal government. Republican attacks on Medicare have become a growing concern: according to EBRI, only 38 percent of workers are confident the program will continue to provide the level of benefits it currently does.

The GOP's obsession with abolishing the Affordable Care Act is the most glaring example of its disdain for aging Americans. Yet Obamacare has been a blessing for Boomers and Gen Xers who have preexisting conditions. The ACA's guaranteed issue plans make no distinction between a 52-year-old American with diabetes, heart disease or asthma and a 52-year-old who has never had any of those illnesses. And AARP notes that under the ACA, the uninsured rate for Americans 50 and older decreased from 15 percent in 2013 to 9 percent in 2016.

According to the Congressional Budget Office, the replacement bills Donald Trump hoped to ram through Congress this year would have resulted in staggering premium hikes for Americans over 50. The CBO's analysis of the American Health Care Act, one of the earlier versions of Trumpcare, showed that a 64-year-old American making $26,500 per year could have gone from paying $1,700 annually in premiums to just over $16,000. The CBO also estimated that the GOP's American Health Care Act would have deprived 23 million Americans of health insurance by 2026.

As 2017 winds down, Americans with health problems are still in the GOP's crosshairs -- this time because of so-called tax reform. The Tax Cuts and Jobs Act (both the House and Senate versions) includes provisions that would undermine Obamacare and cause higher health insurance premiums for older Americans. According to AARP, "Older adults ages 50-64 would be at particularly high risk under the proposal, facing average premium increases of up to $1,500 in 2019 as a result of the bill."

The CBO estimates that the bill will cause premiums to spike an average of 10 percent overall, with average premiums increasing $890 per year for a 50-year-old, $1,100 per year for a 55-year-old, $1,350 per year for a 60-year-old and $1,490 per year for a 64-year-old. Premium increases, according to the CBO, would vary from state to state; in Maine, average premiums for a 64-year-old would rise as much as $1,750 per year.

Countless Americans who are unable to afford those steep premiums would lose their insurance. The CBO estimates that the Tax Cuts and Jobs Act would cause the number of uninsured under 65 to increase 4 million by 2019 and 13 million by 2027. The bill would also imperil Americans 65 and over by cutting $25 billion from Medicare .

As morally reprehensible as the GOP's tax legislation may be, it is merely an acceleration of the redistribution of wealth from the bottom to the top that America has undergone since the mid-1970s. (President Richard Nixon may have been a paranoid right-winger with authoritarian tendencies, but he expanded Medicare and supported universal health care.) Between the decline of labor unions, age discrimination, stagnant wages, an ever-rising cost of living, low interest rates, and a shortage of retirement accounts, millions of Gen Xers and Baby Boomers may never be able to retire.

Traditional defined-benefit pensions were once a mainstay of American labor, especially among unionized workers. But according to Pew Charitable Trusts, only 13 percent of Baby Boomers still have them (among millennials, the number falls to 6 percent). In recent decades, 401(k) plans have become much more prominent, yet a majority of American workers don't have them either.

Analyzing W2 tax records in 2012, U.S. Census Bureau researchers Michael Gideon and Joshua Mitchell found that only 14 percent of private-sector employers in the U.S. were offering a 401(k) or similar retirement packages to their workers. That figure was thought to be closer to 40 percent, but Gideon and Mitchell discovered the actual number was considerably lower when smaller businesses were carefully analyzed, and that larger companies were more likely to offer 401(k) plans than smaller ones.

Today, millions of Americans work in the gig economy who don't have full-time jobs or receive W2s, but instead receive 1099s for freelance work. Tax-deferred SEP-IRAs were once a great, low-risk way for freelancers to save for retirement without relying exclusively on Social Security, but times have changed since the 1980s and '90s when interest rates were considerably higher for certificates of deposit and savings accounts. According to Bankrate.com, average rates for one-year CDs dropped from 11.27 percent in 1984 to 8.1 percent in 1990 to 5.22 percent in 1995 to under 1 percent in 2010, where it currently remains.

The combination of stagnant wages and an increasingly high cost of living have been especially hellish for Americans who are trying to save for retirement. The United States' national minimum wage, a mere $7.25 per hour, doesn't begin to cover the cost of housing at a time when rents have soared nationwide. Never mind the astronomical prices in New York City, San Francisco or Washington, D.C. Median rents for one-bedroom apartments are as high as $1,010 per month in Atlanta, $960 per month in Baltimore, $860 per month in Jacksonville and $750 per month in Omaha, according to ApartmentList.com.

That so many older Americans are renting at all is ominous in its own right. FDR made home ownership a primary goal of the New Deal, considering it a key component of a thriving middle class. But last year, the Urban Institute found that 19 million Americans who previously owned a home are now renting, 31 percent between the ages of 36 and 45. Laurie Goodman, one of the study's authors, contends the Great Recession has "permanently raised the number of renters," and that the explosion of foreclosures has hit Gen Xers especially hard.

The severity of the U.S. retirement crisis is further addressed in journalist Jessica Bruder's new book "Nomadland: Surviving America in the 21st Century," which follows Americans in their 50s, 60s and even 70s living in RVs or vans , barely eking out a living doing physically demanding, seasonal temp work from harvesting sugar beets to cleaning toilets at campgrounds. Several had high-paying jobs before their lives were blown apart by the layoffs, foreclosures and corporate downsizing of the Great Recession. Bruder speaks with former college professors and software professionals who now find themselves destitute, teetering on the brink of homelessness and forced to do backbreaking work for next to nothing. Unlike the big banks, they never received a bailout.

These neo-nomads recall the transients of the 1930s, themselves victims of Wall Street's recklessness. But whereas FDR won in a landslide in 1932 and aggressively pursued a program of progressive economic reforms, Republicans in Congress have set out to shred what little remains of the social safety net, giving huge tax breaks to millionaires and billionaires . The older voters who swept Trump into office may have signed their own death warrants.

If aging Americans are going to be saved from this dystopian future, the U.S. will have to forge a new Great Society. Programs like Social Security, Medicare and Medicaid will need to be strengthened, universal health care must become a reality and age discrimination in the workplace will have to be punished as a civil rights violation like racial and gender-based discrimination. If not, millions of Gen Xers and Boomers will spend their golden years scraping for pennies.

Expat , , December 14, 2017 at 6:29 am

I certainly will never go back to the States for these and other reasons. I have a friend, also an American citizen, who travels frequently back to California to visit his son. He is truly worried about getting sick or having an accident when he is there since he knows it might bankrupt him. As he jokes, he would be happy to have another heart attack here in France since it's free!

For those of you who have traveled the world and talked to people, you probably know that most foreigners are perplexed by America's attitude to health care and social services. The richest nation in the world thinks that health and social security (in the larger sense of not being forced into the street) are not rights at all. Europeans scratch their heads at this.

The only solution is education and information, but they are appalling in America. America remains the most ignorant and worst educated of the developed nations and is probably beaten by many developing nations. It is this ignorance and stupidity that gets Americans to vote for the likes of Trump or any of the other rapacious millionaires they send to office every year.

A first step would be for Americans to insist that Congress eliminate its incredibly generous and life-long healthcare plans for elected officials. They should have to do what the rest of Americans do. Of course, since about 95% of Congress are millionaires, it might not be effective. But it's a start.

vidimi , , December 14, 2017 at 6:40 am

France has its share of problems, but boy do they pale next to the problems in America or even Canada. Life here is overall quite pleasant and I have no desire to go back to N.A.

Marco , , December 14, 2017 at 6:46 am

Canada has problems?

WobblyTelomeres , , December 14, 2017 at 7:47 am

Was in Yellowknife a couple of years ago. The First Nations people have a rough life. From what I've read, such extends across the country.

vidimi , , December 14, 2017 at 8:03 am

yeah, Canada has a neoliberal infestation that is somewhere between the US and the UK. France has got one too, but it is less advanced. I'll enjoy my great healthcare, public transportation, and generous paid time off while I can.

JEHR , , December 14, 2017 at 1:46 pm

The newest neoliberal effort in Canada was put forward by our Minister of Finance (a millionaire) who is touting a bill that will get rid of defined benefit pension plans given to public employees for so-called target benefit pension plans. The risk for target plans is taken by the recipient. Morneau's former firm promotes target benefit pension plans and the change could benefit Morneau himself as he did not put his assets from his firm in a blind trust. At the very least, he has a conflict of interest and should probably resign.

There is always an insidious group of wealthy people here who would like to re-make the world in their own image. I fear for the future.

JEHR , , December 14, 2017 at 1:55 pm

Yes, I agree. There is an effort to "simplify" the financial system of the EU to take into account the business cycle and the financial cycle .

Dita , , December 14, 2017 at 8:25 am

Europeans may scratch their heads, but they should recall their own histories and the long struggle to the universal benefits now enjoyed. Americans are far too complacent. This mildness is viewed by predators as weakness and the attacks will continue.

jefemt , , December 14, 2017 at 10:02 am

We really should be able to turn this around, and have an obligation to ourselves and our 'nation state' , IF there were a group of folks running on a fairness, one-for-all, all-for-one platform. That sure isn't the present two-sides-of-the-same-coin Democraps and Republicrunts.

Not sure if many of the readers here watch non-cable national broadcast news, but Pete Peterson and his foundation are as everpresent an advertiser as the pharma industry. Peterson is the strongest, best organized advocate for gutting social services, social security, and sending every last penny out of the tax-mule consumer's pocket toward wall street. The guy needs an equivalent counterpoint enemy.

Check it out, and be vigilant in dispelling his message and mission. Thanks for running this article.

Running away: the almost-haves run to another nation state, the uber-wealthy want to leave the earth, or live in their private Idaho in the Rockies or on the Ocean. What's left for the least among us? Whatever we create?
https://www.pgpf.org/

Scramjett , , December 14, 2017 at 1:43 pm

I think pathologically optimistic is a better term than complacent. Every time someone dumps on them, their response is usually along the lines of "Don't worry, it'll get better," "Everything works itself out in the end," "maybe we'll win the lottery," my personal favorite "things will get better, just give it time" (honestly it's been 40 years of this neoliberal bullcrap, how much more time are we supposed to give it?), "this is just a phase" or "we can always bring it back later and better than ever." The last one is most troubling because after 20 years of witnessing things in the public sphere disappearing, I've yet to see a single thing return in any form at all.

I'm not sure where this annoying optimism came from but I sure wish it would go away.

sierra7 , , December 14, 2017 at 8:45 pm

The "optimism" comes from having a lack of historical memory. So many social protections that we have/had is seen as somehow coming out of the ether benevolently given without any social struggles. The lack of historical education on this subject in particular is appalling. Now, most would probably look for an "APP" on their "dumbphones" to solve the problem.

The social advantages that we still enjoy were fought in the streets, and on the "bricks" flowing with the participants blood. 8 hr. day; women's right to vote; ability and right for groups of laborers to organize; worker safety laws ..and so many others. There is no historical memory on how those rights were achieved. We are slowly slipping into an oligarchy greased by the idea that the physical possession of material things is all that matters. Sheeple, yes.

Jeremy Grimm , , December 14, 2017 at 4:44 pm

WOW! You must have been outside the U.S. for a long time. Your comment seems to suggest we still have some kind of democracy here. We don't get to pick which rapacious millionaires we get to vote for and it doesn't matter any way since whichever one we pick from the sad offerings ends up with policies dictated from elsewhere.

Expat , , December 14, 2017 at 6:10 pm

Mmm, I think American voters get what they want in the end. They want their politicians because they believe the lies. 19% of Americans believe they are in the top 1% of wealth. A huge percentage of poor people believe they or their kids will (not can, but will) become wealthy. Most Americans can't find France on a map.

So, yes, you DO get to pick your rapacious millionaire. You send the same scumbags back to Washington every year because it's not him, it the other guys who are the problem. One third of Americans support Trump! Really, really support him. They think he is Jesus, MacArthur and Adam Smith all rolled up into one.

I may have been gone for about thirty years, but that has only sharpened my insights into America. It's very hard to see just how flawed America is from the inside but when you step outside and have some perspective, it's frightening.

Disturbed Voter , , December 14, 2017 at 6:29 am

The Democrat party isn't a reform party. Thinking it is so, is because of the "No Other Choice" meme. Not saying that the Republican party works in my favor. They don't. Political reform goes deeper than reforming either main party. It means going to a European plurality system (with its own downside). That way growing Third parties will be viable, if they have popular, as opposed to millionaire, support. I don't see this happening, because of Citizens United, but if all you have is hope, then you have to go with that.

Carolinian , , December 14, 2017 at 8:05 am

Had George W. Bush had his way and turned Social Security over to Wall Street, the economic crash of September 2008 might have left millions of senior citizens homeless.

Substitute Bill Clinton for George Bush in that sentence and it works just as well. Neoliberalism is a bipartisan project.

And many of the potential and actual horrors described above arise from the price distortions of the US medical system with Democratic acquiescence in said system making things worse. The above article reads like a DNC press release.

And finally while Washington politicians of both parties have been threatening Social Security for years that doesn't mean its third rail status has been repealed. The populist tremors of the last election -- which have caused our elites to lose their collective mind -- could be a mere prelude to what will happen in the event of a full scale assault on the safety net.

KYrocky , , December 14, 2017 at 12:05 pm

Substitute Obama's quest for a Grand Bargain as well.

Our government, beginning with Reagan, turned its back on promoting the general welfare. The wealthy soon learned that their best return on investment was the "purchase" of politicians willing to pass the legislation they put in their hands. Much of their investment included creating the right wing media apparatus.

The Class War is real. It has been going on for 40 years, with the Conservative army facing virtually no resistance. Conservatives welcome Russia's help. Conservatives welcome barriers to people voting. Conservatives welcome a populace that believes lies that benefit them. Conservatives welcome the social and financial decline of the entire middle class and poor as long as it profits the rich financially, and by extension enhances their power politically.

If retirees flee our country that will certainly please the Conservatives as that will be fewer critics (enemies). Also less need or demand for social programs.

rps , , December 14, 2017 at 5:01 pm

"Single acts of tyranny may be ascribed to the accidental opinion of the day, but a series of oppressions, begun at a distinguished period and pursued unalterably through every change of ministers, too plainly prove a deliberate, systematic plan of reducing [a people] to slavery" Thomas Jefferson. Rights of British America, 1774 ME 1:193, Papers 1:125

tegnost , , December 14, 2017 at 8:59 am

yes, my problem with the post as well, completely ignores democrat complicity the part where someone with a 26k salary will pay 16k in insurance? No they won't, the system would collapse in that case which will be fine with me.

Marco , , December 14, 2017 at 6:55 am

"President Richard Nixon may have been a paranoid right-winger with authoritarian tendencies, but he expanded Medicare and supported universal health care."

"Gimme that old time Republican!"

One of the reasons I love NC is that most political economic analysis is often more harsh on the Democrats than the Repubs so I am a bit dismayed how this article is way too easy on Team D. How many little (and not so little) knives in the back from Clinton and Obama? Is a knife in the chest that much worse?

OpenThePodBayDoorsHAL , December 14, 2017 at 3:57 pm

This entire thread is simply heartbreaking, Americans have had their money, their freedom, their privacy, their health, and sometimes their very lives taken away from them by the State. But the heartbreaking part is that they feel they are powerless to do anything at all about it so are just trying to leave.

But "People should not fear the government; the government should fear the people"

tagio , December 14, 2017 at 4:39 pm

It's more than a feeling, HAL. https://www.newyorker.com/news/john-cassidy/is-america-an-oligarchy Link to the academic paper embedded in article.

As your quote appears to imply, it's not a problem that can be solved by voting which, let's not forget, is nothing more than expressing an opinion. I am not sticking around just to find out if economically-crushed, opiod-, entertainment-, social media-addled Americans are actually capable of rolling out tumbrils for trips to the guillotines in the city squares. I strongly suspect not.

This is the country where, after the banks crushed the economy in 2008, caused tens of thousands to lose their jobs, and then got huge bailouts, the people couldn't even be bothered to take their money out of the big banks and put it elsewhere. Because, you know, convenience! Expressing an opinion, or mobilizing others to express an opinion, or educating or proselytizing others about what opinion to have, is about the limit of what they are willing, or know how to do.

[Dec 13, 2017] A stunning 33% of job seekers ages 55 and older are long-term unemployed, according to the AARP Public Policy Institute

Notable quotes:
"... And, recent studies have shown, the longer you're out of work - especially if you're older and out of work - the harder it becomes to get a job offer. ..."
Dec 13, 2017 | www.nakedcapitalism.com

Livius Drusus , December 13, 2017 at 2:44 pm

I thought this was an interesting article. Apologies if this has been posted on NC already.

A stunning 33% of job seekers ages 55 and older are long-term unemployed, according to the AARP Public Policy Institute. The average length of unemployment for the roughly 1.2 million people 55+ who are out of work: seven to nine months. "It's emotionally devastating for them," said Carl Van Horn, director of Rutgers University's John J. Heldrich Center for Workforce Development, at a Town Hall his center and the nonprofit WorkingNation held earlier this year in New Brunswick, N.J.

... ... ...

The fight faced by the long-term unemployed

And, recent studies have shown, the longer you're out of work - especially if you're older and out of work - the harder it becomes to get a job offer.

The job-finding rate declines by roughly 50% within eight months of unemployment, according to a 2016 paper by economists Gregor Jarosch of Stanford University and Laura Pilossoph of the Federal Reserve Bank of New York. "Unemployment duration has a strongly negative effect on the likelihood of subsequent employment," wrote researchers from the University of Maryland and the U.S. Census Bureau in another 2016 paper.

"Once upon a time, you could take that first job and it would lead to the next job and the job after that," said Town Hall panelist John Colborn, chief operating officer at the nonprofit JEVS Human Services, of Philadelphia. "The notion of a career ladder offered some hope of getting back into the labor market. The rungs of the ladder are getting harder and harder to find and some of them are broken."

In inner cities, said Kimberly McClain, CEO of The Newark Alliance, "there's an extra layer beyond being older and out of work. There are issues of race and poverty and being defined by your ZIP Code. There's an incredible sense of urgency."

... ... ...

Filling a work gap

If you are over 50, unemployed and have a work gap right now, the Town Hall speakers said, fill it by volunteering, getting an internship, doing project work, job-shadowing someone in a field you want to be in or taking a class to re-skill. These kind of things "make a candidate a lot more attractive," said Colborn. Be sure to note them in your cover letter and résumé.

Town Hall panelist Amanda Mullan, senior vice president and chief human resources officer of the New Jersey Resources Corp. (a utility company based in Wall, N.J.), said that when her company is interviewing someone who has been out of work lately, "we will ask: 'What have you done during that time frame?' If we get 'Nuthin,' that shows something about the individual, from a motivational perspective."

... ... ...

The relief of working again

Finally finding work when you're over 50 and unemployed for a stretch can be a relief for far more than financial reasons.

"Once I landed my job, the thing I most looked forward to was the weekend," said Konopka. "Not to relax, but because I didn't have to think about finding a job anymore. That's 24/7 in your head. You're always thinking on a Saturday: 'If I'm not doing something to find a job, will there be a posting out there?'"

Full article: https://www.marketwatch.com/story/jobs-are-everywhere-just-not-for-people-over-55-2017-12-08

[Dec 13, 2017] Stress of long-term unemployment takes a toll on thousands of Jerseyans who are out of work by Leslie Kwoh

Notable quotes:
"... Leslie Kwoh may be reached at lkwoh@starledger.com or (973) 392-4147. ..."
Jun 13, 2010 | www.nj.com

At 5:30 every morning, Tony Gwiazdowski rolls out of bed, brews a pot of coffee and carefully arranges his laptop, cell phone and notepad like silverware across the kitchen table.

And then he waits.

Gwiazdowski, 57, has been waiting for 16 months. Since losing his job as a transportation sales manager in February 2009, he wakes each morning to the sobering reminder that, yes, he is still unemployed. So he pushes aside the fatigue, throws on some clothes and sends out another flurry of resumes and cheery cover letters.

But most days go by without a single phone call. And around sundown, when he hears his neighbors returning home from work, Gwiazdowski -- the former mayor of Hillsborough -- can't help but allow himself one tiny sigh of resignation.

"You sit there and you wonder, 'What am I doing wrong?'" said Gwiazdowski, who finds companionship in his 2-year-old golden retriever, Charlie, until his wife returns from work.

"The worst moment is at the end of the day when it's 4:30 and you did everything you could, and the phone hasn't rung, the e-mails haven't come through."

Gwiazdowski is one of a growing number of chronically unemployed workers in New Jersey and across the country who are struggling to get through what is becoming one long, jobless nightmare -- even as the rest of the economy has begun to show signs of recovery.

Nationwide, 46 percent of the unemployed -- 6.7 million Americans -- have been without work for at least half a year, by far the highest percentage recorded since the U.S. Labor Department began tracking the data in 1948.

In New Jersey, nearly 40 percent of the 416,000 unemployed workers last year fit that profile, up from about 20 percent in previous years, according to the department, which provides only annual breakdowns for individual states. Most of them were unemployed for more than a year.

But the repercussions of chronic unemployment go beyond the loss of a paycheck or the realization that one might never find the same kind of job again. For many, the sinking feeling of joblessness -- with no end in sight -- can take a psychological toll, experts say.

Across the state, mental health crisis units saw a 20 percent increase in demand last year as more residents reported suffering from unemployment-related stress, according to the New Jersey Association of Mental Health Agencies.

"The longer the unemployment continues, the more impact it will have on their personal lives and mental health," said Shauna Moses, the association's associate executive director. "There's stress in the marriage, with the kids, other family members, with friends."

And while a few continue to cling to optimism, even the toughest admit there are moments of despair: Fear of never finding work, envy of employed friends and embarassment at having to tell acquaintances that, nope, still no luck.

"When they say, 'Hi Mayor,' I don't tell a lot of people I'm out of work -- I say I'm semi-retired," said Gwiazdowski, who maxed out on unemployment benefits several months ago.

"They might think, 'Gee, what's wrong with him? Why can't he get a job?' It's a long story and maybe people really don't care and now they want to get away from you."


SECOND TIME AROUND

Lynn Kafalas has been there before, too. After losing her computer training job in 2000, the East Hanover resident took four agonizing years to find new work -- by then, she had refashioned herself into a web designer.

That not-too-distant experience is why Kafalas, 52, who was laid off again eight months ago, grows uneasier with each passing day. Already, some of her old demons have returned, like loneliness, self-doubt and, worst of all, insomnia. At night, her mind races to dissect the latest interview: What went wrong? What else should she be doing? And why won't even Barnes & Noble hire her?

"It's like putting a stopper on my life -- I can't move on," said Kafalas, who has given up karate lessons, vacations and regular outings with friends. "Everything is about the interviews."

And while most of her friends have been supportive, a few have hinted to her that she is doing something wrong, or not doing enough. The remarks always hit Kafalas with a pang.

In a recent study, researchers at Rutgers University found that the chronically unemployed are prone to high levels of stress, anxiety, depression, loneliness and even substance abuse, which take a toll on their self-esteem and personal relationships.

"They're the forgotten group," said Carl Van Horn, director of the John J. Heldrich Center for Workforce Development at Rutgers, and a co-author of the report. "And the longer you are unemployed, the less likely you are to get a job."

Of the 900 unemployed workers first interviewed last August for the study, only one in 10 landed full-time work by March of this year, and only half of those lucky few expressed satisfaction with their new jobs. Another one in 10 simply gave up searching.

Among those who were still unemployed, many struggled to make ends meet by borrowing from friends or family, turning to government food stamps and forgoing health care, according to the study.

More than half said they avoided all social contact, while slightly less than half said they had lost touch with close friends. Six in 10 said they had problems sleeping.

Kafalas says she deals with her chronic insomnia by hitting the gym for two hours almost every evening, lifting weights and pounding the treadmill until she feels tired enough to fall asleep.

"Sometimes I forget what day it is. Is it Tuesday? And then I'll think of what TV show ran the night before," she said. "Waiting is the toughest part."


AGE A FACTOR

Generally, the likelihood of long-term unemployment increases with age, experts say. A report by the National Employment Law Project this month found that nearly half of those who were unemployed for six months or longer were at least 45 years old. Those between 16 and 24 made up just 14 percent.

Tell that to Adam Blank, 24, who has been living with his girlfriend and her parents at their Martinsville home since losing his sales job at Best Buy a year and half ago.

Blank, who graduated from Rutgers with a major in communications, says he feels like a burden sometimes, especially since his girlfriend, Tracy Rosen, 24, works full-time at a local nonprofit. He shows her family gratitude with small chores, like taking out the garbage, washing dishes, sweeping floors and doing laundry.

Still, he often feels inadequate.

"All I'm doing on an almost daily basis is sitting around the house trying to keep myself from going stir-crazy," said Blank, who dreams of starting a social media company.

When he is feeling particularly low, Blank said he turns to a tactic employed by prisoners of war in Vietnam: "They used to build dream houses in their head to help keep their sanity. It's really just imagining a place I can call my own."


LESSONS LEARNED

Meanwhile, Gwiazdowski, ever the optimist, says unemployment has taught him a few things.

He has learned, for example, how to quickly assess an interviewer's age and play up or down his work experience accordingly -- he doesn't want to appear "threatening" to a potential employer who is younger. He has learned that by occasionally deleting and reuploading his resume to job sites, his entry appears fresh.

"It's almost like a game," he said, laughing. "You are desperate, but you can't show it."

But there are days when he just can't find any humor in his predicament -- like when he finishes a great interview but receives no offer, or when he hears a fellow job seeker finally found work and feels a slight twinge of jealousy.

"That's what I'm missing -- putting on that shirt and tie in the morning and going to work," he said.

The memory of getting dressed for work is still so vivid, Gwiazdowski says, that he has to believe another job is just around the corner.

"You always have to hope that that morning when you get up, it's going to be the day," he said.

"Today is going to be the day that something is going to happen."

Leslie Kwoh may be reached at lkwoh@starledger.com or (973) 392-4147.

DrBuzzard Jun 13, 2010

I collect from the state of iowa, was on tier I and when the gov't recessed without passing extension, iowa stopped paying tier I claims that were already open, i was scheduled to be on tier I until july 15th, and its gone now, as a surprise, when i tried to claim my week this week i was notified. SURPRISE, talk about stress.

berganliz Jun 13, 2010

This is terrible....just wait until RIF'd teachers hit the unemployment offices....but then, this is what NJ wanted...fired teachers who are to blame for the worst recession our country has seen in 150 years...thanks GWB.....thanks Donald Rumsfeld......thanks Dick Cheney....thanks Karl "Miss Piggy" Rove...and thank you Mr. Big Boy himself...Gov Krispy Kreame!

rp121 Jun 13, 2010

For readers who care about this nation's unemployed- Call your Senators to pass HR 4213, the "Extenders" bill. Unfortunately, it does not add UI benefits weeks, however it DOES continue the emergency federal tiers of UI. If it does not pass this week many of us are cut off at 26 wks. No tier 1, 2 -nothing.

[Dec 13, 2017] Unemployment health hazard and stress

The longer you are unemployed, the more you are effected by those factors.
Notable quotes:
"... The good news is that only a relatively small number of people are seriously affected by the stress of unemployment to the extent they need medical assistance. Most people don't get to the serious levels of stress, and much as they loathe being unemployed, they suffer few, and minor, ill effects. ..."
"... Worries about income, domestic problems, whatever, the list is as long as humanity. The result of stress is a strain on the nervous system, and these create the physical effects of the situation over time. The chemistry of stress is complex, but it can be rough on the hormonal system. ..."
"... Not at all surprisingly, people under stress experience strong emotions. It's a perfectly natural response to what can be quite intolerable emotional strains. It's fair to say that even normal situations are felt much more severely by people already under stress. Things that wouldn't normally even be issues become problems, and problems become serious problems. Relationships can suffer badly in these circumstances, and that, inevitably, produces further crises. Unfortunately for those affected, these are by now, at this stage, real crises. ..."
"... Some people are stubborn enough and tough enough mentally to control their emotions ruthlessly, and they do better under these conditions. Even that comes at a cost, and although under control, the stress remains a problem. ..."
"... One of the reasons anger management is now a growth industry is because of the growing need for assistance with severe stress over the last decade. This is a common situation, and help is available. ..."
"... Depression is universally hated by anyone who's ever had it. ..."
"... Very important: Do not, under any circumstances, try to use drugs or alcohol as a quick fix. They make it worse, over time, because they actually add stress. Some drugs can make things a lot worse, instantly, too, particularly the modern made-in-a-bathtub variety. They'll also destroy your liver, which doesn't help much, either. ..."
"... You don't have to live in a gym to get enough exercise for basic fitness. A few laps of the pool, a good walk, some basic aerobic exercises, you're talking about 30-45 minutes a day. It's not hard. ..."
Dec 13, 2017 | www.cvtips.com

It's almost impossible to describe the various psychological impacts, because there are so many. There are sometimes serious consequences, including suicide, and, some would say worse, chronic depression.

There's not really a single cause and effect. It's a compound effect, and unemployment, by adding stress, affects people, often badly.

The world doesn't need any more untrained psychologists, and we're not pretending to give medical advice. That's for professionals. Everybody is different, and their problems are different. What we can do is give you an outline of the common problems, and what you can do about them.

The good news is that only a relatively small number of people are seriously affected by the stress of unemployment to the extent they need medical assistance. Most people don't get to the serious levels of stress, and much as they loathe being unemployed, they suffer few, and minor, ill effects.

For others, there are a series of issues, and the big three are:

Stress

Stress is Stage One. It's a natural result of the situation. Worries about income, domestic problems, whatever, the list is as long as humanity. The result of stress is a strain on the nervous system, and these create the physical effects of the situation over time. The chemistry of stress is complex, but it can be rough on the hormonal system.

Over an extended period, the body's natural hormonal balances are affected, and this can lead to problems. These are actually physical issues, but the effects are mental, and the first obvious effects are, naturally, emotional.

Anger, and other negative emotions

Not at all surprisingly, people under stress experience strong emotions. It's a perfectly natural response to what can be quite intolerable emotional strains. It's fair to say that even normal situations are felt much more severely by people already under stress. Things that wouldn't normally even be issues become problems, and problems become serious problems. Relationships can suffer badly in these circumstances, and that, inevitably, produces further crises. Unfortunately for those affected, these are by now, at this stage, real crises.

If the actual situation was already bad, this mental state makes it a lot worse. Constant aggravation doesn't help people to keep a sense of perspective. Clear thinking isn't easy when under constant stress.

Some people are stubborn enough and tough enough mentally to control their emotions ruthlessly, and they do better under these conditions. Even that comes at a cost, and although under control, the stress remains a problem.

One of the reasons anger management is now a growth industry is because of the growing need for assistance with severe stress over the last decade. This is a common situation, and help is available.

If you have reservations about seeking help, bear in mind it can't possibly be any worse than the problem.

Depression

Depression is universally hated by anyone who's ever had it. This is the next stage, and it's caused by hormonal imbalances which affect serotonin. It's actually a physical problem, but it has mental effects which are sometimes devastating, and potentially life threatening.

The common symptoms are:

It's a disgusting experience. No level of obscenity could possibly describe it. Depression is misery on a level people wouldn't conceive in a nightmare. At this stage the patient needs help, and getting it is actually relatively easy. It's convincing the person they need to do something about it that's difficult. Again, the mental state is working against the person. Even admitting there's a problem is hard for many people in this condition.

Generally speaking, a person who is trusted is the best person to tell anyone experiencing the onset of depression to seek help. Important: If you're experiencing any of those symptoms:

Very important: Do not, under any circumstances, try to use drugs or alcohol as a quick fix. They make it worse, over time, because they actually add stress. Some drugs can make things a lot worse, instantly, too, particularly the modern made-in-a-bathtub variety. They'll also destroy your liver, which doesn't help much, either.

Alcohol, in particular, makes depression much worse. Alcohol is a depressant, itself, and it's also a nasty chemical mix with all those stress hormones.

If you've ever had alcohol problems, or seen someone with alcohol wrecking their lives, depression makes things about a million times worse.

Just don't do it. Steer clear of any so-called stimulants, because they don't mix with antidepressants, either.

Unemployment and staying healthy

The above is what you need to know about the risks of unemployment to your health and mental well being.

These situations are avoidable.

Your best defense against the mental stresses and strains of unemployment, and their related problems is staying healthy.

We can promise you that is nothing less than the truth. The healthier you are, the better your defenses against stress, and the more strength you have to cope with situations.

Basic health is actually pretty easy to achieve:

Diet

Eat real food, not junk, and make sure you're getting enough food. Your body can't work with resources it doesn't have. Good food is a real asset, and you'll find you don't get tired as easily. You need the energy reserves.

Give yourself a good selection of food that you like, that's also worth eating.

The good news is that plain food is also reasonably cheap, and you can eat as much as you need. Basic meals are easy enough to prepare, and as long as you're getting all the protein veg and minerals you need, you're pretty much covered.

You can also use a multivitamin cap, or broad spectrum supplements, to make sure you're getting all your trace elements. Also make sure you're getting the benefits of your food by taking acidophilus or eating yogurt regularly.

Exercise

You don't have to live in a gym to get enough exercise for basic fitness. A few laps of the pool, a good walk, some basic aerobic exercises, you're talking about 30-45 minutes a day. It's not hard.

Don't just sit and suffer

If anything's wrong, check it out when it starts, not six months later. Most medical conditions become serious when they're allowed to get worse.

For unemployed people the added risk is also that they may prevent you getting that job, or going for interviews. If something's causing you problems, get rid of it.

Nobody who's been through the blender of unemployment thinks it's fun.

Anyone who's really done it tough will tell you one thing:

Don't be a victim. Beat the problem, and you'll really appreciate the feeling.

[Dec 13, 2017] Being homeless is better than working for Amazon by Nichole Gracely

Notable quotes:
"... According to Amazon's metrics, I was one of their most productive order pickers -- I was a machine, and my pace would accelerate throughout the course of a shift. What they didn't know was that I stayed fast because if I slowed down for even a minute, I'd collapse from boredom and exhaustion ..."
"... toiling in some remote corner of the warehouse, alone for 10 hours, with my every move being monitored by management on a computer screen. ..."
"... ISS could simply deactivate a worker's badge and they would suddenly be out of work. They treated us like beggars because we needed their jobs. Even worse, more than two years later, all I see is: Jeff Bezos is hiring. ..."
"... I have never felt more alone than when I was working there. I worked in isolation and lived under constant surveillance ..."
"... That was 2012 and Amazon's labor and business practices were only beginning to fall under scrutiny. ..."
"... I received $200 a week for the following six months and I haven't had any source of regular income since those benefits lapsed. I sold everything in my apartment and left Pennsylvania as fast as I could. I didn't know how to ask for help. I didn't even know that I qualified for food stamps. ..."
Nov 28, 2014 | theguardian.com

wa8dzp:

Nichole Gracely has a master's degree and was one of Amazon's best order pickers. Now, after protesting the company, she's homeless.

I am homeless. My worst days now are better than my best days working at Amazon.

According to Amazon's metrics, I was one of their most productive order pickers -- I was a machine, and my pace would accelerate throughout the course of a shift. What they didn't know was that I stayed fast because if I slowed down for even a minute, I'd collapse from boredom and exhaustion.

During peak season, I trained incoming temps regularly. When that was over, I'd be an ordinary order picker once again, toiling in some remote corner of the warehouse, alone for 10 hours, with my every move being monitored by management on a computer screen.

Superb performance did not guarantee job security. ISS is the temp agency that provides warehouse labor for Amazon and they are at the center of the SCOTUS case Integrity Staffing Solutions vs. Busk. ISS could simply deactivate a worker's badge and they would suddenly be out of work. They treated us like beggars because we needed their jobs. Even worse, more than two years later, all I see is: Jeff Bezos is hiring.

I have never felt more alone than when I was working there. I worked in isolation and lived under constant surveillance. Amazon could mandate overtime and I would have to comply with any schedule change they deemed necessary, and if there was not any work, they would send us home early without pay. I started to fall behind on my bills.

At some point, I lost all fear. I had already been through hell. I protested Amazon. The gag order was lifted and I was free to speak. I spent my last days in a lovely apartment constructing arguments on discussion boards, writing articles and talking to reporters. That was 2012 and Amazon's labor and business practices were only beginning to fall under scrutiny. I walked away from Amazon's warehouse and didn't have any other source of income lined up.

I cashed in on my excellent credit, took out cards, and used them to pay rent and buy food because it would be six months before I could receive my first unemployment compensation check.

I received $200 a week for the following six months and I haven't had any source of regular income since those benefits lapsed. I sold everything in my apartment and left Pennsylvania as fast as I could. I didn't know how to ask for help. I didn't even know that I qualified for food stamps.

I furthered my Amazon protest while homeless in Seattle. When the Hachette dispute flared up I "flew a sign," street parlance for panhandling with a piece of cardboard: "I was an order picker at amazon.com. Earned degrees. Been published. Now, I'm homeless, writing and doing this. Anything helps."

I have made more money per word with my signs than I will probably ever earn writing, and I make more money per hour than I will probably ever be paid for my work. People give me money and offer well wishes and I walk away with a restored faith in humanity.

I flew my protest sign outside Whole Foods while Amazon corporate employees were on lunch break, and they gawked. I went to my usual flying spots around Seattle and made more money per hour protesting Amazon with my sign than I did while I worked with them. And that was in Seattle. One woman asked, "What are you writing?" I told her about the descent from working poor to homeless, income inequality, my personal experience. She mentioned Thomas Piketty's book, we chatted a little, she handed me $10 and wished me luck. Another guy said, "Damn, that's a great story! I'd read it," and handed me a few bucks.

[snip]

[Dec 13, 2017] Business Staff brand colleagues as 'lazy'

While lazy people do happen, this compulsive quest for "high performance" is one of the most disgusting futures of neoliberlaism. Cemented by annual "performance reviews" which are the scam.
Aug 19, 2005 | BBC NEWS
An overwhelming majority of bosses and employees think that some of their colleagues consistently underperform.

An Investors in People survey found 75% of bosses and 80% of staff thought some colleagues were "dead wood" - and the main reason was thought to be laziness. Nearly half of employees added they worked closely with someone who they thought was lazy and not up to the job. However, four out of ten workers said that their managers did nothing about colleagues not pulling their weight.

According to Investors in People, the problem of employees not doing their jobs properly seemed to be more prevalent in larger organizations. The survey found that 84% of workers in organizations with more than 1,000 employees thought they had an underperforming colleague, compared with 50% in firms with fewer than 50 staff.

Tell tale signs

The survey identified the tell-tale signs of people not pulling their weight, according to both employers and employees, including:

Both employers and employees agreed that the major reason for someone failing in their job was sheer laziness. "Dead wood" employees can have a stark effect on their colleagues' physical and mental well-being, the survey found. Employees reported that they had to work longer hours to cover for shirking colleagues and felt undervalued as a result. Ultimately, working alongside a lazy colleague could prompt workers to look for a new job the survey found.

But according to Nick Parfitt, spokesman for human resources firm Cubiks, an unproductive worker isn't necessarily lazy.

"It can be too easy to brand a colleague lazy," he said. "They may have genuine personal problems or are being asked to do a job that they have not been given the training to do. "The employer must look out for the warning signs of a worker becoming de-motivated - hold regular conversations and appraisals with staff."

However, Mr Parfitt added that ultimately lazy employees may have to be shown the door. "The cost of sacking someone can be colossal and damaging to team morale but sometimes it maybe the only choice."

[Dec 12, 2017] Can Uber Ever Deliver Part Eleven Annual Uber Losses Now Approaching $5 Billion

Notable quotes:
"... Total 2015 gross passenger payments were 200% higher than 2014, but Uber corporate revenue improved 300% because Uber cut the driver share of passenger revenue from 83% to 77%. This was an effective $500 million wealth transfer from drivers to Uber's investors. ..."
"... Uber's P&L gains were wiped out by higher non-EBIDTAR expense. Thus the 300% Uber revenue growth did not result in any improvement in Uber profit margins. ..."
"... In 2016, Uber unilaterally imposed much larger cuts in driver compensation, costing drivers an additional $3 billion. [6] Prior to Uber's market entry, the take home pay of big-city cab drivers in the US was in the $12-17/hour range, and these earnings were possible only if drivers worked 65-75 hours a week. ..."
"... An independent study of the net earnings of Uber drivers (after accounting for the costs of the vehicles they had to provide) in Denver, Houston and Detroit in late 2015 (prior to Uber's big 2016 cuts) found that driver earnings had fallen to the $10-13/hour range. [7] Multiple recent news reports have documented how Uber drivers are increasing unable to support themselves from their reduced share of passenger payments. [8] ..."
"... Since mass driver defections would cause passenger volume growth to collapse completely, Uber was forced to reverse these cuts in 2017 and increased the driver share from 68% to 80%. This meant that Uber's corporate revenue, which had grown over 300% in 2015 and over 200% in 2016 will probably only grow by about 15% in 2017. ..."
"... Socialize the losses, privatize the gains, VC-ize the subsidies. ..."
"... The cold hard truth is that Uber is backed into a corner with severely limited abilities to tweak the numbers on either the supply or the demand side: cut driver compensation and they trigger driver churn (as has already been demonstrated), increase fare prices for riders and riders defect to cheaper alternatives. ..."
"... "Growth and Efficiency" are the sine qua non of Neoliberalism. Kalanick's "hype brilliance" was to con the market with "revenue growth" and signs ..."
Dec 12, 2017 | www.nakedcapitalism.com

Uber lost $2.5 billion in 2015, probably lost $4 billion in 2016, and is on track to lose $5 billion in 2017.

The top line on the table below shows is total passenger payments, which must be split between Uber corporate and its drivers. Driver gross earnings are substantially higher than actual take home pay, as gross earning must cover all the expenses drivers bear, including fuel, vehicle ownership, insurance and maintenance.

Most of the "profit" data released by Uber over time and discussed in the press is not true GAAP (generally accepted accounting principles) profit comparable to the net income numbers public companies publish but is EBIDTAR contribution. Companies have significant leeway as to how they calculate EBIDTAR (although it would exclude interest, taxes, depreciation, amortization) and the percentage of total costs excluded from EBIDTAR can vary significantly from quarter to quarter, given the impact of one-time expenses such as legal settlements and stock compensation. We only have true GAAP net profit results for 2014, 2015 and the 2nd/3rd quarters of 2017, but have EBIDTAR contribution numbers for all other periods. [5]

Uber had GAAP net income of negative $2.6 billion in 2015, and a negative profit margin of 132%. This is consistent with the negative $2.0 billion loss and (143%) margin for the year ending September 2015 presented in part one of the NC Uber series over a year ago.

No GAAP profit results for 2016 have been disclosed, but actual losses likely exceed $4 billion given the EBIDTAR contribution of negative $3.2 billion. Uber's GAAP losses for the 2nd and 3rd quarters of 2017 were over $2.5 billion, suggesting annual losses of roughly $5 billion.

While many Silicon Valley funded startups suffered large initial losses, none of them lost anything remotely close to $2.6 billion in their sixth year of operation and then doubled their losses to $5 billion in year eight. Reversing losses of this magnitude would require the greatest corporate financial turnaround in history.

No evidence of significant efficiency/scale gains; 2015 and 2016 margin improvements entirely explained by unilateral cuts in driver compensation, but losses soared when Uber had to reverse these cuts in 2017.

Total 2015 gross passenger payments were 200% higher than 2014, but Uber corporate revenue improved 300% because Uber cut the driver share of passenger revenue from 83% to 77%. This was an effective $500 million wealth transfer from drivers to Uber's investors. These driver compensation cuts improved Uber's EBIDTAR margin, but Uber's P&L gains were wiped out by higher non-EBIDTAR expense. Thus the 300% Uber revenue growth did not result in any improvement in Uber profit margins.

In 2016, Uber unilaterally imposed much larger cuts in driver compensation, costing drivers an additional $3 billion. [6] Prior to Uber's market entry, the take home pay of big-city cab drivers in the US was in the $12-17/hour range, and these earnings were possible only if drivers worked 65-75 hours a week.

An independent study of the net earnings of Uber drivers (after accounting for the costs of the vehicles they had to provide) in Denver, Houston and Detroit in late 2015 (prior to Uber's big 2016 cuts) found that driver earnings had fallen to the $10-13/hour range. [7] Multiple recent news reports have documented how Uber drivers are increasing unable to support themselves from their reduced share of passenger payments. [8]

A business model where profit improvement is hugely dependent on wage cuts is unsustainable, especially when take home wages fall to (or below) minimum wage levels. Uber's primary focus has always been the rate of growth in gross passenger revenue, as this has been a major justification for its $68 billion valuation. This growth rate came under enormous pressure in 2017 given Uber efforts to raise fares, major increases in driver turnover as wages fell, [9] and the avalanche of adverse publicity it was facing.

Since mass driver defections would cause passenger volume growth to collapse completely, Uber was forced to reverse these cuts in 2017 and increased the driver share from 68% to 80%. This meant that Uber's corporate revenue, which had grown over 300% in 2015 and over 200% in 2016 will probably only grow by about 15% in 2017.

MKS , December 12, 2017 at 6:19 am

"Uber's business model can never produce sustainable profits"

Two words not in my vocabulary are "Never" and "Always", that is a pretty absolute statement in an non-absolute environment. The same environment that has produced the "Silicon Valley Growth Model", with 15x earnings companies like NVIDA, FB and Tesla (Average earnings/stock price ratio in dot com bubble was 10x) will people pay ridiculous amounts of money for a company with no underlying fundamentals you damn right they will! Please stop with the I know all no body knows anything, especially the psychology and irrationality of markets which are made up of irrational people/investors/traders.

JohnnySacks , December 12, 2017 at 7:34 am

My thoughts exactly. Seems the only possible recovery for the investors is a perfectly engineered legendary pump and dump IPO scheme. Risky, but there's a lot of fools out there and many who would also like to get on board early in the ride in fear of missing out on all the money to be hoovered up from the greater fools. Count me out.

SoCal Rhino , December 12, 2017 at 8:30 am

The author clearly distinguishes between GAAP profitability and valuations, which is after all rather the point of the series. And he makes a more nuanced point than the half sentence you have quoted without context or with an indication that you omitted a portion. Did you miss the part about how Uber would have a strong incentive to share the evidence of a network effect or other financial story that pointed the way to eventual profit? Otherwise (my words) it is the classic sell at a loss, make it up with volume path to liquidation.

tegnost , December 12, 2017 at 9:52 am

apples and oranges comparison, nvidia has lots and lots of patented tech that produces revenue, facebook has a kajillion admittedly irrational users, but those users drive massive ad sales (as just one example of how that company capitalizes itself) and tesla makes an actual car, using technology that inspires it's buyers (the put your money where your mouth is crowd and it can't be denied that tesla, whatever it's faults are, battery tech is not one of them and that intellectual property is worth a lot, and tesla's investors are in on that real business, profitable or otherwise)

Uber is an iphone app. They lose money and have no path to profitability (unless it's the theory you espouse that people are unintelligent so even unintelligent ideas work to fleece them). This article touches on one of the great things about the time we now inhabit, uber drivers could bail en masse, there are two sides to the low attachment employees who you can get rid of easily. The drivers can delete the uber app as soon as another iphone app comes along that gets them a better return

allan , December 12, 2017 at 6:52 am

Yet another source (unintended) of subsidies for Uber, Lyft, etc., which might or might not have been mentioned earlier in the series:

Airports Are Losing Money as Ride-Hailing Services Grow [NYT]

For many air travelers, getting to and from the airport has long been part of the whole miserable experience. Do they drive and park in some distant lot? Take mass transit or a taxi? Deal with a rental car?

Ride-hailing services like Uber and Lyft are quickly changing those calculations. That has meant a bit less angst for travelers.

But that's not the case for airports. Travelers' changing habits, in fact, have begun to shake the airports' financial underpinnings. The money they currently collect from ride-hailing services do not compensate for the lower revenues from the other sources.

At the same time, some airports have had to add staff to oversee the operations of the ride-hailing companies, the report said. And with more ride-hailing vehicles on the roads outside terminals,
there's more congestion.

Socialize the losses, privatize the gains, VC-ize the subsidies.

Thuto , December 12, 2017 at 6:55 am

The cold hard truth is that Uber is backed into a corner with severely limited abilities to tweak the numbers on either the supply or the demand side: cut driver compensation and they trigger driver churn (as has already been demonstrated), increase fare prices for riders and riders defect to cheaper alternatives. The only question is how long can they keep the show going before the lights go out, slick marketing and propaganda can only take you so far, and one assumes the dumb money has a finite supply of patience and will at some point begin asking the tough questions.

Louis Fyne , December 12, 2017 at 8:35 am

The irony is that Uber would have been a perfectly fine, very profitable mid-sized company if Uber stuck with its initial model -- sticking to dense cities with limited parking, limiting driver supply, and charging a premium price for door-to-door delivery, whether by livery or a regular sedan. And then perhaps branching into robo-cars.

But somehow Uber/board/Travis got suckered into the siren call of self-driving cars, triple-digit user growth, and being in the top 100 US cities and on every continent.

Thuto , December 12, 2017 at 11:30 am

I've shared a similar sentiment in one of the previous posts about Uber. But operating profitably in decent sized niche doesn't fit well with ambitions of global domination. For Uber to be "right-sized", an admission of folly would have to be made, its managers and investors would have to transcend the sunk cost fallacy in their strategic decision making, and said investors would have to accept massive hits on their invested capital. The cold, hard reality of being blindsided and kicked to the curb in the smartphone business forced RIM/Blackberry to right-size, and they may yet have a profitable future as an enterprise facing software and services company. Uber would benefit from that form of sober mindedness, but I wouldn't hold my breath.

David Carl Grimes , December 12, 2017 at 6:57 am

The question is: Why did Softbank invest in Uber?

Michael Fiorillo , December 12, 2017 at 9:33 am

I know nothing about Softbank or its management, but I do know that the Japanese were the dumb money rubes in the late '80's, overpaying for trophy real estate they lost billions on.

Until informed otherwise, that's my default assumption

JimTan , December 12, 2017 at 10:50 am

Softbank possibly looking to buy more Uber shares at a 30% discount is very odd. Uber had a Series G funding round in June 2016 where a $3.5 billion investment from Saudi Arabia's Public Investment Fund resulted in its current $68 billion valuation. Now apparently Softbank wants to lead a new $6 billion funding round to buy the shares of Uber employees and early investors at a 30% discount from this last "valuation". It's odd because Saudi Arabia's Public Investment Fund has pledged $45 billion to SoftBank's Vision Fund , an amount which was supposed to come from the proceeds of its pending Aramco IPO. If the Uber bid is linked to SoftBank's Vision Fund, or KSA money, then its not clear why this investor might be looking to literally 'double down' from $3.5 billion o $6 billion on a declining investment.

Yves Smith Post author , December 12, 2017 at 11:38 am

SoftBank has not yet invested. Its tender is still open. If it does not get enough shares at a price it likes, it won't invest.

As to why, I have no idea.

Robert McGregor , December 12, 2017 at 7:04 am

"Growth and Efficiency" are the sine qua non of Neoliberalism. Kalanick's "hype brilliance" was to con the market with "revenue growth" and signs of efficiency, and hopes of greater efficiency, and make most people just overlook the essential fact that Uber is the most unprofitable company of all time!

divadab , December 12, 2017 at 7:19 am

What comprises "Uber Expenses"? 2014 – $1.06 billion; 2015 $3.33 billion; 2016 $9.65 billion; forecast 2017 $11.418 billion!!!!!! To me this is the big question – what are they spending $10 billion per year on?

ALso – why did driver share go from 68% in 2016 to 80% in 2017? If you use 68% as in 2016, 2017 Uber revenue is $11.808 billion, which means a bit better than break-even EBITDA, assuming Uber expenses are as stated $11.428 billion.

Perhaps not so bleak as the article presents, although I would not invest in this thing.

Phil in Kansas City , December 12, 2017 at 7:55 am

I have the same question: What comprises over 11 billion dollars in expenses in 2017? Could it be they are paying out dividends to the early investors? Which would mean they are cannibalizing their own company for the sake of the VC! How long can this go on before they'll need a new infusion of cash?

lyman alpha blob , December 12, 2017 at 2:37 pm

The Saudis have thrown a few billion Uber's way and they aren't necessarily known as the smart money.

Maybe the pole dancers have started chipping in too as they are for bitcoin .

Vedant Desai , December 12, 2017 at 10:37 am

Oh article does answer your 2nd question. Read this paragraph:-

Since mass driver defections would cause passenger volume growth to collapse completely , Uber was forced to reverse these cuts in 2017 and increased the driver share from 68% to 80%. This meant that Uber's corporate revenue, which had grown over 300% in 2015 and over 200% in 2016 will probably only grow by about 15% in 2017.

As for the 1st, read this line in the article:-

There are undoubtedly a number of things Uber could do to reduce losses at the margin, but it is difficult to imagine it could suddenly find the $4-5 billion in profit improvement needed merely to reach breakeven.

Louis Fyne , December 12, 2017 at 8:44 am

in addition to all the points listed in the article/comments, the absolute biggest flaw with Uber is that Uber HQ conditioned its customers on (a) cheap fares and (b) that a car is available within minutes (1-5 if in a big city).

Those two are not mutually compatible in the long-term.

Alfred , December 12, 2017 at 9:49 am

Thus (a) "We cost less" and (b) "We're more convenient" -- aren't those also the advantages that Walmart claims and feeds as a steady diet to its ever hungry consumers? Often if not always, disruption may repose upon delusion.

Martin Finnucane , December 12, 2017 at 11:06 am

Uber's business model could never produce sustainable profits unless it was able to exploit significant anti-competitive market power.

Upon that dependent clause hangs the future of capitalism, and – dare I say it? – its inevitable demise.

Altandmain , December 12, 2017 at 11:09 am

When this Uber madness blows up, I wonder if people will finally begin to discuss the brutal reality of Silicon Valley's so called "disruption".

It is heavily built in around the idea of economic exploitation. Uber drivers are often, especially when the true costs to operate an Uber including the vehicle depreciation are factored in, making not very much per hour driven, especially if they don't get the surge money.

Instacart is another example. They are paying the deliver operators very little.

Jim A. , December 12, 2017 at 12:21 pm

At a fundamental level, I think that the Silicon Valley "disruption" model only works for markets (like software) where the marginal cost for production is de minimus and the products can be protected by IP laws. Volume and market power really work in those cases. But out here in meat-space, where actual material and labor are big inputs to each item sold, you can never just sit back on your laurels and rake in the money. Somebody else will always be able to come and and make an equivalent product. If they can do it more cheaply, you are in trouble.

Altandmain , December 12, 2017 at 5:40 pm

There aren't that many areas in goods and services where the marginal costs are very low.

Software is actually quite unique in that regard, costing merely the bandwidth and permanent storage space to store.

Let's see:

1. From the article, they cannot go public and have limited ways to raise more money. An IPO with its more stringent disclosure requirements would expose them.

2. They tried lowering driver compensation and found that model unsustainable.

3. There are no benefits to expanding in terms of economies of scale.

From where I am standing, it looks like a lot of industries gave similar barriers. Silicon Valley is not going to be able to disrupt those.

Tesla, another Silicon Valley company seems to be struggling to mass produce its Model 3 and deliver an electric car that breaks even, is reliable, while disrupting the industry in the ways that Elon Musk attempted to hype up.

So that basically leaves services and manufacturing out for Silicon Valley disruption.

Joe Bentzel , December 12, 2017 at 2:19 pm

UBER has become a "too big to fail" startup because of all the different tentacles of capital from various Tier 1 VCs and investment bankers.

VCs have admitted openly that UBER is a subsidized business, meaning it's product is sold below market value, and the losses reflect that subsidization. The whole "2 sided platform" argument is just marketecture to hustle more investors. It's a form of service "dumping" that puts legacy businesses into bankruptcy. Back during the dotcom bubble one popular investment banker (Paul Deninger) characterized this model as "Terrorist Competition", i.e. coffers full of invested cash to commoditize the market and drive out competition.

UBER is an absolute disaster that has forked the startup model in Silicon Valley in order to drive total dependence on venture capital by founders. And its current diversification into "autonomous vehicles", food delivery, et al are simply more evidence that the company will never be profitable due to its whacky "blitzscaling" approach of layering on new "businesses" prior to achieving "fit" in its current one.

It's economic model has also metastasized into a form of startup cancer that is killing Silicon Valley as a "technology" innovator. Now it's all cargo cult marketing BS tied to "strategic capital".

UBER is the victory of venture capital and user subsidized startups over creativity by real entrepreneurs.

It's shadow is long and that's why this company should be ..wait for it UNBUNDLED (the new silicon valley word attached to that other BS religion called "disruption"). Call it a great unbundling and you can break up this monster corp any way you want.

Naked Capitalism is a great website.

Phil in KC , December 12, 2017 at 3:20 pm

1. I Agree with your last point.

2. The elevator pitch for Uber: subsidize rides to attract customers, put the competition out of business, and then enjoy an unregulated monopoly, all while exploiting economically ignorant drivers–ahem–"partners."

3. But more than one can play that game, and

4. Cab and livery companies are finding ways to survive!

Phil in KC , December 12, 2017 at 3:10 pm

If subsidizing rides is counted as an expense, (not being an accountant, I would guess it so), then whether the subsidy goes to the driver or the passenger, that would account for the ballooning expenses, to answer my own question. Otherwise, the overhead for operating what Uber describes as a tech company should be minimal: A billion should fund a decent headquarters with staff, plus field offices in, say, 100 U.S. cities. However, their global pretensions are probably burning cash like crazy. On top of that, I wonder what the exec compensation is like?

After reading HH's initial series, I made a crude, back-of-the-envelope calculation that Uber would run out of money sometime in the third fiscal quarter of 2018, but that was based on assuming losses were stabilizing in the range of 3 billion a year. Not so, according to the article. I think crunch time is rapidly approaching. If so, then SoftBank's tender offer may look quite appetizing to VC firms and to any Uber employee able to cash in their options. I think there is a way to make a re-envisioned Uber profitable, and with a more independent board, they may be able to restructure the company to show a pathway to profitability before the IPO. But time is running out.

A not insignificant question is the recruitment and retention of the front line "partners." It would seem to me that at some point, Uber will run out of economically ignorant drivers with good manners and nice cars. I would be very interested to know how many drivers give up Uber and other ride-sharing gigs once the 1099's start flying at the beginning of the year. One of the harsh realities of owning a business or being an contractor is the humble fact that you get paid LAST!

Jan Stickle , December 12, 2017 at 5:00 pm

We became instant Uber riders while spending holidays with relatives in San Diego. While their model is indeed unique from a rider perspective, it was the driver pool that fascinates me. These are not professional livery drivers, but rather freebooters of all stripes driving for various reasons. The remuneration they receive cannot possibly generate much income after expenses, never mind the problems associated with IRS filing as independent contractors.

One guy was just cruising listening to music; cooler to get paid for it than just sitting home! A young lady was babbling and gesticulating non stop about nothing coherent and appeared to be on some sort of stimulant. A foreign gentleman, very professional, drove for extra money when not at his regular job. He was the only one who had actually bought a new Prius for this gig, hoping to pay it off in two years.

This is indeed a brave new world. There was a period in Nicaragua just after the Contra war ended when citizens emerged from their homes and hit the streets in large numbers, desperately looking for income. Every car was a taxi and there was a bipedal mini Walmart at every city intersection as individuals sold everything and anything in a sort of euphoric optimism towards the future. Reality just hadn't caught up with them yet .

[Dec 09, 2017] How to rsync only a specific list of files - Stack Overflow

Notable quotes:
"... The filenames that are read from the FILE are all relative to the source dir ..."
Dec 09, 2017 | stackoverflow.com

ash, May 11, 2015 at 20:05

There is a flag --files-from that does exactly what you want. From man rsync :
--files-from=FILE

Using this option allows you to specify the exact list of files to transfer (as read from the specified FILE or - for standard input). It also tweaks the default behavior of rsync to make transferring just the specified files and directories easier:

The filenames that are read from the FILE are all relative to the source dir -- any leading slashes are removed and no ".." references are allowed to go higher than the source dir. For example, take this command:

rsync -a --files-from=/tmp/foo /usr remote:/backup

If /tmp/foo contains the string "bin" (or even "/bin"), the /usr/bin directory will be created as /backup/bin on the remote host. If it contains "bin/" (note the trailing slash), the immediate contents of the directory would also be sent (without needing to be explicitly mentioned in the file -- this began in version 2.6.4). In both cases, if the -r option was enabled, that dir's entire hierarchy would also be transferred (keep in mind that -r needs to be specified explicitly with --files-from, since it is not implied by -a). Also note that the effect of the (enabled by default) --relative option is to duplicate only the path info that is read from the file -- it does not force the duplication of the source-spec path (/usr in this case).

In addition, the --files-from file can be read from the remote host instead of the local host if you specify a "host:" in front of the file (the host must match one end of the transfer). As a short-cut, you can specify just a prefix of ":" to mean "use the remote end of the transfer". For example:

rsync -a --files-from=:/path/file-list src:/ /tmp/copy

This would copy all the files specified in the /path/file-list file that was located on the remote "src" host.

If the --iconv and --protect-args options are specified and the --files-from filenames are being sent from one host to another, the filenames will be translated from the sending host's charset to the receiving host's charset.

NOTE: sorting the list of files in the --files-from input helps rsync to be more efficient, as it will avoid re-visiting the path elements that are shared between adjacent entries. If the input is not sorted, some path elements (implied directories) may end up being scanned multiple times, and rsync will eventually unduplicate them after they get turned into file-list elements.

Nicolas Mattia, Feb 11, 2016 at 11:06

Note that you still have to specify the directory where the files listed are located, for instance: rsync -av --files-from=file-list . target/ for copying files from the current dir. – Nicolas Mattia Feb 11 '16 at 11:06

ash, Feb 12, 2016 at 2:25

Yes, and to reiterate: The filenames that are read from the FILE are all relative to the source dir . – ash Feb 12 '16 at 2:25

Michael ,Nov 2, 2016 at 0:09

if the files-from file has anything starting with .. rsync appears to ignore the .. giving me an error like rsync: link_stat "/home/michael/test/subdir/test.txt" failed: No such file or directory (in this case running from the "test" dir and trying to specify "../subdir/test.txt" which does exist. – Michael Nov 2 '16 at 0:09

xxx,

--files-from= parameter needs trailing slash if you want to keep the absolute path intact. So your command would become something like below:
rsync -av --files-from=/path/to/file / /tmp/

This could be done like there are a large number of files and you want to copy all files to x path. So you would find the files and throw output to a file like below:

find /var/* -name *.log > file

[Dec 09, 2017] linux - What does the line '!-bin-sh -e' do

Dec 09, 2017 | stackoverflow.com

,

That line defines what program will execute the given script. For sh normally that line should start with the # character as so:
#!/bin/sh -e

The -e flag's long name is errexit , causing the script to immediately exit on the first error.

[Dec 07, 2017] First Rule of Usability Don't Listen to Users

Notable quotes:
"... So, do users know what they want? No, no, and no. Three times no. ..."
Dec 07, 2017 | www.nngroup.com

But ultimately, the way to get user data boils down to the basic rules of usability

... ... ...

So, do users know what they want? No, no, and no. Three times no.

Finally, you must consider how and when to solicit feedback. Although it might be tempting to simply post a survey online, you're unlikely to get reliable input (if you get any at all). Users who see the survey and fill it out before they've used the site will offer irrelevant answers. Users who see the survey after they've used the site will most likely leave without answering the questions. One question that does work well in a website survey is "Why are you visiting our site today?" This question goes to users' motivation and they can answer it as soon as they arrive.

[Dec 07, 2017] The rogue DHCP server

Notable quotes:
"... from Don Watkins ..."
Dec 07, 2017 | opensource.com

from Don Watkins

I am a liberal arts person who wound up being a technology director. With the exception of 15 credit hours earned on my way to a Cisco Certified Network Associate credential, all of the rest of my learning came on the job. I believe that learning what not to do from real experiences is often the best teacher. However, those experiences can frequently come at the expense of emotional pain. Prior to my Cisco experience, I had very little experience with TCP/IP networking and the kinds of havoc I could create albeit innocently due to my lack of understanding of the nuances of routing and DHCP.

At the time our school network was an active directory domain with DHCP and DNS provided by a Windows 2000 server. All of our staff access to the email, Internet, and network shares were served this way. I had been researching the use of the K12 Linux Terminal Server ( K12LTSP ) project and had built a Fedora Core box with a single network card in it. I wanted to see how well my new project worked so without talking to my network support specialists I connected it to our main LAN segment. In a very short period of time our help desk phones were ringing with principals, teachers, and other staff who could no longer access their email, printers, shared directories, and more. I had no idea that the Windows clients would see another DHCP server on our network which was my test computer and pick up an IP address and DNS information from it.

I had unwittingly created a "rogue" DHCP server and was oblivious to the havoc that it would create. I shared with the support specialist what had happened and I can still see him making a bee-line for that rogue computer, disconnecting it from the network. All of our client computers had to be rebooted along with many of our switches which resulted in a lot of confusion and lost time due to my ignorance. That's when I learned that it is best to test new products on their own subnet.

[Dec 03, 2017] Business Has Killed IT With Overspecialization by Charlie Schluting

Highly recommended!
Notable quotes:
"... What happened to the old "sysadmin" of just a few years ago? We've split what used to be the sysadmin into application teams, server teams, storage teams, and network teams. There were often at least a few people, the holders of knowledge, who knew how everything worked, and I mean everything. ..."
"... Now look at what we've done. Knowledge is so decentralized we must invent new roles to act as liaisons between all the IT groups. Architects now hold much of the high-level "how it works" knowledge, but without knowing how any one piece actually does work. In organizations with more than a few hundred IT staff and developers, it becomes nearly impossible for one person to do and know everything. This movement toward specializing in individual areas seems almost natural. That, however, does not provide a free ticket for people to turn a blind eye. ..."
"... Does your IT department function as a unit? Even 20-person IT shops have turf wars, so the answer is very likely, "no." As teams are split into more and more distinct operating units, grouping occurs. One IT budget gets split between all these groups. Often each group will have a manager who pitches his needs to upper management in hopes they will realize how important the team is. ..."
"... The "us vs. them" mentality manifests itself at all levels, and it's reinforced by management having to define each team's worth in the form of a budget. One strategy is to illustrate a doomsday scenario. If you paint a bleak enough picture, you may get more funding. Only if you are careful enough to illustrate the failings are due to lack of capital resources, not management or people. A manager of another group may explain that they are not receiving the correct level of service, so they need to duplicate the efforts of another group and just implement something themselves. On and on, the arguments continue. ..."
Apr 07, 2010 | Enterprise Networking Planet

What happened to the old "sysadmin" of just a few years ago? We've split what used to be the sysadmin into application teams, server teams, storage teams, and network teams. There were often at least a few people, the holders of knowledge, who knew how everything worked, and I mean everything. Every application, every piece of network gear, and how every server was configured -- these people could save a business in times of disaster.

Now look at what we've done. Knowledge is so decentralized we must invent new roles to act as liaisons between all the IT groups. Architects now hold much of the high-level "how it works" knowledge, but without knowing how any one piece actually does work. In organizations with more than a few hundred IT staff and developers, it becomes nearly impossible for one person to do and know everything. This movement toward specializing in individual areas seems almost natural. That, however, does not provide a free ticket for people to turn a blind eye.

Specialization

You know the story: Company installs new application, nobody understands it yet, so an expert is hired. Often, the person with a certification in using the new application only really knows how to run that application. Perhaps they aren't interested in learning anything else, because their skill is in high demand right now. And besides, everything else in the infrastructure is run by people who specialize in those elements. Everything is taken care of.

Except, how do these teams communicate when changes need to take place? Are the storage administrators teaching the Windows administrators about storage multipathing; or worse logging in and setting it up because it's faster for the storage gurus to do it themselves? A fundamental level of knowledge is often lacking, which makes it very difficult for teams to brainstorm about new ways evolve IT services. The business environment has made it OK for IT staffers to specialize and only learn one thing.

If you hire someone certified in the application, operating system, or network vendor you use, that is precisely what you get. Certifications may be a nice filter to quickly identify who has direct knowledge in the area you're hiring for, but often they indicate specialization or compensation for lack of experience.

Resource Competition

Does your IT department function as a unit? Even 20-person IT shops have turf wars, so the answer is very likely, "no." As teams are split into more and more distinct operating units, grouping occurs. One IT budget gets split between all these groups. Often each group will have a manager who pitches his needs to upper management in hopes they will realize how important the team is.

The "us vs. them" mentality manifests itself at all levels, and it's reinforced by management having to define each team's worth in the form of a budget. One strategy is to illustrate a doomsday scenario. If you paint a bleak enough picture, you may get more funding. Only if you are careful enough to illustrate the failings are due to lack of capital resources, not management or people. A manager of another group may explain that they are not receiving the correct level of service, so they need to duplicate the efforts of another group and just implement something themselves. On and on, the arguments continue.

Most often, I've seen competition between server groups result in horribly inefficient uses of hardware. For example, what happens in your organization when one team needs more server hardware? Assume that another team has five unused servers sitting in a blade chassis. Does the answer change? No, it does not. Even in test environments, sharing doesn't often happen between IT groups.

With virtualization, some aspects of resource competition get better and some remain the same. When first implemented, most groups will be running their own type of virtualization for their platform. The next step, I've most often seen, is for test servers to get virtualized. If a new group is formed to manage the virtualization infrastructure, virtual machines can be allocated to various application and server teams from a central pool and everyone is now sharing. Or, they begin sharing and then demand their own physical hardware to be isolated from others' resource hungry utilization. This is nonetheless a step in the right direction. Auto migration and guaranteed resource policies can go a long way toward making shared infrastructure, even between competing groups, a viable option.

Blamestorming

The most damaging side effect of splitting into too many distinct IT groups is the reinforcement of an "us versus them" mentality. Aside from the notion that specialization creates a lack of knowledge, blamestorming is what this article is really about. When a project is delayed, it is all too easy to blame another group. The SAN people didn't allocate storage on time, so another team was delayed. That is the timeline of the project, so all work halted until that hiccup was restored. Having someone else to blame when things get delayed makes it all too easy to simply stop working for a while.

More related to the initial points at the beginning of this article, perhaps, is the blamestorm that happens after a system outage.

Say an ERP system becomes unresponsive a few times throughout the day. The application team says it's just slowing down, and they don't know why. The network team says everything is fine. The server team says the application is "blocking on IO," which means it's a SAN issue. The SAN team say there is nothing wrong, and other applications on the same devices are fine. You've ran through nearly every team, but without an answer still. The SAN people don't have access to the application servers to help diagnose the problem. The server team doesn't even know how the application runs.

See the problem? Specialized teams are distinct and by nature adversarial. Specialized staffers often relegate themselves into a niche knowing that as long as they continue working at large enough companies, "someone else" will take care of all the other pieces.

I unfortunately don't have an answer to this problem. Maybe rotating employees between departments will help. They gain knowledge and also get to know other people, which should lessen the propensity to view them as outsiders

[Dec 03, 2017] Nokia Shareholders Fight Back

On the topic of outsourcing, IMO it can be cheaper if done right. On paper it always seems like a great idea, but in practice it's not always the best idea financially and/or getting the same or better result in comparison to keeping it in-house. I've worked for companies where they have outsourced a particular department/function to companies where I am the one the job is outsourced to. My observation has been the success of getting projects done (e.g.: programing) or facilitating a role (e.g.: sys admin) rely on a few factors regardless of outsourcing or not.
Notable quotes:
"... On the topic of outsourcing, IMO it can be cheaper if done right. On paper it always seems like a great idea, but in practice it's not always the best idea financially and/or getting the same or better result in comparison to keeping it in-house. I've worked for companies where they have outsourced a particular department/function to companies where I am the one the job is outsourced to. My observation has been the success of getting projects done (e.g.: programing) or facilitating a role (e.g.: sys admin) rely on a few factors regardless of outsourcing or not. ..."
Slashdot

noc007 (633443)

On the topic of outsourcing, IMO it can be cheaper if done right. On paper it always seems like a great idea, but in practice it's not always the best idea financially and/or getting the same or better result in comparison to keeping it in-house. I've worked for companies where they have outsourced a particular department/function to companies where I am the one the job is outsourced to. My observation has been the success of getting projects done (e.g.: programing) or facilitating a role (e.g.: sys admin) rely on a few factors regardless of outsourcing or not.

The first is a golden rule of sorts on doing anything:

You can only pick two; NO exceptions. I've encountered so many upper management types that foolishly think they can get away with having all three. In my experience 9/10 of the time it turns out a lack of quality bites them in the butt sometime down the road when they assumed they somehow managed to achieve all three.

The second is communication. Mostly everyone in at least the US has experienced the pain of being subjected to some company's outsourced customer service and/or tech support that can't effectively communicate with both parties on the same page of understanding one another. I really shouldn't need to explain why communication, understanding one another is so important. Sadly this is something I have to constantly explain to my current boss with events like today where my non-outsourced colleague rebooted a number of production critical servers when he was asked to reboot just one secondary server.

Third is the employee's skill in doing the job. Again, another obvious one, but I've observed that it isn't always on the hiring menu. Additionally I've seen some people that interview well, but couldn't create a "Hello World" HTML page for a web developer position as an example. There's no point in hiring or keeping a hired individual to do a job that they lack the skill to do; even if it's an entry-level position with training, that person should be willing to put for the effort to learn and take notes. I accept that everyone has their own unique skills that can aide or hinder their ability to learn and be proficient with a particular task. However, I firmly believe anyone can learn to do anything as long as they put their mind to it. I barely have any artistic ability and my drawing skills are stick figures at best (XKCD is miles ahead of me); if I were to put forth the effort to learn how to draw and paint, I could become a good artist. I taught an A+ technician certification class at a tech school a while back and I had a retired Marine that served in the Vietnam War as one of my students. One could argue his best skill was killing and blowing stuff up. He worked hard and learned to be a technician and passed CompTIA's certification test without a problem. That leads me to the next point.

Lastly is attitude of the end employee doing the actual work. It boggles my mind how so many managers loose the plot when it comes to employee morale and motivation. Productivity generally is improved when those two are improved and it usually doesn't have to involve spending a bunch of money. The employee's attitude should be getting the work done correctly in a reasonable amount of time. Demanding it is a poor approach. Poisoning an employee will result in poisoning the company in a small manner all the way up to the failure of the company. Employees should be encouraged through actual morale improvements, positive motivation, and incentives for doing more work at the same and/or better quality level.

Outsourcing or keeping things in house can be successful and possibly economical if approached correctly with the appropriate support of upper management.

Max Littlemore (1001285)

How dramatic? Isn't outsourcing done (like it or not) to reduce costs?

Outsourcing is done to reduce the projected costs that PHBs see. In reality, outsourcing can lead to increased costs and delays due to time zone differences and language/cultural barriers.

I have seen it work reasonably well, but only when the extra effort and delays caused by the increased need for rework that comes from complex software projects. If you are working with others on software, it is so much quicker to produce quality software if the person who knows the business requirements is sitting right next to the person doing design and the person cutting code and the person doing the testing, etc, etc.

If these people or groups are scattered around the world with different cultures and native languages, communication can suffer, increasing misunderstanding and reducing the quality. I have personally seen this lead to massive increase in code defects in a project that went from in house development to outsourced.

Also, time zone differences cause problems. I have noticed that the further west people live, the less likely they are to take into account how far behind they are. Working with people who fail to realise that their Monday morning is the next day for someone else, or that by the time they are halfway through Friday, others are already on their weekend is not only frustrating, it leads to slow turn around of bug fixes, etc.

Yeah, I'm told outsourcing keeps costs down, but I am yet to see conclusive evidence of that in the real world. At least in complex development. YMMV for support/call centre stuff.

-- I don't therefore I'm not.

[Dec 03, 2017] IT workers voices heard in the Senate, confidentially

The resentment against outsourcing was brewing for a long time.
Notable quotes:
"... Much of the frustration focused on the IT layoffs at Southern California Edison , which is cutting 500 IT workers after hiring two offshore outsourcing firms. This has become the latest example for critics of the visa program's capacity for abuse. ..."
"... Infosys whistleblower Jay Palmer, who testified, and is familiar with the displacement process, told Sessions said these workers will get sued if they speak out. "That's the fear and intimidation that these people go through - they're blindsided," said Palmer. ..."
"... Moreover, if IT workers refuse to train their foreign replacement, "they are going to be terminated with cause, which means they won't even get their unemployment insurance," said Ron Hira, an associate professor at Howard University, who also testified. Affected tech workers who speak out publicly and use their names, "will be blackballed from the industry," he said. ..."
"... Hatch, who is leading the effort to increase the H-1B cap, suggested a willingness to raise wage levels for H-1B dependent employers. They are exempt from U.S. worker protection rules if the H-1B worker is paid at least $60,000 or has a master's degree, a figure that was set in law in 1998. Hatch suggested a wage level of $95,000. ..."
"... Sen. Dick Durbin, (Dem-Ill.), who has joined with Grassley on legislation to impose some restrictions on H-1B visa use -- particularly in offshoring -- has argued for a rule that would keep large firms from having more than 50% of their workers on the visa. This so-called 50/50 rule, as Durbin has noted, has drawn much criticism from India, where most of the affected companies are located. ..."
"... "I want to put the H-1B factories out of business," said Durbin. ..."
"... Hal Salzman, a Rutgers University professor who studies STEM (Science, Technology, Engineering and Math) workforce issues, told the committee that the IT industry now fills about two-thirds of its entry-level positions with guest workers. "At the same time, IT wages have stagnated for over a decade," he said. ..."
"... H-1B supporters use demand for the visa - which will exceed the 85,000 cap -- as proof of economic demand. But Salzman argues that U.S. colleges already graduate more scientists and engineers than find employment in those fields, about 200,000 more. ..."
Mar 18, 2015 | Network World

A Senate Judiciary Committee hearing today on the H-1B visa offered up a stew of policy arguments, positioning and frustration.

Much of the frustration focused on the IT layoffs at Southern California Edison, which is cutting 500 IT workers after hiring two offshore outsourcing firms. This has become the latest example for critics of the visa program's capacity for abuse.

Sen. Charles Grassley (R-Iowa), the committee chair who has long sought H-1B reforms, said he invited Southern California Edison officials "to join us today" and testify. "I thought they would want to defend their actions and explain why U.S. workers have been left high and dry," said Grassley. "Unfortunately, they declined my invitation."

The hearing, by the people picked to testify, was weighted toward critics of the program, prompting a response by industry groups.

Compete America, the Consumer Electronics Association, FWD.us, the U.S. Chamber of Commerce and many others submitted a letter to the committee to rebut the "flawed studies" and "non-representative anecdotes used to create myths that suggest immigration harms American and American workers."

The claim that H-1B critics are using "anecdotes" to make their points (which include layoff reports at firms such as Edison) is a naked example of the pot calling the kettle black. The industry musters anecdotal stories in support of its positions readily and often. It makes available to the press and congressional committees people who came to the U.S. on an H-1B visa who started a business or took on a critical role in a start-up. These people are free to share their often compelling and admirable stories.

The voices of the displaced, who may be in fear of losing their homes, are thwarted by severance agreements.

The committee did hear from displaced workers, including some at Southern California Edison. But the communications with these workers are being kept confidential.

"I got the letters here from people, without the names," said Sen. Jeff Sessions (R-Ala.). "If they say what they know and think about this, they will lose the buy-outs."

Infosys whistleblower Jay Palmer, who testified, and is familiar with the displacement process, told Sessions said these workers will get sued if they speak out. "That's the fear and intimidation that these people go through - they're blindsided," said Palmer.

Moreover, if IT workers refuse to train their foreign replacement, "they are going to be terminated with cause, which means they won't even get their unemployment insurance," said Ron Hira, an associate professor at Howard University, who also testified. Affected tech workers who speak out publicly and use their names, "will be blackballed from the industry," he said.

While lawmakers voiced either strong support or criticism of the program, there was interest in crafting legislation that impose some restrictions on H-1B use.

"America and American companies need more high-skilled workers - this is an undeniable fact," said Sen. Orrin Hatch (R-Utah). "America's high-skilled worker shortage has become a crisis."

Hatch, who is leading the effort to increase the H-1B cap, suggested a willingness to raise wage levels for H-1B dependent employers. They are exempt from U.S. worker protection rules if the H-1B worker is paid at least $60,000 or has a master's degree, a figure that was set in law in 1998. Hatch suggested a wage level of $95,000.

Sen. Dick Durbin, (Dem-Ill.), who has joined with Grassley on legislation to impose some restrictions on H-1B visa use -- particularly in offshoring -- has argued for a rule that would keep large firms from having more than 50% of their workers on the visa. This so-called 50/50 rule, as Durbin has noted, has drawn much criticism from India, where most of the affected companies are located.

"I want to put the H-1B factories out of business," said Durbin.

Durbin got some support for the 50/50 rule from one person testifying in support of expanding the cap, Bjorn Billhardt, the founder and president of Enspire Learning, an Austin-based company. Enspire creates learning development tools; Billhardt came to the U.S. as an exchange student and went from an H-1B visa to a green card to, eventually, citizenship.

"I actually think that's a reasonable provision," said Billhardt of the 50% visa limit. He said it could help, "quite a bit." At the same time, he urged lawmakers to raise the cap to end the lottery system now used to distribute visas once that cap is reached.

Today's hearing went well beyond the impact of H-1B use by outsourcing firms to the displacement of workers overall.

Hal Salzman, a Rutgers University professor who studies STEM (Science, Technology, Engineering and Math) workforce issues, told the committee that the IT industry now fills about two-thirds of its entry-level positions with guest workers. "At the same time, IT wages have stagnated for over a decade," he said.

H-1B supporters use demand for the visa - which will exceed the 85,000 cap -- as proof of economic demand. But Salzman argues that U.S. colleges already graduate more scientists and engineers than find employment in those fields, about 200,000 more.

"Asking domestic graduates, both native-born and immigrant, to compete with guest workers on wages is not a winning strategy for strengthening U.S. science, technology and innovation," said Salzman.

See also

[Dec 02, 2017] BASH Shell How To Redirect stderr To stdout ( redirect stderr to a File )

Dec 02, 2017 | www.cyberciti.biz

BASH Shell: How To Redirect stderr To stdout ( redirect stderr to a File ) Posted on March 12, 2008 March 12, 2008 in Categories BASH Shell , Linux , UNIX last updated March 12, 2008 Q. How do I redirect stderr to stdout? How do I redirect stderr to a file?

A. Bash and other modern shell provides I/O redirection facility. There are 3 default standard files (standard streams) open:

[a] stdin – Use to get input (keyboard) i.e. data going into a program.

[b] stdout – Use to write information (screen)

[c] stderr – Use to write error message (screen)

Understanding I/O streams numbers

The Unix / Linux standard I/O streams with numbers:

Handle Name Description
0 stdin Standard input
1 stdout Standard output
2 stderr Standard error
Redirecting the standard error stream to a file

The following will redirect program error message to a file called error.log:
$ program-name 2> error.log
$ command1 2> error.log

Redirecting the standard error (stderr) and stdout to file

Use the following syntax:
$ command-name &>file
OR
$ command > file-name 2>&1
Another useful example:
# find /usr/home -name .profile 2>&1 | more

Redirect stderr to stdout

Use the command as follows:
$ command-name 2>&1

[Dec 01, 2017] NSA hacks system administrators, new leak reveals

Highly recommended!
"I hunt sysadm" policy is the most realosnableif you you want to get into some coporate netwrok. So republication of this three years old post is just a reminder. Any sysadmin that access corporates netwrok not from a dedicated computer using VPN (corporate laptop) is engangering the corporation. As simple as that. The level of non-professionalism demonstrated by Hillary Clinton IT staff suggests that this can be a problem in government too. After all Snowden documents now are studied by all major intelligence agencies of the world.
This also outlines the main danger of "shadow It".
Notable quotes:
"... Journalist Ryan Gallagher reported that Edward Snowden , a former sys admin for NSA contractor Booz Allen Hamilton, provided The Intercept with the internal documents, including one from 2012 that's bluntly titled "I hunt sys admins." ..."
"... "Who better to target than the person that already has the 'keys to the kingdom'?" ..."
"... "They were written by an NSA official involved in the agency's effort to break into foreign network routers, the devices that connect computer networks and transport data across the Internet," ..."
"... "By infiltrating the computers of system administrators who work for foreign phone and Internet companies, the NSA can gain access to the calls and emails that flow over their networks." ..."
"... The latest leak suggests that some NSA analysts took a much different approach when tasked with trying to collect signals intelligence that otherwise might not be easily available. According to the posts, the author advocated for a technique that involves identifying the IP address used by the network's sys admin, then scouring other NSA tools to see what online accounts used those addresses to log-in. Then by using a ..."
"... that tricks targets into installing malware by being misdirected to fake Facebook servers, the intelligence analyst can hope that the sys admin's computer is sufficiently compromised and exploited. ..."
"... Once the NSA has access to the same machine a sys admin does, American spies can mine for a trove of possibly invaluable information, including maps of entire networks, log-in credentials, lists of customers and other details about how systems are wired. In turn, the NSA has found yet another way to, in theory, watch over all traffic on a targeted network. ..."
"... "Up front, sys admins generally are not my end target. My end target is the extremist/terrorist or government official that happens to be using the network some admin takes care of," the NSA employee says in the documents. ..."
"... "A key part of the protections that apply to both US persons and citizens of other countries is the mandate that information be in support of a valid foreign intelligence requirement, and comply with US Attorney General-approved procedures to protect privacy rights." ..."
"... Coincidentally, outgoing-NSA Director Keith Alexander said last year that he was working on drastically cutting the number of sys admins at that agency by upwards of 90 percent - but didn't say it was because they could be exploited by similar tactics waged by adversarial intelligence groups. ..."
Mar 21, 2014 | news.slashdot.org

In its quest to take down suspected terrorists and criminals abroad, the United States National Security Agency has adopted the practice of hacking the system administrators that oversee private computer networks, new documents reveal.

In its quest to take down suspected terrorists and criminals abroad, the United States National Security Agency has adopted the practice of hacking the system administrators that oversee private computer networks, new documents reveal.

The Intercept has published a handful of leaked screenshots taken from an internal NSA message board where one spy agency specialist spoke extensively about compromising not the computers of specific targets, but rather the machines of the system administrators who control entire networks.

Journalist Ryan Gallagher reported that Edward Snowden, a former sys admin for NSA contractor Booz Allen Hamilton, provided The Intercept with the internal documents, including one from 2012 that's bluntly titled "I hunt sys admins."

According to the posts - some labeled "top secret" - NSA staffers should not shy away from hacking sys admins: a successful offensive mission waged against an IT professional with extensive access to a privileged network could provide the NSA with unfettered capabilities, the analyst acknowledged.

"Who better to target than the person that already has the 'keys to the kingdom'?" one of the posts reads.

"They were written by an NSA official involved in the agency's effort to break into foreign network routers, the devices that connect computer networks and transport data across the Internet," Gallagher wrote for the article published late Thursday. "By infiltrating the computers of system administrators who work for foreign phone and Internet companies, the NSA can gain access to the calls and emails that flow over their networks."

Since last June, classified NSA materials taken by Snowden and provided to certain journalists have exposed an increasing number of previously-secret surveillance operations that range from purposely degrading international encryption standards and implanting malware in targeted machines, to tapping into fiber-optic cables that transfer internet traffic and even vacuuming up data as its moved into servers in a decrypted state.

The latest leak suggests that some NSA analysts took a much different approach when tasked with trying to collect signals intelligence that otherwise might not be easily available. According to the posts, the author advocated for a technique that involves identifying the IP address used by the network's sys admin, then scouring other NSA tools to see what online accounts used those addresses to log-in. Then by using a previously-disclosed NSA tool that tricks targets into installing malware by being misdirected to fake Facebook servers, the intelligence analyst can hope that the sys admin's computer is sufficiently compromised and exploited.

Once the NSA has access to the same machine a sys admin does, American spies can mine for a trove of possibly invaluable information, including maps of entire networks, log-in credentials, lists of customers and other details about how systems are wired. In turn, the NSA has found yet another way to, in theory, watch over all traffic on a targeted network.

"Up front, sys admins generally are not my end target. My end target is the extremist/terrorist or government official that happens to be using the network some admin takes care of," the NSA employee says in the documents.

When reached for comment by The Intercept, NSA spokesperson Vanee Vines said that, "A key part of the protections that apply to both US persons and citizens of other countries is the mandate that information be in support of a valid foreign intelligence requirement, and comply with US Attorney General-approved procedures to protect privacy rights."

Coincidentally, outgoing-NSA Director Keith Alexander said last year that he was working on drastically cutting the number of sys admins at that agency by upwards of 90 percent - but didn't say it was because they could be exploited by similar tactics waged by adversarial intelligence groups. Gen. Alexander's decision came just weeks after Snowden - previously one of around 1,000 sys admins working on the NSA's networks, according to Reuters - walked away from his role managing those networks with a trove of classified information.

[Nov 30, 2017] Will Robots Kill the Asian Century

This aritcle is two years old and not much happned during those two years. But still there is a chance that highly authomated factories can make manufacturing in the USA again profitable. the problme is that they will be even more profible in East Asia;-)
Notable quotes:
"... The National Interest ..."
The National Interest

The rise of technologies such as 3-D printing and advanced robotics means that the next few decades for Asia's economies will not be as easy or promising as the previous five.

OWEN HARRIES, the first editor, together with Robert Tucker, of The National Interest, once reminded me that experts-economists, strategists, business leaders and academics alike-tend to be relentless followers of intellectual fashion, and the learned, as Harold Rosenberg famously put it, a "herd of independent minds." Nowhere is this observation more apparent than in the prediction that we are already into the second decade of what will inevitably be an "Asian Century"-a widely held but rarely examined view that Asia's continued economic rise will decisively shift global power from the Atlantic to the western Pacific Ocean.

No doubt the numbers appear quite compelling. In 1960, East Asia accounted for a mere 14 percent of global GDP; today that figure is about 27 percent. If linear trends continue, the region could account for about 36 percent of global GDP by 2030 and over half of all output by the middle of the century. As if symbolic of a handover of economic preeminence, China, which only accounted for about 5 percent of global GDP in 1960, will likely surpass the United States as the largest economy in the world over the next decade. If past record is an indicator of future performance, then the "Asian Century" prediction is close to a sure thing.

[Nov 29, 2017] Take This GUI and Shove It

Providing a great GUI for complex routers or Linux admin is hard. Of course there has to be a CLI, that's how pros get the job done. But a great GUI is one that teaches a new user to eventually graduate to using CLI.
Notable quotes:
"... Providing a great GUI for complex routers or Linux admin is hard. Of course there has to be a CLI, that's how pros get the job done. But a great GUI is one that teaches a new user to eventually graduate to using CLI. ..."
"... What would be nice is if the GUI could automatically create a shell script doing the change. That way you could (a) learn about how to do it per CLI by looking at the generated shell script, and (b) apply the generated shell script (after proper inspection, of course) to other computers. ..."
"... AIX's SMIT did this, or rather it wrote the commands that it executed to achieve what you asked it to do. This meant that you could learn: look at what it did and find out about which CLI commands to run. You could also take them, build them into a script, copy elsewhere, ... I liked SMIT. ..."
"... Cisco's GUI stuff doesn't really generate any scripts, but the commands it creates are the same things you'd type into a CLI. And the resulting configuration is just as human-readable (barring any weird naming conventions) as one built using the CLI. I've actually learned an awful lot about the Cisco CLI by using their GUI. ..."
"... Microsoft's more recent tools are also doing this. Exchange 2007 and newer, for example, are really completely driven by the PowerShell CLI. The GUI generates commands and just feeds them into PowerShell for you. So you can again issue your commands through the GUI, and learn how you could have done it in PowerShell instead. ..."
"... Moreover, the GUI authors seem to have a penchant to find new names for existing CLI concepts. Even worse, those names are usually inappropriate vagueries quickly cobbled together in an off-the-cuff afterthought, and do not actually tell you where the doodad resides in the menu system. With a CLI, the name of the command or feature set is its location. ..."
"... I have a cheap router with only a web gui. I wrote a two line bash script that simply POSTs the right requests to URL. Simply put, HTTP interfaces, especially if they implement the right response codes, are actually very nice to script. ..."
Slashdot

Deep End's Paul Venezia speaks out against the overemphasis on GUIs in today's admin tools, saying that GUIs are fine and necessary in many cases, but only after a complete CLI is in place, and that they cannot interfere with the use of the CLI, only complement it. Otherwise, the GUI simply makes easy things easy and hard things much harder. He writes, 'If you have to make significant, identical changes to a bunch of Linux servers, is it easier to log into them one-by-one and run through a GUI or text-menu tool, or write a quick shell script that hits each box and either makes the changes or simply pulls down a few new config files and restarts some services? And it's not just about conservation of effort - it's also about accuracy. If you write a script, you're certain that the changes made will be identical on each box. If you're doing them all by hand, you aren't.'"

alain94040 (785132)

Here is a Link to the print version of the article [infoworld.com] (that conveniently fits on 1 page instead of 3).

Providing a great GUI for complex routers or Linux admin is hard. Of course there has to be a CLI, that's how pros get the job done. But a great GUI is one that teaches a new user to eventually graduate to using CLI.

A bad GUI with no CLI is the worst of both worlds, the author of the article got that right. The 80/20 rule applies: 80% of the work is common to everyone, and should be offered with a GUI. And the 20% that is custom to each sysadmin, well use the CLI.

maxwell demon:

What would be nice is if the GUI could automatically create a shell script doing the change. That way you could (a) learn about how to do it per CLI by looking at the generated shell script, and (b) apply the generated shell script (after proper inspection, of course) to other computers.

0123456 (636235) writes:

What would be nice is if the GUI could automatically create a shell script doing the change.

While it's not quite the same thing, our GUI-based home router has an option to download the config as a text file so you can automatically reconfigure it from that file if it has to be reset to defaults. You could presumably use sed to change IP addresses, etc, and copy it to a different router. Of course it runs Linux.

Alain Williams:

AIX's SMIT did this, or rather it wrote the commands that it executed to achieve what you asked it to do. This meant that you could learn: look at what it did and find out about which CLI commands to run. You could also take them, build them into a script, copy elsewhere, ... I liked SMIT.

Ephemeriis:

What would be nice is if the GUI could automatically create a shell script doing the change. That way you could (a) learn about how to do it per CLI by looking at the generated shell script, and (b) apply the generated shell script (after proper inspection, of course) to other computers.

Cisco's GUI stuff doesn't really generate any scripts, but the commands it creates are the same things you'd type into a CLI. And the resulting configuration is just as human-readable (barring any weird naming conventions) as one built using the CLI. I've actually learned an awful lot about the Cisco CLI by using their GUI.

We've just started working with Aruba hardware. Installed a mobility controller last week. They've got a GUI that does something similar. It's all a pretty web-based front-end, but it again generates CLI commands and a human-readable configuration. I'm still very new to the platform, but I'm already learning about their CLI through the GUI. And getting work done that I wouldn't be able to if I had to look up the CLI commands for everything.

Microsoft's more recent tools are also doing this. Exchange 2007 and newer, for example, are really completely driven by the PowerShell CLI. The GUI generates commands and just feeds them into PowerShell for you. So you can again issue your commands through the GUI, and learn how you could have done it in PowerShell instead.

Anpheus:

Just about every Microsoft tool newer than 2007 does this. Virtual machine manager, SQL Server has done it for ages, I think almost all the system center tools do, etc.

It's a huge improvement.

PoV:

All good admins document their work (don't they? DON'T THEY?). With a CLI or a script that's easy: it comes down to "log in as user X, change to directory Y, run script Z with arguments A B and C - the output should look like D". Try that when all you have is a GLUI (like a GUI, but you get stuck): open this window, select that option, drag a slider, check these boxes, click Yes, three times. The output might look a little like this blurry screen shot and the only record of a successful execution is a window that disappears as soon as the application ends.

I suppose the Linux community should be grateful that windows made the fundemental systems design error of making everything graphic. Without that basic failure, Linux might never have even got the toe-hold it has now.

skids:

I think this is a stronger point than the OP: GUIs do not lead to good documentation. In fact, GUIs pretty much are limited to procedural documentation like the example you gave.

The best they can do as far as actual documentation, where the precise effect of all the widgets is explained, is a screenshot with little quote bubbles pointing to each doodad. That's a ridiculous way to document.

This is as opposed to a command reference which can organize, usually in a pretty sensible fashion, exact descriptions of what each command does.

Moreover, the GUI authors seem to have a penchant to find new names for existing CLI concepts. Even worse, those names are usually inappropriate vagueries quickly cobbled together in an off-the-cuff afterthought, and do not actually tell you where the doodad resides in the menu system. With a CLI, the name of the command or feature set is its location.

Not that even good command references are mandatory by today's pathetic standards. Even the big boys like Cisco have shown major degradation in the quality of their documentation during the last decade.

pedantic bore:

I think the author might not fully understand who most admins are. They're people who couldn't write a shell script if their lives depended on it, because they've never had to. GUI-dependent users become GUI-dependent admins.

As a percentage of computer users, people who can actually navigate a CLI are an ever-diminishing group.

arth1: /etc/resolv.conf

/etc/init.d/NetworkManager stop
chkconfig NetworkManager off
chkconfig network on
vi /etc/sysconfig/network
vi /etc/sysconfig/network-scripts/eth0

At least they named it NetworkManager, so experienced admins could recognize it as a culprit. Anything named in CamelCase is almost invariably written by new school programmers who don't grok the Unix toolbox concept and write applications instead of tools, and the bloated drivel is usually best avoided.

Darkness404 (1287218) writes: on Monday October 04, @07:21PM (#33789446)

There are more and more small businesses (5, 10 or so employees) realizing that they can get things done easier if they had a server. Because the business can't really afford to hire a sysadmin or a full-time tech person, its generally the employee who "knows computers" (you know, the person who has to help the boss check his e-mail every day, etc.) and since they don't have the knowledge of a skilled *Nix admin, a GUI makes their administration a lot easier.

So with the increasing use of servers among non-admins, it only makes sense for a growth in GUI-based solutions.

Svartalf (2997) writes: Ah... But the thing is... You don't NEED the GUI with recent Linux systems- you do with Windows.

oatworm (969674) writes: on Monday October 04, @07:38PM (#33789624) Homepage

Bingo. Realistically, if you're a company with less than a 100 employees (read: most companies), you're only going to have a handful of servers in house and they're each going to be dedicated to particular roles. You're not going to have 100 clustered fileservers - instead, you're going to have one or maybe two. You're not going to have a dozen e-mail servers - instead, you're going to have one or two. Consequently, the office admin's focus isn't going to be scalability; it just won't matter to the admin if they can script, say, creating a mailbox for 100 new users instead of just one. Instead, said office admin is going to be more focused on finding ways to do semi-unusual things (e.g. "create a VPN between this office and our new branch office", "promote this new server as a domain controller", "install SQL", etc.) that they might do, oh, once a year.

The trouble with Linux, and I'm speaking as someone who's used YaST in precisely this context, is that you have to make a choice - do you let the GUI manage it or do you CLI it? If you try to do both, there will be inconsistencies because the grammar of the config files is too ambiguous; consequently, the GUI config file parser will probably just overwrite whatever manual changes it thinks is "invalid", whether it really is or not. If you let the GUI manage it, you better hope the GUI has the flexibility necessary to meet your needs. If, for example, YaST doesn't understand named Apache virtual hosts, well, good luck figuring out where it's hiding all of the various config files that it was sensibly spreading out in multiple locations for you, and don't you dare use YaST to manage Apache again or it'll delete your Apache-legal but YaST-"invalid" directive.

The only solution I really see is for manual config file support with optional XML (or some other machine-friendly but still human-readable format) linkages. For example, if you want to hand-edit your resolv.conf, that's fine, but if the GUI is going to take over, it'll toss a directive on line 1 that says "#import resolv.conf.xml" and immediately overrides (but does not overwrite) everything following that. Then, if you still want to use the GUI but need to hand-edit something, you can edit the XML file using the appropriate syntax and know that your change will be reflected on the GUI.

That's my take. Your mileage, of course, may vary.

icebraining (1313345) writes: on Monday October 04, @07:24PM (#33789494) Homepage

I have a cheap router with only a web gui. I wrote a two line bash script that simply POSTs the right requests to URL. Simply put, HTTP interfaces, especially if they implement the right response codes, are actually very nice to script.

devent (1627873) writes:

Why Windows servers have a GUI is beyond me anyway. The servers are running 99,99% of the time without a monitor and normally you just login per ssh to a console if you need to administer them. But they are consuming the extra RAM, the extra CPU cycles and the extra security threats. I don't now, but can you de-install the GUI from a Windows server? Or better, do you have an option for no-GUI installation? Just saw the minimum hardware requirements. 512 MB RAM and 32 GB or greater disk space. My server runs

sirsnork (530512) writes: on Monday October 04, @07:43PM (#33789672)

it's called a "core" install in Server 2008 and up, and if you do that, there is no going back, you can't ever add the GUI back.

What this means is you can run a small subset of MS services that don't need GUI interaction. With R2 that subset grew somwhat as they added the ability to install .Net too, which mean't you could run IIS in a useful manner (arguably the strongest reason to want to do this in the first place).

Still it's a one way trip and you better be damn sure what services need to run on that box for the lifetime of that box or you're looking at a reinstall. Most windows admins will still tell you the risk isn't worth it.

Simple things like network configuration without a GUI in windows is tedious, and, at least last time i looked, you lost the ability to trunk network poers because the NIC manufactuers all assumed you had a GUI to configure your NICs

prichardson (603676) writes: on Monday October 04, @07:27PM (#33789520) Journal

This is also a problem with Max OS X Server. Apple builds their services from open source products and adds a GUI for configuration to make it all clickable and easy to set up. However, many options that can be set on the command line can't be set in the GUI. Even worse, making CLI changes to services can break the GUI entirely.

The hardware and software are both super stable and run really smoothly, so once everything gets set up, it's awesome. Still, it's hard for a guy who would rather make changes on the CLI to get used to.

MrEricSir (398214) writes:

Just because you're used to a CLI doesn't make it better. Why would I want to read a bunch of documentation, mess with command line options, then read whole block of text to see what it did? I'd much rather sit back in my chair, click something, and then see if it worked. Don't make me read a bunch of man pages just to do a simple task. In essence, the question here is whether it's okay for the user to be lazy and use a GUI, or whether the programmer should be too lazy to develop a GUI.

ak_hepcat (468765) writes: <leif@MENCKENdenali.net minus author> on Monday October 04, @07:38PM (#33789626) Homepage Journal

Probably because it's also about the ease of troubleshooting issues.

How do you troubleshoot something with a GUI after you've misconfigured? How do you troubleshoot a programming error (bug) in the GUI -> device communication? How do you scale to tens, hundreds, or thousands of devices with a GUI?

CLI makes all this easier and more manageable.

arth1 (260657) writes:

Why would I want to read a bunch of documentation, mess with command line options, then read whole block of text to see what it did? I'd much rather sit back in my chair, click something, and then see if it worked. Don't make me read a bunch of man pages just to do a simple task. Because then you'll be stuck at doing simple tasks, and will never be able to do more advanced tasks. Without hiring a team to write an app for you instead of doing it yourself in two minutes, that is. The time you spend reading man

fandingo (1541045) writes: on Monday October 04, @07:54PM (#33789778)

I don't think you really understand systems administration. 'Users,' or in this case admins, don't typically do stuff once. Furthermore, they need to know what he did and how to do it again (i.e. new server or whatever) or just remember what he did. One-off stuff isn't common and is a sign of poor administration (i.e. tracking changes and following processes).

What I'm trying to get at is that admins shouldn't do anything without reading the manual. As a Windows/Linux admin, I tend to find Linux easier to properly administer because I either already know how to perform an operation or I have to read the manual (manpage) and learn a decent amount about the operation (i.e. more than click here/use this flag).

Don't get me wrong, GUIs can make unknown operations significantly easier, but they often lead to poor process management. To document processes, screenshots are typically needed. They can be done well, but I find that GUI documentation (created by admins, not vendor docs) tend to be of very low quality. They are also vulnerable to 'upgrades' where vendors change the interface design. CLI programs typically have more stable interfaces, but maybe that's just because they have been around longer...

maotx (765127) writes: <maotx@NoSPAM.yahoo.com> on Monday October 04, @07:42PM (#33789666)

That's one thing Microsoft did right with Exchange 2007. They built it entirely around their new powershell CLI and then built a GUI for it. The GUI is limited in compared to what you can do with the CLI, but you can get most things done. The CLI becomes extremely handy for batch jobs and exporting statistics to csv files. I'd say it's really up there with BASH in terms of scripting, data manipulation, and integration (not just Exchange but WMI, SQL, etc.)

They tried to do similar with Windows 2008 and their Core [petri.co.il] feature, but they still have to load a GUI to present a prompt...Reply to This

Charles Dodgeson (248492) writes: <jeffrey@goldmark.org> on Monday October 04, @08:51PM (#33790206) Homepage Journal

Probably Debian would have been OK, but I was finding admin of most Linux distros a pain for exactly these reasons. I couldn't find a layer where I could do everything that I needed to do without worrying about one thing stepping on another. No doubt there are ways that I could manage a Linux system without running into different layers of management tools stepping on each other, but it was a struggle.

There were other reasons as well (although there is a lot that I miss about Linux), but I think that this was one of the leading reasons.

(NB: I realize that this is flamebait (I've got karma to burn), but that isn't my intention here.)

[Nov 28, 2017] The Stigmatization of the Unemployed

"This overly narrow hiring spec then leads to absurd, widespread complaint that companies can't find people with the right skills" . In the IT job markets such postings are often called purple squirrels
Notable quotes:
"... In particular, there seems to be an extremely popular variant of the above where the starting proposition "God makes moral people rich" is improperly converted to "Rich people are more moral" which is then readily negated to "Poor people are immoral" and then expanded to "Poor people are immoral, thus they DESERVE to suffer for it". It's essentially the theological equivalent of dividing by zero ..."
"... That said, the ranks of the neoliberals are not small. They constitute what Jonathan Schell calls a "mass minority." I suspect the neoliberals have about the same level of popular support that the Nazis did at the time of their takeover of Germany in 1932, or the Bolsheviks had in Russia at the time of their takeover in 1917, which is about 20 or 25% of the total population. ..."
"... The ranks of the neoliberals are made to appear far greater than they really are because they have all but exclusive access to the nation's megaphone. The Tea Party can muster a handful of people to disrupt a town hall meeting and it gets coast to coast, primetime coverage. But let a million people protest against bank bailouts, and it is ignored. Thus, by manipulation of the media, the mass minority is made to appear to be much larger than it really is. ..."
Mar 20, 2011 | naked capitalism

Spencer Thomas:

Very good post. Thank you.

Over the past three decades, large parts of our culture here in the US have internalized the lessons of the new Social Darwinism, with a significant body of literature to explain and justify it. Many of us have internalized, without even realizing it, the ideas of "dog eat dog", "every man for himself", "society should be structured like the animal kingdom, where the weak and sick simply die because they cannot compete, and this is healthy", and "everything that happens to you is your own fault. There is no such thing as circumstance that cannot be overcome, and certainly no birth lottery."

The levers pulled by politicians and the Fed put these things into practice, but even if we managed get different (better) politicians or Fed chairmen, ones who weren't steeped in this culture and ideology, we'd still be left with the culture in the population at large, and things like the "unemployed stigma" are likely to die very, very hard. Acceptance of the "just-world phenomenon" here in the US runs deep.

perfect stranger:

"Religion is just as vulnerable to corporate capture as is the government or the academy."

This is rather rhetorical statement, and wrong one. One need to discern spiritual aspect of religion from the religion as a tool.

Religion, as is structured, is complicit: in empoverishment, obedience, people's preconditioning, and legislative enabler in the institutions such as Supreme – and non-supreme – Court(s). It is a form of PR of the ruling class for the governing class.

DownSouth:

perfect stranger,

Religion, just like human nature, is not that easy to put in a box.

For every example you can cite where religion "is complicit: in empoverishment, obedience, people's preconditioning, and legislative enabler in the institution," I can point to an example of where religion engendered a liberating, emancipatory and revolutionary spirit.

Examples:

•Early Christianity •Nominalism •Early Protestantism •Gandhi •Martin Luther King

Now granted, there don't seem to be any recent examples of this of any note, unless we consider Chris Hedges a religionist, which I'm not sure we can do. Would it be appropriate to consider Hedges a religionist?

perfect stranger:

Yes, that maybe, just maybe be the case in early stages of forming new religion(s). In case of Christianity old rulers from Rome were trying to save own head/throne and the S.P.Q.R. imperia by adopting new religion.

You use examples of Gandhi and MLK which is highly questionable both were fighters for independence and the second, civil rights. In a word: not members of establishment just as I said there were (probably) seeing the religion as spiritual force not tool of enslavement.

Matt:

This link may provide some context:

http://en.wikipedia.org/wiki/Prosperity_theology

In particular, there seems to be an extremely popular variant of the above where the starting proposition "God makes moral people rich" is improperly converted to "Rich people are more moral" which is then readily negated to "Poor people are immoral" and then expanded to "Poor people are immoral, thus they DESERVE to suffer for it". It's essentially the theological equivalent of dividing by zero

DownSouth:

Rex,

I agree.

Poll after poll after poll has shown that a majority of Americans, and a rather significant majority, reject the values, attitudes, beliefs and opinions proselytized by the stealth religion we call "neoclassical economics."

That said, the ranks of the neoliberals are not small. They constitute what Jonathan Schell calls a "mass minority." I suspect the neoliberals have about the same level of popular support that the Nazis did at the time of their takeover of Germany in 1932, or the Bolsheviks had in Russia at the time of their takeover in 1917, which is about 20 or 25% of the total population.

The ranks of the neoliberals are made to appear far greater than they really are because they have all but exclusive access to the nation's megaphone. The Tea Party can muster a handful of people to disrupt a town hall meeting and it gets coast to coast, primetime coverage. But let a million people protest against bank bailouts, and it is ignored. Thus, by manipulation of the media, the mass minority is made to appear to be much larger than it really is.

The politicians love this, because as they carry water for their pet corporations, they can point to the Tea Partiers and say: "See what a huge upwelling of popular support I am responding to."

JTFaraday:

Well, if that's true, then the unemployed are employable but the mass mediated mentality would like them to believe they are literally and inherently unemployable so that they underestimate and under-sell themselves.

This is as much to the benefit of those who would like to pick up "damaged goods" on the cheap as those who promote the unemployment problem as one that inheres in prospective employees rather than one that is a byproduct of a bad job market lest someone be tempted to think we should address it politically.

That's where I see this blame the unemployed finger pointing really getting traction these days.

attempter:

I apologize for the fact that I only read the first few paragraphs of this before quitting in disgust.

I just can no longer abide the notion that "labor" can ever be seen by human beings as a "cost" at all. We really need to refuse to even tolerate that way of phrasing things. Workers create all wealth. Parasites have no right to exist. These are facts, and we should refuse to let argument range beyond them.

The only purpose of civilization is to provide a better way of living and for all people. This includes the right and full opportunity to work and manage for oneself and/or as a cooperative group. If civilization doesn't do that, we're better off without it.

psychohistorian:

I am one of those long term unemployed.

I suppose my biggest employment claim would be as some sort of IT techie, with numerous supply chain systems and component design, development, implementation, interfaces with other systems and ongoing support. CCNP certification and a history of techiedom going back to WEYCOS.

I have a patent (6,209,954) in my name and 12+ years of beating my head against the wall in an industry that buys compliance with the "there is no problem here, move on now" approach.

Hell, I was a junior woodchuck program administrator back in the early 70's working for the Office of the Governor of the state of Washington on CETA PSE or Public Service Employment. The office of the Governor ran the PSE program for 32 of the 39 counties in the state that were not big enough to run their own. I helped organize the project approval process in all those counties to hire folk at ( if memory serves me max of $833/mo.) to fix and expand parks and provide social and other government services as defined projects with end dates. If we didn't have the anti-public congress and other government leadership we have this could be a current component in a rational labor policy but I digress.

I have experience in the construction trades mostly as carpenter but some electrical, plumbing, HVAC, etc. also.

So, of course there is some sort of character flaw that is keeping me and all those others from employment ..right. I may have more of an excuse than others, have paid into SS for 45 years but still would work if it was available ..taking work away from other who may need it more .why set up a society where we have to compete as such for mere existence???????

One more face to this rant. We need government by the people and for the people which we do not have now. Good, public focused, not corporate focused government is bigger than any entities that exist under its jurisdiction and is kept updated by required public participation in elections and potentially other things like military, peace corps, etc. in exchange for advanced education. I say this as someone who has worked at various levels in both the public and private sectors there are ignorant and misguided folks everywhere. At least with ongoing active participation there is a chance that government would, once constructed, be able to evolve as needed within public focus .IMO.

Ishmael:

Some people would say I have been unemployed for 10 years. In 2000 after losing the last of my four CFO gigs for public companies I found it necessary to start consulting. This has lead to two of my three biggest winning years. I am usually consulting on cutting edge area of my profession and many times have large staffs reporting to me that I bring on board to get jobs done. For several years I subcontacted to a large international consulting firm to clean up projects which went wrong. Let me give some insight here.

  1. First, most good positions have gate keepers who are professional recruiters. It is near impossible to get by them and if you are unemployed they will hardly talk to you. One time talking to a recruiter at Korn Fery I was interviewing for a job I have done several times in an industry I have worked in several times. She made a statement that I had never worked at a well known company. I just about fell out of my chair laughing. At one time I was a senior level executive for the largest consulting firm in the world and lived on three continents and worked with companies on six. In addition, I had held senior positions for 2 fortune 500 firms and was the CFO for a company with $4.5 billion in revenue. I am well known at several PE firms and the founder of one of the largest mentioned in a meeting that one of his great mistakes was not investing in a very successful LBO (return of in excess of 20 multiple to investors in 18 months) I was the CFO for. In a word most recruiters are incompetent.
  2. Second, most CEO's any more are just insecure politicians. One time during an interview I had a CEO asked me to talk about some accomplishments. I was not paying to much attention as I rattled off accomplishments and the CEO went nuclear and started yelling at me that he did not know where I thought I was going with this job but the only position above the CFO job was his and he was not going anywhere. I assured him I was only interested in the CFO position and not his, but I knew the job was over. Twice feed back that I got from recruiters which they took at criticism was the "client said I seemed very assured of myself."
  3. Third, government, banking, business and the top MBA schools are based upon lying to move forward. I remember a top human resource executive telling me right before Enron, MCI and Sarbanes Oxley that I needed to learn to be more flexible. My response was that flexibility would get me an orange jump suit. Don't get me wrong, I have a wide grey zone, but it use to be in business the looked for people who could identify problems early and resolve them. Now days I see far more of a demand for people who can come up with PR spins to hide them. An attorney/treasurer consultant who partnered with me on a number of consulting jobs told me some one called me "not very charming." He said he asked what that meant, and the person who said that said, "Ish walks into a meeting and within 10 minutes he is asking about the 10,000 pound guerilla sitting in the room that no one wants to talk about." CEO do not want any challenges in their organization.
  4. Fourth, three above has lead to the hiring of very young and inexperienced people at senior levels. These people are insecure and do not want more senior and experienced people above them and than has resulted in people older than 45 not finding positions.
  5. Fifth, people are considered expendable and are fired for the lamest reasons anymore. A partner at one of the larger and more prestigious recruiting firms one time told me, "If you have a good consulting business, just stick with it. Our average placement does not last 18 months any more." Another well known recruiter in S. Cal. one time commented to me, "Your average consulting gig runs longer than our average placement."

With all of that said, I have a hard time understanding such statements as "@attempter "Workers create all wealth. Parasites have no right to exist." What does that mean? Every worker creates wealth. There is no difference in people. Sounds like communism to me. I make a good living and my net worth has grown working for myself. I have never had a consulting gig terminated by the client but I have terminated several. Usually, I am brought in to fix what several other people have failed at. I deliver basically intellectual properties to companies. Does that mean I am not a worker. I do not usually lift anything heavy or move equipment but I tell people what and where to do it so does that make me a parasite.

Those people who think everyone is equal and everyone deserves equal pay are fools or lazy. My rate is high, but what usually starts as short term projects usually run 6 months or more because companies find I can do so much more than what most of their staff can do and I am not a threat.

I would again like to have a senior challenging role at a decent size company but due to the reasons above will probably never get one. However, you can never tell. I am currently consulting for a midsize very profitable company (grew 400% last year) where I am twice the age of most people there, but everyone speaks to me with respect so you can never tell.

Lidia:

Ishmael, you're quite right. When I showed my Italian husband's resume to try and "network" in the US, my IT friends assumed he was lying about his skills and work history.

Contemporaneously, in Italy it is impossible to get a job because of incentives to hire "youth". Age discrimination is not illegal, so it's quite common to see ads that ask for a programmer under 30 with 5 years of experience in COBOL (the purple squirrel).

Hosswire

Some good points about the foolishness of recruiters, but a great deal of that foolishness is forced by the clients themselves. I used to be a recruiter myself, including at Korn Ferry in Southern California. I described the recruiting industry as "yet more proof that God hates poor people" because my job was to ignore resumes from people seeking jobs and instead "source" aka "poach" people who already had good jobs by dangling a higher salary in front of them. I didn't do it because I disparaged the unemployed, or because I could not do the basic analysis to show that a candidate had analogous or transferrable skills to the opening.

I did it because the client, as Yves said, wanted people who were literally in the same job description already. My theory is that the client wanted to have their ass covered in case the hire didn't work out, by being able to say that they looked perfect "on paper." The lesson I learned for myself and my friends looking for jobs was simple, if morally dubious. Basically, that if prospective employers are going to judge you based on a single piece of paper take full advantage of the fact that you get to write that piece of paper yourself.

Ishmael:

Hosswire - I agree with your comment. There are poor recruiters like the one I sited but in general it is the clients fault. Fear of failure. All hires have at least a 50% chance of going sideways on you. Most companies do not even have the ability to look at a resume nor to interview. I did not mean to same nasty things about recruiters, and I even do it sometimes but mine.

I look at failure in a different light than most companies. You need to be continually experimenting and changing to survive as a company and there will be some failures. The goal is to control the cost of failures while looking for the big pay off on a winner.

Mannwich:

As a former recruiter and HR "professional" (I use that term very loosely for obvious reasons), I can honestly say that you nailed it. Most big companies looking for mid to high level white collar "talent" will almost always take the perceived safest route by hiring those who look the best ON PAPER and in a suit and lack any real interviewing skills to find the real stars. What's almost comical is that companies almost always want to see the most linear resume possible because they want to see "job stability" (e.g. a CYA document in case the person fails in that job) when in many cases nobody cares about the long range view of the company anyway. My question was why should the candidate or employee care about the long range view if the employer clearly doesn't?

Ishmael:

Manwhich another on point comment. Sometimes either interviewing for a job or consulting with a CEO it starts getting to the absurd. I see all the time the requirement for stability in a persons background. Hello, where have they been the last 15 years. In addition, the higher up you go the more likely you will be terminated sometime and that is especially true if you are hired from outside the orgnanization. Companies want loyalty from an employee but offer none in return.

The average tenure for a CFO anymore is something around 18 months. I have been a first party participant (more than once) where I went through an endless recruiting process for a company (lasting more than 6 months) they final hire some one and that person is with the company for 3 months and then resigns (of course we all know it is through mutual agreement).

Ishmael:

Birch:

The real problem has become and maybe this is what you are referring to is the "Crony Capitalism." We have lost control of our financial situation. Basically, PE is not the gods of the universe that everyone thinks they are. However, every bankers secret wet dream is to become a private equity guy. Accordingly, bankers make ridiculous loans to PE because if you say no to them then you can not play in their sand box any more. Since the govt will not let the banks go bankrupt like they should then this charade continues inslaving everyone.

This country as well as many others has a large percentage of its assets tied up in over priced deals that the bankers/governments will not let collapse while the blood sucking vampires suck the life out of the assets.

On the other hand, govt is not the answer. Govt is too large and accomplishes too little.

kevin de bruxelles:

The harsh reality is that, at least in the first few rounds, companies kick to the curb their weakest links and perceived slackers. Therefore when it comes time to hire again, they are loath to go sloppy seconds on what they perceive to be some other company's rejects. They would much rather hire someone who survived the layoffs working in a similar position in a similar company. Of course the hiring company is going to have to pay for this privilege. Although not totally reliable, the fact that someone survived the layoffs provides a form social proof for their workplace abilities.

On the macro level, labor has been under attack for thirty years by off shoring and third world immigration. It is no surprise that since the working classes have been severely undermined that the middle classes would start to feel some pressure. By mass immigration and off-shoring are strongly supported by both parties. Only when the pain gets strong enough will enough people rebel and these two policies will be overturned. We still have a few years to go before this happens.

davver:

Let's say I run a factory. I produce cars and it requires very skilled work. Skilled welding, skilled machinists. Now I introduce some robotic welders and an assembly line system. The plants productivity improves and the jobs actually get easier. They require less skill, in fact I've simplified each task to something any idiot can do. Would wages go up or down? Are the workers really contributing to that increase in productivity or is it the machines and methods I created?

Lets say you think laying off or cutting the wages of my existing workers is wrong. What happens when a new entrant into the business employs a smaller workforce and lower wages, which they can do using the same technology? The new workers don't feel like they were cut down in any way, they are just happy to have a job. Before they couldn't get a job at the old plant because they lacked the skill, but now they can work in the new plant because the work is genuinely easier. Won't I go out of business?

Escariot:

I am 54 and have a ton of peers who are former white collar workers and professionals (project managers, architects, lighting designers, wholesalers and sales reps for industrial and construction materials and equipment) now out of work going on three years. Now I say out of work, I mean out of our trained and experienced fields.

We now work two or three gigs (waiting tables, mowing lawns, doing free lance, working in tourism, truck driving, moving company and fedex ups workers) and work HARD, for much much less than we did, and we are seeing the few jobs that are coming back on line going to younger workers. It is just the reality. And for most of us the descent has not been graceful, so our credit is a wreck, which also breeds a whole other level of issues as now it is common for the credit record to be a deal breaker for employment, housing, etc.

Strangely I don't sense a lot of anger or bitterness as much as humility. And gratitude for ANY work that comes our way. Health insurance? Retirement accounts? not so much.

Mickey Marzick:

Yves and I have disagreed on how extensive the postwar "pact" between management and labor was in this country. But if you drew a line from say, Trenton-Patterson, NJ to Cincinatti, OH to Minneapolis, MN, north and east of it where blue collar manufacturing in steel, rubber, auto, machinery, etc., predominated, this "pact" may have existed but ONLY because physical plant and production were concentrated there and workers could STOP production.

Outside of these heavy industrial pockets, unions were not always viewed favorably. As one moved into the rural hinterlands surrounding them there was jealously and/or outright hostility. Elsewhere, especially in the South "unions" were the exception not the rule. The differences between NE Ohio before 1975 – line from Youngstown to Toledo – and the rest of the state exemplified this pattern. Even today, the NE counties of Ohio are traditional Democratic strongholds with the rest of the state largely Republican. And I suspect this pattern existed elsewhere. But it is changing too

In any case, the demonization of the unemployed is just one notch above the vicious demonization of the poor that has always existed in this country. It's a constant reminder for those still working that you could be next – cast out into the darkness – because you "failed" or worse yet, SINNED. This internalization of the "inner cop" reinforces the dominant ideology in two ways. First, it makes any resistance by individuals still employed less likely. Second, it pits those still working against those who aren't, both of which work against the formation of any significant class consciousness amongst working people. The "oppressed" very often internalize the value system of the oppressor.

As a nation of immigrants ETHNICITY may have more explanatory power than CLASS. For increasingly, it would appear that the dominant ethnic group – suburban, white, European Americans – have thrown their lot in with corporate America. Scared of the prospect of downward social mobility and constantly reminded of URBAN America – the other America – this group is trapped with nowhere to else to go.

It's the divide and conquer strategy employed by ruling elites in this country since its founding [Federalist #10] with the Know Nothings, blaming the Irish [NINA - no Irish need apply] and playing off each successive wave of immigrants against the next. Only when the forces of production became concentrated in the urban industrial enclaves of the North was this strategy less effective. And even then internal immigration by Blacks to the North in search of employment blunted the formation of class consciousness among white ethnic industrial workers.

Wherever the postwar "pact of domination" between unions and management held sway, once physical plant was relocated elsewhere [SOUTH] and eventually offshored, unemployment began to trend upwards. First it was the "rustbelt" now it's a nationwide phenomenon. Needless to say, the "pact" between labor and management has been consigned to the dustbin of history.

White, suburban America has hitched its wagon to that of the corporate horse. Demonization of the unemployed coupled with demonization of the poor only serve to terrorize this ethnic group into acquiescence. And as the workplace becomes a multicultural matrix this ethnic group is constantly reminded of its perilous state. Until this increasingly atomized ethnic group breaks with corporate America once and for all, it's unlikely that the most debilitating scourge of all working people – UNEMPLOYMENT – will be addressed.

Make no mistake about it, involuntary UNEMPLOYMENT/UNDEREMPLYEMT is a form of terrorism and its demonization is terrorism in action. This "quiet violence" is psychological and the intimidation wrought by unemployment and/or the threat of it is intended to dehumanize individuals subjected to it. Much like spousal abuse, the emotional and psychological effects are experienced way before any physical violence. It's the inner cop that makes overt repression unnecessary. We terrorize ourselves into submission without even knowing it because we accept it or come to tolerate it. So long as we accept "unemployment" as an inevitable consequence of progress, as something unfortunate but inevitable, we will continue to travel down the road to serfdom where ARBEIT MACHT FREI!

FULL and GAINFUL EMPLOYMENT are the ultimate labor power.

Eric:

It's delicate since direct age discrimination is illegal, but when circumstances permit separating older workers they have a very tough time getting back into the workforce in an era of high health care inflation. Older folks consume more health care and if you are hiring from a huge surplus of available workers it isn't hard to steer around the more experienced. And nobody gets younger, so when you don't get job A and go for job B 2 weeks later you, you're older still!

James:

Yves said- "This overly narrow hiring spec then leads to absurd, widespread complaint that companies can't find people with the right skills"

In the IT job markets such postings are often called purple squirrels. The HR departments require the applicant to be expert in a dozen programming languages. This is an excuse to hire a foreigner on a temp h1-b or other visa.

Most people aren't aware that this model dominates the sciences. Politicians scream we have a shortage of scientists, yet it seems we only have a shortage of cheap easily exploitable labor. The economist recently pointed out the glut of scientists that currently exists in the USA.

http://www.economist.com/node/17723223

This understates the problem. The majority of PhD recipients wander through years of postdocs only to end up eventually changing fields. My observation is that the top ten schools in biochem/chemistry/physics/ biology produce enough scientists to satisfy the national demand.

The exemption from h1-b visa caps for academic institutions exacerbates the problem, providing academics with almost unlimited access to labor.

The pharmaceutical sector has been decimated over the last ten years with tens of thousands of scientists/ factory workers looking for re-training in a dwindling pool of jobs (most of which will deem you overqualified.)

http://pipeline.corante.com/archives/2011/03/03/a_postdocs_lament.php

Abe, NYC:

I wonder how the demonization of the unemployed can be so strong even in the face of close to 10% unemployment/20% underemployment. It's easy and tempting to demonize an abstract young buck or Cadillac-driving welfare queen, but when a family member or a close friend loses a job, or your kids are stuck at your place because they can't find one, shouldn't that alter your perceptions? Of course the tendency will be to blame it all on the government, but there has to be a limit to that in hard-hit places like Ohio, Colorado, or Arizona. And yet, the dynamics aren't changing or even getting worse. Maybe Wisconsin marks a turning point, I certainly hope it does

damien:

It's more than just stupid recruiting, this stigma. Having got out when the getting was good, years ago, I know that any corporate functionary would be insane to hire me now. Socialization wears off, the deformation process reverses, and the ritual and shibboleths become a joke. Even before I bailed I became a huge pain in the ass as economic exigency receded, every bosses nightmare. I suffered fools less gladly and did the right thing out of sheer anarchic malice.

You really can't maintain corporate culture without existential fear – not just, "Uh oh, I'm gonna get fired," fear, but a visceral feeling that you do not exist without a job. In properly indoctrinated workers that feeling is divorced from economic necessity. So anyone who's survived outside a while is bound to be suspect. That's a sign of economic security, and security of any sort undermines social control.

youniquelikeme:

You hit the proverbial nail with that reply. (Although, sorry, doing the right thing should not be done out of malice) The real fit has to be in the corporate yes-man culture (malleable ass kisser) to be suited for any executive position and beyond that it is the willingness to be manipulated and drained to be able to keep a job in lower echelon.

This is the new age of evolution in the work place. The class wars will make it more of an eventual revolution, but it is coming. The unemployment rate (the actual one, not the Government one) globalization and off shore hiring are not sustainable for much longer.

Something has to give, but it is more likely to snap then to come easily. People who are made to be repressed and down and out eventually find the courage to fight back and by then, it is usually not with words.

down and out in Slicon Valley:

This is the response I got from a recruiter:

"I'm going to be overly honest with you. My firm doesn't allow me to submit any candidate who hasn't worked in 6-12 months or more. Recruiting brokers are probably all similar in that way . You are going to have to go through a connection/relationship you have with a colleague, co-worker, past manager or friend to get your next job .that's my advice for you. Best of luck "

I'm 56 years old with MSEE. Gained 20+ years of experience at the best of the best (TRW, Nortel, Microsoft), have been issued a patent. Where do I sign up to gain skills required to find a job now?

Litton Graft :

"Best of the Best?" I know you're down now, but looking back at these Gov'mint contractors you've enjoyed the best socialism money can by.

Nortel/TRW bills/(ed) the Guvmint at 2x, 3x your salary, you can ride this for decades. At the same time the Inc is attached to the Guvmint ATM localities/counties are giving them a red carpet of total freedom from taxation. Double subsidies.

I've worked many years at the big boy bandits, and there is no delusion in my mind that almost anyone, can do what I do and get paid 100K+. I've never understood the mindset of some folks who work in the Wermacht Inc: "Well, someone has to do this work" or worse "What we do, no one else can do" The reason no one else "can do it" is that they are not allowed to. So, we steal from the poor to build fighter jets, write code or network an agency.

Hosswire:

I used to work as a recruiter and can tell you that I only parroted the things my clients told me. I wanted to get you hired, because I was lazy and didn't want to have to talk to someone else next.

So what do you do? To place you that recruiter needs to see on a piece of paper that you are currently working? Maybe get an email or phone call from someone who will vouch for your employment history. That should not be that hard to make happen.

Francois T :

The "bizarre way that companies now spec jobs" is essentially a coded way for mediocre managers to say without saying so explicitly that "we can afford to be extremely picky, and by God, we shall do so no matter what, because we can!"

Of course, when comes the time to hire back because, oh disaster! business is picking up again, (I'm barely caricaturing here; some managers become despondent when they realize that workers regain a bit of the higher ground; loss of power does that to lesser beings) the same idiots who designed those "overly narrow hiring spec then leads to absurd, widespread complaint that companies can't find people with the right skills" are thrown into a tailspin of despair and misery. Instead of figuring out something as simple as "if demand is better, so will our business", they can't see anything else than the (eeeek!) cost of hiring workers. Unable to break their mental corset of penny-pincher, they fail to realize that lack of qualified workers will prevent them to execute well to begin with.

And guess what: qualified workers cost money, qualified workers urgently needed cost much more.

This managerial attitude must be another factor that explain why entrepreneurship and the formation of small businesses is on the decline in the US (contrary to the confabulations of the US officialdumb and the chattering class) while rising in Europe and India/China.

Kit:

If you are 55-60, worked as a professional (i.e., engineering say) and are now unemployed you are dead meat. Sorry to be blunt but thats the way it is in the US today. Let me repeat that : Dead Meat.

I was terminated at age 59, found absolutely NOTHING even though my qualifications were outstanding. Fortunately, my company had an old style pension plan which I was able to qualify for (at age 62 without reduced benefits). So for the next 2+ years my wife and I survived on unemployment insurance, severance, accumulated vacation pay and odd jobs. Not nice – actually, a living hell.

At age 62, I applied for my pension, early social security, sold our old house (at a good profit) just before the RE crash, moved back to our home state. Then my wife qualified for social security also. Our total income is now well above the US median.

Today, someone looking at us would think we were the typical corporate retiree. We surely don't let on any differently but the experience (to get to this point) almost killed us.

I sympathize very strongly with the millions caught in this unemployment death spiral. I wish I had an answer but I just don't. We were very lucky to survive intact.

Ming:

Thank you Yves for your excellent post, and for bringing to light this crucial issue.

Thank you to all the bloggers, who add to the richness of the this discussion.

I wonder if you could comment on this Yves, and correct me if I am wrong I believe that the power of labor was sapped by the massive available supply of global labor. The favorable economic policies enacted by China (both official and unofficial), and trade negotiations between the US government and the Chinese government were critical to creating the massive supply of labor.

Thank you. No rush of course.

Nexus:

There are some odd comments and notions here that are used to support dogma and positions of prejudice. The world can be viewed in a number of ways. Firstly from a highly individualised and personal perspective – that is what has happened to me and here are my experiences. Or alternatively the world can be viewed from a broader societal perspective.

In the context of labour there has always been an unequal confrontation between those that control capital and those that offer their labour, contrary to some of the views exposed here – Marx was a first and foremost a political economist. The political economist seeks to understand the interplay of production, supply, the state and institutions like the media. Modern day economics branched off from political economy and has little value in explaining the real world as the complexity of the world has been reduced to a simplistic rationalistic model of human behaviour underpinned by other equally simplistic notions of 'supply and demand', which are in turn represented by mathematical models, which in themselves are complex but merely represent what is a simplistic view of the way the world operates. This dogmatic thinking has avoided the need to create an underpinning epistemology. This in turn underpins the notion of free choice and individualism which in itself is an illusion as it ignores the operation of the modern state and the exercise of power and influence within society.

It was stated in one of the comments that the use of capital (machines, robotics, CAD design, etc.) de-skills. This is hardly the case as skills rise for those that remain and support highly automated/continuous production factories. This is symptomatic of the owners of capital wanting to extract the maximum value for labour and this is done via the substitution of labour for capital making the labour that remains to run factories highly productive thus eliminating low skill jobs that have been picked up via services (people move into non productive low skilled occupations warehousing and retail distribution, fast food outlets, etc). Of course the worker does not realise the additional value of his or her labour as this is expropriated for the shareholders (including management as shareholders).

The issue of the US is that since the end of WW2 it is not the industrialists that have called the shots and made investments it is the financial calculus of the investment banker (Finance Capital). Other comments have tried to ignore the existence of the elites in society – I would suggest that you read C.W.Mills – The Power Elites as an analysis of how power is exercised in the US – it is not through the will of the people.

For Finance capital investments are not made on the basis of value add, or contribution through product innovation and the exchange of goods but on basis of the lowest cost inputs. Consequently, the 'elites' that make investment decisions, as they control all forms of capital seek to gain access to the cheapest cost inputs. The reality is that the US worker (a pool of 150m) is now part of a global labour pool of a couple of billion that now includes India and China. This means that the elites, US transnational corporations for instance, can access both cheaper labour pools, relocate capital and avoid worker protection (health and safety is not a concern). The strategies of moving factories via off-shoring (over 40,000 US factories closed or relocated) and out-sourcing/in-sourcing labour is also a representations of this.

The consequence for the US is that the need for domestic labour has diminished and been substituted by cheap labour to extract the arbitrage between US labour rates and those of Chinese and Indians. Ironically, in this context capital has become too successful as the mode of consumption in the US shifted from workers that were notionally the people that created the goods, earned wages and then purchased the goods they created to a new model where the worker was substituted by the consumer underpinned by cheap debt and low cost imports – it is illustrative to note that real wages have not increased in the US since the early 1970's while at the same time debt has steadily increased to underpin the illusion of wealth – the 'borrow today and pay tomorrow' mode of capitalist operation. This model of operation is now broken. The labour force is now being demonized as there is a now surplus of labour and a need to drive down labour rates through changes in legislation and austerity programs to meet those of the emerging Chinese and Indian middle class so workers rights need to be broken. Once this is done a process of in-source may take place as US labour costs will be on par with overseas labour pools.

It is ironic that during the Regan administration a number of strategic thinkers saw the threat from emerging economies and the danger of Finance Capital and created 'Project Socrates' that would have sought to re-orientate the US economy from one that was based on the rationale of Finance Capital to one that focused in productive innovation which entailed an alignment of capital investment, research and training to product innovative goods. Of course this was ignored and the rest is history. The race to the lowest input cost is ultimately self defeating as it is clear that the economy de-industrialises through labour and capital changes and living standards collapse. The elites – bankers, US transnational corporations, media, industrial military complex and the politicians don't care as they make money either way and this way you get other people overseas to work cheap for you.

S P:

Neoliberal orthodoxy treats unemployment as well as wage supression as a necessary means to fight "inflation." If there was too much power in the hands of organized labor, inflationary pressures would spiral out of control as supply of goods cannot keep up with demand.

It also treats the printing press as a necessary means to fight "deflation."

So our present scenario: widespread unemployment along with QE to infinity, food stamps for all, is exactly what you'd expect.

The problem with this orthodoxy is that it assumes unlimited growth on a planet with finite resources, particularly oil and energy. Growth is not going to solve unemployment or wages, because we are bumping up against limits to growth.

There are only two solutions. One is tax the rich and capital gains, slow growth, and reinvest the surplus into jobs/skills programs, mostly to maintain existing infrastructure or build new energy infrastructure. Even liberals like Krugman skirt around this, because they aren't willing to accept that we have the reached the end of growth and we need radical redistribution measures.

The other solution is genuine classical liberalism / libertarianism, along the lines of Austrian thought. Return to sound money, and let the deflation naturally take care of the imbalances. Yes, it would be wrenching, but it would likely be wrenching for everybody, making it fair in a universal sense.

Neither of these options is palatable to the elite classes, the financiers of Wall Street, or the leeches and bureaucrats of D.C.

So this whole experiment called America will fail.

[Nov 27, 2017] This Is Why Hewlett-Packard Just Fired Another 30K

Highly recommended!
Notable quotes:
"... Imagine working at HP and having to listen to Carly Fiorina bulldoze you...she is like a blow-torch...here are 4 minutes of Carly and Ralph Nader (if you can take it): https://www.youtube.com/watch?v=vC4JDwoRHtk ..."
"... My husband has been a software architect for 30 years at the same company. Never before has he seen the sheer unadulterated panic in the executives. All indices are down and they are planning for the worst. Quality is being sacrificed for " just get some relatively functional piece of shit out the door we can sell". He is fighting because he has always produced a stellar product and refuses to have shit tied to his name ( 90% of competitor benchmarks fail against his projects). They can't afford to lay him off, but the first time in my life I see my husband want to quit... ..."
"... HP basically makes computer equipment (PCs, servers, Printers) and software. Part of the problem is that computer hardware has been commodized. Since PCs are cheap and frequent replacements are need, People just by the cheapest models, expecting to toss it in a couple of years and by a newer model (aka the Flat screen TV model). So there is no justification to use quality components. Same is become true with the Server market. Businesses have switched to virtualization and/or cloud systems. So instead of taking a boat load of time to rebuild a crashed server, the VM is just moved to another host. ..."
"... I hung an older sign next to the one saying Information Technology. Somehow MIS-Information Technology seemed appropriate.) ..."
"... Then I got to my first duty assignment. It was about five months after the first moon landing, and the aerospace industry was facing cuts in government aerospace spending. I picked up a copy of an engineering journal in the base library and found an article about job cuts. There was a cartoon with two janitors, buckets at their feet and mops in their hands, standing before a blackboard filled with equations. Once was saying to the other, pointing to one section, "you can see where he made his mistake right here...". It represented two engineers who had been reduced to menial labor after losing their jobs. ..."
"... So while I resent all the H1Bs coming into the US - I worked with several for the last four years of my IT career, and was not at all impressed - and despise the politicians who allow it, I know that it is not the first time American STEM grads have been put out of jobs en masse. In some ways that old saying applies: the more things change, the more they stay the same ..."
"... Just like Amazon, HP will supposedly make billions in profit analyzing things in the cloud that nobody looks at and has no use to the real economy, but it makes good fodder for Power Point presentations. I am amazed how much daily productivity goes into creating fancy charts for meetings that are meaningless to the actual business of the company. ..."
"... 'Computers' cost as much - if not more time than they save, at least in corporate settings. Used to be you'd work up 3 budget projections - expected, worst case and best case, you'd have a meeting, hash it out and decide in a week. Now you have endless alternatives, endless 'tweaking' and changes and decisions take forever, with outrageous amounts of time spent on endless 'analysis' and presentations. ..."
"... A recent lay off here turned out to be quite embarrassing for Parmalat there was nobody left that knew how to properly run the place they had to rehire many ex employees as consultants-at a costly premium ..."
"... HP is laying off 80,000 workers or almost a third of its workforce, converting its long-term human capital into short-term gains for rich shareholders at an alarming rate. The reason that product quality has declined is due to the planned obsolescence that spurs needless consumerism, which is necessary to prop up our debt-backed monetary system and the capitalist-owned economy that sits on top of it. ..."
"... The world is heading for massive deflation. Computers have hit the 14 nano-meter lithography zone, the cost to go from 14nm to say 5nm is very high, and the net benefit to computing power is very low, but lets say we go from 14nm to 5nm over the next 4 years. Going from 5nm to 1nm is not going to net a large boost in computing power and the cost to shrink things down and re-tool will be very high for such an insignificant gain in performance. ..."
"... Another classic "Let's rape all we can and bail with my golden parachute" corporate leaders setting themselves up. Pile on the string of non-IT CEOs that have been leading the company to ruin. To them it is nothing more than a contest of being even worse than their predecessor. Just look at the billions each has lost before their exit. Compaq, a cluster. Palm Pilot, a dead product they paid millions for and then buried. And many others. ..."
"... Let's not beat around the bush, they're outsourcing, firing Americans and hiring cheap labor elsewhere: http://www.bloomberg.com/news/articles/2015-09-15/hewlett-packard-to-cut-up-to-30-000-more-jobs-in-restructuring It's also shifting employees to low-cost areas, and hopes to have 60 percent of its workers located in cheaper countries by 2018, Nefkens said. ..."
"... Carly Fiorina: (LOL, leading a tech company with a degree in medieval history and philosophy) While at ATT she was groomed from the Affirmative Action plan. ..."
"... It is very straightforward. Replace 45,000 US workers with 100,000 offshore workers and you still save millions of USD ! Use the "savings" to buy back stock, then borrow more $$ at ZIRP to buy more stock back. ..."
"... If you look on a site like LinkedIN, it will always say 'We're hiring!'. YES, HP is hiring.....but not YOU, they want Ganesh Balasubramaniamawapbapalooboopawapbamboomtuttifrutti, so that they can work him as modern day slave labor for ultra cheap. We can thank idiot 'leaders' like Meg Pasty Faced Whitman and Bill 'Forced Vaccinations' Gates for lobbying Congress for decades, against the rights of American workers. ..."
"... An era of leadership in computer technology has died, and there is no grave marker, not even a funeral ceremony or eulogy ... Hewlett-Packard, COMPAQ, Digital Equipment Corp, UNIVAC, Sperry-Rand, Data General, Tektronix, ZILOG, Advanced Micro Devices, Sun Microsystems, etc, etc, etc. So much change in so short a time, leaves your mind dizzy. ..."
Sep 15, 2015 | Zero Hedge

SixIsNinE

yeah thanks Carly ... HP made bullet-proof products that would last forever..... I still buy HP workstation notebooks, especially now when I can get them for $100 on ebay .... I sold HP products in the 1990s .... we had HP laserjet IIs that companies would run day & night .... virtually no maintenance ... when PCL5 came around then we had LJ IIIs .... and still companies would call for LJ I's, .... 100 pounds of invincible Printing ! .

This kind of product has no place in the World of Planned-Obsolesence .... I'm currently running an 8510w, 8530w, 2530p, Dell 6420 quad i7, hp printers hp scanners, hp pavilion desktops, .... all for less than what a Laserjet II would have cost in 1994, Total.

Not My Real Name

I still have my HP 15C scientific calculator I bought in 1983 to get me through college for my engineering degree. There is nothing better than a hand held calculator that uses Reverse Polish Notation!

BigJim

HP used to make fantastic products. I remember getting their RPN calculators back in th 80's; built like tanks. Then they decided to "add value" by removing more and more material from their consumer/"prosumer" products until they became unspeakably flimsy. They stopped holding things together with proper fastenings and starting hot melting/gluing it together, so if it died you had to cut it open to have any chance of fixing it.

I still have one of their Laserjet 4100 printers. I expect it to outlast anything they currently produce, and it must be going on 16+ years old now.

Fuck you, HP. You started selling shit and now you're eating through your seed corn. I just wish the "leaders" who did this to you had to pay some kind of penalty greater than getting $25M in a severance package.

Automatic Choke

+100. The path of HP is everything that is wrong about modern business models. I still have a 5MP laserjet (one of the first), still works great. Also have a number of 42S calculators.....my day-to-day workhorse and several spares. I don't think the present HP could even dream of making these products today.

nope-1004

How well will I profit, as a salesman, if I sell you something that works? How valuable are you, as a customer in my database, if you never come back? Confucious say "Buy another one, and if you can't afford it, f'n finance it!" It's the growing trend. Look at appliances. Nothing works anymore.

Normalcy Bias

https://en.wikipedia.org/wiki/Planned_obsolescence

Son of Loki

GE to cut Houston jobs as work moves overseas http://www.bizjournals.com/houston/news/2015/09/15/ge-to-cut-houston-job... " Yes we can! "

Automatic Choke

hey big brother.... if you are curious, there is a damn good android emulator of the HP42S available (Free42). really it is so good that it made me relax about accumulating more spares. still not quite the same as a real calculator. (the 42S, by the way, is the modernization/simplification of the classic HP41, the real hardcord very-programmable, reconfigurable, hackable unit with all the plug-in-modules that came out in the early 80s.)

Miss Expectations

Imagine working at HP and having to listen to Carly Fiorina bulldoze you...she is like a blow-torch...here are 4 minutes of Carly and Ralph Nader (if you can take it): https://www.youtube.com/watch?v=vC4JDwoRHtk

Miffed Microbiologist

My husband has been a software architect for 30 years at the same company. Never before has he seen the sheer unadulterated panic in the executives. All indices are down and they are planning for the worst. Quality is being sacrificed for " just get some relatively functional piece of shit out the door we can sell". He is fighting because he has always produced a stellar product and refuses to have shit tied to his name ( 90% of competitor benchmarks fail against his projects). They can't afford to lay him off, but the first time in my life I see my husband want to quit...

unplugged

I've been an engineer for 31 years - our managements's unspoken motto at the place I'm at (large company) is: "release it now, we'll put in the quality later". I try to put in as much as possible before the product is shoved out the door without killing myself doing it.

AGuy

Do they even make test equipment anymore?

HP test and measurement was spun off many years ago as Agilent. The electronics part of Agilent was spun off as keysight late last year.

HP basically makes computer equipment (PCs, servers, Printers) and software. Part of the problem is that computer hardware has been commodized. Since PCs are cheap and frequent replacements are need, People just by the cheapest models, expecting to toss it in a couple of years and by a newer model (aka the Flat screen TV model). So there is no justification to use quality components. Same is become true with the Server market. Businesses have switched to virtualization and/or cloud systems. So instead of taking a boat load of time to rebuild a crashed server, the VM is just moved to another host.

HP has also adopted the Computer Associates business model (aka Borg). HP buys up new tech companies and sits on the tech and never improves it. It decays and gets replaced with a system from a competitor. It also has a habit of buying outdated tech companies that never generate the revenues HP thinks it will.

BullyBearish

When Carly was CEO of HP, she instituted a draconian "pay for performance" plan. She ended up leaving with over $146 Million because she was smart enough not to specify "what type" of performance.

GeezerGeek

Regarding your statement "All those engineers choosing to pursue other opportunities", we need to realize that tech in general has been very susceptible to the vagaries of government actions. Now the employment problems are due to things like globalization and H1B programs. Some 50 years ago tech - meaning science and engineering - was hit hard as the US space program wound down. Permit me this retrospective:

I graduated from a quite good school with a BS in Physics in 1968. My timing was not all that great, since that was when they stopped granting draft deferments for graduate school. I joined the Air Force, but as an enlisted airman, not an officer. Following basic training, I was sent to learn to operate PCAM operations. That's Punched Card Accounting Machines. Collators. Sorters. Interpreters. Key punches. I was in a class with nine other enlistees. One had just gotten a Masters degree in something. Eight of us had a BS in one thing or another, but all what would now be called STEM fields. The least educated only had an Associate degree. We all enlisted simply to avoid being drafted into the Marines. (Not that there's anything wrong with the Marines, but all of us proclaimed an allergy to energetic lead projectiles and acted accordingly. Going to Canada, as many did, pretty much ensured never getting a job in STEM fields later in life.) So thanks to government action (fighting in VietNam, in this case) a significant portion of educated Americans found themselves diverted from chosen career paths. (In my case, it worked out fine. I learned to program, etc., and spent a total of over 40 years in what is now called IT. I think it was called EDP when I started the trek. Somewhere along the line it became (where I worked) Management Information Systems. MIS. And finally the department became simply Information Technology. I hung an older sign next to the one saying Information Technology. Somehow MIS-Information Technology seemed appropriate.)

Then I got to my first duty assignment. It was about five months after the first moon landing, and the aerospace industry was facing cuts in government aerospace spending. I picked up a copy of an engineering journal in the base library and found an article about job cuts. There was a cartoon with two janitors, buckets at their feet and mops in their hands, standing before a blackboard filled with equations. Once was saying to the other, pointing to one section, "you can see where he made his mistake right here...". It represented two engineers who had been reduced to menial labor after losing their jobs.

So while I resent all the H1Bs coming into the US - I worked with several for the last four years of my IT career, and was not at all impressed - and despise the politicians who allow it, I know that it is not the first time American STEM grads have been put out of jobs en masse. In some ways that old saying applies: the more things change, the more they stay the same.

If you made it this far, thanks for your patience.

adr

Just like Amazon, HP will supposedly make billions in profit analyzing things in the cloud that nobody looks at and has no use to the real economy, but it makes good fodder for Power Point presentations. I am amazed how much daily productivity goes into creating fancy charts for meetings that are meaningless to the actual business of the company.

IT'S ALL BULLSHIT!!!!!

I designed more products in one year for the small company I work for than a $15 billion corporation did throughout their entire design department employing hundreds of people. That is because 90% of their workday is spent preparing crap for meetings and they never really get anything meaningful done.

It took me one week to design a product and send it out for production branded for the company I work for, but it took six months to get the same type of product passed through the multi billion dollar corporation we license for. Because it had to pass through layer after layer of bullshit and through every level of management before it could be signed off. Then a month later somebody would change their mind in middle management and the product would need to be changed and go through the cycle all over again.

Their own bag department made six bags last year, I designed 16. Funny how I out produce a department of six people whose only job is to make bags, yet I only get paid the salary of one.

Maybe I'm just an imbecile for working hard.

Bear

You also have to add all the wasted time of employees having to sit through those presentations and the even more wasted time on Ashley Madison

cynicalskeptic

'Computers' cost as much - if not more time than they save, at least in corporate settings. Used to be you'd work up 3 budget projections - expected, worst case and best case, you'd have a meeting, hash it out and decide in a week. Now you have endless alternatives, endless 'tweaking' and changes and decisions take forever, with outrageous amounts of time spent on endless 'analysis' and presentations.

EVERY VP now has an 'Administrative Assistant' whose primary job is to develop PowerPoint presentations for the endless meetings that take up time - without any decisions ever being made.

Computers stop people from thinking. In ages past when you used a slide rule you had to know the order of magnitude of the end result. Now people make a mistake and come up with a ridiculous number and take it at face value because 'the computer' produced it.

Any exec worht anythign knew what a given line in their department or the total should be +or a small amount. I can't count the number of times budgets and analyses were WRONG because someone left off a few lines on a spreadsheet total.

Yes computer modeling for advanced tech and engineering is a help, CAD/CAM is great and many other applications in the tech/scientific world are a great help but letting computers loose in corporate and finance has produced endless waste AND - worsde - thigns like HFT (e.g. 'better' more effective ways to manipulate and cheat markets.

khnum

A recent lay off here turned out to be quite embarrassing for Parmalat there was nobody left that knew how to properly run the place they had to rehire many ex employees as consultants-at a costly premium

Anopheles

Consultants don't come at that much of a premium becaue the company doesn't have to pay benefits, vacation, sick days, or payroll taxes, etc. Plus it's really easy and cheap to get rid of consultants.

arrowrod

Obviously, you haven't worked as a consultant. You get paid by the hour. To clean up a mess. 100 hours a week are not uncommon. (What?, is it possible to work 100 hours a week? Yes, it is, but only for about 3 months.)

RaceToTheBottom

HP Executives are trying hard to bring the company back to its roots: The ability to fit into one garage...

PrimalScream

ALL THAT Meg Whitman needs to do ... is to FIRE EVERYBODY !! Then have all the products made in China, process all the sales orders in Hong Kong, and sub-contract the accounting and tax paperwork to India. Then HP can use all the profits for stock buybacks, except of course for Meg's salary ... which will keep rising astronomically!

Herdee

That's where education gets you in America.The Government sold out America's manufacturing base to Communist China who holds the debt of the USA.Who would ever guess that right-wing neo-cons(neo-nazis) running the government would sell out to communists just to get the money for war? Very weird.

Really20

"Communist"? The Chinese government, like that of the US, never believed in worker ownership of businesses and never believed that the commerical banking system (whether owned by the state, or private corporations which act like a state) should not control money. Both countries believe in centralization of power among a few shareholders, who take the fruits of working people's labor while contributing nothing of value themselves (money being but a token that represents a claim on real capital, not capital itself.)

Management and investors ought to be separate from each other; management should be chosen by workers by universal equal vote, while a complementary investor board should be chosen by investors much as corporate boards are now. Both of these boards should be legally independent but bound organizations; the management board should run the business while the investor board should negotiate with the management board on the terms of equity issuance. No more buybacks, no more layoffs or early retirements, unless workers as a whole see a need for it to maintain the company.

The purpose of investors is to serve the real economy, not the other way round; and in turn, the purpose of the real economy is to serve humanity, not the other way around. Humans should stop being slaves to perpetual growth.

Really20

HP is laying off 80,000 workers or almost a third of its workforce, converting its long-term human capital into short-term gains for rich shareholders at an alarming rate. The reason that product quality has declined is due to the planned obsolescence that spurs needless consumerism, which is necessary to prop up our debt-backed monetary system and the capitalist-owned economy that sits on top of it.

NoWayJose

HP - that company that sells computers and printers made in China and ink cartridges made in Thailand?

Dominus Ludificatio

Another company going down the drain because their focus is short term returns with crappy products.They will also bring down any company they buy as well.

Barnaby

HP is microcosm of what Carly will do to the US: carve it like a pumpkin and leave the shell out to bake in the sun for a few weeks. But she'll make sure and poison the seeds too! Don't want anything growing out of that pesky Palm division...

Dre4dwolf

The world is heading for massive deflation. Computers have hit the 14 nano-meter lithography zone, the cost to go from 14nm to say 5nm is very high, and the net benefit to computing power is very low, but lets say we go from 14nm to 5nm over the next 4 years. Going from 5nm to 1nm is not going to net a large boost in computing power and the cost to shrink things down and re-tool will be very high for such an insignificant gain in performance.

What does that mean

  1. Computers (atleast non-quantum ones) have hit the point where about 80-90% of the potential for the current science has been tap'd
  2. This means that the consumer is not going to be put in the position where they will have to upgrade to faster systems for atleast another 7-8 years.... (because the new computer wont be that much faster than their existing one).
  3. If no one is upgrading the only IT sectors of the economy that stand to make any money are software companies (Microsoft, Apple, and other small software developers), most software has not caught up with hardware yet.
  4. We are obviously heading for massive deflation, consumer spending levels as a % are probably around where they were in the late 70s - mid 80s, this is a very deflationary environment that is being compounded by a high debt burden (most of everyones income is going to service their debts), that signals monetary tightening is going on... people simply don't have enough discretionary income to spend on new toys.

All that to me screams SELL consumer electronics stocks because profits are GOING TO DECLINE , SALES ARE GOING TO DECLINE. There is no way , no amount of buy backs will float the stocks of corporations like HP/Dell/IBM etc... it is inevitable that these stocks will be worth 30% less over the next 5 - 8 years

But what do I know? maybe I am missing something.

In anycase a lot of pressure is being put on HP to do all it can at any cost to boost the stock valuations, because so much of its stock is institution owned, they will strip the wallpaper off the walls and sell it to a recycling plant if it would give them more money to boost stock valuations. That to me signals that most of the people pressuring the board of HP to boost the stock, want them to gut the company as much as they can to boost it some trivial % points so that the majority of shares can be dumped onto muppets.

To me it pretty much also signals something is terribly wrong at HP and no one is talking about it.

PoasterToaster

Other than die shrinks there really hasn't been a lot going on in the CPU world since Intel abandoned its Netburst architecture and went back to its (Israeli created) Pentium 3 style pipeline. After that they gave up on increasing speed and resorted to selling more cores. Now that wall has been hit, they have been selling "green" and "efficient" nonsense in place of increasing power.

x86 just needs to go, but a lot is invested in it not the least of which is that 1-2 punch of forced, contrived obsolesence carried out in a joint operation with Microsoft. 15 years ago you could watch videos with no problem on your old machine using Windows XP. Fast forward to now and their chief bragging point is still "multitasking" and the ability to process datastreams like video. It's a joke.

The future is not in the current CPU paradigm of instructions per second; it will be in terms of variables per second. It will be more along the lines of what GPU manufacturers are creating with their thousands of "engines" or "processing units" per chip, rather than the 4, 6 or 12 core monsters that Intel is pushing. They have nearly given up on their roadmap to push out to 128 cores as it is. x86 just doesn't work with all that.

Dojidog

Another classic "Let's rape all we can and bail with my golden parachute" corporate leaders setting themselves up. Pile on the string of non-IT CEOs that have been leading the company to ruin. To them it is nothing more than a contest of being even worse than their predecessor. Just look at the billions each has lost before their exit. Compaq, a cluster. Palm Pilot, a dead product they paid millions for and then buried. And many others.

Think the split is going to help? Think again. Rather than taking the opportunity to fix their problems, they have just duplicated and perpetuated them into two separate entities.

HP is a company that is mired in a morass of unmanageable business processes and patchwork of antiquated applications all interconnected to the point they are petrified to try and uncouple them.

Just look at their stock price since January. The insiders know. Want to fix HP? All it would take is a savvy IT based leader with a boatload of common sense. What makes money at HP? Their printers and ink. Not thinking they can provide enterprise solutions to others when they can't even get their own house in order.

I Write Code

Let's not beat around the bush, they're outsourcing, firing Americans and hiring cheap labor elsewhere: http://www.bloomberg.com/news/articles/2015-09-15/hewlett-packard-to-cut-up-to-30-000-more-jobs-in-restructuring It's also shifting employees to low-cost areas, and hopes to have 60 percent of its workers located in cheaper countries by 2018, Nefkens said.

yogibear

Carly Fiorina: (LOL, leading a tech company with a degree in medieval history and philosophy) While at ATT she was groomed from the Affirmative Action plan.

Alma Mater: Stanford University (B.A. in medieval history and philosophy); University of Maryland (MBA); Massachusetts Institute of Technology

==================================================================

Patricia Russo: (Lucent) (Dedree in Political Science). Another lady elevated through the AA plan, Russo got her bachelor's degree from Georgetown University in political science and history in 1973. She finished the advanced management program at Harvard Business School in 1989

Both ladies steered their corporations to failure.

Clowns on Acid

It is very straightforward. Replace 45,000 US workers with 100,000 offshore workers and you still save millions of USD ! Use the "savings" to buy back stock, then borrow more $$ at ZIRP to buy more stock back.

You guys don't know nuthin'.

homiegot

HP: one of the worst places you could work. Souless.

Pancho de Villa

Ladies and Gentlemen! Integrity has left the Building!

space junk

I worked there for a while and it was total garbage. There are still some great folks around, but they are getting paid less and less, and having to work longer hours for less pay while reporting to God knows who, often a foreigner with crappy engrish skills, yes likely another 'diversity hire'. People with DEEP knowledge, decades and decades, have either gotten unfairly fired or demoted, made to quit, or if they are lucky, taken some early retirement and GTFO (along with their expertise - whoopsie! who knew? unintended consequences are a bitch aren't they? )....

If you look on a site like LinkedIN, it will always say 'We're hiring!'. YES, HP is hiring.....but not YOU, they want Ganesh Balasubramaniamawapbapalooboopawapbamboomtuttifrutti, so that they can work him as modern day slave labor for ultra cheap. We can thank idiot 'leaders' like Meg Pasty Faced Whitman and Bill 'Forced Vaccinations' Gates for lobbying Congress for decades, against the rights of American workers.

Remember that Meg 'Pasty Faced' Whitman is the person who came up with the idea of a 'lights out' datacenter....that's right, it's the concept of putting all of your computers in a building, in racks, in the dark, and maybe hiring an intern to come in once a month and keep them going. This is what she actually believed. Along with her other statement to the HP workforce which says basically that the future of HP is one of total automation.....TRANSLATION: If you are a smart admin, engineer, project manager, architect, sw tester, etc.....we (HP management) think you are an IDIOT and can be replaced by a robot, a foreigner, or any other cheap worker.

Race to the bottom is like they say a space ship approaching a black hole......after a while the laws of physics and common sense, just don't apply anymore.

InnVestuhrr

An era of leadership in computer technology has died, and there is no grave marker, not even a funeral ceremony or eulogy ... Hewlett-Packard, COMPAQ, Digital Equipment Corp, UNIVAC, Sperry-Rand, Data General, Tektronix, ZILOG, Advanced Micro Devices, Sun Microsystems, etc, etc, etc. So much change in so short a time, leaves your mind dizzy.

[Nov 27, 2017] College Is Wildly Exploitative Why Arent Students Raising Hell

Highly recommended!
Notable quotes:
"... By David Masciotra, the author of Mellencamp: American Troubadour (University Press of Kentucky). He has also written for Salon, the Atlantic and the Los Angeles Review of Books. For more information visit www.davidmasciotra.com. Originally published at Alternet ..."
"... Robert Reich, in his book Supercapitalism, explains that in the past 30 years the two industries with the most excessive increases in prices are health care and higher education. ..."
"... Using student loan loot and tax subsidies backed by its $3.5 billion endowment, New York University has created a new administrative class of aristocratic compensation. The school not only continues to hire more administrators – many of whom the professors indict as having no visible value in improving the education for students bankrupting themselves to register for classes – but shamelessly increases the salaries of the academic administrative class. The top 21 administrators earn a combined total of $23,590,794 per year. The NYU portfolio includes many multi-million-dollar mansions and luxury condos, where deans and vice presidents live rent-free. ..."
"... As the managerial class grows, in size and salary, so does the full time faculty registry shrink. Use of part time instructors has soared to stratospheric heights at NYU. Adjunct instructors, despite having a minimum of a master's degree and often having a Ph.D., receive only miserly pay-per-course compensation for their work, and do not receive benefits. Many part-time college instructors must transform their lives into daily marathons, running from one school to the next, barely able to breathe between commutes and courses. Adjunct pay varies from school to school, but the average rate is $2,900 per course. ..."
"... New York Times ..."
"... to the people making decisions ..."
"... it's the executives and management generally. Just like Wall Street, many of these top administrators have perfected the art of failing upwards. ..."
"... What is the benefit? What are the risks? ..."
"... Sophomore Noell Conley lives there, too. She shows off the hotel-like room she shares with a roommate . ..."
"... "As you walk in, to the right you see our granite countertops with two sinks, one for each of the residents," she says. A partial wall separates the beds. Rather than trek down the hall to shower, they share a bathroom with the room next door. "That's really nice compared to community bathrooms that I lived in last year," Conley says. To be fair, granite countertops last longer. Tempur-Pedic is a local company - and gave a big discount. The amenities include classrooms and study space that are part of the dorm. Many of the residents are in the university's Honors program. But do student really need Apple TV in the lounges, or a smartphone app that lets them check their laundry status from afar? "Demand has been very high," says the university's Penny Cox, who is overseeing the construction of several new residence halls on campus. Before Central Hall's debut in August, the average dorm was almost half a century old, she says. That made it harder to recruit. " If you visit places like Ohio State, Michigan, Alabama," Cox says, "and you compare what we had with what they have available to offer, we were very far behind." Today colleges are competing for a more discerning consumer. Students grew up with fewer siblings, in larger homes, Cox says. They expect more privacy than previous generations - and more comforts. "These days we seem to be bringing kids up to expect a lot of material plenty," says Jean Twenge, a psychology professor at San Diego State University and author of the book "Generation Me." Those students could be in for some disappointment when they graduate , she says. "When some of these students have all these luxuries and then they get an entry-level job and they can't afford the enormous flat screen and the granite countertops," Twenge says, "then that's going to be a rude awakening." Some on campus also worry about the divide between students who can afford such luxuries and those who can't. The so-called premium dorms cost about $1,000 more per semester. Freshman Josh Johnson, who grew up in a low-income family and lives in one of the university's 1960s-era buildings, says the traditional dorm is good enough for him. ..."
"... "I wouldn't pay more just to live in a luxury dorm," he says. "It seems like I could just pay the flat rate and get the dorm I'm in. It's perfectly fine." In the near future students who want to live on campus won't have a choice. Eventually the university plans to upgrade all of its residence halls. ..."
"... Competition for students who have more sophisticated tastes than in past years is creating the perfect environment for schools to try to outdo each other with ever-more posh on-campus housing. Keeping up in the luxury dorm race is increasingly critical to a school's bottom line: A 2006 study published by the Association of Higher Education Facilities Officers found that "poorly maintained or inadequate residential facilities" was the number-one reason students rejected enrolling at institutions. PHOTO GALLERY: Click Here to See the 10 Schools with Luxury Dorms ..."
"... Private universities get most of the mentions on lists of schools with great dorms, as recent ratings by the Princeton Review, College Prowler, and Campus Splash make clear. But a few state schools that have invested in brand-new facilities are starting to show up on those reviews, too. ..."
"... While many schools offer first dibs on the nicest digs to upperclassmen on campus, as the war for student dollars ratchets up even first-year students at public colleges are living in style. Here are 10 on-campus dormitories at state schools that offer students resort-like amenities. ..."
"... Perhaps some students are afraid to protest for fear of being photographed or videographed and having their face and identity given to every prospective employer throughout America. Perhaps those students are afraid of being blackballed throughout the Great American Workplace if they are caught protesting anything on camera. ..."
"... Mao was perfectly content to promote technical education in the new China. What he deprecated (and fought to suppress) was the typical liberal arts notion of critical thinking. We're witnessing something comparable in the U.S. We're witnessing something comparable in the U.S. ..."
"... Many of the best students feel enormous pressure to succeed and have some inkling that their job prospects are growing narrower, but they almost universally accept this as the natural order of things. Their outlook: if there are 10 or 100 applicants for every available job, well, by golly, I just have to work that much harder and be the exceptional one who gets the job. ..."
"... I read things like this and think about Louis Althusser and his ideas about "Ideological State Apparatuses." While in liberal ideology the education is usually considered to be the space where opportunity to improve one's situation is founded, Althusser reached the complete opposite conclusion. For him, universities are the definitive bourgeois institution, the ideological state apparatus of the modern capitalist state par excellance . The real purpose of the university was not to level the playing field of opportunity but to preserve the advantages of the bourgeoisie and their children, allowing the class system to perpetuate/reproduce itself. ..."
"... My nephew asked me to help him with his college introductory courses in macroeconomics and accounting. I was disappointed to find out what was going on: no lectures by professors, no discussion sessions with teaching assistants; no team projects–just two automated correspondence courses, with automated computer graded problem sets objective tests – either multiple choice, fill in the blank with a number, or fill in the blank with a form answer. This from a public university that is charging tuition for attendance just as though it were really teaching something. All they're really certifying is that the student can perform exercises is correctly reporting what a couple of textbooks said about subjects of marginal relevance to his degree. My nephew understands exactly that this is going on, but still . ..."
"... The reason students accept this has to be the absolutely demobilized political culture of the United States combined with what college represents structurally to students from the middle classes: the only possibility – however remote – of achieving any kind of middle class income. ..."
"... Straight bullshit, but remember our school was just following the national (Neoliberal) model. ..."
Jun 26, 2015 | naked capitalism

Yves here. In May, we wrote up and embedded the report on how NYU exploits students and adjuncts in "The Art of the Gouge": NYU as a Model for Predatory Higher Education. This article below uses that study as a point of departure for for its discussion of how higher education has become extractive.

By David Masciotra, the author of Mellencamp: American Troubadour (University Press of Kentucky). He has also written for Salon, the Atlantic and the Los Angeles Review of Books. For more information visit www.davidmasciotra.com. Originally published at Alternet

Higher education wears the cloak of liberalism, but in policy and practice, it can be a corrupt and cutthroat system of power and exploitation. It benefits immensely from right-wing McCarthy wannabes, who in an effort to restrict academic freedom and silence political dissent, depict universities as left-wing indoctrination centers.

But the reality is that while college administrators might affix "down with the man" stickers on their office doors, many prop up a system that is severely unfair to American students and professors, a shocking number of whom struggle to make ends meet. Even the most elementary level of political science instructs that politics is about power. Power, in America, is about money: who has it? Who does not have it? Who is accumulating it? Who is losing it? Where is it going?

Four hundred faculty members at New York University, one of the nation's most expensive schools, recently released a report on how their own place of employment, legally a nonprofit institution, has become a predatory business, hardly any different in ethical practice or economic procedure than a sleazy storefront payday loan operator. Its title succinctly summarizes the new intellectual discipline deans and regents have learned to master: "The Art of The Gouge."

The result of their investigation reads as if Charles Dickens and Franz Kafka collaborated on notes for a novel. Administrators not only continue to raise tuition at staggering rates, but they burden their students with inexplicable fees, high cost burdens and expensive requirements like mandatory study abroad programs. When students question the basis of their charges, much of them hidden during the enrollment and registration phases, they find themselves lost in a tornadic swirl of forms, automated answering services and other bureaucratic debris.

Often the additional fees add up to thousands of dollars, and that comes on top of the already hefty tuition, currently $46,000 per academic year, which is more than double its rate of 2001. Tuition at NYU is higher than most colleges, but a bachelor's degree, nearly anywhere else, still comes with a punitive price tag. According to the College Board, the average cost of tuition and fees for the 2014–2015 school year was $31,231 at private colleges, $9,139 for state residents at public colleges, and $22,958 for out-of-state residents attending public universities.

Robert Reich, in his book Supercapitalism, explains that in the past 30 years the two industries with the most excessive increases in prices are health care and higher education. Lack of affordable health care is a crime, Reich argues, but at least new medicines, medical technologies, surgeries, surgery techs, and specialists can partially account for inflation. Higher education can claim no costly infrastructural or operational developments to defend its sophisticated swindle of American families. It is a high-tech, multifaceted, but old fashioned transfer of wealth from the poor, working- and middle-classes to the rich.

Using student loan loot and tax subsidies backed by its $3.5 billion endowment, New York University has created a new administrative class of aristocratic compensation. The school not only continues to hire more administrators – many of whom the professors indict as having no visible value in improving the education for students bankrupting themselves to register for classes – but shamelessly increases the salaries of the academic administrative class. The top 21 administrators earn a combined total of $23,590,794 per year. The NYU portfolio includes many multi-million-dollar mansions and luxury condos, where deans and vice presidents live rent-free.

Meanwhile, NYU has spent billions, over the past 20 years, on largely unnecessary real estate projects, buying property and renovating buildings throughout New York. The professors' analysis, NYU's US News and World Report Ranking, and student reviews demonstrate that few of these extravagant projects, aimed mostly at pleasing wealthy donors, attracting media attention, and giving administrators opulent quarters, had any impact on overall educational quality.

As the managerial class grows, in size and salary, so does the full time faculty registry shrink. Use of part time instructors has soared to stratospheric heights at NYU. Adjunct instructors, despite having a minimum of a master's degree and often having a Ph.D., receive only miserly pay-per-course compensation for their work, and do not receive benefits. Many part-time college instructors must transform their lives into daily marathons, running from one school to the next, barely able to breathe between commutes and courses. Adjunct pay varies from school to school, but the average rate is $2,900 per course.

Many schools offer rates far below the average, most especially community colleges paying only $1,000 to $1,500. Even at the best paying schools, adjuncts, as part time employees, are rarely eligible for health insurance and other benefits. Many universities place strict limits on how many courses an instructor can teach. According to a recent study, 25 percent of adjuncts receive government assistance.

The actual scandal of "The Art of the Gouge" is that even if NYU is a particularly egregious offender of basic decency and honesty, most of the report's indictments could apply equally to nearly any American university. From 2003-2013, college tuition increased by a crushing 80 percent. That far outpaces all other inflation. The closest competitor was the cost of medical care, which in the same time period, increased by a rate of 49 percent. On average, tuition in America rises eight percent on an annual basis, placing it far outside the moral universe. Most European universities charge only marginal fees for attendance, and many of them are free. Senator Bernie Sanders recently introduced a bill proposing all public universities offer free education. It received little political support, and almost no media coverage.

In order to obtain an education, students accept the paralytic weight of student debt, the only form of debt not dischargeable in bankruptcy. Before a young person can even think about buying a car, house or starting a family, she leaves college with thousands of dollars in debt: an average of $29,400 in 2012. As colleges continue to suck their students dry of every dime, the US government profits at $41.3 billion per year by collecting interest on that debt. Congress recently cut funding for Pell Grants, yet increased the budget for hiring debt collectors to target delinquent student borrowers.

The university, once an incubator of ideas and entrance into opportunity, has mutated into a tabletop model of America's economic architecture, where the top one percent of income earners now owns 40 percent of the wealth.

"The One Percent at State U," an Institute for Policy Studies report, found that at the 25 public universities with the highest paid presidents, student debt and adjunct faculty increased at dramatically higher rates than at the average state university. Marjorie Wood, the study's co-author, explained told the New York Times that extravagant executive pay is the "tip of a very large iceberg, with universities that have top-heavy executive spending also having more adjuncts, more tuition increases and more administrative spending.

Unfortunately, students seem like passive participants in their own liquidation. An American student protest timeline for 2014-'15, compiled by historian Angus Johnston, reveals that most demonstrations and rallies focused on police violence, and sexism. Those issues should inspire vigilance and activism, but only 10 out of 160 protests targeted tuition hikes for attack, and only two of those 10 events took place outside the state of California.

Class consciousness and solidarity actually exist in Chile, where in 2011 a student movement began to organize, making demands for free college. More than mere theater, high school and college students, along with many of their parental allies, engaged the political system and made specific demands for inexpensive education. The Chilean government announced that in March 2016, it will eliminate all tuition from public universities. Chile's victory for participatory democracy, equality of opportunity and social justice should instruct and inspire Americans. Triumph over extortion and embezzlement is possible.

This seems unlikely to happen in a culture, however, where even most poor Americans view themselves, in the words of John Steinbeck, as "temporarily embarrassed millionaires." The political, educational and economic ruling class of America is comfortable selling out its progeny. In the words of one student quoted in "The Art of the Gouge," "they see me as nothing more than $200,000."

washunate June 26, 2015 at 10:07 am

Awesome question in the headline.

At a basic level, I think the answer is yes, because on balance, college still provides a lot of privatized value to the individual. Being an exploited student with the College Credential Seal of Approval remains relatively much better than being an exploited non student lacking that all important seal. A college degree, for example, is practically a guarantee of avoiding the more unseemly parts of the US "justice" system.

But I think this is changing. The pressure is building from the bottom as academia loses credibility as an institution capable of, never mind interested in, serving the public good rather than simply being another profit center for connected workers. It's actually a pretty exciting time. The kiddos are getting pretty fed up, and the authoritarians at the top of the hierarchy are running out of money with which to buy off younger technocratic enablers and thought leaders and other Serious People.

washunate June 26, 2015 at 10:17 am

P.S., the author in this post demonstrates the very answer to the question. He assumes as true, without any need for support, that the very act of possessing a college degree makes one worthy of a better place in society. That mindset is why colleges can prey upon students. They hold a monopoly on access to resources in American society. My bold:

Adjunct instructors, despite having a minimum of a master's degree and often having a Ph.D., receive only miserly pay-per-course compensation for their work, and do not receive benefits.

What does having a masters degree or PhD have to do with the moral claim of all human beings to a life of dignity and purpose?

flora June 26, 2015 at 11:37 am

There are so many more job seekers per job opening now than, say, 20 or thirty years ago that a degree is used to sort out applications. Now a job that formerly listed a high school degree as a requirement may now list a college degree as a requirement, just to cut down on the number of applications.

So, no, a B.A. or B.S. doesn't confer moral worth, but it does open more job doors than a high school diploma, even if the actual work only requires high school level math, reading, science or technology.

Ben June 26, 2015 at 1:11 pm

I agree a phd often makes someone no more useful in society. However the behaviour of the kids is rational *because* employers demand a masters / phd.

Students are then caught in a trap. Employers demand the paper, often from an expensive institution. The credit is abundant thanks to govt backed loans. They are caught in a situation where as a collective it makes no sense to join in, but as an individual if they opt out they get hurt also.

Same deal for housing. It's a mad world my masters.

What can we do about this? The weak link in the chain seems to me to be employers. Why are they hurting themselves by selecting people who want higher pay but may offer little to no extra value? I work as a programmer and I often think " if we could just 'see' the non-graduate diamonds in the rough".

If employers had perfect knowledge of prospective employees *and* if they saw that a degree would make no difference to their performance universities would crumble overnight.

The state will never stop printing money via student loans. If we can fix recruitment then universities are dead.

washunate June 26, 2015 at 2:22 pm

Why are they hurting themselves by selecting people who want higher pay but may offer little to no extra value?

Yeah, I have thought a lot about that particular question of organizational behavior. It does make sense, conceptually, that somebody would disrupt the system and take people based on ability rather than credentials. Yet we are moving in the opposite direction, toward more rigidity in educational requirements for employment.

For my two cents, I think the bulk of the answer lies in how hiring specifically, and management philosophy more generally, works in practice. The people who make decisions are themselves also subject to someone else's decisions. This is true all up and down the hierarchical ladder, from board members and senior executives to the most junior managers and professionals.

It's true that someone without a degree may offer the same (or better) performance to the company. But they do not offer the same performance to the people making decisions, because those individual people also depend upon their own college degrees to sell their own labor services. To hire significant numbers of employees without degrees into important roles is to sabotage their own personal value.

Very few people are willing to be that kind of martyr. And generally speaking, they tend to self-select away from occupations where they can meaningfully influence decision-making processes in large organizations.

Absolutely, individual business owners can call BS on the whole scam. It is a way that individual people can take action against systemic oppression. Hire workers based upon their fit for the job, not their educational credentials or criminal background or skin color or sexual orientation or all of the other tests we have used. But that's not a systemic solution because the incentives created by public policy are overwhelming at large organizations to restrict who is 'qualified' to fill the good jobs (and increasingly, even the crappy jobs).

Laaughingsong June 26, 2015 at 3:03 pm

I am not so sure that this is so. So many jobs are now crapified. When I was made redundant in 2009, I could not find many jobs that fit my level of experience (just experience! I have no college degree), so I applied for anything that fit my skill set, pretty much regardless of level. I was called Overqualified. I have heard that in the past as well, but never more so during that stretch of job hunting. Remember that's with no degree. Maybe younger people don't hear it as much. But I also think life experience has something to do with it, you need to have something to compare it to. How many times did our parents tell us how different things were when they were kids, how much easier? I didn't take that on board, did y'all?

sam s smith June 26, 2015 at 4:03 pm

I blame HR.

tsk June 27, 2015 at 4:42 pm

For various reasons, people seeking work these days, especially younger job applicants, might not possess the habits of mind and behavior that would make them good employees – i.e., punctuality, the willingness to come to work every day (even when something more fun or interesting comes up, or when one has partied hard the night before), the ability to meet deadlines rather than make excuses for not meeting them, the ability to write competently at a basic level, the ability to read instructions, diagrams, charts, or any other sort of necessary background material, the ability to handle basic computation, the ability to FOLLOW instructions rather than deciding that one will pick and choose which rules and instructions to follow and which to ignore, trainability, etc.

Even if a job applicant's degree is in a totally unrelated field, the fact that he or she has managed to complete an undergraduate degree–or, if relevant, a master's or a doctorate – is often accepted by employers as a sign that the applicant has a sense of personal responsibility, a certain amount of diligence and educability, and a certain level of basic competence in reading, writing, and math.

By the same token, employers often assume that an applicant who didn't bother going to college or who couldn't complete a college degree program is probably not someone to be counted on to be a responsible, trainable, competent employee.

Obviously those who don't go to college, or who go but drop out or flunk out, end up disadvantaged when competing for jobs, which might not be fair at all in individual cases, especially now that college has been priced so far out of the range of so many bright, diligent students from among the poor and and working classes, and now even those from the middle class.

Nevertheless, in general an individual's ability to complete a college degree is not an unreasonable stand-in as evidence of that person's suitability for employment.

Roland June 27, 2015 at 5:14 pm

Nicely put, Ben.

Students are first caught in a trap of "credentials inflation" needed to obtain jobs, then caught by inflation in education costs, then stuck with undischargeable debt. And the more of them who get the credentials, the worse the credentials inflation–a spiral.

It's all fuelled by loose credit. The only beneficiaries are a managerial elite who enjoy palatial facilities.

As for the employers, they're not so bad off. Wages are coming down for credentialled employees due to all the competition. There is such a huge stock of degreed applicants that they can afford to ignore anyone who isn't. The credentials don't cost the employer–they're not spending the money, nor are they lending the money.

Modern money makes it possible for the central authorities to keep this racket going all the way up to the point of general systemic collapse. Why should they stop? Who's going to make them stop?

Bobbo June 26, 2015 at 10:19 am

The only reason the universities can get away with it is easy money. When the time comes that students actually need to pay tuition with real money, money they or their parents have actually saved, then college tuition rates will crash back down to earth. Don't blame the universities. This is the natural and inevitable outcome of easy money.

Jim June 26, 2015 at 10:54 am

Yes, college education in the US is a classic example of the effects of subsidies. Eliminate the subsidies and the whole education bubble would rapidly implode.

washunate June 26, 2015 at 11:03 am

I'm very curious if anyone will disagree with that assessment.

An obvious commonality across higher education, healthcare, housing, criminal justice, and national security is that we spend huge quantities of public money yet hold the workers receiving that money to extremely low standards of accountability for what they do with it.

tegnost June 26, 2015 at 11:38 am

Correct, it's not the universities, it's the culture that contains the universities, but the universities are training grounds for the culture so it is the universities just not only the universities Been remembering the song from my college days "my futures so bright i gotta wear shades". getting rich was the end in itself, and people who didn't make it didn't deserve anything but a whole lot of student debt,creating perverse incentives. And now we all know what the A in type a stands for at least among those who self identify as such, so yes it is the universities

Chris in Paris June 26, 2015 at 12:07 pm

I don't understand why the ability to accept guaranteed loan money doesn't come with an obligation by the school to cap tuition at a certain percentage over maximum loan amount? Would that be so hard to institute?

Ben June 26, 2015 at 1:53 pm

Student loans are debt issuance. Western states are desperate to issue debt as it's fungible with money and marked down as growth.

Borrow 120K over 3 years and it all gets paid into university coffers and reappears as "profit" now. Let some other president deal with low disposable income due to loan repayments. It's in a different electoral cycle – perfect.

jrd2 June 26, 2015 at 11:50 am

You can try to argue, but it will be hard to refute. If you give mortgages at teaser rates to anybody who can fog a mirror, you get a housing bubble. If you give student loans to any student without regard to the prospects of that student paying back the loan, you get a higher education bubble. Which will include private equity trying to catch as much of this money as they possibly can by investing in for profit educational institutions just barely adequate to benefit from federal student loan funds.

jrs June 26, 2015 at 6:16 pm

A lot of background conditions help. It helps to pump a housing bubble if there's nothing else worth investing in (including saving money at zero interest rates). It helps pump an education bubble if most of the jobs have been outsourced so people are competing more and more for fewer and fewer.

Beans June 26, 2015 at 11:51 am

I don't disagree with the statement that easy money has played the biggest role in jacking up tuition. I do strongly disagree that we shouldn't "blame" the universities. The universities are exactly where we should place the blame. The universities have become job training grounds, and yet continue to droll on and on about the importance of noble things like liberal education, the pursuit of knowledge, the importance of ideas, etc. They cannot have it both ways. Years ago, when tuition rates started escalating faster than inflation, the universities should have been the loudest critics – pointing out the cultural problems that would accompany sending the next generation into the future deeply indebted – namely that all the noble ideas learned at the university would get thrown out the window when financial reality forced recent graduates to chose between noble ideas and survival. If universities truly believed that a liberal education was important; that the pursuit of knowledge benefitted humanity – they should have led the charge to hold down tuition.

washunate June 26, 2015 at 12:47 pm

I took it to mean blame as in what allows the system to function. I heartily agree that highly paid workers at universities bear blame for what they do (and don't do) at a granular level.

It's just that they couldn't do those things without the system handing them gobs of resources, from tax deductability of charitable contributions to ignoring anti-competitive behavior in local real estate ownership to research grants and other direct funding to student loans and other indirect funding.

Jim June 26, 2015 at 3:09 pm

Regarding blaming "highly paid workers at universities" – If a society creates incentives for dysfunctional behavior such a society will have a lot of dysfunction. Eliminate the subsidies and see how quicly the educational bubble pops.

James Levy June 26, 2015 at 2:45 pm

You are ignoring the way that the rich bid up the cost of everything. 2% of the population will pay whatever the top dozen or so schools will charge so that little Billy or Sue can go to Harvard or Stanford. This leads to cost creep as the next tier ratchet up their prices in lock step with those above them, etc. The same dynamic happens with housing, at least around wealthy metropolitan areas.

daniel June 26, 2015 at 12:07 pm

Hi to you two,

A European perspective on this: yep, that's true on an international perspective. I belong to the ugly list of those readers of this blog who do not fully share the liberal values of most of you hear. However, may I say that I can agree on a lot of stuff.

US education and health-care are outrageously costly. Every European citizen moving to the states has a question: will he or she be sick whilst there. Every European parent with kids in higher education is aware that having their kids for one closing year in the US is the more they can afford (except if are a banquier d'affaires ). Is the value of the US education good? No doubt! Is is good value for money, of course not. Is the return on the money ok? It will prove disastrous, except if the USD crashed. The main reason? Easy money. As for any kind of investment. Remember that this is indeed a investment plan

Check the level of revenues of "public sector" teaching staff on both sides of the ponds. The figure for US professionals in these area are available on the Web. They are indeed much more costly than, say, North-of-Europe counterparts, "public sector" professionals in those area. Is higher education in the Netherlands sub-par when compared to the US? Of course not.

Yep financing education via the Fed (directly or not) is not only insanely costly. Just insane. The only decent solution: set up public institutions staffed with service-minded professionals that did not have to pay an insane sum to build up the curriculum themselves.

Are "public services" less efficient than private ones here in those area, health-care and higher education. Yep, most certainly. But, sure, having the fed indirectly finance the educational system just destroy any competitive savings made in building a competitive market-orientated educational system and is one of the worst way to handle your educational system.

Yep, you can do a worst use of the money, subprime or China buildings But that's all about it.

US should forget about exceptionnalism and pay attention to what North of Europe is doing in this area. Mind you, I am Southerner (of Europe). But of course I understand that trying to run these services on a federal basis is indeed "mission impossible".

Way to big! Hence the indirect Washington-decided Wall-Street-intermediated Fed-and-deficit-driven financing of higher education. Mind you: we have more and more of this bankers meddling in education in Europe and I do not like what I see.

John Zelnicker June 27, 2015 at 1:36 pm

@washunate – 6/26/15, 11:03 am. I know I'm late to the party, but I disagree. It's not the workers, it's the executives and management generally. Just like Wall Street, many of these top administrators have perfected the art of failing upwards.

IMNSHO everyone needs to stop blaming labor and/or the labor unions. It's not the front line workers, teachers, retail clerks, adjunct instructors, all those people who do the actual work rather than managing other people. Those workers have no bargaining power, and the unions have lost most of theirs, in part due to the horrible labor market, as well as other important reasons.

We have demonized virtually all of the government workers who actually do the work that enables us to even have a government (all levels) and to provide the services we demand, such as public safety, education, and infrastructure. These people are our neighbors, relatives and friends; we owe them better than this.

/end of rant

Roland June 27, 2015 at 5:20 pm

Unionized support staff at Canadian universities have had sub-inflation wage increases for nearly 20 years, while tuition has been rising at triple the rate of inflation.

So obviously one can't blame the unions for rising education costs.

Spring Texan June 28, 2015 at 8:03 am

Thanks for your rant! You said a mouthful. And could not be more correct.

Adam Eran June 26, 2015 at 12:18 pm

Omitted from this account: Federal funding for education has declined 55% since 1972. Part of the Powell memo's agenda.

It's understandable too; one can hardly blame legislators for punishing the educational establishment given the protests of the '60s and early '70s After all, they were one reason Nixon and Reagan rose to power. How dare they propose real democracy! Harumph!

To add to students' burden, there's the recent revision of bankruptcy law: student loans can no longer be retired by bankruptcy (Thanks Hillary!) It'll be interesting to see whether Hillary's vote on that bankruptcy revision becomes a campaign issue.

I also wonder whether employers will start to look for people without degrees as an indication they were intelligent enough to sidestep this extractive scam.

washunate June 26, 2015 at 1:54 pm

I'd be curious what you count as federal funding. Pell grants, for example, have expanded both in terms of the number of recipients and the amount of spending over the past 3 – 4 decades.

More generally, federal support for higher ed comes in a variety of forms. The bankruptcy law you mention is itself a form of federal funding. Tax exemption is another. Tax deductabiliity of contributions is another. So are research grants and exemptions from anti-competitive laws and so forth. There are a range of individual tax credits and deductions. The federal government also does not intervene in a lot of state supports, such as licensing practices in law and medicine that make higher ed gatekeepers to various fiefdoms and allowing universities to take fees for administering (sponsoring) charter schools. The Federal Work-Study program is probably one of the clearest specific examples of a program that offers both largely meaningless busy work and terrible wages.

As far as large employers seeking intelligence, I'm not sure that's an issue in the US? Generally speaking, the point of putting a college credential in a job requirement is precisely to find people participating in the 'scam'. If an employer is genuinely looking for intelligence, they don't have minimum educational requirements.

Laughingsong June 26, 2015 at 3:12 pm

I heard that Congress is cutting those:
http://www.washingtonpost.com/blogs/wonkblog/wp/2014/12/10/congress-cuts-federal-financial-aid-for-needy-students/

different clue June 28, 2015 at 3:06 am

Why would tuition rates come down when students need to pay with "real money, money they or their parents have actually saved. . . " ? Didn't tuition at state universities begin climbing when state governments began boycotting state universities in terms of embargoing former rates of taxpayer support to them? Leaving the state universities to try making up the difference by raising tuition? If people want to limit or reduce the tuition charged to in-state students of state universities, people will have to resume paying former rates of taxes and elect people to state government to re-target those taxes back to state universities the way they used to do before the reductions in state support to state universities.

Jesper June 26, 2015 at 10:29 am

Protest against exploitation and risk being black-listed by exploitative employers -> Only employers left are the ones who actually do want (not pretend to want) ethical people willing to stand up for what they believe in. Not many of those kind of employers around . What is the benefit? What are the risks?

Tammy June 27, 2015 at 4:35 pm

What is the benefit? What are the risks?
I am not a progressive, yet, there is always risk for solidary progress.

Andrew June 26, 2015 at 10:53 am

The author misrepresents the nature and demands of Chile's student movement.

Over the past few decades, university enrollment rates for Chileans expanded dramatically in part due to the creation of many private universities. In Chile, public universities lead the pack in terms of academic reputation and entrance is determined via competitive exams. As a result, students from poorer households who attended low-quality secondary schools generally need to look at private universities to get a degree. And these are the students to which the newly created colleges catered to.

According to Chilean legislation, universities can only function as non-profit entities. However, many of these new institutions were only nominally non-profit entities (for example, the owners of the university would also set up a real estate company that would rent the facilities to the college at above market prices) and they were very much lacking in quality. After a series of high-profile cases of universities that were open and shut within a few years leaving its students in limbo and debt, anger mounted over for-profit education.

The widespread support of the student movement was due to generalized anger about and education system that is dearly lacking in quality and to the violation of the spirit of the law regulating education. Once the student movement's demands became more specific and morphed from opposing for profit institutions to demanding free tuition for everyone, the widespread support waned quickly.

And while the government announced free tuition in public universities, there is a widespread consensus that this is a pretty terrible idea as it is regressive and involves large fiscal costs. In particular because most of the students that attend public universities come from relatively wealthy households that can afford tuition. The students that need the tuition assistance will not benefit under the new rules.

I personally benefited from the fantastically generous financial aid systems that some private American universities have set up which award grants and scholarships based on financial need only. And I believe that it is desirable for the State to guarantee that any qualified student has access to college regardless of his or her wealth I think that by romanticizing the Chilean student movement the author reveals himself to be either is dishonest or, at best, ignorant.

RanDomino June 27, 2015 at 12:23 pm

The protests also involved extremely large riots.

The Insider June 26, 2015 at 10:57 am

Students aren't protesting because they don't feel the consequences until they graduate.

One thing that struck me when I applied for a student loan a few years back to help me get through my last year of graduate school – the living expense allocation was surprisingly high. Not "student sharing an apartment with five random dudes while eating ramen and riding the bus", but more "living alone in a nice one-bedroom apartment while eating takeout and driving a car". Apocryphal stories of students using their student loans to buy new cars or take extravagant vacations were not impossible to believe.

The living expense portion of student loans is often so generous that students can live relatively well while going to school, which makes it that much easier for them to push to the backs of their minds the consequences that will come from so much debt when they graduate. Consequently, it isn't the students who are complaining – it's the former students. But by the time they are out of school and the university has their money in its pocket, it's too late for them to try and change the system.

lord koos June 26, 2015 at 11:42 am

I'm sure many students are simply happy to be in college the ugly truth hits later.

optimader June 26, 2015 at 12:39 pm

http://www.marketplace.org/topics/life/education/compete-students-colleges-roll-out-amenities

Sophomore Noell Conley lives there, too. She shows off the hotel-like room she shares with a roommate.

"As you walk in, to the right you see our granite countertops with two sinks, one for each of the residents," she says.

A partial wall separates the beds. Rather than trek down the hall to shower, they share a bathroom with the room next door.

"That's really nice compared to community bathrooms that I lived in last year," Conley says.

To be fair, granite countertops last longer. Tempur-Pedic is a local company - and gave a big discount. The amenities include classrooms and study space that are part of the dorm. Many of the residents are in the university's Honors program. But do student really need Apple TV in the lounges, or a smartphone app that lets them check their laundry status from afar?

"Demand has been very high," says the university's Penny Cox, who is overseeing the construction of several new residence halls on campus. Before Central Hall's debut in August, the average dorm was almost half a century old, she says. That made it harder to recruit.

"If you visit places like Ohio State, Michigan, Alabama," Cox says, "and you compare what we had with what they have available to offer, we were very far behind."

Today colleges are competing for a more discerning consumer. Students grew up with fewer siblings, in larger homes, Cox says. They expect more privacy than previous generations - and more comforts.

"These days we seem to be bringing kids up to expect a lot of material plenty," says Jean Twenge, a psychology professor at San Diego State University and author of the book "Generation Me."

Those students could be in for some disappointment when they graduate, she says.

"When some of these students have all these luxuries and then they get an entry-level job and they can't afford the enormous flat screen and the granite countertops," Twenge says, "then that's going to be a rude awakening."

Some on campus also worry about the divide between students who can afford such luxuries and those who can't. The so-called premium dorms cost about $1,000 more per semester. Freshman Josh Johnson, who grew up in a low-income family and lives in one of the university's 1960s-era buildings, says the traditional dorm is good enough for him.

"I wouldn't pay more just to live in a luxury dorm," he says. "It seems like I could just pay the flat rate and get the dorm I'm in. It's perfectly fine."

In the near future students who want to live on campus won't have a choice. Eventually the university plans to upgrade all of its residence halls.

So I wonder who on average will fair better navigating the post-college lifestyle/job market reality check, Noell or Josh? Personally, I would bet on the Joshes living in the 60's vintage enamel painted ciderblock dorm rooms.

optimader June 26, 2015 at 12:47 pm

Universities responding to the market

http://www.thefiscaltimes.com/Articles/2012/08/29/10-Public-Colleges-with-Insanely-Luxurious-Dorms

Competition for students who have more sophisticated tastes than in past years is creating the perfect environment for schools to try to outdo each other with ever-more posh on-campus housing. Keeping up in the luxury dorm race is increasingly critical to a school's bottom line: A 2006 study published by the Association of Higher Education Facilities Officers found that "poorly maintained or inadequate residential facilities" was the number-one reason students rejected enrolling at institutions.

PHOTO GALLERY: Click Here to See the 10 Schools with Luxury Dorms

Private universities get most of the mentions on lists of schools with great dorms, as recent ratings by the Princeton Review, College Prowler, and Campus Splash make clear. But a few state schools that have invested in brand-new facilities are starting to show up on those reviews, too.

While many schools offer first dibs on the nicest digs to upperclassmen on campus, as the war for student dollars ratchets up even first-year students at public colleges are living in style. Here are 10 on-campus dormitories at state schools that offer students resort-like amenities.

Jerry Denim June 26, 2015 at 4:37 pm

Bingo! They don't get really mad until they're in their early thirties and they are still stuck doing some menial job with no vacation time, no health insurance and a monstrous mountain of debt. Up until that point they're still working hard waiting for their ship to come in and blaming themselves for any lack of success like Steinbeck's 'embarrassed millionaires.' Then one day maybe a decade after they graduate they realize they've been conned but they've got bills to pay and other problems to worry about so they solider on. 18 year-olds are told by their high school guidance councilors, their parents and all of the adults they trust that college while expensive is a good investment and the only way to succeed. Why should they argue? They don't know any better yet.

different clue June 28, 2015 at 3:09 am

Perhaps some students are afraid to protest for fear of being photographed or videographed and having their face and identity given to every prospective employer throughout America. Perhaps those students are afraid of being blackballed throughout the Great American Workplace if they are caught protesting anything on camera.

Today isn't like the sixties when you could drop out in the confidence that you could always drop back in again. Nowadays there are ten limpets for every scar on the rock.

seabos84 June 26, 2015 at 11:16 am

the average is such a worthless number. The Data we need, and which all these parasitic professional managerial types won't provide –
x axis would be family income, by $5000 increments.
y axis would be the median debt level
we could get fancy, and also throw in how many kids are in school in each of those income increments.

BTW – this 55 yr. old troglodyte believes that 1 of the roles (note – I did NOT say "The Role") of education is preparing people to useful to society. 300++ million Americans, 7 billion humans – we ALL need shelter, reliable and safe food, reliable and safe water, sewage disposal, clothing, transportation, education, sick care, power, leisure, we should ALL have access to family wage jobs and time for BBQs with our various communities several times a year. I know plenty of techno-dweebs here in Seattle who need to learn some of the lessons of 1984, The Prince, and Shakespeare. I know plenty of fuzzies who could be a bit more useful with some rudimentary skills in engineering, or accounting, or finance, or stats, or bio, or chem
I don't know what the current education system is providing, other than some accidental good things for society at large, and mainly mechanisms for the para$ite cla$$e$ to stay parasites.

rmm.

Adam Eran June 26, 2015 at 12:22 pm

Mao was perfectly content to promote technical education in the new China. What he deprecated (and fought to suppress) was the typical liberal arts notion of critical thinking. We're witnessing something comparable in the U.S.

This suppression in China led to an increase in Mao's authority (obviously), but kept him delusional. For example, because China relied on Mao's agricultural advice, an estimated 70 million Chinese died during peacetime. But who else was to be relied upon as an authority?

Back the the U.S.S.A. (the United StateS of America): One Australian says of the American system: "You Yanks don't consult the wisdom of democracy; you enable mobs."

Tammy June 27, 2015 at 4:41 pm

Mao was perfectly content to promote technical education in the new China. What he deprecated (and fought to suppress) was the typical liberal arts notion of critical thinking. We're witnessing something comparable in the U.S. We're witnessing something comparable in the U.S.

Mao liked chaos because he believed in continuous revolution. I would argue what we're experiencing is nothing comparable to what China experienced. (I hope I've understood you correctly.)

Ted June 26, 2015 at 11:20 am

I am pretty sure a tradition of protest to affect political change in the US is a rather rare bird. Most people "protest" by changing their behavior. As an example, by questioning the value of the 46,000 local private college tuition as opposed the the 15k and 9k tiered state college options. My daughter is entering the freshman class next year, we opted for the cheaper state option because, in the end, a private school degree adds nothing, unless it is to a high name recognition institution.

I think, like housing, a downstream consequence of "the gouge" is not to question - much less understand - class relations, but to assess the value of the lifetyle choice once you are stuck with the price of paying for that lifestyle in the form of inflated debt repayments. Eventually "the folk" figure it out and encourage cheaper alternatives toward the same goal.

Jim June 26, 2015 at 3:18 pm

There's probably little point in engaging in political protest. Most people maximise their chances of success by focusing on variables over which they have some degree of control. The ability of most people to have much effect on the overall political-economic system is slight and any returns from political activity are highly uncertain.

jrs June 26, 2015 at 9:53 pm

How does anyone even expect to maintain cheap available state options without political activity? By wishful thinking I suppose?

The value of a private school might be graduating sooner, state schools are pretty overcrowded, but that may not at all be worth the debt (I doubt it almost ever is on a purely economic basis).

RabidGandhi June 27, 2015 at 7:57 pm

Maybe if we just elect the right people with cool posters and a hopey changey slogan, they'll take care of everything for us and we won't have to be politically active.

jrs June 26, 2015 at 10:04 pm

Of course refusal to engage politically because the returns to oneself by doing so are small really IS the tragedy of the commons. Thus one might say it's ethical to engage politically in order to avoid it. Some ethical action focuses on overcoming tragedy of the commons dilemmas. Of course the U.S. system being what it is I have a hard time blaming anyone for giving up.

chairman June 26, 2015 at 11:37 am

The middle class, working class and poor have no voice in politics or policy at all, and they don't know what's going on until it's too late. They've been pushed by all their high school staff that college is the only acceptable option - and often it is. What else are they going to do out of high school, work a 30 hour a week minimum wage retail job? The upper middle class and rich, who entirely monopolize the media, don't have any reason to care about skyrocketing college tuition - their parents are paying for it anyway. They'd rather write about the hip and trendy issues of the day, like trigger warnings.

Fool June 26, 2015 at 1:17 pm

To the contrary, they're hardly advised by "their high school staff"; nonetheless, subway ads for Phoenix, Monroe, etc. have a significant influence.

Uncle Bruno June 26, 2015 at 11:58 am

They're too busy working

Fool June 26, 2015 at 1:20 pm

Also Tinder.

collegestudent June 26, 2015 at 12:39 pm

Speaking as one of these college students, I think that a large part of the reason that the vast majority of students are just accepting the tuition rates is because it has become the societal norm. Growing up I can remember people saying "You need to go to college to find a good job." Because a higher education is seen as a necessity for most people, students think of tuition as just another form of taxes, acceptable and inevitable, which we will expect to get a refund on later in life.

Pitchfork June 26, 2015 at 1:03 pm

I teach at a "good" private university. Most of my students don't have a clue as to how they're being exploited. Many of the best students feel enormous pressure to succeed and have some inkling that their job prospects are growing narrower, but they almost universally accept this as the natural order of things. Their outlook: if there are 10 or 100 applicants for every available job, well, by golly, I just have to work that much harder and be the exceptional one who gets the job.

Incoming freshmen were born in the late 90s - they've never known anything but widespread corruption, financial and corporate oligarchy, i-Pads and the Long Recession.

But as other posters note, the moment of realization usually comes after four years of prolonged adolescence, luxury dorm living and excessive debt accumulation.

Tammy June 27, 2015 at 4:49 pm

Most Ph.D.'s don't either. I'd argue there have been times they have attempted to debate that exploitation is a good–for their employer and himself/herself–with linguistic games. Mind numbing . To be fair, they have a job.

Gottschee June 26, 2015 at 1:34 pm

I have watched the tuition double–double!–at my alma mater in the last eleven years. During this period, administrators have set a goal of increasing enrollment by a third, and from what I hear, they've done so. My question is always this: where is the additional tuition money going? Because as I walk through the campus, I don't really see that many improvements–yes, a new building, but that was supposedly paid for by donations and endowments. I don't see new offices for these high-priced admin people that colleges are hiring, and in fact, what I do see is an increase in the number of part-time faculty and adjuncts. The tenured faculty is not prospering from all this increased revenue, either.

I suspect the tuition is increasing so rapidly simply because the college can get away with it. And that means they are exploiting the students.

While still a student, I once calculated that it cost me $27.00/hour to be in class. (15 weeks x 20 "contact hours" per week =
300 hours/semester, $8000/semester divided by 300 hours = $27.00/hour). A crude calculation, certainly, but a starting point. I did this because I had an instructor who was consistently late to class, and often cancelled class, so much that he wiped out at least $300.00 worth of instruction. I had the gall to ask for a refund of that amount. I'm full of gall. Of course, I was laughed at, not just by the administrators, but also by some students.

Just like medical care, education pricing is "soft," that is, the price is what you are willing to pay. Desirable students get scholarships and stipends, which other students subsidize; similarly, some pre-ACA patients in hospitals were often treated gratis.

Students AND hospital patients alike seem powerless to affect the contract with the provider. Reform will not likely be forthcoming, as students, like patients, are "just passing through."

Martin Finnucane June 26, 2015 at 2:10 pm

Higher education wears the cloak of liberalism, but in policy and practice, it can be a corrupt and cutthroat system of power and exploitation.

I find the "but" in that sentence to be dissonant.

Mark Anderson June 26, 2015 at 3:12 pm

The tuition at most public universities has quadrupled or more over the last 15 to 20 years precisely BECAUSE state government subsidies have been
slashed in the meantime. I was told around 2005 that quadrupled tuition at the University of Minnesota made up for about half of the state money that the legislature had slashed from the university budget over the previous 15 years.

It is on top of that situation that university administrators are building themselves little aristocratic empires, very much modeled on the kingdoms of corporate CEOs
where reducing expenses (cutting faculty) and services to customers (fewer classes, more adjuncts) is seen as the height of responsibility and accountability, perhaps
even the definition of propriety.

Jim June 26, 2015 at 3:23 pm

Everyone should read the introductory chapter to David Graeber's " The Utopia of Rules: On Technology, Stupidity and the Secret Joys of Bureaucracy."

In Chapter One of this book entitled "The Iron law of Liberalism and the Era of Total Bureaucratization" Graeber notes that the US has become the most rigidly credentialised society in the world where

" in field after field from nurses to art teachers, physical therapists, to foreign policy consultants, careers which used to be considered an art (best learned through doing) now require formal professional training and a certificate of completion."

Graeber, in that same chapter, makes another extremely important point. when he notes that career advancement in may large bureaucratic organizations demands a willingness to play along with the fiction that advancement is based on merit, even though most everyone know that this isn't true.

The structure of modern power in the U.S., in both the merging public and private sectors, is built around the false ideology of a giant credentialized meritorcracy rather than the reality of arbitrary extraction by predatory bureaucratic networks.

armchair June 26, 2015 at 3:27 pm

Anecdote: I was speaking to someone who recently started working at as a law school administrator at my alma mater. Enrollment is actually down at law schools (I believe), because word has spread about the lame legal job market. So, the school administration is watching its pennies, and the new administrator says the administrators aren't getting to go on so many of the all expense paid conferences and junkets that they used to back in the heyday. As I hear this, I am thinking about how many of these awesome conferences in San Diego, New Orleans and New York that I'm paying back. Whatever happened to the metaphorical phrase: "when a pig becomes a hog, it goes to slaughter"?

Another anecdote: I see my undergrad alma mater has demolished the Cold War era dorms on one part of campus and replaced it with tons of slick new student housing.

MaroonBulldog June 26, 2015 at 7:15 pm

No doubt those Cold War era dorms had outlived their planned life. Time for replacement. Hell, they had probably become inhabitable and unsafe.

Meanwhile, has your undergraduate school replaced any of its lecture courses with courses presented same model as on-line traffic school? I have a pending comment below about how my nephew's public university "taught" him introductory courses in accounting and macroeconomics that way. Please be assured that the content of those courses was on a par with best practices in the on-line traffic school industry. It would be hilarious if it weren't so desperately sad.

Roquentin June 26, 2015 at 5:04 pm

I read things like this and think about Louis Althusser and his ideas about "Ideological State Apparatuses." While in liberal ideology the education is usually considered to be the space where opportunity to improve one's situation is founded, Althusser reached the complete opposite conclusion. For him, universities are the definitive bourgeois institution, the ideological state apparatus of the modern capitalist state par excellance. The real purpose of the university was not to level the playing field of opportunity but to preserve the advantages of the bourgeoisie and their children, allowing the class system to perpetuate/reproduce itself.

It certainly would explain a lot. It would explain why trying to send everyone to college won't solve this, because not everyone can have a bourgeois job. Some people actually have to do the work. The whole point of the university as an institution was to act as a sorting/distribution hub for human beings, placing them at certain points within the division of labor. A college degree used to mean more because getting it was like a golden ticket, guaranteeing someone who got it at least a petit-bourgeois lifestyle. The thing is, there are only so many slots in corporate America for this kind of employment. That number is getting smaller too. You could hand every man, woman, and child in America a BS and it wouldn't change this in the slightest.

What has happened instead, for college to preserve its role as the sorting mechanism/preservation of class advantage is what I like to call degree inflation and/or an elite formed within degrees themselves. Now a BS or BA isn't enough, one needs an Master's or PhD to really be distinguished. Now a degree from just any institution won't do, it has to be an Ivy or a Tier 1 school. Until we learn to think realistically about what higher education is as an institution little or nothing will change.

Jim June 26, 2015 at 8:14 pm

Any credential is worthless if everybody has it. All information depends on contrast. It's impossible for everybody to "stand out" from the masses. The more people have college degrees the less value a college degree has.

sid_finster June 26, 2015 at 5:49 pm

When I was half-grown, I heard it said that religion is no longer the opiate of the masses, in that no one believes in God anymore, at least not enough for it to change actual behavior.

Instead, buying on credit is the opiate of the masses.

MaroonBulldog June 26, 2015 at 6:58 pm

My nephew asked me to help him with his college introductory courses in macroeconomics and accounting. I was disappointed to find out what was going on: no lectures by professors, no discussion sessions with teaching assistants; no team projects–just two automated correspondence courses, with automated computer graded problem sets objective tests – either multiple choice, fill in the blank with a number, or fill in the blank with a form answer. This from a public university that is charging tuition for attendance just as though it were really teaching something. All they're really certifying is that the student can perform exercises is correctly reporting what a couple of textbooks said about subjects of marginal relevance to his degree. My nephew understands exactly that this is going on, but still .

This is how 21st century America treats its young people: it takes people who are poor, in the sense that they have no assets, and makes them poorer, loading them up with student debt, which they incur in order to finance a falsely-so-called course of university study that can't be a good deal, even for the best students among them.

I am not suggesting the correspondence courses have no worth at all. But they do not have the worth that is being charged for them in this bait-and-switch exercise by Ed Business.

MaroonBulldog June 27, 2015 at 1:39 am

After further thought, I'd compare my nephew's two courses to on-line traffic school: Mechanized "learning" – forget it all as soon as the test is over – Critical thinking not required. Except for the kind of "test preparation" critical thinking that teaches one to spot and eliminate the obviously wrong choices in objective answers–that kind of thinking saves time and so is very helpful.

Not only is he paying full tuition to receive this treatment, but his family and mine are paying taxes to support it, too.

Very useful preparation for later life, where we can all expect to attend traffic school a few times. But no preparation for any activity of conceivable use or benefit to any other person.

Spring Texan June 28, 2015 at 8:07 am

Good story. What a horrible rip-off!

P. Fitzsimon June 27, 2015 at 12:26 pm

I read recently that the business establishment viewed the most important contribution of colleges was that they warehoused young people for four years to allow maturing.

Fred Grosso June 27, 2015 at 4:55 pm

Where are the young people in all this? Is anyone going to start organizing to change things? Any ideas? Any interest? Are we going to have some frustrated, emotional person attempt to kill a university president once every ten years? Then education can appeal for support from the government to beef up security. Meanwhile the same old practices will prevail and the rich get richer and the rest of us get screwed.

Come on people step up.

Unorthodoxmarxist June 27, 2015 at 6:22 pm

The reason students accept this has to be the absolutely demobilized political culture of the United States combined with what college represents structurally to students from the middle classes: the only possibility – however remote – of achieving any kind of middle class income.

Really your choices in the United States are, in terms of jobs, to go into the military (and this is really for working class kids, Southern families with a military history and college-educated officer-class material) or to go to college.

The rest, who have no interest in the military, attend college, much like those who wanted to achieve despite of their class background went into the priesthood in the medieval period. There hasn't been a revolt due to the lack of any idea it could function differently and that American families are still somehow willing to pay the exorbitant rates to give their children a piece of paper that still enables them to claim middle class status though fewer and fewer find jobs. $100k in debt seems preferable to no job prospects at all.

Colleges have become a way for the ruling class to launder money into supposed non-profits and use endowments to purchase stocks, bonds, and real estate. College administrators and their lackeys (the extended school bureaucracy) are propping up another part of the financial sector – just take a look at Harvard's $30+ billion endowment, or Yale's $17 billion – these are just the top of a very large heap. They're all deep into the financial sector. Professors and students are simply there as an excuse for the alumni money machine and real estate scams to keep running, but there's less and less of a reason for them to employ professors, and I say this as a PhD with ten years of teaching experience who has seen the market dry up even more than it was when I entered grad school in the early 2000s.

A Real Black Person purple monkey dishwasher June 28, 2015 at 9:13 pm

"Colleges have become a way for the ruling class to launder money into supposed non-profits and use endowments to purchase stocks, bonds, and real estate. "

Unorthodoxmarxist, I thought I was the only person who was coming to that conclusion. I think there's data out there that could support our thesis that college tuition inflation may be affecting real estate prices. After all, justification a college grad gave to someone who was questioning the value of a college degree was that by obtaining a "a degree" and a professional job, an adult could afford to buy a home in major metropolitan hubs. I'm not sure if he was that ignorant, (business majors, despite the math requirement are highly ideological people. They're no where near as objective as they like to portray themselves as) or if he hasn't been in contact with anyone with a degree trying to buy a home in a metropolitan area.

Anyways, if our thesis is true, then if home prices declined in 2009, then college tuition should have declined as well, but it didn't at most trustworthy schools. Prospective students kept lining up to pay more for education that many insiders believe is "getting worse" because of widespread propaganda and a lack of alternatives, especially for "middle class" women.

Pelham June 27, 2015 at 7:04 pm

It's hard to say, but there ought to be a power keg of students here primed to blow. And Bernie Sanders' proposal for free college could be the fuse.

But first he'd have the light the fuse, and maybe he can. He's getting huge audiences and a lot of interest these days. And here's a timely issue. What would happen if Sanders toured colleges and called for an angry, mass and extended student strike across the country to launch on a certain date this fall or next spring to protest these obscene tuitions and maybe call for something else concrete, like a maximum ratio of administrators to faculty for colleges to receive accreditation?

It could ignite not only a long-overdue movement on campuses but also give a big boost to his campaign. He'd have millions of motivated and even furious students on his side as well as a lot of motivated and furious parents of students (my wife and I would be among them) - and these are just the types of people likely to get out and vote in the primaries and general election.

Sanders' consistent message about the middle class is a strong one. But here's a solid, specific but very wide-ranging issue that could bring that message into very sharp relief and really get a broad class of politically engaged people fired up.

I'm not one of those who think Sanders can't win but applaud his candidacy because it will nudge Hillary Clinton. I don't give a fig about Clinton. I think there's a real chance Sanders can win not just the nomination but also the presidency. This country is primed for a sharp political turn. Sanders could well be the right man in the right place and time. And this glaring and ongoing tuition ripoff that EVERYONE agrees on could be the single issue that puts him front-and-center rather than on the sidelines.

Rosario June 28, 2015 at 1:18 am

I finished graduate school about three years ago. During the pre-graduate terms that I paid out of pocket (2005-2009) I saw a near 70 percent increase in tuition (look up KY college tuition 1987-2009 for proof).

Straight bullshit, but remember our school was just following the national (Neoliberal) model.

Though, realize that I was 19-23 years old. Very immature (still immature) and feeling forces beyond my control. I did not protest out of a) fear [?] (I don't know, maybe, just threw that in there) b) the sheepskin be the path to salvation (include social/cultural pressures from parent, etc.).

I was more affected by b). This is the incredible power of our current Capitalist culture. It trains us well. We are always speaking its language, as if a Classic. Appraising its world through its values.

I wished to protest (i.e. Occupy, etc.) but to which master? All of its targets are post modern, all of it, to me, nonsense, and, because of this undead (unable to be destroyed). This coming from a young man, as I said, still immature, though I fear this misdirection, and alienation is affecting us all.

John June 28, 2015 at 10:42 am

NYU can gouge away. It's filled with Chinese students (spies) who pay full tuition.

[Nov 27, 2017] The Robot Productivity Paradox and the concept of bezel

This concept of "bezel" is an important one
Notable quotes:
"... "In many ways the effect of the crash on embezzlement was more significant than on suicide. To the economist embezzlement is the most interesting of crimes. Alone among the various forms of larceny it has a time parameter. Weeks, months or years may elapse between the commission of the crime and its discovery. (This is a period, incidentally, when the embezzler has his gain and the man who has been embezzled, oddly enough, feels no loss. There is a net increase in psychic wealth.) ..."
"... At any given time there exists an inventory of undiscovered embezzlement in – or more precisely not in – the country's business and banks. ..."
"... This inventory – it should perhaps be called the bezzle – amounts at any moment to many millions [trillions!] of dollars. It also varies in size with the business cycle. ..."
"... In good times people are relaxed, trusting, and money is plentiful. But even though money is plentiful, there are always many people who need more. Under these circumstances the rate of embezzlement grows, the rate of discovery falls off, and the bezzle increases rapidly. ..."
"... In depression all this is reversed. Money is watched with a narrow, suspicious eye. The man who handles it is assumed to be dishonest until he proves himself otherwise. Audits are penetrating and meticulous. Commercial morality is enormously improved. The bezzle shrinks ..."
Feb 22, 2017 | econospeak.blogspot.com

Sandwichman -> Sandwichman ... February 24, 2017 at 08:36 AM

John Kenneth Galbraith, from "The Great Crash 1929":

"In many ways the effect of the crash on embezzlement was more significant than on suicide. To the economist embezzlement is the most interesting of crimes. Alone among the various forms of larceny it has a time parameter. Weeks, months or years may elapse between the commission of the crime and its discovery. (This is a period, incidentally, when the embezzler has his gain and the man who has been embezzled, oddly enough, feels no loss. There is a net increase in psychic wealth.)

At any given time there exists an inventory of undiscovered embezzlement in – or more precisely not in – the country's business and banks.

This inventory – it should perhaps be called the bezzle – amounts at any moment to many millions [trillions!] of dollars. It also varies in size with the business cycle.

In good times people are relaxed, trusting, and money is plentiful. But even though money is plentiful, there are always many people who need more. Under these circumstances the rate of embezzlement grows, the rate of discovery falls off, and the bezzle increases rapidly.

In depression all this is reversed. Money is watched with a narrow, suspicious eye. The man who handles it is assumed to be dishonest until he proves himself otherwise. Audits are penetrating and meticulous. Commercial morality is enormously improved. The bezzle shrinks."

Sanwichman, February 24, 2017 at 05:24 AM

For nearly a half a century, from 1947 to 1996, real GDP and real Net Worth of Households and Non-profit Organizations (in 2009 dollars) both increased at a compound annual rate of a bit over 3.5%. GDP growth, in fact, was just a smidgen faster -- 0.016% -- than growth of Net Household Worth.

From 1996 to 2015, GDP grew at a compound annual rate of 2.3% while Net Worth increased at the rate of 3.6%....

-- Sanwichman

anne -> anne... February 24, 2017 at 05:25 AM

https://fred.stlouisfed.org/graph/?g=cOU6

January 15, 2017

Gross Domestic Product and Net Worth for Households & Nonprofit Organizations, 1952-2016

(Indexed to 1952)

https://fred.stlouisfed.org/graph/?g=cPq1

January 15, 2017

Gross Domestic Product and Net Worth for Households & Nonprofit Organizations, 1992-2016

(Indexed to 1992)

anne -> Sandwichman ... February 24, 2017 at 03:35 PM

The real home price index extends from 1890. From 1890 to 1996, the index increased slightly faster than inflation so that the index was 100 in 1890 and 113 in 1996. However from 1996 the index advanced to levels far beyond any previously experienced, reaching a high above 194 in 2006. Previously the index high had been just above 130.

Though the index fell from 2006, the level in 2016 is above 161, a level only reached when the housing bubble had formed in late 2003-early 2004.

Real home prices are again strikingly high:

http://www.econ.yale.edu/~shiller/data.htm Reply Friday, February 24, 2017 at 03:34 PM anne -> Sandwichman ... February 24, 2017

Valuation

The Shiller 10-year price-earnings ratio is currently 29.34, so the inverse or the earnings rate is 3.41%. The dividend yield is 1.93. So an expected yearly return over the coming 10 years would be 3.41 + 1.93 or 5.34% provided the price-earnings ratio stays the same and before investment costs.

Against the 5.34% yearly expected return on stock over the coming 10 years, the current 10-year Treasury bond yield is 2.32%.

The risk premium for stocks is 5.34 - 2.32 or 3.02%:

http://www.econ.yale.edu/~shiller/data.htm

anne -> anne..., February 24, 2017 at 05:36 AM

What the robot-productivity paradox is puzzles me, other than since 2005 for all the focus on the productivity of robots and on robots replacing labor there has been a dramatic, broad-spread slowing in productivity growth.

However what the changing relationship between the growth of GDP and net worth since 1996 show, is that asset valuations have been increasing relative to GDP. Valuations of stocks and homes are at sustained levels that are higher than at any time in the last 120 years. Bear markets in stocks and home prices have still left asset valuations at historically high levels. I have no idea why this should be.

Sandwichman -> anne... February 24, 2017 at 08:34 AM

The paradox is that productivity statistics can't tell us anything about the effects of robots on employment because both the numerator and the denominator are distorted by the effects of colossal Ponzi bubbles.

John Kenneth Galbraith used to call it "the bezzle." It is "that increment to wealth that occurs during the magic interval when a confidence trickster knows he has the money he has appropriated but the victim does not yet understand that he has lost it." The current size of the gross national bezzle (GNB) is approximately $24 trillion.

Ponzilocks and the Twenty-Four Trillion Dollar Question

http://econospeak.blogspot.ca/2017/02/ponzilocks-and-twenty-four-trillion.html

Twenty-three and a half trillion, actually. But what's a few hundred billion? Here today, gone tomorrow, as they say.

At the beginning of 2007, net worth of households and non-profit organizations exceeded its 1947-1996 historical average, relative to GDP, by some $16 trillion. It took 24 months to wipe out eighty percent, or $13 trillion, of that colossal but ephemeral slush fund. In mid-2016, net worth stood at a multiple of 4.83 times GDP, compared with the multiple of 4.72 on the eve of the Great Unworthing.

When I look at the ragged end of the chart I posted yesterday, it screams "Ponzi!" "Ponzi!" "Ponz..."

To make a long story short, let's think of wealth as capital. The value of capital is determined by the present value of an expected future income stream. The value of capital fluctuates with changing expectations but when the nominal value of capital diverges persistently and significantly from net revenues, something's got to give. Either economic growth is going to suddenly gush forth "like nobody has ever seen before" or net worth is going to have to come back down to earth.

Somewhere between 20 and 30 TRILLION dollars of net worth will evaporate within the span of perhaps two years.

When will that happen? Who knows? There is one notable regularity in the data, though -- the one that screams "Ponzi!"

When the net worth bubble stops going up...
...it goes down.

[Nov 27, 2017] The productivity paradox by Ryan Avent

Notable quotes:
"... But the economy does not feel like one undergoing a technology-driven productivity boom. In the late 1990s, tech optimism was everywhere. At the same time, wages and productivity were rocketing upward. The situation now is completely different. The most recent jobs reports in America and Britain tell the tale. Employment is growing, month after month after month. But wage growth is abysmal. So is productivity growth: not surprising in economies where there are lots of people on the job working for low pay. ..."
"... Increasing labour costs by making the minimum wage a living wage would increase the incentives to boost productivity growth? No, the neoliberals and corporate Democrats would never go for it. They're trying to appeal to the business community and their campaign contributors wouldn't like it. ..."
Mar 20, 2017 | medium.com

People are worried about robots taking jobs. Driverless cars are around the corner. Restaurants and shops increasingly carry the option to order by touchscreen. Google's clever algorithms provide instant translations that are remarkably good.

But the economy does not feel like one undergoing a technology-driven productivity boom. In the late 1990s, tech optimism was everywhere. At the same time, wages and productivity were rocketing upward. The situation now is completely different. The most recent jobs reports in America and Britain tell the tale. Employment is growing, month after month after month. But wage growth is abysmal. So is productivity growth: not surprising in economies where there are lots of people on the job working for low pay.

The obvious conclusion, the one lots of people are drawing, is that the robot threat is totally overblown: the fantasy, perhaps, of a bubble-mad Silicon Valley - or an effort to distract from workers' real problems, trade and excessive corporate power. Generally speaking, the problem is not that we've got too much amazing new technology but too little.

This is not a strawman of my own invention. Robert Gordon makes this case. You can see Matt Yglesias make it here. Duncan Weldon, for his part, writes:

We are debating a problem we don't have, rather than facing a real crisis that is the polar opposite. Productivity growth has slowed to a crawl over the last 15 or so years, business investment has fallen and wage growth has been weak. If the robot revolution truly was under way, we would see surging capital expenditure and soaring productivity. Right now, that would be a nice "problem" to have. Instead we have the reality of weak growth and stagnant pay. The real and pressing concern when it comes to the jobs market and automation is that the robots aren't taking our jobs fast enough.

And in a recent blog post Paul Krugman concluded:

I'd note, however, that it remains peculiar how we're simultaneously worrying that robots will take all our jobs and bemoaning the stalling out of productivity growth. What is the story, really?

What is the story, indeed. Let me see if I can tell one. Last fall I published a book: "The Wealth of Humans". In it I set out how rapid technological progress can coincide with lousy growth in pay and productivity. Start with this:

Low labour costs discourage investments in labour-saving technology, potentially reducing productivity growth.

Peter K. -> Peter K.... Monday, March 20, 2017 at 09:26 AM

Increasing labour costs by making the minimum wage a living wage would increase the incentives to boost productivity growth? No, the neoliberals and corporate Democrats would never go for it. They're trying to appeal to the business community and their campaign contributors wouldn't like it.

anne -> Peter K.... March 20, 2017 at 10:32 AM

https://twitter.com/paulkrugman/status/843167658577182725

Paul Krugman @paulkrugman

But is [Ryan Avent] saying something different from the assertion that recent tech progress is capital-biased?

https://krugman.blogs.nytimes.com/2012/12/26/capital-biased-technological-progress-an-example-wonkish/

If so, what?

anne -> Peter K.... March 20, 2017 at 10:33 AM

http://krugman.blogs.nytimes.com/2012/12/26/capital-biased-technological-progress-an-example-wonkish/

December 26, 2012

Capital-biased Technological Progress: An Example (Wonkish)
By Paul Krugman

Ever since I posted about robots and the distribution of income, * I've had queries from readers about what capital-biased technological change – the kind of change that could make society richer but workers poorer – really means. And it occurred to me that it might be useful to offer a simple conceptual example – the kind of thing easily turned into a numerical example as well – to clarify the possibility. So here goes.

Imagine that there are only two ways to produce output. One is a labor-intensive method – say, armies of scribes equipped only with quill pens. The other is a capital-intensive method – say, a handful of technicians maintaining vast server farms. (I'm thinking in terms of office work, which is the dominant occupation in the modern economy).

We can represent these two techniques in terms of unit inputs – the amount of each factor of production required to produce one unit of output. In the figure below I've assumed that initially the capital-intensive technique requires 0.2 units of labor and 0.8 units of capital per unit of output, while the labor-intensive technique requires 0.8 units of labor and 0.2 units of capital.

[Diagram]

The economy as a whole can make use of both techniques – in fact, it will have to unless it has either a very large amount of capital per worker or a very small amount. No problem: we can just use a mix of the two techniques to achieve any input combination along the blue line in the figure. For economists reading this, yes, that's the unit isoquant in this example; obviously if we had a bunch more techniques it would start to look like the convex curve of textbooks, but I want to stay simple here.

What will the distribution of income be in this case? Assuming perfect competition (yes, I know, but let's deal with that case for now), the real wage rate w and the cost of capital r – both measured in terms of output – have to be such that the cost of producing one unit is 1 whichever technique you use. In this example, that means w=r=1. Graphically, by the way, w/r is equal to minus the slope of the blue line.

Oh, and if you're worried, yes, workers and machines are both paid their marginal product.

But now suppose that technology improves – specifically, that production using the capital-intensive technique gets more efficient, although the labor-intensive technique doesn't. Scribes with quill pens are the same as they ever were; server farms can do more than ever before. In the figure, I've assumed that the unit inputs for the capital-intensive technique are cut in half. The red line shows the economy's new choices.

So what happens? It's obvious from the figure that wages fall relative to the cost of capital; it's less obvious, maybe, but nonetheless true that real wages must fall in absolute terms as well. In this specific example, technological progress reduces the real wage by a third, to 0.667, while the cost of capital rises to 2.33.

OK, it's obvious how stylized and oversimplified all this is. But it does, I think, give you some sense of what it would mean to have capital-biased technological progress, and how this could actually hurt workers.

* http://krugman.blogs.nytimes.com/2012/12/08/rise-of-the-robots/

anne -> Peter K.... March 20, 2017 at 10:34 AM

http://krugman.blogs.nytimes.com/2012/12/08/rise-of-the-robots/

December 8, 2012

Rise of the Robots
By Paul Krugman

Catherine Rampell and Nick Wingfield write about the growing evidence * for "reshoring" of manufacturing to the United States. * They cite several reasons: rising wages in Asia; lower energy costs here; higher transportation costs. In a followup piece, ** however, Rampell cites another factor: robots.

"The most valuable part of each computer, a motherboard loaded with microprocessors and memory, is already largely made with robots, according to my colleague Quentin Hardy. People do things like fitting in batteries and snapping on screens.

"As more robots are built, largely by other robots, 'assembly can be done here as well as anywhere else,' said Rob Enderle, an analyst based in San Jose, California, who has been following the computer electronics industry for a quarter-century. 'That will replace most of the workers, though you will need a few people to manage the robots.' "

Robots mean that labor costs don't matter much, so you might as well locate in advanced countries with large markets and good infrastructure (which may soon not include us, but that's another issue). On the other hand, it's not good news for workers!

This is an old concern in economics; it's "capital-biased technological change," which tends to shift the distribution of income away from workers to the owners of capital.

Twenty years ago, when I was writing about globalization and inequality, capital bias didn't look like a big issue; the major changes in income distribution had been among workers (when you include hedge fund managers and CEOs among the workers), rather than between labor and capital. So the academic literature focused almost exclusively on "skill bias", supposedly explaining the rising college premium.

But the college premium hasn't risen for a while. What has happened, on the other hand, is a notable shift in income away from labor:

[Graph]

If this is the wave of the future, it makes nonsense of just about all the conventional wisdom on reducing inequality. Better education won't do much to reduce inequality if the big rewards simply go to those with the most assets. Creating an "opportunity society," or whatever it is the likes of Paul Ryan etc. are selling this week, won't do much if the most important asset you can have in life is, well, lots of assets inherited from your parents. And so on.

I think our eyes have been averted from the capital/labor dimension of inequality, for several reasons. It didn't seem crucial back in the 1990s, and not enough people (me included!) have looked up to notice that things have changed. It has echoes of old-fashioned Marxism - which shouldn't be a reason to ignore facts, but too often is. And it has really uncomfortable implications.

But I think we'd better start paying attention to those implications.

* http://www.nytimes.com/2012/12/07/technology/apple-to-resume-us-manufacturing.html

** http://economix.blogs.nytimes.com/2012/12/07/when-cheap-foreign-labor-gets-less-cheap/

anne -> anne... March 20, 2017 at 10:41 AM

https://fred.stlouisfed.org/graph/?g=d4ZY

January 30, 2017

Compensation of Employees as a share of Gross Domestic Income, 1948-2015


https://fred.stlouisfed.org/graph/?g=d507

January 30, 2017

Compensation of Employees as a share of Gross Domestic Income, 1948-2015

(Indexed to 1948)

[Nov 27, 2017] Nineteen Ninety-Six: The Robot/Productivity Paradox and the concept of bezel

This concept of "bezel" is an important one
Feb 22, 2017 | econospeak.blogspot.com

Sandwichman -> Sandwichman ... February 24, 2017 at 08:36 AM

John Kenneth Galbraith, from "The Great Crash 1929":

"In many ways the effect of the crash on embezzlement was more significant than on suicide. To the economist embezzlement is the most interesting of crimes. Alone among the various forms of larceny it has a time parameter. Weeks, months or years may elapse between the commission of the crime and its discovery. (This is a period, incidentally, when the embezzler has his gain and the man who has been embezzled, oddly enough, feels no loss. There is a net increase in psychic wealth.)

At any given time there exists an inventory of undiscovered embezzlement in – or more precisely not in – the country's business and banks.

This inventory – it should perhaps be called the bezzle – amounts at any moment to many millions [trillions!] of dollars. It also varies in size with the business cycle.

In good times people are relaxed, trusting, and money is plentiful. But even though money is plentiful, there are always many people who need more. Under these circumstances the rate of embezzlement grows, the rate of discovery falls off, and the bezzle increases rapidly.

In depression all this is reversed. Money is watched with a narrow, suspicious eye.

The man who handles it is assumed to be dishonest until he proves himself otherwise. Audits are penetrating and meticulous. Commercial morality is enormously improved. The bezzle shrinks."

Sanwichman, February 24, 2017 at 05:24 AM

For nearly a half a century, from 1947 to 1996, real GDP and real Net Worth of Households and Non-profit Organizations (in 2009 dollars) both increased at a compound annual rate of a bit over 3.5%. GDP growth, in fact, was just a smidgen faster -- 0.016% -- than growth of Net Household Worth.

From 1996 to 2015, GDP grew at a compound annual rate of 2.3% while Net Worth increased at the rate of 3.6%....

-- Sanwichman

anne -> anne... February 24, 2017 at 05:25 AM

https://fred.stlouisfed.org/graph/?g=cOU6

January 15, 2017

Gross Domestic Product and Net Worth for Households & Nonprofit Organizations, 1952-2016

(Indexed to 1952)

https://fred.stlouisfed.org/graph/?g=cPq1

January 15, 2017

Gross Domestic Product and Net Worth for Households & Nonprofit Organizations, 1992-2016

(Indexed to 1992)

anne -> Sandwichman ... February 24, 2017 at 03:35 PM

The real home price index extends from 1890. From 1890 to 1996, the index increased slightly faster than inflation so that the index was 100 in 1890 and 113 in 1996. However from 1996 the index advanced to levels far beyond any previously experienced, reaching a high above 194 in 2006. Previously the index high had been just above 130.

Though the index fell from 2006, the level in 2016 is above 161, a level only reached when the housing bubble had formed in late 2003-early 2004.

Real home prices are again strikingly high:

http://www.econ.yale.edu/~shiller/data.htm Reply Friday, February 24, 2017 at 03:34 PM anne -> Sandwichman ... February 24, 2017

Valuation

The Shiller 10-year price-earnings ratio is currently 29.34, so the inverse or the earnings rate is 3.41%. The dividend yield is 1.93. So an expected yearly return over the coming 10 years would be 3.41 + 1.93 or 5.34% provided the price-earnings ratio stays the same and before investment costs.

Against the 5.34% yearly expected return on stock over the coming 10 years, the current 10-year Treasury bond yield is 2.32%.

The risk premium for stocks is 5.34 - 2.32 or 3.02%:

http://www.econ.yale.edu/~shiller/data.htm

anne -> anne..., February 24, 2017 at 05:36 AM

What the robot-productivity paradox is puzzles me, other than since 2005 for all the focus on the productivity of robots and on robots replacing labor there has been a dramatic, broad-spread slowing in productivity growth.

However what the changing relationship between the growth of GDP and net worth since 1996 show, is that asset valuations have been increasing relative to GDP. Valuations of stocks and homes are at sustained levels that are higher than at any time in the last 120 years. Bear markets in stocks and home prices have still left asset valuations at historically high levels. I have no idea why this should be.

Sandwichman -> anne... February 24, 2017 at 08:34 AM

The paradox is that productivity statistics can't tell us anything about the effects of robots on employment because both the numerator and the denominator are distorted by the effects of colossal Ponzi bubbles.

John Kenneth Galbraith used to call it "the bezzle." It is "that increment to wealth that occurs during the magic interval when a confidence trickster knows he has the money he has appropriated but the victim does not yet understand that he has lost it." The current size of the gross national bezzle (GNB) is approximately $24 trillion.

Ponzilocks and the Twenty-Four Trillion Dollar Question

http://econospeak.blogspot.ca/2017/02/ponzilocks-and-twenty-four-trillion.html

Twenty-three and a half trillion, actually. But what's a few hundred billion? Here today, gone tomorrow, as they say.

At the beginning of 2007, net worth of households and non-profit organizations exceeded its 1947-1996 historical average, relative to GDP, by some $16 trillion. It took 24 months to wipe out eighty percent, or $13 trillion, of that colossal but ephemeral slush fund. In mid-2016, net worth stood at a multiple of 4.83 times GDP, compared with the multiple of 4.72 on the eve of the Great Unworthing.

When I look at the ragged end of the chart I posted yesterday, it screams "Ponzi!" "Ponzi!" "Ponz..."

To make a long story short, let's think of wealth as capital. The value of capital is determined by the present value of an expected future income stream. The value of capital fluctuates with changing expectations but when the nominal value of capital diverges persistently and significantly from net revenues, something's got to give. Either economic growth is going to suddenly gush forth "like nobody has ever seen before" or net worth is going to have to come back down to earth.

Somewhere between 20 and 30 TRILLION dollars of net worth will evaporate within the span of perhaps two years.

When will that happen? Who knows? There is one notable regularity in the data, though -- the one that screams "Ponzi!"

When the net worth bubble stops going up...
...it goes down.

[Nov 22, 2017] Unemployment is Miserable and Doesn't Spawn an Upsurge in Personal Creativity

Notable quotes:
"... By Bill Mitchell, Professor in Economics and Director of the Centre of Full Employment and Equity at the University of Newcastle, NSW, Australia. Originally published at billy blog ..."
"... The overwhelming importance of having a job for happiness is evident throughout the analysis, and holds across all of the world's regions. ..."
"... The pattern of human concerns ..."
"... The pattern of human concerns ..."
"... Journal of Happiness Studies ..."
"... The results show the differences between having a job and being unemployed are "very large indeed" on the three well-being measures (life evaluation, positive and negative affective states). ..."
"... Psychological Bulletin ..."
"... 1. "unemployment tends to make people more emotionally unstable than they were previous to unemployment". ..."
"... 2. The unemployed experience feelings of "personal threat"; "fear"; "sense of proportion is shattered"; loss of "common sense of values"; "prestige lost in own eyes and as he imagines, in the eyes of his fellow men"; "feelings of inferiority"; loss of "self-confidence" and a general loss of "morale". ..."
"... in the light of the structure of our society where the job one holds is the prime indicator of status and prestige. ..."
"... Psychological Bulletin ..."
"... Related studies found that the "unemployed become so apathetic that they rarely read anything". Other activities, such as attending movies etc were seen as being motivated by the need to "kill time" – "a minimal indication of the increased desire for such attendance". ..."
"... In spite of hopeless attempts the unemployed continually look for work, often going back again and again to their last place of work. Other writers reiterate this point. ..."
"... The non-pecuniary effects of not having a job are significant in terms of lost status, social alienation, abandonment of daily structure etc, and that has not changed much over history. ..."
"... I think what is missing from this article is the term "identity." If you meet new people, often the conversation starts with what you do for a living. Your identity, in part, is what you do. You can call yourself a plumber, a writer, a banker, a consultant, a reporter but the point is this is part of your identity. When you lose your job long term, your identity here loses one of its main anchor points. ..."
"... This is a crucial point that UBI advocates often ignore. There is a deeply entrenched cultural bias towards associating our work status with our general status and prestige and feelings of these standings. ..."
"... When unemployed, the stress of worry about money may suppress the creative juices. Speaking from experience. People may well 'keep looking for jobs' because they know ultimately they need a job with steady income. The great experience of some freelancers notwithstanding, not all are cut out for it. ..."
"... When considering the world's population as a whole, people with a job evaluate the quality of their lives much more favorably than those who are unemployed. ..."
"... Data like that provided by Mitchell is important to demolishing the horrid "economic anxiety" frame much beloved by liberals, especially wonkish Democrats.* It's not (a) just feelings , to be solved by scented candles or training (the liberal version of rugged individualism) and (b) the effects are real and measurable. It's not surprising, when you think about it, that the working class is about work . ..."
Nov 22, 2017 | www.nakedcapitalism.com

Posted on November 21, 2017 by Yves Smith Yves here. Reader UserFriendly sent this post with the message, "I can confirm this." I can too. And before you try to attribute our reactions to being Americans, note that the study very clearly points out that its finding have been confirmed in "all of the world's regions".

By Bill Mitchell, Professor in Economics and Director of the Centre of Full Employment and Equity at the University of Newcastle, NSW, Australia. Originally published at billy blog

Here is a summary of another interesting study I read last week (published March 30, 2017) – Happiness at Work – from academic researchers Jan‐Emmanuel De Neve and George Ward. It explores the relationship between happiness and labour force status, including whether an individual is employed or not and the types of jobs they are doing. The results reinforce a long literature, which emphatically concludes that people are devastated when they lose their jobs and do not adapt to unemployment as its duration increases. The unemployed are miserable and remain so even as they become entrenched in long-term unemployment. Further, they do not seem to sense (or exploit) a freedom to release some inner sense of creativity and purpose. The overwhelming proportion continually seek work – and relate their social status and life happiness to gaining a job, rather than living without a job on income support. The overwhelming conclusion is that "work makes up such an important part of our lives" and that result is robust across different countries and cultures. Being employed leads to much higher evaluations of the quality of life relative to being unemployed. And, nothing much has changed in this regard over the last 80 or so years. These results were well-known in the 1930s, for example. They have a strong bearing on the debate between income guarantees versus employment guarantees. The UBI proponents have produced no robust literature to refute these long-held findings.

While the 'Happiness Study' notes that "the relationship between happiness and employment is a complex and dynamic interaction that runs in both directions" the authors are unequivocal:

The overwhelming importance of having a job for happiness is evident throughout the analysis, and holds across all of the world's regions. When considering the world's population as a whole, people with a job evaluate the quality of their lives much more favorably than those who are unemployed. The importance of having a job extends far beyond the salary attached to it, with non-pecuniary aspects of employment such as social status, social relations, daily structure, and goals all exerting a strong influence on people's happiness.

And, the inverse:

The importance of employment for people's subjective wellbeing shines a spotlight on the misery and unhappiness associated with being unemployed.

There is a burgeoning literature on 'happiness', which the authors aim to contribute to.

They define happiness as "subjective well-being", which is "measured along multiple dimensions":

life evaluation (by way of the Cantril "ladder of life"), positive and negative affect to measure respondents' experienced positive and negative wellbeing, as well as the more domain-specific items of job satisfaction and employee engagement. We find that these diverse measures of subjective wellbeing correlate strongly with each other

Cantril's 'Ladder of Life Scale' (or "Cantril Ladder") is used by polling organisations to assess well-being. It was developed by social researcher Hadley Cantril (1965) and documented in his book The pattern of human concerns .

You can learn more about the use of the 'Cantril Ladder' HERE .

As we read, the "Cantril Self-Anchoring Scale consists of the following":

Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. On which step of the ladder would you say you personally feel you stand at this time? (ladder-present) On which step do you think you will stand about five years from now? (ladder-future)

[Reference: Cantril, H. (1965) The pattern of human concerns , New Brunswick, Rutgers University Press.]

Christian Bjørnskov's 2010 article – How Comparable are the Gallup World Poll Life Satisfaction Data? – also describes how it works.

[Reference: Bjørnskov, C. (2010) 'How Comparable are the Gallup World Poll Life Satisfaction Data?', Journal of Happiness Studies , 11 (1), 41-60.]

The Cantril scale is usually reported as values between 0 and 10.

The authors in the happiness study use poll data from 150 nations which they say "is representative of 98% of the world's population". This survey data is available on a mostly annual basis since 2006.

The following graph (Figure 1 from the Study) shows "the self-reported wellbeing of individuals around the world according to whether or not they are employed."

The "bars measure the subjective wellbeing of individuals of working age" by employment status .

The results show the differences between having a job and being unemployed are "very large indeed" on the three well-being measures (life evaluation, positive and negative affective states).

People employed "evaluate the quality of their lives around 0.6 points higher on average as compared to the unemployed on a scale from 0 to 10."

The authors also conduct more sophisticated (and searching) statistical analysis (multivariate regression) which control for a range of characteristics (gender, age, education, marital status, composition of household) as well as to "account for the many political, economic, and cultural differences between countries as well as year-to-year variation".

The conclusion they reach is simple:

the unemployed evaluate the overall state of their lives less highly on the Cantril ladder and experience more negative emotions in their day-to-day lives as well as fewer positive ones. These are among the most widely accepted and replicated findings in the science of happiness Here, income is being held constant along with a number of other relevant covariates, showing that these unemployment effects go well beyond the income loss associated with losing one's job.

These results are not surprising. The earliest study of this sort of outcome was from the famous study published by Philip Eisenberg and Paul Lazersfeld in 1938. [Reference: Eisenberg, P. and Lazarsfeld, P. (1938) 'The psychological effects of unemployment', Psychological Bulletin , 35(6), 358-390.]

They explore four dimensions of unemployment:

I. The Effects of Unemployment on Personality.

II. Socio-Political Attitudes Affected by Unemployment.

III. Differing Attitudes Produced by Unemployment and Related Factors.

IV. The Effects of Unemployment on Children and Youth.

On the first dimension, they conclude that:

1. "unemployment tends to make people more emotionally unstable than they were previous to unemployment".

2. The unemployed experience feelings of "personal threat"; "fear"; "sense of proportion is shattered"; loss of "common sense of values"; "prestige lost in own eyes and as he imagines, in the eyes of his fellow men"; "feelings of inferiority"; loss of "self-confidence" and a general loss of "morale".

Devastation, in other words. They were not surprised because they note that:

in the light of the structure of our society where the job one holds is the prime indicator of status and prestige.

This is a crucial point that UBI advocates often ignore. There is a deeply entrenched cultural bias towards associating our work status with our general status and prestige and feelings of these standings. That hasn't changed since Eisenberg and Lazersfeld wrote up the findings of their study in 1938.

It might change over time but that will take a long process of re-education and cultural shift. Trying to dump a set of new cultural values that only a small minority might currently hold to onto a society that clearly still values work is only going to create major social tensions. Eisenberg and Lazarsfeld also considered an earlier 1937 study by Cantril who explored whether "the unemployed tend to evolve more imaginative schemes than the employed".

[Reference: Cantril, H. (1934) 'The Social Psychology of Everyday Life', Psychological Bulletin , 31, 297-330.]

The proposition was (is) that once unemployed, do people then explore new options that were not possible while working, which deliver them with the satisfaction that they lose when they become jobless. The specific question asked in the research was: "Have there been any changes of interests and habits among the unemployed?" Related studies found that the "unemployed become so apathetic that they rarely read anything". Other activities, such as attending movies etc were seen as being motivated by the need to "kill time" – "a minimal indication of the increased desire for such attendance".

On the third dimension, Eisenberg and Lazersfeld examine the questions – "Are there unemployed who don't want to work? Is the relief situation likely to increase this number?", which are still a central issue today – the bludger being subsidized by income support.

They concluded that:

the number is few. In spite of hopeless attempts the unemployed continually look for work, often going back again and again to their last place of work. Other writers reiterate this point.

So for decades, researchers in this area, as opposed to bloggers who wax lyrical on their own opinions, have known that the importance of work in our lives goes well beyond the income we earn. The non-pecuniary effects of not having a job are significant in terms of lost status, social alienation, abandonment of daily structure etc, and that has not changed much over history. The happiness paper did explore "how short-lived is the misery associated with being out of work" in the current cultural settings.

The proposition examined was that:

If the pain is only fleeting and people quickly get used to being unemployed, then we might see joblessness as less of a key public policy priority in terms of happiness.

They conclude that:

a number of studies have demonstrated that people do not adapt much, if at all, to being unemployed there is a large initial shock to becoming unemployed, and then as people stay unemployed over time their levels of life satisfaction remain low . several studies have shown that even once a person becomes re-employed, the prior experience of unemployment leaves a mark on his or her happiness.

So there is no sudden or even medium-term realisation that being jobless endows the individual with a new sense of freedom to become their creative selves, freed from the yoke of work. To bloom into musicians, artists, or whatever.

The reality is that there is an on-going malaise – a deeply entrenched sense of failure is overwhelming, which stifles happiness and creativity, even after the individual is able to return to work.

This negativity, borne heavily by the individual, however, also impacts on society in general.

The paper recognises that:

A further canonical finding in the literature on unemployment and subjective wellbeing is that there are so-called "spillover" effects.

High levels of unemployment "increase fear and heighten the sense of job insecurity". Who will lose their job next type questions?

The researchers found in their data that the higher is the unemployment rate the greater the anxiety among those who remain employed.

Conclusion

The overwhelming conclusion is that "work makes up such an important part of our lives" and that result is robust across different countries and cultures.

Being employed leads to much higher evaluations of the quality of life relative to being unemployed.

The unemployed are miserable and remain so even as they become entrenched in long-term unemployment. They do not seem to sense (or exploit) a freedom to release some inner sense of creativity and purpose.

The overwhelming proportion continually seek work – and relate their social status and life happiness to gaining a job, rather than living without a job on income support.

Modern Monetary Theory (MMT) allows us to understand that it is the government that chooses the unemployment rate – it is a political choice.

For currency-issuing governments it means their deficits are too low relative to the spending and saving decisions of the non-government sector.

For Eurozone-type nations, it means that in surrendering their currencies and adopting a foreign currency, they are unable to guarantee sufficient work in the face of negative shifts in non-government spending. Again, a political choice.

The Job Guarantee can be used as a vehicle to not only ensure their are sufficient jobs available at all times but also to start a process of wiping out the worst jobs in the non-government sector.

That can be done by using the JG wage to ensure low-paid private employers have to restructure their workplaces and pay higher wages and achieve higher productivity in order to attract labour from the Job Guarantee pool.

The Series So Far

This is a further part of a series I am writing as background to my next book with Joan Muysken analysing the Future of Work . More instalments will come as the research process unfolds.

The series so far:

  1. When Austrians ate dogs .
  2. Employment as a human right .
  3. The rise of the "private government .
  4. The evolution of full employment legislation in the US .
  5. Automation and full employment – back to the 1960s .
  6. Countering the march of the robots narrative .
  7. Unemployment is miserable and does not spawn an upsurge in personal creativity .

The blogs in these series should be considered working notes rather than self-contained topics. Ultimately, they will be edited into the final manuscript of my next book due in 2018. The book will likely be published by Edward Elgar (UK).

That is enough for today!

divadab , November 21, 2017 at 6:11 am

Perhaps I'm utterly depressed but I haven't had a job job for over 5 years. Plenty of work, however, more than I can handle and it requires priorisation. But I am deliberately not part of the organized herd. I stay away from big cities – it's scary how managed the herd is in large groups – and I suppose that unemployment for a herd animal is rather distressing as it is effectively being kicked out of the herd.

Anyway my advice, worth what you pay for it but let he who has ears, etc. – is to go local, very local, grow your own food, be part of a community, manage your own work, and renounce the energy feast herd dynamics. "Unemployment", like "recession", is a mechanism of control. Not very practical advice for most, I realize, trapped in the herd as they are in car payments and mortgages, but perhaps aspirational?

The Rev Kev , November 21, 2017 at 6:35 am

I think what is missing from this article is the term "identity." If you meet new people, often the conversation starts with what you do for a living. Your identity, in part, is what you do. You can call yourself a plumber, a writer, a banker, a consultant, a reporter but the point is this is part of your identity. When you lose your job long term, your identity here loses one of its main anchor points.

Worse, there is a deliberate stigma attached with being long term unemployed. In that article you have seen the word bludger being used. In parts of the US I have read of the shame of 'living off the county'. And yes, I have been there, seen that, and got the t-shirt. It's going to be interesting as mechanization and computers turn large portions of the population from workers to 'gig' workers. Expect mass demoralization.

nonclassical , November 21, 2017 at 10:24 am

yes the lives many of us have lived, no longer exist though we appear not notice, as we "can" live in many of same "ways" ..rather well known psychologist defined some 40 years ago, best to "drop through cracks"

jrs , November 21, 2017 at 12:13 pm

Well, you also lose money, maybe you become homeless etc. as you have nowhere else to turn (if there are kids involved to support it gets even scarier though there are some programs). Or maybe you become dependent on another person(s) to support you which is of course degrading as you know you must rely on them to live, whether it's a spouse or lover when you want to work and bring in money, or mom and dads basement, or the kindest friend ever who lets you sleep on their couch. I mean these are the things that really matter.

Privileged people whose main worry in unemployment would be losing identity, wow out of touch much? Who cares about some identity for parties, but the ability to have a stable decent life (gig work hardly counts) is what is needed.

sgt_doom , November 21, 2017 at 2:20 pm

I believe your comment sums up the situation the best -- and most realistically.

jgordon , November 21, 2017 at 7:08 pm

I normally wouldn't comment like this, but you have brought up some extremely important points about identity that I would like to address.

Recently I had the most intense mushroom experience of my entire life–so intense that my identity had been completely stripped and I was left in a formless state, at the level of seeing my bare, unvarnished animal neural circuitry in operation. Suddenly with a flash of inspiration I realized that the identity of everyone, all of us, is inextricably tied up in what we do and what we do for other people.

Following from that, I understood that if we passively rely on others for survival, whether it be relying on friends, family, or government, then we do not have an identity or reason for existing. And the inner self, the animal core of who we are, will realise this lack of identity (even if the concious mind denies it), and will continually generate feelings of profound depression and intense nihilism that will inevitably destroy us if the root cause is not addressed.

Before this experience I was somewhat ambivalent about my politics, but immediately after I knew that the political right was correct on everything important, from attitudes on sex to economic philosophy. People need a core of cultural stability and hard work to grow and become actualized. The alternative is rudderless dissatisfaction and envy that leads nowhere.

On the topic of giving "out of kindnes and goodwill", giving without demanding anything in return is a form of abuse, as it deprives those who receive our feel-good generosity the motivation to form a coherent identity. If the parents of a basement-dweller were truly good people, instead of supporting said dweller they'd drag her out by the ear and make her grow food in the yard or some such. Likewise, those who have supported you without also giving concrete demands and expecations in return have been unkind, and for your own good I hope that you will immediately remove yourself from their support. On the other hand, if you have been thoughtlessly giving because it warms the cockles of your heart, then stop it now. You are ruining other people this way, and if your voting habits are informed by this kind of malevolence I'd encourage you to change those as well.

Anyway the original poster is right about everything. Working and having a purpose in life is an entirely different animal from making money and being "successful" in the government-sponsored commercial economy. Society and government deliberately try to conflate the two for various reasons, primarily graft of labor and genius, but that is only a deliberate mis-framing that needlessly harms people when the mainstream economic system is in catastrophic decline, as ours is today. You should try to clear up this misconception within yourself as a way of getting better.

Well, I hope this message can give you a few different thoughts and help you find your way out of the existential angst you're caught in. Don't wallow in helplessness. Think of something useful to do, anything, whether it earns you money or not, and go out and start doing it. You'll be surprised at how much better you feel about yourself in no time.

skippy , November 22, 2017 at 12:45 am

The problem is you said – I – had an extreme experience [burning bush], the truth was reviled to – I – and I alone during this extreme chemically altered state. Which by the way just happens to conform to a heap of environmental biases I collected. This is why sound methodology demands peer review. disheveled some people think Mister Toads Wild ride at Disneyland on psychotropics is an excellent adventure too.

Jeremy Grimm , November 21, 2017 at 12:33 pm

I think your observation about the importance of work to identity is most perceptive. This post makes too little distinction between work and a job and glosses over the place of work in defining who we are to ourselves and to others. I recall the scene in the movie "About a Boy" when the hero meets someone he cares about and she asks him what he does for a living.

I believe there's another aspect of work -- related to identity -- missing in the analysis of this post. Work can offer a sense of mission -- of acting as part of an effort toward a larger goal no individual could achieve alone. However you may regard the value in putting man on the moon there is no mistaking the sense of mission deeply felt by the engineers and technicians working on the project. What jobs today can claim service to a mission someone might value?

Henry Moon Pie , November 21, 2017 at 7:00 am

Agreed on your points. Wage slavery is nothing to aspire to. Self-determination within a context of an interdependent community is a much better way to live. We do our thing in the city, however.

ambrit , November 21, 2017 at 8:29 am

Finding that "interdependent community" is the hard part. My experience has been that this endeavour is almost chance based; Serendipity if you will.
Here Down South, the churches still seem to have a stranglehold on small and mid scale social organization. One of the big effects of 'churching' is the requirement that the individual gave up personal critical thinking. Thus, the status quo is reinforced. One big happy 'Holy Circlejerk.'

UserFriendly , November 21, 2017 at 10:10 am

from the article

This is a crucial point that UBI advocates often ignore. There is a deeply entrenched cultural bias towards associating our work status with our general status and prestige and feelings of these standings.

That hasn't changed since Eisenberg and Lazersfeld wrote up the findings of their study in 1938. It might change over time but that will take a long process of re-education and cultural shift. Trying to dump a set of new cultural values that only a small minority might currently hold to onto a society that clearly still values work is only going to create major social tensions.

FelicityT , November 21, 2017 at 3:07 pm

I would agree about the entenched cultural norms, etc. But not the pessimism and timeline for change. An individual can communicate a complex idea to millions in seconds, things move fast these days.

For me, it seems that what we (we being UBI/radical change proponents) are lacking is a compelling easily accessible story. Not just regarding UBI (as that is but one part of the trully revolutionary transformations that must occur) but encompassing everything.

We have countless think pieces, bits of academic writing, books, etc that focus on individual pieces and changes in isolation. But we've largely abandoned the all-encompassing narrative, which at their heart is precisely what religion offers and why it can be so seductive, successful, and resilient for so long.

The status quo has this type of story, it's not all that compelling but given the fact that it is the status quo and has inertia and tradition on its side (along with the news media, political, entertainment, etc) it doesn't have to be.

We need to abandon the single narrow issue activism that has become so prominent over the years and get back to engaging with issues as unseparable and intimately interconnected.

Tinkering around the edges will do nothing, a new political religion is what is required.

Yves Smith Post author , November 21, 2017 at 4:23 pm

Sorry, I disagree vehemently. Deeply held cultural attitudes are very slow to change and the study found that work being critical to happiness examined a large number of societies.

Look at feminism. I was a half-generation after the time when women were starting to get a shot at real jobs. IIRC, the first class that accepted women at Harvard Law School was in the 1950 and at Harvard Business School, 1965. And the number of first attendees was puny. The 1965 class at HBS had 10 8 women out of a graduating class of over 800; my class in 1981 had only 11% women.

In the 1980s, you saw a shift from the belief that women could do what men could do to promotion of the idea that women could/should be feminine as well as successful. This looked like seriously mixed messages, in that IMHO the earlier tendency to de-emphasize gender roles in the workplace looked like a positive development.

Women make less than 80% of what men do in the US. Even female doctors in the same specialities make 80% of their male peers.

The Speenhamland in the UK had what amounted to an income guarantee from the 1790s to 1832. Most people didn't want to be on it and preferred to work. Two generations and being on the support of local governments was still seen as carrying a stigma.

More generally, social animals have strongly ingrained tendencies to resent situations they see as unfair. Having someone who is capable of working not work elicits resentment from many, which is why most people don't want to be in that position. You aren't going to change that.

And people need a sense of purpose. There are tons of cases of rich heirs falling into drug addiction or alcoholism and despair because they have no sense of purpose in life. Work provides that, even if it's mundane work to support a family. That is one of the great dissservices the Democrats have done to the citizenry at large: sneering at ordinary work when blue-collar men were the anchors of families and able to take pride in that.

FelicityT , November 21, 2017 at 5:11 pm

So a few points.

Regarding the large number of societies, we often like to think we're more different than we actually are focusing on a few glaringly obvious differences and generalizing from there. Even going back a few hundred years when ideas travelled slower we were still (especially the "west" though the "east" wasn't all that much more different either) quite similar. So I'm less inclined to see the large number of societies as evidence.

Generally on societal changes and movements: The issue here is that the leadership has not changed, they may soften some edges here or there (only to resharpen them again when we're looking elsewhere) but their underlying ideologies are largely unchanged. A good mass of any population will go along to survive, whether they agree or not (and we find increasing evidence that many do not agree, though certainly that they do not agree on a single alternative).

It may be impossible to implement such changes in who controls the levers of power in a democratic fashion but it also may be immoral not implement such changes. Of course this is also clearly a similar path to that walked by many a demonized (in most cases rightfully so) dictator and despot. 'Tread carefully' are wise words to keep in mind.

Today we have a situation which reflects your example re: social animals and resentment of unfairness: the elite (who falls into this category is of course debatable, some individuals moreso than others). But they have intelligently, for their benefit, redirected that resentment towards those that have little. Is there really any logical connection between not engaging in wage labor (note: NOT equivalent to not working) and unfairness? Or is it a myth crafted by those who currently benefit the most?

That resentment is also precisely why it is key that a Basic income be universal with no means testing, everyone gets the same.

I think we should not extrapolate too much from the relatively small segment of the population falling into the the inherited money category. Correlation is not causation and all that.

It also seems that so often individuals jump to the hollywood crafted image of the layabout stoner sitting on the couch giggling at cartoons (or something similarly negative) when the concept of less wage labor is brought up. A reduction of wage labor does not equate to lack of work being done, it simply means doing much of that work for different reasons and rewards and incentives.

As I said in the Links thread today, we produce too much, we consume too much, we grow too much. More wage labor overall as a requirement for survival is certainly not the solution to any real problem that we face, its a massively inefficient use of resources and a massive strain on the ecosystems.

Yves Smith Post author , November 21, 2017 at 8:34 pm

I am really gobsmacked at the sense of entitlement on display here. Why are people entitled to an income with no work? Being an adult means toil: cleaning up after yourself, cleaning up after your kids if you have them, if you are subsistence farmer, tending your crops and livestock, if you are a modern society denizen, paying your bills and your taxes on time. The idea that people are entitled to a life of leisure is bollocks. Yet you promote that.

Society means we have obligations to each other. That means work. In rejecting work you reject society.

And the touting of "creativity" is a top 10% trope that Thomas Frank called out in Listen, Liberal. It's a way of devaluing what the bottom 90% do.

WobblyTelomeres , November 21, 2017 at 8:53 pm

My argument with the article is that, to me, it smacks of Taylorism. A follow-on study would analyze how many hours a laborer must work before the acquired sense of purpose and dignity and associated happiness began to decline. Would it be 30 hours a week of backbreaking labor before dignity found itself eroded? 40? 50? 60? When does the worker break? Just how far can we push the mule before it collapses?

The author alludes to this: "The overwhelming proportion relate their social status and life happiness to gaining a job"

Work equals happiness. Got it.

But, as a former robotics instructor, and as one who watches the industry (and former students), I see an automated future as damn near inevitable. Massive job displacement is coming, life as a minimum wage burger flipper will cease, with no future employment prospects short of government intervention (WPA and CCC for all, I say). I'm not a Luddite, obviously, but there are going to be a lot of people, billions, worldwide, with no prospect of employment. Saying, "You're lazy and entitled" is a bit presumptuous, Yves. Not everyone has your ability, not everyone has my ability. When the burger flipping jobs are gone, where do they go? When roombas mop the floors, where do the floor moppers go?

flora , November 21, 2017 at 9:38 pm

"WPA and CCC for all, I say. "

+1

We could use a new Civilian Conservation Corps and and a Works Progress Administration. There's lots of work that needs doing that isn't getting done by private corporations.

nihil obstet , November 21, 2017 at 10:05 pm

The outrage at non-work wealth and income would be more convincing if it were aimed also at owners of capital. About 30% of national income is passive -- interest, rents, dividends. Why are the owners of capital "entitled to an income with no work?" It's all about the morality that underlies the returns to capital while sugaring over a devaluation of labor. As a moral issue, everyone should share the returns on capital or we should tax away the interest, rents, and dividends. If it's an economic issue, berating people for their beliefs isn't a reason.

WobblyTelomeres , November 21, 2017 at 10:14 pm

Why are the owners of capital "entitled to an income with no work?"

THIS!!!! So much, THIS!!!! But, what else is a Wobbly to say, eh?

Yves Smith Post author , November 22, 2017 at 2:27 am

The overwhelming majority do work. The top 0.1% is almost entirely private equity managers who are able to classify labor income as capital gains through the carried interest loophole. Go look at the Forbes 400.

The 1% are mainly CEOs, plus elite professionals, like partners at top law and consulting firms and specialty surgeons (heart, brain, oncology). The CEOs similarly should be seen as getting labor income but have a lot of stock incentive pay (that is how they get seriously rich) which again gets capital gains treatment.

You are mistaking clever taking advantage of the tax code for where the income actually comes from. Even the kids of rich people are under pressure to act like entrepreneurs from their families and peers. Look at Paris Hilton and Ivanka as examples. They both could have sat back and enjoyed their inheritance, but both went and launched businesses. I'm not saying the kids of the rich succeed, or would have succeed to the extent they do without parental string-pulling, but the point is very few hand their fortune over to a money manager and go sailing or play the cello.

IsotopeC14 , November 22, 2017 at 2:58 am

Isn't the brother of the infamous Koch duo doing exactly that? Actually, if all the .001%ers were like him, we'd all be better off

IsotopeC14 , November 22, 2017 at 1:34 am

What's your take on Rutger Bergman's ted talk? i think most jobs aren't real jobs at all, like marketing and ceo's. why can't we do 20 hour work weeks so we don't have huge amounts of unemployment? Note, I was "unemployed" for years since "markets" decide not to fund science in the US. Yay Germany At least I was fortunate enough to not be forced to work at Walmart or McDonalds like the majority of people with absolutely no life choices. Ah the sweet coercion of capitalism.

flora , November 21, 2017 at 9:09 pm

Your hopes for a UBI are undone by some of the real world observations I've made over many years, with regard to how a guaranteed income increase, of any measure, for a whole population of an area, affects prices. Shorter: income going up means prices are raised by merchants to capture the new income.

Your assumption that any UBI would not be instantly captured by raised prices is naive, at best. It's also naive to assume companies would continue to pay wages at the same level to people still employed, instead of reducing wages and letting UBI fill in the rest. Some corporations already underpay their workers, then encourage the workers to apply for food stamps and other public supports to make up for the reduced wage.

The point of the paper is the importance of paid employment to a person's sense of well being. I agree with the paper.

Andrew Dodds , November 22, 2017 at 2:48 am

For the vast majority, a UBI would be income-neutral – it would have to be, to avoid massive inflation. So people would receive a UBI, but pay more tax to compensate. The effect on prices would be zero.

The advantage of a UBI is mostly felt at the lower end, where insecure/seasonal work does now pay. At the moment, a person who went from farm labourer to Christmas work to summer resort work in the UK would certainly be working hard, but also relentlessly hounded by the DWP over universal credit. A UBI would make this sort of lifestyle possible.

jsn , November 21, 2017 at 11:28 am

Davidab, Good for you, but your perspicacity is not scalable. People are social animals and your attitude toward "the herd", at least as expressed here, is that of a predator, even if your taste doesn't run toward predation. Social solutions will necessarily be scalable or they won't be solutions for long.

Lambert Strether , November 22, 2017 at 1:44 am

> the organized herd a herd animal trapped in the herd

I don't think throwing 80% to 90% of the population into the "prey" bucket is especially perspicacious politically (except, of course, for predators or parasites). I also don't think it's especially perspicacious morally. You write:

Not very practical advice for most, I realize, trapped in the herd as they are in car payments and mortgages, but perhaps aspirational?

Let me translate that: "Trapped in the herd as many are to support spouses and children." In other words, taking the cares of the world on themselves in order to care for others.

BJ , November 21, 2017 at 6:37 am

Unemployed stay at home dad here. My children are now old enough to no longer need a stay at home dad. Things I have done: picked up two musical instruments and last year dug a natural swimming pond by hand. Further, one would need to refute all the increased happiness in retirement (NBER). Why social security but not UBI? I get being part of the precariat is painful and this is a reality for most the unemployed no matter where you live in the world. A UBI is unworkable because it will never be large enough to make people's lives unprecarious. Having said that, I am almost positive if you gave every unemployed person 24 k a year and health benefits, there would be a mass of non working happy creative folks.

divadab , November 21, 2017 at 7:41 am

UBI seems to me to encourage non-virtuous behavior – sloth, irresponsibility, fecklessness, and spendthriftness. I like the Finnish model – unemployment insurance is not limited – except if you refuse work provided by the local job center. Lots of work is not being done all over America – we could guarantee honest work to all with some imagination. Start with not spraying roundup and rather using human labor to control weeds and invasive species.

I do agree that universal health insurance is necessary and sadly Obamacare is not that.

ambrit , November 21, 2017 at 8:34 am

The crux of this problem is the definition used for "non-virtuous behaviour." A new CCC is a good place to start though. (Your Tax Dollars At Work! [For some definition of tax dollars.]) As for BJ above, I would suppose that child rearing was his "employment" for years. good so far, but his follow-up is untypical. The 'Empty Nester' mother is a well known meme.

a different chris , November 21, 2017 at 9:19 am

Spendthriftness on 24K a year? Seriously? If we are disgorging unprofessional opinions, I will add my own: sloth and irresponsibility are more signs of depression rather than freedom from having to work. In fact, I believe (and I think much of the stuff here) supports the idea that people want to be seen as useful in some way. Doesn't include me! :) .. unfortunately, I have the charmingly named "dependents" so there you have it.

BJ , November 21, 2017 at 11:18 am

I lived 6 years as a grad student on 24k a year and would say it was easy. Only thing I would have to had worried about was awful health insurance. A two household each with 24k would be even easier, especially if you could do it in a low cost area. So I am not sure what you mean by spendthrift. But again it will never happen, so we will be stuck with what we have or most likely an even more sinister system. I guess I am advocating for a JG with unlimited number of home makers per household.

roadrider , November 21, 2017 at 9:23 am

except if you refuse work provided by the local job center

And who's to say that the local "job center" has work that would be appropriate for every person's specific talents and interests? This is no better than saying that you should be willing to go work for some minimum-wage retail job with unpredictable scheduling and other forms of employer abuses after you lose a high-paying job requiring special talents. I have to call bullshit on this model. I went through a two-year stretch if unemployment in no small part because the vast majority of the available jobs for my skill set were associated with the MIC, surveillance state or the parasitic FIRE sector. I was able to do this because I had saved up enough FY money and had no debts or family to support.

I can also attest to the negative aspects of unemployment that the post describes. Its all true and I can't really say that I'e recovered even now, 2.5 years after finding another suitable job.

Jesper , November 21, 2017 at 10:55 am

The job center in the neighbouring Sweden had the same function. Had is the important word. My guess is that the last time someone lost their unemployment insurance payout due to not accepting a job was in the early 1980s. Prior to that companies might, maybe, possibly have considered hiring someone assigned to them – full employment forced companies to accept what was offered. Companies did not like the situation and the situation has since changed.

Now, when full employment is a thing of the past, the way to lose unemployment insurance payouts is by not applying to enough jobs. An easily gamed system by people not wanting to work: just apply to completely unsuitable positions and the number of applications will be high. Many companies are therefore overwhelmed by applications and are therefore often forced to hire more people in HR to filter out the unsuitable candidates.
People in HR tend not to know much about qualifications and or personalities for the job so they tend to filter out too many. We're all familiar with the skills-shortage .
Next step of this is that the companies who do want to hire have to use recruitment agencies. Basically outsourcing the HR to another company whose people are working on commission. Recruiters sometimes know how to find 'talent', often they are the same kind of people with the same skills and backgrounds as people working in HR.

To even get to the hiring manager a candidate has to go through two almost identical and often meaningless interviews. Recruiter and then HR. Good for the GDP I suppose, not sure if it is good for anything else.

But back on topic again, there is a second way of losing unemployment insurance payout: Time. Once the period covered has passed there is no more payouts of insurance. After that it it is time to live on savings, then sell all assets, and then once that is done finally go to the welfare office and prove that savings are gone and all assets are sold and maybe welfare might be paid out. People on welfare in Sweden are poor and the indignities they are being put through are many. Forget about hobbies and forget about volunteering as the money for either of those activities simply aren't available. Am I surprised by a report saying unemployed in Sweden are unhappy? Nope.

nonclassical , November 21, 2017 at 10:42 am

meanwhile NYTimes testimonials Friday, show average family of 4 healthprofit costs (tripled, due to trump demise ACA) to be $30,000. per year, with around $10,000. deductible end of any semblance of affordable access, "murKa"

https://www.nytimes.com/2017/11/16/us/politics/obamacare-premiums-middle-class.html

Jeremy Grimm , November 21, 2017 at 1:53 pm

What do you mean by virtuous behavior?

Where does a character like Bertie Wooster in "Jeeves" fit in your notions of virtuous behavior? Would you consider him more virtuous working in the management of a firm, controlling the lives and labor of others -- and humorously helped by his his brilliant valet, Jeeves, getting him out of trouble?

For contrast -- in class and social status -- take a beer-soaked trailer trash gentleman of leisure -- and for sake of argument blessed with less than average intelligence -- where would you put him to work where you'd feel pleased with his product or his service? Would you feel better about this fellow enjoying a six-pack after working 8 hours a day 5 days a week virtuously digging and then filling a hole in the ground while carefully watched and goaded by an overseer? [Actually -- how different is that from "using human labor to control weeds and invasive species"? I take it you're a fan of chain-gangs and making the poor pick up trash on the highways?]

What about some of our engineers and scientists virtuously serving the MIC? Is their behavior virtuous because they're not guilty of sloth, irresponsibility [in executing their work], fecklessness, and spendthriftness? On this last quality how do you feel about our government who pay the salaries for all these jobs building better ways to kill and maim?

Bill Smith , November 21, 2017 at 8:01 am

How big is the swimming pool and how long did it take? Where did you put the dirt?

BJ , November 21, 2017 at 11:07 am

It is a design by David Pagan Butler. It is his plunge pool design, deepend is 14 by 8 by 7 deep. I used the dirt to make swales around some trees. Win win all around.

tegnost , November 21, 2017 at 9:32 am

curious to know whether you are married to someone with a job?

BJ , November 21, 2017 at 11:25 am

The answer is yes my spouse works. So I do have a schedule of waking up to make her lunch everyday, meeting her at lunch to walk, and making dinner when she gets home, but we do all those things on her days off so .

But again we would need to explain away, why people who are retired are happier? Just because they think they payed into social security? Try explaining to someone on the SS dole how the government spends money into existence and is not paid by taxes or that the government never saved their tax money, so there are not entitled to this money.

David Kane Miller , November 21, 2017 at 6:55 am

I hated working for other people and doing what they wanted. I began to feel some happiness when I had a half acre on which I could create my own projects. Things improved even more when I could assure myself of some small guaranteed income by claiming Social Security at age 62. To arise in the morning when I feel rested, with interesting projects like gardens, fences, small buildings ahead and work at my own pace is the essence of delight for me. I've been following your arguments against UBI for years and disagree vehemently.

a different chris , November 21, 2017 at 9:23 am

I feel I would behave the same as you, if I had the chance. *But* no statements about human beings are absolute, and because UBI would work for either of us does not mean it would work for the majority. Nothing devised by man is perfect.

Mel , November 21, 2017 at 9:42 am

It's not you; it's not me. It's those deplorable people.

tegnost , November 21, 2017 at 9:37 am

first you had to buy the half acre in a suitable location, then you had to work many years to qualify for social security, the availability of which you paid for and feel you deserve. You also have to buy stuff for fences gardens and small buildings. At most that rhymes with a ubi but is significantly different in it's make up.

Lambert Strether , November 22, 2017 at 1:56 am

> when I had a half acre on which I could create my own projects

That is, when you acquired the half acre, which not everyone can do. It seems to me there's a good deal of projecting going on with this thread from people who are, in essence, statistical outliers. But Mitchell summarizes the literature:

So for decades, researchers in this area, as opposed to bloggers who wax lyrical on their own opinions, have known that the importance of work in our lives goes well beyond the income we earn.

If the solution that works for you is going to scale, that implies that millions more will have to own land. If UBI depends on that, how does that happen? (Of course, in a post-collapse scenario, the land might be taken , but that same scenario makes the existence of institutions required to convey the UBI highly unlikely. )

Carla , November 21, 2017 at 7:16 am

Very glad to hear that Bill Mitchell is working on the "Future of Work" book, and to have this post, and the links to the other segments. Thank you, Yves!

Andrew , November 21, 2017 at 7:25 am

I don't agree with this statement. Never will. I'm the complete opposite. Give me more leisure time and you'll find me painting, writing, playing instruments and doing things that I enjoy. I recall back to when I was a student, I relished in the free time I got (believe me University gave me a lot of free time) between lectures, meaning I could enjoy this time pursuing creative activities. Sure I might be different than most people but I know countless people who are the same.

My own opinion is that root problem lies in the pathology of the working mentality, that 'work' and having a 'job' is so engrained into our society and mindset that once you give most people the time to enjoy other things, they simply can't. They don't know what to do with themselves and they eventually become unhappy, watching daytime TV sat on the sofa.

I recall back to a conversation with my mother about my father, she said to me, 'I don't know how your father is going to cope once he retires and has nothing to do' and it's that very example of where work for so many people becomes so engrained in their mindset, that they are almost scared of having 'nothing to do' as they say. It's a shame, it's this systemic working mentality that has led to this mindset. I'm glad I'm the opposite of this and proud by mother brought me up to be this way. Work, and job are not in my vocabulary. I work to live, not live to work.

I_Agree , November 21, 2017 at 11:26 am

I agree with Andrew. I think this data on the negative effects says more about how being employed fundamentally breaks the human psyche and turns them into chattel, incapable of thinking for themselves and destroying their natural creativity. The more a human is molded into a "good worker" the less they become a full fledged human being. The happiest people are those that have never placed importance on work, that have always lived by the maxim "work to live, not live to work". From my own experience every assertion in this article is the opposite of reality. It is working that makes me apathethic, uncreative, and miserable. The constant knowing that you're wasting your life, day after day, engaged in an activity merely to build revenue streams for the rich, instead of doing things that help society or that please you on a personal level, is what I find misery inducing.

nycTerrierist , November 21, 2017 at 12:18 pm

I agree. If financial insecurity is removed from the equation -- free time can be used creatively for self-actualization, whatever form that may take: cultivating the arts, hobbies, community activities, worthy causes and projects. The ideology wafting from Mitchell's post smells to me like a rationale for wage slavery (market driven living, neo-liberalism, etc.)

jrs , November 21, 2017 at 12:48 pm

Besides how are people supposed to spend their time "exploring other opportunities" when unemployed anyway? To collect unemployment which isn't exactly paying that much anyway, they have to show they are applying to jobs. To go to the movies the example given costs money, which one may tend to be short on when unemployed. They probably are looking for work regardless (for the income). There may still be some free time. But they could go back to school? Uh in case one just woke up from a rock they were under for 100 years, that costs money, which one may tend to be short on when unemployed, plus there is no guarantee the new career will pan out either, no guarantee someone is just chomping at the bit to hire a newly trained 50 year old or something. I have always taken classes when unemployed, and paid for it and it's not cheap.

Yes to use one's time wisely in unemployment in the existing system requires a kind of deep psychological maturity that few have, a kind of Surrender To Fate, to the uncertainty of whether one will have an income again or not (either that or a sugar daddy or a trust fund). Because it's not easy to deal with that uncertainty. And uncertainty is the name of the game in unemployment, that and not having an income may be the pain in it's entirety.

FelicityT , November 21, 2017 at 3:18 pm

Sadly this breaking down into a "good worker" begins for most shortly after they begin school. This type of education harms society in a myriad of ways including instilling a dislike of learning, deference to authority (no matter how irrational and unjust), and a destruction of a child's natural curiosity.

Yves Smith Post author , November 21, 2017 at 5:21 pm

I don't buy your premise that people are "creative". The overwhelming majority do not have creative projects they'd be pursuing if they had leisure and income. Go look at retirees, ones that have just retired, are healthy, and have money.

Yves Smith Post author , November 21, 2017 at 4:29 pm

You are really misconstruing what the studies have found and misapplied it to your situation. Leisure time when you have a job or a role (being a student) is not at all the same as having time when you are unemployed, with or without a social safety net.

Summer , November 21, 2017 at 6:25 pm
jrs , November 21, 2017 at 6:37 pm

one often has a role when unemployed: finding work. But it's not a very fulfilling one! But if one is trying to find work, it's not exactly the absence of a role either even if it still leaves significantly more free time than otherwise, maybe winning the lottery is the absence of a role.

But then it's also not like we give people a UBI even for a few years (at any time in adult life) to get an education. Only if they take out a student loan approaching the size of a mortgage or have parents willing to pony up are they allowed that (to pay not just for the education but to live because having a roof over one's head etc. is never free, a UBI via debt it might be called).

Lambert Strether , November 22, 2017 at 2:00 am

> Give me more leisure time and you'll find me painting, writing, playing instruments and doing things that I enjoy.
Nothing to breed resentment of "the creative class" here! Blowback from Speenhamland brought on the workhouses, so be careful what you wish for.

Jesper , November 21, 2017 at 7:47 am

Again the UBI vs JG debate .

UBI won't happen and JG has been tried (and failed).

The argument that JG would allow the public sector to hire more people is demeaning to people already employed in the public sector and demonstrably false – people are hired into the public sector without there being a JG. It is most certainly possible to be against a JG while wanting more people working in the public sector.

The way forward is to have a government acting for people instead of for corporations. Increase the amount of paid vacations, reduce the pension age and stop with the Soviet style worship of work: While some people are apparently proud of their friends and relatives who died while at work it is also possible to feel sad about that.

diptherio , November 21, 2017 at 10:00 am

JG has been tried (and failed).

When and where? The NCCC seemed to work pretty good here in the Western US.

Jesper , November 21, 2017 at 10:27 am

The JG was tried in Communist countries in Europe, Asia and Americas. The arguments then and there were the same as here and now, made by the same type of social 'scientists' (economists).

Would a JG be different here and now as the Republicans and Democrats are representing the best interests of the people? Or are they representing the same kind of interests as the Communist parties did?

Yves Smith Post author , November 21, 2017 at 4:39 pm

Data, please. The USSR fell because it was spending on its military to keep up with the US, a much larger economy. Countering your assertion we have this:

tegnost , November 21, 2017 at 10:00 am

As long as people argue that "it's not fair" to fix the inequality issue and employ things like debt jubilee or student loan forgiveness, or if we fix the ridiculous cost of health care what will all those insurance agents do then we will wind up with the real kind of class warfare, rather than the current punching from the top down, the punching will come from the bottom, because the situation is not fair now, it's just TINA according to those who profit from it. In my own life there is a balance of creativity and work, and I find work enables my creativity by putting some pressure on my time, i.e., I get up earlier, I practice at 8:30 am instead of sleeping til 10 and winding up with S.A..D., I go to bed rather than watch tv or drink to excess.. in other words i have some kind of weird schedule, I have days off sort of When I've been unemployed I feel the way s described in the article. I find the arguments in favor of ubi tend to come from people who already have assets, or jobs, or family who they take care of which is actually a job although uncommonly described as such. The only truth I see in real life is that the unemployed I am intimately familiar with first are mentally oppressed by the notion that to repair their situation will require they work every waking hour at substandard wages for the rest of their life and that is a major barrier to getting started, and that is a policy choice the gov't and elite classes purposefully made which created the precariat and will be their undoing if they are unable to see this.

tegnost , November 21, 2017 at 10:15 am

Hey look, even the msm is looking at it
https://www.seattletimes.com/seattle-news/is-uprising-the-only-way-out-of-gross-inequality-maybe-so/

j84ustin , November 21, 2017 at 10:08 am

As someone who works in the public sector I never quite thought of it like that, thanks.

hunkerdown , November 21, 2017 at 7:53 am

Disappointing that there's no analysis in this context of less employment, as in shorter work weeks and/or days, as opposed to merely all or none.

nonclassical , November 21, 2017 at 10:45 am

see – hear

(but no possibility without healthcare access, rather than healthtprofit)

Vatch , November 21, 2017 at 11:31 am

Interesting point. I read a science fiction story in which the protagonist arrives for work at his full time job at 10:00 AM, and he's finished for the day at 4:00 PM. I can't remember the name of the story or novel, unfortunately.

jrs , November 21, 2017 at 1:04 pm

Agreed. And they already have it in places like Denmark. Why don't we talk about that? It actually exists unlike utopian schemes for either total UBI or total work guarantee (government job creation is not utopian, but imagining it will employ everyone is, and I would like the UBI to be more widely tried, but in this country we are nowhere close). Funny how utopia becomes more interesting to people than actual existing arrangements, even though of course those could be improved on too.

The Danish work arrangement is less than a 40 hour week, and mothers especially often work part-time but both sexes can. It's here in this country where work is either impossibly grueling or you are not working. No other choice. In countries with more flexible work arrangements more women actually work, but it's flexible and flexible for men who choose to do the parenting as well. I'm not saying this should be for parents only of course.

Lambert Strether , November 22, 2017 at 2:02 am

Because the JG sets the baseline for employment, which private companies must meet, the JG (unlike the UBI) can do this.

Otis B Driftwood , November 21, 2017 at 7:58 am

My own situation is that I am unhappy in my well-paying job and would like nothing more than to devote myself to other interests. I'm thirty years on in a relationship with someone who grew up in bad financial circumstances and panics whenever I talk about leaving my job. I tell her that we have 2 years of living expenses in the bank but I can't guarantee making the same amount of money if I do leave my job. She has a job that she loves and is important and pays barely 1/2 of my own income. So she worries about her future with me. She worries about losing her home. I suppose that makes me the definition of a wage slave. And it makes for an increasingly unhappy marriage. I admire those who have faced similar circumstances and found a way through this. Sorry to vent, but this topic and the comments hit a nerve with me and I'm still trying to figure this out.

ambrit , November 21, 2017 at 8:38 am

Otis; We are presently going through a period where that "two year cushion" has evaporated, for various reasons. We are seeing our way through this, straight into penury and privation. Take nothing for granted in todays' economy.

jrs , November 21, 2017 at 1:11 pm

yes find the lower paying job that you like more first. If you just quit for nothing in the hopes of finding one it might not happen. Of course unemployment also happens sometimes, whether we want it or not.

bronco , November 21, 2017 at 12:47 pm

The newer generations are worse when it comes to lifestyle. Those of that are older can at least remember a time without cellphones internet streaming services leasing a new car every 2 years etc.

What about the young? My niece and her husband should be all set , his mom sunk money into a home on the condition she moved into a mother in law apartment. So far so good right? 2 years in they are imploding even with the free child care she provides. Combined their wireless bill a month is over $300. The sit on the couch side by side and stream netflix shows to dueling iphones in front of a 65 inch tv that is not even turned on. Wearing headphones in silence.

Both driving new vehicles , both have gym memberships they don't use . They buy lattes 3 or 4 times a day which is probably another 500 a month.

My uncle passed away recently and my niece asked if she was in the will. It was literally her only communication on the subject. They are going under and could easily trim a few thousand a month from the budget but simply won't. No one in the family is going to lift a finger for them at this point they burned every possible bridge already. I have seen people living in cars plenty lately but I think these will be the first I see to living in brand new cars .

Somewhere along the line they got the impression that the american dream was a leased car a starbucks in one hand and an iphone in the other .

Confront them with the concept of living within a paycheck and they react like a patient hearing he has 3 months to live.

Lambert Strether , November 22, 2017 at 2:03 am

Ah. Reagan's "welfare queens" updated. Kids these days!

JBird , November 22, 2017 at 3:00 am

Yeah being poor, never mind growing up poor, just well and truly sucks and it can really @@@@ you up. Gives people all sorts of issues. I'm rather like her, but I have had the joy of multi-hour commutes to unexciting soul crushing work. Happy, happy, joy, joy! However don't forget that with the current political economy things are likely to go bad in all sorts of ways. This whole site is devoted to that. My suggestion is to keep the job unless you have something lined up. Not being able to rent has it own stresses too. Take my word for it.

Thuto , November 21, 2017 at 8:00 am

I may be engaging in semantics but I think conflating work and jobs makes this article a bit of a mixed bag. I know plenty of people who are terribly unhappy in their jobs, but nonetheless extract a sense of wellbeing from having a stable source of INCOME to pay their bills (anecdotally speaking, acute stress from recent job losses is closely linked to uncertainty about how bills are going to be paid, that's why those with a safety net of accumulated savings report less stress than those without). Loss of status, social standing and identity and the chronic stress borne from these become evident much later I.e. when the unemployment is prolonged, accompanied of course by the still unresolved top-of-mind concern of "how to pay the bills".

As such, acute stress for the recently unemployed is driven by financial/income uncertainty (I.e. how am I going to pay the bills) whereas chronic stress from prolonged unemployment brings into play the more identity driven aspects like loss of social standing and status. For policy interventions to have any effects, policy makers would have to delineate the primary drivers of stress (or lack of wellbeing as the author calls it) during the various phases of the unemployment lifecycle. An Unemployment Insurance Fund (UIF) like we have here in South Africa appears to address the early stages of unemployment, and the accompanying acute stress, quite well by providing the income guarantee (for six months) that cushions the shock of losing a job. What's still missing of course are interventions that promote the quick return to employment for those on UIF, so maybe a middle of the road solution between UBI and a jobs guarantee scheme is how policy makers should be framing this, instead of the binary either/or we currently have.

TroyMcClure , November 21, 2017 at 9:19 am

Lots' of people think they're unhappy with their jobs. Let them sit unemployed for 9 months and ask them if they want that job back. The usual parade of anecdata is on display here in the comments. Mitchell's real data and analysis in the article above still stand.

Thuto , November 21, 2017 at 10:06 am

If you'd read through my comment, and not rushed through it with a view of dishing out a flippant response, you'd have seen that nowhere do I question the validity of his data, I merely question how the argument is presented in some areas (NC discourages unquestioning deference to the views of experts no??). By the way, anecdotes do add to richer understanding of a nuanced and layered topic (as this one is) so your dismissal of them in your haste to invalidate people's observations is hardly helpful.

jrs , November 21, 2017 at 1:15 pm

Yes people many not like their jobs but prefer the security of having them to not. Yes even if the boss sexually harasses one (as we are seeing is very common). Yes even if there is other workplace abuse. Yes even when it causes depression or PTSD (but if one stays with such a job long term it ruins the self confidence that is one prerequisite to get another job!). Yes even if one is in therapy because of job stress, sexual harassment or you name it. The job allows the having health insurance, allows the therapy, allows the complaining about the job in therapy to make it through another week.

Lambert Strether , November 22, 2017 at 2:04 am

> The usual parade of anecdata is on display here in the comments. Mitchell's real data and analysis in the article above still stand.

Ding ding ding!

Democrita , November 21, 2017 at 8:13 am

When unemployed, the stress of worry about money may suppress the creative juices. Speaking from experience. People may well 'keep looking for jobs' because they know ultimately they need a job with steady income. The great experience of some freelancers notwithstanding, not all are cut out for it.

I would love to see some more about happiness or its lack in retirement–referenced by stay-at-home dad BJ , above.

I wonder, too, about the impact of *how* one loses one's job. Getting laid off vs fired vs quitting vs involuntary retirement vs voluntary, etc feel very different. Speaking from experience on that, too. I will search on these points and post anything of interest.

jrs , November 21, 2017 at 1:40 pm

There are also other things that are degrading about the very process of being unemployed not mentioned here. What about the constant rejection that it can entail? One is unemployed and looking for work, one sends out resumes, many of them will never be answered, that's rejection. Then if one is lucky they get interviews, many will never lead to jobs, yet more rejection. Does the process of constant rejection itself have a negative effect on a human being whether it's looking for jobs or dates or whatever? Isn't it learned helplessness to if one keeps trying for something and keeps failing. Isn't that itself demoralizing entirely independent of any doubtful innate demoralizing quality of leisure.

freedeomny , November 21, 2017 at 10:23 am

I am not so sure if I agree with this article. I think it really depends on whether or not you have income to support yourself, hate or love your job, and the amount of outside interests you have, among other things. Almost everyone I know who lives in the NYC area and commutes into the city .doesn't like their job and finds the whole situation "soul-crushing".

Those that live in Manhattan proper are (feel) a bit better off. I for one stopped working somewhat voluntarily last year. I write somewhat because I began to dislike my job so much that it was interfering with my state of well being, however, if I had been allowed to work remotely I probably would have stuck it out for another couple of years.

I am close enough to 62 that I can make do before SS kicks in although I have completely changed my lifestyle – i.e. I've given up a materialistic lifestyle and live very frugally.

Additionally I saved for many years once I decided to embark on this path. I do not find myself depressed at all and the path this year has been very enriching and exciting (and scary) as I reflect on what I want for the future. I'm pretty sure I will end up moving and buying a property so that I can become as self sufficient as possible. Also, I probably will get a job down the line – but if I can't get one because I am deemed too old that will be ok as well. The biggest unknown for me is how much health insurance will cost in the future .

Yves Smith Post author , November 21, 2017 at 5:15 pm

The article made clear that the studies included "unemployed but with income" from government support. It is amazing the degree to which readers ignore that and want to make the findings about "unemployed with no income".

JBird , November 22, 2017 at 3:30 am

That's because we Americans all have work=good=worthy=blessed by God while workless=scum=worthless=accursed by God engraved into our collective soul. Our politics, our beliefs, are just overlays to that.

Even when we agree that the whole situation just crushes people into paste, and for which they have no defense regardless of how hard they work, how carefully they plan, or what they do, that underlay makes use feel that this is their/our fault. Any suggestions that at least some support can be decoupled from work, and that maybe work, and how much you earn, should not determine their value, brings the atavistic fear of being the "undeserving poor," parasites and therefore reprobated scum.

So we don't hear what you are saying without extra effort because it's bypassing our conscious thoughts.

Jamie , November 21, 2017 at 10:43 am

Add my voice to those above who feel that forced labor is the bane of existence, not the wellspring. All this study says to me is that refusing to employ someone in capitalist society does not make them happy. It makes them outcasts.

So, I say yes to a JG, because anyone who wants work should be offered work. But at the same time, a proper JG is not forced labor. And the only way to ensure that it is not forced labor, is to decouple basic needs from wage slavery.

Left in Wisconsin , November 21, 2017 at 12:02 pm

I am critical of those who distinguish between the job and the income. Of course the income is critical to the dignity of the job. For many jobs, it is the primary source of that dignity. The notion that all jobs should provide some intrinsic dignity unrelated to the income, or that people whose dignity is primarily based on the income they earn rather than the work they do are deluded, is to buy in to the propaganda of "passion" being a requirement for your work and to really be blind to what is required to make a society function. Someone has to change the diapers, and wipe the butts of old people. (yes, I've done both.) It doesn't require passion and any sense of satisfaction is gone by about the second day. But if you could make a middle class living doing it, there would be a lot fewer unhappy people in the world.

It is well known that auto factory jobs were not perceived as good jobs until the UAW was able to make them middle class jobs. The nature of the actual work itself hasn't changed all that much over the years – mostly it is still very repetitive work that requires little specialized training, even if the machine technology is much improved. Indeed, I would guess that more intrinsic satisfaction came from bashing metal than pushing buttons on a CNC machine, and so the jobs may even be less self-actualizing than they used to be.

The capitalist myth is that the private sector economy generates all the wealth and the public sector is a claim on that wealth. Yet human development proves to us that this is not true – a substantial portion of "human capital" is developed outside the paid economy, government investment in R&D generates productivity growth, etc. And MMT demonstrates that we do not require private sector savings to fund public investment.

We are still a ways from having the math to demonstrate that government investment in caring and nurturing is always socially productive – first we need productivity numbers that reflect more than just private sector "product." But I think we are moving in that direction. Rather than prioritize a minimum wage JG of make-work, we should first simply pay people good wages to raise their own children or look after their elderly and disabled relatives. The MMT JG, as I understand it, would still require people to leave their kids with others to look after them in order to perform some minimum wage task. That is just dumb.

jrs , November 21, 2017 at 1:31 pm

Maybe it's dumb, it's certainly dumb in a system like the U.S. where work is brutal and often low paid and paid childcare is not well remunerated either. But caretakers also working seems to work in countries with greater income equality, good job protections, flexible work arrangements, and a decent amount of paid parental leave – yea Denmark, they think their children should be raised by professionals, but also work-life balance is still pretty good.

Whiskey Bob , November 21, 2017 at 1:34 pm

My take is that capitalism has made the benefits and malus of having a job so ingrained into culture and so reinforced. Having a job is so closely linked to happiness because it gives you the money needed to pursue it.

A job affords you the ability to pursue whatever goals you want within a capitalist framework. "Everything" costs money and so having a job gives you the money to pay for those costs and go on to fulfill your pursuit of happiness.

Analyzing whether people are happy or not under these conditions seem apparent that it is going to lead to results heavily biased towards finding happiness through employment.

The unemployed are often living off someone else's income and feel like an undeserving parasite. Adults are generally ingrained with the culture that they have to grow up and be independent and be able to provide for a new family that they will start up. Becoming unemployed is like being emasculated and infantile, the opposite of what is expected of adults.

There's also that not having a job is increasingly being punished especially in the case of America. American wages have stayed either largely static or have worsened, making being unemployed that much more of a burden on family or friends. Unemployment has been demonized by Reaganism and has become systematically punishable for the long term unemployed. If you are unemployed for too long, you start losing government support. This compounds the frantic rush to get out of unemployment once unemployed.

There is little luxury to enjoy while unemployed. Life while unemployed is a frustrating and often disappointing hell of constant job applications and having many of them lead to nothing. The people providing support often start to become less so over time and become more convinced of laziness or some kind of lack of character or willpower or education or ability or whatever. Any sense of systemic failure is transplanted into a sense of personal failure, especially under neoliberalism.

I am not so sure about the case of Europe and otherwise. I am sure that the third world often has little or no social safety nets so having work (in exploitative conditions in many cases) is a must for survival.

Anyways, I wonder about the exact methodologies of these studies and I think they often take the current feelings about unemployment and then attempt to extrapolate talking points for UBI/JG from them. Yes, UBI wouldn't change culture overnight and it would take a very, very long time for people to let down their guard and adjust if UBI is to be implemented in a manner that would warrant trust. This article seems to understand the potential for that, but decides against it being a significant factor due to the studies emphasizing the malus of unemployment.

I wonder how different the results would be if there were studies that asked people how they would feel if they were unemployed under a UBI system versus the current system. I know a good number of young people (mostly under 30) who would love to drop out and just play video games all day. Though the significance of such a drastic demographic shift would probably lead to great political consequences. It would probably prove the anti-UBI crowd right in that under a capitalist framework, the capitalists and the employed wouldn't tolerate the unemployed and would seek to turn them into an underclass.

Personally I think a combination of UBI and JG should be pursued. JG would work better within the current capitalist framework. I don't think it is without its pitfalls due to similar possible issues (with the similar policy of full employment) either under Keynesianism (e.g. Milton Friedman sees it as inefficient) or in the USSR (e.g. bullshit jobs). There is the possibility of UBI having benefits (not having the unemployed be a burden but a subsidized contributer to the economy) so I personally don't think it should be fully disregarded until it is understood better. I would like it if there were better scientific studies to expand upon the implications of UBI and better measure if it would work or not. The upcoming studies testing an actual UBI system should help to end the debates once and for all.

redleg , November 21, 2017 at 2:28 pm

My $0.02: I have a creative pursuit (no money) and a engineering/physical science technical career (income!). I am proficient in and passionate about both. Over the last few years, the technical career became tenuous due to consolidation of regional consulting firms (endemic to this era)- wages flat to declining, higher work stress, less time off, conversation to contact employment, etc.- which has resulted in two layoffs.

During the time of tenuous employment, my art took on a darker tone. During unemployment the art stopped altogether.

I'm recently re-employed in a field that I'm not proficient. Both the peter principle and imposter syndrome apply. My art has resumed, but the topics are singular about despair and work, to the point that I feel like I'm constantly reworking the same one piece over and over again. And the quality has plummeted too.

In some fields (e.g. engineering), being a wage slave is the only realistic option due to the dominance of a small number of large firms. The big players crowd out independents and free lancers, while pressuring their own employees through just-high-enough wages and limiting time off. Engineering services is a relationship- based field, and the big boys (and they are nearly all boys) have vastly bigger networks to draw work from than a small firm unless that small firm has a big contact to feed them work (until they get gobbled up). The big firms also have more areas of expertise which limits how useful a boutique firm is to a client pool, except under very narrow circumstances. And if you are an introvert like most engineering people, there's no way to compete with big firms and their marketing staff to expand a network enough to compete.

In that way, consulting is a lot like art. To make a living at it you need either contacts or a sponsor. Or an inheritance.

ChrisPacific , November 21, 2017 at 5:30 pm

I would be interested to know what the definition of unemployment was for the purpose of this study (I couldn't find it in the supplied links). If it's simply "people who don't have a job," for example, then it would include the likes of the idle rich, retirees, wards of the state, and so on. Binary statements like this one do make it sound like the broad definition is the one in use:

When considering the world's population as a whole, people with a job evaluate the quality of their lives much more favorably than those who are unemployed.

The conclusion seems at odds with results I've seen for some of those groups – for example, I thought it was fairly well accepted that retirees who are supported by a government plan that is sufficient for them to live on were generally at least as happy as they had been during their working life.

If, on the other hand, the study uses a narrow definition (e.g. people who are of working age, want a job or need one to support themselves financially, but can't find one) then the conclusion seems a lot more reasonable. But that's a heavily loaded definition in economic and cultural terms. In that case, the conclusion (people are happier if they have a job) only holds true in the current prevailing model of society. It doesn't rule out the possibility of structuring society or the economy differently in such a way that people can be non-working and happy. The existence of one such population already (retirees) strongly suggests that outcomes like this are possible. A UBI would be an example of just such a restructuring of society, and therefore I don't think that this study and its result are necessarily a valid argument against it.

nihil obstet , November 21, 2017 at 6:07 pm

Which makes a person happier -- being considered worthless by one's society or valuable? How many studies do we need to answer that question? Apparently, a lot, because studies like this one keep on going. The underlying assumption is that jobs make one valuable. So if you don't have a job you're worthless. Now, who's happier on the whole, people with jobs or the unemployed? That's surely good for a few more studies. Did you know that members of socially devalued groups (minorities, non-heteros, and the like) have higher rates of dysfunction, rather like the unemployed? Hmm, I wonder if there's maybe a similar principle at work. And my solution is not to turn all the people of color white nor to change all the women to men nor to "cure" gays. Well, maybe a few more conclusive studies of this kind will convince me that we must all be the same, toeing the line for those whom it has pleased God to dictate our values to us.

I am convinced that we shouldn't outlaw jobs, because I believe the tons of stories about happy people in their jobs However, I also believe we shouldn't force everyone into jobs, because I know tons of stories about happy people without jobs. You know, the stories that the JG people explain away: parents caring for their children (JG -- "oh, we'll make that a job!"), volunteers working on local planning issues (JG -- "oh, we'll make that a job, too. In fact, we'll make everything worth doing a job. The important thing is to be able to force people to work schedules and bosses, because otherwise, they'll all lie around doing nothing and be miserable"), the retired (JG -- "that's not really the same, but they'd be better off staying in a job"). And this is all before we get to those who can't really hold a job because of disability or geography or other responsibilities.

I support the JG over the current situation, but as to what we should be working for, the more I read the JG arguments, the more paternalistic and just plain narrow minded judgmental they seem.

Summer , November 21, 2017 at 6:52 pm

If someone else gives you a sense of purpose and takes it away what was the purpose?

Lambert Strether , November 22, 2017 at 1:24 am

Data like that provided by Mitchell is important to demolishing the horrid "economic anxiety" frame much beloved by liberals, especially wonkish Democrats.* It's not (a) just feelings , to be solved by scented candles or training (the liberal version of rugged individualism) and (b) the effects are real and measurable. It's not surprising, when you think about it, that the working class is about work .

* To put this another way, anybody who has really suffered the crawling inwardness of anxiety, in the clinical sense, knows that it affects every aspect of one's being. Anxiety is not something deplorables deploy as cover for less than creditable motives.

[Nov 13, 2017] 20 Sed (Stream Editor) Command Examples for Linux Users

Nov 13, 2017 | www.linuxtechi.com

20 Sed (Stream Editor) Command Examples for Linux Users

by Pradeep Kumar · Published November 9, 2017 · Updated November 9, 2017

Sed command or Stream Editor is very powerful utility offered by Linux/Unix systems. It is mainly used for text substitution , find & replace but it can also perform other text manipulations like insertion deletion search etc. With SED, we can edit complete files without actually having to open it. Sed also supports the use of regular expressions, which makes sed an even more powerful test manipulation tool

In this article, we will learn to use SED command with the help some examples. Basic syntax for using sed command is,

sed OPTIONS [SCRIPT] [INPUTFILE ]

Now let's see some examples.

Example :1) Displaying partial text of a file

With sed, we can view only some part of a file rather than seeing whole file. To see some lines of the file, use the following command,

[linuxtechi@localhost ~]$ sed -n 22,29p testfile.txt

here, option 'n' suppresses printing of whole file & option 'p' will print only line lines from 22 to 29.

Example :2) Display all except some lines

To display all content of a file except for some portion, use the following command,

[linuxtechi@localhost ~]$ sed 22,29d testfile.txt

Option 'd' will remove the mentioned lines from output.

Example :3) Display every 3rd line starting with Nth line

Do display content of every 3rd line starting with line number 2 or any other line, use the following command

[linuxtechi@localhost ~]$ sed -n '2-3p' file.txt
Example :4 ) Deleting a line using sed command

To delete a line with sed from a file, use the following command,

[linuxtechi@localhost ~]$ sed Nd testfile.txt

where 'N' is the line number & option 'd' will delete the mentioned line number. To delete the last line of the file, use

[linuxtechi@localhost ~]$ sed $d testfile.txt
Example :5) Deleting a range of lines

To delete a range of lines from the file, run

[linuxtechi@localhost ~]$ sed '29-34d' testfile.txt

This will delete lines 29 to 34 from testfile.txt file.

Example :6) Deleting lines other than the mentioned

To delete lines other than the mentioned lines from a file, we will use '!'

[linuxtechi@localhost ~]$ sed '29-34!d' testfile.txt

here '!' option is used as not, so it will reverse the condition i.e. will not delete the lines mentioned. All the lines other 29-34 will be deleted from the files testfile.txt.

Example :7) Adding Blank lines/spaces

To add a blank line after every non-blank line, we will use option 'G',

[linuxtechi@localhost ~]$ sed G testfile.txt
Example :8) Search and Replacing a string using sed

To search & replace a string from the file, we will use the following example,

[linuxtechi@localhost ~]$ sed 's/danger/safety/' testfile.txt

here option 's' will search for word 'danger' & replace it with 'safety' on every line for the first occurrence only.

Example :9) Search and replace a string from whole file using sed

To replace the word completely from the file, we will use option 'g' with 's',

[linuxtechi@localhost ~]$ sed 's/danger/safety/g' testfile.txt
Example :10) Replace the nth occurrence of string pattern

We can also substitute a string on nth occurrence from a file. Like replace 'danger' with 'safety' only on second occurrence,

[linuxtechi@localhost ~]$ sed 's/danger/safety/2' testfile.txt

To replace 'danger' on 2nd occurrence of every line from whole file, use

[linuxtechi@localhost ~]$ sed 's/danger/safety/2g' testfile.txt
Example :11) Replace a string on a particular line

To replace a string only from a particular line, use

[linuxtechi@localhost ~]$ sed '4 s/danger/safety/' testfile.txt

This will only substitute the string from 4th line of the file. We can also mention a range of lines instead of a single line,

[linuxtechi@localhost ~]$  sed '4-9 s/danger/safety/' testfile.txt
Example :12) Add a line after/before the matched search

To add a new line with some content after every pattern match, use option 'a' ,

[linuxtechi@localhost ~]$ sed '/danger/a "This is new line with text after match"' testfile.txt

To add a new line with some content a before every pattern match, use option 'i',

[linuxtechi@localhost ~]$ sed '/danger/i "This is new line with text before match" ' testfile.txt
Example :13) Change a whole line with matched pattern

To change a whole line to a new line when a search pattern matches we need to use option 'c' with sed,

[linuxtechi@localhost ~]$ sed '/danger/c "This will be the new line" ' testfile.txt

So when the pattern matches 'danger', whole line will be changed to the mentioned line.

Advanced options with sed

Up until now we were only using simple expressions with sed, now we will discuss some advanced uses of sed with regex,

Example :14) Running multiple sed commands

If we need to perform multiple sed expressions, we can use option 'e' to chain the sed commands,

[linuxtechi@localhost ~]$  sed -e 's/danger/safety/g' -e 's/hate/love/' testfile.txt
Example :15) Making a backup copy before editing a file

To create a backup copy of a file before we edit it, use option '-i.bak',

[linuxtechi@localhost ~]$ sed -i.bak -e 's/danger/safety/g'  testfile.txt

This will create a backup copy of the file with extension .bak. You can also use other extension if you like.

Example :16) Delete a file line starting with & ending with a pattern

To delete a file line starting with a particular string & ending with another string, use

[linuxtechi@localhost ~]$ sed -e 's/danger.*stops//g' testfile.txt

This will delete the line with 'danger' on start & 'stops' in the end & it can have any number of words in between , '.*' defines that part.

Example :17) Appending lines

To add some content before every line with sed & regex, use

[linuxtechi@localhost ~]$ sed -e 's/.*/testing sed &/' testfile.txt

So now every line will have 'testing sed' before it.

Example :18) Removing all commented lines & empty lines

To remove all commented lines i.e. lines with # & all the empty lines, use

[linuxtechi@localhost ~]$ sed -e 's/#.*//;/^$/d' testfile.txt

To only remove commented lines, use

[linuxtechi@localhost ~]$ sed -e 's/#.*//' testfile.txt
Example :19) Get list of all usernames from /etc/passwd

To get the list of all usernames from /etc/passwd file, use

[linuxtechi@localhost ~]$  sed 's/\([^:]*\).*/\1/' /etc/passwd

a complete list all usernames will be generated on screen as output.

Example :20) Prevent overwriting of system links with sed command

'sed -i' command has been know to remove system links & create only regular files in place of the link file. So to avoid such a situation & prevent ' sed -i ' from destroying the links, use ' –follow-symklinks ' options with the command being executed.

Let's assume i want to disable SELinux on CentOS or RHEL Severs

[linuxtechi@localhost ~]# sed -i --follow-symlinks 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/sysconfig/selinux

These were some examples to show sed, we can use these reference to employ them as & when needed. If you guys have any queries related to this or any article, do share with us.

[Nov 12, 2017] Installing Nagios 3.4.4 On CentOS 6.3

Nov 12, 2017 | www.howtoforge.com

Installing Nagios 3.4.4 On CentOS 6.3 Introduction

Nagios is a monitoring tool under GPL licence. This tool lets you monitor servers, network hardware (switches, routers, ...) and applications. A lot of plugins are available and its big community makes Nagios the biggest open source monitoring tool. This tutorial shows how to install Nagios 3.4.4 on CentOS 6.3.

Prerequisites

After installing your CentOS server, you have to disable selinux & install some packages to make nagios work.

To disable selinux, open the file: /etc/selinux/config

# vi /etc/selinux/config

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=permissive // change this value to disabled
# SELINUXTYPE= can take one of these two values:
#     targeted - Targeted processes are protected,
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

Now, download all packages you need:

# yum install gd gd-devel httpd php gcc glibc glibc-common

Nagios Installation

Create a directory:

# mkdir /root/nagios

Navigate to this directory:

# cd /root/nagios

Download nagios-core & plugin:

# wget http://prdownloads.sourceforge.net/sourceforge/nagios/nagios-3.4.4.tar.gz
# wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.16.tar.gz

Untar nagios core:

# tar xvzf nagios-3.4.4.tar.gz

Go to the nagios dir:

# cd nagios

Configure before make:

# ./configure

Make all necessary files for Nagios:

# make all

Installation:

# make install

# make install-init

# make install-commandmode

# make install-config

# make install-webconf

Create a password to log into the web interface:

# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

Start the service and start it on boot:

# chkconfig nagios on
# service nagios start

Now, you have to install the plugins:

# cd ..
# tar xvzf nagios-plugins-1.4.15.tar.gz
# cd nagios-plugins-1.4.15
# ./configure
# make
# make install

Start the apache service and enable it on boot:

# service httpd start
# chkconfig httpd on

Now, connect to your nagios system:

http://Your-Nagios-IP/nagios and enter login : nagiosadmin & password you have chosen above.

And after the installation ?

After the installation you have to configure all your host & services in nagios configuration files.This step is performed in command line and is complicated, so I recommand to install tool like Centreon, that is a beautiful front-end to add you host & services.

To go further, I recommend you to read my article on Nagios & Centreon monitoring .

[Nov 12, 2017] How to Install Nagios 4 in Ubuntu and Debian

Nov 12, 2017 | www.tecmint.com

Requirements

  1. Debian 9 Minimal Installation
  2. Ubuntu 16.04 Minimal Installation
Step 1: Install Pre-requirements for Nagios

1. Before installing Nagios Core from sources in Ubuntu or Debian , first install the following LAMP stack components in your system, without MySQL RDBMS database component, by issuing the below command.

# apt install apache2 libapache2-mod-php7.0 php7.0

2. On the next step, install the following system dependencies and utilities required to compile and install Nagios Core from sources, by issuing the follwoing command.

# apt install wget unzip zip  autoconf gcc libc6 make apache2-utils libgd-dev
Step 2: Install Nagios 4 Core in Ubuntu and Debian

3. On the first step, create nagios system user and group and add nagios account to the Apache www-data user, by issuing the below commands.

# useradd nagios
# usermod -a -G nagios www-data

4. After all dependencies, packages and system requirements for compiling Nagios from sources are present in your system, go to Nagios webpage and grab the latest version of Nagios Core stable source archive by issuing the following command.

# wget https://assets.nagios.com/downloads/nagioscore/releases/nagios-4.3.4.tar.gz

5. Next, extract Nagios tarball and enter the extracted nagios directory, with the following commands. Issue ls command to list nagios directory content.

# tar xzf nagios-4.3.4.tar.gz 
# cd nagios-4.3.4/
# ls
List Nagios Content

List Nagios Content

6. Now, start to compile Nagios from sources by issuing the below commands. Make sure you configure Nagios with Apache sites-enabled directory configuration by issuing the below command.

# ./configure --with-httpd-conf=/etc/apache2/sites-enabled

7. In the next step, build Nagios files by issuing the following command.

# make all

8. Now, install Nagios binary files, CGI scripts and HTML files by issuing the following command.

# make install

9. Next, install Nagios daemon init and external command mode configuration files and make sure you enable nagios daemon system-wide by issuing the following commands.

# make install-init
# make install-commandmode
# systemctl enable nagios.service

10. Next, run the following command in order to install some Nagios sample configuration files needed by Nagios to run properly by issuing the below command.

# make install-config

11. Also, install Nagios configuration file for Apacahe web server, which can be fount in /etc/apacahe2/sites-enabled/ directory, by executing the below command.

# make install-webconf

12. Next, create nagiosadmin account and a password for this account necessary by Apache server to log in to Nagios web panel by issuing the following command.

# htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin

13. To allow Apache HTTP server to execute Nagios cgi scripts and to access Nagios admin panel via HTTP, first enable cgi module in Apache and then restart Apache service and start and enable Nagios daemon system-wide by issuing the following commands.

# a2enmod cgi
# systemctl restart apache2
# systemctl start nagios
# systemctl enable nagios

14. Finally, log in to Nagios Web Interface by pointing a browser to your server's IP address or domain name at the following URL address via HTTP protocol. Log in to Nagios with nagiosadmin user the password setup with htpasswd script.

http://IP-Address/nagios
OR
http://DOMAIN/nagios

[Nov 09, 2017] Make sudo work harder

www.linux.com
www.ibm.com/developerworks

Managing sudoers

Over time, your sudoers file will grow with more and more entries, which is to be expected. This could be because more application environments are being placed on the server, or because of splitting the delegation of currents tasks down further to segregate responsibility. With many entries, typos can occur, which is common. Making the sudoers file more manageable by the root user makes good administrative sense. Let's look at two ways this can be achieved, or at least a good standard to build on. If you have many static entries (meaning the same command is run on every machine where sudo is), put these into a separate sudoers file, which can be achieved using the include directive.

Having many entries for individual users can also be time consuming when adding or amending entries. With many user entries, it is good practice to put these into groups. Using groups, you can literally group users together, and the groups are valid AIX groups.

Now look at these two methods more closely.

Include file

Within large-enterprise environments, keeping the sudoers file maintained is an important and regularly required task. A solution to make this chore easier is to reorganize the sudoers file. One way to do this is to extract entries that are static or reusable, where the same commands are run on every box. Like audit/security or storix backups or general performance reports, with sudo you can now use the include directive. The main sudoers file can then contain the local entries, and the include file would barely need editing as those entries are static. When visudo is invoked, it will scan sudoers when it sees the include entry. It will scan that file, then come back to the main sudoers and carry on scanning. In reality, it works like this. When you exit out of visudo from the main sudoers file, it will take you to the include file for editing. Once you quit the include, you are back to the AIX prompt. You can have more than one include file, but I cannot think of a reason why you would want more than one.

Let's call our secondary sudoers file sudo_static.<hostname>. In the examples in this demonstration the hostname I am using is rs6000. In the main sudoers file, make the entry as follows:

1 #include /etc/sudo_static.rs6000

Next, add some entries to the /etc/sudo_static.rs6000 file. You do not have to put in all the sudoers directives or stanzas. If this file contains entries where they are not required, don't include them. For example, my include file contains only the following text, and nothing more.

You can use the %h, instead of typing the actual hostname:

I personally do not use this method because I have experienced returning extra characters on the hostname. This issue is fixed in sudo 1.7.2 p1.

1 2 3 4 bravo rs6000 = (root) NOPASSWD: /usr/opt/db2_08_01/adm/db2licd -end bravo rs6000 = (root) NOPASSWD: /usr/opt/db2_08_01/adm/db2licd bravo rs6000 = (db2inst) NOPASSWD: /home/db2inst/sqllib/adm/db2start bravo rs6000 = (db2inst) NOPASSWD: /home/db2inst/sqllib/adm/db2stop force

When you run visudo, and you save and quit the file, visudo will inform you to click Enter to edit the include sudoers file. Once you have edited the file, sudo will pick up on syntax errors if any, as with the main file. Alternatively, to edit the include file directly, use:

1 visudo -f /etc/sudo_static.rs6000

.

Using groups

Users belonging to a valid AIX group can be included in sudoers, making the sudoers file more manageable with fewer entries per user. When reorganizing the sudoers entries to include groups, you may have to create a new groups under AIX to include users that are only allowed to use sudo for certain commands. To use groups, simply prefix the entries with a '%'. Assume you have groups called devops and devuat , and with those groups you have the following users:

1 2 3 4 5 6 7 8 # lsgroup -f -a users devops devops: users=joex,delta,charlie,tstgn # lsgroup -f -a users devuat devuat: users=zebra,spsys,charlie

For the group devops to be allowed to run the /usr/local/bin/data_ext.sh command as dbdftst.

For the group devuat to be allowed to run the commands :/usr/local/bin/data_mvup.sh, /usr/local/bin/data_rep.sh as dbukuat.

We could have the following sudoers entries:

1 2 3 %devops rs6000 = (dbdftst) NOPASSWD: /usr/local/bin/data_ext.sh %devuat rs6000 = (dbukuat) /usr/local/bin/data_mvup.sh %devuat rs6000 = (dbukuat) /usr/local/bin/data_rep.sh

Notice in the previous entries, the group devops users will not be prompted for their password when executing /usr/local/bin/data_ext.sh; however, the group devuat users will be prompted for their password. User "charlie" is a member of both groups ( devops and devuat ), so he can execute all the above commands.

Timeout with sudo

Sudo has a feature that uses time tickets to determine how long since the last sudo command was run. During this time period, the user can re-run the command without being prompted for the password (that's the user's own password). Once this time allotment has ended, the user is prompted for the password again to re-run the command. If the user gives the correct password, the command is executed, the ticket is then re-set, and the time clock starts all over again. The ticket feature will not work if you have NOPASSWD in the user's entry in sudoers. The default timeout is five minutes. If you wish to change the default value, simply put an entry in sudoers. For example, to set the timeout value for user "bravo" on any commands he runs to 20 minutes, you could use:

1 Defaults:bravo timestamp_timeout=20

To destroy the ticket, as the user, use:

1 $ sudo -k

When the ticket is destroyed, the user will be prompted for his password again, when running a sudo command.

Please do not set the timeout value for all users, as this will cause problems, especially when running jobs in batch and the batch takes longer to run than normal. To disable this feature, use the value -1 in the timestamp_timeout variable. The time tickets are directory entries with the name of the user located in /var/run/sudo.

Those variables

As discussed earlier, sudo will strip out potentially dangerous system variables. To check out what variables are kept and which ones are stripped, use sudo -V . The output will give you a listing of preserved and stripped variables. Stripping out the LIBPATH is clearly an inconvenience. There are a couple of ways around this--either write a wrapper script or specify the environments on the command line. Looking at the wrapper script solution first, suppose you have an application that stops or starts a DB2® instance. You could create a bare-bones script that would keep the variables intact. In Listing 1. rc.db2 , notice that you source the instance profile, which in turn exports various LIBPATH and DB2 environment variables, keeping the environment variable intact, by using:

1 . /home/$inst/sqllib/db2profile

For completeness, the entries in sudoers to execute this is and not strip out any system environment variables are:

1 2 3 4 bravo rs6000 = (dbinst4) NOPASSWD: /home/dbinst4/sqllib/adm/db2start bravo rs6000 = (dbinst4) NOPASSWD: /home/dbinst4/sqllib/adm/db2stop force bravo rs6000 = (dbinst4) NOPASSWD: /usr/local/bin/rc.db2 stop db2inst4 bravo rs6000 = (dbinst4) NOPASSWD: /usr/local/bin/rc.db2 start db2inst4

Note in this example, user "bravo" can execute the above commands as user "dbinst4." Typically, the user would run:

1 2 sudo -u dbinst4 /usr/local/bin/rc.db2 stop db2inst4 sudo -u dbinst4 /usr/local/bin/rc.db2 start db2inst4
Listing 1. rc.db2
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 #!/bin/sh # rc.db2 # stop/start db2 instances # check to see if db2 inst is runningdb2_running(){state=`ps -ef |grep db2sysc |grep -v grep| awk '$1=="'${inst}'" { print $1 }'` if [ "$state" = "" ] then return 1 else return 0 fi} usage () { echo "`basename $0` start | stop <instance>" } # stop db2 stop_db2 () { echo "stopping db2 instance as user $inst" if [ -f /home/$inst/sqllib/db2profile ]; then . /home/$inst/sqllib/db2profile else echo "Cannot source DB2..exiting" exit 1 fi /home/$inst/sqllib/adm/db2stop force } # start db2 start_db2 () { echo "starting db2 instance as user $inst" if [ -f /home/$inst/sqllib/db2profile ]; then . /home/$inst/sqllib/db2profile else echo "Cannot source DB2..exiting" exit 1 fi /home/$inst/sqllib/adm/db2start } # check we get 2 params if [ $# != 2 ] then usage exit 1 fi inst=$2 case "$1" in Start|start) if db2_running then echo "db2 instance $inst appears to be already running" exit 0 else echo " instance not running as user $inst..attempting to start it" start_db2 $inst fi ;; Stop|stop) if db2_running then echo "instance running as $inst..attempting to stop it" stop_db2 $inst else echo "db2 instance $inst appears to be not running anyway" exit 0 fi ;; *) usage ;; esac

The other way to preserve system environment variables is to use the Defaults !env_reset directive, like in sudoers:

1 Defaults !env_reset

Then from the command line, specify the environment variable name with its value:

1 $ sudo LIBPATH=″/usr/lib:/opt/db2_09_05/lib64″ -u delta /usr/local/bin/datapmp

If you do not put the !env_reset entry in, you will get the following error from sudo when you try to run the command:

1 sudo: sorry, you are not allowed to set the following environment variables: LIBPATH

If you find that sudo is also stripping out other environment variables, you can specify the variable name in sudoers so that sudo keeps those variables intact (with the Defaults env_keep += directive). For instance, suppose sudo was stripping out the application variables DSTAGE_SUP and DSTAGE_META from one of my suodo-ised scripts. To preserve these variables, I could put the following entries in sudoers:

1 2 Defaults env_keep += "DSTAGE_SUP" Defaults env_keep += "DSTAGE_META"

Notice that I give the variable name and not the variable value. The values are already contained in my script like this:

1 export DSTAGE_SUP=/opt/dstage/dsengine; export DSTAGE_META=/opt/dstage/db2

Now when the sudo script is executed, the above environment variables are preserved.

Securing the sudo path

A default PATH within sudoers can be imposed using the secure_path directive. This directive specifies where to look for binaries and commands when a user executes a sudo command. This option clearly tries to lock down specific areas where a user runs a sudo command, which is good practice. Use the following directive in sudoers, specifying the secure PATH with its search directories:

1 Defaults secure_path="/usr/local/sbin:/usr/local/bin:/opt/freeware/bin:/usr/sbin"
Getting restrictive

Restrictions can be put in place to restrict certain commands to users. Assume you have a group called dataex , whose members are "alpha," "bravo," and "charlie." Now, that group has been allowed to run the sudo command /usr/local/bin/mis_ext * , where the asterisk represents the many parameters passed to the script. However, user "charlie" is not allowed to execute that script if the parameter is import . This type of condition can be met by using the logical NOT '!' operator. Here is how that is achieved in sudoers:

1 2 %dataex rs6000 = (dbmis) NOPASSWD: /usr/local/bin/mis_ext * charlie rs6000 = (dbmis) NOPASSWD: !/usr/local/bin/mis_ext import

Note that the logical NOT operator entries go after the non-restrictive entry. Many conditional NOT entries can be applied on the same line; just make sure that they are comma separated, like so:

1 2 3 4 charlie rs6000 = (dbmis) NOPASSWD: /usr/local/bin/aut_pmp * charlie rs6000 = (dbmis) NOPASSWD: !/usr/local/bin/aut_pmp create, !/usr/local/bin/aut_pmp delete, !/usr/local/bin/aut_pmp amend
When in visudo, do not think just saving the sudo entry and staying in visudo will make the changes effective; it won't. You must exit visudo for the changes to take effect. Rolling out sudo commands

Rolling out sudo commands to remote hosts in an enterprise environment is best done using a ssh script as root, and the keys should have been exchanged between the hosts, for password-less logins. Let's look at one example of how to do this. With geographically remote machines, if you get a hardware issue of some sort (disk or memory), the IBM® engineer will be on-site to replace the failing hardware. There will be occasions when they require the root password to carry out their task. One procedure you might want to put in place is for the engineer to gain access to root they must use sudo. Informing the engineer prior to the visit of the password would be advantageous. Listing 2 demonstrates one way you could roll out this configuration. Looking more closely at Listing 2 , use a for loop containing a list of hosts you are pushing out to. (Generally, though, you would have these hosts in a text file and read them in using a while loop.) Using the 'here' document method, make a backup copy of sudoers, and an entry is then appended to sudoers, like so:

1 2 # -- ibmeng sudo root ibmeng host1 = (root) NOPASSWD:ALL

Next, the user "ibmeng" is created, and the password is set for the user using chpasswd . In this demonstration, it is ibmpw . A message is then appended to their profile, informing the user how to sudo to root. So when the engineer logs in, he is presented with the message:

1 IBM Engineer, to access root account type: sudo -u root su -

Of course the account for ibmeng would be locked after the visit.

Listing 2. dis_ibm
Nov 09, 2017 | www.ibm.com
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 #!/bin/sh # dis_ibm dest_hosts='host1 host2 host3 host4' for host in $dest_hosts do echo "doing [$host]" $ssh -T -t -l root $host<<'mayday' host=`hostname` cp /etc/sudoers /etc/sudoers.bak if [ $? != 0 ] then echo "error: unable to cp sudoers file" exit 1 fi echo "# -- ibmeng sudo root\nibmeng $host = (root) NOPASSWD:ALL">>/etc/sudoers mkuser su=false ibmeng if [ $? = 0 ] then echo "ibmeng:ibmpw" | chpasswd -c else echo "error: unable to create user ibmeng and or passwd" exit 1 fi chuser gecos='IBM engineer acc' ibmeng if [ -f /home/ibmeng/.profile ] then echo "echo \"IBM Engineer, to access root account type: sudo -u root su -"\" >>/home/ibmeng/.profile fi mayday done
Conclusion

Sudo allows you to control who can run what commands as whom. But you must be able to understand the features of sudoers fully to gain maximum understanding of its implications and responsibility.


Downloadable resources
Related topics

[Nov 09, 2017] Understanding sudoers syntax

Nov 09, 2017 | toroid.org

User specifications

The /etc/sudoers file contains "user specifications" that define the commands that users may execute. When sudo is invoked, these specifications are checked in order, and the last match is used. A user specification looks like this at its most basic:

User Host = (Runas) Command

Read this as "User may run Command as the Runas user on Host".

Any or all of the above may be the special keyword ALL, which always matches.

User and Runas may be usernames, group names prefixed with %, numeric UIDs prefixed with #, or numeric GIDs prefixed with %#. Host may be a hostname, IP address, or a whole network (e.g., 192.0.2.0/24), but not 127.0.0.1.

Runas

This optional clause controls the target user (and group) sudo will run the Command as, or in other words, which combinations of the -u and -g arguments it will accept.

If the clause is omitted, the user will be permitted to run commands only as root. If you specify a username, e.g., (postgres), sudo will accept "-u postgres" and run commands as that user. In both cases, sudo will not accept -g.

If you also specify a target group, e.g., (postgres:postgres), sudo will accept any combination of the listed users and groups (see the section on aliases below). If you specify only a target group, e.g., (:postgres), sudo will accept and act on "-g postgres" but run commands only as the invoking user.

This is why you sometimes see (ALL:ALL) in the 90% example.

Commands

In the simplest case, a command is the full path to an executable, which permits it to be executed with any arguments. You may specify a list of arguments after the path to permit the command only with those exact arguments, or write "" to permit execution only without any arguments.

A command may also be the full path to a directory (including a trailing /). This permits execution of all the files in that directory, but not in any subdirectories.

ams ALL=/bin/ls, /bin/df -h /, /bin/date "", \
        /usr/bin/, sudoedit /etc/hosts, \
        OTHER_COMMANDS

The keyword sudoedit is also recognised as a command name, and arguments can be specified as with other commands. Use this instead of allowing a particular editor to be run with sudo, because it runs the editor as the user and only installs the editor's output file into place as root (or other target user).

As shown above, comma-separated lists of commands and aliases may be specified. Commands may also use shell wildcards either in the path or in the argument list (but see the warning below about the latter).

Sudo is very flexible, and it's tempting to set up very fine-grained access, but it can be difficult to understand the consequences of a complex setup, and you can end up with unexpected problems . Try to keep things simple.

Options

Before the command, you can specify zero or more options to control how it will be executed. The most important options are NOPASSWD (to not require a password) and SETENV (to allow the user to set environment variables for the command).

ams ALL=(ALL) NOPASSWD: SETENV: /bin/ls

Other available options include NOEXEC, LOG_INPUT and LOG_OUTPUT, and SELinux role and type specifications. These are all documented in the manpage.

Digests

The path to a binary (i.e., not a directory or alias) may also be prefixed with a digest:

ams ALL=(ALL) sha224:IkotndXGTmZtH5ZNFtRfIwkG0WuiuOs7GoZ+6g== /bin/ls

The specified binary will then be executed only if it matches the digest. SHA-2 digests of 224, 256, 384, and 512-bits are accepted in hex or Base64 format. The values can be generated using, e.g., sha512sum or openssl.

Aliases

In addition to the things listed above, a User, Host, Runas, or Command may be an alias, which is a named list of comma-separated values of the corresponding type. An alias may be used wherever a User, Host, Runas, or Command may occur. They are always named in uppercase, and can be defined as shown in these examples:

# Type_Alias NAME = a, b : NAME_2 = c, d,  

User_Alias TRUSTED = %admin, !ams
Runas_Alias LEGACYUSERS = oldapp1, oldapp2
Runas_Alias APPUSERS = app1, app2, LEGACYUSERS
Host_Alias PRODUCTION = www1, www2, \
    192.0.2.1/24, !192.0.2.222
Cmnd_Alias DBA = /usr/pgsql-9.4/bin, \
    /usr/local/bin/pgadmin

An alias definition can also include another alias of the same type (e.g., LEGACYUSERS above). You cannot include options like NOPASSWD: in command aliases.

Any term in a list may be prefixed with ! to negate it. This can be used to include a group but exclude a certain user, or to exclude certain addresses in a network, and so on. Negation can also be used in command lists, but note the manpage's warning that trying to "subtract" commands from ALL using ! is generally not effective .

Use aliases whenever you need rules involving multiple users, hosts, or commands.

Default options

Sudo has a number of options whose values may be set in the configuration file, overriding the defaults either unconditionally, or only for a given user, host, or command. The defaults are sensible, so you do not need to care about options unless you're doing something special.

Option values are specified in one or more "Defaults" lines. The example below switches on env_reset, turns off insults (read !insults as "not insults"), sets password_tries to 4, and so on. All the values are set unconditionally, i.e. they apply to every user specification.

Defaults env_reset, !insults, password_tries=4, \
    lecture=always
Defaults passprompt="Password for %p:"

Options may also be set only for specific hosts, users, or commands, as shown below. Defaults@host sets options for a host, Defaults:user for a (requesting) user, Defaults!command for a command, and Defaults>user for a target user. You can also use aliases in these definitions.

Defaults@localhost insults
Defaults:ams insults, !lecture
Defaults>root mail_always, mailto="foo@example.org"

Cmnd_Alias FOO = /usr/bin/foo, /usr/bin/bar, \
    /usr/local/bin/baz
Defaults!FOO always_set_home

Unconditional defaults are parsed first, followed by host and user defaults, then runas defaults, then command defaults.

The many available options are explained well in the manpage.

Complications

In addition to the alias mechanism, a User, Host, Runas, or Command may each be a comma-separated list of things of the corresponding type. Also, a user specification may contain multiple host and command sets for a single User. Please be sparing in your use of this syntax, in case you ever have to make sense of it again.

Users and hosts can also be a +netgroup or other more esoteric things, depending on plugins. Host names may also use shell wildcards (see the fqdn option).

If Runas is omitted but the () are not, sudo will reject -u and -g and run commands only as the invoking user.

You can use wildcards in command paths and in arguments, but their meaning is different. In a path, a * will not match a /, so /usr/bin/* will match /usr/bin/who but not /usr/bin/X11/xterm. In arguments, a * does match /; also, arguments are matched as a single string (not a list of separate words), so * can match across words. The manpage includes the following problematic example, which permits additional arguments to be passed to /bin/cat without restriction:

%operator ALL = /bin/cat /var/log/messages*

Warning : Sudo will not work if /etc/sudoers contains syntax errors, so you should only ever edit it using visudo, which performs basic sanity checks, and installs the new file only if it parses correctly.

Another warning: if you take the EBNF in the manpage seriously enough, you will discover that the implementation doesn't follow it. You can avoid this sad fate by linking to this article instead of trying to write your own. Happy sudoing!

[Nov 09, 2017] Example of the sudoers file

Nov 09, 2017 | support.symantec.com

Example of the sudoers file

This is an example of the contents of the sudoers file is located in the /etc directory of the UNIX target computer. This example contains sample configurations required to use the sudo functionality as mentioned in the section Using sudo functionality for querying Oracle UNIX targets .

# User alias specification
##
User_Alias UNIX_USERS = unix1, unix2, unix3
User_Alias BV_CONTROL_USERS = bvunix1, bvunix2, bvunix3
##
# Runas alias specification
Defaults:UNIX_USERS !authenticate
Defaults:BV_CONTROL_USERS !authenticate
##
Runas_Alias SUPER_USERS = root
Defaults logfile=/var/log/sudolog
##
# Cmnd alias specification
##
Cmnd_Alias APPLICATIONS = /usr/sbin/named
Cmnd_Alias AIX_ADMINCMDS = /usr/sbin/lsps, /usr/sbin/lsattr
Cmnd_Alias ADMINCMDS = /usr/sbin/prtconf, /sbin/runlevel, ulimit, AIX_ADMINCMDS,
Cmnd_Alias NETWORKCMDS = /sbin/ifconfig, /usr/local/bin/nslookup, inetadm -p
Cmnd_Alias FILECMDS = /bin/cat, /bin/date '+%Z', /usr/bin/strings -n, \
/usr/bin/diff, /usr/bin/cmp, /usr/bin/find, \
/bin/echo, /usr/bin/file, /bin/df -P, \
/usr/bin/cksum, /bin/ls -la, /bin/ls -lad, \
/bin/ls -lac, /bin/ls -lau
#Cmnd_Alias COMMONCMDS = /usr/bin, /bin, /usr/local/bin
Cmnd_Alias SU = /usr/bin/su
Cmnd_Alias SYSADMCMD = /usr/lib/sendmail
Cmnd_Alias ACTIVEADMCMDS = /usr/sbin/adduser
UNIX_USERS ALL = (SUPER_USERS) APPLICATIONS, NETWORKCMDS, ADMINCMDS, FILECMDS, !SU, !ACTIVEADMCMDS,
!SYSADMCMD, NOPASSWD: ALL
BV_CONTROL_USERS ALL = NOPASSWD: ALL

See Using sudo functionality for querying Oracle UNIX targets .

See Disabling password prompt in the sudoers file .

See Minimum required privileges to query an Oracle database .

[Nov 09, 2017] Add an netgroup in sudoers instead a group

Nov 09, 2017 | hd.free.fr

5 thoughts on "sudo command & sudoers file : Concepts and Practical examples"

  1. Pingback: sudo | Site Title
  2. Andres Ferreo July 16, 2014 at 21:18

    I'll like to add an netgroup in sudoers instead a group. That is possible? How should I do this setup

    Thanks.

    Reply ↓
    1. Pier Post author July 17, 2014 at 22:50

      In order to use a netgroup in the sudoers file, you just need to explicitly define it as a netgroup by using the " + " sign (instead of a " % " sign that would be used for a system group).

      You will need to include this netgroup inside a User_Alias (you may want to create a new User_Alias for this purpose)

      Please check the " 3.1.2 User_Alias " section for more infos, and feel free to ask for more detailed explanation.

      Hope this helps.

      Pier.

      Reply ↓
  3. Matthew February 14, 2014 at 15:43

    Great info, just diving into the world of this, and was trying to figure out how to limit a login to run a cache clearing command

    user ALL=NOPASSWD: rm -rf /usr/nginx/cache/*

    but i got a syntax error

    Reply ↓
    1. Pier Post author February 17, 2014 at 07:22

      Hi,

      Looks like you forgot the following part of the command specs :
      3. (ALL) : This is the part that specify which user(s) you may act as.

      Check the 2.1 Section of the current page, you may want to have something like :
      user ALL=(ALL) NOPASSWD: /sbin/rm -rf /usr/nginx/cache/*

      Always use the full path for any given command : This will prevent you from using a bad aliased command.

[Nov 09, 2017] Sudoers - Community Help Wiki

Nov 09, 2017 | help.ubuntu.com

... ... ...

Aliases

There are four kinds of aliases: User_Alias, Runas_Alias, Host_Alias and Cmnd_Alias. Each alias definition is of the form:

Where Alias_Type is one of User_Alias, Runas_Alias, Host_Alias or Cmnd_Alias. A name is a string of uppercase letters, numbers and underscores starting with an uppercase letter. You can put several aliases of the same type on one line by separating them with colons (:) as so:

You can include other aliases in an alias specification provided they would normally fit there. For example you can use a user alias wherever you would normally expect to see a list of users (for example in a user or runas alias).

There are also built in aliases called ALL which match everything where they are used. If you used ALL in place of a user list it matches all users for example. If you try and set an alias of ALL it will be overridden by this built in alias so don't even try.

User Aliases

User aliases are used to specify groups of users. You can specify usernames, system groups (prefixed by a %) and netgroups (prefixed by a +) as follows:

 # Everybody in the system group "admin" is covered by the alias ADMINS
 User_Alias ADMINS = %admin
 # The users "tom", "dick", and "harry" are covered by the USERS alias
 User_Alias USERS = tom, dick, harry
 # The users "tom" and "mary" are in the WEBMASTERS alias
 User_Alias WEBMASTERS = tom, mary
 # You can also use ! to exclude users from an alias
 # This matches anybody in the USERS alias who isn't in WEBMASTERS or ADMINS aliases
 User_Alias LIMITED_USERS = USERS, !WEBMASTERS, !ADMINS
Runas Aliases

Runas Aliases are almost the same as user aliases but you are allowed to specify users by uid's. This is helpful as usernames and groups are matched as strings so two users with the same uid but different usernames will not be matched by entering a single username but can be matched with a uid. For example:

 # UID 0 is normally used for root
 # Note the hash (#) on the following line indicates a uid, not a comment.
 Runas_Alias ROOT = #0
 # This is for all the admin users similar to the User_Alias of ADMINS set earlier 
 # with the addition of "root"
 Runas_Alias ADMINS = %admin, root
Host Aliases

A host alias is a list of hostname, ip addresses, networks and netgroups (prefixed with a +). If you do not specify a netmask with a network the netmask of the hosts ethernet interface(s) will be used when matching.

 # This is all the servers
 Host_Alias SERVERS = 192.168.0.1, 192.168.0.2, server1
 # This is the whole network
 Host_Alias NETWORK = 192.168.0.0/255.255.255.0
 # And this is every machine in the network that is not a server
 Host_Alias WORKSTATIONS = NETWORK, !SERVER
 # This could have been done in one step with 
 # Host_Alias WORKSTATIONS = 192.168.0.0/255.255.255.0, !SERVERS
 # but I think this method is clearer.
Command Aliases

Command aliases are lists of commands and directories. You can use this to specify a group of commands. If you specify a directory it will include any file within that directory but not in any subdirectories.

The special command '"sudoedit"' allows users to run sudo with the -e flag or as the command sudoedit . If you include command line arguments in a command in an alias these must exactly match what the user enters on the command line. If you include any of the following they will need to be escaped with a backslash (\): ",", "\", ":", "=".

Examples:

 # All the shutdown commands
 Cmnd_Alias SHUTDOWN_CMDS = /sbin/poweroff, /sbin/reboot, /sbin/halt
 # Printing commands
 Cmnd_Alias PRINTING_CMDS = /usr/sbin/lpc, /usr/sbin/lprm
 # Admin commands
 Cmnd_Alias ADMIN_CMDS = /usr/sbin/passwd, /usr/sbin/useradd, /usr/sbin/userdel, /usr/sbin/usermod, /usr/sbin/visudo
 # Web commands
 Cmnd_Alias WEB_CMDS = /etc/init.d/apache2
User Specifications

User Specifications are where the sudoers file sets who can run what as who. It is the key part of the file and all the aliases have just been set up for this very point. If this was a film this part is where all the key threads of the story come together in the glorious unveiling before the final climatic ending. Basically it is important and without this you ain't going anywhere.

A user specification is in the format

<user list> <host list> = <operator list> <tag list> <command list>

The user list is a list of users or a user alias that has already been set, the host list is a list of hosts or a host alias, the operator list is a list of users they must be running as or a runas alias and the command list is a list of commands or a cmnd alias.

The tag list has not been covered yet and allows you set special things for each command. You can use PASSWD and NOPASSWD to specify whether the user has to enter a password or not and you can also use NOEXEC to prevent any programs launching shells themselves (as once a program is running with sudo it has full root privileges so could launch a root shell to circumvent any restrictions in the sudoers file.

For example (using the aliases and users from earlier)

 # This lets the webmasters run all the web commands on the machine 
 # "webserver" provided they give a password
 WEBMASTERS webserver= WEB_CMDS
 # This lets the admins run all the admin commands on the servers
 ADMINS SERVERS= ADMIN_CMDS
 # This lets all the USERS run admin commands on the workstations provided 
 # they give the root password or and admin password (using "sudo -u <username>")
 USERS WORKSTATIONS=(ADMINS) ADMIN_CMDS
 # This lets "harry" shutdown his own machine without a password
 harry harrys-machine= NOPASSWD: SHUTDOWN_CMDS
 # And this lets everybody print without requiring a password
 ALL ALL=(ALL) NOPASSWD: PRINTING_CMDS
The Default Ubuntu Sudoers File

The sudoers file that ships with Ubuntu 8.04 by default is included here so if you break everything you can restore it if needed and also to highlight some key things.

# /etc/sudoers
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the man page for details on how to write a sudoers file.
#

Defaults    env_reset

# Uncomment to allow members of group sudo to not need a password
# %sudo ALL=NOPASSWD: ALL

# Host alias specification

# User alias specification

# Cmnd alias specification

# User privilege specification
root    ALL=(ALL) ALL

# Members of the admin group may gain root privileges
%admin ALL=(ALL) ALL

This is pretty much empty and only has three rules in it. The first ( Defaults env_reset ) resets the terminal environment after switching to root. So, ie: all user set variables are removed. The second ( root ALL=(ALL) ALL ) just lets root do everything on any machine as any user. And the third ( %admin ALL=(ALL) ALL ) lets anybody in the admin group run anything as any user. Note that they will still require a password (thus giving you the normal behaviour you are so used to).

If you want to add your own specifications and you are a member of the admin group then you will need to add them after this line. Otherwise all your changes will be overridden by this line saying you (as part of the admin group) can do anything on any machine as any user provided you give a password.

Common Tasks

This section includes some common tasks and how to accomplish them using the sudoers file.

Shutting Down From The Console Without A Password

Often people want to be able to shut their computers down without requiring a password to do so. This is particularly useful in media PCs where you want to be able to use the shutdown command in the media centre to shutdown the whole computer.

To do this you need to add some cmnd aliases as follows:

Cmnd_Alias SHUTDOWN_CMDS = /sbin/poweroff, /sbin/halt, /sbin/reboot

You also need to add a user specification (at the end of the file after the " %admin ALL = (ALL) ALL " line so it takes effect - see above for details):

<your username> ALL=(ALL) NOPASSWD: SHUTDOWN_CMDS

Obviously you need to replace "<your username>" with the username of the user who needs to be able to shutdown the pc without a password. You can use a user alias here as normal.

Multiple tags on a line

There are times where you need to have both NOPASSWD and NOEXEC or other tags on the same configuration line. The man page for sudoers is less than clear, so here is an example of how this is done:

myuser ALL = (root) NOPASSWD:NOEXEC: /usr/bin/vim

This example lets the user "myuser" run as root the "vim" binary without a password, and without letting vim shell out (the :shell command).

Enabling Visual Feedback when Typing Passwords

As of Ubuntu 10.04 (Lucid), you can enable visual feedback when you are typing a password at a sudo prompt.

Simply edit /etc/sudoers and change the Defaults line to read:

Defaults        env_reset,pwfeedback
Troubleshooting

If your changes don't seem to have had any effect, check that they are not trying to use aliases that are not defined yet and that no other user specifications later in the file are overriding what you are trying to accomplish.

[Nov 09, 2017] TERM strings by Tom Ryder

Jan 26, 2013 | sanctum.geek.nz

A certain piece of very misleading advice is often given online to users having problems with the way certain command-line applications are displaying in their terminals. This is to suggest that the user change the value of their TERM environment variable from within the shell, doing something like this:

$ TERM=xterm-256color

This misinformation sometimes extends to suggesting that users put the forced TERM change into their shell startup scripts. The reason this is such a bad idea is that it forces your shell to assume what your terminal is, and thereby disregards the initial terminal identity string sent by the emulator. This leads to a lot of confusion when one day you need to connect with a very different terminal emulator.

Accounting for differences

All terminal emulators are not created equal. Certainly, not all of them are xterm(1) , although many other terminal emulators do a decent but not comprehensive job of copying it. The value of the TERM environment variable is used by the system running the shell to determine what the terminal connecting to it can and cannot do, what control codes to send to the program to use those features, and how the shell should understand the input of certain key codes, such as the Home and End keys. These things in particular are common causes of frustration for new users who turn out to be using a forced TERM string.

Instead, focus on these two guidelines for setting TERM :

  1. Avoid setting TERM from within the shell, especially in your startup scripts like .bashrc or .bash_profile . If that ever seems like the answer, then you are probably asking the wrong question! The terminal identification string should always be sent by the terminal emulator you are using; if you do need to change it, then change it in the settings for the emulator.
  2. Always use an appropriate TERM string that accurately describes what your choice of terminal emulator can and cannot display. Don't make an rxvt(1) terminal identify itself as xterm ; don't make a linux console identify itself as vt100 ; and don't make an xterm(1) compiled without 256 color support refer to itself as xterm-256color .

In particular, note that sometimes for compatibility reasons, the default terminal identification used by an emulator is given as something generic like xterm , when in fact a more accurate or comprehensive terminal identity file is more than likely available for your particular choice of terminal emulator with a little searching.

An example that surprises a lot of people is the availability of the putty terminal identity file, when the application defaults to presenting itself as an imperfect xterm(1) emulator.

Configuring your emulator's string

Before you change your terminal string in its settings, check whether the default it uses is already the correct one, with one of these:

$ echo $TERM
$ tset -q

Most builds of rxvt(1) , for example, should already use the correct TERM string by default, such as rxvt-unicode-256color for builds with 256 colors and Unicode support.

Where to configure which TERM string your terminal uses will vary depending on the application. For xterm(1) , your .Xresources file should contain a definition like the below:

XTerm*termName: xterm-256color

For rxvt(1) , the syntax is similar:

URxvt*termName: rxvt-unicode-256color

Other GTK and Qt emulators sometimes include the setting somewhere in their preferences. Look for mentions of xterm , a common fallback default.

For Windows PuTTY, it's configurable under the "'Connections > Data"' section:

Setting the terminal string in PuTTY

More detail about configuring PuTTY for connecting to modern systems can be found in my article on configuring PuTTY .

Testing your TERM string

On GNU/Linux systems, an easy way to test the terminal capabilities (particularly effects like colors and reverse video) is using the msgcat(1) utility:

$ msgcat --color=test

This will output a large number of tests of various features to the terminal, so that you can check their appearance is what you expect.

Finding appropriate terminfo(5) definitions

On GNU/Linux systems, the capabilities and behavior of various terminal types is described using terminfo(5) files, usually installed as part of the ncurses package. These files are often installed in /lib/terminfo or /usr/share/terminfo , in subdirectories by first letter.

In order to use a particular TERM string, an appropriate file must exist in one of these directories. On Debian-derived systems, a large collection of terminal types can be installed to the system with the ncurses-term package.

For example, the following variants of the rxvt terminal emulator are all available:

$ cd /usr/share/terminfo/r
$ ls rxvt*
rxvt-16color  rxvt-256color  rxvt-88color  rxvt-color  rxvt-cygwin
rxvt-cygwin-native  rxvt+pcfkeys  rxvt-unicode-256color  rxvt-xpm
Private and custom terminfo(5) files

If you connect to a system that doesn't have a terminfo(5) definition to match the TERM definition for your particular terminal, you might get a message similar to this on login:

setterm: rxvt-unicode-256color: unknown terminal type
tput: unknown terminal "rxvt-unicode-256color"
$

If you're not able to install the appropriate terminal definition system-wide, one technique is to use a private .terminfo directory in your home directory containing the definitions you need:

$ cd ~/.terminfo
$ find
.
./x
./x/xterm-256color
./x/xterm
./r
./r/rxvt-256color
./r/rxvt-unicode-256color
./r/rxvt
./s
./s/screen
./s/screen-256color
./p
./p/putty-256color
./p/putty

You can copy this to your home directory on the servers you manage with a tool like scp :

$ scp -r .terminfo server:
TERM and multiplexers

Terminal multiplexers like screen(1) and tmux(1) are special cases, and they cause perhaps the most confusion to people when inaccurate TERM strings are used. The tmux FAQ even opens by saying that most of the display problems reported by people are due to incorrect TERM settings, and a good portion of the codebase in both multiplexers is dedicated to negotiating the differences between terminal capacities.

This is because they are "terminals within terminals", and provide their own functionality only within the bounds of what the outer terminal can do. In addition to this, they have their own type for terminals within them; both of them use screen and its variants, such as screen-256color .

It's therefore very important to check that both the outer and inner definitions for TERM are correct. In .screenrc it usually suffices to use a line like the following:

term screen

Or in .tmux.conf :

set-option -g default-terminal screen

If the outer terminals you use consistently have 256 color capabilities, you may choose to use the screen-256color variant instead.

If you follow all of these guidelines, your terminal experience will be much smoother, as your terminal and your system will understand each other that much better. You may find that this fixes a lot of struggles with interactive tools like vim(1) , for one thing, because if the application is able to divine things like the available color space directly from terminal information files, it saves you from having to include nasty hacks on the t_Co variable in your .vimrc . Posted in Terminal Tagged term strings , terminal types , terminfo

[Nov 09, 2017] PuTTY configuration by Tom Ryder

Dec 22, 2012 | sanctum.geek.nz

Posted on PuTTY is a terminal emulator with a free software license, including an SSH client. While it has cross-platform ports, it's used most frequently on Windows systems, because they otherwise lack a built-in terminal emulator that interoperates well with Unix-style TTY systems.

While it's very popular and useful, PuTTY's defaults are quite old, and are chosen for compatibility reasons rather than to take advantage of all the features of a more complete terminal emulator. For new users, this is likely an advantage as it can avoid confusion, but more advanced users who need to use a Windows client to connect to a modern GNU/Linux system may find the defaults frustrating, particularly when connecting to a more capable and custom-configured server.

Here are a few of the problems with the default configuration:

All of these things are fixable.

Terminal type

Usually the most important thing in getting a terminal working smoothly is to make sure it identifies itself correctly to the machine to which it's connecting, using an appropriate $TERM string. By default, PuTTY identifies itself as an xterm(1) terminal emulator, which most systems will support.

However, there's a terminfo(5) definition for putty and putty-256color available as part of ncurses , and if you have it available on your system then you should use it, as it slightly more precisely describes the features available to PuTTY as a terminal emulator.

You can check that you have the appropriate terminfo(5) definition installed by looking in /usr/share/terminfo/p :

$ ls -1 /usr/share/terminfo/p/putty*
/usr/share/terminfo/p/putty  
/usr/share/terminfo/p/putty-256color  
/usr/share/terminfo/p/putty-sco  
/usr/share/terminfo/p/putty-vt100

On Debian and Ubuntu systems, these files can be installed with:

# apt-get install ncurses-term

If you can't install the files via your system's package manager, you can also keep a private repository of terminfo(5) files in your home directory, in a directory called .terminfo :

$ ls -1 $HOME/.terminfo/p
putty
putty-256color

Once you have this definition installed, you can instruct PuTTY to identify with that $TERM string in the Connection > Data section:

Correct terminal definition in PuTTY

Here, I've used putty-256color ; if you don't need or want a 256 color terminal you could just use putty .

Once connected, make sure that your $TERM string matches what you specified, and hasn't been mangled by any of your shell or terminal configurations:

$ echo $TERM
putty-256color
Color space

Certain command line applications like Vim and Tmux can take advantage of a full 256 colors in the terminal. If you'd like to use this, set PuTTY's $TERM string to putty-256color as outlined above, and select Allow terminal to use xterm 256-colour mode in Window > Colours

256 colours in PuTTY

You can test this is working by using a 256 color application, or by trying out the terminal colours directly in your shell using tput :

$ for ((color = 0; color <= 255; color++)); do
> tput setaf "$color"
> printf "test"
> done

If you see the word test in many different colors, then things are probably working. Type reset to fix your terminal after this:

$ reset
Using UTF-8

If you're connecting to a modern GNU/Linux system, it's likely that you're using a UTF-8 locale. You can check which one by typing locale . In my case, I'm using the en_NZ locale with UTF-8 character encoding:

$ locale
LANG=en_NZ.UTF-8
LANGUAGE=en_NZ:en
LC_CTYPE="en_NZ.UTF-8"
LC_NUMERIC="en_NZ.UTF-8"
LC_TIME="en_NZ.UTF-8"
LC_COLLATE="en_NZ.UTF-8"
LC_MONETARY="en_NZ.UTF-8"
LC_MESSAGES="en_NZ.UTF-8"
LC_PAPER="en_NZ.UTF-8"
LC_NAME="en_NZ.UTF-8"
LC_ADDRESS="en_NZ.UTF-8"
LC_TELEPHONE="en_NZ.UTF-8"
LC_MEASUREMENT="en_NZ.UTF-8"
LC_IDENTIFICATION="en_NZ.UTF-8"
LC_ALL=

If the output of locale does show you're using a UTF-8 character encoding, then you should configure PuTTY to interpret terminal output using that character set; it can't detect it automatically (which isn't PuTTY's fault; it's a known hard problem). You do this in the Window > Translation section:

Using UTF-8 encoding in PuTTY

While you're in this section, it's best to choose the Use Unicode line drawing code points option as well. Line-drawing characters are most likely to work properly with this setting for UTF-8 locales and modern fonts:

Using Unicode line-drawing points in PuTTY

If Unicode and its various encodings is new to you, I highly recommend Joel Spolsky's classic article about what programmers should know about both.

Fonts

Courier New is a workable monospace font, but modern Windows systems include Consolas , a much nicer terminal font. You can change this in the Window > Appearance section:

Using Consolas font in PuTTY

There's no reason you can't use another favourite Bitmap or TrueType font instead once it's installed on your system; DejaVu Sans Mono , Inconsolata , and Terminus are popular alternatives. I personally favor Ubuntu Mono .

Bells

Terminal bells by default in PuTTY emit the system alert sound. Most people find this annoying; some sort of visual bell tends to be much better if you want to use the bell at all. Configure this in Terminal > Bell

Given the purpose of the alert is to draw attention to the window, I find that using a flashing taskbar icon works well; I use this to draw my attention to my prompt being displayed after a long task completes, or if someone mentions my name or directly messages me in irssi(1) .

Another option is using the Visual bell (flash window) option, but I personally find this even worse than the audible bell.

Default palette

The default colours for PuTTY are rather like those used in xterm(1) , and hence rather harsh, particularly if you're used to the slightly more subdued colorscheme of terminal emulators like gnome-terminal(1) , or have customized your palette to something like Solarized .

If you have decimal RGB values for the colours you'd prefer to use, you can enter those in the Window > Colours section, making sure that Use system colours and Attempt to use logical palettes are unchecked:

There are a few other default annoyances in PuTTY, but the above are the ones that seem to annoy advanced users most frequently. Dag Wieers has a similar post with a few more defaults to fix.

[Nov 04, 2017] Time to move away from HPE Software by Lindsay Hill

Nov 04, 2017 | lkhill.com

Time to move away from HPE Software 15 September 2016 · Filed in Opinion

If you are still using HPE Software, you should actively plan to migrate away. The recent divestiture does not look good to me - I think existing customers are going to get soaked. Plan your migration now.

I've said it before, that I retain a soft spot for Hewlett-Packard. They gave me my first professional job out of university. I served my sentence doing HP OpenView consulting, and HP-UX Administration, but still: it got me started. Once you have some professional experience, it's much easier to move to the next role.

It saddens me to watch HP's ongoing struggles. It's sad to watch a big ship get broken up for parts. But things had to change. They need to do something to adapt to the realities of modern IT demands.

There was one line in the recent announcement about divesting HPE's software assets that stood out to me:

Micro Focus expects to improve the margin on HPE's software assets by approximately 20 percentage points by the end of the third full financial year following the closing of the transaction

Press Release

(Emphasis added).

It has been clear for a while that HP Software was no longer a core asset for HPE. It was clear that it was not adapting, and was being starved for investment. Revenues have seen decline. Smart customers have seen this coming, and have been actively migrating away from HPE Software.

But if you're still using it, you should pay attention to that press release. How do you think Micro Focus plans to improve margins by 20 percentage points? That's a lot of margin. You've got three options:

  1. Increase sales. Software development has high fixed costs, so margin improves with additional sales.
  2. Increase prices, collecting more money from existing customers.
  3. Reduce investment, spending less to improve margins, and hope customers don't notice.

This is a mature business. They will have a low percentage of new customers. Most revenue will be coming from existing customers. It is not a growth market. So what's left? Raise prices, and reduce investment.

If you're an existing customer, expect to see more license audits, and higher renewal quotes. Expect to see feature stagnation.

It won't happen straight away, but it will happen. If you're still delaying that migration, time to get a move on.

[Nov 04, 2017] Has HP Abandoned Operations Manager Lindsay Hill

Nov 04, 2017 | lkhill.com

HP OM has not adapted well to modern demands. It does not deal well with VMs being deployed at a high rate. It does not offer service monitoring capabilities. It does not offer any way to connect to cloud provider APIs. The agents have continued to be unstable. The administrative interface for OML/OMU looks like something I wrote over a weekend, based on a dodgy PHP shopping cart. It does not look like a piece of software that costs tens of thousands of dollars. Or actually maybe it does - Enterprise software in general tends to be ugly. HP didn't even develop it themselves - they licensed the admin interface from Blue Elephant Systems . The Java GUI for OML/OMU was a disgrace in 2002 - and it hasn't changed since.

William , April 13, 2016 10:30 AM

Again, at another site where they are attempting to move to OMi ( BSM ) Just a note here. BSM is the top tier interface through which other products flow. A crude anology would be MicroSoft Office is the suite in which many other products like Access, Outlook, One Note...etc...etc are pieces or parts. OMi is a piece of the comprehensive suite of tools offered by HP Software. Just like "OpenView" was the umbrella word used for all HP tools like OM-W, OM-U, NNM, OV...PI, TA, SI, PM and a host of other products. The jury is still out on whether the products are viable as a management suite. One major consideration is ROI. Problems still exist in ALL the tools, SiS does not provide capabilities or granularity agents have. I could write or borrow scripts ( Perl, Shell, VB, Powershell ) to effectivly do everything it does. OMi loses CI's, does not get critical mesaages forwarded, loses communication with the agents it is supposedly managing for starters, NNMi has issues not finding nodes that it should discover when discovery filters are configured. And I could add a dozen other "dirty diapers" in the suite. Yet, one can see where HP is trying to go here. IF a few of those 400 million in development dollars are thrown at the suite it could prove a valuable suite in any IT departments arsenal.

Hank Williams , October 9, 2013 10:08 AM

Theoretically BSM/OMi looks like an HPOM alternative, but looking at the scalability, the TCO, the complexity (and, and, and ...) it isn't. If you are wary about migrating to BSM, be provocative and ask HP for a reference implementation and analyse the length and cost of the implementation.

William Linn Hank Williams , July 28, 2017 3:45 PM

OMi is a little cleaner, on my last customer site it at least functioned in the 10.12 version. HP couldn't sell BSM with all the integrations like they thought. I personally know of several large enterprises that unceremoniously dumped ALL HP products, like Data Protector, HP-UX when the monitoring tool became an albatross around their necks as far as implementation and complexity of BSM/OMi. So, HP has done what HP always does when they have a major malfunction in marketing. They "REBRANDED"!!!! Seems that is what you see in companies that go out of business in one location, then move to a secondary spot OR, better yet they have a huge "going out of business" sale and the products never get lowered in price, they actually mark them up, if they sell great, if not then they close for a couple of weeks and company A opens as company B with all the same inventory at a marked up price. Maybe not really the scenario HP is using, but close. OMi by itself without the uCMDB ( which causes other issues when reconciliation occurs between agent based CI' and CIT and what is found via the scripts uCMDB uses to collect data, mismatches as one sees it differently and then the CI or CIT is removed, IF a critical system...boom...no monitoring as the policies are gone and there is only a reference to the CI in OMi. ) but...as noted, OMi by itself seems stable, though they are not in version 10.61...and by the way...the patch from 10.60 to 61 is flawed. But aside from the complications of TQL's, RTSM, etc...etc...it looks a whole lot more stable.

Lindsay Hill Hank Williams , October 9, 2013 11:57 PM

You're right - I've seen those implementation plans, and it gets very expensive, very quickly. You have to put in a lot of effort just into getting software installed and integrated - none of which is of any direct value to the customer. Maybe justifiable in huge environments, but for the rest of us? Not really.
Customers shouldn't have to pay for fixing broken integrations, they should be able to just start using the software to solve their business problems. We're years away from reaching that point though.

[Nov 04, 2017] No Dress Code AHAHAHAHAHA by Lindsay Hill

Nov 04, 2017 | lkhill.com

[Nov 01, 2017] Cron best practices by Tom Ryder

May 08, 2016 | sanctum.geek.nz

The time-based job scheduler cron(8) has been around since Version 7 Unix, and its crontab(5) syntax is familiar even for people who don't do much Unix system administration. It's standardised , reasonably flexible, simple to configure, and works reliably, and so it's trusted by both system packages and users to manage many important tasks.

However, like many older Unix tools, cron(8) 's simplicity has a drawback: it relies upon the user to know some detail of how it works, and to correctly implement any other safety checking behaviour around it. Specifically, all it does is try and run the job at an appropriate time, and email the output. For simple and unimportant per-user jobs, that may be just fine, but for more crucial system tasks it's worthwhile to wrap a little extra infrastructure around it and the tasks it calls.

There are a few ways to make the way you use cron(8) more robust if you're in a situation where keeping track of the running job is desirable.

Apply the principle of least privilege

The sixth column of a system crontab(5) file is the username of the user as which the task should run:

0 * * * *  root  cron-task

To the extent that is practical, you should run the task as a user with only the privileges it needs to run, and nothing else. This can sometimes make it worthwhile to create a dedicated system user purely for running scheduled tasks relevant to your application.

0 * * * *  myappcron  cron-task

This is not just for security reasons, although those are good ones; it helps protect you against nasties like scripting errors attempting to remove entire system directories .

Similarly, for tasks with database systems such as MySQL, don't use the administrative root user if you can avoid it; instead, use or even create a dedicated user with a unique random password stored in a locked-down ~/.my.cnf file, with only the needed permissions. For a MySQL backup task, for example, only a few permissions should be required, including SELECT , SHOW VIEW , and LOCK TABLES .

In some cases, of course, you really will need to be root . In particularly sensitive contexts you might even consider using sudo(8) with appropriate NOPASSWD options, to allow the dedicated user to run only the appropriate tasks as root , and nothing else.

Test the tasks

Before placing a task in a crontab(5) file, you should test it on the command line, as the user configured to run the task and with the appropriate environment set. If you're going to run the task as root , use something like su or sudo -i to get a root shell with the user's expected environment first:

$ sudo -i -u cronuser
$ cron-task

Once the task works on the command line, place it in the crontab(5) file with the timing settings modified to run the task a few minutes later, and then watch /var/log/syslog with tail -f to check that the task actually runs without errors, and that the task itself completes properly:

May  7 13:30:01 yourhost CRON[20249]: (you) CMD (cron-task)

This may seem pedantic at first, but it becomes routine very quickly, and it saves a lot of hassles down the line as it's very easy to make an assumption about something in your environment that doesn't actually hold in the one that cron(8) will use. It's also a necessary acid test to make sure that your crontab(5) file is well-formed, as some implementations of cron(8) will refuse to load the entire file if one of the lines is malformed.

If necessary, you can set arbitrary environment variables for the tasks at the top of the file:

MYVAR=myvalue

0 * * * *  you  cron-task
Don't throw away errors or useful output

You've probably seen tutorials on the web where in order to keep the crontab(5) job from sending standard output and/or standard error emails every five minutes, shell redirection operators are included at the end of the job specification to discard both the standard output and standard error. This kluge is particularly common for running web development tasks by automating a request to a URL with curl(1) or wget(1) :

*/5 * * *  root  curl https://example.com/cron.php >/dev/null 2>&1

Ignoring the output completely is generally not a good idea, because unless you have other tasks or monitoring ensuring the job does its work, you won't notice problems (or know what they are), when the job emits output or errors that you actually care about.

In the case of curl(1) , there are just way too many things that could go wrong, that you might notice far too late:

The author has seen all of the above happen, in some cases very frequently.

As a general policy, it's worth taking the time to read the manual page of the task you're calling, and to look for ways to correctly control its output so that it emits only the output you actually want. In the case of curl(1) , for example, I've found the following formula works well:

curl -fLsS -o /dev/null http://example.com/

This way, the curl(1) request should stay silent if everything is well, per the old Unix philosophy Rule of Silence .

You may not agree with some of the choices above; you might think it important to e.g. log the complete output of the returned page, or to fail rather than silently accept a 301 redirect, or you might prefer to use wget(1) . The point is that you take the time to understand in more depth what the called program will actually emit under what circumstances, and make it match your requirements as closely as possible, rather than blindly discarding all the output and (worse) the errors. Work with Murphy's law ; assume that anything that can go wrong eventually will.

Send the output somewhere useful

Another common mistake is failing to set a useful MAILTO at the top of the crontab(5) file, as the specified destination for any output and errors from the tasks. cron(8) uses the system mail implementation to send its messages, and typically, default configurations for mail agents will simply send the message to an mbox file in /var/mail/$USER , that they may not ever read. This defeats much of the point of mailing output and errors.

This is easily dealt with, though; ensure that you can send a message to an address you actually do check from the server, perhaps using mail(1) :

$ printf '%s\n' 'Test message' | mail -s 'Test subject' you@example.com

Once you've verified that your mail agent is correctly configured and that the mail arrives in your inbox, set the address in a MAILTO variable at the top of your file:

MAILTO=you@example.com

0 * * * *    you  cron-task-1
*/5 * * * *  you  cron-task-2

If you don't want to use email for routine output, another method that works is sending the output to syslog with a tool like logger(1) :

0 * * * *   you  cron-task | logger -it cron-task

Alternatively, you can configure aliases on your system to forward system mail destined for you on to an address you check. For Postfix, you'd use an aliases(5) file.

I sometimes use this setup in cases where the task is expected to emit a few lines of output which might be useful for later review, but send stderr output via MAILTO as normal. If you'd rather not use syslog , perhaps because the output is high in volume and/or frequency, you can always set up a log file /var/log/cron-task.log but don't forget to add a logrotate(8) rule for it!

Put the tasks in their own shell script file

Ideally, the commands in your crontab(5) definitions should only be a few words, in one or two commands. If the command is running off the screen, it's likely too long to be in the crontab(5) file, and you should instead put it into its own script. This is a particularly good idea if you want to reliably use features of bash or some other shell besides POSIX/Bourne /bin/sh for your commands, or even a scripting language like Awk or Perl; by default, cron(8) uses the system's /bin/sh implementation for parsing the commands.

Because crontab(5) files don't allow multi-line commands, and have other gotchas like the need to escape percent signs % with backslashes, keeping as much configuration out of the actual crontab(5) file as you can is generally a good idea.

If you're running cron(8) tasks as a non-system user, and can't add scripts into a system bindir like /usr/local/bin , a tidy method is to start your own, and include a reference to it as part of your PATH . I favour ~/.local/bin , and have seen references to ~/bin as well. Save the script in ~/.local/bin/cron-task , make it executable with chmod +x , and include the directory in the PATH environment definition at the top of the file:

PATH=/home/you/.local/bin:/usr/local/bin:/usr/bin:/bin
MAILTO=you@example.com

0 * * * *  you  cron-task

Having your own directory with custom scripts for your own purposes has a host of other benefits, but that's another article

Avoid /etc/crontab

If your implementation of cron(8) supports it, rather than having an /etc/crontab file a mile long, you can put tasks into separate files in /etc/cron.d :

$ ls /etc/cron.d
system-a
system-b
raid-maint

This approach allows you to group the configuration files meaningfully, so that you and other administrators can find the appropriate tasks more easily; it also allows you to make some files editable by some users and not others, and reduces the chance of edit conflicts. Using sudoedit(8) helps here too. Another advantage is that it works better with version control; if I start collecting more than a few of these task files or to update them more often than every few months, I start a Git repository to track them:

$ cd /etc/cron.d
$ sudo git init
$ sudo git add --all
$ sudo git commit -m "First commit"

If you're editing a crontab(5) file for tasks related only to the individual user, use the crontab(1) tool; you can edit your own crontab(5) by typing crontab -e , which will open your $EDITOR to edit a temporary file that will be installed on exit. This will save the files into a dedicated directory, which on my system is /var/spool/cron/crontabs .

On the systems maintained by the author, it's quite normal for /etc/crontab never to change from its packaged template.

Include a timeout

cron(8) will normally allow a task to run indefinitely, so if this is not desirable, you should consider either using options of the program you're calling to implement a timeout, or including one in the script. If there's no option for the command itself, the timeout(1) command wrapper in coreutils is one possible way of implementing this:

0 * * * *  you  timeout 10s cron-task

Greg's wiki has some further suggestions on ways to implement timeouts .

Include file locking to prevent overruns

cron(8) will start a new process regardless of whether its previous runs have completed, so if you wish to avoid locking for long-running task, on GNU/Linux you could use the flock(1) wrapper for the flock(2) system call to set an exclusive lockfile, in order to prevent the task from running more than one instance in parallel.

0 * * * *  you  flock -nx /var/lock/cron-task cron-task

Greg's wiki has some more in-depth discussion of the file locking problem for scripts in a general sense, including important information about the caveats of "rolling your own" when flock(1) is not available.

If it's important that your tasks run in a certain order, consider whether it's necessary to have them in separate tasks at all; it may be easier to guarantee they're run sequentially by collecting them in a single shell script.

Do something useful with exit statuses

If your cron(8) task or commands within its script exit non-zero, it can be useful to run commands that handle the failure appropriately, including cleanup of appropriate resources, and sending information to monitoring tools about the current status of the job. If you're using Nagios Core or one of its derivatives, you could consider using send_nsca to send passive checks reporting the status of jobs to your monitoring server. I've written a simple script called nscaw to do this for me:

0 * * * *  you  nscaw CRON_TASK -- cron-task
Consider alternatives to cron(8)

If your machine isn't always on and your task doesn't need to run at a specific time, but rather needs to run once daily or weekly, you can install anacron and drop scripts into the cron.hourly , cron.daily , cron.monthly , and cron.weekly directories in /etc , as appropriate. Note that on Debian and Ubuntu GNU/Linux systems, the default /etc/crontab contains hooks that run these, but they run only if anacron(8) is not installed.

If you're using cron(8) to poll a directory for changes and run a script if there are such changes, on GNU/Linux you could consider using a daemon based on inotifywait(1) instead.

Finally, if you require more advanced control over when and how your task runs than cron(8) can provide, you could perhaps consider writing a daemon to run on the server consistently and fork processes for its task. This would allow running a task more often than once a minute, as an example. Don't get too bogged down into thinking that cron(8) is your only option for any kind of asynchronous task management!

[Nov 01, 2017] Listing files

www.tecmint.com

Using ls is probably one of the first commands an administrator will learn for getting a simple list of the contents of the directory. Most administrators will also know about the -a and -l switches, to show all files including dot files and to show more detailed data about files in columns, respectively.

There are other switches to GNU ls which are less frequently used, some of which turn out to be very useful for programming:

Since the listing is text like anything else, you could, for example, pipe the output of this command into a vim process, so you could add explanations of what each file is for and save it as an inventory file or add it to a README:

$ ls -XR | vim -

This kind of stuff can even be automated by make with a little work, which I'll cover in another article later in the series.

[Nov 01, 2017] Functions by Tom Ryder

Nov 01, 2017 | sanctum.geek.nz

A more flexible method for defining custom commands for an interactive shell (or within a script) is to use a shell function. We could declare our ll function in a Bash startup file as a function instead of an alias like so:

# Shortcut to call ls(1) with the -l flag
ll() {
    command ls -l "$@"
}

Note the use of the command builtin here to specify that the ll function should invoke the program named ls , and not any function named ls . This is particularly important when writing a function wrapper around a command, to stop an infinite loop where the function calls itself indefinitely:

# Always add -q to invocations of gdb(1)
gdb() {
    command gdb -q "$@"
}

In both examples, note also the use of the "$@" expansion, to add to the final command line any arguments given to the function. We wrap it in double quotes to stop spaces and other shell metacharacters in the arguments causing problems. This means that the ll command will work correctly if you were to pass it further options and/or one or more directories as arguments:

$ ll -a
$ ll ~/.config

Shell functions declared in this way are specified by POSIX for Bourne-style shells, so they should work in your shell of choice, including Bash, dash , Korn shell, and Zsh. They can also be used within scripts, allowing you to abstract away multiple instances of similar commands to improve the clarity of your script, in much the same way the basics of functions work in general-purpose programming languages.

Functions are a good and portable way to approach adding features to your interactive shell; written carefully, they even allow you to port features you might like from other shells into your shell of choice. I'm fond of taking commands I like from Korn shell or Zsh and implementing them in Bash or POSIX shell functions, such as Zsh's vared or its two-argument cd features.

If you end up writing a lot of shell functions, you should consider putting them into separate configuration subfiles to keep your shell's primary startup file from becoming unmanageably large.

Examples from the author

You can take a look at some of the shell functions I have defined here that are useful to me in general shell usage; a lot of these amount to implementing convenience features that I wish my shell had, especially for quick directory navigation, or adding options to commands:

Other examples Variables in shell functions

You can manipulate variables within shell functions, too:

# Print the filename of a path, stripping off its leading path and
# extension
fn() {
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

This works fine, but the catch is that after the function is done, the value for name will still be defined in the shell, and will overwrite whatever was in there previously:

$ printf '%s\n' "$name"
foobar
$ fn /home/you/Task_List.doc
Task_List
$ printf '%s\n' "$name"
Task_List

This may be desirable if you actually want the function to change some aspect of your current shell session, such as managing variables or changing the working directory. If you don't want that, you will probably want to find some means of avoiding name collisions in your variables.

If your function is only for use with a shell that provides the local (Bash) or typeset (Ksh) features, you can declare the variable as local to the function to remove its global scope, to prevent this happening:

# Bash-like
fn() {
    local name
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

# Ksh-like
# Note different syntax for first line
function fn {
    typeset name
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
}

If you're using a shell that lacks these features, or you want to aim for POSIX compatibility, things are a little trickier, since local function variables aren't specified by the standard. One option is to use a subshell , so that the variables are only defined for the duration of the function:

# POSIX; note we're using plain parentheses rather than curly brackets, for
# a subshell
fn() (
    name=$1
    name=${name##*/}
    name=${name%.*}
    printf '%s\n' "$name"
)

# POSIX; alternative approach using command substitution:
fn() {
    printf '%s\n' "$(
        name=$1
        name=${name##*/}
        name=${name%.*}
        printf %s "$name"
    )"
}

This subshell method also allows you to change directory with cd within a function without changing the working directory of the user's interactive shell, or to change shell options with set or Bash options with shopt only temporarily for the purposes of the function.

Another method to deal with variables is to manipulate the positional parameters directly ( $1 , $2 ) with set , since they are local to the function call too:

# POSIX; using positional parameters
fn() {
    set -- "${1##*/}"
    set -- "${1%.*}"
    printf '%s\n' "$1"
}

These methods work well, and can sometimes even be combined, but they're awkward to write, and harder to read than the modern shell versions. If you only need your functions to work with your modern shell, I recommend just using local or typeset . The Bash Guide on Greg's Wiki has a very thorough breakdown of functions in Bash, if you want to read about this and other aspects of functions in more detail.

Keeping functions for later

As you get comfortable with defining and using functions during an interactive session, you might define them in ad-hoc ways on the command line for calling in a loop or some other similar circumstance, just to solve a task in that moment.

As an example, I recently made an ad-hoc function called monit to run a set of commands for its hostname argument that together established different types of monitoring system checks, using an existing script called nmfs :

$ monit() { nmfs "$1" Ping Y ; nmfs "$1" HTTP Y ; nmfs "$1" SNMP Y ; }
$ for host in webhost{1..10} ; do
> monit "$host"
> done

After that task was done, I realized I was likely to use the monit command interactively again, so I decided to keep it. Shell functions only last as long as the current shell, so if you want to make them permanent, you need to store their definitions somewhere in your startup files. If you're using Bash, and you're content to just add things to the end of your ~/.bashrc file, you could just do something like this:

$ declare -f monit >> ~/.bashrc

That would append the existing definition of monit in parseable form to your ~/.bashrc file, and the monit function would then be loaded and available to you for future interactive sessions. Later on, I ended up converting monit into a shell script, as its use wasn't limited to just an interactive shell.

If you want a more robust approach to keeping functions like this for Bash permanently, I wrote a tool called Bashkeep , which allows you to quickly store functions and variables defined in your current shell into separate and appropriately-named files, including viewing and managing the list of names conveniently:

$ keep monit
$ keep
monit
$ ls ~/.bashkeep.d
monit.bash
$ keep -d monit

[Nov 01, 2017] 4 Ways to Watch or Monitor Log Files in Real Time by Matei Cezar

Oct 31, 2017 | www.tecmint.com
How can I see the content of a log file in real time in Linux? Well there are a lot of utilities out there that can help a user to output the content of a file while the file is changing or continuously updating. Some of the most known and heavily used utility to display a file content in real time in Linux is the tail command (manage files effectively).

Read Also : 4 Good Open Source Log Monitoring and Management Tools for Linux

1. tail Command – Monitor Logs in Real Time

As said, tail command is the most common solution to display a log file in real time. However, the command to display the file has two versions, as illustrated in the below examples.

In the first example the command tail needs the -f argument to follow the content of a file.

$ sudo tail -f /var/log/apache2/access.log
Monitor Apache Logs in Real Time

Monitor Apache Logs in Real Time

The second version of the command is actually a command itself: tailf . You won't need to use the -f switch because the command is built-in with the -f argument.

$ sudo tailf /var/log/apache2/access.log
Real Time Apache Logs Monitoring

Real Time Apache Logs Monitoring

Usually, the log files are rotated frequently on a Linux server by the logrotate utility. To watch log files that get rotated on a daily base you can use the -F flag to tail command

Read Also : How to Manage System Logs (Configure, Rotate and Import Into Database) in Linux

The tail -F will keep track if new log file being created and will start following the new file instead of the old file.

$ sudo tail -F /var/log/apache2/access.log

However, by default, tail command will display the last 10 lines of a file. For instance, if you want to watch in real time only the last two lines of the log file, use the -n file combined with the -f flag, as shown in the below example.

$ sudo tail -n2 -f /var/log/apache2/access.log
Watch Last Two Lines of Logs

Watch Last Two Lines of Logs 2. Multitail Command – Monitor Multiple Log Files in Real Time

Another interesting command to display log files in real time is multitail command . The name of the command implies that multitail utility can monitor and keep track of multiple files in real time. Multitail also lets you navigate back and forth in the monitored file.

To install mulitail utility in Debian and RedHat based systems issue the below command.

$ sudo apt install multitail   [On Debian & Ubuntu]
$ sudo yum install multitail   [On RedHat & CentOS]
$ sudo dnf install multitail   [On Fedora 22+ version]

To display the output of two log file simultaneous, execute the command as shown in the below example.

$ sudo multitail /var/log/apache2/access.log /var/log/apache2/error.log