|Home||Switchboard||Unix Administration||Red Hat||TCP/IP Networks||Neoliberalism||Toxic Managers|
May the source be with you, but remember the KISS principle ;-)
Bigger doesn't imply better. Bigger often is a sign of obesity, of lost control, of overcomplexity, of cancerous cells
|News||Event Correlation||Recommended Links||Conditions||Policies|
|Correlation Composer||Duplicate Message Suppression||ECS Designer||Humor||Etc|
Note: HP renamed the product called now HP operations manager way too many times. Also it is very inconsistent with using abbreviations. Here we will assume that the term "HP Operations manager" and abbreviations HPOM, OMU, and OVO mean the same thing :-)
Sometimes, you may want to acknowledge messages automatically. The following example illustrates the approach that can be used:
If you would want to the second message to automatically acknowledge the first message. HPOM enables you to automate this type of correlation (for example, an application shutting down and starting up again).
Please note that if you acknowledge messages automatically, just two states can exist in the browser.
In effect, the message browser has become a state-based browser. (For suggestions on how to implement this concept for threshold monitors, see “A Default Message Key and Message Key Relation”)
When you work with related message, the relationship between message that "opens" the situation and message the "closes" it is established through message keys.
A message key acknowledges all previous messages with matching the message key. The pattern used for performing matching is specified with MSGKEYRELATIONS ACK keywords in the policy body.
When a message is acknowledged automatically by another message, both messages are annotated, as follows:
For threshold monitor policies, HPOM can generate a default message key, along with a message key relation, for each condition. To generate a default message key and a message key relation for each condition, use the AUTOMATIC_MSGKEY keyword. The following default values are generated:
This message key indicates that the monitor disk_util has detected more than 90% used disk space in the root directory (/) on the node managed_node.hp.com.
The message key relation ensures that all those messages are acknowledged that are triggered by the monitor disk_util and report that disk utilization of / on the node managed_node.hp.com has exceeded or fallen below a threshold. Sending a Reset Message Automatically
HPOM also automatically acknowledges the last message of a monitored object by sending a reset message if a threshold was first exceeded and the monitored value then drops below all possible reset values. In other words, no condition matches any longer. The reset message is provided with a message-key relation that acknowledges the last message that was sent for a certain monitor. This last message must contain a message key. Otherwise, no reset message is sent.
The reset message cannot be configured, but has the following default text: <monitor_name>[(<instance>)]:<value> (below reset). The reset message does not display in the message browser, but is sent directly to the history database. The severity of the reset message is set to normal. Automatic or operator-initiated actions are not defined.
For details on how the monitor agent behaves when multiple conditions exist for one monitored object, see “Threshold Monitoring with Multiple Conditions” on page 403. Example of an Automatic Reset Message Figure 4-9 shows an example of a reset message sent automatically.
Implementing Message Policies Strategies for Optimal Message Filtering
MONITOR "my_mon" ... MAXTHRESHOLD ... DESCRIPTION "my_mon 1" CONDITION ... THRESHOLD 99.0000 RESET 95.0000 ... SET ... SEVERITY Critical ... MSGKEY "<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<$THRESHOLD>" MSGKEYRELATION ACK "^<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<*>$" DESCRIPTION "my_mon 2" CONDITION ... THRESHOLD 90.0000 RESET 85.0000 ... SET ... SEVERITY Major ... MSGKEY "<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<$THRESHOLD>" MSGKEYRELATION ACK "^<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<*>$" DESCRIPTION "my_mon 3" CONDITION ... THRESHOLD 80.0000 RESET 75.0000 ... SET ... SEVERITY Warning ... MSGKEY "<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<$THRESHOLD>" MSGKEYRELATION ACK "^<$NAME>:<$MSG_NODE_NAME>:<$MSG_OBJECT>:<*>$" Reset Message: monitored value below lowest possible reset value (< 75%).In this example, the my_mon monitor has three conditions:
Feb 28, 2010 | IT Resource Center
I've got an open message interface policy with 5 rules, each with a different threhold of Critical, Major, Minor and Warning and each has a message key of .....
On the Correlation tab in the acknowledge messages with message key field i've entered this .......... <$OPTION(hostname)>:<$MSG_OBJECT>:<*>
In hopes that each time a rule in the policy is matched the new message that is created will acknowledge any messages in the browser that were generated previously by another rule in the policy. This is not working. Instead what i'm seeing is that if the same rule (eg. warning) is matched multiple times the counter is increased (which is fine by me) and then if a different rule is matched (eg. minor) it creates a new message in the message browser leaving the previous warning message. Acknowledgement is not working and i'm not sure why not. Any thoughts?
You have to send the message to the acknowledged browser, not the active browser.
I tried that and its better, except that I don't (obviously) have the lastest alarm in my active broswer. Also, if i use a pretty aggressive polling interval like i'd like to i'm going to get notified via email each time it polls. I was hoping i could suppress and count duplicates of the same severity and then once any change in severity was detected by the policy it would send those to the active message broswer and ack previous messages. Is that not possible?
Hi, use <![$MSG_SEV]> for the message severity clause (instead of '<*>')
This will close any active events with a severity that is different from the 'current' event.
That did the trick. I tried passing that in every which way but that one. Seems so obvious now. Thanks very much!
I spoke too soon. I thought it was working, then went back to do some more testing and it wasn't after all.
Seems like it should be ok. Below is what is in my policy for msg key and ack msg key Message Key: <$OPTION(hostname)>:<$MSG_OBJECT>:<$MSG_SEV> ack message key: <$OPTION(hostname)>:<$MSG_OBJECT>:<![$MSG_SEV]>
This is what i get in the msg properties of the active message browser...... Message Key: lfkndrac01.consolidated.com :DATA:Minor Ack Mesg Key: lfkndrac01.consolidated.com :DATA:<![$MSG_SEV]>
I've got to be overlooking something, I'm just not seeing it. Any ideas?
Should probably add more detail. What is happening is still that if there is, for instance, a minor message in the browser triggered by the policy rule and then another policy rule catches a major violation it creates a major event in the browser an leaves the minor as well instead of ack'ing it.
John, what you see in the keys looks right... We do this all the time. I also ran a test to verify via a test open message interface policy, and, it ran just fine. Did all the things it sounds like you are wanting... Same severity messages incremented the dup count and differing severity messages acknowledged any current messages. If this is OMW, we (frequently) have WMI problems and have to bounce WMI to straighten things out... You may try that and test agin...
Thanks for taking the time to respond again. I'm glad to hear it should work as expected and that i'm on the right track.
This is omw8.10 and a linux 8.65 agent. I bounced wmi but still the same thing. I'm perplexed, I've been staring at this thing for hours and am now at a loss.
I suppose it could still be something in the policy but I did like the wmi thought. Seems like the issue still might be on the mgt sever side. Any other thoughts on some possible troubleshooting i can do to try and figure out what the problem might be?
John, I would simplify first try to isolate the location of the problem.
For instance, set up an open message interface policy that traps on something like app=tstapp, obj=tstobj, and all severities. Then have that policy set the msg key and ack key to something like...
deploy to a given server and, then, from that server (I just did mine on the mgt server), just start issuing opcmsg commands like...
opcmsg a=tstapp o=tstobj msg_t="some text" sev=.... (normal, minor, major, critical for sev).
and see what happens... if it works like you want, then it is (maybe?) the other policy, if not, then it is *maybe?) your server or agent.... Then, just issue an opcmsg command from a commandline somewhere
Dennis, thx for the tips, working great now. Once i tested as you suggested i realized it had to be the policy.
At some point, early on, when testing the policy I couldn't get <$MSG_NODE_NAME> to come through in the message key and couldn't figure out why not. So, in the perl script that makes the opcmsg call I was grabbing the $HOST_NAME=`uname -n`; and passing it to the opcmsg call as an -option and then set the msg key to ............
I switched the message key back to <$MSG_NODE_NAME>:<$MSG_OBJECT>:<$MSG_SEV>
noticed the node name was now being populated in the $MSG_NODE_NAME and things are fine.
I was actually passing a couple of variables using the opcmsg -option, so maybe that contributed to the problem.
I wanted to let you know you were extremely helpfull and I appreciate you taking the time to respond back.
No problem, glad you got it fixed... I remember having some problem in the past with $MSG_NODE_NAME vs. $MSG_NODE before, but, that was long ago and don't remember the details... I see that I am using $MSG_NODE now... so... not sure what the difference is... but, as long as it works... D
Sep 20, 2002 | IT Resource Center forums
I have a message key set with the same message key relation set for a message template. I would expect that new messages arriving from the same node with the same object and severity to acknowledge the old out of the browser but that is not happening. The new message is becoming an annotation of the old message. The key is <$MSG_NODE_NAME>:<$MSG_OBJECT>:<$MSG_SEV>
Did you specify a pattern for message key acknoledgement in the message correlation window of the template definition ? I don't have acces to ITO, it's the field under the message key definition.
To have the result you expect, you have to specify : Message key : <$MSG_NODE_NAME>:<$MSG_OBJECT>:<$MSG_SEV> And in the message key acknoledment pattern : ^<$MSG_NODE_NAME>:<$MSG_OBJECT>:<$MSG_SEV>$
With these settings, any new message arriving will have the message key set, and will acknowledge all previous messages with the same key.
I guess the new message is added as an annotation of the previous message because the suppress and count duplicate messages is enabled on your management server, with the "add duplicates as annotation" checked.
no go with that set up
sounds like you have the duplicate counter set to annotate duplicates. try it with that feature turned off (Actions->Server->Configure)
If that doesn't work, try setting the relationship to ^<$MSG_NODE_NAME>:<$MSG_OBJECT>:<*>$
That's the normal relationship, so that any subsequent message that refers to the same object (not just the same condition, which is what the object and severity together refer to) would be acknowledged automatically.
I haven't seen anyone do what you're trying, which is to use message correlation to ack the same exact message. usually the duplicate counter is enough to keep the browser from filling up because of the same message occurring.
hope this helps, greg
Greg I have tried this and still it is not working. I have two conditions, one for warning and one for minor. the warning comes in and I pump up the FS size to get the minor alarm and it just becomes a counted duplicate even though it is a different severity. Any thoughts.
I think i misunderstood your original message. you don't want the severity in your key, you want the value of the object you're measuring. you should use pattern matching in your message text to get the value into a variable, and then put that variable at the end of your message key.
your key will look like: <$MSG_NODE_NAME>:<$MSG_OBJECT>:<value>
("value" is the variable you create in the main part of the condition)
and your relationship like: <$MSG_NODE_NAME>:<$MSG_OBJECT>:<*>
note that if the exact same value comes in, you'll get a duplicate, but if a different value (even with the same severity) comes in, you'll get the old message ack'd and the new one in the browser.
Greg thanks for that bit of info, it works with only one issue. When an alarm comes in, for example "Warning node1 /tmp space is at 18%". Ten minutes later another comes in, "Minor node1 /tmp space is at 25%". The Minor alarm acknowledges the Warning and continues that way if the % goes up. But when the % goes back down into the Warning level the alarm is surpressed and never comes into the browser. I have tested dropping the % back down into the Normal range and still the higher alarm stays in the browser. Any thoughts?
Jun 22, 2006 | IT Resource Center forums
In that case,
In the template/policy condition for the active message, in the 'message key', enter a key e.g. <node>:<app>:<object>:down.
This if for event that specs something is down.
Then for your next message, the UP, mention in the condition to acknowledge active messages with the above key.
Make sure every key is tuned, to avoid that other messages are ack'd as well.
All variables you can use are in the help pages as well.
BTW : you posted this in NNM thread, better place it in Operations
Groupthink : Two Party System as Polyarchy : Corruption of Regulators : Bureaucracies : Understanding Micromanagers and Control Freaks : Toxic Managers : Harvard Mafia : Diplomatic Communication : Surviving a Bad Performance Review : Insufficient Retirement Funds as Immanent Problem of Neoliberal Regime : PseudoScience : Who Rules America : Neoliberalism : The Iron Law of Oligarchy : Libertarian Philosophy
War and Peace : Skeptical Finance : John Kenneth Galbraith :Talleyrand : Oscar Wilde : Otto Von Bismarck : Keynes : George Carlin : Skeptics : Propaganda : SE quotes : Language Design and Programming Quotes : Random IT-related quotes : Somerset Maugham : Marcus Aurelius : Kurt Vonnegut : Eric Hoffer : Winston Churchill : Napoleon Bonaparte : Ambrose Bierce : Bernard Shaw : Mark Twain Quotes
Vol 25, No.12 (December, 2013) Rational Fools vs. Efficient Crooks The efficient markets hypothesis : Political Skeptic Bulletin, 2013 : Unemployment Bulletin, 2010 : Vol 23, No.10 (October, 2011) An observation about corporate security departments : Slightly Skeptical Euromaydan Chronicles, June 2014 : Greenspan legacy bulletin, 2008 : Vol 25, No.10 (October, 2013) Cryptolocker Trojan (Win32/Crilock.A) : Vol 25, No.08 (August, 2013) Cloud providers as intelligence collection hubs : Financial Humor Bulletin, 2010 : Inequality Bulletin, 2009 : Financial Humor Bulletin, 2008 : Copyleft Problems Bulletin, 2004 : Financial Humor Bulletin, 2011 : Energy Bulletin, 2010 : Malware Protection Bulletin, 2010 : Vol 26, No.1 (January, 2013) Object-Oriented Cult : Political Skeptic Bulletin, 2011 : Vol 23, No.11 (November, 2011) Softpanorama classification of sysadmin horror stories : Vol 25, No.05 (May, 2013) Corporate bullshit as a communication method : Vol 25, No.06 (June, 2013) A Note on the Relationship of Brooks Law and Conway Law
Fifty glorious years (1950-2000): the triumph of the US computer engineering : Donald Knuth : TAoCP and its Influence of Computer Science : Richard Stallman : Linus Torvalds : Larry Wall : John K. Ousterhout : CTSS : Multix OS Unix History : Unix shell history : VI editor : History of pipes concept : Solaris : MS DOS : Programming Languages History : PL/1 : Simula 67 : C : History of GCC development : Scripting Languages : Perl history : OS History : Mail : DNS : SSH : CPU Instruction Sets : SPARC systems 1987-2006 : Norton Commander : Norton Utilities : Norton Ghost : Frontpage history : Malware Defense History : GNU Screen : OSS early history
The Peter Principle : Parkinson Law : 1984 : The Mythical Man-Month : How to Solve It by George Polya : The Art of Computer Programming : The Elements of Programming Style : The Unix Hater’s Handbook : The Jargon file : The True Believer : Programming Pearls : The Good Soldier Svejk : The Power Elite
Most popular humor pages:
Manifest of the Softpanorama IT Slacker Society : Ten Commandments of the IT Slackers Society : Computer Humor Collection : BSD Logo Story : The Cuckoo's Egg : IT Slang : C++ Humor : ARE YOU A BBS ADDICT? : The Perl Purity Test : Object oriented programmers of all nations : Financial Humor : Financial Humor Bulletin, 2008 : Financial Humor Bulletin, 2010 : The Most Comprehensive Collection of Editor-related Humor : Programming Language Humor : Goldman Sachs related humor : Greenspan humor : C Humor : Scripting Humor : Real Programmers Humor : Web Humor : GPL-related Humor : OFM Humor : Politically Incorrect Humor : IDS Humor : "Linux Sucks" Humor : Russian Musical Humor : Best Russian Programmer Humor : Microsoft plans to buy Catholic Church : Richard Stallman Related Humor : Admin Humor : Perl-related Humor : Linus Torvalds Related humor : PseudoScience Related Humor : Networking Humor : Shell Humor : Financial Humor Bulletin, 2011 : Financial Humor Bulletin, 2012 : Financial Humor Bulletin, 2013 : Java Humor : Software Engineering Humor : Sun Solaris Related Humor : Education Humor : IBM Humor : Assembler-related Humor : VIM Humor : Computer Viruses Humor : Bright tomorrow is rescheduled to a day after tomorrow : Classic Computer Humor
The Last but not Least Technology is dominated by two types of people: those who understand what they do not manage and those who manage what they do not understand ~Archibald Putt. Ph.D
Copyright © 1996-2018 by Dr. Nikolai Bezroukov. www.softpanorama.org was initially created as a service to the (now defunct) UN Sustainable Development Networking Programme (SDNP) in the author free time and without any remuneration. This document is an industrial compilation designed and created exclusively for educational use and is distributed under the Softpanorama Content License. Original materials copyright belong to respective owners. Quotes are made for educational purposes only in compliance with the fair use doctrine.
FAIR USE NOTICE This site contains copyrighted material the use of which has not always been specifically authorized by the copyright owner. We are making such material available to advance understanding of computer science, IT technology, economic, scientific, and social issues. We believe this constitutes a 'fair use' of any such copyrighted material as provided by section 107 of the US Copyright Law according to which such material can be distributed without profit exclusively for research and educational purposes.
This is a Spartan WHYFF (We Help You For Free) site written by people for whom English is not a native language. Grammar and spelling errors should be expected. The site contain some broken links as it develops like a living tree...
|You can use PayPal to make a contribution, supporting development of this site and speed up access. In case softpanorama.org is down you can use the at softpanorama.info|
The statements, views and opinions presented on this web page are those of the author (or referenced source) and are not endorsed by, nor do they necessarily reflect, the opinions of the author present and former employers, SDNP or any other organization the author may be associated with. We do not warrant the correctness of the information provided or its fitness for any purpose.
Last modified: September 12, 2017