Large number of Nagios alerts after a host comes up from being down.
Hi,
We are running Nagios 4.0.7 and whenever a host goes down (ping results time out) we get an alert that the host is down and nothing else, which is great. However, when the host comes back up, all of the other service checks immediately time out and start sending a massive amount of alerts about each service. Then, as soon as the services come back up, we get another massive amount of alerts stating that the services are recovered.
Is there a way to delay service alerts after a host goes down and comes back up? For instance, a host goes down, we get an alert regarding the down'd host. Host comes up, and we get an alert that the host is up. If the services aren't okay after the host has been recoered for, say, 5 minutes THEN we start to get service alerts. Is this possible?
Thank you
We are running Nagios 4.0.7 and whenever a host goes down (ping results time out) we get an alert that the host is down and nothing else, which is great. However, when the host comes back up, all of the other service checks immediately time out and start sending a massive amount of alerts about each service. Then, as soon as the services come back up, we get another massive amount of alerts stating that the services are recovered.
Is there a way to delay service alerts after a host goes down and comes back up? For instance, a host goes down, we get an alert regarding the down'd host. Host comes up, and we get an alert that the host is up. If the services aren't okay after the host has been recoered for, say, 5 minutes THEN we start to get service alerts. Is this possible?
Thank you
You can also use host a service dependencies so when a host goes down, any hosts or services that are dependencies will suppress their alerts. When the host comes back up the dependencies will follow the same process. If the services goes down on its own it will alert you as configured.
ASKER
Seth, this increase would delay service alerts across the board and not just if a host went down and came back up, correct? My hope was that there was a way to tell service alerts to hold off for a while only if the host went down and came back up. Otherwise we'll be waiting 5 minutes to be alerted if a service just decides to die
Sanga, That's how we have it set now. If a host goes down, the services don't report that they are down, but the problem is when the host comes back up, all of the services are still marked as down so we get a flood of alerts
Thanks for the help
Sanga, That's how we have it set now. If a host goes down, the services don't report that they are down, but the problem is when the host comes back up, all of the services are still marked as down so we get a flood of alerts
Thanks for the help
depends how you have things configured
you might have templates that all hosts follow or some might be customized
is it that important that you need to be notified that soon? do you need service check intervals that short?
as far as dependencies go, services associated with a host are automatically dependent of a host
using dependencies is more for something in between to prevent false positives
for example, a remote site goes down, a system there could be reported down when it isn't. having the gateway/router as dependency will make that system 'unknown' because the parent is down and not the system itself
you might have templates that all hosts follow or some might be customized
is it that important that you need to be notified that soon? do you need service check intervals that short?
as far as dependencies go, services associated with a host are automatically dependent of a host
using dependencies is more for something in between to prevent false positives
for example, a remote site goes down, a system there could be reported down when it isn't. having the gateway/router as dependency will make that system 'unknown' because the parent is down and not the system itself
What Sanga said. Do you have dependencies configured?
ASKER
How do I check to make sure I have dependencies setup/configured?
could you post your configuration file(s) to review?
ASKER
I've posted the configuration file for one of the servers I am monitoring (with details scrubbed). Are there any other files you need me to upload?
there is nothing attached
ASKER CERTIFIED SOLUTION
membership
This solution is only available to members.
To access this solution, you must be a member of Experts Exchange.
ASKER
Found solution
if either of those are too small, the hard alert would be triggered faster causing that to happen