Adjust RRD database to provide better resolution

TheFlyingCorpse
TheFlyingCorpse used Ask the Experts™
on
Hi,

I am trying to fix an issue I've come across when it comes to the use of RRD databases for storing the performance data from Nagios. (It processed by a perl script and then used in a web frontend for Nagios)
The data from the same day is accurate, ie the max thats reported back out from the rrdtool is correct.
The issue is that it averages the data over time, ie weeks, months and year.

I've played a bit with creating a new RRD from scratch, but I see an issue in how I can make it scale better.
It appears tho as I've made it scale up to a week with the proper reporting of the graph's, where the code I used to create the RRD is posted below.

What I want is a RRD that lets me have the proper resolution for an entire year, not just the previous week.
The service I am monitoring is checked once a day, every day, so I dont need the "10 minute" scale to this as much as I want to see a proper resolution for the last 2 years, ~730 rows(if its 1 row per check).

Any idea on how I can achieve this? I dont want the "average" except as an overview on what the average actually is, not break the data being displayed in the graph thats output from the RRD.


Thanks!
TheFlyingCorpse
rrdtool create test.rrd \
--step 60 DS:1:GAUGE:86400:0:100000 \
RRA:AVERAGE:0.5:1:2880 \
RRA:MAX:0.5:1:2880 \
RRA:MIN:0.5:1:2880

Open in new window

Comment
Watch Question

Do more with

Expert Office
EXPERT OFFICE® is a registered trademark of EXPERTS EXCHANGE®
nociSoftware Engineer
Distinguished Expert 2018

Commented:
so you create a rrd file, with 60 second intervals, if there is one day of NO samples it will enter the unknown value as a data point.
And valid values are between 0 and 100000 inclusive.

Three storage streams: AVERAGE, MIN, MAX may contain for half of unknown values before considered unknown.
You take 1 datapoint for each step and allow for 2880 datapoints =>
2880 minutes stored = 2 days. (there are no derived datapoints).
Just the moving avarage, min & max.
For year you would need: 365.25*1440 = 525.960
for two years you need: 1.051.920 datapoints...

That would create quite big files.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
To answer your question:
you would replace the 2880 with 1051920 in the rrdcreate call.

I am not quite sure if the system has ever been tested with these kind of values..
it also quite a big graph to draw... 1M horizontal datapoints.
This can easily become giga-pixel resolution.

Author

Commented:
Yeah, I did that, and it seems to work both for the daily and weekly view.

There is however still some wierd stuff going on if I go to the monthly and yearly view, however one should be able to zoom in to the graph when the resolution is set this high, am I not?
(I mean Zoom in as with pnp4Nagios, which can zoom in the graphs)
Success in ‘20 With a Profitable Pricing Strategy

Do you wonder if your IT business is truly profitable or if you should raise your prices? Learn how to calculate your overhead burden using our free interactive tool and use it to determine the right price for your IT services. Start calculating Now!

nociSoftware Engineer
Distinguished Expert 2018

Commented:
With the RRD you have shown (1M datapoints .. then there are no data tracks for MONTH & YEAR.. (Or Week for that matter).

There is only one track....
(A regular entries exerpt)
RRA:MAX:0.5:1:2880
RRA:MAX:0.5:5:2880
RRA:MAX:0.5:30:4320
RRA:MAX:0.5:360:5840

store 2880 data points and store every sample
then also keep 2880 data points but avarage them for every 5  datapoints that have been entered in to the RRD file. Then a 3rd track: every 30 data points is stored, and 4320 are kept, then also a 4th dataset that stores an avaraged datapoint every 360 source datapoints and keeps 5840 values. ==> these 4 are the rules for avaraging out.

As these are the default (as well as show 4 pictures) create the regular graphs...,
In your case you need a specialized graphing routine that selects a range of datapoints from ONE stream only as there are no others.

please checkout 'man rrdgraph' and 'man rrdgraph_examples' for detailed information.
nociSoftware Engineer
Distinguished Expert 2018

Commented:
The zooming option in pnp4nagios (thanks btw, I didn't known that one, I just set it up) is done by selecting a different period from the file.
I need to collect some data to see what is actualy represented.
Obviously you cannot expect to have about 4000 points shown in a say 400 pixel graph.., so values have been compressed there too.

Author

Commented:
yeah, I just wonder how its done so I can reverse it, I only want my 365 points :)
Software Engineer
Distinguished Expert 2018
Commented:

http://linux.die.net/man/1/rrdgraph
http://linux.die.net/man/1/rrdgraph_examples

Just specify a different --start & --end  to change the drawing.

In your case the dsname = 1

Do more with

Expert Office
Submit tech questions to Ask the Experts™ at any time to receive solutions, advice, and new ideas from leading industry professionals.

Start 7-Day Free Trial