Naga Ravi Chaitanya Elluri ed82418799
Documentation: Add etcd database quota alerts (#12249)
This commit:
- Fires a critical alert when the etcd database quota is 95% full
  at any given point of time to alert the user to defrag or increase
  the quota in order to avoid the alarm getting triggered which blocks
  all the writes to etcd meaning there can't be any new objects created.
  This is needed to make sure the cluster supports running large number
  of nodes and objects.
- Fires a warning when there is a sudden surge in etcd writes leading to
  increase in the etcd database quota size at an alarming rate as it
  is disruptive. It might be because of a rougue process and it's
  important to alert the admin.
2020-09-25 11:03:04 -07:00
..

Prometheus Monitoring Mixin for etcd

NOTE: This project is alpha stage. Flags, configuration, behaviour and design may change significantly in following releases.

A set of customisable Prometheus alerts for etcd.

Instructions for use are the same as the kubernetes-mixin.

Background

  • For more information about monitoring mixins, see this design doc.

Testing alerts

Make sure to have jsonnet and gojsontoyaml installed.

First compile the mixin to a YAML file, which the promtool will read:

jsonnet -e '(import "mixin.libsonnet").prometheusAlerts' | gojsontoyaml > mixin.yaml

Then run the unit test:

promtool test rules test.yaml