#

Victoria Metrics Alert - executes a list of given MetricsQL expressions (rules) and sends alerts to Alert Manager.

Prerequisites #

Install the follow packages: git, kubectl, helm, helm-docs. See this tutorial.

How to install #

Access a Kubernetes cluster.

Setup chart repository (can be omitted for OCI repositories) #

Add a chart helm repository with follow commands:

              1
2
3
4
                          helm repo add vm https://victoriametrics.github.io/helm-charts/

helm repo update

            

List versions of vm/victoria-metrics-alert chart available to installation:

              1
2
                          helm search repo vm/victoria-metrics-alert -l

            

Install `victoria-metrics-alert` chart #

Export default values of victoria-metrics-alert chart to file values.yaml:

For HTTPS repository

              1
2
                          helm show values vm/victoria-metrics-alert > values.yaml

            

For OCI repository

              1
2
                          helm show values oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-alert > values.yaml

            

Change the values according to the need of the environment in values.yaml file.

Test the installation with command:

For HTTPS repository

              1
2
                          helm install vma vm/victoria-metrics-alert -f values.yaml -n NAMESPACE --debug --dry-run

            

For OCI repository

              1
2
                          helm install vma oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-alert -f values.yaml -n NAMESPACE --debug --dry-run

            

Install chart with command:

For HTTPS repository

              1
2
                          helm install vma vm/victoria-metrics-alert -f values.yaml -n NAMESPACE

            

For OCI repository

              1
2
                          helm install vma oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-alert -f values.yaml -n NAMESPACE

            

Get the pods lists by running this commands:

              1
2
                          kubectl get pods -A | grep 'vma'

            

Get the application by running this command:

              1
2
                          helm list -f vma -n NAMESPACE

            

See the history of versions of vma application with command.

              1
2
                          helm history vma -n NAMESPACE

            

HA configuration for Alertmanager #

There is no option on this chart to set up Alertmanager with HA mode. To enable the HA configuration, you can use:

VictoriaMetrics Operator
official Alertmanager Helm chart

How to uninstall #

Remove application with command.

              1
2
                          helm uninstall vma -n NAMESPACE

            

Documentation of Helm Chart #

Install helm-docs following the instructions on this tutorial.

Generate docs with helm-docs command.

              1
2
3
                          cd charts/victoria-metrics-alert

helm-docs
            

The markdown generation is entirely go template driven. The tool parses metadata from charts and generates a number of sub-templates that can be referenced in a template file (by default README.md.gotmpl). If no template file is provided, the tool has a default internal template that will generate a reasonably formatted README.

Parameters #

The following tables lists the configurable parameters of the chart and their default values.

Change the values according to the need of the environment in victoria-metrics-alert/values.yaml file.

Key	Description
`alertmanager.baseURL: ""`	`(string)` External URL, that alertmanager will expose to receivers
`alertmanager.baseURLPrefix: ""`	`(string)` External URL Prefix, Prefix for the internal routes of web endpoints
`alertmanager.config: global: resolve_timeout: 5m receivers: - name: devnull route: group_by: - alertname group_interval: 10s group_wait: 30s receiver: devnull repeat_interval: 24h`	`(object)` Alertmanager configuration
`alertmanager.configMap: ""`	`(string)` Use existing configmap if specified otherwise .config values will be used
`alertmanager.emptyDir: {}`	`(object)` Empty dir configuration if persistence is disabled for Alertmanager
`alertmanager.enabled: false`	`(bool)` Create alertmanager resources
`alertmanager.envFrom: []`	`(list)` Specify alternative source for env variables
`alertmanager.extraArgs: {}`	`(object)` Extra command line arguments for container of component
`alertmanager.extraContainers: []`	`(list)` Extra containers to run in a pod with alertmanager
`alertmanager.extraHostPathMounts: []`	`(list)` Additional hostPath mounts
`alertmanager.extraVolumeMounts: []`	`(list)` Extra Volume Mounts for the container
`alertmanager.extraVolumes: []`	`(list)` Extra Volumes for the pod
`alertmanager.fullnameOverride: ""`	`(string)` Override Alertmanager resources fullname
`alertmanager.image: registry: "" repository: prom/alertmanager tag: v0.27.0`	`(object)` Alertmanager image configuration
`alertmanager.imagePullSecrets: []`	`(list)` Image pull secrets
`alertmanager.ingress.annotations: {}`	`(object)` Ingress annotations
`alertmanager.ingress.enabled: false`	`(bool)` Enable deployment of ingress for alertmanager component
`alertmanager.ingress.extraLabels: {}`	`(object)` Ingress extra labels
`alertmanager.ingress.hosts: - name: alertmanager.local path: - / port: web`	`(list)` Array of host objects
`alertmanager.ingress.ingressClassName: ""`	`(string)` Ingress controller class name
`alertmanager.ingress.pathType: Prefix`	`(string)` Ingress path type
`alertmanager.ingress.tls: []`	`(list)` Array of TLS objects
`alertmanager.initContainers: []`	`(list)` Additional initContainers to initialize the pod
`alertmanager.listenAddress: 0.0.0.0:9093`	`(string)` Alertmanager listen address
`alertmanager.nodeSelector: {}`	`(object)` Pod’s node selector. Details are here
`alertmanager.persistentVolume.accessModes: - ReadWriteOnce`	`(list)` Array of access modes. Must match those of existing PV or dynamic provisioner. Details are here
`alertmanager.persistentVolume.annotations: {}`	`(object)` Persistent volume annotations
`alertmanager.persistentVolume.enabled: false`	`(bool)` Create/use Persistent Volume Claim for alertmanager component. Empty dir if false
`alertmanager.persistentVolume.existingClaim: ""`	`(string)` Existing Claim name. If defined, PVC must be created manually before volume will be bound
`alertmanager.persistentVolume.mountPath: /data`	`(string)` Mount path. Alertmanager data Persistent Volume mount root path.
`alertmanager.persistentVolume.size: 50Mi`	`(string)` Size of the volume. Better to set the same as resource limit memory property.
`alertmanager.persistentVolume.storageClassName: ""`	`(string)` StorageClass to use for persistent volume. Requires alertmanager.persistentVolume.enabled: true. If defined, PVC created automatically
`alertmanager.persistentVolume.subPath: ""`	`(string)` Mount subpath
`alertmanager.podAnnotations: {}`	`(object)` Alertmanager Pod annotations
`alertmanager.podLabels: {}`	`(object)` Alertmanager Pod labels
`alertmanager.podSecurityContext: enabled: false`	`(object)` Pod’s security context. Details are here
`alertmanager.priorityClassName: ""`	`(string)` Name of Priority Class
`alertmanager.probe.liveness: httpGet: path: '{{ ternary "" .app.baseURLPrefix (empty .app.baseURLPrefix) }}/-/healthy' port: web`	`(object)` Liveness probe
`alertmanager.probe.readiness: httpGet: path: '{{ ternary "" .app.baseURLPrefix (empty .app.baseURLPrefix) }}/-/ready' port: web`	`(object)` Readiness probe
`alertmanager.probe.startup: httpGet: path: '{{ ternary "" .app.baseURLPrefix (empty .app.baseURLPrefix) }}/-/ready' port: web`	`(object)` Startup probe
`alertmanager.resources: {}`	`(object)` Resource object. Details are here
`alertmanager.retention: 120h`	`(string)` Alertmanager retention
`alertmanager.securityContext: enabled: false`	`(object)` Security context to be added to server pods
`alertmanager.service.annotations: {}`	`(object)` Service annotations
`alertmanager.service.clusterIP: ""`	`(string)` Service ClusterIP
`alertmanager.service.externalIPs: []`	`(list)` Service external IPs. Check here for details
`alertmanager.service.externalTrafficPolicy: ""`	`(string)` Service external traffic policy. Check here for details
`alertmanager.service.healthCheckNodePort: ""`	`(string)` Health check node port for a service. Check here for details
`alertmanager.service.ipFamilies: []`	`(list)` List of service IP families. Check here for details.
`alertmanager.service.ipFamilyPolicy: ""`	`(string)` Service IP family policy. Check here for details.
`alertmanager.service.labels: {}`	`(object)` Service labels
`alertmanager.service.loadBalancerIP: ""`	`(string)` Service load balancer IP
`alertmanager.service.loadBalancerSourceRanges: []`	`(list)` Load balancer source range
`alertmanager.service.servicePort: 9093`	`(int)` Service port
`alertmanager.service.type: ClusterIP`	`(string)` Service type
`alertmanager.templates: {}`	`(object)` Alertmanager extra templates
`alertmanager.tolerations: []`	`(list)` Node tolerations for server scheduling to nodes with taints. Details are here
`alertmanager.webConfig: {}`	`(object)` Alertmanager web configuration
`extraObjects: []`	`(list)` Add extra specs dynamically to this chart
`global.cluster.dnsDomain: cluster.local.`	`(string)` K8s cluster domain suffix, uses for building storage pods’ FQDN. Details are here
`global.compatibility: openshift: adaptSecurityContext: auto`	`(object)` Openshift security context compatibility configuration
`global.image.registry: ""`	`(string)` Image registry, that can be shared across multiple helm charts
`global.imagePullSecrets: []`	`(list)` Image pull secrets, that can be shared across multiple helm charts
`license: key: "" secret: key: "" name: ""`	`(object)` Enterprise license key configuration for VictoriaMetrics enterprise. Required only for VictoriaMetrics enterprise. Check docs here, for more information, visit site. Request a trial license here Supported starting from VictoriaMetrics v1.94.0
`license.key: ""`	`(string)` License key
`license.secret: key: "" name: ""`	`(object)` Use existing secret with license key
`license.secret.key: ""`	`(string)` Key in secret with license key
`license.secret.name: ""`	`(string)` Existing secret name
`nameOverride: ""`	`(string)` Override chart name
`server.affinity: {}`	`(object)` Pod affinity
`server.annotations: {}`	`(object)` Annotations to be added to the deployment
`server.config: alerts: groups: []`	`(object)` VMAlert configuration
`server.configMap: ""`	`(string)` VMAlert alert rules configuration. Use existing configmap if specified
`server.datasource: basicAuth: password: "" username: "" bearer: token: "" tokenFile: "" url: ""`	`(object)` VMAlert reads metrics from source, next section represents its configuration. It can be any service which supports MetricsQL or PromQL.
`server.datasource.basicAuth: password: "" username: ""`	`(object)` Basic auth for datasource
`server.datasource.bearer.token: ""`	`(string)` Token with Bearer token. You can use one of token or tokenFile. You don’t need to add “Bearer” prefix string
`server.datasource.bearer.tokenFile: ""`	`(string)` Token Auth file with Bearer token. You can use one of token or tokenFile
`server.env: []`	`(list)` Additional environment variables (ex.: secret tokens, flags). Check here for details.
`server.envFrom: []`	`(list)` Specify alternative source for env variables
`server.extraArgs: envflag.enable: true envflag.prefix: VM_ httpListenAddr: :8880 loggerFormat: json rule: - /config/alert-rules.yaml`	`(object)` Extra command line arguments for container of component
`server.extraContainers: []`	`(list)` Additional containers to run in the same pod
`server.extraHostPathMounts: []`	`(list)` Additional hostPath mounts
`server.extraVolumeMounts: []`	`(list)` Extra Volume Mounts for the container. Expects a lice of volume mounts
`server.extraVolumes: []`	`(list)` Extra Volumes for the pod
`server.fullnameOverride: ""`	`(string)` Override vmalert resources fullname
`server.image: pullPolicy: IfNotPresent registry: "" repository: victoriametrics/vmalert tag: "" variant: ""`	`(object)` VMAlert image configuration
`server.imagePullSecrets: []`	`(list)` Image pull secrets
`server.ingress.annotations: {}`	`(object)` Ingress annotations
`server.ingress.enabled: false`	`(bool)` Enable deployment of ingress for vmalert component
`server.ingress.extraLabels: {}`	`(object)` Ingress extra labels
`server.ingress.hosts: - name: vmalert.local path: - / port: http`	`(list)` Array of host objects
`server.ingress.ingressClassName: ""`	`(string)` Ingress controller class name
`server.ingress.pathType: Prefix`	`(string)` Ingress path type
`server.ingress.tls: []`	`(list)` Array of TLS objects
`server.initContainers: []`	`(list)` Additional initContainers to initialize the pod
`server.labels: {}`	`(object)` Labels to be added to the deployment
`server.minReadySeconds: 0`	`(int)` Specifies the minimum number of seconds for which a newly created Pod should be ready without any of its containers crashing/terminating 0 is the standard k8s default
`server.name: ""`	`(string)` Override default `app` label name
`server.nodeSelector: {}`	`(object)` Pod’s node selector. Details are here
`server.notifier: alertmanager: basicAuth: password: "" username: "" bearer: token: "" tokenFile: "" url: ""`	`(object)` Notifier to use for alerts. Multiple notifiers can be enabled by using `notifiers` section
`server.notifier.alertmanager.basicAuth: password: "" username: ""`	`(object)` Basic auth for alertmanager
`server.notifier.alertmanager.bearer.token: ""`	`(string)` Token with Bearer token. You can use one of token or tokenFile. You don’t need to add “Bearer” prefix string
`server.notifier.alertmanager.bearer.tokenFile: ""`	`(string)` Token Auth file with Bearer token. You can use one of token or tokenFile
`server.notifiers: []`	`(list)` Additional notifiers to use for alerts
`server.podAnnotations: {}`	`(object)` Annotations to be added to pod
`server.podDisruptionBudget: enabled: false labels: {}`	`(object)` See `kubectl explain poddisruptionbudget.spec` for more. Or check docs
`server.podLabels: {}`	`(object)` Pod’s additional labels
`server.podSecurityContext: enabled: true`	`(object)` Pod’s security context. Details are here
`server.priorityClassName: ""`	`(string)` Name of Priority Class
`server.probe.liveness: failureThreshold: 3 initialDelaySeconds: 5 periodSeconds: 15 tcpSocket: {} timeoutSeconds: 5`	`(object)` Liveness probe
`server.probe.readiness: failureThreshold: 3 httpGet: {} initialDelaySeconds: 5 periodSeconds: 15 timeoutSeconds: 5`	`(object)` Readiness probe
`server.probe.startup: {}`	`(object)` Startup probe
`server.remote.read.basicAuth: password: "" username: ""`	`(object)` Basic auth for remote read
`server.remote.read.bearer: token: "" tokenFile: ""`	`(object)` Auth based on Bearer token for remote read
`server.remote.read.bearer.token: ""`	`(string)` Token with Bearer token. You can use one of token or tokenFile. You don’t need to add “Bearer” prefix string
`server.remote.read.bearer.tokenFile: ""`	`(string)` Token Auth file with Bearer token. You can use one of token or tokenFile
`server.remote.read.url: ""`	`(string)` VMAlert remote read URL
`server.remote.write.basicAuth: password: "" username: ""`	`(object)` Basic auth for remote write
`server.remote.write.bearer: token: "" tokenFile: ""`	`(object)` Auth based on Bearer token for remote write
`server.remote.write.bearer.token: ""`	`(string)` Token with Bearer token. You can use one of token or tokenFile. You don’t need to add “Bearer” prefix string
`server.remote.write.bearer.tokenFile: ""`	`(string)` Token Auth file with Bearer token. You can use one of token or tokenFile
`server.remote.write.url: ""`	`(string)` VMAlert remote write URL
`server.replicaCount: 1`	`(int)` Replica count
`server.resources: {}`	`(object)` Resource object. Details are here
`server.securityContext: enabled: true`	`(object)` Security context to be added to server pods
`server.service.annotations: {}`	`(object)` Service annotations
`server.service.clusterIP: ""`	`(string)` Service ClusterIP
`server.service.externalIPs: []`	`(list)` Service external IPs. Check here for details
`server.service.externalTrafficPolicy: ""`	`(string)` Service external traffic policy. Check here for details
`server.service.healthCheckNodePort: ""`	`(string)` Health check node port for a service. Check here for details
`server.service.ipFamilies: []`	`(list)` List of service IP families. Check here for details.
`server.service.ipFamilyPolicy: ""`	`(string)` Service IP family policy. Check here for details.
`server.service.labels: {}`	`(object)` Service labels
`server.service.loadBalancerIP: ""`	`(string)` Service load balancer IP
`server.service.loadBalancerSourceRanges: []`	`(list)` Load balancer source range
`server.service.servicePort: 8880`	`(int)` Service port
`server.service.type: ClusterIP`	`(string)` Service type
`server.strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate`	`(object)` Deployment strategy, set to standard k8s default
`server.tolerations: []`	`(list)` Node tolerations for server scheduling to nodes with taints. Details are here
`server.verticalPodAutoscaler: enabled: false`	`(object)` Vertical Pod Autoscaler
`server.verticalPodAutoscaler.enabled: false`	`(bool)` Use VPA for vmalert
`serviceAccount.annotations: {}`	`(object)` Annotations to add to the service account
`serviceAccount.automountToken: true`	`(bool)` Mount API token to pod directly
`serviceAccount.create: true`	`(bool)` Specifies whether a service account should be created
`serviceAccount.name: null`	`(string)` The name of the service account to use. If not set and create is true, a name is generated using the fullname template
`serviceMonitor.annotations: {}`	`(object)` Service Monitor annotations
`serviceMonitor.basicAuth: {}`	`(object)` Basic auth params for Service Monitor
`serviceMonitor.enabled: false`	`(bool)` Enable deployment of Service Monitor for server component. This is Prometheus operator object
`serviceMonitor.extraLabels: {}`	`(object)` Service Monitor labels
`serviceMonitor.metricRelabelings: []`	`(list)` Service Monitor metricRelabelings
`serviceMonitor.relabelings: []`	`(list)` Service Monitor relabelings

#

Prerequisites #

How to install #

Setup chart repository (can be omitted for OCI repositories) #

Install victoria-metrics-alert chart #

HA configuration for Alertmanager #

How to uninstall #

Documentation of Helm Chart #

Parameters #

Install `victoria-metrics-alert` chart #