Victoria Metrics Operator
Prerequisites #
- Install the follow packages:
git
,kubectl
,helm
,helm-docs
. See this tutorial. - PV support on underlying infrastructure.
ArgoCD issues #
When running operator using ArgoCD without Cert Manager (.Values.admissionWebhooks.certManager.enabled: false
) it will rerender webhook certificates
on each sync since Helm lookup
function is not respected by ArgoCD. To prevent this please update you operator Application spec.syncPolicy
and spec.ignoreDifferences
with a following:
apiVersion: argoproj.io/v1alpha1
kind: Application
...
spec:
...
syncPolicy:
syncOptions:
# https://argo-cd.readthedocs.io/en/stable/user-guide/sync-options/#respect-ignore-difference-configs
# argocd must also ignore difference during apply stage
# otherwise it ll silently override changes and cause a problem
- RespectIgnoreDifferences=true
ignoreDifferences:
- group: ""
kind: Secret
name: <fullname>-validation
namespace: kube-system
jsonPointers:
- /data
- group: admissionregistration.k8s.io
kind: ValidatingWebhookConfiguration
name: <fullname>-admission
jqPathExpressions:
- '.webhooks[]?.clientConfig.caBundle'
where <fullname>
is output of {{ include "vm-operator.fullname" }}
for your setup
Upgrade guide #
During release an issue with helm CRD was discovered. So for upgrade from version less then 0.1.3 you have to two options:
- use helm management for CRD, enabled by default.
- use own management system, need to add variable: –set createCRD=false.
If you choose helm management, following steps must be done before upgrade:
- define namespace and helm release name variables
export NAMESPACE=default
export RELEASE_NAME=operator
execute kubectl commands:
kubectl get crd | grep victoriametrics.com | awk '{print $1 }' | xargs -i kubectl label crd {} app.kubernetes.io/managed-by=Helm --overwrite
kubectl get crd | grep victoriametrics.com | awk '{print $1 }' | xargs -i kubectl annotate crd {} meta.helm.sh/release-namespace="$NAMESPACE" meta.helm.sh/release-name="$RELEASE_NAME" --overwrite
run helm upgrade command.
Chart Details #
This chart will do the following:
- Rollout victoria metrics operator
How to install #
Access a Kubernetes cluster.
Setup chart repository (can be omitted for OCI repositories) #
Add a chart helm repository with follow commands:
helm repo add vm https://victoriametrics.github.io/helm-charts/
helm repo update
List versions of vm/victoria-metrics-operator
chart available to installation:
helm search repo vm/victoria-metrics-operator -l
Install victoria-metrics-operator
chart
#
Export default values of victoria-metrics-operator
chart to file values.yaml
:
For HTTPS repository
helm show values vm/victoria-metrics-operator > values.yaml
For OCI repository
helm show values oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator > values.yaml
Change the values according to the need of the environment in values.yaml
file.
Test the installation with command:
For HTTPS repository
helm install vmo vm/victoria-metrics-operator -f values.yaml -n NAMESPACE --debug --dry-run
For OCI repository
helm install vmo oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator -f values.yaml -n NAMESPACE --debug --dry-run
Install chart with command:
For HTTPS repository
helm install vmo vm/victoria-metrics-operator -f values.yaml -n NAMESPACE
For OCI repository
helm install vmo oci://ghcr.io/victoriametrics/helm-charts/victoria-metrics-operator -f values.yaml -n NAMESPACE
Get the pods lists by running this commands:
kubectl get pods -A | grep 'vmo'
Get the application by running this command:
helm list -f vmo -n NAMESPACE
See the history of versions of vmo
application with command.
helm history vmo -n NAMESPACE
Validation webhook #
Its possible to use validation of created resources with operator. For now, you need cert-manager to easily certificate management https://cert-manager.io/docs/
admissionWebhooks:
enabled: true
# what to do in case, when operator not available to validate request.
certManager:
# enables cert creation and injection by cert-manager
enabled: true
How to uninstall #
Remove application with command.
helm uninstall vmo -n NAMESPACE
Documentation of Helm Chart #
Install helm-docs
following the instructions on this tutorial.
Generate docs with helm-docs
command.
cd charts/victoria-metrics-operator
helm-docs
The markdown generation is entirely go template driven. The tool parses metadata from charts and generates a number of sub-templates that can be referenced in a template file (by default README.md.gotmpl
). If no template file is provided, the tool has a default internal template that will generate a reasonably formatted README.
Parameters #
The following tables lists the configurable parameters of the chart and their default values.
Change the values according to the need of the environment in victoria-metrics-operator/values.yaml
file.
Key | Type | Default | Description |
---|---|---|---|
admissionWebhooks | object |
certManager:
enabled: false
issuer: {}
enabled: true
enabledCRDValidation:
vlogs: true
vmagent: true
vmalert: true
vmalertmanager: true
vmalertmanagerconfig: true
vmauth: true
vmcluster: true
vmrule: true
vmsingle: true
vmuser: true
keepTLSSecret: true
policy: Fail
tls:
caCert: null
cert: null
key: null
| Configures resource validation |
admissionWebhooks.certManager | object |
enabled: false
issuer: {}
| with keys: tls.key, tls.crt, ca.crt |
admissionWebhooks.certManager.enabled | bool | false | Enables cert creation and injection by cert-manager. |
admissionWebhooks.certManager.issuer | object |
{}
| If needed, provide own issuer. Operator will create self-signed if empty. |
admissionWebhooks.enabled | bool | true | Enables validation webhook. |
admissionWebhooks.policy | string | Fail | What to do in case, when operator not available to validate request. |
affinity | object |
{}
| Pod affinity |
annotations | object |
{}
| Annotations to be added to the all resources |
crd.cleanup.enabled | bool | false | Tells helm to clean up all the vm resources under this release’s namespace when uninstalling |
crd.cleanup.image | object |
pullPolicy: IfNotPresent
repository: bitnami/kubectl
tag: ""
| Image configuration for CRD cleanup Job |
crd.create | bool | true | with this option, if you remove this chart, all crd resources will be deleted with it. |
env | list |
[]
| extra settings for the operator deployment. Full list here |
envFrom | list |
[]
| |
extraArgs | object |
{}
| operator container additional commandline arguments |
extraContainers | list |
[]
| |
extraHostPathMounts | list |
[]
| Additional hostPath mounts |
extraLabels | object |
{}
| Labels to be added to the all resources |
extraObjects | list |
[]
| Add extra specs dynamically to this chart |
extraVolumeMounts | list |
[]
| Extra Volume Mounts for the container |
extraVolumes | list |
[]
| Extra Volumes for the pod |
fullnameOverride | string | "" | Overrides the full name of server component |
global.cluster.dnsDomain | string | cluster.local | |
global.image.registry | string | "" | |
global.imagePullSecrets | list |
[]
| |
image | object |
pullPolicy: IfNotPresent
registry: ""
repository: victoriametrics/operator
tag: ""
variant: ""
| operator image configuration |
image.pullPolicy | string | IfNotPresent | Image pull policy |
image.registry | string | "" | Image registry |
image.repository | string | victoriametrics/operator | Image repository |
image.tag | string | "" | Image tag override Chart.AppVersion |
imagePullSecrets | list |
[]
| Secret to pull images |
logLevel | string | info | possible values: info and error. |
nameOverride | string | "" | VM operatror deployment name override |
nodeSelector | object |
{}
| Pod’s node selector. Details are here |
operator.disable_prometheus_converter | bool | false | By default, operator converts prometheus-operator objects. |
operator.enable_converter_ownership | bool | false | Enables ownership reference for converted prometheus-operator objects, it will remove corresponding victoria-metrics objects in case of deletion prometheus one. |
operator.prometheus_converter_add_argocd_ignore_annotations | bool | false | Compare-options and sync-options for prometheus objects converted by operator for properly use with ArgoCD |
operator.useCustomConfigReloader | bool | false | Enables custom config-reloader, bundled with operator. It should reduce vmagent and vmauth config sync-time and make it predictable. |
podDisruptionBudget | object |
enabled: false
labels: {}
| See |
podLabels | object |
{}
| |
podSecurityContext | object |
{}
| |
probe.liveness | object |
failureThreshold: 3
initialDelaySeconds: 5
periodSeconds: 15
tcpSocket:
port: probe
timeoutSeconds: 5
| Liveness probe |
probe.readiness | object |
failureThreshold: 3
httpGet:
port: probe
initialDelaySeconds: 5
periodSeconds: 15
timeoutSeconds: 5
| Readiness probe |
probe.startup | object |
{}
| Startup probe |
rbac.aggregatedClusterRoles | object |
enabled: true
labels:
admin:
rbac.authorization.k8s.io/aggregate-to-admin: "true"
view:
rbac.authorization.k8s.io/aggregate-to-view: "true"
| create aggregated clusterRoles for CRD readonly and admin permissions |
rbac.aggregatedClusterRoles.labels | object |
admin:
rbac.authorization.k8s.io/aggregate-to-admin: "true"
view:
rbac.authorization.k8s.io/aggregate-to-view: "true"
| labels attached to according clusterRole |
rbac.create | bool | true | Specifies whether the RBAC resources should be created |
replicaCount | int | 1 | |
resources | object |
{}
| Resource object |
securityContext | object |
{}
| |
service.annotations | object |
{}
| |
service.clusterIP | string | "" | |
service.externalIPs | string | "" | |
service.externalTrafficPolicy | string | "" | |
service.healthCheckNodePort | string | "" | |
service.ipFamilies | list |
[]
| |
service.ipFamilyPolicy | string | "" | |
service.labels | object |
{}
| |
service.loadBalancerIP | string | "" | |
service.loadBalancerSourceRanges | list |
[]
| |
service.servicePort | int | 8080 | |
service.type | string | ClusterIP | |
service.webhookPort | int | 9443 | |
serviceAccount.create | bool | true | Specifies whether a service account should be created |
serviceAccount.name | string | "" | The name of the service account to use. If not set and create is true, a name is generated using the fullname template |
serviceMonitor | object |
annotations: {}
basicAuth: {}
enabled: false
extraLabels: {}
interval: ""
relabelings: []
scheme: ""
scrapeTimeout: ""
tlsConfig: {}
| configures monitoring with serviceScrape. VMServiceScrape must be pre-installed |
tolerations | list |
[]
| Array of tolerations object. Spec is here |
topologySpreadConstraints | list |
[]
| Pod Topology Spread Constraints. Spec is here |
watchNamespaces | list |
[]
| By default, the operator will watch all the namespaces If you want to override this behavior, specify the namespace. Operator supports multiple namespaces for watching. |