VictoriaLogs Multilevel #

The VictoriaLogs multilevel Helm chart deploys global read for multiple storage groups.

Prerequisites #

Before installing this chart, ensure your environment meets the following requirements:

Kubernetes cluster - A running Kubernetes cluster with sufficient resources
Helm - Helm package manager installed and configured

Additional requirements depend on your configuration:

Persistent storage - Required if you enable persistent volumes for data retention (enabled by default)
kubectl - Needed for cluster management and troubleshooting

For installation instructions, refer to the official documentation:

Quick start #

The chart deploys VictoriaLogs global read entrypoint for multi-level cluster setup . It deploys the following components according to VictoriaLogs cluster architecture :

vmauth provides balancing across vlselect instances
vlselect provides API for querying the ingested logs

This chart is not self-sufficient, it’s a building block for multi-level cluster setup that configures global read endpoint for multiple VictoriaLogs clusters. Check VictoriaLogs Multi-level component below:

graph TD
    Users["Users/Grafana/vlogscli"] --> VLG
    subgraph VLG["VictoriaLogs Multi-level"]
        AUTH["VMAuth LB"] --> VLQ1["vlselect-1"]
        AUTH["VMAuth LB"] --> VLQ2["vlselect-2"]
    end
       
    VLQ1 --> VL1["VictoriaLogs Cluster-1"]
    VLQ1 --> VL2["VictoriaLogs Cluster-2"]
    VLQ1 --> VL3["VictoriaLogs Cluster-3"]
    VLQ2 --> VL1["VictoriaLogs Cluster-1"]
    VLQ2 --> VL2["VictoriaLogs Cluster-2"]
    VLQ2 --> VL3["VictoriaLogs Cluster-3"]

Chart has the only required parameter .Values.storageNodes, which defines list of all endpoints to query data from. Let’s deploy multilevel chart that points to vlselect-1, vlselect-2 and vlselect-3 nodes. If there no VictoriaLogs cluster configured yet, please check victoria-logs-cluster chart quick start section

      export NAMESPACE=logging

# Install victoria-logs-multicluster chart
helm install vlm vm/victoria-logs-multilevel --namespace $NAMESPACE --wait \
    --set "storageNodes={vlselect-1,vlselect-2,vlselect-n}"
    

To access the data within the cluster, use the vmauth address: http://vlm-victoria-logs-multilevel-vmauth.logging.svc.cluster.local:8427. You can use this address as Data Source URL in Grafana. The vmui interface is available at: http://vlm-victoria-logs-multilevel-vmauth.logging.svc.cluster.local:8427/select/vmui/.

To uninstall these charts, run: helm uninstall vlm collector.

For finer control and easier maintenance, it is recommended to set these values in a separate values file and use it during installation. See how to install victoria-logs-single for an example. You can do this later if you want to configure more settings than shown in the default example.

How to install #

Access a Kubernetes cluster.

Setup chart repository (can be omitted for OCI repositories) #

Add a chart helm repository with follow commands:

      helm repo add vm https://victoriametrics.github.io/helm-charts/

helm repo update

List versions of vm/victoria-logs-multilevel chart available to installation:

      helm search repo vm/victoria-logs-multilevel -l

Install `victoria-logs-multilevel` chart #

Export default values of victoria-logs-multilevel chart to file values.yaml:

For HTTPS repository

      helm show values vm/victoria-logs-multilevel > values.yaml

For OCI repository

      helm show values oci://ghcr.io/victoriametrics/helm-charts/victoria-logs-multilevel > values.yaml

Change the values according to the need of the environment in values.yaml file.

Note

Consider setting .Values.nameOverride to a small value like vlm to avoid hitting resource name limits of 63 characters

Test the installation with command:

For HTTPS repository

      helm install vlm vm/victoria-logs-multilevel -f values.yaml -n NAMESPACE --debug

For OCI repository

      helm install vlm oci://ghcr.io/victoriametrics/helm-charts/victoria-logs-multilevel -f values.yaml -n NAMESPACE --debug

Install chart with command:

For HTTPS repository

      helm install vlm vm/victoria-logs-multilevel -f values.yaml -n NAMESPACE

For OCI repository

      helm install vlm oci://ghcr.io/victoriametrics/helm-charts/victoria-logs-multilevel -f values.yaml -n NAMESPACE

Get the pods lists by running this commands:

      kubectl get pods -A | grep 'vlm'

Get the application by running this command:

      helm list -f vlm -n NAMESPACE

See the history of versions of vlm application with command.

      helm history vlm -n NAMESPACE

How to uninstall #

Remove application with command.

      helm uninstall vlm -n NAMESPACE

Documentation of Helm Chart #

Install helm-docs following the instructions on this tutorial .

Generate docs with helm-docs command.

      cd charts/victoria-logs-multilevel

helm-docs

The markdown generation is entirely go template driven. The tool parses metadata from charts and generates a number of sub-templates that can be referenced in a template file (by default README.md.gotmpl). If no template file is provided, the tool has a default internal template that will generate a reasonably formatted README.

Parameters #

The following tables lists the configurable parameters of the chart and their default values.

Change the values according to the need of the environment in victoria-logs-multilevel/values.yaml file.

Key	Description
`common.image: tag: ""`	`(object)` common for all components image configuration
`extraObjects: []`	`(list)` Add extra specs dynamically to this chart
`global.cluster: dnsDomain: cluster.local.`	`(object)` k8s cluster domain suffix, uses for building storage pods’ FQDN. Details are here
`global.compatibility: openshift: adaptSecurityContext: auto`	`(object)` Openshift security context compatibility configuration
`global.image.registry: ""`	`(string)` Image registry, that can be shared across multiple helm charts
`global.imagePullSecrets: []`	`(list)` Image pull secrets, that can be shared across multiple helm charts
`nameOverride: ""`	`(string)` Override chart name
`printNotes: true`	`(bool)` Print information after deployment
`serviceAccount.annotations: {}`	`(object)` Service account annotations
`serviceAccount.automountToken: true`	`(bool)` mount API token to pod directly
`serviceAccount.create: false`	`(bool)` Specifies whether a service account should be created
`serviceAccount.extraLabels: {}`	`(object)` Service account labels
`serviceAccount.name: null`	`(string)` The name of the service account to use. If not set and create is true, a name is generated using the fullname template
`storageNodes: []`	`(list)` Storage nodes for multilevel setuo
`vlselect.affinity: {}`	`(object)` Pod affinity
`vlselect.annotations: {}`	`(object)` Deployment annotations
`vlselect.containerWorkingDir: ""`	`(string)` Container workdir
`vlselect.enabled: true`	`(bool)` Enable deployment of vlselect component.
`vlselect.env: []`	`(list)` Additional environment variables (ex.: secret tokens, flags). Check here for details.
`vlselect.envFrom: []`	`(list)` Specify alternative source for env variables
`vlselect.extraArgs: envflag.enable: true http.shutdownDelay: 15s httpListenAddr: :9471 loggerFormat: json`	`(object)` Extra command line arguments for vlselect component
`vlselect.extraContainers: []`	`(list)` Extra containers to run in a pod with vlselect
`vlselect.extraLabels: {}`	`(object)` Deployment additional labels
`vlselect.extraVolumeMounts: []`	`(list)` Extra Volume Mounts for the container
`vlselect.extraVolumes: []`	`(list)` Extra Volumes for the pod
`vlselect.fullnameOverride: ""`	`(string)` Overrides the full name of vlselect component
`vlselect.horizontalPodAutoscaler.behavior: {}`	`(object)` Behavior settings for scaling by the HPA
`vlselect.horizontalPodAutoscaler.enabled: false`	`(bool)` Use HPA for vlselect component
`vlselect.horizontalPodAutoscaler.maxReplicas: 10`	`(int)` Maximum replicas for HPA to use to to scale the vlselect component
`vlselect.horizontalPodAutoscaler.metrics: []`	`(list)` Metric for HPA to use to scale the vlselect component
`vlselect.horizontalPodAutoscaler.minReplicas: 2`	`(int)` Minimum replicas for HPA to use to scale the vlselect component
`vlselect.image.pullPolicy: IfNotPresent`	`(string)` Image pull policy
`vlselect.image.registry: ""`	`(string)` Image registry
`vlselect.image.repository: victoriametrics/victoria-logs`	`(string)` Image repository
`vlselect.image.tag: ""`	`(string)` Image tag override Chart.AppVersion
`vlselect.image.variant: ""`	`(string)` Image tag suffix, which is appended to `Chart.AppVersion` if no `server.image.tag` is defined
`vlselect.ingress.annotations: {}`	`(object)` Ingress annotations
`vlselect.ingress.enabled: false`	`(bool)` Enable deployment of ingress for vlselect component
`vlselect.ingress.extraLabels: {}`	`(object)` Ingress extra labels
`vlselect.ingress.hosts: - name: vlselect.local path: - /select port: http`	`(list)` Array of host objects
`vlselect.ingress.ingressClassName: ""`	`(string)` Ingress controller class name
`vlselect.ingress.pathType: Prefix`	`(string)` Ingress path type
`vlselect.ingress.tls: []`	`(list)` Array of TLS objects
`vlselect.initContainers: []`	`(list)` Init containers for vlselect
`vlselect.lifecycle: {}`	`(object)` Specify pod lifecycle
`vlselect.name: ""`	`(string)` Override default `app` label name
`vlselect.nodeSelector: {}`	`(object)` Pod’s node selector. Details are here
`vlselect.podAnnotations: {}`	`(object)` Pod’s annotations
`vlselect.podDisruptionBudget: enabled: false labels: {}`	`(object)` See `kubectl explain poddisruptionbudget.spec` for more. Details are here
`vlselect.podDisruptionBudget.enabled: false`	`(bool)` See `kubectl explain poddisruptionbudget.spec` for more. Details are here
`vlselect.podLabels: {}`	`(object)` Pod’s additional labels
`vlselect.podSecurityContext: enabled: true fsGroup: 1000`	`(object)` Pod’s security context. Details are here
`vlselect.ports.name: http`	`(string)` vlselect http port name
`vlselect.priorityClassName: ""`	`(string)` Name of Priority Class
`vlselect.probe: liveness: failureThreshold: 3 initialDelaySeconds: 5 periodSeconds: 15 tcpSocket: {} timeoutSeconds: 5 readiness: failureThreshold: 3 httpGet: {} initialDelaySeconds: 5 periodSeconds: 5 timeoutSeconds: 5 startup: {}`	`(object)` Readiness & Liveness probes
`vlselect.probe.liveness: failureThreshold: 3 initialDelaySeconds: 5 periodSeconds: 15 tcpSocket: {} timeoutSeconds: 5`	`(object)` vlselect liveness probe
`vlselect.probe.readiness: failureThreshold: 3 httpGet: {} initialDelaySeconds: 5 periodSeconds: 5 timeoutSeconds: 5`	`(object)` vlselect readiness probe
`vlselect.probe.startup: {}`	`(object)` vlselect startup probe
`vlselect.replicaCount: 2`	`(int)` Count of vlselect pods
`vlselect.resources: {}`	`(object)` Resource object. Details are here
`vlselect.route.annotations: {}`	`(object)` HTTPRoute annotations
`vlselect.route.enabled: false`	`(bool)` Enable deployment of HTTPRoute for select component
`vlselect.route.extraRules: []`	`(list)` Extra rules to prepend to route. This is useful when working with annotation based services.
`vlselect.route.filters: []`	`(list)` Filters for a default rule in HTTPRoute
`vlselect.route.hostnames: []`	`(list)` Array of hostnames
`vlselect.route.labels: {}`	`(object)` HTTPRoute extra labels
`vlselect.route.matches: - path: type: PathPrefix value: '{{ dig "extraArgs" "http.pathPrefix" "/select" .Values.vlselect }}'`	`(list)` Matches for a default rule in HTTPRoute
`vlselect.route.parentRefs: []`	`(list)` HTTPGateway objects refs
`vlselect.securityContext: enabled: true runAsGroup: 1000 runAsNonRoot: true runAsUser: 1000`	`(object)` Check here for details.
`vlselect.service.annotations: {}`	`(object)` Service annotations
`vlselect.service.clusterIP: ""`	`(string)` Service ClusterIP
`vlselect.service.enabled: true`	`(bool)` Create vlselect service
`vlselect.service.externalIPs: []`	`(list)` Service external IPs. Details are here
`vlselect.service.externalTrafficPolicy: ""`	`(string)` Service external traffic policy. Check here for details
`vlselect.service.extraPorts: []`	`(list)` Extra service ports
`vlselect.service.healthCheckNodePort: ""`	`(string)` Health check node port for a service. Check here for details
`vlselect.service.ipFamilies: []`	`(list)` List of service IP families. Check here for details.
`vlselect.service.ipFamilyPolicy: ""`	`(string)` Service IP family policy. Check here for details.
`vlselect.service.labels: {}`	`(object)` Service labels
`vlselect.service.loadBalancerIP: ""`	`(string)` Service load balancer IP
`vlselect.service.loadBalancerSourceRanges: []`	`(list)` Load balancer source range
`vlselect.service.servicePort: 9471`	`(int)` Service port
`vlselect.service.targetPort: http`	`(string)` Target port
`vlselect.service.trafficDistribution: ""`	`(string)` Service traffic distribution. Details are here
`vlselect.service.type: ClusterIP`	`(string)` Service type
`vlselect.strategy: {}`	`(object)`
`vlselect.terminationGracePeriodSeconds: 60`	`(int)` Pod’s termination grace period in seconds
`vlselect.tolerations: []`	`(list)` Array of tolerations object. Details are here
`vlselect.topologySpreadConstraints: []`	`(list)` Pod topologySpreadConstraints
`vlselect.vmServiceScrape.annotations: {}`	`(object)` VMServiceScrape annotations
`vlselect.vmServiceScrape.enabled: false`	`(bool)` Enable VMServiceScrape for vlselect component, which is managed by vm-operator.
`vlselect.vmServiceScrape.extraLabels: {}`	`(object)` VMServiceScrape labels
`vlselect.vmServiceScrape.namespace: ""`	`(string)` Target namespace of VMServiceScrape manifest
`vlselect.vmServiceScrape.spec.endpoints[0].port: http`	`(string)`
`vlselect.vmServiceScrape.useServiceMonitor: false`	`(bool)` Create ServiceMonitor instead of VMServiceScrape, which is managed by prometheus-operator
`vmauth.affinity: {}`	`(object)` Pod affinity
`vmauth.annotations: {}`	`(object)` VMAuth annotations
`vmauth.config: {}`	`(object)`
`vmauth.configSecretName: ""`	`(string)` VMAuth configuration secret name
`vmauth.containerWorkingDir: ""`	`(string)` Container workdir
`vmauth.enabled: true`	`(bool)` Enable deployment of vmauth component, to help better balance the read and write requests to vlselect components. With vmauth enabled, please also set `service.clusterIP: None` and `service.type: ClusterIP` for `vlselect`.
`vmauth.env: []`	`(list)` Additional environment variables (ex.: secret tokens, flags). Check here for details
`vmauth.envFrom: []`	`(list)` Specify alternative source for env variables
`vmauth.extraArgs: envflag.enable: true http.shutdownDelay: 15s httpListenAddr: :8427 loggerFormat: json`	`(object)` Extra command line arguments for vmauth component
`vmauth.extraContainers: []`	`(list)` Extra containers to run in a pod with vmauth
`vmauth.extraLabels: {}`	`(object)` VMAuth additional labels
`vmauth.extraVolumeMounts: []`	`(list)` Extra Volume Mounts for the container
`vmauth.extraVolumes: []`	`(list)` Extra Volumes for the pod
`vmauth.fullnameOverride: ""`	`(string)` Overrides the full name of vmauth component
`vmauth.horizontalPodAutoscaler.behavior: {}`	`(object)` Behavior settings for scaling by the HPA
`vmauth.horizontalPodAutoscaler.enabled: false`	`(bool)` Use HPA for vmauth component
`vmauth.horizontalPodAutoscaler.maxReplicas: 10`	`(int)` Maximum replicas for HPA to use to to scale the vmauth component
`vmauth.horizontalPodAutoscaler.metrics: []`	`(list)` Metric for HPA to use to scale the vmauth component
`vmauth.horizontalPodAutoscaler.minReplicas: 2`	`(int)` Minimum replicas for HPA to use to scale the vmauth component
`vmauth.image.pullPolicy: IfNotPresent`	`(string)` Image pull policy
`vmauth.image.registry: ""`	`(string)` Image registry
`vmauth.image.repository: victoriametrics/vmauth`	`(string)` Image repository
`vmauth.image.tag: v1.116.0`	`(string)` Image tag override the default Chart.AppVersion, which is for vlogs components
`vmauth.ingress.annotations: {}`	`(object)` Ingress annotations
`vmauth.ingress.enabled: false`	`(bool)` Enable deployment of ingress for vmauth component
`vmauth.ingress.extraLabels: {}`	`(object)`
`vmauth.ingress.hosts: - name: vmauth.local path: - /insert port: http`	`(list)` Array of host objects
`vmauth.ingress.pathType: Prefix`	`(string)` pathType is only for k8s >= 1.1=
`vmauth.ingress.tls: []`	`(list)` Array of TLS objects
`vmauth.initContainers: []`	`(list)` Init containers for vmauth
`vmauth.lifecycle: {}`	`(object)` Specify pod lifecycle
`vmauth.name: ""`	`(string)` Override default `app` label name
`vmauth.nodeSelector: {}`	`(object)` Pod’s node selector. Details are here
`vmauth.podAnnotations: {}`	`(object)` Pod’s annotations
`vmauth.podDisruptionBudget: enabled: false labels: {}`	`(object)` See `kubectl explain poddisruptionbudget.spec` for more. Details are here
`vmauth.podLabels: {}`	`(object)` VMAuth pod labels
`vmauth.podSecurityContext: enabled: true fsGroup: 1000`	`(object)` Pod’s security context. Details are here
`vmauth.ports.name: http`	`(string)` VMAuth http port name
`vmauth.priorityClassName: ""`	`(string)` Name of Priority Class
`vmauth.probe.liveness: failureThreshold: 3 initialDelaySeconds: 5 periodSeconds: 15 tcpSocket: {} timeoutSeconds: 5`	`(object)` VMAuth liveness probe
`vmauth.probe.readiness: failureThreshold: 3 httpGet: {} initialDelaySeconds: 5 periodSeconds: 5 timeoutSeconds: 5`	`(object)` VMAuth readiness probe
`vmauth.probe.startup: {}`	`(object)` VMAuth startup probe
`vmauth.replicaCount: 2`	`(int)` Count of vmauth pods
`vmauth.resources: {}`	`(object)` Resource object
`vmauth.route.annotations: {}`	`(object)` HTTPRoute annotations
`vmauth.route.enabled: false`	`(bool)` Enable deployment of HTTPRoute for vmauth component
`vmauth.route.extraRules: []`	`(list)` Extra rules to prepend to route. This is useful when working with annotation based services.
`vmauth.route.filters: []`	`(list)` Filters for a default rule in HTTPRoute
`vmauth.route.hostnames: []`	`(list)` Array of hostnames
`vmauth.route.labels: {}`	`(object)` HTTPRoute extra labels
`vmauth.route.matches: - path: type: PathPrefix value: '{{ dig "extraArgs" "http.pathPrefix" "/" .Values.vmauth }}'`	`(list)` Matches for a default rule in HTTPRoute
`vmauth.route.parentRefs: []`	`(list)` HTTPGateway objects refs
`vmauth.securityContext: enabled: true runAsGroup: 1000 runAsNonRoot: true runAsUser: 1000`	`(object)` Check here for details.
`vmauth.service.annotations: {}`	`(object)` Service annotations
`vmauth.service.clusterIP: ""`	`(string)` Service ClusterIP
`vmauth.service.enabled: true`	`(bool)` Create VMAuth service
`vmauth.service.externalIPs: []`	`(list)` Service External IPs. Details are here
`vmauth.service.externalTrafficPolicy: ""`	`(string)` Service external traffic policy. Check here for details
`vmauth.service.extraPorts: []`	`(list)` Extra service ports
`vmauth.service.healthCheckNodePort: ""`	`(string)` Health check node port for a service. Check here for details
`vmauth.service.ipFamilies: []`	`(list)` List of service IP families. Check here for details.
`vmauth.service.ipFamilyPolicy: ""`	`(string)` Service IP family policy. Check here for details.
`vmauth.service.labels: {}`	`(object)` Service labels
`vmauth.service.loadBalancerIP: ""`	`(string)` Service load balancer IP
`vmauth.service.loadBalancerSourceRanges: []`	`(list)` Load balancer source range
`vmauth.service.servicePort: 8427`	`(int)` Service port
`vmauth.service.targetPort: http`	`(string)` Target port
`vmauth.service.trafficDistribution: ""`	`(string)` Service traffic distribution. Details are here
`vmauth.service.type: ClusterIP`	`(string)` Service type
`vmauth.strategy: {}`	`(object)` VMAuth Deployment strategy
`vmauth.tolerations: []`	`(list)` Array of tolerations object. Details are here
`vmauth.topologySpreadConstraints: []`	`(list)` Pod topologySpreadConstraints
`vmauth.vmServiceScrape.annotations: {}`	`(object)` VMServiceScrape annotations
`vmauth.vmServiceScrape.enabled: false`	`(bool)` Enable VMServiceScrape for vmauth component, which is managed by vm-operator.
`vmauth.vmServiceScrape.extraLabels: {}`	`(object)` VMServiceScrape labels
`vmauth.vmServiceScrape.namespace: ""`	`(string)` Target namespace of VMServiceScrape manifest
`vmauth.vmServiceScrape.spec.endpoints[0].port: http`	`(string)`
`vmauth.vmServiceScrape.useServiceMonitor: false`	`(bool)` Create ServiceMonitor instead of VMServiceScrape, which is managed by prometheus-operator

VictoriaLogs Multilevel #

Prerequisites #

Quick start #

How to install #

Setup chart repository (can be omitted for OCI repositories) #

Install victoria-logs-multilevel chart #

How to uninstall #

Documentation of Helm Chart #

Parameters #

Install `victoria-logs-multilevel` chart #