vtagent

vtagent is an agent for accepting trace spans from OTLP exporters and remote write them to VictoriaTraces .

Features #

The main role of vtagent is to replicate collected trace spans among multiple VictoriaTraces instances, without 3rd party collectors.

It also works smoothly in environments with unstable connections to VictoriaTraces instances. If the remote storage is unavailable, the collected trace spans are buffered at the directory specified via -remoteWrite.tmpDataPath command-line flag. The buffered spans are sent to remote storage as soon as the connection to the remote storage is repaired.

vtagent will support the following features under the VictoriaTraces roadmap soon:

  • OTLP output support to remotely write trace spans via the OpenTelemetry protocol.
  • Efficient tail-based sampling with on-disk buffering of trace spans.
  • Retroactive sampling.

When to use vtagent #

Use vtagent when:

  • You need to replicate trace spans to multiple VictoriaTraces instances for high availability.
  • You have unstable connectivity to VictoriaTraces and need on-disk buffering.

Quick Start #

Please download and unpack the vtutils archive from releases page ( vtagent is also available as Docker images on Docker Hub and Quay ), then pass the following command-line flags to the vtagent-prod binary:

  • -remoteWrite.url - the VictoriaTraces endpoint for sending the accepted trace spans to. It must end with /insert/native. The -remoteWrite.url may refer to DNS SRV address. See these docs for details.

Example command, which starts vtagent for accepting trace spans over OTLP/HTTP at the port 10429 and sends the collected trace spans to VictoriaTraces instance at victoria-traces-host:10428:

      /path/to/vtagent-prod -remoteWrite.url=http://victoria-traces-host:10428/insert/native

vtagent can also accept data over OTLP/gRPC at the port specified by -otlpGRPCListenAddr.

Pass -help to vtagent in order to see the full list of supported command-line flags with their descriptions .

Replication and high availability #

vtagent can accept multiple -remoteWrite.url command-line flags. In this case it replicates the collected spans among all the VictoriaTraces instances mentioned in -remoteWrite.url command-line flags.

If some of VictoriaTraces instances are temporarily unavailable, then the collected spans are buffered at the directory specified via -remoteWrite.tmpDataPath command-line flag. The buffered spans are sent to the VictoriaTraces instance as soon as it becomes available. This guarantees the delivery of all the spans across all the VictoriaTraces instances specified via -remoteWrite.url command-line flags.

The on-disk buffer is limited by the available disk space at the -remoteWrite.tmpDataPath by default. It is possible to limit it to the given size via -remoteWrite.maxDiskUsagePerURL command-line flag (this flag can be specified individually per each -remoteWrite.url). When the buffer size reaches this limit for the given -remoteWrite.url, then the oldest spans are dropped from the buffer and are replaced by the newly received spans.

vtagent maintains independent buffers for each -remoteWrite.url, so the collected spans are delivered to the remaining available VictoriaTraces instances in a timely manner when some of the VictoriaTraces instances are unavailable.

Advanced usage #

vtagent can be fine-tuned with various command-line flags. Run ./vtagent -help in order to see the full list of these flags with their descriptions and default values:
Previous Data ingestion Next Querying