Monitoring

There are 2 models to monitor VictoriaMetrics Anomaly Detection behavior - push and pull. Parameters for each of them should be specified in the config file, monitoring section.

Pull Model Config parameters#

Parameter Default Description
addr "0.0.0.0" Server IP Address
port 8080 Port

Push Config parameters#

Parameter Default Description
url Link where to push metrics to. Example: "http://localhost:8480/"
tenant_id Tenant ID for cluster version. Example: "0:0"
health_path "health" Deprecated since v1.8.0. Absolute, to override /health path
user BasicAuth username
password BasicAuth password
timeout "5s" Stop waiting for a response after a given number of seconds.
extra_labels Section for custom labels specified by user.

Monitoring section config example#

monitoring:
  pull: # Enable /metrics endpoint.
    addr: "0.0.0.0"
    port: 8080
  push:
    url: "http://localhost:8480/"
    tenant_id: "0:0" # For cluster version only
    user: "USERNAME"
    password: "PASSWORD"
    timeout: "5s"
    extra_labels:
      job: "vmanomaly-push"
      test: "test-1"

Metrics generated by vmanomaly#

Metric Type Description
vmanomaly_start_time_seconds Gauge vmanomaly start time in UNIX time

Models Behaviour Metrics#

Label names description

Note: There is a new label key model_alias introduced in multi-model support v1.10.0. This label key adjustment was made to preserve unique label set production during writing produced metrics back to VictoriaMetrics.

Metric Type Description Labelnames
vmanomaly_model_runs Counter How many times models ran (per model) stage, query_key, model_alias
vmanomaly_model_run_duration_seconds Summary How much time (in seconds) model invocations took stage, query_key, model_alias
vmanomaly_model_datapoints_accepted Counter How many datapoints did models accept stage, query_key, model_alias
vmanomaly_model_datapoints_produced Counter How many datapoints were generated by models stage, query_key, model_alias
vmanomaly_models_active Gauge How many models are currently inferring query_key
vmanomaly_model_runs_skipped Counter How many times a run was skipped (per model) stage, query_key, model_alias

Writer Behaviour Metrics#

Label names description

Metric Type Description Labelnames
vmanomaly_writer_request_duration_seconds Summary How much time (in seconds) did requests to VictoriaMetrics take url, query_key
vmanomaly_writer_response_count Counter Response code counts we got from VictoriaMetrics url, query_key, code
vmanomaly_writer_sent_bytes Counter How much bytes were sent to VictoriaMetrics url, query_key
vmanomaly_writer_request_serialize_seconds Summary How much time (in seconds) did serializing take query_key
vmanomaly_writer_datapoints_sent Counter How many datapoints were sent to VictoriaMetrics query_key
vmanomaly_writer_timeseries_sent Counter How many timeseries were sent to VictoriaMetrics query_key

Reader Behaviour Metrics#

Label names description

Metric Type Description Labelnames
vmanomaly_reader_request_duration_seconds Summary How much time (in seconds) did queries to VictoriaMetrics take url, query_key
vmanomaly_reader_response_count Counter Response code counts we got from VictoriaMetrics url, query_key, code
vmanomaly_reader_received_bytes Counter How much bytes were received in responses query_key
vmanomaly_reader_response_parsing_seconds Summary How much time (in seconds) did parsing take for each step step
vmanomaly_reader_timeseries_received Counter How many timeseries were received from VictoriaMetrics query_key
vmanomaly_reader_datapoints_received Counter How many rows were received from VictoriaMetrics query_key

Labelnames#

stage - stage of model - ‘fit’, ‘infer’ or ‘fit_infer’ for models that do it simultaneously.

query_key - query alias from reader config section.

model_alias - model alias from models config section. Introduced in v1.10.0.

url - writer or reader url endpoint.

code - response status code or connection_error, timeout.

step - json or dataframe reading step.