Querying #

Metrics, logs, and traces have different structures and query patterns. VictoriaStack uses signal-specific query languages and integrations, so each database can optimize for the data it stores:

Run queries from Grafana or Perses , from the built-in vmui , or from any HTTP client (see Query APIs ). See how to use multitenancy for read queries .

If there is no data to query yet, start with Ingestion docs .

Querying metrics #

VictoriaMetrics query language MetricsQL is inspired by PromQL . MetricsQL is backward-compatible with PromQL, so Grafana dashboards backed by a Prometheus datasource or alerting/recording rules should work the same after switching from Prometheus to VictoriaMetrics. However, there are some intentional differences between these two languages.

If you’re not familiar with PromQL or MetricsQL, please read the basic concepts of MetricsQL .

On top of PromQL, MetricsQL adds conveniences to improve everyday queries:

• the lookbehind window in square brackets may be omitted, so rate(requests_total) is a valid query that auto-adjusts the lookbehind window based on the real interval between samples and query step;
• series selectors accept multiple or filters, such as {env="prod",job="a" or env="dev",job="b"};
• MetricsQL doesn’t extrapolate rate and increase function results. For example, it returns integer results from increase() over slow-changing integer counters;
• WITH expressions extract repeating parts of a query into named templates.
• About 100+ additional functions are supported. See MetricsQL functions .

For example, the following query returns the request rate per route, and the one below it the 99th percentile of request duration:

      sum(rate({__name__="http.server.request.duration_count"})) by ("http.route")

histogram_quantile(0.99, sum(rate({__name__="http.server.request.duration_bucket"})) by (le, "http.route"))
    

Additionally, VictoriaMetrics implements Graphite API for querying, which makes it a drop-in replacement for Graphite.

Querying logs #

VictoriaLogs query language LogsQL is specifically designed for querying logs and provides the following features:

• Full-text search in any field.
• Ability to combine filters into arbitrary complex logical filters .
• Ability to extract structured fields from unstructured logs at query time via transformations .
• Ability to calculate various stats over the selected log entries via stats .

A LogsQL query starts with filters and optionally uses pipes to process the selected logs. The simplest query matches all logs:

      *
    

Filters combine, and every field of a log entry can be filtered, including numeric comparisons. The following query selects logs with word quote where the context.total field exceeds 20:

      "quote" context.total:>20
    

Pipes allow delimiting actions, applying them one by one:

      "quote" context.total:>20 | stats by (service.name, severity_text) count() as logs | sort by (logs)
    

To get the basic concepts of LogsQL, please read LogsQL tutorial .

Querying traces #

VictoriaTraces stores spans as wide events, so it queries them with LogsQL as well. That makes filtering and analytics over spans work the same way as over logs: filter by any field, extract fields at query time, and calculate stats over the matching spans.

For reading traces, VictoriaTraces also serves Jaeger and Tempo compatible APIs. Explore traces through Grafana or Perses connected via these APIs. Use LogsQL for alerting on traces .

Query APIs #

Every database answers queries over HTTP, which is what the UIs integrations , alerting , and any script or exporter use:

• Metrics: /api/v1/query, /api/v1/query_range and other handlers from Prometheus querying API. Additionally, see Graphite API .
• Logs: /select/logsql/query and other handlers from VictoriaLogs HTTP API, including tailing, statistics calculation, fields exploration, etc.
• Traces: the same LogsQL endpoints as for logs, plus the Jaeger and Tempo APIs .

Read requests select their tenant with the AccountID and ProjectID HTTP headers, the same way in all three databases.