Scheduler

Scheduler defines how often to run and make inferences, as well as what timerange to use to train the model. Is specified in scheduler section of a config for VictoriaMetrics Anomaly Detection.

Note: Starting from v1.11.0 scheduler section in config supports multiple schedulers via aliasing.
Also, vmanomaly expects scheduler section to be named schedulers. Using old (flat) format with scheduler key is deprecated and will be removed in future versions.

schedulers:
  scheduler_periodic_1m:
    # class: "scheduler.periodic.PeriodicScheduler"
    infer_every: "1m"
    fit_every: "2m"
    fit_window: "3h"
  scheduler_periodic_5m:
    # class: "scheduler.periodic.PeriodicScheduler"
    infer_every: "5m"
    fit_every: "10m"
    fit_window: "3h"
...

Old-style configs (< 1.11.0)

scheduler:
  # class: "scheduler.periodic.PeriodicScheduler"
  infer_every: "1m"
  fit_every: "2m"
  fit_window: "3h"
...

will be implicitly converted to

schedulers:
  default_scheduler:  # default scheduler alias, backward compatibility
    # class: "scheduler.periodic.PeriodicScheduler"
    infer_every: "1m"
    fit_every: "2m"
    fit_window: "3h"
...

Parameters#

class: str, default="scheduler.periodic.PeriodicScheduler", options={"scheduler.periodic.PeriodicScheduler", "scheduler.oneoff.OneoffScheduler", "scheduler.backtesting.BacktestingScheduler"}

• "scheduler.periodic.PeriodicScheduler": periodically runs the models on new data. Useful for consecutive re-trainings to counter data drift and model degradation over time.
• "scheduler.oneoff.OneoffScheduler": runs the process once and exits. Useful for testing.
• "scheduler.backtesting.BacktestingScheduler": imitates consecutive backtesting runs of OneoffScheduler. Runs the process once and exits. Use to get more granular control over testing on historical data.

Depending on selected class, different parameters should be used

Periodic scheduler#

Parameters#

For periodic scheduler parameters are defined as differences in times, expressed in difference units, e.g. days, hours, minutes, seconds.

Examples: "50s", "4m", "3h", "2d", "1w".

Periodic scheduler config example#

scheduler:
  class: "scheduler.periodic.PeriodicScheduler"
  fit_window: "14d" 
  infer_every: "1m" 
  fit_every: "1h" 

This part of the config means that vmanomaly will calculate the time window of the previous 14 days and use it to train a model. Every hour model will be retrained again on 14 days’ data, which will include + 1 hour of new data. The time window is strictly the same 14 days and doesn’t extend for the next retrains. Every minute vmanomaly will produce model inferences for newly added data points by using the model that is kept in memory at that time.

Oneoff scheduler#

Parameters#

For Oneoff scheduler timeframes can be defined in Unix time in seconds or ISO 8601 string format. ISO format supported time zone offset formats are:

• Z (UTC)
• ±HH:MM
• ±HHMM
• ±HH

If a time zone is omitted, a timezone-naive datetime is used.

Defining fitting timeframe#

Defining inference timeframe#

ISO format scheduler config example#

scheduler:
 class: "scheduler.oneoff.OneoffScheduler"
 fit_start_iso: "2022-04-01T00:00:00Z"
 fit_end_iso: "2022-04-10T00:00:00Z"
 infer_start_iso: "2022-04-11T00:00:00Z"
 infer_end_iso: "2022-04-14T00:00:00Z"

UNIX time format scheduler config example#

scheduler:
 class: "scheduler.oneoff.OneoffScheduler"
 fit_start_iso: 1648771200
 fit_end_iso: 1649548800
 infer_start_iso: 1649635200
 infer_end_iso: 1649894400

Backtesting scheduler#

Parameters#

As for Oneoff scheduler, timeframes can be defined in Unix time in seconds or ISO 8601 string format. ISO format supported time zone offset formats are:

• Z (UTC)
• ±HH:MM
• ±HHMM
• ±HH

If a time zone is omitted, a timezone-naive datetime is used.

Defining overall timeframe#

This timeframe will be used for slicing on intervals (fit_window, infer_window == fit_every), starting from the latest available time point, which is to_* and going back, until no full fit_window + infer_window interval exists within the provided timeframe.

Defining training timeframe#

The same explicit logic as in Periodic scheduler

Defining inference timeframe#

In BacktestingScheduler, the inference window is implicitly defined as a period between 2 consecutive model fit_every runs. The latest inference window starts from to_s - fit_every and ends on the latest available time point, which is to_s. The previous periods for fit/infer are defined the same way, by shifting fit_every seconds backwards until we get the last full fit period of fit_window size, which start is >= from_s.

ISO format scheduler config example#

scheduler:
 class: "scheduler.backtesting.BacktestingScheduler"
 from_start_iso: '2021-01-01T00:00:00Z'
 to_end_iso: '2021-01-14T00:00:00Z'
 fit_window: 'P14D'
 fit_every: 'PT1H'

UNIX time format scheduler config example#

scheduler:
 class: "scheduler.backtesting.BacktestingScheduler"
 from_start_s: 167253120
 to_end_s: 167443200
 fit_window: '14d'
 fit_every: '1h'