Every workload region (Earth, Mars, Venus) has a vmagent that sends data to multiple regions with a monitoring setup. The monitoring setup (Ground Control 1,2) contains VictoriaMetrics Time Series Database(TSDB) cluster or single.
Using this schema, you can achieve:
-remoteWrite.urlcommand-line options to
/path/to/vmagent-prod \ -remoteWrite.url=<ground-control-1-remote-write> \ -remoteWrite.url=<ground-control-2-remote-write>
-promscrape.configparameter as well.
Here is a Quickstart guide for vmagent
You can use one of the following options:
The data is duplicated twice, and every region contains a full copy of the data. That means one region can be offline.
You don't need to set up a replication factor using the VictoriaMetrics cluster.
You can set up vmalert in each Ground control region that evaluates recording and alerting rules. As every region contains a full copy of the data, you don't need to synchronize recording rules from one region to another.
For alert deduplication, please use cluster mode in Alertmanager.
We also recommend adopting these alerts:
An additional VictoriaMetrics single can be set up in every region, scraping metrics from the main TSDB.
You also may evaluate the option to send these metrics to the neighbour region to achieve HA.
Setup vmagents in Ground Control regions. That allows it to accept data close to storage and add more reliability if storage is temporarily offline.