Release process guidance

Pre-reqs#

  1. Make sure you have enterprise remote configured
git remote add enterprise <url>
  1. Make sure you have singing key configured
  2. Make sure you have github token with at least read:org, repo, write:packages permissions exported under GITHUB_TOKEN env variable. You can create token here
  3. Make sure you’re authorized for pushing docker images

For MacOS users#

Make sure you have GNU version of utilities zip, tar, sha256sum. To install them run the following commands:

brew install coreutils
brew install gnu-tar
export PATH="/usr/local/opt/coreutils/libexec/gnubin:$PATH"

Docker may need additional configuration changes:

docker buildx create --use --name=qemu
docker buildx inspect --bootstrap  

By default, docker on MacOS has limited amount of resources (CPU, mem) to use. Bumping the limits may significantly improve build speed.

Release version and Docker images#

  1. Make sure all the changes are documented in CHANGELOG.md. Ideally, every change must be documented in the commit with the change. Alternatively, the change must be documented immediately after the commit, which adds the change.
  2. Make sure all the changes are synced between master, cluster, enterprise-single-node and enterprise-cluster branches. Changes in these branches must be synced immediately after they are committed in at least a single branch.
  3. Make sure that the release branches have no security issues.
  4. Update release versions if needed in SECURITY.md.
  5. Add (available starting from v1.xx.y) line to feature docs introduced in the upcoming release.
  6. Cut new version in CHANGELOG.md and make it merged. See example in this commit.
  7. Cherry-pick bug fixes relevant for LTS releases.
  8. Make sure you get all changes fetched git fetch --all.
  9. Create the following release tags:
    • git tag -s v1.xx.y in master branch
    • git tag -s v1.xx.y-cluster in cluster branch
    • git tag -s v1.xx.y-enterprise in enterprise-single-node branch
    • git tag -s v1.xx.y-enterprise-cluster in enterprise-cluster branch
  10. Run TAG=v1.xx.y make publish-release. This command performs the following tasks: a) Build and package binaries in *.tar.gz release archives with the corresponding _checksums.txt files inside bin directory. This step can be run manually with the command make release from the needed git tag. b) Build and publish multi-platform Docker images for the given TAG, TAG-cluster, TAG-enterprise and TAG-enterprise-cluster. The multi-platform Docker image is built for the following platforms:
    • linux/amd64
    • linux/arm64
    • linux/arm
    • linux/ppc64le
    • linux/386 This step can be run manually with the command make publish from the needed git tag.
  11. Push the tags v1.xx.y and v1.xx.y-cluster created at step 8 to public GitHub repository at https://github.com/VictoriaMetrics/VictoriaMetrics. Push the tags v1.xx.y, v1.xx.y-cluster, v1.xx.y-enterprise and v1.xx.y-enterprise-cluster to the corresponding branches in private repository. Important note: do not push enterprise tags to public GitHub repository - they must be pushed only to private repository.
  12. Run TAG=v1.xx.y make github-create-release github-upload-assets. This command performs the following tasks: a) Create draft GitHub release with the name TAG. This step can be run manually with the command TAG=v1.xx.y make github-create-release. The release id is stored at /tmp/vm-github-release file. b) Upload all the binaries and checksums created at step 9a to that release. This step can be run manually with the command make github-upload-assets. It is expected that the needed release id is stored at /tmp/vm-github-release file, which must be created at the step a. If the upload process is interrupted by any reason, then the following recovery steps must be performed:
    • To delete the created draft release by running the command make github-delete-release. This command expects that the id of the release to delete is located at /tmp/vm-github-release file created at the step a.
    • To run the command TAG=v1.xx.y make github-create-release github-upload-assets, so new release is created and all the needed assets are re-uploaded to it.
  13. Go to https://github.com/VictoriaMetrics/VictoriaMetrics/releases and verify that draft release with the name TAG has been created and this release contains all the needed binaries and checksums.
  14. Update the release description with the content of CHANGELOG for this release.
  15. Publish release by pressing “Publish release” green button in GitHub’s UI.
  16. Bump version of the VictoriaMetrics cluster in the sandbox environment by opening and merging PR.
  17. Bump VictoriaMetrics version at deployment/docker/docker-compose.yml and at deployment/docker/docker-compose-cluster.yml.
  18. Follow the instructions in release follow-up.

Public Announcement#

Operator#

The operator repository https://github.com/VictoriaMetrics/operator/

Bump the version of images#

  • Bump Version field in file internal/config/config.go with new release version for:
    • vmalert in BaseOperatorConf.VMAlertDefault.Version,
    • vmagent in BaseOperatorConf.VMAgentDefault.Version,
    • vmsingle in BaseOperatorConf.VMSingleDefault.Version,
    • vmselect in BaseOperatorConf.VMClusterDefault.VMSelectDefault.Version,
    • vmstorage in BaseOperatorConf.VMClusterDefault.VMStorageDefault.Version,
    • vminsert in BaseOperatorConf.VMClusterDefault.VMInsertDefault.Version,
    • vmbackupmanager in BaseOperatorConf.VMBackup.Version (should be enterprise version),
    • vmauth in BaseOperatorConf.VMAuthDefault.Version.
  • Run make operator-conf.
  • Rename “Next release” section in docs/CHANGELOG.md to the new release version and create new empty “Next release” section.
  • Commit and push changes to master.
  • Create and push a new tag with the new release version.
  • Create github release from this tag with “Release notes” from docs/CHANGELOG.md for this version in description.

Helm Charts#

The helm chart repository https://github.com/VictoriaMetrics/helm-charts/

Bump the version of images#

Bump tag field in values.yaml with new release version. Bump appVersion field in Chart.yaml with new release version. Add new line to “Next release” section in CHANGELOG.md about version update (the line must always start with “-”). Do NOT change headers in CHANGELOG.md. Bump version field in Chart.yaml with incremental semver version (based on the CHANGELOG.md analysis).

Do these updates to the following charts:

  1. Update vmagent chart version in values.yaml and Chart.yaml
  2. Update vmalert chart version in values.yaml and Chart.yaml
  3. Update vmauth chart version in values.yaml and Chart.yaml
  4. Update cluster chart versions in values.yaml, bump version for vmselect, vminsert and vmstorage and Chart.yaml
  5. Update k8s-stack chart versions in values.yaml, bump version for vmselect, vminsert, vmstorage, vmsingle, vmalert, vmagent and Chart.yaml
  6. Update single-node chart version in values.yaml and Chart.yaml
  7. Update vmgateway chart version in values.yaml and Chart.yaml

Once updated, run the following commands:

  1. Commit and push changes to master.
  2. Run “Release” action on Github: release helm charts
  3. Merge new PRs “Automatic update CHANGELOGs and READMEs” and “Synchronize docs” after pipelines are complete.

Ansible Roles#

Bump the version of images#

Repository https://github.com/VictoriaMetrics/ansible-playbooks

  1. Update vmagent version in main.yml
  2. Update vmalert version in main.yml
  3. Update cluster version in main.yml
  4. Update single version in main.yml
  5. Commit changes
  6. Create a new tag
  7. Create a new release. This automatically publishes the new versions to galaxy.ansible.com

RPM packages#

Bump the version of components#

Repository https://github.com/VictoriaMetrics/victoriametrics-lts-rpm

  1. Update vmagent version in vmagent.spec
  2. Update vmalert version in vmalert.spec
  3. Update vmauth version in vmauth.spec
  4. Update vmbackup version in vmbackup.spec
  5. Update vmctl version in vmctl.spec
  6. Update vmrestore version in vmrestore.spec
  7. Update vminsert version in vminsert.spec
  8. Update vmselect version in vmselect.spec
  9. Update vmstorage version in vmstorage.spec
  10. Update vmsingle version in vmsingle.spec
  11. Commit and push changes to the repository. This will automatically build and publish new versions of RPM packages.