An upgrade of Deckhouse Code installs a new version over the current one: the configuration and the data are kept, and the configuration run and the database migrations start automatically. The rules below apply to every installation type.

Version order

Upgrade sequentially, without skipping major versions: the database migrations are designed for the transition from the previous major version. If several major releases lie between the current and the target version, install them one by one and wait for the background migrations to finish between the steps. Ask the vendor for the upgrade path for your pair of versions when you obtain the new version.

Background migrations

An upgrade applies the main database migrations and does not wait for the background ones: they keep running on the instance that is already serving users. All background migrations must finish before the next upgrade, otherwise the upgrade stops with an error.

To check the state of the migrations, select “Admin” in the top right corner, then go to “Monitoring” → “Background migrations” in the left pane. The “Queued” and “Finalizing” tabs must have no jobs left and the “Failed” tab must be empty: every migration is in the “Finished” status. On a typical instance this takes minutes, and longer on large amounts of data.

Before the upgrade

  • Linux package
  • Omnibus Docker
  • Helm Chart

Create a backup as described in Backup and restore and check that fresh copies of the /etc/gitlab/gitlab-secrets.json and /etc/gitlab/gitlab.rb files exist:

sudo gitlab-backup create

Create a backup of the data and of the configuration, as described in Backup and restore, and check that the archives are outside the host.

Write down the tag the container runs, because the rollback returns to it:

docker inspect --format '{{.Config.Image}}' code

Every chart release states the Deckhouse Code version it deploys. Check it before upgrading:

helm show chart deckhouse-code --version <CHART_VERSION>

Create a backup as described in Backup and restore: the rollback of a schema change needs it.

Upgrading

  • Linux package
  • Omnibus Docker
  • Helm Chart

The configuration in /etc/gitlab/gitlab.rb and the data are kept.

  1. Install the new package; the configuration run and the migrations start automatically:

    # RED OS.
sudo rpm -Uvh ./deckhouse-code-<NEW_VERSION>.el8.x86_64.rpm
# Ubuntu.
sudo apt install -y ./deckhouse-code_<NEW_VERSION>_amd64.deb

  2. Check the state of the instance and the installed version:

    # Every service is in the run state.
sudo gitlab-ctl status
# The installed version on RED OS.
rpm -q deckhouse-code
# The installed version on Ubuntu.
dpkg -l deckhouse-code
# health=200 is expected.
curl --cacert /etc/gitlab/ssl/<HOSTNAME>.crt \
     --resolve '<HOSTNAME>:443:127.0.0.1' -o /dev/null \
     -w "health=%{http_code}\n" https://<HOSTNAME>/-/health

While the migrations run, the instance answers more slowly; on a typical instance the upgrade takes minutes.

The container is replaced with one created from the new image tag; the volumes stay, and the instance data and configuration on them are migrated on the first start. Every tag names a version, and no tag follows the latest one, so the target version is named explicitly.

  1. Pull the image of the target version:

    docker pull <REGISTRY>/<FLAVOR>:<VERSION>

  2. Stop and remove the container. The volumes are host directories and stay in place:

    docker stop code
docker rm code

  3. Create the container from the new tag with the same name, ports, volumes and environment variables as in Quick start:

    docker run -d --name code \
  --shm-size 256m \
  -p 80:80 -p 443:443 -p 22:22 \
  -e GITLAB_OMNIBUS_CONFIG="external_url 'http://<HOSTNAME>'" \
  -v /srv/code/config:/etc/gitlab \
  -v /srv/code/logs:/var/log/gitlab \
  -v /srv/code/data:/var/opt/gitlab \
  <REGISTRY>/<FLAVOR>:<VERSION>

  4. Follow the start. The database migrations run during the configuration, and the instance answers 502 until it finishes:

    docker logs -f code

  5. Check the version and the services:

    docker logs code | grep 'Current version'
docker exec code gitlab-ctl status
docker inspect --format '{{.State.Health.Status}}' code

The start-up script reads the version of the data on the volume and compares it with the version in the image. When the instance cannot move to the new version in one step, the output names the version to install first and the start stops: run the tag of that version, wait for the background migrations to finish, then run the target tag.

After the configuration, the script brings the database to the PostgreSQL version of the new image. A failed database upgrade is rolled back, and the container exits with the message Upgrading the existing database failed and was reverted. Create the container with -e GITLAB_SKIP_PG_UPGRADE=true to bring the instance up on the current PostgreSQL version, and upgrade the database separately.

Upgrade the release:

helm repo update
helm upgrade code deckhouse-code \
  -n code \
  -f values.yaml \
  --version <CHART_VERSION>

The command reuses the existing values.yaml. Follow the version order when the mapping spans more than one Deckhouse Code version; skipping a version can leave a later migration unable to run.

A migrations job runs once per upgrade, before the components that depend on the new database schema restart. Check its progress:

d8 k -n code get jobs
d8 k -n code logs job/<MIGRATIONS_JOB_NAME>

Background migrations continue on the running instance after the upgrade and have to finish before the next one.

Rollback

A rollback is only possible from a backup taken before the upgrade: the new version has migrated the data.

  • Linux package
  • Omnibus Docker
  • Helm Chart
Install the package version on which the backup was taken and restore the data as described in Restoring. Ask the vendor for the exact downgrade procedure for your configuration.

  1. Stop and remove the container:

    docker stop code
docker rm code

  2. Create the container from the previous tag with the same volumes.

  3. Restore the data from the backup taken before the upgrade, as described in Restoring.

Roll the release back to the previous revision:

helm rollback code <REVISION>

helm rollback reverts the release’s Kubernetes objects only. It does not reverse a database schema change made by the upgrade’s migrations.

When the upgrade you are rolling back changed the schema, restore the backup taken before the upgrade after the rollback completes, as described in Restoring.

Moving between the package, Docker and the module

Moving an instance from the Linux package or Omnibus Docker to the Deckhouse Kubernetes Platform module and back is described in the Migration section of the module documentation.

Deckhouse Kubernetes Platform module

In the module delivery, the upgrade is started by the platform. With backup.enabled and backup.backupBeforeUpdate set in the CodeInstance resource, the operator creates a backup first and updates the remaining components after the backup job has finished successfully; a failed job postpones the update and raises the D8CodeOperatorUpdatePostpone alert. The order is described in the Automatic backup creation before module updates section of the module documentation.

←
Administration / Backup and restore
Usage / Overview
→