A Deckhouse Code backup consists of an archive with the instance data and of files that the archive does not contain. The contents of the archive are the same for the Linux package and for Omnibus Docker: both deliveries run on the same engine, and the archive in them is created by the gitlab-backup command. In a container the same commands run through docker exec, and the archive is written to the data volume.

What the backup contains

The gitlab-backup create command collects into a single archive:

  • Repositories
  • The database
  • Uploads
  • CI artifacts
  • LFS objects
  • The package registry

What is stored separately

The archive does not contain these files, so copy them along with every archive:

  • /etc/gitlab/gitlab-secrets.json: Encryption keys of the database. Without this file the restore is incomplete: two-factor authentication settings, tokens and encrypted CI variables are lost.
  • /etc/gitlab/gitlab.rb: The instance configuration.
  • Certificates and keys from the /etc/gitlab/ssl/ directory.

Store the secret files with the same level of protection as passwords.

  • Linux package
  • Omnibus Docker
The files are in the /etc/gitlab/ directory on the instance machine. Copy them together with every archive.

The files are on the configuration volume. Archive them with a command of its own:

docker exec -t code gitlab-ctl backup-etc

The command writes /etc/gitlab/config_backup/gitlab_config_<TIMESTAMP>_<DATE>.tar with the content of /etc/gitlab: gitlab.rb, gitlab-secrets.json and the TLS certificates. The directory is on the configuration volume, /srv/code/config/config_backup on the host.

Creating a backup

  • Linux package
  • Omnibus Docker
  • Helm Chart

A backup is taken with a single command:

sudo gitlab-backup create

The archive appears in the /var/opt/gitlab/backups/ directory under a name of the form <TIMESTAMP>_<DATE>_<ENGINE_VERSION>_gitlab_backup.tar. The version in the name is the version of the built-in engine, which differs from the version of the deckhouse-code package, so record the package version next to the archive when you take a backup:

# RED OS.
rpm -q deckhouse-code
# Ubuntu.
dpkg -l deckhouse-code

Run the backup in the running container:

docker exec -t code gitlab-backup create

The archive appears in /var/opt/gitlab/backups inside the container, which is /srv/code/data/backups on the host with the volumes from Quick start. Copy it off the host together with the archive of the configuration and the secrets.

The backup-utility utility runs inside the toolbox pod the Helm Chart deploys:

d8 k -n code exec deploy/code-toolbox -- backup-utility

The command creates a backup archive and uploads it to the object storage bucket configured through global.appConfig.object_store (Object storage).

Archive retention and schedule

Archives older than backup_keep_time are deleted when the next backup is created.

  • Linux package
  • Omnibus Docker

The retention time is set in the /etc/gitlab/gitlab.rb file in seconds:

# Seven days.
gitlab_rails['backup_keep_time'] = 604800

The change takes effect after sudo gitlab-ctl reconfigure.

Regular backups are set up with a cron job, for example daily at 02:00:

echo '0 2 * * * root /opt/gitlab/bin/gitlab-backup create CRON=1' \
  | sudo tee /etc/cron.d/deckhouse-code-backup

How long archives are kept is set by gitlab_rails['backup_keep_time'] in the configuration on the configuration volume; its value and behaviour are described in the Linux package tab.

For a schedule, run the same command from the host cron:

echo '0 2 * * * root /usr/bin/docker exec -t code gitlab-backup create CRON=1' \
  | sudo tee /etc/cron.d/deckhouse-code-backup

Copying off the host

Copy fresh archives to a separate server regularly.

  • Linux package
  • Omnibus Docker
Move the archives from the /var/opt/gitlab/backups/ directory, for example with the rsync command. Copy the /etc/gitlab/gitlab-secrets.json and /etc/gitlab/gitlab.rb files together with the archive: without them the archive is not restored completely.
Move the archives from the /srv/code/data/backups directory on the host together with the configuration archive from /srv/code/config/config_backup: without the configuration and the secrets the archive is not restored completely.

Restoring

Restoring replaces the instance data with the contents of the archive. Before you start, check that the conditions are met:

  • The installed version is the version the backup was taken on.
  • The gitlab.rb and gitlab-secrets.json files of the source instance are in place. When moving to a new server, copy them before the restore and run the configuration.
  • The certificates and keys from the /etc/gitlab/ssl/ directory are in place: gitlab.rb refers to them by path, and without these files the configuration run fails to start nginx.
  • There is enough free space for the unpacked archive.

In the commands of the Linux package and Omnibus Docker, <BACKUP_NAME> is the archive file name without the _gitlab_backup.tar suffix: for the 1784800000_2026_07_23_19.1.2_gitlab_backup.tar archive it is 1784800000_2026_07_23_19.1.2.

  • Linux package
  • Omnibus Docker
  • Helm Chart

Check the installed package version with rpm -q deckhouse-code on RED OS or dpkg -l deckhouse-code on Ubuntu. When the configuration and the secrets come from another machine, run sudo gitlab-ctl reconfigure after copying them.

  1. Copy the archive to the backup directory and hand the file over to the git user:

    sudo cp <BACKUP_NAME>_gitlab_backup.tar /var/opt/gitlab/backups/
sudo chown git:git /var/opt/gitlab/backups/<BACKUP_NAME>_gitlab_backup.tar

  2. Stop the services that write to the database; the other services keep running:

    sudo gitlab-ctl stop puma
sudo gitlab-ctl stop sidekiq

  3. Restore the data from the archive:

    sudo gitlab-backup restore BACKUP=<BACKUP_NAME>

    The command asks for confirmation twice: before recreating the authorized_keys file and before dropping the current database tables. Answer yes both times.

  4. Restart the instance and run the self-check:

    sudo gitlab-ctl reconfigure
sudo gitlab-ctl restart
sudo gitlab-rake gitlab:check SANITIZE=true

Check that the container runs the image tag of the version the backup was taken on and that the configuration and the secrets of the source instance are on the configuration volume.

  1. Copy the archive into the backup directory on the data volume and give it the owner the instance uses:

    sudo cp <BACKUP_NAME>_gitlab_backup.tar /srv/code/data/backups/
docker exec code chown git:git /var/opt/gitlab/backups/<BACKUP_NAME>_gitlab_backup.tar

  2. Stop the services that write to the database:

    docker exec code gitlab-ctl stop puma
docker exec code gitlab-ctl stop sidekiq

  3. Start the restore. The command asks for confirmation twice, before the authorized_keys file is rebuilt and before the current database tables are dropped; answer yes both times:

    docker exec -it code gitlab-backup restore BACKUP=<BACKUP_NAME>

  4. Apply the configuration and restart the container:

    docker exec code gitlab-ctl reconfigure
docker restart code

  5. Check the instance:

    docker exec code gitlab-ctl status
docker exec code gitlab-rake gitlab:check SANITIZE=true

  1. Put the instance in maintenance mode.

  2. Confirm the object storage bucket configured through global.appConfig.object_store holds the archive you want to restore.

  3. Run the restore command inside the toolbox pod:

    d8 k -n code exec deploy/code-toolbox -- backup-utility restore --backup-id <BACKUP_ID>

  4. Wait for the command to finish and confirm every pod is Running:

    d8 k -n code get pods

  5. Take the instance out of maintenance mode.

After the restore, sign in to the web interface and clone a repository to check that the instance works with its own data.

Deckhouse Kubernetes Platform module

In the module delivery, backups are created by the operator on the schedule from the CodeInstance resource and uploaded to object storage: the contents of the archive, the restore preconditions and the command matrix are described in the Backups and restore section of the module documentation.

←
Administration / Advanced search
Administration / Upgrade
→