Migrating from Single Node Docker to Kubernetes

Audience: System Administrators

Content Summary: You can import backups from your existing Immuta instance into Kubernetes. Deployment types that can be imported include Kubernetes and Single Node Docker. This page outlines how to import backups from an Immuta instance into Kubernetes.

Backing up your Existing Single Node Docker Instance

Backup the database roles for your existing Immuta instance separately. If you have a combined database instance both of these commands will be run from that database instance.

Exec into the immuta-db container:
```
docker exec -it immuta-db /bin/bash
```

Backup the Immuta role:

backup-immuta.sh \
  -r immuta \
  -d /var/lib/pgsql/11/backups \
  -f immuta-00000000000000.tar.gz

Backup the Metadata role:

backup-immuta.sh \
  -r bometadata \
  -d /var/lib/pgsql/11/backups \
  -f metadata-00000000000000.tar.gz

Copy these output files from the existing instance to a location that has kubectl access to the new cluster. The remainder of this guide will assume that it is being run from a directory containing these two backup files.

Create PersistentVolumeClaims

StorageClassName

Depending on the cloud provider, the storageClassName will change in the below file. Please make sure you test which ones are available to you by using kubectl get storageclass and selecting one that is recommended.

It's generally safe to use aws-efs (if using AWS) or azurefile-csi (if using AKS).

Create a PersistentVolumeClaim using the appropriate storage class for the cloud provider.

Create a file, immuta-backup-pvc.yaml, with the following contents.

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: immuta-backups
  labels:
    app: immuta
    volume-use: backup
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: foo
  resources:
    requests:
      storage: 100Gi

Create the PersistentVolumeClaim by running the following command:
```
kubectl create -f immuta-backup-pvc.yaml
```

Verify that the claim is successfully bound:

kubectl get persistentvolumeclaims -l app=immuta,volume-use=backup

The volume claim should be present and report status "Bound".

Populating PersistentVolumeClaims with Backups

The easiest way to populate existing backups into the PersistentVolumeClaims is to create a temporary deployment that mounts the PersistentVolumeClaims, and copy the backups in using kubectl cp.

Create a file, immuta-backup-importer-deployment.yaml, with the following contents.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: immuta-backup-importer
  labels:
    app: immuta
    component: backup-importer
spec:
  replicas: 1
  selector:
    matchLabels:
      app: immuta
      component: backup-importer
  template:
    metadata:
      labels:
        app: immuta
        component: backup-importer
    spec:
      containers:
      - name: importer
        image: nginx
        tty: true
        stdin: true
        command: ["/bin/bash"]
        volumeMounts:
        - mountPath: "/var/run/immuta/backup/database"
          name: immuta-backup
          subPath: database
        - mountPath: "/var/run/immuta/backup/query-engine"
          name: immuta-backup
          subPath: query-engine
      volumes:
        - name: immuta-backup
          persistentVolumeClaim:
            claimName: immuta-backups

Create the deployment:

kubectl create -f immuta-backup-importer-deployment.yaml

Once the immuta-backup-importer Pod has successfully started running, you can copy the backups into the appropriate locations.

First, get the pod name, and save it into a variable. Next copy the backups you took above into the volume paths specified in the Deployment.

backup_importer_pod_name=$(kubectl get pod \
  -l app=immuta,component=backup-importer \
  -o go-template='{{(index .items 0).metadata.name}}')

kubectl cp \
  immuta-00000000000000.tar.gz \
  ${backup_importer_pod_name}:/var/run/immuta/backup/query-engine/

kubectl cp \
  metadata-00000000000000.tar.gz \
  ${backup_importer_pod_name}:/var/run/immuta/backup/database/immuta-00000000000000.tar.gz

Once this has completed successfully, you can safely delete the deployment. The PersistentVolumeClaims will be re-used for the Immuta deployment.

kubectl delete -f immuta-backup-importer-deployment.yaml

Finish the Restore

Data Persistence

If you are persisting data from a previous deployment, it is best to destroy the existing PVCs for database and query-engine; otherwise, the backups will not be restored properly. If this is the case, uninstall the Helm release, delete the PVCs, and then install again using the restore method.

Make sure the following is set in your immuta-values.yaml in order to ensure backups are pulled from the PVC.

backup:
  enabled: true
  type: volume
  volume:
    claimName: immuta-backups
  restore:
    enabled: true
    queryEngineFile: immuta-00000000000000.tar.gz # This can be whatever filename you are using for your backup
    databaseFile: immuta-00000000000000.tar.gz # This can be whatever filename you are using for your backup

You may now install or upgrade Immuta using Helm:

helm install <release name> immuta/immuta --values <path to immuta-values.yaml> -n <immuta namespace>

Importing Backups from AWS and Azure

For further detail on importing backups from bucket or blob storage, refer to the backup sections of the below documentation.

AWS

Azure

Helm Installation