Clean up kubernetes config for maintenance-worker

We're going into the direction of removing the backend and database
deployments, accessing `/uploads` and `/data` through the maintenance
worker pod and carrying out tasks from there.

deployment/legacy-migration/README.md

@@ -5,7 +5,7 @@ server which is running our legacy code and you want to import that data. It
 will import the uploads folder and migrate a dump of the legacy Mongo database
 into our new Neo4J graph database.
 
-**Prepare migration of Human Connection legacy server**
+## Configure Maintenance-Worker Pod
 
 Create a configmap with the specific connection data of your legacy server:
 
@@ -19,7 +19,6 @@ $ kubectl create configmap maintenance-worker          \
   --from-literal=MONGODB_AUTH_DB=hc_api                 \
   --from-literal=MONGODB_DATABASE=hc_api                \
   --from-literal=UPLOADS_DIRECTORY=/var/www/api/uploads \
-  --from-literal=NEO4J_URI=bolt://localhost:7687
 ```
 
 Create a secret with your public and private ssh keys. As the [kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/#use-case-pod-with-ssh-keys) points out, you should be careful with your ssh keys. Anyone with access to your cluster will have access to your ssh keys. Better create a new pair with `ssh-keygen` and copy the public key to your legacy server with `ssh-copy-id`:
@@ -32,24 +31,44 @@ $ kubectl create secret generic ssh-keys          \
   --from-file=known_hosts=/path/to/.ssh/known_hosts
 ```
 
-**Migrate legacy database**
+## Deploy a Temporary Maintenance-Worker Pod
 
-Patch the existing deployments to use a multi-container setup:
+Bring the application into maintenance mode.
 
+{% hint style="info" %} TODO: implement maintenance mode {% endhint %}
+
+
+Then temporarily delete backend and database deployments
 ```bash
-cd legacy-migration
-kubectl apply -f volume-claim-mongo-export.yaml
-kubectl patch --namespace=human-connection deployment nitro-backend --patch "$(cat deployment-backend.yaml)"
-kubectl patch --namespace=human-connection deployment nitro-neo4j   --patch "$(cat deployment-neo4j.yaml)"
-cd ..
+$ kubectl --namespace=human-connection get deployments
+NAME            READY   UP-TO-DATE   AVAILABLE   AGE
+nitro-backend   1/1     1            1           3d11h
+nitro-neo4j     1/1     1            1           3d11h
+nitro-web       2/2     2            2           73d
+$ kubectl --namespace=human-connection delete deployment nitro-neo4j
+deployment.extensions "nitro-neo4j" deleted
+$ kubectl --namespace=human-connection delete deployment nitro-backend
+deployment.extensions "nitro-backend" deleted
 ```
 
-Run the migration:
+Deploy one-time maintenance-worker pod:
+```
+# in deployment/legacy-migration/
+$ kubectl apply -f db-migration-worker.yaml
+pod/nitro-maintenance-worker created
+```
+
+Import legacy database and uploads:
 
 ```text
-$ kubectl --namespace=human-connection get pods
-# change <POD_IDs> below
-$ kubectl --namespace=human-connection exec -it nitro-neo4j-65bbdb597c-nc2lv migrate
-$ kubectl --namespace=human-connection exec -it nitro-backend-c6cc5ff69-8h96z sync_uploads
+$ kubectl --namespace=human-connection exec -it nitro-maintenance-worker bash
+$ import_legacy_db
+$ import_uploads
+$ exit
+```
+
+Delete the pod when you're done:
+```
+$ kubectl --namespace=human-connection delete pod nitro-maintenance-worker
 ```
 

deployment/legacy-migration/db-migration-worker.yaml

@@ -10,30 +10,15 @@
       secret:
         secretName: ssh-keys
         defaultMode: 0400
-    - name: mongo-export
-      persistentVolumeClaim:
-        claimName: mongo-export-claim
     containers:
     - name: nitro-maintenance-worker
       image: humanconnection/maintenance-worker:latest
       envFrom:
       - configMapRef:
           name: maintenance-worker
+      - configMapRef:
+          name: configmap
       volumeMounts:
       - name: secret-volume
         readOnly: false
         mountPath: /root/.ssh
-      - name: mongo-export
-        mountPath: /mongo-export/
----
-  kind: PersistentVolumeClaim
-  apiVersion: v1
-  metadata:
-    name: mongo-export-claim
-    namespace: human-connection
-  spec:
-    accessModes:
-      - ReadWriteOnce
-    resources:
-      requests:
-        storage: 1Gi

deployment/legacy-migration/deployment-backend.yaml

@@ -1,27 +0,0 @@
----
-  apiVersion: extensions/v1beta1
-  kind: Deployment
-  metadata:
-    name: nitro-backend
-    namespace: human-connection
-  spec:
-    template:
-      spec:
-        containers:
-        - name: nitro-maintenance-worker
-          image: humanconnection/maintenance-worker:latest
-          imagePullPolicy: Always
-          envFrom:
-          - configMapRef:
-              name: maintenance-worker
-          volumeMounts:
-          - name: secret-volume
-            readOnly: false
-            mountPath: /root/.ssh
-          - name: uploads
-            mountPath: /uploads/
-        volumes:
-        - name: secret-volume
-          secret:
-            secretName: ssh-keys
-            defaultMode: 0400

deployment/legacy-migration/deployment-neo4j.yaml

@@ -1,39 +0,0 @@
----
-  apiVersion: extensions/v1beta1
-  kind: Deployment
-  metadata:
-    name: nitro-neo4j
-    namespace: human-connection
-  spec:
-    template:
-      spec:
-        containers:
-        - name: nitro-maintenance-worker
-          image: humanconnection/maintenance-worker:latest
-          imagePullPolicy: Always
-          envFrom:
-          - configMapRef:
-              name: maintenance-worker
-          env:
-          - name: COMMIT
-            value: <BACKEND_COMMIT>
-          - name: NEO4J_URI
-            value: bolt://localhost:7687
-          volumeMounts:
-          - name: secret-volume
-            readOnly: false
-            mountPath: /root/.ssh
-          - name: mongo-export
-            mountPath: /mongo-export/
-        - name: nitro-neo4j
-          volumeMounts:
-            - mountPath: /mongo-export/
-              name: mongo-export
-        volumes:
-        - name: secret-volume
-          secret:
-            secretName: ssh-keys
-            defaultMode: 0400
-        - name: mongo-export
-          persistentVolumeClaim:
-            claimName: mongo-export-claim

deployment/legacy-migration/maintenance-worker/README.md

@@ -1,40 +0,0 @@
-# Legacy Migration
-
-This guide helps you to import data from our legacy servers, which are using FeathersJS and MongoDB.
-
-**You can skip this if you don't plan to migrate any legacy applications!**
-
-## Prerequisites
-
-You need [docker](https://www.docker.com/) installed on your machine. Furthermore you need SSH access to the server and you need to know the following login credentials and server settings:
-
-| Environment variable | Description |
-| :--- | :--- |
-| SSH\_USERNAME | Your ssh username on the server |
-| SSH\_HOST | The IP address of the server |
-| MONGODB\_USERNAME | Mongo username on the server |
-| MONGODB\_PASSWORD | Mongo password on the server |
-| MONGODB\_AUTH\_DB | Mongo authentication database |
-| MONGODB\_DATABASE | The name of the mongo database |
-| UPLOADS\_DIRECTORY | Path to remote uploads folder |
-
-## Run the database migration
-
-Run `docker-compose` with all environment variables specified:
-
-```bash
-SSH_USERNAME=username SSH_HOST=some.server.com MONGODB_USERNAME='hc-api' MONGODB_PASSWORD='secret' MONGODB_DATABASE=hc_api MONGODB_AUTH_DB=hc_api UPLOADS_DIRECTORY=/var/www/api/uploads  docker-compose up
-```
-
-Download the remote mongo database:
-
-```bash
-docker-compose exec maintenance-worker ./import.sh
-```
-
-Import the local download into Neo4J:
-
-```bash
-docker-compose exec neo4j import/import.sh
-```
-

deployment/legacy-migration/maintenance-worker/docker-compose.yml

@@ -4,11 +4,11 @@ services:
   maintenance:
     image: humanconnection/maintenance-worker:latest
     build:
-      context: deployment/legacy-migration/maintenance-worker
+      context: .
     volumes:
       - uploads:/uploads
       - neo4j-data:/data
-      - ./deployment/legacy-migration/maintenance-worker/migration/:/migration
+      - ./migration/:/migration
     networks:
       - hc-network
     environment:

deployment/legacy-migration/volume-claim-mongo-export.yaml

@@ -1,12 +0,0 @@
----
-  kind: PersistentVolumeClaim
-  apiVersion: v1
-  metadata:
-    name: mongo-export-claim
-    namespace: human-connection
-  spec:
-    accessModes:
-      - ReadWriteOnce
-    resources:
-      requests:
-        storage: 1Gi