Merge maintenance with deployment

This commit is contained in:
Robert Schäfer 2019-04-24 00:26:37 +02:00
parent f1c28ee0cc
commit 6ed5ad58d5
21 changed files with 58 additions and 57 deletions

View File

@ -26,7 +26,7 @@
* [Kubernetes Dashboard](deployment/digital-ocean/dashboard/README.md)
* [HTTPS](deployment/digital-ocean/https/README.md)
* [Neo4J DB Backup](deployment/backup.md)
* [Maintenance](maintenance/README.md)
* [Legacy Migration](deployment/legacy-migration/README.md)
* [Feature Specification](cypress/features.md)
* [Code of conduct](CODE_OF_CONDUCT.md)
* [License](LICENSE.md)

View File

@ -45,57 +45,3 @@ $ kubectl apply -f human-connection/
```
This can take a while because kubernetes will download the docker images. Sit back and relax and have a look into your kubernetes dashboard. Wait until all pods turn green and they don't show a warning `Waiting: ContainerCreating` anymore.
#### Legacy data migration
This setup is completely optional and only required if you have data on a server which is running our legacy code and you want to import that data. It will import the uploads folder and migrate a dump of mongodb into neo4j.
**Prepare migration of Human Connection legacy server**
Create a configmap with the specific connection data of your legacy server:
```bash
$ kubectl create configmap maintenance-worker \
--namespace=human-connection \
--from-literal=SSH_USERNAME=someuser \
--from-literal=SSH_HOST=yourhost \
--from-literal=MONGODB_USERNAME=hc-api \
--from-literal=MONGODB_PASSWORD=secretpassword \
--from-literal=MONGODB_AUTH_DB=hc_api \
--from-literal=MONGODB_DATABASE=hc_api \
--from-literal=UPLOADS_DIRECTORY=/var/www/api/uploads \
--from-literal=NEO4J_URI=bolt://localhost:7687
```
Create a secret with your public and private ssh keys. As the [kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/#use-case-pod-with-ssh-keys) points out, you should be careful with your ssh keys. Anyone with access to your cluster will have access to your ssh keys. Better create a new pair with `ssh-keygen` and copy the public key to your legacy server with `ssh-copy-id`:
```bash
$ kubectl create secret generic ssh-keys \
--namespace=human-connection \
--from-file=id_rsa=/path/to/.ssh/id_rsa \
--from-file=id_rsa.pub=/path/to/.ssh/id_rsa.pub \
--from-file=known_hosts=/path/to/.ssh/known_hosts
```
**Migrate legacy database**
Patch the existing deployments to use a multi-container setup:
```bash
cd legacy-migration
kubectl apply -f volume-claim-mongo-export.yaml
kubectl patch --namespace=human-connection deployment nitro-backend --patch "$(cat deployment-backend.yaml)"
kubectl patch --namespace=human-connection deployment nitro-neo4j --patch "$(cat deployment-neo4j.yaml)"
cd ..
```
Run the migration:
```text
$ kubectl --namespace=human-connection get pods
# change <POD_IDs> below
$ kubectl --namespace=human-connection exec -it nitro-neo4j-65bbdb597c-nc2lv migrate
$ kubectl --namespace=human-connection exec -it nitro-backend-c6cc5ff69-8h96z sync_uploads
```

View File

@ -0,0 +1,55 @@
# Legacy data migration
This setup is **completely optional** and only required if you have data on a
server which is running our legacy code and you want to import that data. It
will import the uploads folder and migrate a dump of the legacy Mongo database
into our new Neo4J graph database.
**Prepare migration of Human Connection legacy server**
Create a configmap with the specific connection data of your legacy server:
```bash
$ kubectl create configmap maintenance-worker \
--namespace=human-connection \
--from-literal=SSH_USERNAME=someuser \
--from-literal=SSH_HOST=yourhost \
--from-literal=MONGODB_USERNAME=hc-api \
--from-literal=MONGODB_PASSWORD=secretpassword \
--from-literal=MONGODB_AUTH_DB=hc_api \
--from-literal=MONGODB_DATABASE=hc_api \
--from-literal=UPLOADS_DIRECTORY=/var/www/api/uploads \
--from-literal=NEO4J_URI=bolt://localhost:7687
```
Create a secret with your public and private ssh keys. As the [kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/#use-case-pod-with-ssh-keys) points out, you should be careful with your ssh keys. Anyone with access to your cluster will have access to your ssh keys. Better create a new pair with `ssh-keygen` and copy the public key to your legacy server with `ssh-copy-id`:
```bash
$ kubectl create secret generic ssh-keys \
--namespace=human-connection \
--from-file=id_rsa=/path/to/.ssh/id_rsa \
--from-file=id_rsa.pub=/path/to/.ssh/id_rsa.pub \
--from-file=known_hosts=/path/to/.ssh/known_hosts
```
**Migrate legacy database**
Patch the existing deployments to use a multi-container setup:
```bash
cd legacy-migration
kubectl apply -f volume-claim-mongo-export.yaml
kubectl patch --namespace=human-connection deployment nitro-backend --patch "$(cat deployment-backend.yaml)"
kubectl patch --namespace=human-connection deployment nitro-neo4j --patch "$(cat deployment-neo4j.yaml)"
cd ..
```
Run the migration:
```text
$ kubectl --namespace=human-connection get pods
# change <POD_IDs> below
$ kubectl --namespace=human-connection exec -it nitro-neo4j-65bbdb597c-nc2lv migrate
$ kubectl --namespace=human-connection exec -it nitro-backend-c6cc5ff69-8h96z sync_uploads
```

View File

@ -4,11 +4,11 @@ services:
maintenance:
image: humanconnection/maintenance-worker:latest
build:
context: maintenance
context: deployment/legacy-migration/maintenance-worker
volumes:
- uploads:/uploads
- neo4j-data:/data
- ./maintenance/migration/:/migration
- ./deployment/legacy-migration/maintenance-worker/migration/:/migration
networks:
- hc-network
environment: