remains from the deployment folder of the ocelot-social repository
This commit is contained in:
parent
8e1708284b
commit
dfc07d88af
45
deployment/old/Maintenance.md
Normal file
45
deployment/old/Maintenance.md
Normal file
@ -0,0 +1,45 @@
|
|||||||
|
# Maintenance mode
|
||||||
|
|
||||||
|
> Despite our best efforts, systems sometimes require downtime for a variety of reasons.
|
||||||
|
|
||||||
|
Quote from [here](https://www.nrmitchi.com/2017/11/easy-maintenance-mode-in-kubernetes/)
|
||||||
|
|
||||||
|
We use our maintenance mode for manual database backup and restore. Also we
|
||||||
|
bring the database into maintenance mode for manual database migrations.
|
||||||
|
|
||||||
|
## Deploy the service
|
||||||
|
|
||||||
|
We prepared sample configuration, so you can simply run:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# in folder deployment/
|
||||||
|
$ kubectl apply -f ./ocelot-social/maintenance/
|
||||||
|
```
|
||||||
|
|
||||||
|
This will fire up a maintenance service.
|
||||||
|
|
||||||
|
## Bring application into maintenance mode
|
||||||
|
|
||||||
|
Now if you want to have a controlled downtime and you want to bring your
|
||||||
|
application into maintenance mode, you can edit your global ingress server.
|
||||||
|
|
||||||
|
E.g. copy file [`deployment/digital-ocean/https/templates/ingress.template.yaml`](../../digital-ocean/https/templates/ingress.template.yaml) to new file `deployment/digital-ocean/https/ingress.yaml` and change the following:
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
...
|
||||||
|
|
||||||
|
- host: develop-k8s.ocelot.social
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
- path: /
|
||||||
|
backend:
|
||||||
|
# serviceName: web
|
||||||
|
serviceName: maintenance
|
||||||
|
# servicePort: 3000
|
||||||
|
servicePort: 80
|
||||||
|
```
|
||||||
|
|
||||||
|
Then run `$ kubectl apply -f deployment/digital-ocean/https/ingress.yaml`. If you
|
||||||
|
want to deactivate the maintenance server, just undo the edit and apply the
|
||||||
|
configuration again.
|
||||||
|
|
||||||
39
deployment/old/digital-ocean/README.md
Normal file
39
deployment/old/digital-ocean/README.md
Normal file
@ -0,0 +1,39 @@
|
|||||||
|
# Digital Ocean
|
||||||
|
|
||||||
|
As a start, read the [introduction into Kubernetes](https://www.digitalocean.com/community/tutorials/an-introduction-to-kubernetes) by the folks at Digital Ocean. The following section should enable you to deploy ocelot.social to your Kubernetes cluster.
|
||||||
|
|
||||||
|
## Connect to your local cluster
|
||||||
|
|
||||||
|
1. Create a cluster at [Digital Ocean](https://www.digitalocean.com/).
|
||||||
|
2. Download the `***-kubeconfig.yaml` from the Web UI.
|
||||||
|
3. Move the file to the default location where kubectl expects it to be: `mv ***-kubeconfig.yaml ~/.kube/config`. Alternatively you can set the config on every command: `--kubeconfig ***-kubeconfig.yaml`
|
||||||
|
4. Now check if you can connect to the cluster and if its your newly created one by running: `kubectl get nodes`
|
||||||
|
|
||||||
|
The output should look about like this:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl get nodes
|
||||||
|
NAME STATUS ROLES AGE VERSION
|
||||||
|
nifty-driscoll-uu1w Ready <none> 69d v1.13.2
|
||||||
|
nifty-driscoll-uuiw Ready <none> 69d v1.13.2
|
||||||
|
nifty-driscoll-uusn Ready <none> 69d v1.13.2
|
||||||
|
```
|
||||||
|
|
||||||
|
If you got the steps right above and see your nodes you can continue.
|
||||||
|
|
||||||
|
Digital Ocean Kubernetes clusters don't have a graphical interface, so I suggest
|
||||||
|
to setup the [Kubernetes dashboard](./dashboard/README.md) as a next step.
|
||||||
|
Configuring [HTTPS](./https/README.md) is bit tricky and therefore I suggest to
|
||||||
|
do this as a last step.
|
||||||
|
|
||||||
|
## Spaces
|
||||||
|
|
||||||
|
We are storing our images in the s3-compatible [DigitalOcean Spaces](https://www.digitalocean.com/docs/spaces/).
|
||||||
|
|
||||||
|
We still want to take backups of our images in case something happens to the images in the cloud. See these [instructions](https://www.digitalocean.com/docs/spaces/resources/s3cmd-usage/) about getting set up with `s3cmd` to take a copy of all images in a `Spaces` namespace, i.e. `ocelot-social-uploads`.
|
||||||
|
|
||||||
|
After configuring `s3cmd` with your credentials, etc. you should be able to make a backup with this command.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
s3cmg get --recursive --skip-existing s3://ocelot-social-uploads
|
||||||
|
```
|
||||||
55
deployment/old/digital-ocean/dashboard/README.md
Normal file
55
deployment/old/digital-ocean/dashboard/README.md
Normal file
@ -0,0 +1,55 @@
|
|||||||
|
# Install Kubernetes Dashboard
|
||||||
|
|
||||||
|
The kubernetes dashboard is optional but very helpful for debugging. If you want to install it, you have to do so only **once** per cluster:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/digital-ocean/
|
||||||
|
$ kubectl apply -f dashboard/
|
||||||
|
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.0-beta4/aio/deploy/recommended.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
### Login to your dashboard
|
||||||
|
|
||||||
|
Proxy the remote kubernetes dashboard to localhost:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl proxy
|
||||||
|
```
|
||||||
|
|
||||||
|
Visit:
|
||||||
|
|
||||||
|
[http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/](http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/)
|
||||||
|
|
||||||
|
You should see a login screen.
|
||||||
|
|
||||||
|
To get your token for the dashboard you can run this command:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n kube-system describe secret $(kubectl -n kube-system get secret | grep admin-user | awk '{print $1}')
|
||||||
|
```
|
||||||
|
|
||||||
|
It should print something like:
|
||||||
|
|
||||||
|
```text
|
||||||
|
Name: admin-user-token-6gl6l
|
||||||
|
Namespace: kube-system
|
||||||
|
Labels: <none>
|
||||||
|
Annotations: kubernetes.io/service-account.name=admin-user
|
||||||
|
kubernetes.io/service-account.uid=b16afba9-dfec-11e7-bbb9-901b0e532516
|
||||||
|
|
||||||
|
Type: kubernetes.io/service-account-token
|
||||||
|
|
||||||
|
Data
|
||||||
|
====
|
||||||
|
ca.crt: 1025 bytes
|
||||||
|
namespace: 11 bytes
|
||||||
|
token: eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJhZG1pbi11c2VyLXRva2VuLTZnbDZsIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImFkbWluLXVzZXIiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC51aWQiOiJiMTZhZmJhOS1kZmVjLTExZTctYmJiOS05MDFiMGU1MzI1MTYiLCJzdWIiOiJzeXN0ZW06c2VydmljZWFjY291bnQ6a3ViZS1zeXN0ZW06YWRtaW4tdXNlciJ9.M70CU3lbu3PP4OjhFms8PVL5pQKj-jj4RNSLA4YmQfTXpPUuxqXjiTf094_Rzr0fgN_IVX6gC4fiNUL5ynx9KU-lkPfk0HnX8scxfJNzypL039mpGt0bbe1IXKSIRaq_9VW59Xz-yBUhycYcKPO9RM2Qa1Ax29nqNVko4vLn1_1wPqJ6XSq3GYI8anTzV8Fku4jasUwjrws6Cn6_sPEGmL54sq5R4Z5afUtv-mItTmqZZdxnkRqcJLlg2Y8WbCPogErbsaCDJoABQ7ppaqHetwfM_0yMun6ABOQbIwwl8pspJhpplKwyo700OSpvTT9zlBsu-b35lzXGBRHzv5g_RA
|
||||||
|
```
|
||||||
|
|
||||||
|
Grab the token from above and paste it into the [login screen](http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/)
|
||||||
|
|
||||||
|
When you are logged in, you should see sth. like:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
Feel free to save the login token from above in your password manager. Unlike the `kubeconfig` file, this token does not expire.
|
||||||
5
deployment/old/digital-ocean/dashboard/admin-user.yaml
Normal file
5
deployment/old/digital-ocean/dashboard/admin-user.yaml
Normal file
@ -0,0 +1,5 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ServiceAccount
|
||||||
|
metadata:
|
||||||
|
name: admin-user
|
||||||
|
namespace: kube-system
|
||||||
BIN
deployment/old/digital-ocean/dashboard/dashboard-screenshot.png
Normal file
BIN
deployment/old/digital-ocean/dashboard/dashboard-screenshot.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 178 KiB |
12
deployment/old/digital-ocean/dashboard/role-binding.yaml
Normal file
12
deployment/old/digital-ocean/dashboard/role-binding.yaml
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
apiVersion: rbac.authorization.k8s.io/v1
|
||||||
|
kind: ClusterRoleBinding
|
||||||
|
metadata:
|
||||||
|
name: admin-user
|
||||||
|
roleRef:
|
||||||
|
apiGroup: rbac.authorization.k8s.io
|
||||||
|
kind: ClusterRole
|
||||||
|
name: cluster-admin
|
||||||
|
subjects:
|
||||||
|
- kind: ServiceAccount
|
||||||
|
name: admin-user
|
||||||
|
namespace: kube-system
|
||||||
126
deployment/old/digital-ocean/https/README.md
Normal file
126
deployment/old/digital-ocean/https/README.md
Normal file
@ -0,0 +1,126 @@
|
|||||||
|
## Create Letsencrypt Issuers and Ingress Services
|
||||||
|
|
||||||
|
Copy the configuration templates and change the file according to your needs.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/digital-ocean/https/
|
||||||
|
cp templates/issuer.template.yaml ./issuer.yaml
|
||||||
|
cp templates/ingress.template.yaml ./ingress.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
At least, **change email addresses** in `issuer.yaml`. For sure you also want
|
||||||
|
to _change the domain name_ in `ingress.yaml`.
|
||||||
|
|
||||||
|
Once you are done, apply the configuration:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/digital-ocean/https/
|
||||||
|
$ kubectl apply -f .
|
||||||
|
```
|
||||||
|
|
||||||
|
{% hint style="info" %}
|
||||||
|
CAUTION: It seems that the behaviour of Digital Ocean has changed and the load balancer is not created automatically anymore.
|
||||||
|
And to create a load balancer costs money. Please refine the following documentation if required.
|
||||||
|
{% endhint %}
|
||||||
|
|
||||||
|
{% tabs %}
|
||||||
|
{% tab title="Without Load Balancer" %}
|
||||||
|
|
||||||
|
A solution without a load balance you can find [here](../no-loadbalancer/README.md).
|
||||||
|
|
||||||
|
{% endtab %}
|
||||||
|
{% tab title="With Digital Ocean Load Balancer" %}
|
||||||
|
|
||||||
|
{% hint style="info" %}
|
||||||
|
CAUTION: It seems that the behaviour of Digital Ocean has changed and the load balancer is not created automatically anymore.
|
||||||
|
Please refine the following documentation if required.
|
||||||
|
{% endhint %}
|
||||||
|
|
||||||
|
In earlier days by now, your cluster should have a load balancer assigned with an external IP
|
||||||
|
address. On Digital Ocean, this is how it should look like:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
If the load balancer isn't created automatically you have to create it your self on Digital Ocean under Networks.
|
||||||
|
In case you don't need a Digital Ocean load balancer (which costs money by the way) have a look in the tab `Without Load Balancer`.
|
||||||
|
|
||||||
|
{% endtab %}
|
||||||
|
{% endtabs %}
|
||||||
|
|
||||||
|
Check the ingress server is working correctly:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ curl -kivL -H 'Host: <DOMAIN_NAME>' 'https://<IP_ADDRESS>'
|
||||||
|
<page HTML>
|
||||||
|
```
|
||||||
|
|
||||||
|
If the response looks good, configure your domain registrar for the new IP address and the domain.
|
||||||
|
|
||||||
|
Now let's get a valid HTTPS certificate. According to the tutorial above, check your tls certificate for staging:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social describe certificate tls
|
||||||
|
<
|
||||||
|
...
|
||||||
|
Spec:
|
||||||
|
...
|
||||||
|
Issuer Ref:
|
||||||
|
Group: cert-manager.io
|
||||||
|
Kind: ClusterIssuer
|
||||||
|
Name: letsencrypt-staging
|
||||||
|
...
|
||||||
|
Events:
|
||||||
|
<no errors>
|
||||||
|
>
|
||||||
|
$ kubectl -n ocelot-social describe secret tls
|
||||||
|
<
|
||||||
|
...
|
||||||
|
Annotations: ...
|
||||||
|
cert-manager.io/issuer-kind: ClusterIssuer
|
||||||
|
cert-manager.io/issuer-name: letsencrypt-staging
|
||||||
|
...
|
||||||
|
>
|
||||||
|
```
|
||||||
|
|
||||||
|
If everything looks good, update the cluster-issuer of your ingress. Change the annotation `cert-manager.io/cluster-issuer` from `letsencrypt-staging` (for testing by getting a dummy certificate – no blocking by letsencrypt, because of to many request cycles) to `letsencrypt-prod` (for production with a real certificate – possible blocking by letsencrypt for several days, because of to many request cycles) in your ingress configuration in `ingress.yaml`.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/digital-ocean/https/
|
||||||
|
$ kubectl apply -f ingress.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
Take a minute and have a look if the certificate is now newly generated by `letsencrypt-prod`, the cluster-issuer for production:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social describe certificate tls
|
||||||
|
<
|
||||||
|
...
|
||||||
|
Spec:
|
||||||
|
...
|
||||||
|
Issuer Ref:
|
||||||
|
Group: cert-manager.io
|
||||||
|
Kind: ClusterIssuer
|
||||||
|
Name: letsencrypt-prod
|
||||||
|
...
|
||||||
|
Events:
|
||||||
|
<no errors>
|
||||||
|
>
|
||||||
|
$ kubectl -n ocelot-social describe secret tls
|
||||||
|
<
|
||||||
|
...
|
||||||
|
Annotations: ...
|
||||||
|
cert-manager.io/issuer-kind: ClusterIssuer
|
||||||
|
cert-manager.io/issuer-name: letsencrypt-prod
|
||||||
|
...
|
||||||
|
>
|
||||||
|
```
|
||||||
|
|
||||||
|
In case the certificate is not newly created delete the former secret to force a refresh:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social delete secret tls
|
||||||
|
```
|
||||||
|
|
||||||
|
Now, HTTPS should be configured on your domain. Congrats!
|
||||||
|
|
||||||
|
For troubleshooting have a look at the cert-manager's [Troubleshooting](https://cert-manager.io/docs/faq/troubleshooting/) or [Troubleshooting Issuing ACME Certificates](https://cert-manager.io/docs/faq/acme/).
|
||||||
BIN
deployment/old/digital-ocean/https/ip-address.png
Normal file
BIN
deployment/old/digital-ocean/https/ip-address.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 141 KiB |
85
deployment/old/legacy-migration/README.md
Normal file
85
deployment/old/legacy-migration/README.md
Normal file
@ -0,0 +1,85 @@
|
|||||||
|
# Legacy data migration
|
||||||
|
|
||||||
|
This setup is **completely optional** and only required if you have data on a
|
||||||
|
server which is running our legacy code and you want to import that data. It
|
||||||
|
will import the uploads folder and migrate a dump of the legacy Mongo database
|
||||||
|
into our new Neo4J graph database.
|
||||||
|
|
||||||
|
## Configure Maintenance-Worker Pod
|
||||||
|
|
||||||
|
Create a configmap with the specific connection data of your legacy server:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl create configmap maintenance-worker \
|
||||||
|
-n ocelot-social \
|
||||||
|
--from-literal=SSH_USERNAME=someuser \
|
||||||
|
--from-literal=SSH_HOST=yourhost \
|
||||||
|
--from-literal=MONGODB_USERNAME=hc-api \
|
||||||
|
--from-literal=MONGODB_PASSWORD=secretpassword \
|
||||||
|
--from-literal=MONGODB_AUTH_DB=hc_api \
|
||||||
|
--from-literal=MONGODB_DATABASE=hc_api \
|
||||||
|
--from-literal=UPLOADS_DIRECTORY=/var/www/api/uploads
|
||||||
|
```
|
||||||
|
|
||||||
|
Create a secret with your public and private ssh keys. As the [kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/#use-case-pod-with-ssh-keys) points out, you should be careful with your ssh keys. Anyone with access to your cluster will have access to your ssh keys. Better create a new pair with `ssh-keygen` and copy the public key to your legacy server with `ssh-copy-id`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl create secret generic ssh-keys \
|
||||||
|
-n ocelot-social \
|
||||||
|
--from-file=id_rsa=/path/to/.ssh/id_rsa \
|
||||||
|
--from-file=id_rsa.pub=/path/to/.ssh/id_rsa.pub \
|
||||||
|
--from-file=known_hosts=/path/to/.ssh/known_hosts
|
||||||
|
```
|
||||||
|
|
||||||
|
## Deploy a Temporary Maintenance-Worker Pod
|
||||||
|
|
||||||
|
Bring the application into maintenance mode.
|
||||||
|
|
||||||
|
{% hint style="info" %} TODO: implement maintenance mode {% endhint %}
|
||||||
|
|
||||||
|
|
||||||
|
Then temporarily delete backend and database deployments
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social get deployments
|
||||||
|
NAME READY UP-TO-DATE AVAILABLE AGE
|
||||||
|
backend 1/1 1 1 3d11h
|
||||||
|
neo4j 1/1 1 1 3d11h
|
||||||
|
webapp 2/2 2 2 73d
|
||||||
|
$ kubectl -n ocelot-social delete deployment neo4j
|
||||||
|
deployment.extensions "neo4j" deleted
|
||||||
|
$ kubectl -n ocelot-social delete deployment backend
|
||||||
|
deployment.extensions "backend" deleted
|
||||||
|
```
|
||||||
|
|
||||||
|
Deploy one-time develop-maintenance-worker pod:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in deployment/legacy-migration/
|
||||||
|
$ kubectl apply -f maintenance-worker.yaml
|
||||||
|
pod/develop-maintenance-worker created
|
||||||
|
```
|
||||||
|
|
||||||
|
Import legacy database and uploads:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social exec -it develop-maintenance-worker bash
|
||||||
|
$ import_legacy_db
|
||||||
|
$ import_legacy_uploads
|
||||||
|
$ exit
|
||||||
|
```
|
||||||
|
|
||||||
|
Delete the pod when you're done:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
$ kubectl -n ocelot-social delete pod develop-maintenance-worker
|
||||||
|
```
|
||||||
|
|
||||||
|
Oh, and of course you have to get those deleted deployments back. One way of
|
||||||
|
doing it would be:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/
|
||||||
|
$ kubectl apply -f human-connection/deployment-backend.yaml -f human-connection/deployment-neo4j.yaml
|
||||||
|
```
|
||||||
|
|
||||||
40
deployment/old/legacy-migration/maintenance-worker.yaml
Normal file
40
deployment/old/legacy-migration/maintenance-worker.yaml
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
---
|
||||||
|
kind: Pod
|
||||||
|
apiVersion: v1
|
||||||
|
metadata:
|
||||||
|
name: develop-maintenance-worker
|
||||||
|
namespace: ocelot-social
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: develop-maintenance-worker
|
||||||
|
image: ocelotsocialnetwork/develop-maintenance-worker:latest
|
||||||
|
imagePullPolicy: Always
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
memory: "2G"
|
||||||
|
limits:
|
||||||
|
memory: "8G"
|
||||||
|
envFrom:
|
||||||
|
- configMapRef:
|
||||||
|
name: maintenance-worker
|
||||||
|
- configMapRef:
|
||||||
|
name: configmap
|
||||||
|
volumeMounts:
|
||||||
|
- name: secret-volume
|
||||||
|
readOnly: false
|
||||||
|
mountPath: /root/.ssh
|
||||||
|
- name: uploads
|
||||||
|
mountPath: /uploads
|
||||||
|
- name: neo4j-data
|
||||||
|
mountPath: /data/
|
||||||
|
volumes:
|
||||||
|
- name: secret-volume
|
||||||
|
secret:
|
||||||
|
secretName: ssh-keys
|
||||||
|
defaultMode: 0400
|
||||||
|
- name: uploads
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: uploads-claim
|
||||||
|
- name: neo4j-data
|
||||||
|
persistentVolumeClaim:
|
||||||
|
claimName: neo4j-data-claim
|
||||||
@ -0,0 +1 @@
|
|||||||
|
.ssh/
|
||||||
2
deployment/old/legacy-migration/maintenance-worker/.gitignore
vendored
Normal file
2
deployment/old/legacy-migration/maintenance-worker/.gitignore
vendored
Normal file
@ -0,0 +1,2 @@
|
|||||||
|
.ssh/
|
||||||
|
ssh/
|
||||||
@ -0,0 +1,21 @@
|
|||||||
|
FROM ocelotsocialnetwork/develop-neo4j:latest
|
||||||
|
|
||||||
|
ENV NODE_ENV=maintenance
|
||||||
|
EXPOSE 7687 7474
|
||||||
|
|
||||||
|
ENV BUILD_DEPS="gettext" \
|
||||||
|
RUNTIME_DEPS="libintl"
|
||||||
|
|
||||||
|
RUN set -x && \
|
||||||
|
apk add --update $RUNTIME_DEPS && \
|
||||||
|
apk add --virtual build_deps $BUILD_DEPS && \
|
||||||
|
cp /usr/bin/envsubst /usr/local/bin/envsubst && \
|
||||||
|
apk del build_deps
|
||||||
|
|
||||||
|
|
||||||
|
RUN apk upgrade --update
|
||||||
|
RUN apk add --no-cache mongodb-tools openssh nodejs yarn rsync
|
||||||
|
|
||||||
|
COPY known_hosts /root/.ssh/known_hosts
|
||||||
|
COPY migration /migration
|
||||||
|
COPY ./binaries/* /usr/local/bin/
|
||||||
@ -0,0 +1,6 @@
|
|||||||
|
# SSH Access
|
||||||
|
# SSH_USERNAME='username'
|
||||||
|
# SSH_HOST='example.org'
|
||||||
|
|
||||||
|
# UPLOADS_DIRECTORY=/var/www/api/uploads
|
||||||
|
OUTPUT_DIRECTORY='/uploads/'
|
||||||
2
deployment/old/legacy-migration/maintenance-worker/binaries/idle
Executable file
2
deployment/old/legacy-migration/maintenance-worker/binaries/idle
Executable file
@ -0,0 +1,2 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
tail -f /dev/null
|
||||||
12
deployment/old/legacy-migration/maintenance-worker/binaries/import_legacy_db
Executable file
12
deployment/old/legacy-migration/maintenance-worker/binaries/import_legacy_db
Executable file
@ -0,0 +1,12 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -e
|
||||||
|
for var in "SSH_USERNAME" "SSH_HOST" "MONGODB_USERNAME" "MONGODB_PASSWORD" "MONGODB_DATABASE" "MONGODB_AUTH_DB"
|
||||||
|
do
|
||||||
|
if [[ -z "${!var}" ]]; then
|
||||||
|
echo "${var} is undefined"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
/migration/mongo/export.sh
|
||||||
|
/migration/neo4j/import.sh
|
||||||
@ -0,0 +1,17 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# import .env config
|
||||||
|
set -o allexport
|
||||||
|
source $(dirname "$0")/.env
|
||||||
|
set +o allexport
|
||||||
|
|
||||||
|
for var in "SSH_USERNAME" "SSH_HOST" "UPLOADS_DIRECTORY"
|
||||||
|
do
|
||||||
|
if [[ -z "${!var}" ]]; then
|
||||||
|
echo "${var} is undefined"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
rsync --archive --update --verbose ${SSH_USERNAME}@${SSH_HOST}:${UPLOADS_DIRECTORY}/ ${OUTPUT_DIRECTORY}
|
||||||
@ -0,0 +1,3 @@
|
|||||||
|
|1|GuOYlVEhTowidPs18zj9p5F2j3o=|sDHJYLz9Ftv11oXeGEjs7SpVyg0= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM5N29bI5CeKu1/RBPyM2fwyf7fuajOO+tyhKe1+CC2sZ1XNB5Ff6t6MtCLNRv2mUuvzTbW/HkisDiA5tuXUHOk=
|
||||||
|
|1|2KP9NV+Q5g2MrtjAeFSVcs8YeOI=|nf3h4wWVwC4xbBS1kzgzE2tBldk= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNhRK6BeIEUxXlS0z/pOfkUkSPfn33g4J1U3L+MyUQYHm+7agT08799ANJhnvELKE1tt4Vx80I9UR81oxzZcy3E=
|
||||||
|
|1|HonYIRNhKyroUHPKU1HSZw0+Qzs=|5T1btfwFBz2vNSldhqAIfTbfIgQ= ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNhRK6BeIEUxXlS0z/pOfkUkSPfn33g4J1U3L+MyUQYHm+7agT08799ANJhnvELKE1tt4Vx80I9UR81oxzZcy3E=
|
||||||
@ -0,0 +1,17 @@
|
|||||||
|
# SSH Access
|
||||||
|
# SSH_USERNAME='username'
|
||||||
|
# SSH_HOST='example.org'
|
||||||
|
|
||||||
|
# Mongo DB on Remote Maschine
|
||||||
|
# MONGODB_USERNAME='mongouser'
|
||||||
|
# MONGODB_PASSWORD='mongopassword'
|
||||||
|
# MONGODB_DATABASE='mongodatabase'
|
||||||
|
# MONGODB_AUTH_DB='admin'
|
||||||
|
|
||||||
|
# Export Settings
|
||||||
|
# On Windows this resolves to C:\Users\dornhoeschen\AppData\Local\Temp\mongo-export (MinGW)
|
||||||
|
EXPORT_PATH='/tmp/mongo-export/'
|
||||||
|
EXPORT_MONGOEXPORT_BIN='mongoexport'
|
||||||
|
MONGO_EXPORT_SPLIT_SIZE=6000
|
||||||
|
# On Windows use something like this
|
||||||
|
# EXPORT_MONGOEXPORT_BIN='C:\Program Files\MongoDB\Server\3.6\bin\mongoexport.exe'
|
||||||
53
deployment/old/legacy-migration/maintenance-worker/migration/mongo/export.sh
Executable file
53
deployment/old/legacy-migration/maintenance-worker/migration/mongo/export.sh
Executable file
@ -0,0 +1,53 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# import .env config
|
||||||
|
set -o allexport
|
||||||
|
source $(dirname "$0")/.env
|
||||||
|
set +o allexport
|
||||||
|
|
||||||
|
# Export collection function defintion
|
||||||
|
function export_collection () {
|
||||||
|
"${EXPORT_MONGOEXPORT_BIN}" --db ${MONGODB_DATABASE} --host localhost -d ${MONGODB_DATABASE} --port 27018 --username ${MONGODB_USERNAME} --password ${MONGODB_PASSWORD} --authenticationDatabase ${MONGODB_AUTH_DB} --collection $1 --out "${EXPORT_PATH}$1.json"
|
||||||
|
mkdir -p ${EXPORT_PATH}splits/$1/
|
||||||
|
split -l ${MONGO_EXPORT_SPLIT_SIZE} -a 3 ${EXPORT_PATH}$1.json ${EXPORT_PATH}splits/$1/
|
||||||
|
}
|
||||||
|
|
||||||
|
# Export collection with query function defintion
|
||||||
|
function export_collection_query () {
|
||||||
|
"${EXPORT_MONGOEXPORT_BIN}" --db ${MONGODB_DATABASE} --host localhost -d ${MONGODB_DATABASE} --port 27018 --username ${MONGODB_USERNAME} --password ${MONGODB_PASSWORD} --authenticationDatabase ${MONGODB_AUTH_DB} --collection $1 --out "${EXPORT_PATH}$1_$3.json" --query "$2"
|
||||||
|
mkdir -p ${EXPORT_PATH}splits/$1_$3/
|
||||||
|
split -l ${MONGO_EXPORT_SPLIT_SIZE} -a 3 ${EXPORT_PATH}$1_$3.json ${EXPORT_PATH}splits/$1_$3/
|
||||||
|
}
|
||||||
|
|
||||||
|
# Delete old export & ensure directory
|
||||||
|
rm -rf ${EXPORT_PATH}*
|
||||||
|
mkdir -p ${EXPORT_PATH}
|
||||||
|
|
||||||
|
# Open SSH Tunnel
|
||||||
|
ssh -4 -M -S my-ctrl-socket -fnNT -L 27018:localhost:27017 -l ${SSH_USERNAME} ${SSH_HOST}
|
||||||
|
|
||||||
|
# Export all Data from the Alpha to json and split them up
|
||||||
|
export_collection "badges"
|
||||||
|
export_collection "categories"
|
||||||
|
export_collection "comments"
|
||||||
|
export_collection_query "contributions" '{"type": "DELETED"}' "DELETED"
|
||||||
|
export_collection_query "contributions" '{"type": "post"}' "post"
|
||||||
|
# export_collection_query "contributions" '{"type": "cando"}' "cando"
|
||||||
|
export_collection "emotions"
|
||||||
|
# export_collection_query "follows" '{"foreignService": "organizations"}' "organizations"
|
||||||
|
export_collection_query "follows" '{"foreignService": "users"}' "users"
|
||||||
|
# export_collection "invites"
|
||||||
|
# export_collection "organizations"
|
||||||
|
# export_collection "pages"
|
||||||
|
# export_collection "projects"
|
||||||
|
# export_collection "settings"
|
||||||
|
export_collection "shouts"
|
||||||
|
# export_collection "status"
|
||||||
|
export_collection_query "users" '{"isVerified": true }' "verified"
|
||||||
|
# export_collection "userscandos"
|
||||||
|
# export_collection "usersettings"
|
||||||
|
|
||||||
|
# Close SSH Tunnel
|
||||||
|
ssh -S my-ctrl-socket -O check -l ${SSH_USERNAME} ${SSH_HOST}
|
||||||
|
ssh -S my-ctrl-socket -O exit -l ${SSH_USERNAME} ${SSH_HOST}
|
||||||
@ -0,0 +1,16 @@
|
|||||||
|
# Neo4J Settings
|
||||||
|
# NEO4J_USERNAME='neo4j'
|
||||||
|
# NEO4J_PASSWORD='letmein'
|
||||||
|
|
||||||
|
# Import Settings
|
||||||
|
# On Windows this resolves to C:\Users\dornhoeschen\AppData\Local\Temp\mongo-export (MinGW)
|
||||||
|
IMPORT_PATH='/tmp/mongo-export/'
|
||||||
|
IMPORT_CHUNK_PATH='/tmp/mongo-export/splits/'
|
||||||
|
|
||||||
|
IMPORT_CHUNK_PATH_CQL='/tmp/mongo-export/splits/'
|
||||||
|
# On Windows this path needs to be windows style since the cypher-shell runs native - note the forward slash
|
||||||
|
# IMPORT_CHUNK_PATH_CQL='C:/Users/dornhoeschen/AppData/Local/Temp/mongo-export/splits/'
|
||||||
|
|
||||||
|
IMPORT_CYPHERSHELL_BIN='cypher-shell'
|
||||||
|
# On Windows use something like this
|
||||||
|
# IMPORT_CYPHERSHELL_BIN='C:\Program Files\neo4j-community\bin\cypher-shell.bat'
|
||||||
@ -0,0 +1,52 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[?] image: {
|
||||||
|
[?] path: { // Path is incorrect in Nitro - is icon the correct name for this field?
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[ ] alt: { // If we use an image - should we not have an alt?
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[?] status: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] enum: ['permanent', 'temporary'],
|
||||||
|
[ ] default: 'permanent', // Default value is missing in Nitro
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] type: {
|
||||||
|
[?] type: String, // in nitro this is a defined enum - seems good for now
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[X] id: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] createdAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[?] updatedAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as badge
|
||||||
|
MERGE(b:Badge {id: badge._id["$oid"]})
|
||||||
|
ON CREATE SET
|
||||||
|
b.id = badge.key,
|
||||||
|
b.type = badge.type,
|
||||||
|
b.icon = replace(badge.image.path, 'https://api-alpha.human-connection.org', ''),
|
||||||
|
b.status = badge.status,
|
||||||
|
b.createdAt = badge.createdAt.`$date`,
|
||||||
|
b.updatedAt = badge.updatedAt.`$date`
|
||||||
|
;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (n:Badge) DETACH DELETE n;
|
||||||
@ -0,0 +1,129 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[X] title: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] slug: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[ ] unique: true // Unique value is not enforced in Nitro?
|
||||||
|
},
|
||||||
|
[?] icon: { // Nitro adds required: true
|
||||||
|
[X] type: String,
|
||||||
|
[ ] unique: true // Unique value is not enforced in Nitro?
|
||||||
|
},
|
||||||
|
[?] createdAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[?] updatedAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as category
|
||||||
|
MERGE(c:Category {id: category._id["$oid"]})
|
||||||
|
ON CREATE SET
|
||||||
|
c.name = category.title,
|
||||||
|
c.slug = category.slug,
|
||||||
|
c.icon = category.icon,
|
||||||
|
c.createdAt = category.createdAt.`$date`,
|
||||||
|
c.updatedAt = category.updatedAt.`$date`
|
||||||
|
;
|
||||||
|
|
||||||
|
// Transform icon names
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-justforfun")
|
||||||
|
SET c.icon = 'smile'
|
||||||
|
SET c.slug = 'just-for-fun'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-luck")
|
||||||
|
SET c.icon = 'heart-o'
|
||||||
|
SET c.slug = 'happiness-values'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-health")
|
||||||
|
SET c.icon = 'medkit'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-environment")
|
||||||
|
SET c.icon = 'tree'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-animal-justice")
|
||||||
|
SET c.icon = 'paw'
|
||||||
|
SET c.slug = 'animal-protection'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-human-rights")
|
||||||
|
SET c.icon = 'balance-scale'
|
||||||
|
SET c.slug = 'human-rights-justice'
|
||||||
|
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-education")
|
||||||
|
SET c.icon = 'graduation-cap'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-cooperation")
|
||||||
|
SET c.icon = 'users'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-politics")
|
||||||
|
SET c.icon = 'university'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-economy")
|
||||||
|
SET c.icon = 'money'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-technology")
|
||||||
|
SET c.icon = 'flash'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-internet")
|
||||||
|
SET c.icon = 'mouse-pointer'
|
||||||
|
SET c.slug = 'it-internet-data-privacy'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-art")
|
||||||
|
SET c.icon = 'paint-brush'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-freedom-of-speech")
|
||||||
|
SET c.icon = 'bullhorn'
|
||||||
|
SET c.slug = 'freedom-of-speech'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-sustainability")
|
||||||
|
SET c.icon = 'shopping-cart'
|
||||||
|
;
|
||||||
|
|
||||||
|
MATCH (c:Category)
|
||||||
|
WHERE (c.icon = "categories-peace")
|
||||||
|
SET c.icon = 'angellist'
|
||||||
|
SET c.slug = 'global-peace-nonviolence'
|
||||||
|
;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (n:Category) DETACH DELETE n;
|
||||||
@ -0,0 +1,67 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[?] userId: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] contributionId: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[X] content: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] contentExcerpt: { // Generated from content
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true // Not required in Nitro
|
||||||
|
},
|
||||||
|
[ ] hasMore: { type: Boolean },
|
||||||
|
[ ] upvotes: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
},
|
||||||
|
[ ] upvoteCount: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[?] deleted: {
|
||||||
|
[X] type: Boolean,
|
||||||
|
[ ] default: false, // Default value is missing in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as comment
|
||||||
|
MERGE (c:Comment {id: comment._id["$oid"]})
|
||||||
|
ON CREATE SET
|
||||||
|
c.content = comment.content,
|
||||||
|
c.contentExcerpt = comment.contentExcerpt,
|
||||||
|
c.deleted = comment.deleted,
|
||||||
|
c.createdAt = comment.createdAt.`$date`,
|
||||||
|
c.updatedAt = comment.updatedAt.`$date`,
|
||||||
|
c.disabled = false
|
||||||
|
WITH c, comment, comment.contributionId as postId
|
||||||
|
MATCH (post:Post {id: postId})
|
||||||
|
WITH c, post, comment.userId as userId
|
||||||
|
MATCH (author:User {id: userId})
|
||||||
|
MERGE (c)-[:COMMENTS]->(post)
|
||||||
|
MERGE (author)-[:WROTE]->(c)
|
||||||
|
;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (n:Comment) DETACH DELETE n;
|
||||||
@ -0,0 +1,156 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
[?] { //Modeled incorrect as Post
|
||||||
|
[?] userId: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] organizationId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[X] categoryIds: {
|
||||||
|
[X] type: Array,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[X] title: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] slug: { // Generated from title
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[?] unique: true, // Unique value is not enforced in Nitro?
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] type: { // db.getCollection('contributions').distinct('type') -> 'DELETED', 'cando', 'post'
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] cando: {
|
||||||
|
[ ] difficulty: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] enum: ['easy', 'medium', 'hard']
|
||||||
|
},
|
||||||
|
[ ] reasonTitle: { type: String },
|
||||||
|
[ ] reason: { type: String }
|
||||||
|
},
|
||||||
|
[X] content: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true
|
||||||
|
},
|
||||||
|
[?] contentExcerpt: { // Generated from content
|
||||||
|
[X] type: String,
|
||||||
|
[?] required: true // Not required in Nitro
|
||||||
|
},
|
||||||
|
[ ] hasMore: { type: Boolean },
|
||||||
|
[X] teaserImg: { type: String },
|
||||||
|
[ ] language: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] shoutCount: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] meta: {
|
||||||
|
[ ] hasVideo: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] embedds: {
|
||||||
|
[ ] type: Object,
|
||||||
|
[ ] default: {}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[?] visibility: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] enum: ['public', 'friends', 'private'],
|
||||||
|
[ ] default: 'public', // Default value is missing in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] isEnabled: {
|
||||||
|
[X] type: Boolean,
|
||||||
|
[ ] default: true, // Default value is missing in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] tags: { type: Array }, // ensure this is working properly
|
||||||
|
[ ] emotions: {
|
||||||
|
[ ] type: Object,
|
||||||
|
[-] index: true,
|
||||||
|
[ ] default: {
|
||||||
|
[ ] angry: {
|
||||||
|
[ ] count: 0,
|
||||||
|
[ ] percent: 0
|
||||||
|
[ ] },
|
||||||
|
[ ] cry: {
|
||||||
|
[ ] count: 0,
|
||||||
|
[ ] percent: 0
|
||||||
|
[ ] },
|
||||||
|
[ ] surprised: {
|
||||||
|
[ ] count: 0,
|
||||||
|
[ ] percent: 0
|
||||||
|
},
|
||||||
|
[ ] happy: {
|
||||||
|
[ ] count: 0,
|
||||||
|
[ ] percent: 0
|
||||||
|
},
|
||||||
|
[ ] funny: {
|
||||||
|
[ ] count: 0,
|
||||||
|
[ ] percent: 0
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[?] deleted: { // THis field is not always present in the alpha-data
|
||||||
|
[?] type: Boolean,
|
||||||
|
[ ] default: false, // Default value is missing in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] createdAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[?] updatedAt: {
|
||||||
|
[?] type: Date, // Type is modeled as string in Nitro which is incorrect
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as post
|
||||||
|
MERGE (p:Post {id: post._id["$oid"]})
|
||||||
|
ON CREATE SET
|
||||||
|
p.title = post.title,
|
||||||
|
p.slug = post.slug,
|
||||||
|
p.image = replace(post.teaserImg, 'https://api-alpha.human-connection.org', ''),
|
||||||
|
p.content = post.content,
|
||||||
|
p.contentExcerpt = post.contentExcerpt,
|
||||||
|
p.visibility = toLower(post.visibility),
|
||||||
|
p.createdAt = post.createdAt.`$date`,
|
||||||
|
p.updatedAt = post.updatedAt.`$date`,
|
||||||
|
p.deleted = COALESCE(post.deleted, false),
|
||||||
|
p.disabled = COALESCE(NOT post.isEnabled, false)
|
||||||
|
WITH p, post
|
||||||
|
MATCH (u:User {id: post.userId})
|
||||||
|
MERGE (u)-[:WROTE]->(p)
|
||||||
|
WITH p, post, post.categoryIds as categoryIds
|
||||||
|
UNWIND categoryIds AS categoryId
|
||||||
|
MATCH (c:Category {id: categoryId})
|
||||||
|
MERGE (p)-[:CATEGORIZED]->(c)
|
||||||
|
WITH p, post.tags AS tags
|
||||||
|
UNWIND tags AS tag
|
||||||
|
WITH apoc.text.replace(tag, '[^\\p{L}0-9]', '') as tagNoSpacesAllowed
|
||||||
|
CALL apoc.when(tagNoSpacesAllowed =~ '^((\\p{L}+[\\p{L}0-9]*)|([0-9]+\\p{L}+[\\p{L}0-9]*))$', 'RETURN tagNoSpacesAllowed', '', {tagNoSpacesAllowed: tagNoSpacesAllowed})
|
||||||
|
YIELD value as validated
|
||||||
|
WHERE validated.tagNoSpacesAllowed IS NOT NULL
|
||||||
|
MERGE (t:Tag { id: validated.tagNoSpacesAllowed, disabled: false, deleted: false })
|
||||||
|
MERGE (p)-[:TAGGED]->(t)
|
||||||
|
;
|
||||||
@ -0,0 +1,2 @@
|
|||||||
|
MATCH (n:Post) DETACH DELETE n;
|
||||||
|
MATCH (n:Tag) DETACH DELETE n;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (n) DETACH DELETE n;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (u:User)-[e:EMOTED]->(c:Post) DETACH DELETE e;
|
||||||
@ -0,0 +1,58 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[X] userId: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[X] contributionId: {
|
||||||
|
[X] type: String,
|
||||||
|
[X] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] rated: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[?] enum: ['funny', 'happy', 'surprised', 'cry', 'angry']
|
||||||
|
},
|
||||||
|
[X] createdAt: {
|
||||||
|
[X] type: Date,
|
||||||
|
[X] default: Date.now
|
||||||
|
},
|
||||||
|
[X] updatedAt: {
|
||||||
|
[X] type: Date,
|
||||||
|
[X] default: Date.now
|
||||||
|
},
|
||||||
|
[-] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as emotion
|
||||||
|
MATCH (u:User {id: emotion.userId}),
|
||||||
|
(c:Post {id: emotion.contributionId})
|
||||||
|
MERGE (u)-[e:EMOTED {
|
||||||
|
id: emotion._id["$oid"],
|
||||||
|
emotion: emotion.rated,
|
||||||
|
createdAt: datetime(emotion.createdAt.`$date`),
|
||||||
|
updatedAt: datetime(emotion.updatedAt.`$date`)
|
||||||
|
}]->(c)
|
||||||
|
RETURN e;
|
||||||
|
/*
|
||||||
|
// Queries
|
||||||
|
// user sets an emotion emotion:
|
||||||
|
// MERGE (u)-[e:EMOTED {id: ..., emotion: "funny", createdAt: ..., updatedAt: ...}]->(c)
|
||||||
|
// user removes emotion
|
||||||
|
// MATCH (u)-[e:EMOTED]->(c) DELETE e
|
||||||
|
// contribution distributions over every `emotion` property value for one post
|
||||||
|
// MATCH (u:User)-[e:EMOTED]->(c:Post {id: "5a70bbc8508f5b000b443b1a"}) RETURN e.emotion,COUNT(e.emotion)
|
||||||
|
// contribution distributions over every `emotion` property value for one user (advanced - "whats the primary emotion used by the user?")
|
||||||
|
// MATCH (u:User{id:"5a663b1ac64291000bf302a1"})-[e:EMOTED]->(c:Post) RETURN e.emotion,COUNT(e.emotion)
|
||||||
|
// contribution distributions over every `emotion` property value for all posts created by one user (advanced - "how do others react to my contributions?")
|
||||||
|
// MATCH (u:User)-[e:EMOTED]->(c:Post)<-[w:WROTE]-(a:User{id:"5a663b1ac64291000bf302a1"}) RETURN e.emotion,COUNT(e.emotion)
|
||||||
|
// if we can filter the above an a variable timescale that would be great (should be possible on createdAt and updatedAt fields)
|
||||||
|
*/
|
||||||
@ -0,0 +1 @@
|
|||||||
|
MATCH (u1:User)-[f:FOLLOWS]->(u2:User) DETACH DELETE f;
|
||||||
@ -0,0 +1,36 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[?] userId: {
|
||||||
|
[-] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] foreignId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] foreignService: { // db.getCollection('follows').distinct('foreignService') returns 'organizations' and 'users'
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
index:
|
||||||
|
[?] { userId: 1, foreignId: 1, foreignService: 1 },{ unique: true } // is the unique constrain modeled?
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as follow
|
||||||
|
MATCH (u1:User {id: follow.userId}), (u2:User {id: follow.foreignId})
|
||||||
|
MERGE (u1)-[:FOLLOWS]->(u2)
|
||||||
|
;
|
||||||
108
deployment/old/legacy-migration/maintenance-worker/migration/neo4j/import.sh
Executable file
108
deployment/old/legacy-migration/maintenance-worker/migration/neo4j/import.sh
Executable file
@ -0,0 +1,108 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -e
|
||||||
|
|
||||||
|
# import .env config
|
||||||
|
set -o allexport
|
||||||
|
source $(dirname "$0")/.env
|
||||||
|
set +o allexport
|
||||||
|
|
||||||
|
# Delete collection function defintion
|
||||||
|
function delete_collection () {
|
||||||
|
# Delete from Database
|
||||||
|
echo "Delete $2"
|
||||||
|
"${IMPORT_CYPHERSHELL_BIN}" < $(dirname "$0")/$1/delete.cql > /dev/null
|
||||||
|
# Delete index file
|
||||||
|
rm -f "${IMPORT_PATH}splits/$2.index"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Import collection function defintion
|
||||||
|
function import_collection () {
|
||||||
|
# index file of those chunks we have already imported
|
||||||
|
INDEX_FILE="${IMPORT_PATH}splits/$1.index"
|
||||||
|
# load index file
|
||||||
|
if [ -f "$INDEX_FILE" ]; then
|
||||||
|
readarray -t IMPORT_INDEX <$INDEX_FILE
|
||||||
|
else
|
||||||
|
declare -a IMPORT_INDEX
|
||||||
|
fi
|
||||||
|
# for each chunk import data
|
||||||
|
for chunk in ${IMPORT_PATH}splits/$1/*
|
||||||
|
do
|
||||||
|
CHUNK_FILE_NAME=$(basename "${chunk}")
|
||||||
|
# does the index not contain the chunk file name?
|
||||||
|
if [[ ! " ${IMPORT_INDEX[@]} " =~ " ${CHUNK_FILE_NAME} " ]]; then
|
||||||
|
# calculate the path of the chunk
|
||||||
|
export IMPORT_CHUNK_PATH_CQL_FILE="${IMPORT_CHUNK_PATH_CQL}$1/${CHUNK_FILE_NAME}"
|
||||||
|
# load the neo4j command and replace file variable with actual path
|
||||||
|
NEO4J_COMMAND="$(envsubst '${IMPORT_CHUNK_PATH_CQL_FILE}' < $(dirname "$0")/$2)"
|
||||||
|
# run the import of the chunk
|
||||||
|
echo "Import $1 ${CHUNK_FILE_NAME} (${chunk})"
|
||||||
|
echo "${NEO4J_COMMAND}" | "${IMPORT_CYPHERSHELL_BIN}" > /dev/null
|
||||||
|
# add file to array and file
|
||||||
|
IMPORT_INDEX+=("${CHUNK_FILE_NAME}")
|
||||||
|
echo "${CHUNK_FILE_NAME}" >> ${INDEX_FILE}
|
||||||
|
else
|
||||||
|
echo "Skipping $1 ${CHUNK_FILE_NAME} (${chunk})"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# Time variable
|
||||||
|
SECONDS=0
|
||||||
|
|
||||||
|
# Delete all Neo4J Database content
|
||||||
|
echo "Deleting Database Contents"
|
||||||
|
delete_collection "badges" "badges"
|
||||||
|
delete_collection "categories" "categories"
|
||||||
|
delete_collection "users" "users"
|
||||||
|
delete_collection "follows" "follows_users"
|
||||||
|
delete_collection "contributions" "contributions_post"
|
||||||
|
delete_collection "contributions" "contributions_cando"
|
||||||
|
delete_collection "shouts" "shouts"
|
||||||
|
delete_collection "comments" "comments"
|
||||||
|
delete_collection "emotions" "emotions"
|
||||||
|
|
||||||
|
#delete_collection "invites"
|
||||||
|
#delete_collection "notifications"
|
||||||
|
#delete_collection "organizations"
|
||||||
|
#delete_collection "pages"
|
||||||
|
#delete_collection "projects"
|
||||||
|
#delete_collection "settings"
|
||||||
|
#delete_collection "status"
|
||||||
|
#delete_collection "systemnotifications"
|
||||||
|
#delete_collection "userscandos"
|
||||||
|
#delete_collection "usersettings"
|
||||||
|
echo "DONE"
|
||||||
|
|
||||||
|
# Import Data
|
||||||
|
echo "Start Importing Data"
|
||||||
|
import_collection "badges" "badges/badges.cql"
|
||||||
|
import_collection "categories" "categories/categories.cql"
|
||||||
|
import_collection "users_verified" "users/users.cql"
|
||||||
|
import_collection "follows_users" "follows/follows.cql"
|
||||||
|
#import_collection "follows_organizations" "follows/follows.cql"
|
||||||
|
import_collection "contributions_post" "contributions/contributions.cql"
|
||||||
|
#import_collection "contributions_cando" "contributions/contributions.cql"
|
||||||
|
#import_collection "contributions_DELETED" "contributions/contributions.cql"
|
||||||
|
import_collection "shouts" "shouts/shouts.cql"
|
||||||
|
import_collection "comments" "comments/comments.cql"
|
||||||
|
import_collection "emotions" "emotions/emotions.cql"
|
||||||
|
|
||||||
|
# import_collection "invites"
|
||||||
|
# import_collection "notifications"
|
||||||
|
# import_collection "organizations"
|
||||||
|
# import_collection "pages"
|
||||||
|
# import_collection "systemnotifications"
|
||||||
|
# import_collection "userscandos"
|
||||||
|
# import_collection "usersettings"
|
||||||
|
|
||||||
|
# does only contain dummy data
|
||||||
|
# import_collection "projects"
|
||||||
|
|
||||||
|
# does only contain alpha specifc data
|
||||||
|
# import_collection "status
|
||||||
|
# import_collection "settings""
|
||||||
|
|
||||||
|
echo "DONE"
|
||||||
|
|
||||||
|
echo "Time elapsed: $SECONDS seconds"
|
||||||
@ -0,0 +1,39 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] email: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true,
|
||||||
|
[ ] unique: true
|
||||||
|
},
|
||||||
|
[ ] code: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] role: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] enum: ['admin', 'moderator', 'manager', 'editor', 'user'],
|
||||||
|
[ ] default: 'user'
|
||||||
|
},
|
||||||
|
[ ] invitedByUserId: { type: String },
|
||||||
|
[ ] language: { type: String },
|
||||||
|
[ ] badgeIds: [],
|
||||||
|
[ ] wasUsed: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as invite;
|
||||||
@ -0,0 +1,48 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] userId: { // User this notification is sent to
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] type: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] enum: ['comment','comment-mention','contribution-mention','following-contribution']
|
||||||
|
},
|
||||||
|
[ ] relatedUserId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] relatedContributionId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] relatedOrganizationId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] relatedCommentId: {type: String },
|
||||||
|
[ ] unseen: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as notification;
|
||||||
@ -0,0 +1,137 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] name: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] slug: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] unique: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] followersCounts: {
|
||||||
|
[ ] users: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] organizations: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] projects: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] followingCounts: {
|
||||||
|
[ ] users: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] organizations: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] projects: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] categoryIds: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] logo: { type: String },
|
||||||
|
[ ] coverImg: { type: String },
|
||||||
|
[ ] userId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] description: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] descriptionExcerpt: { type: String }, // will be generated automatically
|
||||||
|
[ ] publicEmail: { type: String },
|
||||||
|
[ ] url: { type: String },
|
||||||
|
[ ] type: {
|
||||||
|
[ ] type: String,
|
||||||
|
[-] index: true,
|
||||||
|
[ ] enum: ['ngo', 'npo', 'goodpurpose', 'ev', 'eva']
|
||||||
|
},
|
||||||
|
[ ] language: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] default: 'de',
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] addresses: {
|
||||||
|
[ ] type: [{
|
||||||
|
[ ] street: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] zipCode: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] city: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] country: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] lat: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] lng: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] required: true
|
||||||
|
}
|
||||||
|
}],
|
||||||
|
[ ] default: []
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] isEnabled: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] reviewedBy: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] default: null,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] tags: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] deleted: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as organisation;
|
||||||
@ -0,0 +1,55 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] title: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] slug: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] type: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] default: 'page'
|
||||||
|
},
|
||||||
|
[ ] key: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] content: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] language: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] active: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
index:
|
||||||
|
[ ] { slug: 1, language: 1 },{ unique: true }
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as page;
|
||||||
@ -0,0 +1,44 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] name: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] slug: { type: String },
|
||||||
|
[ ] followerIds: [],
|
||||||
|
[ ] categoryIds: { type: Array },
|
||||||
|
[ ] logo: { type: String },
|
||||||
|
[ ] userId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] description: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] content: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] addresses: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as project;
|
||||||
@ -0,0 +1,36 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] key: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] default: 'system',
|
||||||
|
[-] index: true,
|
||||||
|
[ ] unique: true
|
||||||
|
},
|
||||||
|
[ ] invites: {
|
||||||
|
[ ] userCanInvite: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] maxInvitesByUser: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] default: 1
|
||||||
|
},
|
||||||
|
[ ] onlyUserWithBadgesCanInvite: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] maintenance: false
|
||||||
|
}, {
|
||||||
|
[ ] timestamps: true
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as setting;
|
||||||
@ -0,0 +1 @@
|
|||||||
|
// this is just a relation between users and contributions - no need to delete
|
||||||
@ -0,0 +1,36 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[?] userId: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] foreignId: {
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[?] foreignService: { // db.getCollection('shots').distinct('foreignService') returns 'contributions'
|
||||||
|
[X] type: String,
|
||||||
|
[ ] required: true, // Not required in Nitro
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
index:
|
||||||
|
[?] { userId: 1, foreignId: 1 },{ unique: true } // is the unique constrain modeled?
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as shout
|
||||||
|
MATCH (u:User {id: shout.userId}), (p:Post {id: shout.foreignId})
|
||||||
|
MERGE (u)-[:SHOUTED]->(p)
|
||||||
|
;
|
||||||
@ -0,0 +1,19 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] maintenance: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as status;
|
||||||
@ -0,0 +1,61 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] type: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] default: 'info',
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] title: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] content: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] slot: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] language: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] permanent: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] requireConfirmation: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] active: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: true,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] totalCount: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as systemnotification;
|
||||||
@ -0,0 +1,2 @@
|
|||||||
|
MATCH (n:User) DETACH DELETE n;
|
||||||
|
MATCH (e:EmailAddress) DETACH DELETE e;
|
||||||
@ -0,0 +1,124 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[?] email: {
|
||||||
|
[X] type: String,
|
||||||
|
[-] index: true,
|
||||||
|
[X] required: true,
|
||||||
|
[?] unique: true //unique constrain missing in Nitro
|
||||||
|
},
|
||||||
|
[?] password: { // Not required in Alpha -> verify if always present
|
||||||
|
[X] type: String
|
||||||
|
},
|
||||||
|
[X] name: { type: String },
|
||||||
|
[X] slug: {
|
||||||
|
[X] type: String,
|
||||||
|
[-] index: true
|
||||||
|
},
|
||||||
|
[ ] gender: { type: String },
|
||||||
|
[ ] followersCounts: {
|
||||||
|
[ ] users: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] organizations: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] projects: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] followingCounts: {
|
||||||
|
[ ] users: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] organizations: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
},
|
||||||
|
[ ] projects: {
|
||||||
|
[ ] type: Number,
|
||||||
|
[ ] default: 0
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] timezone: { type: String },
|
||||||
|
[X] avatar: { type: String },
|
||||||
|
[X] coverImg: { type: String },
|
||||||
|
[ ] doiToken: { type: String },
|
||||||
|
[ ] confirmedAt: { type: Date },
|
||||||
|
[?] badgeIds: [], // Verify this is working properly
|
||||||
|
[?] deletedAt: { type: Date }, // The Date of deletion is not saved in Nitro
|
||||||
|
[?] createdAt: {
|
||||||
|
[?] type: Date, // Modeled as String in Nitro
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[?] updatedAt: {
|
||||||
|
[?] type: Date, // Modeled as String in Nitro
|
||||||
|
[ ] default: Date.now // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[ ] lastActiveAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] isVerified: { type: Boolean },
|
||||||
|
[?] role: {
|
||||||
|
[X] type: String,
|
||||||
|
[-] index: true,
|
||||||
|
[?] enum: ['admin', 'moderator', 'manager', 'editor', 'user'], // missing roles manager & editor in Nitro
|
||||||
|
[ ] default: 'user' // Default value is missing in Nitro
|
||||||
|
},
|
||||||
|
[ ] verifyToken: { type: String },
|
||||||
|
[ ] verifyShortToken: { type: String },
|
||||||
|
[ ] verifyExpires: { type: Date },
|
||||||
|
[ ] verifyChanges: { type: Object },
|
||||||
|
[ ] resetToken: { type: String },
|
||||||
|
[ ] resetShortToken: { type: String },
|
||||||
|
[ ] resetExpires: { type: Date },
|
||||||
|
[X] wasSeeded: { type: Boolean },
|
||||||
|
[X] wasInvited: { type: Boolean },
|
||||||
|
[ ] language: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] default: 'en'
|
||||||
|
},
|
||||||
|
[ ] termsAndConditionsAccepted: { type: Date }, // we display the terms and conditions on registration
|
||||||
|
[ ] systemNotificationsSeen: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as user
|
||||||
|
MERGE(u:User {id: user._id["$oid"]})
|
||||||
|
ON CREATE SET
|
||||||
|
u.name = user.name,
|
||||||
|
u.slug = COALESCE(user.slug, apoc.text.random(20, "[A-Za-z]")),
|
||||||
|
u.email = user.email,
|
||||||
|
u.encryptedPassword = user.password,
|
||||||
|
u.avatar = replace(user.avatar, 'https://api-alpha.human-connection.org', ''),
|
||||||
|
u.coverImg = replace(user.coverImg, 'https://api-alpha.human-connection.org', ''),
|
||||||
|
u.wasInvited = user.wasInvited,
|
||||||
|
u.wasSeeded = user.wasSeeded,
|
||||||
|
u.role = toLower(user.role),
|
||||||
|
u.createdAt = user.createdAt.`$date`,
|
||||||
|
u.updatedAt = user.updatedAt.`$date`,
|
||||||
|
u.deleted = user.deletedAt IS NOT NULL,
|
||||||
|
u.disabled = false
|
||||||
|
MERGE (e:EmailAddress {
|
||||||
|
email: user.email,
|
||||||
|
createdAt: toString(datetime()),
|
||||||
|
verifiedAt: toString(datetime())
|
||||||
|
})
|
||||||
|
MERGE (e)-[:BELONGS_TO]->(u)
|
||||||
|
MERGE (u)-[:PRIMARY_EMAIL]->(e)
|
||||||
|
WITH u, user, user.badgeIds AS badgeIds
|
||||||
|
UNWIND badgeIds AS badgeId
|
||||||
|
MATCH (b:Badge {id: badgeId})
|
||||||
|
MERGE (b)-[:REWARDED]->(u)
|
||||||
|
;
|
||||||
@ -0,0 +1,35 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] userId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] contributionId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] done: {
|
||||||
|
[ ] type: Boolean,
|
||||||
|
[ ] default: false
|
||||||
|
},
|
||||||
|
[ ] doneAt: { type: Date },
|
||||||
|
[ ] createdAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
},
|
||||||
|
[ ] wasSeeded: { type: Boolean }
|
||||||
|
}
|
||||||
|
index:
|
||||||
|
[ ] { userId: 1, contributionId: 1 },{ unique: true }
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as usercando;
|
||||||
@ -0,0 +1,43 @@
|
|||||||
|
/*
|
||||||
|
// Alpha Model
|
||||||
|
// [ ] Not modeled in Nitro
|
||||||
|
// [X] Modeled in Nitro
|
||||||
|
// [-] Omitted in Nitro
|
||||||
|
// [?] Unclear / has work to be done for Nitro
|
||||||
|
{
|
||||||
|
[ ] userId: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true,
|
||||||
|
[ ] unique: true
|
||||||
|
},
|
||||||
|
[ ] blacklist: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
},
|
||||||
|
[ ] uiLanguage: {
|
||||||
|
[ ] type: String,
|
||||||
|
[ ] required: true
|
||||||
|
},
|
||||||
|
[ ] contentLanguages: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] default: []
|
||||||
|
},
|
||||||
|
[ ] filter: {
|
||||||
|
[ ] categoryIds: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] index: true
|
||||||
|
},
|
||||||
|
[ ] emotions: {
|
||||||
|
[ ] type: Array,
|
||||||
|
[ ] index: true
|
||||||
|
}
|
||||||
|
},
|
||||||
|
[ ] hideUsersWithoutTermsOfUseSigniture: {type: Boolean},
|
||||||
|
[ ] updatedAt: {
|
||||||
|
[ ] type: Date,
|
||||||
|
[ ] default: Date.now
|
||||||
|
}
|
||||||
|
}
|
||||||
|
*/
|
||||||
|
|
||||||
|
CALL apoc.load.json("file:${IMPORT_CHUNK_PATH_CQL_FILE}") YIELD value as usersetting;
|
||||||
40
deployment/old/mailserver/Deployment.yaml
Normal file
40
deployment/old/mailserver/Deployment.yaml
Normal file
@ -0,0 +1,40 @@
|
|||||||
|
{{- if .Values.developmentMailserverDomain }}
|
||||||
|
apiVersion: apps/v1
|
||||||
|
kind: Deployment
|
||||||
|
metadata:
|
||||||
|
name: {{ .Release.Name }}-mailserver
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||||
|
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||||
|
app.kubernetes.io/name: ocelot-social
|
||||||
|
app.kubernetes.io/version: {{ .Chart.AppVersion }}
|
||||||
|
helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
|
||||||
|
spec:
|
||||||
|
replicas: 1
|
||||||
|
minReadySeconds: 15
|
||||||
|
progressDeadlineSeconds: 60
|
||||||
|
selector:
|
||||||
|
matchLabels:
|
||||||
|
ocelot.social/selector: deployment-mailserver
|
||||||
|
template:
|
||||||
|
metadata:
|
||||||
|
labels:
|
||||||
|
ocelot.social/selector: deployment-mailserver
|
||||||
|
name: mailserver
|
||||||
|
spec:
|
||||||
|
containers:
|
||||||
|
- name: mailserver
|
||||||
|
image: djfarrelly/maildev
|
||||||
|
imagePullPolicy: {{ .Values.image.pullPolicy }}
|
||||||
|
ports:
|
||||||
|
- containerPort: 80
|
||||||
|
- containerPort: 25
|
||||||
|
envFrom:
|
||||||
|
- configMapRef:
|
||||||
|
name: {{ .Release.Name }}-configmap
|
||||||
|
- secretRef:
|
||||||
|
name: {{ .Release.Name }}-secrets
|
||||||
|
restartPolicy: Always
|
||||||
|
terminationGracePeriodSeconds: 30
|
||||||
|
status: {}
|
||||||
|
{{- end}}
|
||||||
18
deployment/old/mailserver/README.md
Normal file
18
deployment/old/mailserver/README.md
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
# Development Mail Server
|
||||||
|
|
||||||
|
You can deploy a fake smtp server which captures all send mails and displays
|
||||||
|
them in a web interface. The [sample configuration](../templates/configmap.template.yaml)
|
||||||
|
is assuming such a dummy server in the `SMTP_HOST` configuration and points to
|
||||||
|
a cluster-internal SMTP server.
|
||||||
|
|
||||||
|
To deploy the SMTP server just uncomment the relevant code in the
|
||||||
|
[ingress server configuration](../../https/templates/ingress.template.yaml) and
|
||||||
|
run the following:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# in folder deployment/ocelot-social
|
||||||
|
$ kubectl apply -f mailserver/
|
||||||
|
```
|
||||||
|
|
||||||
|
You might need to refresh the TLS secret to enable HTTPS on the publicly
|
||||||
|
available web interface.
|
||||||
22
deployment/old/mailserver/Service.yaml
Normal file
22
deployment/old/mailserver/Service.yaml
Normal file
@ -0,0 +1,22 @@
|
|||||||
|
{{- if .Values.developmentMailserverDomain }}
|
||||||
|
apiVersion: v1
|
||||||
|
kind: Service
|
||||||
|
metadata:
|
||||||
|
name: {{ .Release.Name }}-mailserver
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/instance: {{ .Release.Name }}
|
||||||
|
app.kubernetes.io/managed-by: {{ .Release.Service }}
|
||||||
|
app.kubernetes.io/name: ocelot-social
|
||||||
|
app.kubernetes.io/version: {{ .Chart.AppVersion }}
|
||||||
|
helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
|
||||||
|
spec:
|
||||||
|
ports:
|
||||||
|
- name: web
|
||||||
|
port: 80
|
||||||
|
targetPort: 80
|
||||||
|
- name: smtp
|
||||||
|
port: 25
|
||||||
|
targetPort: 25
|
||||||
|
selector:
|
||||||
|
ocelot.social/selector: deployment-mailserver
|
||||||
|
{{- end}}
|
||||||
42
deployment/old/mailserver/ingress.yaml
Normal file
42
deployment/old/mailserver/ingress.yaml
Normal file
@ -0,0 +1,42 @@
|
|||||||
|
kind: Ingress
|
||||||
|
apiVersion: networking.k8s.io/v1
|
||||||
|
metadata:
|
||||||
|
name: ingress-{{ .Release.Name }}-webapp
|
||||||
|
labels:
|
||||||
|
app.kubernetes.io/name: "{{ .Chart.Name }}"
|
||||||
|
app.kubernetes.io/instance: "{{ .Release.Name }}"
|
||||||
|
app.kubernetes.io/version: "{{ .Chart.AppVersion }}"
|
||||||
|
app.kubernetes.io/component: "ingress webapp"
|
||||||
|
app.kubernetes.io/part-of: "{{ .Chart.Name }}"
|
||||||
|
app.kubernetes.io/managed-by: "{{ .Release.Service }}"
|
||||||
|
helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
|
||||||
|
annotations:
|
||||||
|
kubernetes.io/ingress.class: "nginx"
|
||||||
|
cert-manager.io/cluster-issuer: {{ .Values.LETSENCRYPT.ISSUER }}
|
||||||
|
nginx.ingress.kubernetes.io/proxy-body-size: {{ .Values.NGINX.PROXY_BODY_SIZE }}
|
||||||
|
spec:
|
||||||
|
tls:
|
||||||
|
- hosts:
|
||||||
|
- {{ .Values.LETSENCRYPT.DOMAIN }}
|
||||||
|
secretName: tls
|
||||||
|
rules:
|
||||||
|
- host: {{ .Values.LETSENCRYPT.DOMAIN }}
|
||||||
|
http:
|
||||||
|
paths:
|
||||||
|
- path: /
|
||||||
|
pathType: ImplementationSpecific
|
||||||
|
backend:
|
||||||
|
service:
|
||||||
|
name: {{ .Release.Name }}-webapp
|
||||||
|
port:
|
||||||
|
number: 3000
|
||||||
|
|
||||||
|
#{{- if .Values.developmentMailserverDomain }}
|
||||||
|
# - host: {{ .Values.developmentMailserverDomain }}
|
||||||
|
# http:
|
||||||
|
# paths:
|
||||||
|
# - path: /
|
||||||
|
# backend:
|
||||||
|
# serviceName: {{ .Release.Name }}-mailserver
|
||||||
|
# servicePort: 80
|
||||||
|
#{{- end }}
|
||||||
43
deployment/old/monitoring/README.md
Normal file
43
deployment/old/monitoring/README.md
Normal file
@ -0,0 +1,43 @@
|
|||||||
|
# Metrics
|
||||||
|
|
||||||
|
You can optionally setup [prometheus](https://prometheus.io/) and
|
||||||
|
[grafana](https://grafana.com/) for metrics.
|
||||||
|
|
||||||
|
We follow this tutorial [here](https://medium.com/@chris_linguine/how-to-monitor-your-kubernetes-cluster-with-prometheus-and-grafana-2d5704187fc8):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
kubectl proxy # proxy to your kubernetes dashboard
|
||||||
|
|
||||||
|
helm repo list
|
||||||
|
# If using helm v3, the stable repository is not set, so you need to manually add it.
|
||||||
|
helm repo add stable https://kubernetes-charts.storage.googleapis.com
|
||||||
|
# Create a monitoring namespace for your cluster
|
||||||
|
kubectl create namespace monitoring
|
||||||
|
helm --namespace monitoring install prometheus stable/prometheus
|
||||||
|
kubectl -n monitoring get pods # look for 'server'
|
||||||
|
kubectl port-forward -n monitoring <PROMETHEUS_SERVER_ID> 9090
|
||||||
|
# You can now see your prometheus server on: http://localhost:9090
|
||||||
|
|
||||||
|
# Make sure you are in folder `deployment/`
|
||||||
|
kubectl apply -f monitoring/grafana/config.yml
|
||||||
|
helm --namespace monitoring install grafana stable/grafana -f monitoring/grafana/values.yml
|
||||||
|
# Get the admin password for grafana from your kubernetes dashboard.
|
||||||
|
kubectl --namespace monitoring port-forward <POD_NAME> 3000
|
||||||
|
# You can now see your grafana dashboard on: http://localhost:3000
|
||||||
|
# Login with user 'admin' and the password you just looked up.
|
||||||
|
# In your dashboard import this dashboard:
|
||||||
|
# https://grafana.com/grafana/dashboards/1860
|
||||||
|
# Enter ID 180 and choose "Prometheus" as datasource.
|
||||||
|
# You got metrics!
|
||||||
|
```
|
||||||
|
|
||||||
|
Now you should see something like this:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
You can set up a grafana dashboard, by visiting https://grafana.com/dashboards, finding one that is suitable and copying it's id.
|
||||||
|
You then go to the left hand menu in localhost, choose `Dashboard` > `Manage` > `Import`
|
||||||
|
Paste in the id, click `Load`, select `Prometheus` for the data source, and click `Import`
|
||||||
|
|
||||||
|
When you just installed prometheus and grafana, the data will not be available
|
||||||
|
immediately, so wait for a couple of minutes and reload.
|
||||||
16
deployment/old/monitoring/grafana/config.yml
Normal file
16
deployment/old/monitoring/grafana/config.yml
Normal file
@ -0,0 +1,16 @@
|
|||||||
|
apiVersion: v1
|
||||||
|
kind: ConfigMap
|
||||||
|
metadata:
|
||||||
|
name: prometheus-grafana-datasource
|
||||||
|
namespace: monitoring
|
||||||
|
labels:
|
||||||
|
grafana_datasource: '1'
|
||||||
|
data:
|
||||||
|
datasource.yaml: |-
|
||||||
|
apiVersion: 1
|
||||||
|
datasources:
|
||||||
|
- name: Prometheus
|
||||||
|
type: prometheus
|
||||||
|
access: proxy
|
||||||
|
orgId: 1
|
||||||
|
url: http://prometheus-server.monitoring.svc.cluster.local
|
||||||
BIN
deployment/old/monitoring/grafana/metrics.png
Normal file
BIN
deployment/old/monitoring/grafana/metrics.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 206 KiB |
4
deployment/old/monitoring/grafana/values.yml
Normal file
4
deployment/old/monitoring/grafana/values.yml
Normal file
@ -0,0 +1,4 @@
|
|||||||
|
sidecar:
|
||||||
|
datasources:
|
||||||
|
enabled: true
|
||||||
|
label: grafana_datasource
|
||||||
37
deployment/old/volumes/README.md
Normal file
37
deployment/old/volumes/README.md
Normal file
@ -0,0 +1,37 @@
|
|||||||
|
# Persistent Volumes
|
||||||
|
|
||||||
|
At the moment, the application needs two persistent volumes:
|
||||||
|
|
||||||
|
* The `/data/` folder where `neo4j` stores its database and
|
||||||
|
* the folder `/develop-backend/public/uploads` where the backend stores uploads, in case you don't use Digital Ocean Spaces (an AWS S3 bucket) for this purpose.
|
||||||
|
|
||||||
|
As a matter of precaution, the persistent volume claims that setup these volumes
|
||||||
|
live in a separate folder. You don't want to accidently loose all your data in
|
||||||
|
your database by running
|
||||||
|
|
||||||
|
```sh
|
||||||
|
kubectl delete -f ocelot-social/
|
||||||
|
```
|
||||||
|
|
||||||
|
or do you?
|
||||||
|
|
||||||
|
## Create Persistent Volume Claims
|
||||||
|
|
||||||
|
Run the following:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# in folder deployments/
|
||||||
|
$ kubectl apply -f volumes
|
||||||
|
persistentvolumeclaim/neo4j-data-claim created
|
||||||
|
persistentvolumeclaim/uploads-claim created
|
||||||
|
```
|
||||||
|
|
||||||
|
## Backup And Restore
|
||||||
|
|
||||||
|
We tested a couple of options how to do disaster recovery in kubernetes. First,
|
||||||
|
there is the [offline backup strategy](./neo4j-offline-backup/README.md) of the
|
||||||
|
community edition of Neo4J, which you can also run on a local installation.
|
||||||
|
Kubernetes also offers so-called [volume snapshots](./volume-snapshots/README.md).
|
||||||
|
Changing the [reclaim policy](./reclaim-policy/README.md) of your persistent
|
||||||
|
volumes might be an additional safety measure. Finally, there is also a
|
||||||
|
kubernetes specific disaster recovery tool called [Velero](./velero/README.md).
|
||||||
12
deployment/old/volumes/neo4j-data.yaml
Normal file
12
deployment/old/volumes/neo4j-data.yaml
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
---
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
apiVersion: v1
|
||||||
|
metadata:
|
||||||
|
name: neo4j-data-claim
|
||||||
|
namespace: ocelot-social
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: "10Gi" # see requirements for Neo4j v3.5.14 https://neo4j.com/docs/operations-manual/3.5/installation/requirements/
|
||||||
88
deployment/old/volumes/neo4j-offline-backup/README.md
Normal file
88
deployment/old/volumes/neo4j-offline-backup/README.md
Normal file
@ -0,0 +1,88 @@
|
|||||||
|
# Backup (offline)
|
||||||
|
|
||||||
|
This tutorial explains how to carry out an offline backup of your Neo4J
|
||||||
|
database in a kubernetes cluster.
|
||||||
|
|
||||||
|
An offline backup requires the Neo4J database to be stopped. Read
|
||||||
|
[the docs](https://neo4j.com/docs/operations-manual/current/tools/dump-load/).
|
||||||
|
Neo4J also offers online backups but this is available in enterprise edition
|
||||||
|
only.
|
||||||
|
|
||||||
|
The tricky part is to stop the Neo4J database *without* stopping the container.
|
||||||
|
Neo4J's docker container image starts `neo4j` by default, so we have to override
|
||||||
|
this command with sth. that keeps the container spinning but does not terminate
|
||||||
|
it.
|
||||||
|
|
||||||
|
## Stop and Restart Neo4J Database in Kubernetes
|
||||||
|
|
||||||
|
[This tutorial](http://bigdatums.net/2017/11/07/how-to-keep-docker-containers-running/)
|
||||||
|
explains how to keep a docker container running. For kubernetes, the way to
|
||||||
|
override the docker image `CMD` is explained [here](https://kubernetes.io/docs/tasks/inject-data-application/define-command-argument-container/#define-a-command-and-arguments-when-you-create-a-pod).
|
||||||
|
|
||||||
|
So, all we have to do is edit the kubernetes deployment of our Neo4J database
|
||||||
|
and set a custom `command` every time we have to carry out tasks like backup,
|
||||||
|
restore, seed etc.
|
||||||
|
|
||||||
|
First bring the application into [maintenance mode](https://github.com/Ocelot-Social-Community/Ocelot-Social/blob/master/deployment/ocelot-social/maintenance/README.md) to ensure there are no
|
||||||
|
database connections left and nobody can access the application.
|
||||||
|
|
||||||
|
Run the following:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl -n ocelot-social edit deployment develop-neo4j
|
||||||
|
```
|
||||||
|
|
||||||
|
Add the following to `spec.template.spec.containers`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
["tail", "-f", "/dev/null"]
|
||||||
|
```
|
||||||
|
|
||||||
|
and write the file which will update the deployment.
|
||||||
|
|
||||||
|
The command `tail -f /dev/null` is the equivalent of *sleep forever*. It is a
|
||||||
|
hack to keep the container busy and to prevent its shutdown. It will also
|
||||||
|
override the default `neo4j` command and the kubernetes pod will not start the
|
||||||
|
database.
|
||||||
|
|
||||||
|
Now perform your tasks!
|
||||||
|
|
||||||
|
When you're done, edit the deployment again and remove the `command`. Write the
|
||||||
|
file and trigger an update of the deployment.
|
||||||
|
|
||||||
|
## Create a Backup in Kubernetes
|
||||||
|
|
||||||
|
First stop your Neo4J database, see above. Then:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl -n ocelot-social get pods
|
||||||
|
# Copy the ID of the pod running Neo4J.
|
||||||
|
$ kubectl -n ocelot-social exec -it <POD-ID> bash
|
||||||
|
# Once you're in the pod, dump the db to a file e.g. `/root/neo4j-backup`.
|
||||||
|
> neo4j-admin dump --to=/root/neo4j-backup
|
||||||
|
> exit
|
||||||
|
# Download the file from the pod to your computer.
|
||||||
|
$ kubectl cp human-connection/<POD-ID>:/root/neo4j-backup ./neo4j-backup
|
||||||
|
```
|
||||||
|
|
||||||
|
Revert your changes to deployment `develop-neo4j` which will restart the database.
|
||||||
|
|
||||||
|
## Restore a Backup in Kubernetes
|
||||||
|
|
||||||
|
First stop your Neo4J database. Then:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl -n ocelot-social get pods
|
||||||
|
# Copy the ID of the pod running Neo4J.
|
||||||
|
# Then upload your local backup to the pod. Note that once the pod gets deleted
|
||||||
|
# e.g. if you change the deployment, the backup file is gone with it.
|
||||||
|
$ kubectl cp ./neo4j-backup human-connection/<POD-ID>:/root/
|
||||||
|
$ kubectl -n ocelot-social exec -it <POD-ID> bash
|
||||||
|
# Once you're in the pod restore the backup and overwrite the default database
|
||||||
|
# called `graph.db` with `--force`.
|
||||||
|
# This will delete all existing data in database `graph.db`!
|
||||||
|
> neo4j-admin load --from=/root/neo4j-backup --force
|
||||||
|
> exit
|
||||||
|
```
|
||||||
|
|
||||||
|
Revert your changes to deployment `develop-neo4j` which will restart the database.
|
||||||
59
deployment/old/volumes/neo4j-online-backup/README.md
Normal file
59
deployment/old/volumes/neo4j-online-backup/README.md
Normal file
@ -0,0 +1,59 @@
|
|||||||
|
# Backup (online)
|
||||||
|
|
||||||
|
## Online backups are only avaible with a Neo4j Enterprise and a license, see https://neo4j.com/licensing/ for the different licenses available
|
||||||
|
|
||||||
|
This tutorial explains how to carry out an online backup of your Neo4J
|
||||||
|
database in a kubernetes cluster.
|
||||||
|
|
||||||
|
One of the benefits of doing an online backup is that the Neo4j database does not need to be stopped, so there is no downtime. Read [the docs](https://neo4j.com/docs/operations-manual/current/backup/performing/)
|
||||||
|
|
||||||
|
To use Neo4j Enterprise you must add this line to your configmap, if using, or your deployment `develop-neo4j` env.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
NEO4J_ACCEPT_LICENSE_AGREEMENT: "yes"
|
||||||
|
```
|
||||||
|
|
||||||
|
## Create a Backup in Kubernetes
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# Backup the database with one command, this will get the develop-neo4j pod, ssh into it, and run the backup command
|
||||||
|
$ kubectl -n=human-connection exec -it $(kubectl -n=human-connection get pods | grep develop-neo4j | awk '{ print $1 }') -- neo4j-admin backup --backup-dir=/var/lib/neo4j --name=neo4j-backup
|
||||||
|
# Download the file from the pod to your computer.
|
||||||
|
$ kubectl cp human-connection/$(kubectl -n=human-connection get pods | grep develop-neo4j | awk '{ print $1 }'):/var/lib/neo4j/neo4j-backup ./neo4j-backup/
|
||||||
|
```
|
||||||
|
|
||||||
|
You should now have a backup of the database locally. If you want, you can simulate disaster recovery by sshing into the develop-neo4j pod, deleting all data and restoring from backup
|
||||||
|
|
||||||
|
## Disaster where database data is gone somehow
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl -n=human-connection exec -it $(kubectl -n=human-connection get pods | grep develop-neo4j |awk '{ print $1 }') bash
|
||||||
|
# Enter cypher-shell
|
||||||
|
$ cypher-shell
|
||||||
|
# Delete all data
|
||||||
|
> MATCH (n) DETACH DELETE (n);
|
||||||
|
|
||||||
|
> exit
|
||||||
|
```
|
||||||
|
|
||||||
|
## Restore a backup in Kubernetes
|
||||||
|
|
||||||
|
Restoration must be done while the database is not running, see [our docs](https://docs.human-connection.org/human-connection/deployment/volumes/neo4j-offline-backup#stop-and-restart-neo-4-j-database-in-kubernetes) for how to stop the database, but keep the container running
|
||||||
|
|
||||||
|
After, you have stopped the database, and have the pod running, you can restore the database by running these commands:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl -n ocelot-social get pods
|
||||||
|
# Copy the ID of the pod running Neo4J.
|
||||||
|
# Then upload your local backup to the pod. Note that once the pod gets deleted
|
||||||
|
# e.g. if you change the deployment, the backup file is gone with it.
|
||||||
|
$ kubectl cp ./neo4j-backup/ human-connection/<POD-ID>:/root/
|
||||||
|
$ kubectl -n ocelot-social exec -it <POD-ID> bash
|
||||||
|
# Once you're in the pod restore the backup and overwrite the default database
|
||||||
|
# called `graph.db` with `--force`.
|
||||||
|
# This will delete all existing data in database `graph.db`!
|
||||||
|
> neo4j-admin restore --from=/root/neo4j-backup --force
|
||||||
|
> exit
|
||||||
|
```
|
||||||
|
|
||||||
|
Revert your changes to deployment `develop-neo4j` which will restart the database.
|
||||||
12
deployment/old/volumes/uploads.yaml
Normal file
12
deployment/old/volumes/uploads.yaml
Normal file
@ -0,0 +1,12 @@
|
|||||||
|
---
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
apiVersion: v1
|
||||||
|
metadata:
|
||||||
|
name: uploads-claim
|
||||||
|
namespace: ocelot-social
|
||||||
|
spec:
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: "10Gi"
|
||||||
112
deployment/old/volumes/velero/README.md
Normal file
112
deployment/old/volumes/velero/README.md
Normal file
@ -0,0 +1,112 @@
|
|||||||
|
# Velero
|
||||||
|
|
||||||
|
{% hint style="danger" %}
|
||||||
|
I tried Velero and it did not work reliably all the time. Sometimes the
|
||||||
|
kubernetes cluster crashes during recovery or data is not fully recovered.
|
||||||
|
|
||||||
|
Feel free to test it out and update this documentation once you feel that it's
|
||||||
|
working reliably. It is very likely that Digital Ocean had some bugs when I
|
||||||
|
tried out the steps below.
|
||||||
|
{% endhint %}
|
||||||
|
|
||||||
|
We use [velero](https://github.com/heptio/velero) for on premise backups, we
|
||||||
|
tested on version `v0.11.0`, you can find their
|
||||||
|
documentation [here](https://heptio.github.io/velero/v0.11.0/).
|
||||||
|
|
||||||
|
Our kubernets configurations adds some annotations to pods. The annotations
|
||||||
|
define the important persistent volumes that need to be backed up. Velero will
|
||||||
|
pick them up and store the volumes in the same cluster but in another namespace
|
||||||
|
`velero`.
|
||||||
|
|
||||||
|
## Prequisites
|
||||||
|
|
||||||
|
You have to install the binary `velero` on your computer and get a tarball of
|
||||||
|
the latest release. We use `v0.11.0` so visit the
|
||||||
|
[release](https://github.com/heptio/velero/releases/tag/v0.11.0) page and
|
||||||
|
download and extract e.g. [velero-v0.11.0-linux-arm64.tar.gz](https://github.com/heptio/velero/releases/download/v0.11.0/velero-v0.11.0-linux-amd64.tar.gz).
|
||||||
|
|
||||||
|
|
||||||
|
## Setup Velero Namespace
|
||||||
|
|
||||||
|
Follow their [getting started](https://heptio.github.io/velero/v0.11.0/get-started)
|
||||||
|
instructions to setup the Velero namespace. We use
|
||||||
|
[Minio](https://docs.min.io/docs/deploy-minio-on-kubernetes) and
|
||||||
|
[restic](https://github.com/restic/restic), so check out Velero's instructions
|
||||||
|
how to setup [restic](https://heptio.github.io/velero/v0.11.0/restic):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# run from the extracted folder of the tarball
|
||||||
|
$ kubectl apply -f config/common/00-prereqs.yaml
|
||||||
|
$ kubectl apply -f config/minio/
|
||||||
|
```
|
||||||
|
|
||||||
|
Once completed, you should see the namespace in your kubernetes dashboard.
|
||||||
|
|
||||||
|
## Manually Create an On-Premise Backup
|
||||||
|
|
||||||
|
When you create your deployments for Human Connection the required annotations
|
||||||
|
should already be in place. So when you create a backup of namespace
|
||||||
|
`human-connection`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ velero backup create hc-backup --include-namespaces=human-connection
|
||||||
|
```
|
||||||
|
|
||||||
|
That should backup your persistent volumes, too. When you enter:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ velero backup describe hc-backup --details
|
||||||
|
```
|
||||||
|
|
||||||
|
You should see the persistent volumes at the end of the log:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
....
|
||||||
|
|
||||||
|
Restic Backups:
|
||||||
|
Completed:
|
||||||
|
human-connection/develop-backend-5b6dd96d6b-q77n6: uploads
|
||||||
|
human-connection/develop-neo4j-686d768598-z2vhh: neo4j-data
|
||||||
|
```
|
||||||
|
|
||||||
|
## Simulate a Disaster
|
||||||
|
|
||||||
|
Feel free to try out if you loose any data when you simulate a disaster and try
|
||||||
|
to restore the namespace from the backup:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ kubectl delete namespace human-connection
|
||||||
|
```
|
||||||
|
|
||||||
|
Wait until the wrongdoing has completed, then:
|
||||||
|
```sh
|
||||||
|
$ velero restore create --from-backup hc-backup
|
||||||
|
```
|
||||||
|
|
||||||
|
Now, I keep my fingers crossed that everything comes back again. If not, I feel
|
||||||
|
very sorry for you.
|
||||||
|
|
||||||
|
|
||||||
|
## Schedule a Regular Backup
|
||||||
|
|
||||||
|
Check out the [docs](https://heptio.github.io/velero/v0.11.0/get-started). You
|
||||||
|
can create a regular schedule e.g. with:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ velero schedule create hc-weekly-backup --schedule="@weekly" --include-namespaces=human-connection
|
||||||
|
```
|
||||||
|
|
||||||
|
Inspect the created backups:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
$ velero schedule get
|
||||||
|
NAME STATUS CREATED SCHEDULE BACKUP TTL LAST BACKUP SELECTOR
|
||||||
|
hc-weekly-backup Enabled 2019-05-08 17:51:31 +0200 CEST @weekly 720h0m0s 6s ago <none>
|
||||||
|
|
||||||
|
$ velero backup get
|
||||||
|
NAME STATUS CREATED EXPIRES STORAGE LOCATION SELECTOR
|
||||||
|
hc-weekly-backup-20190508155132 Completed 2019-05-08 17:51:32 +0200 CEST 29d default <none>
|
||||||
|
|
||||||
|
$ velero backup describe hc-weekly-backup-20190508155132 --details
|
||||||
|
# see if the persistent volumes are backed up
|
||||||
|
```
|
||||||
50
deployment/old/volumes/volume-snapshots/README.md
Normal file
50
deployment/old/volumes/volume-snapshots/README.md
Normal file
@ -0,0 +1,50 @@
|
|||||||
|
# Kubernetes Volume Snapshots
|
||||||
|
|
||||||
|
It is possible to backup persistent volumes through volume snapshots. This is
|
||||||
|
especially handy if you don't want to stop the database to create an [offline
|
||||||
|
backup](../neo4j-offline-backup/README.md) thus having a downtime.
|
||||||
|
|
||||||
|
Kubernetes announced this feature in a [blog post](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/). Please make yourself familiar with it before you continue.
|
||||||
|
|
||||||
|
## Create a Volume Snapshot
|
||||||
|
|
||||||
|
There is an example in this folder how you can e.g. create a volume snapshot for
|
||||||
|
the persistent volume claim `neo4j-data-claim`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# in folder deployment/volumes/volume-snapshots/
|
||||||
|
kubectl apply -f snapshot.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
If you are on Digital Ocean the volume snapshot should show up in the Web UI:
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
## Provision a Volume based on a Snapshot
|
||||||
|
|
||||||
|
Edit your persistent volume claim configuration and add a `dataSource` pointing
|
||||||
|
to your volume snapshot. [The blog post](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/) has an example in section "Provision a new volume from a snapshot with
|
||||||
|
Kubernetes".
|
||||||
|
|
||||||
|
There is also an example in this folder how the configuration could look like.
|
||||||
|
If you apply the configuration new persistent volume claim will be provisioned
|
||||||
|
with the data from the volume snapshot:
|
||||||
|
|
||||||
|
```
|
||||||
|
# in folder deployment/volumes/volume-snapshots/
|
||||||
|
kubectl apply -f neo4j-data.yaml
|
||||||
|
```
|
||||||
|
|
||||||
|
## Data Consistency Warning
|
||||||
|
|
||||||
|
Note that volume snapshots do not guarantee data consistency. Quote from the
|
||||||
|
[blog post](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/):
|
||||||
|
|
||||||
|
> Please note that the alpha release of Kubernetes Snapshot does not provide
|
||||||
|
> any consistency guarantees. You have to prepare your application (pause
|
||||||
|
> application, freeze filesystem etc.) before taking the snapshot for data
|
||||||
|
> consistency.
|
||||||
|
|
||||||
|
In case of Neo4J this probably means that enterprise edition is required which
|
||||||
|
supports [online backups](https://neo4j.com/docs/operations-manual/current/backup/).
|
||||||
|
|
||||||
Binary file not shown.
|
After Width: | Height: | Size: 118 KiB |
18
deployment/old/volumes/volume-snapshots/neo4j-data.yaml
Normal file
18
deployment/old/volumes/volume-snapshots/neo4j-data.yaml
Normal file
@ -0,0 +1,18 @@
|
|||||||
|
---
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
|
apiVersion: v1
|
||||||
|
metadata:
|
||||||
|
name: neo4j-data-claim
|
||||||
|
namespace: ocelot-social
|
||||||
|
labels:
|
||||||
|
app: ocelot-social
|
||||||
|
spec:
|
||||||
|
dataSource:
|
||||||
|
name: neo4j-data-snapshot
|
||||||
|
kind: VolumeSnapshot
|
||||||
|
apiGroup: snapshot.storage.k8s.io
|
||||||
|
accessModes:
|
||||||
|
- ReadWriteOnce
|
||||||
|
resources:
|
||||||
|
requests:
|
||||||
|
storage: 1Gi
|
||||||
10
deployment/old/volumes/volume-snapshots/snapshot.yaml
Normal file
10
deployment/old/volumes/volume-snapshots/snapshot.yaml
Normal file
@ -0,0 +1,10 @@
|
|||||||
|
---
|
||||||
|
apiVersion: snapshot.storage.k8s.io/v1alpha1
|
||||||
|
kind: VolumeSnapshot
|
||||||
|
metadata:
|
||||||
|
name: uploads-snapshot
|
||||||
|
namespace: ocelot-social
|
||||||
|
spec:
|
||||||
|
source:
|
||||||
|
name: uploads-claim
|
||||||
|
kind: PersistentVolumeClaim
|
||||||
Loading…
x
Reference in New Issue
Block a user