diff --git a/SUMMARY.md b/SUMMARY.md index e3503e48d..4ee2bd3c6 100644 --- a/SUMMARY.md +++ b/SUMMARY.md @@ -27,9 +27,10 @@ * [HTTPS](deployment/digital-ocean/https/README.md) * [Human Connection](deployment/human-connection/README.md) * [Volumes](deployment/volumes/README.md) - * [Disaster Recovery](deployment/disaster-recovery/README.md) - * [Neo4J Offline-Backups](deployment/disaster-recovery/neo4j-offline-backup/README.md) - * [On-Premise Backups](deployment/disaster-recovery/velero/README.md) + * [Neo4J Offline-Backups](deployment/volumes/neo4j-offline-backup/README.md) + * [Volume Snapshots](deployment/volumes/volume-snapshots/README.md) + * [Reclaim Policy](deployment/volumes/reclaim-policy/README.md) + * [Velero](deployment/volumes/velero/README.md) * [Neo4J DB Backup](deployment/backup.md) * [Legacy Migration](deployment/legacy-migration/README.md) * [Feature Specification](cypress/features.md) diff --git a/deployment/disaster-recovery/README.md b/deployment/disaster-recovery/README.md deleted file mode 100644 index b6b2cd51c..000000000 --- a/deployment/disaster-recovery/README.md +++ /dev/null @@ -1,8 +0,0 @@ -# Disaster Recovery - -We tested a couple of options how to do disaster recovery in kubernetes. First, -there is the [offline backup strategy](./neo4j-offline-backup/README.md) of the -community edition of Neo4J, which you can also run on a local installation. -Next up is a kubernetes specific disaster recovery tool called -[Velero](./velero/README.md). - diff --git a/deployment/volumes/README.md b/deployment/volumes/README.md index b838794d5..2d08a34cb 100644 --- a/deployment/volumes/README.md +++ b/deployment/volumes/README.md @@ -7,7 +7,13 @@ At the moment, the application needs two persistent volumes: As a matter of precaution, the persistent volume claims that setup these volumes live in a separate folder. You don't want to accidently loose all your data in -your database by running `kubectl delete -f human-connection/`, do you? +your database by running + +```sh +kubectl delete -f human-connection/ +``` + +or do you? ## Create Persistent Volume Claims @@ -19,24 +25,12 @@ persistentvolumeclaim/neo4j-data-claim created persistentvolumeclaim/uploads-claim created ``` -## Change Reclaim Policy +## Backup and Restore -We recommend to change the `ReclaimPolicy`, so if you delete the persistent -volume claims, the associated volumes will be released, not deleted: - -```sh -$ kubectl --namespace=human-connection get pv - -NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE -pvc-bd02a715-66d0-11e9-be52-ba9c337f4551 1Gi RWO Delete Bound human-connection/neo4j-data-claim do-block-storage 4m24s -pvc-bd208086-66d0-11e9-be52-ba9c337f4551 2Gi RWO Delete Bound human-connection/uploads-claim do-block-storage 4m12s -``` - -Get the volume id from above, then change `ReclaimPolicy` with: -```sh -kubectl patch pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' - -# in the above example -kubectl patch pv pvc-bd02a715-66d0-11e9-be52-ba9c337f4551 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' -kubectl patch pv pvc-bd208086-66d0-11e9-be52-ba9c337f4551 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' -``` +We tested a couple of options how to do disaster recovery in kubernetes. First, +there is the [offline backup strategy](./neo4j-offline-backup/README.md) of the +community edition of Neo4J, which you can also run on a local installation. +Kubernetes also offers so-called [volume snapshots](./volume-snapshots/README.md). +Changing the [reclaim policy](./reclaim-policy/README.md) of your persistent +volumes might be an additional safety measure. Finally, there is also a +kubernetes specific disaster recovery tool called [Velero](./velero/README.md). diff --git a/deployment/disaster-recovery/neo4j-offline-backup/README.md b/deployment/volumes/neo4j-offline-backup/README.md similarity index 100% rename from deployment/disaster-recovery/neo4j-offline-backup/README.md rename to deployment/volumes/neo4j-offline-backup/README.md diff --git a/deployment/volumes/reclaim-policy/README.md b/deployment/volumes/reclaim-policy/README.md new file mode 100644 index 000000000..00c91c319 --- /dev/null +++ b/deployment/volumes/reclaim-policy/README.md @@ -0,0 +1,30 @@ +# Change Reclaim Policy + +We recommend to change the `ReclaimPolicy`, so if you delete the persistent +volume claims, the associated volumes will be released, not deleted. + +This procedure is optional and an additional security measure. It might prevent +you from loosing data if you accidently delete the namespace and the persistent +volumes along with it. + +```sh +$ kubectl --namespace=human-connection get pv + +NAME CAPACITY ACCESS MODES RECLAIM POLICY STATUS CLAIM STORAGECLASS REASON AGE +pvc-bd02a715-66d0-11e9-be52-ba9c337f4551 1Gi RWO Delete Bound human-connection/neo4j-data-claim do-block-storage 4m24s +pvc-bd208086-66d0-11e9-be52-ba9c337f4551 2Gi RWO Delete Bound human-connection/uploads-claim do-block-storage 4m12s +``` + +Get the volume id from above, then change `ReclaimPolicy` with: +```sh +kubectl patch pv -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' + +# in the above example +kubectl patch pv pvc-bd02a715-66d0-11e9-be52-ba9c337f4551 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' +kubectl patch pv pvc-bd208086-66d0-11e9-be52-ba9c337f4551 -p '{"spec":{"persistentVolumeReclaimPolicy":"Retain"}}' +``` + +Given that you changed the reclaim policy as described above, you should be able +to create a persistent volume claim based on a volume snapshot content. See +the general kubernetes documentation [here](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/) +and our specific documentation for snapshots [here](../snapshot/README.md). diff --git a/deployment/disaster-recovery/velero/README.md b/deployment/volumes/velero/README.md similarity index 100% rename from deployment/disaster-recovery/velero/README.md rename to deployment/volumes/velero/README.md diff --git a/deployment/volumes/volume-snapshots/README.md b/deployment/volumes/volume-snapshots/README.md new file mode 100644 index 000000000..47f71530a --- /dev/null +++ b/deployment/volumes/volume-snapshots/README.md @@ -0,0 +1,48 @@ +# Kubernetes Volume Snapshots + +It is possible to backup persistent volumes through volume snapshots. This is +especially handy if you don't want to stop the database to create an [offline +backup](../neo4j-offline-backup/README.md) thus having a downtime. + +## Create a Volume Snapshot + +There is an example in this folder how you can e.g. create a volume snapshot for +the persistent volume claim `neo4j-data-claim`: + +```sh +# in folder deployment/volumes/volume-snapshots/ +kubectl apply -f snapshot.yaml +``` + +If you are on Digital Ocean the volume snapshot should show up in the Web UI: + +![Digital Ocean Web UI showing a volume snapshot](./digital-ocean-volume-snapshots.png) + +## Provision a Volume based on a Snapshot + +Edit your persistent volume claim configuration and add a `dataSource` pointing +to your volume snapshot. [This blog post](https://kubernetes.io/blog/2018/10/09/introducing-volume-snapshot-alpha-for-kubernetes/) has an example in section "Provision a new volume from a snapshot with +Kubernetes". + +There is also an example in this folder how the configuration could look like. +If you apply the configuration new persistent volume claim will be provisioned +with the data from the volume snapshot: + +``` +# in folder deployment/volumes/volume-snapshots/ +kubectl apply -f neo4j-data.yaml +``` + +## Data Consistency Warning + +Note that volume snapshots do not guarantee data consistency. Quote from the +blog post above: + +> Please note that the alpha release of Kubernetes Snapshot does not provide +> any consistency guarantees. You have to prepare your application (pause +> application, freeze filesystem etc.) before taking the snapshot for data +> consistency. + +In case of Neo4J this probably means that enterprise edition is required which +supports [online backups](https://neo4j.com/docs/operations-manual/current/backup/). + diff --git a/deployment/volumes/volume-snapshots/digital-ocean-volume-snapshots.png b/deployment/volumes/volume-snapshots/digital-ocean-volume-snapshots.png new file mode 100644 index 000000000..cb6599616 Binary files /dev/null and b/deployment/volumes/volume-snapshots/digital-ocean-volume-snapshots.png differ diff --git a/deployment/volumes/volume-snapshots/neo4j-data.yaml b/deployment/volumes/volume-snapshots/neo4j-data.yaml new file mode 100644 index 000000000..7de9e19dc --- /dev/null +++ b/deployment/volumes/volume-snapshots/neo4j-data.yaml @@ -0,0 +1,18 @@ +--- + kind: PersistentVolumeClaim + apiVersion: v1 + metadata: + name: neo4j-data-claim + namespace: human-connection + labels: + app: human-connection + spec: + dataSource: + name: neo4j-data-snapshot + kind: VolumeSnapshot + apiGroup: snapshot.storage.k8s.io + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 1Gi diff --git a/deployment/volumes/snapshots.yaml b/deployment/volumes/volume-snapshots/snapshot.yaml similarity index 86% rename from deployment/volumes/snapshots.yaml rename to deployment/volumes/volume-snapshots/snapshot.yaml index 9415a331d..3c3487e14 100644 --- a/deployment/volumes/snapshots.yaml +++ b/deployment/volumes/volume-snapshots/snapshot.yaml @@ -2,7 +2,7 @@ apiVersion: snapshot.storage.k8s.io/v1alpha1 kind: VolumeSnapshot metadata: - name: neo4j-data-snapshot-2 + name: neo4j-data-snapshot namespace: human-connection spec: source: