20 Commits

Author SHA1 Message Date
Robert Schäfer
9656bd4eb7 Fix comments import
@appinteractive
2019-02-05 20:02:59 +01:00
Robert Schäfer
0705ab1bf5 Fix missing contribution author
Bug: Only if a tag and a category was present, the author was assigned.
2019-02-05 14:48:13 +01:00
Robert Schäfer
82c679aaf1 Check the environment variables early on
I was missing NEO4J_URI and was told very late - let's check env vars at
the very beginning.
2019-02-04 21:42:44 +01:00
Robert Schäfer
5b674cd95f Hopefully fix migrate on Digital Ocean
It kept crashing, maybe because we're piping too much stuff to STDOUT.
Let's see.
2019-02-04 21:33:48 +01:00
Robert Schäfer
2bc1e88920 Make CodeFactor happy 2019-02-04 19:55:59 +01:00
Robert Schäfer
084f610ec8 Turnaround: *use* volumes for sharing data
... but use it in the same pod. It seems to be possible to have shared
volumes in the same pod:
https://kubernetes.io/docs/tasks/access-application-cluster/communicate-containers-same-pod-shared-volume/
2019-02-04 19:44:39 +01:00
Robert Schäfer
01a145be29 Container db-migration-worker need no volumes
You can simply run `docker-compose exec db-migration-worker migrate` and
that's it. Of course you have to provide all the environment variables.
2019-02-04 17:49:48 +01:00
Robert Schäfer
5492b282bc Refactoring: No uploads/ vol,access neo4j directly
Digital Ocean does not support shared directories. So we have to uploads
the images in `/uploads` via `kubectl cp` or something similar.
Likewise, it is not possible to share the exported mongodb .json files
with neo4j container. Therefore let's install `cypher-shell`, included
in `neo4j` package, to directly open a neo4j connection and bulk import
the data.
2019-02-04 14:31:40 +01:00
Robert Schäfer
20908cd062 Docker and .gitignore entire .ssh folder
On kubernetes, the ssh keys are added as a volume from a secret
2019-02-04 12:41:03 +01:00
Robert Schäfer
064e98943d Remove sub-directories in /mongo-export
On Digital Ocean a persistent volume has a folder `lost+found` by
default. This script stops throws error messages because it can't remove
that folder if we don't add `-r` flag.
2019-02-04 12:37:03 +01:00
Robert Schäfer
5bdc728028 Don't copy private SSH key into docker image
We can mount the .ssh at runtime. This will allow us to push the docker
image to dockerhub without exposing any secrects.
2019-01-28 17:28:52 +01:00
Robert Schäfer
2cab6fe9cf Replace scp with rsync
`rsync` is faster on multiple runs. The `--update` flag will only
download newer images and does not overwrite existing files. Thus, it will
only download missing images.
2019-01-22 12:48:06 +01:00
Robert Schäfer
509e4feeb9 Add workflow to download remote uploads
* remote uploads directory
* download content of uploads directory with `scp`
* share uploads volume with backend container
2019-01-22 12:48:06 +01:00
Robert Schäfer
8591816dfa Fix CodeFactor 2019-01-19 00:57:12 +01:00
Robert Schäfer
4534f1ae9e Add production server to known_hosts 2019-01-18 21:18:59 +01:00
Robert Schäfer
f0c18e2cca Clean up Dockerfile, add known_hosts file
@appinteractive it's troublesome to add the SSH private key via
environment variable. You have to convert newlines to spaces and convert
them back - which I think is error prone. I hope we can transfer the
private key file on to our deployed container later on.
2019-01-18 21:07:24 +01:00
Robert Schäfer
bfad621306 Disable automatic know-hosts for now
This will require user interaction when running the import script
2019-01-16 02:19:02 +01:00
Robert Schäfer
433cd7a52e Export all relevant collection to a shared folder 2019-01-16 02:10:42 +01:00
Robert Schäfer
1c04b8f3de Import mongodb on-the-fly 2019-01-16 02:10:42 +01:00
Robert Schäfer
0970014a59 Create db-migration-worker as a docker container
The idea is to import/dump the remote database via SSH, restore it to
the local mongodb, export .json collections to a shared volume and
import the json collections with cypher-shell.
2019-01-16 02:10:42 +01:00