16 Commits

Author SHA1 Message Date
roschaefer
af3bdaef52 Don't create accounts for unverified users 2019-09-17 19:16:56 +02:00
roschaefer
cce4ce07df Increase default split size a little 2019-09-05 12:52:20 +02:00
mattwr18
60659626c8 Merge branch 'master' of github.com:Human-Connection/Human-Connection into update-legacy-migration-config 2019-09-05 08:11:01 +02:00
roschaefer
66ed78e19b Remove a lot of unused code, reduce graphql schema 2019-09-05 03:20:32 +02:00
mattwr18
17941678b1 Update export syntax, update maintenance worker/neo4j config
- I was getting an error locally  Failed: error parsing query as Extended JSON: invalid JSON input
- updating the syntax to wrap the json in single quotes, and key/values
in double quotes fixed the issue
- update memory limits
2019-09-04 16:53:13 +02:00
Matt Rider
907233cbc9 Increase nginx client max body size, split size legacy migration
- allows the client to send files at least the size we agreed in the vue-dropzone
- we don't need such a small split size as we don't run it on the remote server
2019-08-27 18:03:50 +02:00
roschaefer
bb4b3196a7 Fix various issues in legacy db import
Most important: Remove `#` symbol from Tags and snakeCase them.
2019-08-16 00:29:42 +02:00
aa54433afb
improved accuracy of import to reflect different types of contributions & follows. Cleaned neo4j import folder structure 2019-06-19 13:35:21 +02:00
Robert Schäfer
df70ae451c Parameterize split size
@ulfgebhardt that was a really good suggestion, thanks!
2019-06-03 13:11:12 +02:00
Robert Schäfer
115022fc2e Decrease split size
This might take longer but will require less memory. I guess this might
work even on our kubernetes cluster.
2019-05-31 18:38:58 +02:00
Robert Schäfer
f9ac22e560 Add executable UNIX permissions to export script
I encourage @ulfgebhardt to run the following command once:

```
SH_USERNAME=ulf SSH_HOST=***** MONGODB_USERNAME='hc-api' MONGODB_PASSWORD=***** MONGODB_DATABASE=hc_api MONGODB_AUTH_DB=admin UPLOADS_DIRECTORY=/data/api/uploads docker-compose -f docker-compose.maintenance.yml up --build
```

Once you're done with everything. You don't have to run docker for
development, but this procedure would ensure docker environment works
as expected.
2019-05-28 18:14:21 +02:00
4b5138880d
missing default .env file 2019-05-28 17:07:54 +02:00
301e7fa60c
fixed data export for alpha:
- include all collections
- refactored mongodb export script
- renamed to export
- use of .env file for (additional) configurations / configuration overrides
2019-05-28 17:03:44 +02:00
Robert Schäfer
b5d91cffef Implement @appinteractive's suggestions
This:
https://github.com/Human-Connection/Human-Connection/pull/529#discussion_r280065855
2019-05-07 19:15:47 +02:00
Robert Schäfer
497f77ae10 Breakthrough! Use split+indices for performance
@appinteractive thanks for pointing out `split`. You just saved me some
days of work to refactor the import statements to use CSV instead of
JSON files.

@Tirokk when I enter `:schema` in Neo4J web UI, I see the following:
```
:schema
Indexes
   ON :Badge(id) ONLINE
   ON :Category(id) ONLINE
   ON :Comment(id) ONLINE
   ON :Post(id) ONLINE
   ON :Tag(id) ONLINE
   ON :User(id) ONLINE

No constraints
```

So I temporarily removed the unique constraints on `slug` and added
plain indices on `id` for all relevant node types. We cannot omit the
`:Label` unfortunately, neo4j does not allow this. So I had to add all
indices for all known node labels instead.

With indices the import finishes in:
```
Time elapsed: 351 seconds
```
🎉

@appinteractive when I keep the unique indices on slug, I get an error
during import that a node with label `:User` and slug `tobias` already
exists. Ie. we have unqiue constraint violations in our production data.

@mattwr18 @ulfgebhardt @ogerly I started the application on my machine
on the production data and it turns out that the index page
http://localhost:3000/ takes way to long. Visiting my profile page at
http://localhost:3000/profile/5b1693daf850c11207fa6109/robert-schafer
is fine, though. Even pagination works. When I visit a post page with
not too many comments, the application is fast enough, too:
http://localhost:3000/post/5bbf49ebc428ea001c7ca89c/neues-video-format-human-connection-tech-news
2019-05-01 12:25:28 +02:00
Robert Schäfer
6ed5ad58d5 Merge maintenance with deployment 2019-04-24 00:27:58 +02:00