61 Commits

Author SHA1 Message Date
mattwr18
f275799856
Merge pull request #1351 from Human-Connection/1333-fix_checks_for_deleted_and_disabled
Fix database import, pagination and counters on user profile page
2019-08-30 13:51:41 +02:00
roschaefer
e4d57f80aa Fix #1333
Ok, so here are multiple issues:
1. In cypher, `NOT NULL` will return `NULL` not `FALSE`. If we want
`FALSE` to be set in the database import, we should use `COAELESCE`
to find the first not-null value.
See:
https://neo4j.com/docs/cypher-manual/current/syntax/working-with-null/
https://markhneedham.com/blog/2017/02/22/neo4j-null-values-even-work/

2. I removed the `disabled` and `deleted` checks on the commented
counter. With `neo4j-graphql-js` it is not possible to filter on the
join models (at least not without a lot of complexity) for disabled or
deleted items. Let's live with the fact that the list of commented posts
will include those posts, where the user has deleted his comment or where
the user's comment was disabled. It's being displayed as "not available"
so I think this is OK for now.

3. De-couple the pagination counters from the "commented", "shouted"
etc. counters. It might be that the list of posts is different for
different users. E.g. if the user has blocked you, the "posts" list
will be empty. The "shouted" or "commented" list will not have the
posts of the author. If you are a moderator, the list will include
disabled posts. So the counters are not in sync with the actual list
coming from the backend. Therefore I implemented "fetch and check if
resultSet < pageSize" instead of a global counter.
2019-08-28 12:57:32 +02:00
Matt Rider
907233cbc9 Increase nginx client max body size, split size legacy migration
- allows the client to send files at least the size we agreed in the vue-dropzone
- we don't need such a small split size as we don't run it on the remote server
2019-08-27 18:03:50 +02:00
mattwr18
90e5579cab
Merge pull request #1278 from Human-Connection/fix_hashtag_db_import
Fix hashtag db import
2019-08-16 14:37:09 +02:00
roschaefer
bb4b3196a7 Fix various issues in legacy db import
Most important: Remove `#` symbol from Tags and snakeCase them.
2019-08-16 00:29:42 +02:00
roschaefer
d3b256e99e Fix user mention menu on production data
Apparently some of our users in production have no slug, this will
assign an (hopefully unique) random slug to them.
2019-08-15 21:07:23 +02:00
Matt Rider
ee93efd6d9 Localise categories 2019-08-14 13:57:39 +02:00
Matt Rider
08504ad2ff Gitignore ssh directory 2019-07-29 08:36:16 +02:00
Robert Schäfer
904142cf6e Create a separate node for email in data import 2019-07-15 18:35:38 +02:00
Robert Schäfer
43a2c502c9 Fix regression of data import 2019-07-14 01:43:44 +02:00
mattwr18
4c91d8fbc5
Merge pull request #1016 from Human-Connection/277_reward_badges
Refactor reward/unreward Badges in backend
2019-07-13 10:08:52 -03:00
Robert Schäfer
99740e6ad6 Follow @Tirokk's suggestion and rename key=>id 2019-07-13 14:27:19 +02:00
44205e7268
delete follows correctly 2019-07-01 16:18:49 +02:00
760f899379
import emotions 2019-07-01 16:18:14 +02:00
5653242c32
Merge branch 'master' into 2019/kw25/improve_import_accuracy 2019-06-24 12:16:05 +02:00
c14a4f3134
fixed deleted flag import 2019-06-24 12:06:09 +02:00
9cbb736d70
cleanup import todo 2019-06-19 13:54:20 +02:00
aa54433afb
improved accuracy of import to reflect different types of contributions & follows. Cleaned neo4j import folder structure 2019-06-19 13:35:21 +02:00
98dbf801ab
Merge branch 'master' into 2019/kw24/fix_imported_img_urls
# Conflicts:
#	backend/src/middleware/fixImageUrlsMiddleware.js
#	backend/src/middleware/fixImageUrlsMiddleware.spec.js
2019-06-17 13:24:29 +02:00
65df4c5a20
use replace instead of substring 2019-06-17 12:16:15 +02:00
bc83e6bfd9
configurable output folder, local .env file to override config and define local environment 2019-06-14 14:13:41 +02:00
d024d7df91
fixed importing of urls - remove url prefix 2019-06-14 14:03:21 +02:00
Robert Schäfer
df70ae451c Parameterize split size
@ulfgebhardt that was a really good suggestion, thanks!
2019-06-03 13:11:12 +02:00
Robert Schäfer
f9e9aa47b7 Disable collections without import statement
Even if the import statement does literally nothing, this slows down the
import. Let's avoid this until we do sth. with the data.
2019-05-31 20:14:08 +02:00
Robert Schäfer
115022fc2e Decrease split size
This might take longer but will require less memory. I guess this might
work even on our kubernetes cluster.
2019-05-31 18:38:58 +02:00
Robert Schäfer
9a9118c721 Merge remote-tracking branch 'origin/master' into 2019/kw22/alpha_data_import_status_schema_split 2019-05-31 16:26:06 +02:00
Robert Schäfer
58add8fc5f Install envsubst in Dockerfile
@ulfgebhardt please setup docker on your machine or a remote machine.

Installing `envsubst` on alpine fails with circular dependencies
(awkward). So this repo here has a solution:
https://github.com/cirocosta/alpine-envsubst/blob/master/Dockerfile#L6
2019-05-31 16:26:00 +02:00
Robert Schäfer
10864e7d18 Remove command line arguments -u and -p
@ulfgebhardt: The docs at `man cypher-shell` say that you can pass
`NEO4J_USERNAME` and `NEO4J_PASSWORD` per environment variable. So the
command line arguments are obsolete here.
2019-05-31 16:26:00 +02:00
37be5481c0
remaining legacy table descriptions and dummy import scripts 2019-05-29 20:43:58 +02:00
355205028f
removed debug output 2019-05-29 20:04:03 +02:00
271af7dde2
keep index file for already imported files, do not reimport already imported file, invites, notifications, emotions and organisations scripts and descriptions 2019-05-29 20:02:30 +02:00
968578f10a
delete cql files 2019-05-29 17:30:46 +02:00
816fbdf7cd
fixed slug required - it was not in Nitro 2019-05-29 16:41:27 +02:00
Robert Schäfer
58224381a8 Add missing environment var to maintenance-worker 2019-05-29 16:12:03 +02:00
Robert Schäfer
a83aad3f60 Fix wrong path in Dockerfile 2019-05-29 16:00:45 +02:00
67595f6400
missing change for windows comment 2019-05-29 15:45:32 +02:00
7e44667dd1
modified .env file for neo4j import since its had problems with file locks on windows (cypher-shell) 2019-05-29 15:45:02 +02:00
6f0447515a
- fixed several errors handling import
- split graphql schema into parts
- which data is imported which is not - a list
2019-05-29 15:37:28 +02:00
Robert Schäfer
e1a113e7e4 Fix wrong mountpath
We're saving the files to /uploads. If the maintenance-worker does not
mount the uploads persistent volume there, we don't get persistent
files.
2019-05-29 15:16:08 +02:00
Robert Schäfer
2c8dcaa592 Yet another typo 2019-05-29 15:08:14 +02:00
Robert Schäfer
95fe115198 Fix typo 2019-05-29 15:01:56 +02:00
860a0d41d0
Merge pull request #697 from Human-Connection/2019/kw22/alpha_data_import
🍰 2019/kw22/alpha_data_import
2019-05-29 00:43:10 +02:00
4ccfe3822c
added existing data imports 2019-05-29 00:20:43 +02:00
Robert Schäfer
ab7a4ffc3e
Fix duplicate tags by using the name as the id
@ulfgebhardt: I wondered about the list of tags after importing the
legacy db. It seems, each tag has at most 1 contribution. I guess it's
because we create a unique id for each tag, so two tags with the same
`name` e.g. `#hashtag` and `#hashtag` are not de-duplicated.

I'm currently sitting in the train and cannot run the data import myself, could
you double-check?
2019-05-28 22:41:51 +02:00
014104e055
fixed data import from alpha data:
- include all collections (commented out)
- refactored neo4j import script
- use of .env file for (additional) configurations / configuration overrides
- lots of fiddeling with neo4j cql files and cypher shell
2019-05-28 18:53:05 +02:00
Robert Schäfer
f9ac22e560 Add executable UNIX permissions to export script
I encourage @ulfgebhardt to run the following command once:

```
SH_USERNAME=ulf SSH_HOST=***** MONGODB_USERNAME='hc-api' MONGODB_PASSWORD=***** MONGODB_DATABASE=hc_api MONGODB_AUTH_DB=admin UPLOADS_DIRECTORY=/data/api/uploads docker-compose -f docker-compose.maintenance.yml up --build
```

Once you're done with everything. You don't have to run docker for
development, but this procedure would ensure docker environment works
as expected.
2019-05-28 18:14:21 +02:00
4b5138880d
missing default .env file 2019-05-28 17:07:54 +02:00
301e7fa60c
fixed data export for alpha:
- include all collections
- refactored mongodb export script
- renamed to export
- use of .env file for (additional) configurations / configuration overrides
2019-05-28 17:03:44 +02:00
Robert Schäfer
dfef37b3f6 Prevent argument list too long error 2019-05-08 00:46:07 +02:00
Robert Schäfer
bcc2c4dbbb Configure scripts and docker-compose.yml
After endless try/error I found the way to share volumes between
multiple docker-compose.ymls: You have to place those files in the same
folder. Also the import scripts must be adapted.
2019-05-07 21:48:09 +02:00