Robert Schäfer 497f77ae10 Breakthrough! Use split+indices for performance
@appinteractive thanks for pointing out `split`. You just saved me some
days of work to refactor the import statements to use CSV instead of
JSON files.

@Tirokk when I enter `:schema` in Neo4J web UI, I see the following:
```
:schema
Indexes
   ON :Badge(id) ONLINE
   ON :Category(id) ONLINE
   ON :Comment(id) ONLINE
   ON :Post(id) ONLINE
   ON :Tag(id) ONLINE
   ON :User(id) ONLINE

No constraints
```

So I temporarily removed the unique constraints on `slug` and added
plain indices on `id` for all relevant node types. We cannot omit the
`:Label` unfortunately, neo4j does not allow this. So I had to add all
indices for all known node labels instead.

With indices the import finishes in:
```
Time elapsed: 351 seconds
```
🎉

@appinteractive when I keep the unique indices on slug, I get an error
during import that a node with label `:User` and slug `tobias` already
exists. Ie. we have unqiue constraint violations in our production data.

@mattwr18 @ulfgebhardt @ogerly I started the application on my machine
on the production data and it turns out that the index page
http://localhost:3000/ takes way to long. Visiting my profile page at
http://localhost:3000/profile/5b1693daf850c11207fa6109/robert-schafer
is fine, though. Even pagination works. When I visit a post page with
not too many comments, the application is fast enough, too:
http://localhost:3000/post/5bbf49ebc428ea001c7ca89c/neues-video-format-human-connection-tech-news
2019-05-01 12:25:28 +02:00

16 lines
538 B
SQL

CALL apoc.load.json('file:/tmp/mongo-export/splits/current-chunk.json') YIELD value as json
MERGE (comment:Comment {id: json._id["$oid"]})
ON CREATE SET
comment.content = json.content,
comment.contentExcerpt = json.contentExcerpt,
comment.deleted = json.deleted,
comment.disabled = false
WITH comment, json, json.contributionId as postId
MATCH (post:Post {id: postId})
WITH comment, post, json.userId as userId
MATCH (author:User {id: userId})
MERGE (comment)-[:COMMENTS]->(post)
MERGE (author)-[:WROTE]->(comment)
;