From 6d80eabb0eff39e7f0d3a829d6fcc3f4a235ee66 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Claus-Peter=20H=C3=BCbner?= Date: Fri, 29 Apr 2022 03:58:36 +0200 Subject: [PATCH] more details in stage 2 - Authentication --- .gitignore | 2 + .../TechnicalRequirements/Federation.md | 162 ++++++++++-------- 2 files changed, 96 insertions(+), 68 deletions(-) diff --git a/.gitignore b/.gitignore index b02b9d6ec..32e11f545 100644 --- a/.gitignore +++ b/.gitignore @@ -1,3 +1,5 @@ +.dbeaver +.project *.log /node_modules/* messages.pot diff --git a/docu/Concepts/TechnicalRequirements/Federation.md b/docu/Concepts/TechnicalRequirements/Federation.md index f86a94fb3..959aa8afe 100644 --- a/docu/Concepts/TechnicalRequirements/Federation.md +++ b/docu/Concepts/TechnicalRequirements/Federation.md @@ -27,7 +27,7 @@ The Variant A with an internal server contains the benefit to be as independent The Varaint B with an external server contains the benefit to reduce the implementation efforts and the responsibility for an own ActivitPub-Server. But it will cause an additional dependency to a third party service provider and the growing hosting costs. -## HyperSwarm {HyperSwarm} +## HyperSwarm The decision to switch from ActivityPub to HyperSwarm base on the arguments, that the *hyperswarm/dht* library will satify the most federation requirements out of the box. It is now to design the business requirements of the [gradido community communication](../BusinessRequirements/CommunityVerwaltung.md#UC-createCommunity) in a technical conception. @@ -41,23 +41,23 @@ To enable such a relationship between an existing community and a new community 2. Authentication 3. Autorized Communication -### Overview {Overview} +### Overview At first the following diagramm gives an overview of the three stages and shows the handshake between an existing community-A and a new created community-B including the data exchange for buildup such a federated, authenticated and autorized relationship. ![FederationHyperSwarm.png](./image/FederationHyperSwarm.png) -### Prerequisits {Prerequisits} +### Prerequisits Before starting in describing the details of the federation handshake, some prerequisits have to be defined. -#### Database {Database} +#### Database With the federation additional data tables/entities have to be created. ##### Community-Entity -Create the new *Community* table to store attributes of the own community. This table is used more like a frame for lists of federated foreign communities, users, AUF- and Welfare-account and the profile data of the own community: +Create the new *Community* table to store attributes of the own community. This table is used more like a frame for own community data in the future like the list of federated foreign communities, own users, own futher accounts like AUF- and Welfare-account and the profile data of the own community: | Attributes | Type | Description | | ----------- | ------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | @@ -83,7 +83,6 @@ Create the new *CommunityFederation* table to store at this point of time only t | authenticatedAt | timestamp | the timestamp of the last successfull authentication with this foreign community | | | | for the near future additional attributes will follow with the next level of Multi-Community Readyness | - ##### CommunityApiVersion-Entity Create the new *CommunityApiVersion* table to support several urls and apiversion of one community at once. It references the table *CommunityFederation* with the foreignkey *communityFederationID* (naming pattern foreignkea = `ID`) for a 1:n relationship @@ -97,9 +96,7 @@ Create the new *CommunityApiVersion* table to support several urls and apiversio | validFrom | timestamp | the timestamp as of the url and api are provided by the community | | verifiedAt | timestamp | the timestamp the url and apiversion of this foreign community is verified (for the own community its always null) | - - -#### Configuration {Configuration} +#### Configuration The preparation of a community infrastructure needs some application-outside configuration, which will be read during the start phase of the gradido components. The configuration will be defined in a file based manner like key-value-pairs as properties or in a structured way like xml or json files. @@ -116,7 +113,7 @@ The preparation of a community infrastructure needs some application-outside con | federation.dhtnode.host | | | defines the host where the DHT-Node is hosted, if outside apollo | | federation.dhtnode.port | | | defines the port on which the DHT-node will provide its services, if outside apollo | -#### 1st Start of a community {1st Start of Community} +#### 1st Start of a community The first time a new community infrastructure on a server is started, the start-phase has to check and prepair the own community database for federation. That means the application has to read the configuration and check against the database, if all current configured data is propagated in the database especially in the *CommunityXXX* entities. @@ -124,7 +121,7 @@ The first time a new community infrastructure on a server is started, the start- * community.id = next sequence value * community.uuid = generated UUID (version 4) - * community.name = federation.communityname + * community.name = Configuration.federation.communityname * community.description = null * prepare the *CommunityFederation* table @@ -140,31 +137,31 @@ The first time a new community infrastructure on a server is started, the start- * communityApiVersion.id = next sequence value * communityApiVersion.communityFederationID = communityFederation.id - * communityApiVersion.url = federation.apiversion.``.url - * communityApiVersion.apiversion = federation.apiversion - * communityApiVersion.validFrom = federation.apiversion.``.validFrom + * communityApiVersion.url = Configuration.federation.apiversion.``.url + * communityApiVersion.apiversion = Configuration.federation.apiversion + * communityApiVersion.validFrom = Configuration.federation.apiversion.``.validFrom * communityApiVersion.verifiedAt = null -### Stage1 - Federation {Stage1 - Federation} +### Stage1 - Federation -For the 1st stage the *hyperswarm dht library* will be used. It supports an easy way to connect a new community with other existing communities. As shown in the picture above the *hyperswarm dht library* will be part of the component *DHT-Node* separated from the *apollo server* component. The background for this separation is to keep off the federation activity from the business processes or to enable component specific scaling in the future. In consequence for the inter-component communication between *DHT-Node*, *apollo server* and other components like *database* the interface and security has to be defined. +For the 1st stage the *hyperswarm dht library* will be used. It supports an easy way to connect a new community with other existing communities. As shown in the picture above the *hyperswarm dht library* will be part of the component *DHT-Node* separated from the *apollo server* component. The background for this separation is to keep off the federation activity from the business processes or to enable component specific scaling in the future. In consequence for the inter-component communication between *DHT-Node*, *apollo server* and other components like *database* the interface and security has to be defined during development on using technical standards. -For the first federation release the DHT-node will be part of the apollo server, but internally designed and implemented as a logical saparated component. +For the first federation release the *DHT-Node* will be part of the *apollo server*, but internally designed and implemented as a logical saparated component. -#### Sequence join&connect {join&connect} +#### Sequence join&connect 1. In the own database of community_A the entites *Community*, *CommunityFederation* and *CommunityApiVersion* are initialized 2. When starting the *DHT-Node* of community_A it search per *apollo-ORM* for the own community entry and check on existing keypair *CommunityFederation.pubKey* and *CommunityFederation.privateKey* in the database. If they not exist, the *DHT-Node* generates the keypair *pubkey* and *privatekey* and writes them per *apollo-ORM* in the database 3. For joining with the correct channel of *hyperswarm dht* a topic has to be used. The *DHT-Node* reads the configured value of the property *federation.dhtnode.topic*. 4. with the *CommunityFederation.pubKey* and the *federation.dhtnode.topic* the *DHT-Node* joins the *hyperswarm dht* and listen for other *DHT-nodes* on the topic. -5. As soon as a the *hyperswarm dht* notifies an additional node in the topic the *DHT-node* reads the *pubKey* of this additional node and search it per *apollo-ORM* in the *CommunityFederation* table by filtering with *CommunityFederation.foreign* = TRUE -6. if an entry with the C*ommunityFederation.pubKey* of the additional node still exists and the *CommunityFederation.pubKeyVerifiedAt* is not NULL the *DHT-node* do nothing and join and listen again on the topic -7. if an entry with the *CommunityFederation.pubKey* of the additional node can't be found, the *DHT-Node* starts with the next step *direct exchange* of the federation handshake +5. As soon as a the *hyperswarm dht* notifies an additional node in the topic, the *DHT-node* reads the *pubKey* of this additional node and search it per *apollo-ORM* in the *CommunityFederation* table by filtering with *CommunityFederation.foreign* = TRUE +6. if an entry with the C*ommunityFederation.pubKey* of the foreign node still exists and the *CommunityFederation.pubKeyVerifiedAt* is not NULL both the *DHT-node* and the foreign node had pass through the federation process before. Nevertheless the following steps and stages have to be processed for updating e.g the api versions or other meanwhile changed date. +7. if an entry with the *CommunityFederation.pubKey* of the additional node can't be found, the *DHT-Node* starts with the next step *direct exchange* of the federation handshake anyway. -#### Sequence direct exchange {direct exchange} +#### Sequence direct exchange -1. if the *CommunityFederation.pubKey* of the additional node does not exists in the *CommunityFederation* table the *DHT-node* starts a *direct exchange* with this additional node -2. the *DHT-node* opens a direct connection per hyperswarm with the additional node and exchange respectively the *url* and *apiversion* between each other. +1. if the *CommunityFederation.pubKey* of the additional node does not exists in the *CommunityFederation* table the *DHT-node* starts a *direct exchange* with this foreign node to gets the data the first time, otherwise to update previous exchanged data. +2. the *DHT-node* opens a direct connection per *hyperswarm* with the additional node and exchange respectively the *url* and *apiversion* between each other. 1. to support the future feature that one community can provide several urls and apiversion at once the exchanged data should be in a format, which can represent structured information, like JSON (or simply CSV - feel free how to implement, but be aware about security aspects to avoid possible attacks during parsing the exchanged data and the used parsers for it) ``` @@ -173,13 +170,13 @@ For the first federation release the DHT-node will be part of the apollo server, { "url" : "comB.com", "version" : "1.0", - "validFrom" : "2020.01.01" + "validFrom" : "2022.01.01" } "API" : { "url" : "comB.com", "version" : "1.1", - "validFrom" : "2020.04.15" + "validFrom" : "2022.04.15" } "API" : { @@ -189,82 +186,111 @@ For the first federation release the DHT-node will be part of the apollo server, } } ``` - 2. the *DHT-Node* writes per *apollo-ORM* the received and parsed data from the other node in the database + 2. the *DHT-Node* writes per *apollo-ORM* the received and parsed data from the foreign node in the database 1. For the future an optimization step will be introduced here to avoid possible attacks of a foreign node by polute our database with mass of data. 1. before the *apollo-ORM* writes the data in the database, the *apollo-graphQL* invokes for all received urls and apiversions at the foreign node the request https.//`//getPubKey()`. - 2. Normally the foreign node will repsonse in a very short time with its publicKey - 3. if such a request runs in a timeout, the previous exchanged data with the foreign node will be a fake and can be refused without storing in database. - 4. if the response is in time the received publicKey must be equals with the pubKey of the foreign node the DHT-Node gets from hyperswarm dht per topic before - 5. if both keys are the same, the writing of the exchanged data per apollo-ORM can go on. - 6. if both keys will not match the exchanged data during the direct connection will be a fake and can be refused without storing in database. - 2. the *apollo-ORM* write the received data as follow + 2. Normally the foreign node will response in a very short time with its publicKey, because there will be nothing to en- or decrypt or other complex processing steps. + 3. if such a request runs in a timeout anyhow, the previous exchanged data with the foreign node will be almost certainly a fake and can be refused without storing in database. Break the further federation processing steps and stages and return back to stage1 join&connect. + 4. if the response is in time the received publicKey must be equals with the pubKey of the foreign node the *DHT-Node* gets from *hyperswarm dht* per topic during the join&connect stage before + 5. if both keys are the same, the writing of the exchanged data per *apollo-ORM* can go on. + 6. if both keys will not match the exchanged data during the direct connection will be almost certainly a fake and can be refused without storing in database. Break the further federation processing steps and stages and return back to stage1 join&connect. + 2. the *apollo-ORM* inserts / updates or deletes the received data as follow - * insert in the *CommunityFederation* table for this foreign node: - * communityFederation.id = next sequence value - * communityFederation.uuid = null - * communityFederation.foreign = TRUE - * communityFederation.createdAt = NOW - * communityFederation.privateKey = null - * communityFederation.pubKey = exchangedData.pubKey - * communityFederation.pubKeyVerifiedAt = null - * communityFederation.authenticatedAt = null - * insert in the *CommunityApiVersion* table for all exchangedData APIs: - * communityApiVersion.id = next sequence value - * communityApiVersion.communityFederationID = communityFederation.id - * communityApiVersion.url = exchangedData.API.url - * communityApiVersion.apiversion = exchangedData.API.version - * communityApiVersion.validFrom = exchangedData.API.validFrom - * communityApiVersion.verifiedAt = null - 3. After all received data is stored successfully, the DHT-Node starts the *stage2 - Authentication* of the federation handshake + * insert/update in the *CommunityFederation* table for this foreign node: + + | Column | insert | update | + | ------------------------------------ | -------------------- | :------------------ | + | communityFederation.id | next sequence value | keep existing value | + | communityFederation.uuid | null | keep existing value | + | communityFederation.foreign | TRUE | keep existing value | + | communityFederation.createdAt | NOW | keep existing value | + | communityFederation.privateKey | null | keep existing value | + | communityFederation.pubKey | exchangedData.pubKey | keep existing value | + | communityFederation.pubKeyVerifiedAt | null | keep existing value | + | communityFederation.authenticatedAt | null | keep existing value | + * for each exchangedData API + + if API not exists in database then insert in the *CommunityApiVersion* table: + + | Column | insert | + | ----------------------------------------- | --------------------------- | + | communityApiVersion.id | next sequence value | + | communityApiVersion.communityFederationID | communityFederation.id | + | communityApiVersion.url | exchangedData.API.url | + | communityApiVersion.apiversion | exchangedData.API.version | + | communityApiVersion.validFrom | exchangedData.API.validFrom | + | communityApiVersion.verifiedAt | null | + + if API exists in database but was not part of the last data exchange, then delete it from the *CommunityApiVersion* table + + if API exists in database and was part of the last data exchange, then update it in the *CommunityApiVersion* table + + | Column | update | + | ----------------------------------------- | --------------------------- | + | communityApiVersion.id | keep existing value | + | communityApiVersion.communityFederationID | keep existing value | + | communityApiVersion.url | keep existing value | + | communityApiVersion.apiversion | keep existing value | + | communityApiVersion.validFrom | exchangedData.API.validFrom | + | communityApiVersion.verifiedAt | keep existing value | + * + 3. After all received data is stored successfully, the *DHT-Node* starts the *stage2 - Authentication* of the federation handshake ### Stage2 - Authentication -The 2nd stage of federation is called *authentication*, because during the 1st stage the *hyperswarm dht* only ensures the knowledge that one node is the owner of its keypairs *pubkKy* and *privateKey*. The exchanged data between two nodes during the *direct exchange* on the *hyperswarm dht channel* must be verified, means ensure if the proclaimed *url(s)* and *apiversion(s)* of a node is the correct address to reach the same node outside the hyperswarm infrastructure. +The 2nd stage of federation is called *authentication*, because during the 1st stage the *hyperswarm dht* only ensures the knowledge that one node is the owner of its keypairs *pubKey* and *privateKey*. The exchanged data between two nodes during the *direct exchange* on the *hyperswarm dht channel* must be verified, means ensure if the proclaimed *url(s)* and *apiversion(s)* of a node is the correct address to reach the same node outside the hyperswarm infrastructure. -As mentioned before the *DHT-node* invokes the *authentication* stage on *apollo server* *graphQL* with the previous stored data of the foreign node. +As mentioned before the *DHT-node* invokes the *authentication* stage on *apollo server* *graphQL* with the previous stored data of the foreign node. #### Sequence - view of existing Community 1. the authentication stage starts by reading for the *foreignNode* from the previous federation step all necessary data - 1. select with the *foreignNode.pubKey* from the tables *CommunityFederation* and *CommunityApiVersion* where *CommunityApiVersion.validFrom* <= NOW + 1. select with the *foreignNode.pubKey* from the tables *CommunityFederation* and *CommunityApiVersion* where *CommunityApiVersion.validFrom* <= NOW and *CommunityApiVersion.verifiedAt* = null 2. the resultSet will be a list of data with the following attributes * foreignNode.pubKey * foreignNode.url * foreignNode.apiVersion 2. read the own keypair and uuid by `select uuid, privateKey, pubKey from CommunityFederation cf where cf.foreign = FALSE` -3. for each entry of the resultSet do - 1. encryptedURL = encrypting the foreignNode.url and foreignNode.apiVersion with the foreignNode.pubKey - 2. signedAndEncryptedURL = sign the result of the encryption with the own privateKey - 3. invoke the request `https://///openConnection(own.pubKey, signedAndEncryptedURL )` + 4. the foreign node will response immediately with an empty response OK, otherwise break the authentication stage with an error 4. the foreign node will process the request on its side - see [description below](#Sequence - view of new Community) - and invokes a redirect request base on the previous exchanged data during stage1 - Federation. This could be more than one redirect request depending on the amount of supported urls and apiversions we propagate to the foreignNode before. -5. for each received request `https://///openConnectionRedirect(onetimecode, foreignNode.url, encryptedRedirectURL )` do 1. with the given parameter the following steps will be done - 1. search for the foreignNode.pubKey by `select cf.pubKey from CommunityApiVersion cav, CommunityFederation cf where cav.url = foreignNode.url and cav.communityFederationID = cf.id` + 1. search for the *foreignNode.pubKey* by `select cf.pubKey from CommunityApiVersion cav, CommunityFederation cf where cav.url = foreignNode.url and cav.communityFederationID = cf.id` 2. decrypt with the `own.privateKey` the received `encryptedRedirectURL` parameter, which contains a full qualified url inc. apiversion and route - 3. verify signature of `encryptedRedirectURL` with the previous found foreignNode.pubKey from the own database - 4. if the decryption and signature verification are successful then encrypt the own.uuid with the own.privateKey + 3. verify signature of `encryptedRedirectURL` with the previous found *foreignNode.pubKey* from the own database + 4. if the decryption and signature verification are successful then encrypt the *own.uuid* with the *own.privateKey* to *encryptedOwnUUID* 5. invoke the redirect request with https://`(onetimecode, encryptedOwnUUID)` and 6. wait for the response with the `encryptedForeignUUID` - 7. decrypt the `encrpytedForeignUUID` with the foreignNode.pubKey - 8. write the encrypted foreignNode.UUID in the database by updating the CommunityFederation table per `update CommunityFederation cf set values (cf.uuid = foreignNode.UUID, cf.pubKeyVerifiedAt = NOW) where cf.pubKey = foreignNode.pubkey` + 7. decrypt the `encrpytedForeignUUID` with the *foreignNode.pubKey* + 8. write the encrypted *foreignNode.UUID* in the database by updating the CommunityFederation table per `update CommunityFederation cf set values (cf.uuid = foreignNode.UUID, cf.pubKeyVerifiedAt = NOW) where cf.pubKey = foreignNode.pubkey` After all redirect requests are process, all relevant authentication data of the new community are well know here and stored in the database. +#### Sequence - view of new Community -#### Sequence - view of new Community {Sequence - view of new Community} - -ongoing +This chapter contains the description of the Authentication Stage on the new community side as the request `openConnection(pubKey, signedAndEncryptedURL)` +As soon the *openConnection* request is invoked: +1. decrypted the 2nd `parameter.signedAndEncryptedURL` with the own *privatKey* +2. with the 1st parameter *pubKey* search in the own database `select uuid, url, pubKey from CommunityFederation cf where cf.foreign = TRUE and cf.pubKey = parameter.pubKey` +3. check if the decrypted `parameter.signedAndEncryptedURL` is equals the selected url from the previous selected CommunityFederationEntry + 1. if not then break the further processing of this request by only writing an error-log event. There will be no answer to the invoker community, because this community will only go on with a `openConnectionRedirect`-request from this community. + 2. if yes then verify the signature of `parameter.signedAndEncryptedURL` with the `cf.pubKey` read in step 2 before + 3. +4. ### Stage3 - Autorized Business Communication ongoing - # Review von Ulf ## Communication concept