docs(deployment): Explain how to setup metrics (#2825)

* docs(deployment): Explain how to setup metrics

close #2411
close #2777

* Update docs with some info I found useful

Co-authored-by: mattwr18 <mattwr18@gmail.com>

SUMMARY.md

@@ -32,6 +32,7 @@
     * [Volume Snapshots](deployment/volumes/volume-snapshots/README.md)
     * [Reclaim Policy](deployment/volumes/reclaim-policy/README.md)
     * [Velero](deployment/volumes/velero/README.md)
+  * [Metrics](deployment/monitoring/README.md)
   * [Legacy Migration](deployment/legacy-migration/README.md)
 * [Feature Specification](cypress/features.md)
 * [Code of conduct](CODE_OF_CONDUCT.md)

deployment/monitoring/README.md

@@ -0,0 +1,43 @@
+# Metrics
+
+You can optionally setup [prometheus](https://prometheus.io/) and
+[grafana](https://grafana.com/) for metrics.
+
+We follow this tutorial [here](https://medium.com/@chris_linguine/how-to-monitor-your-kubernetes-cluster-with-prometheus-and-grafana-2d5704187fc8):
+
+```bash
+kubectl proxy # proxy to your kubernetes dashboard
+
+helm repo list
+# If using helm v3, the stable repository is not set, so you need to manually add it.
+helm repo add stable https://kubernetes-charts.storage.googleapis.com
+# Create a monitoring namespace for your cluster
+kubectl create namespace monitoring
+helm --namespace monitoring install prometheus stable/prometheus
+kubectl -n monitoring get pods # look for 'server'
+kubectl port-forward -n monitoring <PROMETHEUS_SERVER_ID> 9090
+# You can now see your prometheus server on: http://localhost:9090
+
+# Make sure you are in folder `deployment/`
+kubectl apply -f monitoring/grafana/config.yml
+helm --namespace monitoring install grafana stable/grafana -f monitoring/grafana/values.yml
+# Get the admin password for grafana from your kubernetes dashboard.
+kubectl --namespace monitoring port-forward <POD_NAME> 3000
+# You can now see your grafana dashboard on: http://localhost:3000
+# Login with user 'admin' and the password you just looked up.
+# In your dashboard import this dashboard:
+# https://grafana.com/grafana/dashboards/1860
+# Enter ID 180 and choose "Prometheus" as datasource.
+# You got metrics!
+```
+
+Now you should see something like this:
+
+![Grafana dashboard](./grafana/metrics.png)
+
+You can set up a grafana dashboard, by visiting https://grafana.com/dashboards, finding one that is suitable and copying it's id.
+You then go to the left hand menu in localhost, choose `Dashboard` > `Manage` > `Import`
+Paste in the id, click `Load`, select `Prometheus` for the data source, and click `Import`
+
+When you just installed prometheus and grafana, the data will not be available
+immediately, so wait for a couple of minutes and reload.

deployment/monitoring/grafana/config.yml

@@ -0,0 +1,16 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: prometheus-grafana-datasource
+  namespace: monitoring
+  labels:
+    grafana_datasource: '1'
+data:
+  datasource.yaml: |-
+    apiVersion: 1
+    datasources:
+    - name: Prometheus
+      type: prometheus
+      access: proxy
+      orgId: 1
+      url: http://prometheus-server.monitoring.svc.cluster.local

deployment/monitoring/grafana/values.yml

@@ -0,0 +1,4 @@
+sidecar:
+  datasources:
+    enabled: true
+    label: grafana_datasource