Prometheus

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
UNAMESPACE=$UNAMESPACE envsubst < prometheus.yml   > prometheus.my-ns.yaml
helm install prometheus prometheus-community/prometheus -f prometheus.my-ns.yaml --namespace $UNAMESPACE
                

Copy

Install prometheus in the same namespace.

Grafana

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm install grafana grafana/grafana --namespace  $UNAMESPACE
kubectl get secret --namespace $UNAMESPACE grafana -o jsonpath="{.data.admin-password}" | base64 --decode ; echo
UNAMESPACE=$UNAMESPACE envsubst < c8-ingress-grafana.yaml | kubectl apply -f -
                

Copy

Install Grafana. It provides you the password for admin account. Please make a note of this.
Also note that we are adding ingress for accessing Grafana app. You can sign-in to Grafana

Configure Grafana - add prometheus datasource

            
####### Grafana UI - Setup data source and import dashboard
# Step 1.  Go to Datasources. Add prometheus as data source. Save and test. (Get 'prometheus-server' IP address from Lens UI)
# Step 2.a Go to Dashboard -> Import. 
#      2.b Use link to copy-and-paste dashboard template ->> https://github.com/camunda/camunda/blob/stable/8.6/monitor/grafana/zeebe.json    
#      2.c Select data source from step 1 and Save


# To check prometheus data feed - run this from toolkit pod where curl is supported
curl -s http://prometheus-server.$UNAMESPACE:9090/api/v1/targets            
                

Copy

This completes the Grafana configuration. You can now view the dashboard.