diff --git a/troubleshooting/custom-prometheus.md b/troubleshooting/custom-prometheus.md new file mode 100644 index 0000000000..fb81eecf24 --- /dev/null +++ b/troubleshooting/custom-prometheus.md @@ -0,0 +1,37 @@ +# Using Custom Prometheus + +When using custom prometheus with Lens app, Lens expects certain things for prometheus rules and labels. Below is listed the changes required to see metrics properly. + +## kube-prometheus + +1. To see node metrics properly, please add + +``` +- action: replace + regex: (.*) + replacement: $1 + sourceLabels: + - __meta_kubernetes_pod_node_name + targetLabel: kubernetes_node +``` + +relabeling to `node-exporter` servicemonitor crd (for example `kubectl edit -n monitoring servicemonitors node-exporter`). + +2. To see cluster's pod usage on cluster overview properly, please add `metricsRelabeling` to `kubelet` service monitor (`kubectl edit -n monitoring servicemonitors node-exporter`) + +``` +metricRelabelings: +- action: replace + sourceLabels: + - node + targetLabel: instance +``` + +## Helm chart + +1. To see cpu metrics properly, please set value of `server.global.scrape_timeout` less than 1 minute, for example + +``` +helm upgrade --set server.global.scrape_timeout=30s prometheus stable/prometheus +``` +