diff --git a/doc/README.md b/doc/README.md index 78030ce4e72d7ed164be0101e1cbbc5034053232..1943d656aa78624cfcf4da2b622f913921fa587e 100644 --- a/doc/README.md +++ b/doc/README.md @@ -56,7 +56,7 @@ - [Housekeeping](administration/housekeeping.md) Keep your Git repository tidy and fast. - [GitLab Pages configuration](administration/pages/index.md) Configure GitLab Pages. - [GitLab performance monitoring with InfluxDB](administration/monitoring/performance/introduction.md) Configure GitLab and InfluxDB for measuring performance metrics. -- [GitLab performance monitoring with Prometheus](administration/monitoring/performance/prometheus.md) Configure GitLab and Prometheus for measuring performance metrics. +- [GitLab performance monitoring with Prometheus](administration/monitoring/prometheus/index.md) Configure GitLab and Prometheus for measuring performance metrics. - [Request Profiling](administration/monitoring/performance/request_profiling.md) Get a detailed profile on slow requests. - [Monitoring uptime](user/admin_area/monitoring/health_check.md) Check the server status using the health check endpoint. - [Debugging Tips](administration/troubleshooting/debug.md) Tips to debug problems when things go wrong diff --git a/doc/administration/monitoring/performance/introduction.md b/doc/administration/monitoring/performance/introduction.md index 8b106e89cc23b3cb15519ad0ef832f805a4064e0..17c2b4b70d3af770161ea10c95d2c1cf4b3618cf 100644 --- a/doc/administration/monitoring/performance/introduction.md +++ b/doc/administration/monitoring/performance/introduction.md @@ -15,7 +15,7 @@ documents in order to understand and properly configure GitLab Performance Monit >**Note:** Omnibus GitLab 8.16 includes Prometheus as an additional tool to collect metrics. It will eventually replace InfluxDB when their metrics collection is -on par. Read more in the [Prometheus documentation](prometheus.md). +on par. Read more in the [Prometheus documentation](../prometheus/index.md). ## Introduction to GitLab Performance Monitoring diff --git a/doc/administration/monitoring/performance/prometheus.md b/doc/administration/monitoring/performance/prometheus.md index 51c63325064c16f20fe4c635164f6a476ead7fe5..d73ef5d1789cacb3ab13c734dc4993c527853c49 100644 --- a/doc/administration/monitoring/performance/prometheus.md +++ b/doc/administration/monitoring/performance/prometheus.md @@ -1,102 +1 @@ -# GitLab Prometheus - ->**Notes:** -- Prometheus and the node exporter are bundled in the Omnibus GitLab package - since GitLab 8.16. For installations from source you will have to install - them yourself. Over subsequent releases additional GitLab metrics will be - captured. -- Prometheus services are off by default but will be on starting with GitLab 9.0. - -[Prometheus] is a powerful time-series monitoring service, providing a flexible -platform for monitoring GitLab and other software products. -GitLab provides out of the box monitoring with Prometheus, providing easy -access to high quality time-series monitoring of GitLab services. - -## Overview - -Prometheus works by periodically connecting to data sources and collecting their -performance metrics. To view and work with the monitoring data, you can either -connect directly to Prometheus or utilize a dashboard tool like [Grafana]. - -## Configuring Prometheus - ->**Note:** -Available since Omnibus GitLab 8.16. For installations from source you'll -have to install and configure it yourself. - -To enable Prometheus: - -1. Edit `/etc/gitlab/gitlab.rb` -1. Find and uncomment the following line, making sure it's set to `true`: - - ```ruby - prometheus['enable'] = true - ``` - -1. Save the file and [reconfigure GitLab][reconfigure] for the changes to - take effect - -By default, Prometheus will run as the `gitlab-prometheus` user and listen on -TCP port `9090` under localhost. If the [node exporter](#node-exporter) service -has been enabled, it will automatically be set up as a monitoring target for -Prometheus. - -## Viewing Performance Metrics - -After you have [enabled Prometheus](#configuring-prometheus), you can visit -`:9090` for the dashboard that Prometheus offers by default. - -The performance data collected by Prometheus can be viewed directly in the -Prometheus console or through a compatible dashboard tool. -The Prometheus interface provides a [flexible query language][prom-query] to work -with the collected data where you can visualize their output. -For a more fully featured dashboard, Grafana can be used and has -[official support for Prometheus][prom-grafana]. - -## Prometheus exporters - -There are a number of libraries and servers which help in exporting existing -metrics from third-party systems as Prometheus metrics. This is useful for cases -where it is not feasible to instrument a given system with Prometheus metrics -directly (for example, HAProxy or Linux system stats). You can read more in the -[Prometheus exporters and integrations documentation][prom-exporters]. - -While you can use any exporter you like with your GitLab installation, the -following ones documented here are bundled in the Omnibus GitLab packages -making it easy to configure and use. - -### Node exporter - ->**Note:** -Available since Omnibus GitLab 8.16. For installations from source you'll -have to install and configure it yourself. - -The [node exporter] allows you to measure various machine resources such as -memory, disk and CPU utilization. - -To enable the node exporter: - -1. [Enable Prometheus](#configuring-prometheus) -1. Edit `/etc/gitlab/gitlab.rb` -1. Find and uncomment the following line, making sure it's set to `true`: - - ```ruby - node_exporter['enable'] = true - ``` - -1. Save the file and [reconfigure GitLab][reconfigure] for the changes to - take effect - -Prometheus it will now automatically begin collecting performance data from -the node exporter. You can visit `:9100/metrics` for a real -time representation of the metrics that are collected. Refresh the page and -you will see the data change. - -[grafana]: https://grafana.net -[node exporter]: https://github.com/prometheus/node_exporter -[prometheus]: https://prometheus.io -[prom-query]: https://prometheus.io/docs/querying/basics -[prom-grafana]: https://prometheus.io/docs/visualization/grafana/ -[scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E -[prom-exporters]: https://prometheus.io/docs/instrumenting/exporters/ -[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure +This document was moved to [monitoring/prometheus](../prometheus/index.md). diff --git a/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md new file mode 100644 index 0000000000000000000000000000000000000000..86ef9d167e294dc7bbcff27d05019664af527443 --- /dev/null +++ b/doc/administration/monitoring/prometheus/gitlab_monitor_exporter.md @@ -0,0 +1,30 @@ +# GitLab monitor exporter + +>**Note:** +Available since [Omnibus GitLab 8.17][1132]. For installations from source +you'll have to install and configure it yourself. + +The [GitLab monitor exporter] allows you to measure various GitLab metrics. + +To enable the GitLab monitor exporter: + +1. [Enable Prometheus](index.md#configuring-prometheus) +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line, making sure it's set to `true`: + + ```ruby + gitlab_monitor_exporter['enable'] = true + ``` + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +Prometheus will now automatically begin collecting performance data from +the GitLab monitor exporter exposed under `localhost:9168`. + +[← Back to the main Prometheus page](index.md) + +[1132]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1132 +[GitLab monitor exporter]: https://gitlab.com/gitlab-org/gitlab-monitor +[prometheus]: https://prometheus.io +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/administration/monitoring/prometheus/index.md b/doc/administration/monitoring/prometheus/index.md new file mode 100644 index 0000000000000000000000000000000000000000..3a394c561db18574d3b566ce4d4e051d982a142e --- /dev/null +++ b/doc/administration/monitoring/prometheus/index.md @@ -0,0 +1,147 @@ +# GitLab Prometheus + +>**Notes:** +- Prometheus and the various exporters listed in this page are bundled in the + Omnibus GitLab package. Check each exporter's documentation for the timeline + they got added. For installations from source you will have to install + them yourself. Over subsequent releases additional GitLab metrics will be + captured. +- Prometheus services are off by default but will be on starting with GitLab 9.0. +- Prometheus and its exporters do not authenticate users, and will be available + to anyone who can access them. + +[Prometheus] is a powerful time-series monitoring service, providing a flexible +platform for monitoring GitLab and other software products. +GitLab provides out of the box monitoring with Prometheus, providing easy +access to high quality time-series monitoring of GitLab services. + +## Overview + +Prometheus works by periodically connecting to data sources and collecting their +performance metrics via the [various exporters](#prometheus-exporters). To view +and work with the monitoring data, you can either +[connect directly to Prometheus](#viewing-performance-metrics) or utilize a +dashboard tool like [Grafana]. + +## Configuring Prometheus + +>**Note:** +Available since Omnibus GitLab 8.16. For installations from source you'll +have to install and configure it yourself. + +To enable Prometheus: + +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line, making sure it's set to `true`: + + ```ruby + prometheus['enable'] = true + ``` + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +By default, Prometheus will run as the `gitlab-prometheus` user and listen on +`http://localhost:9090`. If the [node exporter](#node-exporter) service +has been enabled, it will automatically be set up as a monitoring target for +Prometheus. + +## Changing the port Prometheus listens on + +>**Note:** +The following change was added in [GitLab Omnibus 8.17][1261]. Although possible, +it's not recommended to change the default address and port Prometheus listens +on as this might affect or conflict with other services running on the GitLab +server. Proceed at your own risk. + +To change the address/port that Prometheus listens on: + +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line: + + ```ruby + prometheus['listen_address'] = 'localhost:9090' + ``` + + Replace `localhost:9090` with the address/port you want Prometheus to + listen on. + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +## Viewing performance metrics + +After you have [enabled Prometheus](#configuring-prometheus), you can visit +`http://localhost:9090` for the dashboard that Prometheus offers by default. + +>**Note:** +If SSL has been enabled on your GitLab instance, you may not be able to access +Prometheus on the same browser as GitLab due to [HSTS][hsts]. We plan to +[provide access via GitLab][multi-user-prometheus], but in the interim there are +some workarounds: using a separate browser for Prometheus, resetting HSTS, or +having [Nginx proxy it][nginx-custom-config]. Follow issue [#27069] for more +information. + +The performance data collected by Prometheus can be viewed directly in the +Prometheus console or through a compatible dashboard tool. +The Prometheus interface provides a [flexible query language][prom-query] to work +with the collected data where you can visualize their output. +For a more fully featured dashboard, Grafana can be used and has +[official support for Prometheus][prom-grafana]. + +Sample Prometheus queries: + +- **% Memory used:** `(1 - ((node_memory_MemFree + node_memory_Cached) / node_memory_MemTotal)) * 100` +- **% CPU load:** `1 - rate(node_cpu{mode="idle"}[5m])` +- **Data transmitted:** `irate(node_network_transmit_bytes[5m])` +- **Data received:** `irate(node_network_receive_bytes[5m])` + +## Prometheus exporters + +There are a number of libraries and servers which help in exporting existing +metrics from third-party systems as Prometheus metrics. This is useful for cases +where it is not feasible to instrument a given system with Prometheus metrics +directly (for example, HAProxy or Linux system stats). You can read more in the +[Prometheus exporters and integrations upstream documentation][prom-exporters]. + +While you can use any exporter you like with your GitLab installation, the +following ones documented here are bundled in the Omnibus GitLab packages +making it easy to configure and use. + +### Node exporter + +The node exporter allows you to measure various machine resources such as +memory, disk and CPU utilization. + +[➔ Read more about the node exporter.](node_exporter.md) + +### Redis exporter + +The Redis exporter allows you to measure various Redis metrics. + +[➔ Read more about the Redis exporter.](redis_exporter.md) + +### Postgres exporter + +The Postgres exporter allows you to measure various PostgreSQL metrics. + +[➔ Read more about the Postgres exporter.](postgres_exporter.md) + +### GitLab monitor exporter + +The GitLab monitor exporter allows you to measure various GitLab metrics. + +[➔ Read more about the GitLab monitor exporter.](gitlab_monitor_exporter.md) + +[grafana]: https://grafana.net +[hsts]: https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security +[multi-user-prometheus]: https://gitlab.com/gitlab-org/multi-user-prometheus +[nginx-custom-config]: https://docs.gitlab.com/omnibus/settings/nginx.html#inserting-custom-nginx-settings-into-the-gitlab-server-block +[prometheus]: https://prometheus.io +[prom-exporters]: https://prometheus.io/docs/instrumenting/exporters/ +[prom-query]: https://prometheus.io/docs/querying/basics +[prom-grafana]: https://prometheus.io/docs/visualization/grafana/ +[scrape-config]: https://prometheus.io/docs/operating/configuration/#%3Cscrape_config%3E +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure +[#27069]: https://gitlab.com/gitlab-org/gitlab-ce/issues/27069 +[1261]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1261 diff --git a/doc/administration/monitoring/prometheus/node_exporter.md b/doc/administration/monitoring/prometheus/node_exporter.md new file mode 100644 index 0000000000000000000000000000000000000000..aef7758a88f7baaee007aa8f294a69aa38d17caf --- /dev/null +++ b/doc/administration/monitoring/prometheus/node_exporter.md @@ -0,0 +1,30 @@ +# Node exporter + +>**Note:** +Available since Omnibus GitLab 8.16. For installations from source you'll +have to install and configure it yourself. + +The [node exporter] allows you to measure various machine resources such as +memory, disk and CPU utilization. + +To enable the node exporter: + +1. [Enable Prometheus](index.md#configuring-prometheus) +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line, making sure it's set to `true`: + + ```ruby + node_exporter['enable'] = true + ``` + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +Prometheus will now automatically begin collecting performance data from +the node exporter exposed under `localhost:9100`. + +[← Back to the main Prometheus page](index.md) + +[node exporter]: https://github.com/prometheus/node_exporter +[prometheus]: https://prometheus.io +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/administration/monitoring/prometheus/postgres_exporter.md b/doc/administration/monitoring/prometheus/postgres_exporter.md new file mode 100644 index 0000000000000000000000000000000000000000..8e2d3162f8855ab5c9dff9fef56e27e2dd1f15ce --- /dev/null +++ b/doc/administration/monitoring/prometheus/postgres_exporter.md @@ -0,0 +1,30 @@ +# Postgres exporter + +>**Note:** +Available since [Omnibus GitLab 8.17][1131]. For installations from source +you'll have to install and configure it yourself. + +The [postgres exporter] allows you to measure various PostgreSQL metrics. + +To enable the postgres exporter: + +1. [Enable Prometheus](index.md#configuring-prometheus) +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line, making sure it's set to `true`: + + ```ruby + postgres_exporter['enable'] = true + ``` + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +Prometheus will now automatically begin collecting performance data from +the postgres exporter exposed under `localhost:9187`. + +[← Back to the main Prometheus page](index.md) + +[1131]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1131 +[postgres exporter]: https://github.com/wrouesnel/postgres_exporter +[prometheus]: https://prometheus.io +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure diff --git a/doc/administration/monitoring/prometheus/redis_exporter.md b/doc/administration/monitoring/prometheus/redis_exporter.md new file mode 100644 index 0000000000000000000000000000000000000000..d54d409dbb6ce6a508e15d7d6b2801334a13a998 --- /dev/null +++ b/doc/administration/monitoring/prometheus/redis_exporter.md @@ -0,0 +1,33 @@ +# Redis exporter + +>**Note:** +Available since [Omnibus GitLab 8.17][1118]. For installations from source +you'll have to install and configure it yourself. + +The [Redis exporter] allows you to measure various [Redis] metrics. For more +information on what's exported [read the upstream documentation][redis-exp]. + +To enable the Redis exporter: + +1. [Enable Prometheus](index.md#configuring-prometheus) +1. Edit `/etc/gitlab/gitlab.rb` +1. Add or find and uncomment the following line, making sure it's set to `true`: + + ```ruby + redis_exporter['enable'] = true + ``` + +1. Save the file and [reconfigure GitLab][reconfigure] for the changes to + take effect + +Prometheus will now automatically begin collecting performance data from +the Redis exporter exposed under `localhost:9121`. + +[← Back to the main Prometheus page](index.md) + +[1118]: https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/1118 +[redis]: https://redis.io +[redis exporter]: https://github.com/oliver006/redis_exporter +[redis-exp]: https://github.com/oliver006/redis_exporter/blob/master/README.md#whats-exported +[prometheus]: https://prometheus.io +[reconfigure]: ../../restart_gitlab.md#omnibus-gitlab-reconfigure