# Geo for multiple nodes

> 原文：[https://docs.gitlab.com/ee/administration/geo/replication/multiple_servers.html](https://docs.gitlab.com/ee/administration/geo/replication/multiple_servers.html)

*   [Architecture overview](#architecture-overview)
*   [Redis and PostgreSQL for multiple nodes](#redis-and-postgresql-for-multiple-nodes)
*   [Prerequisites: Two working GitLab multi-node clusters](#prerequisites-two-working-gitlab-multi-node-clusters)
*   [Configure the GitLab cluster to be the **primary** node](#configure-the-gitlab-cluster-to-be-the-primary-node)
    *   [Step 1: Configure the **primary** frontend servers](#step-1-configure-the-primary-frontend-servers)
    *   [Step 2: Configure the **primary** database](#step-2-configure-the-primary-database)
*   [Configure a **secondary** node](#configure-a-secondary-node)
    *   [Step 1: Configure the Redis and Gitaly services on the **secondary** node](#step-1-configure-the-redis-and-gitaly-services-on-the-secondary-node)
    *   [Step 2: Configure the main read-only replica PostgreSQL database on the **secondary** node](#step-2-configure-the-main-read-only-replica-postgresql-database-on-the-secondary-node)
    *   [Step 3: Configure the tracking database on the **secondary** node](#step-3-configure-the-tracking-database-on-the-secondary-node)
    *   [Step 4: Configure the frontend application servers on the **secondary** node](#step-4-configure-the-frontend-application-servers-on-the-secondary-node)
    *   [Step 5: Set up the LoadBalancer for the **secondary** node](#step-5-set-up-the-loadbalancer-for-the-secondary-node)
    *   [Step 6: Configure the backend application servers on the **secondary** node](#step-6-configure-the-backend-application-servers-on-the-secondary-node)

# Geo for multiple nodes[](#geo-for-multiple-nodes-premium-only "Permalink")

本文档介绍了用于在多节点配置中运行 Geo 的最小参考架构. 如果您的多节点设置与所描述的不同，则可以根据需要调整这些说明.

## Architecture overview[](#architecture-overview "Permalink")

[![Geo multi-node diagram](img/e28a59687a00b33f8174f2bcba18ac5d.png)](../../high_availability/img/geo-ha-diagram.png)

*[diagram source - GitLab employees only](https://docs.google.com/drawings/d/1z0VlizKiLNXVVVaERFwgsIOuEgjcUqDTWPdQYsE7Z4c/edit)*

上面的拓扑假定**主要**和**辅助**地理集群位于两个单独的位置，位于它们自己的具有专用 IP 地址的虚拟网络上. 网络配置为使一个地理位置内的所有计算机都可以使用其专用 IP 地址相互通信. 给出的 IP 地址只是示例，可能会因部署的网络拓扑而有所不同.

在上述示例中，访问这两个 Geo 部署的唯一外部方法是通过 HTTPS 分别位于`gitlab.us.example.com`和`gitlab.eu.example.com` .

**注意：** **主要**和**辅助** Geo 部署必须能够通过 HTTPS 相互通信.

## Redis and PostgreSQL for multiple nodes[](#redis-and-postgresql-for-multiple-nodes "Permalink")

地理位置支持：

*   Redis 和 PostgreSQL 在**为主**节点上配置了多个节点.
*   Redis 在为多个节点配置的**辅助**节点上.

**注意：** [已计划](https://gitlab.com/groups/gitlab-org/-/epics/2536)在多节点配置中在**辅助**节点上支持 PostgreSQL.

由于为 PostgreSQL 和 Redis 设置此配置涉及额外的复杂性，因此此 Geo 多节点文档未涵盖该配置.

有关使用 omnibus 软件包设置多节点 PostgreSQL 集群和 Redis 集群的更多信息，请分别参见[PostgreSQL](../../postgresql/replication_and_failover.html)和[Redis](../../redis/replication_and_failover.html)的多节点文档.

**注意：**可以将云托管服务用于 PostgreSQL 和 Redis，但这不在本文档的范围之内.

## Prerequisites: Two working GitLab multi-node clusters[](#prerequisites-two-working-gitlab-multi-node-clusters "Permalink")

一个群集将充当**主**节点. 使用[GitLab 多节点文档](../../reference_architectures/index.html)进行设置. 如果您已经有一个正在使用的工作 GitLab 实例，则可以将其用作**primary** .

The second cluster will serve as the **secondary** node. Again, use the [GitLab multi-node documentation](../../reference_architectures/index.html) to set this up. It’s a good idea to log in and test it, however, note that its data will be wiped out as part of the process of replicating from the **primary**.

## Configure the GitLab cluster to be the **primary** node[](#configure-the-gitlab-cluster-to-be-the-primary-node "Permalink")

通过以下步骤，GitLab 集群可以用**作主**节点.

### Step 1: Configure the **primary** frontend servers[](#step-1-configure-the-primary-frontend-servers "Permalink")

1.  编辑`/etc/gitlab/gitlab.rb`并添加以下内容：

    ```
    ##
    ## Enable the Geo primary role
    ##
    roles ['geo_primary_role']

    ##
    ## The unique identifier for the Geo node.
    ##
    gitlab_rails['geo_node_name'] = '<node_name_here>'

    ##
    ## Disable automatic migrations
    ##
    gitlab_rails['auto_migrate'] = false 
    ```

进行这些更改后，请[重新配置 GitLab，](../../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效.

**注意：**在正常的 GitLab 多节点设置过程中，应已在应用程序服务器上禁用 PostgreSQL 和 Redis，并且已配置从应用程序服务器到后端服务器上的那些服务的连接. 请参阅[PostgreSQL](../../postgresql/replication_and_failover.html#configuring-the-application-nodes)和[Redis 的](../../redis/replication_and_failover.html#example-configuration-for-the-gitlab-application)多节点配置文档.

### Step 2: Configure the **primary** database[](#step-2-configure-the-primary-database "Permalink")

1.  编辑`/etc/gitlab/gitlab.rb`并添加以下内容：

    ```
    ##
    ## Configure the Geo primary role and the PostgreSQL role
    ##
    roles ['geo_primary_role', 'postgres_role'] 
    ```

## Configure a **secondary** node[](#configure-a-secondary-node "Permalink")

**辅助**群集与任何其他 GitLab 多节点群集相似，但有两个主要区别：

*   主 PostgreSQL 数据库是**主**节点的 PostgreSQL 数据库的只读副本.
*   **辅助**集群还有一个单独的 PostgreSQL 数据库，称为"跟踪数据库"，该数据库跟踪各种资源的同步状态.

因此，我们将一个接一个地设置多节点组件，并包括与常规多节点设置的偏差. 但是，我们强烈建议您首先配置一个全新的群集，就像它不是地理设置一样，以便可以对其进行测试和验证. 只有这样，才可以对其进行修改以用作 Geo **辅助对象** . 这有助于分离与地理设置无关的问题.

### Step 1: Configure the Redis and Gitaly services on the **secondary** node[](#step-1-configure-the-redis-and-gitaly-services-on-the-secondary-node "Permalink")

再次使用非 Geo 多节点文档配置以下服务：

*   为多个节点[配置 Redis for GitLab](../../redis/replication_and_failover.html#example-configuration-for-the-gitlab-application) .
*   [Gitaly](../../high_availability/gitaly.html) ，它将存储从**主**节点同步的数据.

**注意：** [NFS](../../high_availability/nfs.html)可以代替 Gitaly 使用，但不建议使用.

### Step 2: Configure the main read-only replica PostgreSQL database on the **secondary** node[](#step-2-configure-the-main-read-only-replica-postgresql-database-on-the-secondary-node "Permalink")

**注意：**以下文档假定数据库将仅在单个节点上运行. [当前不支持](https://gitlab.com/groups/gitlab-org/-/epics/2536) **辅助**节点上的多节点 PostgreSQL.

配置[**辅助**数据库](database.html)作为**主**数据库的一个只读副本. 使用以下内容作为指导.

1.  为数据库用户生成所需密码的 MD5 哈希，GitLab 应用程序将使用该密码来访问只读副本数据库：

    请注意，用户名（默认情况下为`gitlab` ）已合并到哈希中.

    ```
    gitlab-ctl pg-password-md5 gitlab
    # Enter password: <your_password_here>
    # Confirm password: <your_password_here>
    # fca0b89a972d69f00eb3ec98a5838484 
    ```

    在下一步中使用此哈希值填写`<md5_hash_of_your_password>` .

2.  在副本数据库计算机中编辑`/etc/gitlab/gitlab.rb` ，并添加以下内容：

    ```
    ##
    ## Configure the Geo secondary role and the PostgreSQL role
    ##
    roles ['geo_secondary_role', 'postgres_role']

    ##
    ## Secondary address
    ## - replace '<secondary_node_ip>' with the public or VPC address of your Geo secondary node
    ## - replace '<tracking_database_ip>' with the public or VPC address of your Geo tracking database node
    ##
    postgresql['listen_address'] = '<secondary_node_ip>'
    postgresql['md5_auth_cidr_addresses'] = ['<secondary_node_ip>/32', '<tracking_database_ip>/32']

    ##
    ## Database credentials password (defined previously in primary node)
    ## - replicate same values here as defined in primary node
    ##
    postgresql['sql_user_password'] = '<md5_hash_of_your_password>'
    gitlab_rails['db_password'] = '<your_password_here>'

    ##
    ## When running the Geo tracking database on a separate machine, disable it
    ## here and allow connections from the tracking database host. And ensure
    ## the tracking database IP is in postgresql['md5_auth_cidr_addresses'] above.
    ##
    geo_postgresql['enable'] = false

    ##
    ## Disable `geo_logcursor` service so Rails doesn't get configured here
    ##
    geo_logcursor['enable'] = false 
    ```

进行这些更改后，请[重新配置 GitLab，](../../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效.

如果使用外部 PostgreSQL 实例，另请参阅使用外部 PostgreSQL 实例的[Geo](external_database.html) .

### Step 3: Configure the tracking database on the **secondary** node[](#step-3-configure-the-tracking-database-on-the-secondary-node "Permalink")

**注意：**本文档假定跟踪数据库将仅在一台计算机上运行，​​而不是作为 PostgreSQL 集群运行.

配置跟踪数据库.

1.  Generate an MD5 hash of the desired password for the database user that the GitLab application will use to access the tracking database:

    请注意，用户名（默认为`gitlab_geo` ）已合并到哈希中.

    ```
    gitlab-ctl pg-password-md5 gitlab_geo
    # Enter password: <your_password_here>
    # Confirm password: <your_password_here>
    # fca0b89a972d69f00eb3ec98a5838484 
    ```

    在下一步中，使用此哈希值填写`<tracking_database_password_md5_hash>` .

2.  在跟踪数据库计算机中编辑`/etc/gitlab/gitlab.rb` ，并添加以下内容：

    ```
    ##
    ## Enable the Geo secondary tracking database
    ##
    geo_postgresql['enable'] = true
    geo_postgresql['listen_address'] = '<ip_address_of_this_host>'
    geo_postgresql['sql_user_password'] = '<tracking_database_password_md5_hash>'

    ##
    ## Configure FDW connection to the replica database
    ##
    geo_secondary['db_fdw'] = true
    geo_postgresql['fdw_external_password'] = '<replica_database_password_plaintext>'
    geo_postgresql['md5_auth_cidr_addresses'] = ['<replica_database_ip>/32']
    gitlab_rails['db_host'] = '<replica_database_ip>'

    # Prevent reconfigure from attempting to run migrations on the replica DB
    gitlab_rails['auto_migrate'] = false

    ##
    ## Disable all other services that aren't needed, since we don't have a role
    ## that does this.
    ##
    alertmanager['enable'] = false
    consul['enable'] = false
    gitaly['enable'] = false
    gitlab_exporter['enable'] = false
    gitlab_workhorse['enable'] = false
    nginx['enable'] = false
    node_exporter['enable'] = false
    pgbouncer_exporter['enable'] = false
    postgresql['enable'] = false
    prometheus['enable'] = false
    redis['enable'] = false
    redis_exporter['enable'] = false
    repmgr['enable'] = false
    sidekiq['enable'] = false
    puma['enable'] = false 
    ```

进行这些更改后，请[重新配置 GitLab，](../../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效.

如果使用外部 PostgreSQL 实例，另请参阅使用外部 PostgreSQL 实例的[Geo](external_database.html) .

### Step 4: Configure the frontend application servers on the **secondary** node[](#step-4-configure-the-frontend-application-servers-on-the-secondary-node "Permalink")

在体系结构概述中，有两台机器运行 GitLab 应用程序服务. 这些服务在配置中有选择地启用.

在[为多个节点配置 GitLab](../../high_availability/gitlab.html)之后， [配置](../../high_availability/gitlab.html)应用程序服务器，然后进行以下修改：

1.  在**辅助**集群中的每个应用程序服务器上编辑`/etc/gitlab/gitlab.rb` ，并添加以下内容：

    ```
    ##
    ## Enable the Geo secondary role
    ##
    roles ['geo_secondary_role', 'application_role']

    ##
    ## The unique identifier for the Geo node.
    ##
    gitlab_rails['geo_node_name'] = '<node_name_here>'

    ##
    ## Disable automatic migrations
    ##
    gitlab_rails['auto_migrate'] = false

    ##
    ## Configure the connection to the tracking DB. And disable application
    ## servers from running tracking databases.
    ##
    geo_secondary['db_host'] = '<geo_tracking_db_host>'
    geo_secondary['db_password'] = '<geo_tracking_db_password>'
    geo_postgresql['enable'] = false

    ##
    ## Configure connection to the streaming replica database, if you haven't
    ## already
    ##
    gitlab_rails['db_host'] = '<replica_database_host>'
    gitlab_rails['db_password'] = '<replica_database_password>'

    ##
    ## Configure connection to Redis, if you haven't already
    ##
    gitlab_rails['redis_host'] = '<redis_host>'
    gitlab_rails['redis_password'] = '<redis_password>'

    ##
    ## If you are using custom users not managed by Omnibus, you need to specify
    ## UIDs and GIDs like below, and ensure they match between servers in a
    ## cluster to avoid permissions issues
    ##
    user['uid'] = 9000
    user['gid'] = 9000
    web_server['uid'] = 9001
    web_server['gid'] = 9001
    registry['uid'] = 9002
    registry['gid'] = 9002 
    ```

**注意：**如果使用 omnibus 软件包设置了 PostgreSQL 集群，并且设置了`postgresql['sql_user_password'] = 'md5 digest of secret'`设置，请记住`gitlab_rails['db_password']`和`geo_secondary['db_password']`上面提到的包含明文密码. 这用于让 Rails 服务器连接到数据库.**注意：**确保在远程数据库的`postgresql['md5_auth_cidr_addresses']`设置中列出了当前节点 IP.

进行这些更改后，请[重新配置 GitLab，](../../restart_gitlab.html#omnibus-gitlab-reconfigure)以使更改生效.

在辅助服务器上，将启用以下 GitLab 前端服务：

*   `geo-logcursor`
*   `gitlab-pages`
*   `gitlab-workhorse`
*   `logrotate`
*   `nginx`
*   `registry`
*   `remote-syslog`
*   `sidekiq`
*   `puma`

通过在前端应用程序服务器上运行`sudo gitlab-ctl status`来验证这些服务.

### Step 5: Set up the LoadBalancer for the **secondary** node[](#step-5-set-up-the-loadbalancer-for-the-secondary-node "Permalink")

在这种拓扑中，每个地理位置都需要一个负载平衡器，以将流量路由到应用程序服务器.

有关更多信息，请参见[具有多个节点的 GitLab 的 Load Balancer](../../high_availability/load_balancer.html) .

### Step 6: Configure the backend application servers on the **secondary** node[](#step-6-configure-the-backend-application-servers-on-the-secondary-node "Permalink")

上面的最小参考体系结构图显示了在同一台计算机上一起运行的所有应用程序服务. 但是，对于多个节点，我们[强烈建议单独运行所有服务](../../reference_architectures/index.html) .

例如，可以将 Sidekiq 服务器配置为与上面的前端应用程序服务器类似，但要进行一些更改以仅运行`sidekiq`服务：

1.  在**辅助**群集中的每个 Sidekiq 服务器上编辑`/etc/gitlab/gitlab.rb` ，并添加以下内容：

    ```
    ##
    ## Enable the Geo secondary role
    ##
    roles ['geo_secondary_role']

    ##
    ## Enable the Sidekiq service
    ##
    sidekiq['enable'] = true

    ##
    ## Ensure unnecessary services are disabled
    ##
    alertmanager['enable'] = false
    consul['enable'] = false
    geo_logcursor['enable'] = false
    gitaly['enable'] = false
    gitlab_exporter['enable'] = false
    gitlab_workhorse['enable'] = false
    nginx['enable'] = false
    node_exporter['enable'] = false
    pgbouncer_exporter['enable'] = false
    postgresql['enable'] = false
    prometheus['enable'] = false
    redis['enable'] = false
    redis_exporter['enable'] = false
    repmgr['enable'] = false
    puma['enable'] = false

    ##
    ## The unique identifier for the Geo node.
    ##
    gitlab_rails['geo_node_name'] = '<node_name_here>'

    ##
    ## Disable automatic migrations
    ##
    gitlab_rails['auto_migrate'] = false

    ##
    ## Configure the connection to the tracking DB. And disable application
    ## servers from running tracking databases.
    ##
    geo_secondary['db_host'] = '<geo_tracking_db_host>'
    geo_secondary['db_password'] = '<geo_tracking_db_password>'
    geo_postgresql['enable'] = false

    ##
    ## Configure connection to the streaming replica database, if you haven't
    ## already
    ##
    gitlab_rails['db_host'] = '<replica_database_host>'
    gitlab_rails['db_password'] = '<replica_database_password>'

    ##
    ## Configure connection to Redis, if you haven't already
    ##
    gitlab_rails['redis_host'] = '<redis_host>'
    gitlab_rails['redis_password'] = '<redis_password>'

    ##
    ## If you are using custom users not managed by Omnibus, you need to specify
    ## UIDs and GIDs like below, and ensure they match between servers in a
    ## cluster to avoid permissions issues
    ##
    user['uid'] = 9000
    user['gid'] = 9000
    web_server['uid'] = 9001
    web_server['gid'] = 9001
    registry['uid'] = 9002
    registry['gid'] = 9002 
    ```

    您可以类似地将服务器配置为仅运行`geo_logcursor['enable'] = true`的`geo-logcursor`服务，并使用`sidekiq['enable'] = false`禁用 Sidekiq.

    这些服务器不需要连接到负载平衡器.