From 4ddd58cc5979894bb7243e1183ebe95886da2944 Mon Sep 17 00:00:00 2001 From: Mel Kiyama Date: Fri, 13 Oct 2017 13:04:03 -0700 Subject: [PATCH] doc: Warning about add cgroup kernel issue to resource group docs. (#3492) * doc: Warning about add cgroup kernel issue to resource group docs. * docs: fix typo --- gpdb-doc/dita/admin_guide/wlmgmt.xml | 136 +++-- .../admin_guide/workload_mgmt_resgroups.xml | 529 ++++++++++++------ 2 files changed, 422 insertions(+), 243 deletions(-) diff --git a/gpdb-doc/dita/admin_guide/wlmgmt.xml b/gpdb-doc/dita/admin_guide/wlmgmt.xml index 783d564ca0..41d7d11c5c 100644 --- a/gpdb-doc/dita/admin_guide/wlmgmt.xml +++ b/gpdb-doc/dita/admin_guide/wlmgmt.xml @@ -11,71 +11,83 @@ amount of memory used to execute a query, and the relative amount of CPU devoted to processing a query. Greenplum Database provides two schemes to manage resources - Resource Queues and Resource Groups.

- Resource groups are an experimental feature and are not intended for use - in a production environment. Experimental features are subject to change without notice in - future releases. + Resource groups are an experimental feature and are not intended for use in + a production environment. Experimental features are subject to change without notice in future + releases.

Significant Pivotal Greenplum Database performance + degradation has been observed when enabling resource group-based workload management on supported versions of RedHat 6.x, CentOS 6.x, and SuSE 11 + systems. The issue is due to a Linux cgroup kernel bug. The kernel bug has been fixed on + CentOS 7.x and Red Hat 7.x systems.

Either the resource queue or the resource group management scheme can be active in Greenplum Database; both schemes cannot be active at the same time.

Resource queues are enabled by default when you install your Greenplum Database cluster. While you can create and assign resource groups when resource queues are active, you must explicitly enable resource groups to start using that management scheme.

-

The following table summarizes some of the differences between Resource Queues and Resource Groups.

- - - - - - - - Metric - Resource Queues - Resource Groups - - - - - Concurrency - Managed at the query level - Managed at the transaction level - - - CPU - Specify query priority - Specify percentage of CPU resources; uses Linux Control Groups - - - Memory - Managed at the queue and operator level; users can over-subscribe - Managed at the transaction level, with enhanced allocation and tracking; users cannot over-subscribe - - - Memory Isolation - None - Memory is isolated between resource groups and between transactions within the same resource group - - - Users - Limits are applied only to non-admin users - Limits are applied to SUPERUSER and non-admin users alike - - - Queueing - Queue only when no slot available - Queue when no slot is available or not enough available memory - - - Query Failure - Query may fail immediately if not enough memory - Query may fail after reaching transaction memory limit - - - Limit Bypass - Limits are not enforced for SUPERUSER roles and certain operators and functions - No limit bypass cases - - - -
- - +

The following table summarizes some of the differences between Resource Queues and Resource + Groups.

+ + + + + + + + Metric + Resource Queues + Resource Groups + + + + + Concurrency + Managed at the query level + Managed at the transaction level + + + CPU + Specify query priority + Specify percentage of CPU resources; uses Linux Control + Groups + + + Memory + Managed at the queue and operator level; users can + over-subscribe + Managed at the transaction level, with enhanced allocation and + tracking; users cannot over-subscribe + + + Memory Isolation + None + Memory is isolated between resource groups and between + transactions within the same resource group + + + Users + Limits are applied only to non-admin users + Limits are applied to SUPERUSER and non-admin + users alike + + + Queueing + Queue only when no slot available + Queue when no slot is available or not enough available + memory + + + Query Failure + Query may fail immediately if not enough memory + Query may fail after reaching transaction memory limit + + + Limit Bypass + Limits are not enforced for SUPERUSER roles and + certain operators and functions + No limit bypass cases + + + +
+ + diff --git a/gpdb-doc/dita/admin_guide/workload_mgmt_resgroups.xml b/gpdb-doc/dita/admin_guide/workload_mgmt_resgroups.xml index 74bd42d51e..7954beee02 100644 --- a/gpdb-doc/dita/admin_guide/workload_mgmt_resgroups.xml +++ b/gpdb-doc/dita/admin_guide/workload_mgmt_resgroups.xml @@ -4,116 +4,214 @@ Using Resource Groups - Resource groups are an experimental feature and are not intended for use in a production environment. Experimental features are subject to change without notice in future releases. -

You can use resource groups to manage the number of active queries that may execute concurrently in your Greenplum Database cluster. With resource groups, you can also manage the amount of CPU and memory resources Greenplum allocates to each query.

+ Resource groups are an experimental feature and are not intended for use in + a production environment. Experimental features are subject to change without notice in future + releases. +

You can use resource groups to manage the number of active queries that may execute + concurrently in your Greenplum Database cluster. With resource groups, you can also manage the + amount of CPU and memory resources Greenplum allocates to each query.

-

When the user executes a query, Greenplum Database evaluates the query against a set of limits defined for the resource group. Greenplum Database executes the query immediately if the group's resource limits have not yet been reached and the query does not cause the group to exceed the concurrent transaction limit. If these conditions are not met, Greenplum Database queues the query. For example, if the maximum number of concurrent transactions for the resource group has already been reached, a subsequent query is queued and must wait until other queries complete before it runs. Greenplum Database may also execute a pending query when the resource group's concurrency and memory limits are altered to large enough values.

-

Within a resource group, transactions are evaluated on a first in, first out basis. Greenplum Database periodically assesses the active workload of the system, reallocating resources and starting/queuing jobs as necessary.

-

When you create a resource group, you provide a set of limits that determine the amount of CPU and memory resources available to transactions executed within the group. These limits are:

- - - - - - - Limit Type - Description - - - - - CONCURRENCY - The maximum number of concurrent transactions, including active and idle transactions, that are permitted for this resource group. - - - CPU_RATE_LIMIT - The percentage of CPU resources available to this resource group. - - - MEMORY_LIMIT - The percentage of memory resources available to this resource group. - - - MEMORY_SHARED_QUOTA - The percentage of memory to share across transactions submitted in this resource group. - - - MEMORY_SPILL_RATIO - The memory usage threshold for memory-intensive transactions. When a transaction reaches this threshold, it spills to disk. - - - -
+

When the user executes a query, Greenplum Database evaluates the query against a set of + limits defined for the resource group. Greenplum Database executes the query immediately if + the group's resource limits have not yet been reached and the query does not cause the group + to exceed the concurrent transaction limit. If these conditions are not met, Greenplum + Database queues the query. For example, if the maximum number of concurrent transactions for + the resource group has already been reached, a subsequent query is queued and must wait until + other queries complete before it runs. Greenplum Database may also execute a pending query + when the resource group's concurrency and memory limits are altered to large enough + values.

+

Within a resource group, transactions are evaluated on a first in, first out basis. Greenplum + Database periodically assesses the active workload of the system, reallocating resources and + starting/queuing jobs as necessary.

+

When you create a resource group, you provide a set of limits that determine the amount of + CPU and memory resources available to transactions executed within the group. These limits + are:

+ + + + + + + Limit Type + Description + + + + + CONCURRENCY + The maximum number of concurrent transactions, including active + and idle transactions, that are permitted for this resource group. + + + CPU_RATE_LIMIT + The percentage of CPU resources available to this resource + group. + + + MEMORY_LIMIT + The percentage of memory resources available to this resource + group. + + + MEMORY_SHARED_QUOTA + The percentage of memory to share across transactions submitted in + this resource group. + + + MEMORY_SPILL_RATIO + The memory usage threshold for memory-intensive transactions. When + a transaction reaches this threshold, it spills to disk. + + + +
Transaction Concurrency Limit -

The CONCURRENCY limit controls the maximum number of concurrent transactions permitted for the resource group. Each resource group is logically divided into a fixed number of slots equal to the CONCURRENCY limit. Greenplum Database allocates these slots an equal, fixed percentage of memory resources.

+

The CONCURRENCY limit controls the maximum number of concurrent + transactions permitted for the resource group. Each resource group is logically divided into + a fixed number of slots equal to the CONCURRENCY limit. Greenplum Database + allocates these slots an equal, fixed percentage of memory resources.

The default CONCURRENCY limit value for a resource group is 20.

-

Greenplum Database queues any transactions submitted after the resource group reaches its CONCURRENCY limit. When a running transaction completes, Greenplum Database un-queues and executes the earliest queued transaction if sufficient memory resources exist.

+

Greenplum Database queues any transactions submitted after the resource group reaches its + CONCURRENCY limit. When a running transaction completes, Greenplum + Database un-queues and executes the earliest queued transaction if sufficient memory + resources exist.

 CPU Limit -

The server configuration parameter identifies the maximum percentage of system CPU resources to allocate to resource groups on each Greenplum Database segment node. The remaining CPU resources are used for the OS kernel and the Greenplum Database daemon processes. The default gp_resource_group_cpu_limit value is .9 (90%).

- The default gp_resource_group_cpu_limit value may not leave sufficient CPU resources if you are running other workloads on your Greenplum Database cluster nodes, so be sure to adjust this server configuration parameter accordingly. -

The Greenplum Database node CPU percentage is further divided equally among each segment on the Greenplum node. Each resource group reserves a percentage of the segment CPU for resource management. You identify this percentage via the CPU_RATE_LIMIT value you provide when you create the resource group.

-

The minimum CPU_RATE_LIMIT percentage you can specify for a resource group is 1, the maximum is 100.

-

The sum of CPU_RATE_LIMITs specified for all resource groups you define in your Greenplum Database cluster must not exceed 100.

-

CPU resource assignment is elastic in that Greenplum Database may allocate the CPU resources of an idle resource group to a busier one(s). In such situations, CPU resources are re-allocated to the previously idle resource group when that resource group next becomes active. If multiple resource groups are busy, they are allocated the CPU resources of any idle resource groups based on the ratio of their CPU_RATE_LIMITs. For example, a resource group created with a CPU_RATE_LIMIT of 40 will be allocated twice as much extra CPU resource as a resource group you create with a CPU_RATE_LIMIT of 20.

+

The server configuration parameter identifies the maximum + percentage of system CPU resources to allocate to resource groups on each Greenplum Database + segment node. The remaining CPU resources are used for the OS kernel and the Greenplum + Database daemon processes. The default gp_resource_group_cpu_limit value is + .9 (90%).

+ The default gp_resource_group_cpu_limit value may not leave sufficient + CPU resources if you are running other workloads on your Greenplum Database cluster nodes, + so be sure to adjust this server configuration parameter accordingly. +

The Greenplum Database node CPU percentage is further divided equally among each segment on + the Greenplum node. Each resource group reserves a percentage of the segment CPU for + resource management. You identify this percentage via the CPU_RATE_LIMIT + value you provide when you create the resource group.

+

The minimum CPU_RATE_LIMIT percentage you can specify for a resource group + is 1, the maximum is 100.

+

The sum of CPU_RATE_LIMITs specified for all resource groups you define in + your Greenplum Database cluster must not exceed 100.

+

CPU resource assignment is elastic in that Greenplum Database may allocate the CPU + resources of an idle resource group to a busier one(s). In such situations, CPU resources + are re-allocated to the previously idle resource group when that resource group next becomes + active. If multiple resource groups are busy, they are allocated the CPU resources of any + idle resource groups based on the ratio of their CPU_RATE_LIMITs. For + example, a resource group created with a CPU_RATE_LIMIT of 40 will be + allocated twice as much extra CPU resource as a resource group you create with a + CPU_RATE_LIMIT of 20.

 Memory Limits -

When resource groups are enabled, memory usage is managed at the Greenplum Database node, segment, resource group, and transaction levels.

+

When resource groups are enabled, memory usage is managed at the Greenplum Database node, + segment, resource group, and transaction levels.

-

The server configuration parameter identifies the maximum percentage of system memory resources to allocate to resource groups on each Greenplum Database segment node. The default gp_resource_group_memory_limit value is .9 (90%).

-

The memory resource available on a Greenplum Database node is further divided equally among each segment on the node. Each resource group reserves a percentage of the segment memory for resource management. You identify this percentage via the MEMORY_LIMIT value you specify when you create the resource group. The minimum MEMORY_LIMIT percentage you can specify for a resource group is 1, the maximum is 100.

-

The sum of MEMORY_LIMITs specified for all resource groups you define in your Greenplum Database cluster must not exceed 100.

-

The memory reserved by the resource group is divided into fixed and shared components. The MEMORY_SHARED_QUOTA value you specify when you create the resource group identifies the percentage of reserved resource group memory that may be shared among the currently running transactions. This memory is allotted on a first-come, first-served basis. A running transaction may use none, some, or all of the MEMORY_SHARED_QUOTA.

+

The server configuration parameter identifies the maximum + percentage of system memory resources to allocate to resource groups on each Greenplum + Database segment node. The default gp_resource_group_memory_limit value is + .9 (90%).

+

The memory resource available on a Greenplum Database node is further divided equally among + each segment on the node. Each resource group reserves a percentage of the segment memory + for resource management. You identify this percentage via the MEMORY_LIMIT + value you specify when you create the resource group. The minimum + MEMORY_LIMIT percentage you can specify for a resource group is 1, the + maximum is 100.

+

The sum of MEMORY_LIMITs specified for all resource groups you define in + your Greenplum Database cluster must not exceed 100.

+

The memory reserved by the resource group is divided into fixed and shared components. The + MEMORY_SHARED_QUOTA value you specify when you create the resource group + identifies the percentage of reserved resource group memory that may be shared among the + currently running transactions. This memory is allotted on a first-come, first-served basis. + A running transaction may use none, some, or all of the + MEMORY_SHARED_QUOTA.

-

The minimum MEMORY_SHARED_QUOTA you can specify is 0, the maximum is 100. The default MEMORY_SHARED_QUOTA is 20.

+

The minimum MEMORY_SHARED_QUOTA you can specify is 0, the maximum is 100. + The default MEMORY_SHARED_QUOTA is 20.

-

As mentioned previously, CONCURRENCY identifies the maximum number of concurrently running transactions permitted in the resource group. The fixed memory reserved by a resource group is divided into CONCURRENCY number of transaction slots. Each slot is allocated a fixed, equal amount of resource group memory. Greenplum Database guarantees this fixed memory to each transaction. +

As mentioned previously, CONCURRENCY identifies the maximum number of + concurrently running transactions permitted in the resource group. The fixed memory reserved + by a resource group is divided into CONCURRENCY number of transaction + slots. Each slot is allocated a fixed, equal amount of resource group memory. Greenplum + Database guarantees this fixed memory to each transaction. + Resource Group Memory Allotments + +

- - Resource Group Memory Allotments - -

+

When a query's memory usage exceeds the fixed per-transaction memory usage amount, + Greenplum Database allocates available resource group shared memory to the query. The + maximum amount of resource group memory available to a specific transaction slot is the sum + of the transaction's fixed memory and the full resource group shared memory allotment.

-

When a query's memory usage exceeds the fixed per-transaction memory usage amount, Greenplum Database allocates available resource group shared memory to the query. The maximum amount of resource group memory available to a specific transaction slot is the sum of the transaction's fixed memory and the full resource group shared memory allotment.

+
+ Spill to Disk +

MEMORY_SPILL_RATIO identifies the memory usage threshold for + memory-intensive operators in a transaction. When the transaction reaches this memory + threshold, it spills to disk. Greenplum Database uses the + MEMORY_SPILL_RATIO to determine the initial memory to allocate to a + transaction.

+

The minimum MEMORY_SPILL_RATIO percentage you can specify for a + resource group is 0. The maximum is 100. The default MEMORY_SPILL_RATIO + is 20.

+

You define the MEMORY_SPILL_RATIO when you create a resource group. You + can selectively set this limit on a per-query basis at the session level with the + server configuration parameter.

+
-
- Spill to Disk -

MEMORY_SPILL_RATIO identifies the memory usage threshold for memory-intensive operators in a transaction. When the transaction reaches this memory threshold, it spills to disk. Greenplum Database uses the MEMORY_SPILL_RATIO to determine the initial memory to allocate to a transaction.

-

The minimum MEMORY_SPILL_RATIO percentage you can specify for a resource group is 0. The maximum is 100. The default MEMORY_SPILL_RATIO is 20.

-

You define the MEMORY_SPILL_RATIO when you create a resource group. You can selectively set this limit on a per-query basis at the session level with the server configuration parameter.

-
- Using Resource Groups -
- Prerequisite -

Greenplum Database resource groups use Linux Control Groups (cgroups) to manage CPU resources. With cgroups, Greenplum isolates the CPU usage of your Greenplum processes from other processes on the node. This allows Greenplum to support CPU usage restrictions on a per-resource-group basis.

-

For detailed information about cgroups, refer to the Control Groups documentation for your Linux distribution.

-

Complete the following tasks on each node in your Greenplum Database cluster to set up cgroups for use with resource groups:

+ Significant Pivotal Greenplum Database + performance degradation has been observed when enabling resource group-based workload + management on supported versions of RedHat 6.x, CentOS 6.x, + and SuSE 11 systems. The issue is due to a Linux cgroup kernel bug. The kernel bug has been + fixed on CentOS 7.x and Red Hat 7.x systems. +
+ Prerequisite +

Greenplum Database resource groups use Linux Control Groups (cgroups) to manage CPU + resources. With cgroups, Greenplum isolates the CPU usage of your Greenplum processes from + other processes on the node. This allows Greenplum to support CPU usage restrictions on a + per-resource-group basis.

+

For detailed information about cgroups, refer to the Control Groups documentation for + your Linux distribution.

+

Complete the following tasks on each node in your Greenplum Database cluster to set up + cgroups for use with resource groups:

    -
  1. If you are running the Suse 11+ operating system on your Greenplum Database cluster nodes, you must enable swap accounting on each node and restart your Greenplum Database cluster. The swapaccount kernel boot parameter governs the swap accounting setting on Suse 11+ systems. After setting this boot parameter, you must reboot your systems. For details, refer to the Cgroup Swap Control discussion in the Suse 11 release notes. You must be the superuser or have sudo access to configure kernel boot parameters and reboot systems. -
    2. -
  2. Create the Greenplum Database cgroups configuration file /etc/cgconfig.d/gpdb.conf. You must be the superuser or have sudo access to create this file: - sudo vi /etc/cgconfig.d/gpdb.conf +
  3. If you are running the Suse 11+ operating system on your Greenplum Database cluster + nodes, you must enable swap accounting on each node and restart your Greenplum Database + cluster. The swapaccount kernel boot parameter governs the swap + accounting setting on Suse 11+ systems. After setting this boot parameter, you must + reboot your systems. For details, refer to the Cgroup Swap Control discussion in the Suse 11 + release notes. You must be the superuser or have sudo access to + configure kernel boot parameters and reboot systems.
    4. +
  4. Create the Greenplum Database cgroups configuration file + /etc/cgconfig.d/gpdb.conf. You must be the superuser or have + sudo access to create this file: + sudo vi /etc/cgconfig.d/gpdb.conf
    5. -
  5. Add the following configuration information to /etc/cgconfig.d/gpdb.conf: - group gpdb { +
  6. Add the following configuration information to + /etc/cgconfig.d/gpdb.conf: group gpdb { perm { task { uid = gpadmin; @@ -128,68 +226,90 @@ } cpuacct { } - }

    This content configures CPU and CPU accounting control groups managed by the gpadmin user.

    -
    7. -
  7. If not already installed and running, install the Control Groups operating system package and start the cgroups service on each Greenplum Database node. The commands you run to perform these tasks will differ based on the operating system installed on the node. You must be the superuser or have sudo access to run these commands: -
      -
    • Redhat/CentOS 7.x systems: - sudo yum install libcgroup-tools -sudo cgconfigparser -l /etc/cgconfig.d/gpdb.conf + } +

      This content configures CPU and CPU accounting control groups managed by the + gpadmin user.

      • -
    • Redhat/CentOS 6.x systems: - sudo yum install libcgroup +
    • If not already installed and running, install the Control Groups operating system + package and start the cgroups service on each Greenplum Database node. The commands you + run to perform these tasks will differ based on the operating system installed on the + node. You must be the superuser or have sudo access to run these + commands:
        +
      • Redhat/CentOS 7.x systems: + sudo yum install libcgroup-tools +sudo cgconfigparser -l /etc/cgconfig.d/gpdb.conf +
        • +
      • Redhat/CentOS 6.x systems: + sudo yum install libcgroup sudo service cgconfig start -
        • -
      • Suse 11+ systems: - sudo zypper install libcgroup-tools +
        • +
      • Suse 11+ systems: + sudo zypper install libcgroup-tools sudo cgconfigparser -l /etc/cgconfig.d/gpdb.conf +
        • +
      • -
    -
  8. Identify the cgroup directory mount point for the node: - grep cgroup /proc/mounts

    The first line of output identifies the cgroup mount point.

    + grep cgroup /proc/mounts

    The first line of output identifies + the cgroup mount point.

    9. -
  9. Verify that you set up the Greenplum Database cgroups configuration correctly by running the following commands. Replace cgroup_mount_point with the mount point you identified in the previous step: - ls -l cgroup_mount_point/cpu/gpdb +
  10. Verify that you set up the Greenplum Database cgroups configuration correctly by + running the following commands. Replace cgroup_mount_point with the + mount point you identified in the previous step: ls -l cgroup_mount_point/cpu/gpdb ls -l cgroup_mount_point/cpuacct/gpdb -

    If these directories exist and are owned by gpadmin:gpadmin, you have successfully configured cgroups for Greenplum Database CPU resource management.

    + +

    If these directories exist and are owned by gpadmin:gpadmin, you + have successfully configured cgroups for Greenplum Database CPU resource + management.

-
-
- Procedure - -

To use resource groups in your Greenplum Database cluster, you:

-
    -
  1. Enable resource groups for your Greenplum Database cluster.
    2. -
  2. Create resource groups.
    3. -
  3. Assign the resource groups to one or more roles.
    4. -
  4. Use resource management system views to monitor and manage the resource groups.
    5. -
-
+
+
+ Procedure +

To use resource groups in your Greenplum Database cluster, you:

+
    +
  1. Enable resource groups for your + Greenplum Database cluster.
    2. +
  2. Create resource groups.
    3. +
  3. Assign the resource groups to one or + more roles.
    4. +
  4. Use resource management system views + to monitor and manage the resource groups.
    5. +
+
Enabling Resource Groups -

When you install Greenplum Database, resource queues are enabled by default. To use resource groups instead of resource queues, you must set the server configuration parameter.

+

When you install Greenplum Database, resource queues are enabled by default. To use + resource groups instead of resource queues, you must set the server configuration parameter.

    -
  1. Set the gp_resource_manager server configuration parameter to the value "group": +
  2. Set the gp_resource_manager server configuration parameter to the value + "group": gpconfig -s gp_resource_manager gpconfig -c gp_resource_manager -v "group"
    3. -
  3. Restart Greenplum Database: - gpstop +
  4. Restart Greenplum Database: gpstop gpstart
-

Once enabled, any transaction submitted by a role is directed to the resource group assigned to the role, and is governed by that resource group's concurrency, memory, and CPU limits.

-

Greenplum Database creates two default resource groups named admin_group and default_group. When you enable resources groups, any role that was not explicitly assigned a resource group is assigned the default group for the role's capability. SUPERUSER roles are assigned the admin_group, non-admin roles are assigned the group named default_group.

-

The default resource groups admin_group and default_group are created with the following resource limits:

- +

Once enabled, any transaction submitted by a role is directed to the resource group + assigned to the role, and is governed by that resource group's concurrency, memory, and CPU + limits.

+

Greenplum Database creates two default resource groups named admin_group + and default_group. When you enable resources groups, any role that was not + explicitly assigned a resource group is assigned the default group for the role's + capability. SUPERUSER roles are assigned the admin_group, + non-admin roles are assigned the group named default_group.

+

The default resource groups admin_group and default_group + are created with the following resource limits:

+
@@ -229,80 +349,118 @@ gpstart -
+ Creating Resource Groups -

When you create a resource group, you provide a name, CPU limit, and memory limit. You can optionally provide a concurrent transaction limit and memory shared quota and spill ratio. Use the command to create a new resource group.

-

You must have SUPERUSER privileges to create a resource group. The maximum number of resource groups allowed in your Greenplum Database cluster is 100.

-

When you create a resource group, you must provide CPU_RATE_LIMIT and MEMORY_LIMIT limit values. - These limits identify the percentage of Greenplum Database resources - to allocate to this resource group. For example, to create a resource group named - rgroup1 with a CPU limit of 20 and a memory limit of 25:

-

- =# CREATE RESOURCE GROUP rgroup1 WITH (CPU_RATE_LIMIT=20, MEMORY_LIMIT=25); +

When you create a resource group, you provide a name, CPU limit, and memory limit. You can + optionally provide a concurrent transaction limit and memory shared quota and spill ratio. + Use the command to create a new resource group.

+

When you create a resource group, you must provide + CPU_RATE_LIMIT and MEMORY_LIMIT limit values. These + limits identify the percentage of Greenplum Database resources to allocate to this resource + group. For example, to create a resource group named rgroup1 with a CPU limit of 20 + and a memory limit of 25:

+

+ =# CREATE RESOURCE GROUP rgroup1 WITH (CPU_RATE_LIMIT=20, MEMORY_LIMIT=25); -

-

The CPU limit of 20 is shared by every role to which rgroup1 is assigned. Similarly, the memory limit of 25 is shared by every role to which rgroup1 is assigned. rgroup1 utilizes the default CONCURRENCY setting of 20.

-

The command updates the limits of a resource group. - To change the limits of a resource group, specify the new values you want for the group. - For example:

-

- =# ALTER RESOURCE GROUP rg_light SET CONCURRENCY 7; +

+

The CPU limit of 20 is shared by every role to which rgroup1 is assigned. + Similarly, the memory limit of 25 is shared by every role to which rgroup1 + is assigned. rgroup1 utilizes the default CONCURRENCY + setting of 20.

+

The command updates the limits of a resource group. To + change the limits of a resource group, specify the new values you want for the group. For + example:

+

+ =# ALTER RESOURCE GROUP rg_light SET CONCURRENCY 7; =# ALTER RESOURCE GROUP exec SET MEMORY_LIMIT 25; -

- You cannot set or alter the CONCURRENCY value for the admin_group to zero (0). -

The command drops a resource group. To drop a resource group, the group cannot be assigned to any role, nor can there be any transactions active or waiting in the resource group. - To drop a resource group:

-

- =# DROP RESOURCE GROUP exec; -

+

+ You cannot set or alter the CONCURRENCY value for the + admin_group to zero (0). +

The command drops a resource group. To drop a resource + group, the group cannot be assigned to any role, nor can there be any transactions active or + waiting in the resource group. To drop a resource group:

+

+ =# DROP RESOURCE GROUP exec; +

Assigning a Resource Group to a Role -

When you create a resource group, the group is available for assignment to one or more roles (users). You assign a resource group to a database role using the RESOURCE GROUP clause of the or commands. If you do not specify a resource group for a role, the role is assigned the default group for the role's capability. SUPERUSER roles are assigned the admin_group, non-admin roles are assigned the group named default_group.

-

Use the ALTER ROLE or CREATE ROLE commands to assign a resource group to a role. For example:

+

When you create a resource group, the group is available for assignment to + one or more roles (users). You assign a resource group to a database role using the + RESOURCE GROUP clause of the or commands. If you do not specify a resource group + for a role, the role is assigned the default group for the role's capability. + SUPERUSER roles are assigned the admin_group, non-admin + roles are assigned the group named default_group.

+

Use the ALTER ROLE or CREATE ROLE commands to assign a + resource group to a role. For example:

=# ALTER ROLE bill RESOURCE GROUP rg_light; =# CREATE ROLE mary RESOURCE GROUP exec;

-

You can assign a resource group to one or more roles. If you have defined a role hierarchy, assigning a resource group to a parent role does not propagate down to the members of that role group.

-

If you wish to remove a resource group assignment from a role and assign the role the default group, change the role's group name assignment to NONE. - For example:

-

- =# ALTER ROLE mary RESOURCE GROUP NONE; +

You can assign a resource group to one or more roles. If you have defined a role hierarchy, + assigning a resource group to a parent role does not propagate down to the members of that + role group.

+

If you wish to remove a resource group assignment from a role and assign the role the + default group, change the role's group name assignment to NONE. For + example:

+

+ =# ALTER ROLE mary RESOURCE GROUP NONE; -

- - +

+ + - Monitoring Resource Group Status - -

Monitoring the status of your resource groups and queries may involve the following tasks:

- - - + Monitoring Resource Group Status + +

Monitoring the status of your resource groups and queries may involve the following + tasks:

+ + + Viewing Resource Group Limits -

The gp_toolkit system view displays the current and proposed limits for a resource group. The proposed limit differs from the current limit when you alter the limit but the new value can not be immediately applied. To view the limits of all resource groups:

+

The + gp_toolkit system view displays the current and proposed limits for a + resource group. The proposed limit differs from the current limit when you alter the limit + but the new value can not be immediately applied. To view the limits of all resource + groups:

=# SELECT * FROM gp_toolkit.gp_resgroup_config; @@ -313,7 +471,12 @@ gpstart Viewing Resource Group Query Status and CPU/Memory Usage -

The gp_toolkit system view enables you to view the status and activity of a resource group. The view displays the number of running and queued transactions. It also displays the real-time CPU and memory usage of the resource group. To view this information:

+

The + gp_toolkit system view enables you to view the status and activity of a + resource group. The view displays the number of running and queued transactions. It also + displays the real-time CPU and memory usage of the resource group. To view this + information:

=# SELECT * FROM gp_toolkit.gp_resgroup_status; @@ -325,8 +488,10 @@ gpstart Viewing the Resource Group Assigned to a Role

To view the resource group-to-role assignments, perform the following query on the - and - system catalog tables:

+ and system catalog tables:

=# SELECT rolname, rsgname FROM pg_roles, pg_resgroup     WHERE pg_roles.rolresgroup=pg_resgroup.oid; @@ -338,10 +503,10 @@ gpstart Viewing a Resource Group's Running and Pending Queries -

To view a resource group's running queries, pending queries, and how long - the pending queries have been queued, examine the - - system catalog table:

+

To view a resource group's running queries, pending queries, and how long the pending + queries have been queued, examine the system catalog table:

=# SELECT current_query, waiting, rsgname, rsgqueueduration FROM pg_stat_activity; @@ -356,16 +521,14 @@ gpstart

There may be cases when you want to cancel a queued transaction in a resource group. For example, you may want to remove a query that is waiting in the resource group queue but has not yet been executed. Or, you may want to stop a running query that is taking too - long to execute, or one that is sitting idle in a transaction and taking up resource - group transaction slots that are needed by other users.

-

To cancel a queued transaction, you must first determine the process id (pid) - associated with the transaction. Once you have obtained the - process id, you can invoke pg_cancel_backend() to end that process, - as shown below.

+ long to execute, or one that is sitting idle in a transaction and taking up resource group + transaction slots that are needed by other users.

+

To cancel a queued transaction, you must first determine the process id (pid) associated + with the transaction. Once you have obtained the process id, you can invoke + pg_cancel_backend() to end that process, as shown below.

For example, to view the process information associated with all statements currently - active or waiting in all resource groups, run the following query. If the query - returns no results, then there are no running or queued transactions - in any resource group.

+ active or waiting in all resource groups, run the following query. If the query returns no + results, then there are no running or queued transactions in any resource group.

=# SELECT rolname, g.rsgname, procpid, waiting, current_query, datname FROM pg_roles, gp_toolkit.gp_resgroup_status g, pg_stat_activity @@ -379,14 +542,18 @@ gpstart ---------+----------+---------+---------+-----------------------+---------   sammy | rg_light | 31861 | f | <IDLE> in transaction | testdb  billy | rg_light | 31905 | t | SELECT * FROM topten; | testdb -

Use this output to identify the process id (procpid) of the transaction you want to cancel, and then cancel the process. For example, to cancel the pending query identified in the sample output above:

+

Use this output to identify the process id (procpid) of the transaction + you want to cancel, and then cancel the process. For example, to cancel the pending query + identified in the sample output above:

=# SELECT pg_cancel_backend(31905);

-

You can provide an optional message in a second argument to pg_cancel_backend() - to indicate to the user why the process was cancelled.

+

You can provide an optional message in a second argument to + pg_cancel_backend() to indicate to the user why the process was + cancelled.

-

Do not use an operating system KILL command to cancel any Greenplum Database process.

+

Do not use an operating system KILL command to cancel any Greenplum + Database process.

 -- GitLab