From d5a442cd188618d89f2d4729e22f77b45ab322e8 Mon Sep 17 00:00:00 2001 From: Lisa Owen Date: Wed, 25 Apr 2018 09:56:26 -0700 Subject: [PATCH] docs - SQL ref page updates for resgroup memory_auditor (#4860) * docs - sql ref page updates for resgroup memory_auditor * edits from engineering review * some of the edits requested by david * use plural where appropriate --- .../sql_commands/ALTER_RESOURCE_GROUP.xml | 10 +++--- .../ref_guide/sql_commands/ALTER_ROLE.xml | 6 ++-- .../sql_commands/CREATE_RESOURCE_GROUP.xml | 32 +++++++++++++------ .../ref_guide/sql_commands/CREATE_ROLE.xml | 2 +- .../sql_commands/DROP_RESOURCE_GROUP.xml | 6 ++-- 5 files changed, 37 insertions(+), 19 deletions(-) diff --git a/gpdb-doc/dita/ref_guide/sql_commands/ALTER_RESOURCE_GROUP.xml b/gpdb-doc/dita/ref_guide/sql_commands/ALTER_RESOURCE_GROUP.xml index a6b7a811ed..27960ee3a8 100644 --- a/gpdb-doc/dita/ref_guide/sql_commands/ALTER_RESOURCE_GROUP.xml +++ b/gpdb-doc/dita/ref_guide/sql_commands/ALTER_RESOURCE_GROUP.xml @@ -19,11 +19,13 @@ MEMORY_SPILL_RATIO integer Description

ALTER RESOURCE GROUP changes the limits of a resource group. Only a superuser can alter a resource group.

-

You can set or reset the concurrency limit of a resource group to control the maximum +

You can set or reset the concurrency limit of a resource group that you create for roles to control the maximum number of active concurrent statements in that group. You can also reset the memory or CPU rate limit of a resource group to control the amount of memory or CPU resources that all queries submitted through the group can consume on each segment host.

-

The new resource limit is immediately applied if current resource usage is less than or equal to the new value and there are no running transactions in the resource group. If current resource usage exceeds the new limit value, or there are running transactions in other resource groups holding some of the resource, Greenplum Database will defer the new limit assignment until resource usage is within the range of the new value.

+

When you alter the CPU limit of a resource group, the new limit is immediately applied.

+

When you alter a memory limit of a resource group that you create for roles, the new resource limit is immediately applied if current resource usage is less than or equal to the new value and there are no running transactions in the resource group. If the current resource usage exceeds the new memory limit value, or if there are running transactions in other resource groups that hold some of the resource, then Greenplum Database defers assigning the new limit until resource usage falls within the range of the new value.

+

When you increase the memory limit of a resource group that you create for external components, the new resource limit is phased in as system memory resources become available. If you decrease the memory limit of a resource group that you create for external components, the behavior is component-specific. For example, if you decrease the memory limit of a resource group that you create for a PL/Container runtime, queries in a running container may fail with an out of memory error.

You can alter one limit type in a single ALTER RESOURCE GROUP call.

@@ -35,13 +37,13 @@ MEMORY_SPILL_RATIO integer CONCURRENCY integer - The maximum number of concurrent transactions, including active and idle transactions, that are permitted for this resource group. + The maximum number of concurrent transactions, including active and idle transactions, that are permitted for resource groups that you assign to roles. Any transactions submitted after the CONCURRENCY value limit is reached are queued. When a running transaction completes, the earliest queued transaction is executed. The CONCURRENCY value must be an integer in the range [0 .. max_connections]. The default - CONCURRENCY value is 20. + CONCURRENCY value for a resource group that you create for roles is 20. You cannot set the CONCURRENCY value for the admin_group to zero (0). diff --git a/gpdb-doc/dita/ref_guide/sql_commands/ALTER_ROLE.xml b/gpdb-doc/dita/ref_guide/sql_commands/ALTER_ROLE.xml index 249b7411b8..d30c1c2b1f 100644 --- a/gpdb-doc/dita/ref_guide/sql_commands/ALTER_ROLE.xml +++ b/gpdb-doc/dita/ref_guide/sql_commands/ALTER_ROLE.xml @@ -65,7 +65,8 @@ ALTER ROLE name [ [WITH] option [ ... ] ]< more information.
  • RESOURCE GROUP — Assigns a resource group to the role. The role would then be subject to the concurrent transaction, memory, and CPU limits configured for - the resource group. You can assign a single resource group to one or more roles. See + the resource group. You can assign a single resource group to one or more roles. You + cannot assign a resource group that you create for an external component to a role. See for additional information.
  • WITH option — Changes many of the role @@ -105,7 +106,8 @@ ALTER ROLE name [ [WITH] option [ ... ] ]< NONE removes the role's current resource group assignment and assigns a default resource group based on the role's capability. SUPERUSER roles are assigned the admin_group resource group, while the - default_group resource group is assigned to non-admin roles. + default_group resource group is assigned to non-admin roles. + You cannot assign a resource group that you create for an external component to a role. queue_name diff --git a/gpdb-doc/dita/ref_guide/sql_commands/CREATE_RESOURCE_GROUP.xml b/gpdb-doc/dita/ref_guide/sql_commands/CREATE_RESOURCE_GROUP.xml index 2b557b790c..1ce5a13d90 100644 --- a/gpdb-doc/dita/ref_guide/sql_commands/CREATE_RESOURCE_GROUP.xml +++ b/gpdb-doc/dita/ref_guide/sql_commands/CREATE_RESOURCE_GROUP.xml @@ -13,23 +13,29 @@ MEMORY_LIMIT=integer [ CONCURRENCY=integer ] [ MEMORY_SHARED_QUOTA=integer ] -[ MEMORY_SPILL_RATIO=integer ] +[ MEMORY_SPILL_RATIO=integer ] +[ MEMORY_AUDITOR= {vmtracker | cgroup} ]
    • Description -

    Creates a new resource group for Greenplum Database resource management. A resource - group is assigned to one or more roles and identifies concurrent transaction, memory, - and CPU limits for the role when resource groups are enabled.

    +

    Creates a new resource group for Greenplum Database resource management. You can create resource groups to manage resources for roles or + to manage the resources of a Greenplum Database external component such as PL/Container.

    +

    A resource + group that you create to manage a user role identifies concurrent transaction, memory, + and CPU limits for the role when resource groups are enabled. You may assign such resource groups to one or more roles.

    +

    A resource group that you create to manage the resources of a Greenplum Database external component such as PL/Container identifies the memory and CPU limits for the component when resource groups are enabled. These resource groups use cgroups to manage both CPU and memory management. Assignment of resource groups to external components is component-specific. For example, you assign a PL/Container resource group when you configure a PL/Container runtime. You cannot assign a resource group that you create for external components to a role, nor can you assign a resource group that you create for roles to an external component.

    You must have SUPERUSER privileges to create a resource group. The maximum number of resource groups allowed in your Greenplum Database cluster is 100.

    Greenplum Database pre-defines two default resource groups: admin_group and default_group. These group names, as well as the group name none, are reserved.

    To set appropriate limits for resource groups, the Greenplum Database administrator must - be familiar with the queries typically executed on the system, as well as the users/roles executing those queries.

    + be familiar with the queries typically executed on the system, as well as the users/roles executing those queries and the external components they may be using, such as PL/Containers.

    - After creating a resource group, assign the group to one or more roles using the or commands.

    +

    + After you create a resource group to manage the CPU and memory resources of an external component, configure the external component to use the resource group. For example, configure the PL/Container runtime resource_group_id.

    Parameters @@ -42,8 +48,8 @@ MEMORY_LIMIT=integer CONCURRENCY integer - The maximum number of concurrent transactions, including active and idle transactions, that are permitted for this resource group. The CONCURRENCY value must be an integer in the range [0 .. max_connections]. The default CONCURRENCY value is 20. - + The maximum number of concurrent transactions, including active and idle transactions, that are permitted for this resource group. The CONCURRENCY value must be an integer in the range [0 .. max_connections]. The default CONCURRENCY value for resource groups defined for roles is 20. + You must set CONCURRENCY to zero (0) for resource groups that you create for external components. You cannot set the CONCURRENCY value for the admin_group to zero (0). @@ -63,6 +69,11 @@ MEMORY_LIMIT=integer MEMORY_SPILL_RATIO integer The memory usage threshold for memory-intensive operators in a transaction. When the transaction reaches this threshold, it spills to disk. The minimum memory spill ratio percentage you can specify for a resource group is 0. The maximum is 100. The default MEMORY_SPILL_RATIO value is 20. + + MEMORY_AUDITOR {vmtracker | cgroup} + The memory auditor for the resource group. Greenplum Database employs virtual memory tracking for role resources and cgroup memory tracking for resources used by external components. The default MEMORY_AUDITOR is vmtracker. When you create a resource group with vmtracker memory auditing, Greenplum Database tracks that resource group's memory internally. + When you create a resource group specifying the cgroup MEMORY_AUDITOR, Greenplum Database defers the accounting of memory used by that resource group to cgroups. CONCURRENCY must be zero (0) for a resource group that you create for external components such as PL/Container. You cannot assign a resource group that you create for external components to a Greenplum Database role. +
    Notes @@ -75,9 +86,12 @@ MEMORY_LIMIT=integer Examples

    Create a resource group with CPU and memory limit percentages of 35:

    CREATE RESOURCE GROUP rgroup1 WITH (CPU_RATE_LIMIT=35, MEMORY_LIMIT=35); -

    Create a resource group with a concurrent transaction limit of 30, a memory limit of 15, and a CPU limit of 25:

    +

    Create a resource group with a concurrent transaction limit of 20, a memory limit of 15, and a CPU limit of 25:

    CREATE RESOURCE GROUP rgroup2 WITH (CONCURRENCY=20, MEMORY_LIMIT=15, CPU_RATE_LIMIT=25); +

    Create a resource group to manage PL/Container resources specifying a memory limit of 10, and a CPU limit of 10:

    + CREATE RESOURCE GROUP plc_run1 WITH (MEMORY_LIMIT=10, CPU_RATE_LIMIT=10, + CONCURRENCY=0, MEMORY_AUDITOR=cgroup);
    Compatibility diff --git a/gpdb-doc/dita/ref_guide/sql_commands/CREATE_ROLE.xml b/gpdb-doc/dita/ref_guide/sql_commands/CREATE_ROLE.xml index 3838aee3cb..9aa8e20258 100644 --- a/gpdb-doc/dita/ref_guide/sql_commands/CREATE_ROLE.xml +++ b/gpdb-doc/dita/ref_guide/sql_commands/CREATE_ROLE.xml @@ -179,7 +179,7 @@ having the SUPERUSER attribute. You can assign the default_group resource group to any role. - + You cannot assign a resource group that you create for an external component to a role. RESOURCE QUEUE queue_name The name of the resource queue to which the new user-level role is to be diff --git a/gpdb-doc/dita/ref_guide/sql_commands/DROP_RESOURCE_GROUP.xml b/gpdb-doc/dita/ref_guide/sql_commands/DROP_RESOURCE_GROUP.xml index 188bfcd9cb..529945dc66 100644 --- a/gpdb-doc/dita/ref_guide/sql_commands/DROP_RESOURCE_GROUP.xml +++ b/gpdb-doc/dita/ref_guide/sql_commands/DROP_RESOURCE_GROUP.xml @@ -4,9 +4,9 @@ DROP RESOURCE GROUP

    Removes a resource group.

    SynopsisDROP RESOURCE GROUP group_name
    Description

    This command removes a resource group from Greenplum Database. Only a superuser -can drop a resource group.

    To drop a resource group, the group cannot be assigned to any roles, -nor can it have any statements pending or running in the group.

    You cannot drop the pre-defined admin_group and default_group resource groups.

    -
    Parametersgroup_nameThe name of the resource group to remove.
    Notes

    You cannot submit a DROP RESOURCE GROUP command in an explicit transaction or sub-transaction.

    Use to remove the resource group assigned +can drop a resource group.

    To drop a role resource group, the group cannot be assigned to any roles, +nor can it have any statements pending or running in the group. If you drop a resource group that you created for an external component, the behavior is determined by the external component. For example, dropping a resource group that you assigned to a PL/Container runtime kills running containers in the group.

    You cannot drop the pre-defined admin_group and default_group resource groups.

    +
    Parametersgroup_nameThe name of the resource group to remove.
    Notes

    You cannot submit a DROP RESOURCE GROUP command in an explicit transaction or sub-transaction.

    Use to remove a resource group assigned to a specific user/role.

    Perform the following query to view all of the currently active queries for all resource groups:

    SELECT usename, current_query, waiting, procpid, rsgid, rsgname, rsgqueueduration -- GitLab