Skip to content

G
Gpdb

Greenplum / Gpdb

通知 7
Star 1
Fork 0
G
Gpdb

体验新版 GitCode，发现更多精彩内容 >>

提交 310fb67c 编写于 作者: C Chuck Litzell 提交者： David Yozie
浏览文件
操作

Docs - add diskquota module (#7942) 

* docs - add diskquota module to reference guide

* Add missing quote; minor edits

* Make module doc more consistent with other module docs.

* Updates from reviews

* Add section about shared memory and diskquota.max_active_tables
上级 706d7774
隐藏空白更改
内联 并排
Showing with 237 addition and 0 deletion
gpdb-doc/dita/ref_guide/modules/diskquota.xml 0 → 100644
浏览文件 @ 310fb67c
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE topic PUBLIC "-//OASIS//DTD DITA Topic//EN" "topic.dtd">
<topic id="topic_gzw_2wz_13b">
<title>diskquota</title>
<body>
<p>The <codeph>diskquota</codeph> module allows Greenplum Database administrators to limit the
amount of disk space used by schemas or roles in a database. </p>
</body>
<topic id="topic_ofb_gb1_b3b">
<title>Installing and Registering the Module</title>
<body>
<p>The <codeph>diskquota</codeph> module is installed when you install Greenplum Database.</p>
<p>Before you can use the module, you must perform these steps:</p>
<ol id="ol_pfb_gb1_b3b">
<li>Create the <codeph>diskquota</codeph> database. The <codeph>diskquota</codeph> module
uses this database to store the list of databases where the module is enabled.
<codeblock>$ createdb diskquota;</codeblock></li>
<li> Add the <codeph>diskquota</codeph> shared library to the Greenplum Database
<codeph>shared_preload_libraries</codeph> server configuration parameter and restart
Greenplum
Database.<codeblock>$ gpconfig -c shared_preload_libraries -v 'diskquota'
$ gpstop -ar</codeblock></li>
<li> Register the <codeph>diskquota</codeph> extension in databases where you want to
enforce disk usage quotas. <codeph>diskquota</codeph> can be registered in up to ten
databases. <codeblock>$ psql -d testdb -c "CREATE EXTENSION diskquota"</codeblock></li>
<li>If you register the <codeph>diskquota</codeph> extension in a database that already
contains data, you must initialize the <codeph>diskquota</codeph> table size data by
running the <codeph>diskquota.init_table_size_table()</codeph> UDF in the database. In a
database with many files, this can take a long time. The <codeph>diskquota</codeph> module
cannot be used until the initialization is
complete.<codeblock>=# SELECT diskquota.init_table_size_table();</codeblock>
</li>
</ol>
</body>
</topic>
<topic id="topic_ndp_4wy_c3b">
<title>About the diskquota Module</title>
<body>
<p>A Greenplum Database superuser can set disk usage quotas for schemas and roles. A schema
quota sets a limit on disk space used by all tables that belong to a schema. A role quota
sets a limit on disk space used by all tables that are owned by a role.</p>
<p>Diskquota processes running on the master and segment hosts check disk usage periodically
and place schemas or roles on a blacklist when they reach their quota. </p>
<p>When a query plan has been generated for a query that would add data, and the schema or
role is on the blacklist, the query is cancelled before it can start. An error message
reports the quota that has been exceeded. A query that does not add data, such as a simple
<codeph>SELECT</codeph> query, is allowed to run even when the role or schema is on the
blacklist.</p>
<p>Diskquota enforces <i>soft limits</i> for disk usage. Quotas are only checked before a
query executes. If the quota is not exceeded when the query is about to run, the query is
allowed to run, even if it causes the quota to be exceeded. </p>
<p>There is some delay after a quota has been reached before the schema or role is added to
the blacklist. Other queries could add more data during the delay. The delay occurs because
<codeph>diskquota</codeph> processes that calculate the disk space used by each table
execute periodically with a pause between executions (two seconds by default). The delay
also occurs when disk usage falls beneath a quota, due to operations such as
<codeph>DROP</codeph>, <codeph>TRUNCATE</codeph>, or <codeph>VACUUM FULL</codeph> that
remove data. Administrators can change the amount of time between disk space checks by
setting the <codeph>diskquota.naptime</codeph> server configuration parameter.</p>
<p>If a query is unable to run because the schema or role has been blacklisted, an
administrator can increase the exceeded quota to allow the query to execute. The
<codeph>show_fast_schema_quota_view</codeph> and
<codeph>show_fast_role_quota_view</codeph> views can be used to find the schemas or roles
that have exceeded their limits.</p>
</body>
</topic>
<topic id="topic_qfb_gb1_b3b">
<title>Using the diskquota Module</title>
<body>
<section>
<title>Setting Disk Quotas</title>
<p>Use the <codeph>diskquota.set_schema_quota()</codeph> and
<codeph>diskquota.set_role_quota()</codeph> user-defined functions in a database to set,
update, or delete disk quota limits for schemas and roles in the database. The functions
take two arguments: the schema or role name, and the quota to set. The quota can be
specified in units of MB, GB, TB, or PB, for example '2TB'. </p>
<p>The following example sets a 250GB quota for the <codeph>acct</codeph> schema:</p>
<codeblock>$ SELECT diskquota.set_schema_quota('acct', '250GB');</codeblock>
<p>This example sets a 500MB quota for the <codeph>nickd</codeph> role:</p>
<codeblock>$ SELECT diskquota.set_role_quota(nickd, '500MB');</codeblock>
<p>To change a quota, call the <codeph>diskquota.set_schema_quota()</codeph> or
<codeph>diskquota.set_role_quota()</codeph> function again with the new quota value.</p>
<p>To remove a quota, set the quota value to <codeph>'-1'</codeph>.</p>
</section>
<section><title>Displaying Disk Quotas and Disk Usage</title><p>The <codeph>diskquota</codeph>
module provides two views to display active quotas and the current computed disk space
used.</p>The <codeph>diskquota.show_fast_schema_quota_view</codeph> view lists active
quotas for schemas in the current database. The <codeph>nspsize_in_bytes</codeph> column
contains the calculated size for all tables that belong to the schema.
<codeblock>=# <b>SELECT * FROM diskquota.show_fast_schema_quota_view;</b>
schema_name | schema_oid | quota_in_mb | nspsize_in_bytes
-------------+------------+-------------+------------------
acct | 16561 | 256000 | 131072
analytics | 16519 | 1073741824 | 144670720
eng | 16560 | 5242880 | 117833728
public | 2200 | 250 | 3014656
(4 rows)</codeblock><p>The
<codeph>diskquota.show_fast_role_quota_view</codeph> view lists the active quotas for
roles in the current database. The <codeph>rolsize_in_bytes</codeph> column contains the
calculated size for all tables that are owned by the
role.</p><codeblock>=# <b>SELECT * FROM diskquota.show_fast_role_quota_view;</b>
role_name | role_oid | quota_in_mb | rolsize_in_bytes
-----------+----------+-------------+------------------
mdach | 16558 | 500 | 131072
adam | 16557 | 300 | 117833728
nickd | 16577 | 500 | 144670720
(3 rows)</codeblock></section>
<section><title>Setting the Delay Between Disk Usage Updates</title>The
<codeph>diskquota.naptime</codeph> server configuration parameter specifies how frequently
(in seconds) the table sizes are recalculated. The smaller the <codeph>naptime</codeph>
value, the less delay in detecting changes in disk usage. This example sets the
<codeph>naptime</codeph> to ten seconds.
<codeblock>$ gpconfig -c diskquota.naptime -v 10</codeblock></section>
<section>
<title>Configuring diskquota Shared Memory</title>
<p>The <codeph>diskquota</codeph> module uses shared memory to save the blacklist and to
save the active table list.</p>
<p>The blacklist shared memory can hold up to 1MiB of database objects that exceed the quota
limit. If the blacklist shared memory fills, data may be loaded into some schemas or roles
after they have reached their quota limit. </p>
<p>Active table shared memory holds up to 1MiB of active tables by default. Active tables
are tables that may have changed sizes since <codeph>diskquota</codeph> last recalculated
the table sizes. <codeph>diskquota</codeph> hook functions are called when the storage
manager on each Greenplum Database segment creates, extends, or truncates a table file.
The hook functions store the identity of the file in shared memory so that its file size
can be recalculated the next time the table size data is refreshed. </p>
<p>If the shared memory for active tables fills, <codeph>diskquota</codeph> may fail to
detect a change in disk usage. The amount of active table shared memory can be adjusted by
setting the <codeph>diskquota.max_active_tables</codeph> server configuration parameter.
This example changes the active table shared memory to
2MiB:<codeblock>$ gpconfig -c diskquota.max_active_tables -v '2MiB'</codeblock></p>
<p>Shared memory is allocated when Greenplum Database starts up, so a server restart is
required after you change the value of the <codeph>diskquota.max_active_tables</codeph>
parameter.</p>
</section>
</body>
</topic>
<topic id="topic_sfb_gb1_b3b">
<title>Notes</title>
<body>
<p>The <codeph>diskquota</codeph> module can be enabled in up to ten databases. One diskquota
worker process is created on the Greenplum Database master host for each diskquota-enabled
database.</p>
<p>The disk usage for a role is defined as the total of disk usage on all segments for all
tables the role owns. Although a role is a cluster-level database object, the disk usage for
roles is calculated separately for each database. </p>
<p>The disk usage of a schema is defined as the total of disk usage on all segments for all
tables in the schema. </p>
<p>The disk usage for a table includes the table data, indexes, toast tables, and free space
map. For append-optimized tables, the calculation includes the visibility map and index, and
the block directory table.</p>
<p>The <codeph>diskquota</codeph> module cannot detect a newly created table inside of an
uncommited transaction. The size of the new table is not included in the disk usage
calculated for the corresponding schema or role until after the transaction has committed.
Similarly, a table created using the <codeph>CREATE TABLE AS</codeph> command is not
included in disk usage statistics until the command has completed.</p>
<p>Deleting rows or running <codeph>VACUUM</codeph> on a table does not release disk space, so
these operations cannot alone remove a schema or role from the <codeph>diskquota</codeph>
blacklist. The disk space used by a table can be reduced by running <codeph>VACUUM
FULL</codeph> or <codeph>TRUNCATE TABLE</codeph>.</p>
<p>The <codeph>diskquota</codeph> module supports high availability features provided by the
background worker framework. The <codeph>diskquota</codeph> launcher process only runs on
the active master node. The postmaster on the standby master does not start the
<codeph>diskquota</codeph> launcher process when it is in standby mode. When the master is
down and the administrator runs the <codeph><xref
href="../../utility_guide/admin_utilities/gpactivatestandby.xml#topic1"/></codeph>
command, the standby master changes its role to master and the <codeph>diskquota</codeph>
launcher process is forked automatically. Using the <codeph>diskquota</codeph>-enabled
database list in the <codeph>diskquota</codeph> database, the <codeph>diskquota</codeph>
launcher creates the <codeph>diskquota</codeph> worker processes that manage disk quotas for
each database.</p>
</body>
</topic>
<topic id="topic_v2z_jrv_b3b">
<title>Examples</title>
<body>
<p>This example demonstrates how to set up a schema quota and then observe diskquota behavior
as data is added to the schema.</p>
<ol id="ol_rfb_gb1_b3b">
<li>Create a database named <codeph>test</codeph> and log in to
it.<codeblock>$ <b>createdb test</b>
$ <b>psql -d test</b></codeblock></li>
<li>Create the diskquota extension in the
database.<codeblock>=# <b>CREATE EXTENSION diskquota;</b>
CREATE EXTENSION</codeblock></li>
<li> Create the <codeph>s1</codeph>
schema.<codeblock>=# <b>CREATE SCHEMA s1;</b>
CREATE SCHEMA</codeblock></li>
<li>Set a 1MB disk quota for the <codeph>s1</codeph>
schema.<codeblock>=# <b>SELECT diskquota.set_schema_quota('s1', '1MB');</b>
set_schema_quota
------------------
(1 row)</codeblock></li>
<li>The following commands create a table in the <codeph>s1</codeph> schema and insert a
small amount of data into it. The schema has no data yet, so it is not on the blacklist.
<codeblock>=# <b>SET search_path TO s1;
SET
</b>=# <b>CREATE TABLE a(i int);</b>
CREATE TABLE
=# <b>INSERT INTO a SELECT generate_series(1,100);</b>
INSERT 0 100
</codeblock></li>
<li>This command inserts a large amount of data, enough to exceed the 1MB quota that was set
for the schema. Before the <codeph>INSERT</codeph> command, the <codeph>s1</codeph> schema
is still not on the blacklist, so this command should be allowed to run, even though it
will exceed the limit set for the
schema.<codeblock>=# <b>INSERT INTO a SELECT generate_series(1,10000000);</b>
INSERT 0 10000000
</codeblock></li>
<li>This command attempts to insert a small amount of data. Because the previous command
exceeded the schema's disk quota limit, the schema should be blacklisted and any data
loading command should be cancelled.
<codeblock>=# <b>INSERT INTO a SELECT generate_series(1,100);</b>
ERROR: schema's disk space quota exceeded with name:s1
</codeblock></li>
<li>This command removes the quota from the <codeph>s1</codeph> schema by setting it to
<codeph>-1</codeph> and again inserts a small amount of data. A 5-second sleep before
the <codeph>INSERT</codeph> command ensures that the <codeph>diskquota</codeph> table size
data is updated before the command is
run.<codeblock>=# <b>SELECT diskquota.set_schema_quota('s1', '-1');</b>
set_schema_quota
------------------
(1 row)
# Wait for 5 seconds to ensure the blacklist is updated
#= <b>SELECT pg_sleep(5);</b>
#= <b>INSERT INTO a SELECT generate_series(1,100);</b>
INSERT 0 100
</codeblock></li>
</ol>
</body>
</topic>
</topic>
gpdb-doc/dita/ref_guide/modules/intro.xml
浏览文件 @ 310fb67c
......@@ -20,6 +20,8 @@
data type.</li>
<li><codeph><xref href="dblink.xml" format="dita"
>dblink</xref></codeph> - Provides connections to other Greenplum databases.</li>
<li><codeph><xref href="diskquota.xml#topic_gzw_2wz_13b"/></codeph> - Allows administrators to
set disk usage quotas for Greenplum Database roles and schemas.</li>
<li><codeph><xref href="fuzzystrmatch.xml"
format="dita">fuzzystrmatch</xref></codeph> - Determines similarities and differences
between strings.</li>
......
gpdb-doc/dita/ref_guide/modules/modules.ditamap
浏览文件 @ 310fb67c
......@@ -5,6 +5,7 @@
<topicref href="auto-explain.xml"/>
<topicref href="citext.xml" linking="none"/>
<topicref href="dblink.xml" linking="none"/>
<topicref href="diskquota.xml" linking="none"/>
<topicref href="fuzzystrmatch.xml" linking="none"/>
<topicref href="gp_sparse_vector.xml" linking="none"/>
<topicref href="hstore.xml" linking="none"/>
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或