From 2d114130841e9480e0b7de458ed477f02efd42fd Mon Sep 17 00:00:00 2001 From: dyozie Date: Tue, 7 Nov 2017 15:18:22 -0800 Subject: [PATCH] Docs - add info about filtering leaf partitions with gpbackup --- .../admin_guide/managing/backup-gpbackup.xml | 59 ++++++++++++++++++- .../admin_utilities/gpbackup.xml | 24 +++++--- 2 files changed, 74 insertions(+), 9 deletions(-) diff --git a/gpdb-doc/dita/admin_guide/managing/backup-gpbackup.xml b/gpdb-doc/dita/admin_guide/managing/backup-gpbackup.xml index 144e71980a..4539f02237 100644 --- a/gpdb-doc/dita/admin_guide/managing/backup-gpbackup.xml +++ b/gpdb-doc/dita/admin_guide/managing/backup-gpbackup.xml @@ -20,9 +20,10 @@ format="dita"/> for more information.) Storing the table data in CSV files also provides opportunities for using other restore utilities, such as gpload, to load the data either in the same cluster or another cluster. By default, one file is created for each - table on the segment. You can specify the -leaf-partition-data option with + table on the segment. You can specify the -leaf-partition-data option with gpbackup to create one data file per leaf partition of a partitioned table, - instead of a single file.

+ instead of a single file. This option also enables you to filter backup sets by leaf + partitions.

Each gpbackup task uses a single transaction in Greenplum Database. During this transaction, metadata is backed up on the master host, and data for each table on each segment host is written to CSV backup files using COPY .. ON SEGMENT commands @@ -41,6 +42,10 @@ requires a distinct timestamp.

  • Database object filtering is currently limited to schemas and tables.
  • Filtering objects with gprestore is not yet supported.
    • +
  • You cannot use the -exclude-table-file with + -leaf-partition-data. Although you can specify leaf partition names + in a file specified with -exclude-table-file, gpbackup + ignores the partition names.
  • Incremental backups are not supported.

    • @@ -231,6 +236,56 @@ water.tonic

    gpbackup options -include-table-file and -exclude-table-file. For example:$ gpbackup -dbname demo -include-table-file /home/gpadmin/table-list.txt

    +
    + Filtering by Leaf Partition +

    By default, gpbackup creates one file for each table on a segment. You + can specify the -leaf-partition-data option with to create one data file + per leaf partition of a partitioned table, instead of a single file. This option also + enables you to filter backups to specific leaf partitions by listing the partition names + in a text file to include. For example, consider a table that was created using the + statement:demo=# CREATE TABLE sales (id int, date date, amt decimal(10,2)) +DISTRIBUTED BY (id) +PARTITION BY RANGE (date) +( PARTITION Jan17 START (date '2017-01-01') INCLUSIVE , +PARTITION Feb17 START (date '2017-02-01') INCLUSIVE , +PARTITION Mar17 START (date '2017-03-01') INCLUSIVE , +PARTITION Apr17 START (date '2017-04-01') INCLUSIVE , +PARTITION May17 START (date '2017-05-01') INCLUSIVE , +PARTITION Jun17 START (date '2017-06-01') INCLUSIVE , +PARTITION Jul17 START (date '2017-07-01') INCLUSIVE , +PARTITION Aug17 START (date '2017-08-01') INCLUSIVE , +PARTITION Sep17 START (date '2017-09-01') INCLUSIVE , +PARTITION Oct17 START (date '2017-10-01') INCLUSIVE , +PARTITION Nov17 START (date '2017-11-01') INCLUSIVE , +PARTITION Dec17 START (date '2017-12-01') INCLUSIVE +END (date '2018-01-01') EXCLUSIVE ); +NOTICE: CREATE TABLE will create partition "sales_1_prt_jan17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_feb17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_mar17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_apr17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_may17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_jun17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_jul17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_aug17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_sep17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_oct17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_nov17" for table "sales" +NOTICE: CREATE TABLE will create partition "sales_1_prt_dec17" for table "sales" +CREATE TABLE

    +

    To back up only data for the last quarter of the year, first create a text file that + lists those leaf partition names instead of the full table + name:public.sales_1_prt_oct17 +public.sales_1_prt_nov17 +public.sales_1_prt_dec17 +

    +

    Then specify the file with the -include-table-file option, and also + specify -leaf-partition-data to generate one data file per leaf + partition:$ gpbackup -dbname demo -include-table-file last-quarter.txt -leaf-partition-data

    + You cannot use the -exclude-table-file with + -leaf-partition-data. Although you can specify leaf partition names in + a file specified with -exclude-table-file, gpbackup + ignores the partition names. +
    diff --git a/gpdb-doc/dita/utility_guide/admin_utilities/gpbackup.xml b/gpdb-doc/dita/utility_guide/admin_utilities/gpbackup.xml index fffabee2b3..46f192dcc3 100644 --- a/gpdb-doc/dita/utility_guide/admin_utilities/gpbackup.xml +++ b/gpdb-doc/dita/utility_guide/admin_utilities/gpbackup.xml @@ -106,9 +106,13 @@ backup. Each line in the text file must define a single table using the format <schema-name>.<table-name>. The file must not include trailing lines. If a table or schema name uses any character other than a lowercase letter, - number, or an underscore character, then you must include that name in double quotes. - See for - more information. + number, or an underscore character, then you must include that name in double quotes. + You cannot use this option in combination with -leaf-partition-data. + Although you can specify leaf partition names in a file specified with + -exclude-table-file, gpbackup ignores the partition + names. + See + for more information. @@ -133,16 +137,22 @@ <schema-name>.<table-name>. The file must not include trailing lines. If a table or schema name uses any character other than a lowercase letter, number, or an underscore character, then you must include that name in double quotes. - Any tables not listed in this file are omitted from the backup set. See for more - information. + Any tables not listed in this file are omitted from the backup set. + You can optionally specify a table leaf partition name in place of the table name, to + include only specific leaf partitions in a backup with the + -leaf-partition-data option. + See + for more information. -leaf-partition-data Optional. For partitioned tables, creates one data file per leaf partition instead of - one data file for the entire table (the default). + one data file for the entire table (the default). Using this option also enables you to + specify individual leaf partitions to include in a backup, with the + -include-table-file option. You cannot use this option in combination + with -exclude-table-file. -- GitLab