As we merge with upstream and by that keep refining the Postgres
planner, legacy planner is no longer a suitable name. This changes
all variations of the spelling (legacy planner, legacy optimizer,
legacy query optimizer etc) to say "Postgres" rather than "legacy".
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>
Reviewed-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>

* docs - GDD - add information about specific type of deadlocks.

* docs - GDD - update deadlock information based on review comments.

* typo fix

* Small edit for clarity

* Removing duplicate info

Changing version in path from /600 to /6-0 for consistency with other docs and to recognize that only the minor version digit is significant for the doc build (#7141)

The GUC which enables recursive CTEs is in the currently released
version called gp_recursive_cte_prototype, but in order to reflect
the current state of the code it's now renamed to gp_recursive_cte.
By default the GUC is still off, but that might change before we
ship the next release.

The previous GUC name is still supported, but marked as deprecated,
in order to make upgrades easier.
Reviewed-by: NIvan Novick <inovick@pivotal.io>
Reviewed-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>

This method was introduced to improve the data redistribution
performance during gpexpand phase2, however per benchmark results the
effect does not reach our expectation.  For example when expanding a
table from 7 segments to 8 segments the reshuffle method is only 30%
faster than the traditional CTAS method, when expanding from 4 to 8
segments reshuffle is even 10% slower than CTAS.  When there are indexes
on the table the reshuffle performance can be worse, and extra VACUUM is
needed to actually free the disk space.  According to our experiments
the bottleneck of reshuffle method is on the tuple deletion operation,
it is much slower than the insertion operation used by CTAS.

The reshuffle method does have some benefits, it requires less extra
disk space, it also requires less network bandwidth (similar to CTAS
method with the new JCH reduce method, but less than CTAS + MOD).  And
it can be faster in some cases, however as we can not automatically
determine when it is faster it is not easy to get benefit from it in
practice.

On the other side the reshuffle method is less tested, it is possible to
have bugs in corner cases, so it is not production ready yet.

In such a case we decided to retire it entirely for now, we might add it
back in the future if we can get rid of the slow deletion or find out
reliable ways to automatically choose between reshuffle and ctas
methods.

Discussion: https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/8xknWag-SkI/5OsIhZWdDgAJReviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>

Make sure there is whitespace before xref tags when used in normal
text to avoid links squashed together with text.
Reviewed-by: NLisa Owen <lowen@pivotal.io>

* docs - CTE available with INSERT, UPDATE, DELETE

-updated GUC
-updated Admin Guide topic WITH Queries (Common Table Expressions)

updates to SELECT, INSERT, UPDATE DELETE will be part of postgres 9.2 merge.

* docs - CTE updates from review comments

* docs - CTE more updates from review comments

* docs - CTE - updates from review comments

* Experimental -> Beta wording

* vacuumdb - add -t option support for multiple tables

* reindexdb - add -t option support for multiple tables; misc edits

* psql - misc additions, edits, and reformats

* pg_dump - misc edits, additions, formatting

* add * syntax for descendant tables

* pg_dumpall - add -d, --dbname option

* PREPARE - add note about search_path

* TRUNCATE - add * option

* SELECT - add LATERAL, NO KEY UPDATE, KEY SHARE, and related edits to locking clause

* ALTER ROLE. Re-order syntax and descriptions.

* ALTER RULE. Add new command and fix some conref issues

* COPY. Add FREEZE option and edits.

* CREATE INDEX. Minor synopsis update.

* CREATE SCHEMA. Add IF NOT EXISTS forms of command

* CREATE TABLE. Syntax changes. Add SET CONSTRAINTS command.

* bgworker. Add bgworker topic to reference guide

* CREATE TABLE AS. Add WITH/WITHOUT OIDS

* Changes from review

* remove some parens, add trigger note

* remove extra whitespace in CREATE TABLE

* CREATE TABLE AS - correct DISTRIBUTED BY syntax

* Review comments on CREATE TABLE.

* pg_dump - add parallelization note

* CREATE TABLE - fix whitespace again?

* Remove docs for LATERAL

* Add LATERAL to unsupported feature list

* reword from 'experimental' to 'beta'

* Experimental -> Beta in markdown source

* typo fix

* Removing SuSE 11 details in Beta notes

* Warn not to run gpbackup/gprestore during expand.

* Updates from review

* docs - add new host/seg resource group usage views

* clarifications requested by ning

* vacuumdb - add missing equal signs =

* CREATE/ALTER/DROP COLLATION. Adds new references for these commands.

* remove space

* DROP COLLATION. Remove redundant privileges statement.

* SELECT. Add DISTINCT to several clauses, and may edits.

* CREATE TABLE. Adds NO INHERIT, IF NOT EXISTS, column_reference_storage_directive syntax, edits.

* ALTER DOMAIN. Add new forms of command.

* ALTER EXTENSION. Small edit only.

* ALTER FUNCTION. Add LEAKPROOF.

* ALTER_INDEX. Change SET/RESET FILLFACTOR to SET (fillfactor=)

* ALTER OPERATOR FAMILY. Add SP_GiST to descriptions.

* ALTER SEQUENCE. Add IF EXISTS.

* ALTER SERVER. Small edits.

* Add Range Type section & related changes

* ALTER TYPE - add UAGE privilege requirement

* ALTER VIEW - add IF EXISTS keyword; add syntax for view settings

* ANALYZE - add info about foreign tables

* CHECKPOINT - remove WAL paragraph; other edits

* ALTER TABLE. Add IF EXISTS, constraints, edits

* CREATE VIEW - add view option syntax

* CREATE OPERATOR TYPE - minor edits

* CREATE OPERATOR - add USAGE requirement

* createdb - new maintenance-db option; minor edits; simplify synopsis to be consistent with util output & postgresql docs

* createlang - add note about lower-casing the name

* createuser - add default notices, --interactive option, update examples

* DELETE - fix codeph style

* DROP INDEX - add CONCURRENTLY option

* DROP TABLE - small edit to permissions required

* dropdb - add --maintenance-db option

* droplang - add lowercase notice

* dropuser - add --if-exists; edits around prompting.

* clusterdb. Add --maintenance-db connection option.

* COMMENT. Replace table_name argument with relation_name

* CREATE AGGREGATE. Add privileges paragraph.

* CREATE CAST. Add privileges required.

* CREATE DOMAIN. Add required privileges, edit example.

* CREATE FUNCTION. Add LEAKPROOF function attribute.

* CREATE INDEX. add BUFFERING storage parameter.

* CREATE LANGUAGE. minor edit.

* CREATE TABLE AS. Edits. Deprecate GLOBAL/LOCAL. Also update SELECT and CREATE TABLE to enable links.

CREATE TABLE AS. Edits. Deprecate GLOBAL/LOCAL. Also update CREATE TABLE to enable links.

* CREATE ROLE. minor edits.

* GRANT - add USAGE ON DOMAIN, ON TYPE, with related notes

* EXPLAIN. Add BUFFERS option. Fix missing query in example.

* PREPARE. Edits.

* SELECT. add values clause  to with_query, updates select list, ORDER BY, LIMIT, FOR UPDATE/FOR SHARE clauses

* DELETE. updates to with_query for data-modifying commands.

* INSERT - updates to with_query for data-modifying commands.

* UPDATE - updates to with_query for data-modifying commands.

* REVOKE. add DOMAIN and TYPE variants

* SET. minor edit.

* pg_dump. Add --section and related parameters.

* pg_restore - add --section option and related text/edits

* reindexdb - add --maintenancedb connection option

* vacuumdb - add --maintenancedb connection option

* SET TRANSACTION. - snapshot not supported. minor edit.

* postgresql - edits, additions introduced in 9.2; major omissions and re-writes were sourced from the 9.4 docs

* Changes from review

* misc edits

* misc edit

* Remove outputclass tags from sgml conversion

* reformat converted files

rank column has been deleted in e2c8b178
As this column is very useful for user to adjust the expansion table
order, so we get it back.

- Remove GP/WLM
- Change GPCC entries to gpperfmon (for gpmmon/gpsmon)
- Add GPCC gpccws http and rpc ports and ccagent
- Add GPText ZooKeepr and Solr ports

Update ports and protocols page for gp-wlm, gpcc, and gptext

Clarify GPText use a range of ports. Make GPCC and GPText rows conditional.

* vacuumdb - add missing equal signs =

* CREATE/ALTER/DROP COLLATION. Adds new references for these commands.

* ALTER CONVERSION. Add SET SCHEMA variant.

* ALTER OPERATOR. add SET SCHEMA.

* ALTER OPERATOR CLASS. add SET SCHEMA

* ALTER OPERATOR FAMILY. add SET SCHEMA, FOR SEARCH/FOR ORDER BY

* ALTER ROLE. add REPLICATION/NOREPLICATION options

* ALTER TABLE. Adds collation order, table constraints, many edits.

* reindexdb - add equal signs =

* pg_restore - misc edits, reorg and add some new options

* pg_dumpall - misc edits, new/rearrange options

* pg_dump - misc edits, new/rearrange options, new example

* dropuser - add equal signs =

* droplang - misc edits, add equal signs

* dropdb - add equal signs

* createuser - misc edits, add equal signs

* ALTER TYPE. reorganize synopsis. edits.

* ALTER USER. Add replication/noreplication

* BEGIN. Add DEFERRABLE options

* CLUSTER. Edits to usage notes

* COMMENT. Add new object types, examples

* createlang - misc edits, note, add equal signs

* createdb - misc edits, rearrange options, add equal signs

* clusterdb - add equal signs

* remove space

* note should be info, not warning

* COPY. Add encoding option.

* CREATE DOMAIN. Add collation option.

* CREATE INDEX. A collation.

* CREATE ROLE. Add replication/no replication

* CREATE OPERATOR CLASS. Add FOR SEARCH/ORDER BY.

* CREATE TYPE. Add collatable. Update Compatibility.

* CREATE USER. Add REPLICATION/NOREPLICATION. Make consistent with create role.

* CREATE VIEW. Edits to usage

* CREATE TABLE. Add UNLOGGED table type and COLLATE for table column.
updated - UNLOGGED warnings are per segment.

* CREATE TABLE. Add UNLOGGED table type. UNLOGGED information/warnings are per segment.

* GRANT. minor edits.

* psql - add equal signs, many additions, regorgs, and edits

* COPY - change literal to codeph

* DELETE. Add WITH query clause

* LOCK. SERIALIZABLE xact locking clarifications.

* SET TRANSACTION. Adds deferrable mode.

* DROP COLLATION. Remove redundant privileges statement.

* DROP TYPE. Qualify/hedge type extension compatibility

* EXPLAIN. Add JSON and YAML format examples.

* SET TRANSACTION. Add DEFERRED syntax and note that it is inoperative in gpdb

* REVOKE. add missing spaces in syntax.

* SELECT. Add DISTINCT to several clauses, and may edits.

* TRUNCATE. edits about RESTART and triggers

* VACUUM. deprecate notice for unparenthized syntax.

* SELECT INTO. Add DISTINCT keyword (syntax only), UNLOGGED table keyword and definition.

* INSERT. add WITH [RECURSIVE] clause

* UPDATE. Add WITH [RECURSIVE] clause

* createlang/droplang - remove deprecation note

* CREATE LANGUAGE. Use create extension instead for languages repackaged as extensions.

* pg_dump - --serializable-deferrable is a no-op

* SET TRANSACTION. Add transaction_default_deferred GUC and another unsupported notice

* BEGIN. Note DEFERRABLE has no effect in Greenplum Database

* EXPLAIN. Add missing query from example

* Address review comments.

* edits from Chuck

* docs - support for special characters in schema/table names for --include-table option.

* docs - support for special characters in schema/table names for --include-table-file option.

* docs - remove misplaced word "support"

docs - exchange partition, add limitation exchanging with a partitioned table is not supported. (#6961)

* docs - appendoptimized is an alias for appendonly storage option

* option names lower case, add legacy, misc edits

* docs - update compression information
-Add a topic that lists features that support configuring compression.
-update gp_workfile_compression GUC. Remove requirement to install zstd.

* docs - update compression information based on review comments.
-changed description to point to feature/utility for specific compression information.
-added general, pivotal, and OSS specific information about compression requirements.

* docs - compression information - reivew edit.

Unique indexes are now enforced throughout gpexpand, so there's no need to
warn or treat them specially. (gpexpand used to first make all tables
randomly distributed, and while in that state, unique indexes could not be
enforced. But it doesn't do that anymore.)

This also removes the gp_status_detail.rank column. There's no particular
reason to process tables with indexes before others anymore, and that was
the only criteria used for ranking.
Reviewed-by: NMelanie Plageman <mplageman@pivotal.io>

* docs - update template database information.

Updated ddl-database.xml topic - section About Template Databases
Also removed draft text from expand topics, gpexpand and pg_tablespace

* docs - update template database information based on review comments.
-Changed text - postgres is not a template database
-changed title to About Template and Default Databases

* Small line edit

* docs - update util ref pages for 9.0beta3

* edits requested by mel

* address comments from chuck

* ALTER DEFAULT PRIVILEGES. Add new command to sql ref

* ALTER ROLE. Substantive changes are already in the dita.

* ALTER TABLE. add [IF EXISTS], STATISTICS DISTINCT

* CREATE CAST. Add elipses to argtype

* CREATE DATABASE. Change dbowner to user_name

* CREATE FUNCTION. Remove SETOF and LIKE from RETURNS clause, notes on argument names

* CREATE ROLE. add USER, SYSID options

* CREATE TABLE.  LIKE clause syntax reorg and behavior. CHECK clause.

* psql. add \ddp and \drds meta commands

* REVOKE. add optional GROUP keyword

* SELECT. Many changes. Lock behavior changes. Compatibility notes.

* VACUUM. new parenthesized form of command

* psql. Add missing square bracket

* ALTER ROLE. Edits only.

* ALTER TABLE. Change STATISTICS DISTINCT to (attribute_option = n)

* ALTER TABLESPACE. Add tablespace_parameter

* ANALYZE. Change STATISTICS DISTNCT to SET (n_distinct = ...)

* CHECKPOINT. forces a restartpoint if run during recovery.

* CLUSTER. Mention FILLFACTOR and ALTER TABLE cluster options

* CREATE DOMAIN. domain can become null in spite of NOT NULL

* CREATE CONVERSION. Note command is similar to standard create translation.

* CREATE INDEX. Index name is optional, GiST example, edits.

* CREATE LANGUAGE. Add OR REPLACE clause, edits.

* CREATE OPERATOR. Deprecate => operator, note about lexical precedence

* REASSIGN OWNED. Add required privileges and other notes

* REINDEX. Clarify which indexes are reindexed for each option

* DROP CONVERSION. Note command is similar to standard DROP TRANSLATION

* SELECT. RANGE/ROWS edits, mostly made in earlier commits.

* SHOW. Add examples with output

* VACUUM. VACUUM FULL caveats.

* Formatting fixes

* Updates from reviews

* analyze - add paragraph about distinct values per column

* copy - new options and syntax

* COPY - add legacy syntax

* COPY - changes from 9.0Beta3 merge

* CREATE VIEW - add note about creating existing view

* EXPLAIN - new options; examples not updated.

* FETCH - small edits

* GRANT - small consistency edits; syntax consistency for optional GROUP keyword

* MOVE - optional FROM, IN keywords

* MOVE - optional GROUP keyword, consistency edits

* ROLLBACK TO SAVEPOINT - clarification/edits around cursor rollback

* bump postgresql url reference to 9.4

* bump postgresql url reference to 9.4

* bump postgresql url reference to 9.4

* delimiter -> delimiter_character

* Add a few more words around defaults for omitted options

* bump postgresql url reference to 9.4

* bump postgresql url reference to 9.4

* analyze - add paragraph about distinct values per column

* copy - new options and syntax

* COPY - add legacy syntax

* COPY - changes from 9.0Beta3 merge

* CREATE VIEW - add note about creating existing view

* EXPLAIN - new options; examples not updated.

* FETCH - small edits

* GRANT - small consistency edits; syntax consistency for optional GROUP keyword

* MOVE - optional FROM, IN keywords

* MOVE - optional GROUP keyword, consistency edits

* ROLLBACK TO SAVEPOINT - clarification/edits around cursor rollback

* delimiter -> delimiter_character

* Add a few more words around defaults for omitted options

* ALTER DEFAULT PRIVILEGES. Add new command to sql ref

* ALTER ROLE. Substantive changes are already in the dita.

* ALTER TABLE. add [IF EXISTS], STATISTICS DISTINCT

* CREATE CAST. Add elipses to argtype

* CREATE DATABASE. Change dbowner to user_name

* CREATE FUNCTION. Remove SETOF and LIKE from RETURNS clause, notes on argument names

* CREATE ROLE. add USER, SYSID options

* CREATE TABLE.  LIKE clause syntax reorg and behavior. CHECK clause.

* psql. add \ddp and \drds meta commands

* REVOKE. add optional GROUP keyword

* SELECT. Many changes. Lock behavior changes. Compatibility notes.

* VACUUM. new parenthesized form of command

* psql. Add missing square bracket

* Review comments

This re-implements compression that was lost in the WorkFile/BufFile
refactoring. This new implementation uses Zstandard rather than zlib.

Compression can be enabled with "gp_workfile_compression=on". It is
disabled by default.
Reviewed-by: NMel Kiyama <mkiyama@pivotal.io>
Reviewed-by: NYandong Yao <yyao@pivotal.io>

Temporary files have been somewhat inconsistent across different
operations. Some operations used the Greenplum-specific "workfile" API,
while others used the upstream BufFile API directly.

The workfile API provides some extra features: workfiles are visible in
the gp_toolkit views, and you can limit their size with the
gp_workfile_limit_* GUCs. The temporary files that didn't go through the
workfile API were exempt, which is not cool.

To make things consistent, remove the workfile APIs. Use BufFiles
directly everywhere. Re-implement the user-facing view and tracking the
limits, on top of the BufFile API, so that those features are not lost.

The workfile API also supported compressing the temporary files using
zlib. That feature is lost with this commit, but will be re-introduced by
the next commit.

Another feature that this removes, is checksumming temporary files.
That doesn't seem very useful, so we can probably live without it. But
if it's still needed, then that should also be re-implemented on top
of the BufFile API later.

Discussion: https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/8Xe9MGor0pM/SjqiOo83BAAJReviewed-by: NMel Kiyama <mkiyama@pivotal.io>
Reviewed-by: NYandong Yao <yyao@pivotal.io>

The way dropped columns are handled in ALTER TABLE was changed back in
2017, commit 62d66c06, but gpexpand didn't get the memo. Dropped columns
of custom datatypes no longer pose any problems.
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

* docs - add GUC gp_enable_global_deadlock_detector

Add Note text to DELETE, LOCK, UPDATE about GDD.
Also, update lock descriptions in dml.xml to point to GDD section.

* docs - review updates for gp_enable_global_deadlock_detector GUC