As we merge with upstream and by that keep refining the Postgres
planner, legacy planner is no longer a suitable name. This changes
all variations of the spelling (legacy planner, legacy optimizer,
legacy query optimizer etc) to say "Postgres" rather than "legacy".
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>
Reviewed-by: NDavid Yozie <dyozie@pivotal.io>
Reviewed-by: NGeorgios Kokolatos <gkokolatos@pivotal.io>

* vacuumdb - add -t option support for multiple tables

* reindexdb - add -t option support for multiple tables; misc edits

* psql - misc additions, edits, and reformats

* pg_dump - misc edits, additions, formatting

* add * syntax for descendant tables

* pg_dumpall - add -d, --dbname option

* PREPARE - add note about search_path

* TRUNCATE - add * option

* SELECT - add LATERAL, NO KEY UPDATE, KEY SHARE, and related edits to locking clause

* ALTER ROLE. Re-order syntax and descriptions.

* ALTER RULE. Add new command and fix some conref issues

* COPY. Add FREEZE option and edits.

* CREATE INDEX. Minor synopsis update.

* CREATE SCHEMA. Add IF NOT EXISTS forms of command

* CREATE TABLE. Syntax changes. Add SET CONSTRAINTS command.

* bgworker. Add bgworker topic to reference guide

* CREATE TABLE AS. Add WITH/WITHOUT OIDS

* Changes from review

* remove some parens, add trigger note

* remove extra whitespace in CREATE TABLE

* CREATE TABLE AS - correct DISTRIBUTED BY syntax

* Review comments on CREATE TABLE.

* pg_dump - add parallelization note

* CREATE TABLE - fix whitespace again?

* Remove docs for LATERAL

* Add LATERAL to unsupported feature list

* vacuumdb - add missing equal signs =

* CREATE/ALTER/DROP COLLATION. Adds new references for these commands.

* remove space

* DROP COLLATION. Remove redundant privileges statement.

* SELECT. Add DISTINCT to several clauses, and may edits.

* CREATE TABLE. Adds NO INHERIT, IF NOT EXISTS, column_reference_storage_directive syntax, edits.

* ALTER DOMAIN. Add new forms of command.

* ALTER EXTENSION. Small edit only.

* ALTER FUNCTION. Add LEAKPROOF.

* ALTER_INDEX. Change SET/RESET FILLFACTOR to SET (fillfactor=)

* ALTER OPERATOR FAMILY. Add SP_GiST to descriptions.

* ALTER SEQUENCE. Add IF EXISTS.

* ALTER SERVER. Small edits.

* Add Range Type section & related changes

* ALTER TYPE - add UAGE privilege requirement

* ALTER VIEW - add IF EXISTS keyword; add syntax for view settings

* ANALYZE - add info about foreign tables

* CHECKPOINT - remove WAL paragraph; other edits

* ALTER TABLE. Add IF EXISTS, constraints, edits

* CREATE VIEW - add view option syntax

* CREATE OPERATOR TYPE - minor edits

* CREATE OPERATOR - add USAGE requirement

* createdb - new maintenance-db option; minor edits; simplify synopsis to be consistent with util output & postgresql docs

* createlang - add note about lower-casing the name

* createuser - add default notices, --interactive option, update examples

* DELETE - fix codeph style

* DROP INDEX - add CONCURRENTLY option

* DROP TABLE - small edit to permissions required

* dropdb - add --maintenance-db option

* droplang - add lowercase notice

* dropuser - add --if-exists; edits around prompting.

* clusterdb. Add --maintenance-db connection option.

* COMMENT. Replace table_name argument with relation_name

* CREATE AGGREGATE. Add privileges paragraph.

* CREATE CAST. Add privileges required.

* CREATE DOMAIN. Add required privileges, edit example.

* CREATE FUNCTION. Add LEAKPROOF function attribute.

* CREATE INDEX. add BUFFERING storage parameter.

* CREATE LANGUAGE. minor edit.

* CREATE TABLE AS. Edits. Deprecate GLOBAL/LOCAL. Also update SELECT and CREATE TABLE to enable links.

CREATE TABLE AS. Edits. Deprecate GLOBAL/LOCAL. Also update CREATE TABLE to enable links.

* CREATE ROLE. minor edits.

* GRANT - add USAGE ON DOMAIN, ON TYPE, with related notes

* EXPLAIN. Add BUFFERS option. Fix missing query in example.

* PREPARE. Edits.

* SELECT. add values clause  to with_query, updates select list, ORDER BY, LIMIT, FOR UPDATE/FOR SHARE clauses

* DELETE. updates to with_query for data-modifying commands.

* INSERT - updates to with_query for data-modifying commands.

* UPDATE - updates to with_query for data-modifying commands.

* REVOKE. add DOMAIN and TYPE variants

* SET. minor edit.

* pg_dump. Add --section and related parameters.

* pg_restore - add --section option and related text/edits

* reindexdb - add --maintenancedb connection option

* vacuumdb - add --maintenancedb connection option

* SET TRANSACTION. - snapshot not supported. minor edit.

* postgresql - edits, additions introduced in 9.2; major omissions and re-writes were sourced from the 9.4 docs

* Changes from review

* misc edits

* misc edit

* Remove outputclass tags from sgml conversion

* reformat converted files

* vacuumdb - add missing equal signs =

* CREATE/ALTER/DROP COLLATION. Adds new references for these commands.

* ALTER CONVERSION. Add SET SCHEMA variant.

* ALTER OPERATOR. add SET SCHEMA.

* ALTER OPERATOR CLASS. add SET SCHEMA

* ALTER OPERATOR FAMILY. add SET SCHEMA, FOR SEARCH/FOR ORDER BY

* ALTER ROLE. add REPLICATION/NOREPLICATION options

* ALTER TABLE. Adds collation order, table constraints, many edits.

* reindexdb - add equal signs =

* pg_restore - misc edits, reorg and add some new options

* pg_dumpall - misc edits, new/rearrange options

* pg_dump - misc edits, new/rearrange options, new example

* dropuser - add equal signs =

* droplang - misc edits, add equal signs

* dropdb - add equal signs

* createuser - misc edits, add equal signs

* ALTER TYPE. reorganize synopsis. edits.

* ALTER USER. Add replication/noreplication

* BEGIN. Add DEFERRABLE options

* CLUSTER. Edits to usage notes

* COMMENT. Add new object types, examples

* createlang - misc edits, note, add equal signs

* createdb - misc edits, rearrange options, add equal signs

* clusterdb - add equal signs

* remove space

* note should be info, not warning

* COPY. Add encoding option.

* CREATE DOMAIN. Add collation option.

* CREATE INDEX. A collation.

* CREATE ROLE. Add replication/no replication

* CREATE OPERATOR CLASS. Add FOR SEARCH/ORDER BY.

* CREATE TYPE. Add collatable. Update Compatibility.

* CREATE USER. Add REPLICATION/NOREPLICATION. Make consistent with create role.

* CREATE VIEW. Edits to usage

* CREATE TABLE. Add UNLOGGED table type and COLLATE for table column.
updated - UNLOGGED warnings are per segment.

* CREATE TABLE. Add UNLOGGED table type. UNLOGGED information/warnings are per segment.

* GRANT. minor edits.

* psql - add equal signs, many additions, regorgs, and edits

* COPY - change literal to codeph

* DELETE. Add WITH query clause

* LOCK. SERIALIZABLE xact locking clarifications.

* SET TRANSACTION. Adds deferrable mode.

* DROP COLLATION. Remove redundant privileges statement.

* DROP TYPE. Qualify/hedge type extension compatibility

* EXPLAIN. Add JSON and YAML format examples.

* SET TRANSACTION. Add DEFERRED syntax and note that it is inoperative in gpdb

* REVOKE. add missing spaces in syntax.

* SELECT. Add DISTINCT to several clauses, and may edits.

* TRUNCATE. edits about RESTART and triggers

* VACUUM. deprecate notice for unparenthized syntax.

* SELECT INTO. Add DISTINCT keyword (syntax only), UNLOGGED table keyword and definition.

* INSERT. add WITH [RECURSIVE] clause

* UPDATE. Add WITH [RECURSIVE] clause

* createlang/droplang - remove deprecation note

* CREATE LANGUAGE. Use create extension instead for languages repackaged as extensions.

* pg_dump - --serializable-deferrable is a no-op

* SET TRANSACTION. Add transaction_default_deferred GUC and another unsupported notice

* BEGIN. Note DEFERRABLE has no effect in Greenplum Database

* EXPLAIN. Add missing query from example

* Address review comments.

* edits from Chuck

* docs - appendoptimized is an alias for appendonly storage option

* option names lower case, add legacy, misc edits

* ALTER DEFAULT PRIVILEGES. Add new command to sql ref

* ALTER ROLE. Substantive changes are already in the dita.

* ALTER TABLE. add [IF EXISTS], STATISTICS DISTINCT

* CREATE CAST. Add elipses to argtype

* CREATE DATABASE. Change dbowner to user_name

* CREATE FUNCTION. Remove SETOF and LIKE from RETURNS clause, notes on argument names

* CREATE ROLE. add USER, SYSID options

* CREATE TABLE.  LIKE clause syntax reorg and behavior. CHECK clause.

* psql. add \ddp and \drds meta commands

* REVOKE. add optional GROUP keyword

* SELECT. Many changes. Lock behavior changes. Compatibility notes.

* VACUUM. new parenthesized form of command

* psql. Add missing square bracket

* ALTER ROLE. Edits only.

* ALTER TABLE. Change STATISTICS DISTINCT to (attribute_option = n)

* ALTER TABLESPACE. Add tablespace_parameter

* ANALYZE. Change STATISTICS DISTNCT to SET (n_distinct = ...)

* CHECKPOINT. forces a restartpoint if run during recovery.

* CLUSTER. Mention FILLFACTOR and ALTER TABLE cluster options

* CREATE DOMAIN. domain can become null in spite of NOT NULL

* CREATE CONVERSION. Note command is similar to standard create translation.

* CREATE INDEX. Index name is optional, GiST example, edits.

* CREATE LANGUAGE. Add OR REPLACE clause, edits.

* CREATE OPERATOR. Deprecate => operator, note about lexical precedence

* REASSIGN OWNED. Add required privileges and other notes

* REINDEX. Clarify which indexes are reindexed for each option

* DROP CONVERSION. Note command is similar to standard DROP TRANSLATION

* SELECT. RANGE/ROWS edits, mostly made in earlier commits.

* SHOW. Add examples with output

* VACUUM. VACUUM FULL caveats.

* Formatting fixes

* Updates from reviews

* ALTER DEFAULT PRIVILEGES. Add new command to sql ref

* ALTER ROLE. Substantive changes are already in the dita.

* ALTER TABLE. add [IF EXISTS], STATISTICS DISTINCT

* CREATE CAST. Add elipses to argtype

* CREATE DATABASE. Change dbowner to user_name

* CREATE FUNCTION. Remove SETOF and LIKE from RETURNS clause, notes on argument names

* CREATE ROLE. add USER, SYSID options

* CREATE TABLE.  LIKE clause syntax reorg and behavior. CHECK clause.

* psql. add \ddp and \drds meta commands

* REVOKE. add optional GROUP keyword

* SELECT. Many changes. Lock behavior changes. Compatibility notes.

* VACUUM. new parenthesized form of command

* psql. Add missing square bracket

* Review comments

Replace the use of the built-in hashing support for built-in datatypes, in
cdbhash.c, with the normal PostgreSQL hash functions. Now is a good time
to do this, since we've already made the change to use jump consistent
hashing in GPDB 6, so we'll need to deal with the upgrade problems
associated with changing the hash functions, anyway.

It is no longer enough to track which columns/expressions are used to
distribute data. You also need to know the hash function used. For that,
a new field is added to gp_distribution_policy, to record the hash
operator class used for each distribution key column. In the planner,
a new opfamily field is added to DistributionKey, to track that throughout
the planning.

Normally, if you do "CREATE TABLE ... DISTRIBUTED BY (column)", the
default hash operator class for the datatype is used. But this patch
extends the syntax so that you can specify the operator class explicitly,
like "... DISTRIBUTED BY (column opclass)". This is similar to how an
operator class can be specified for each column in CREATE INDEX.

To support upgrade, the old hash functions have been converted to special
(non-default) operator classes, named cdbhash_*_ops. For example, if you
want to use the old hash function for an integer column, you could do
"DISTRIBUTED BY (intcol cdbhash_int4_ops)". The old hard-coded whitelist
of operators that have "compatible" cdbhash functions has been replaced
by putting the compatible hash opclasses in the same operator family. For
example, all legacy integer operator classes, cdbhash_int2_ops,
cdbhash_int4_ops and cdbhash_int8_ops, are all part of the
cdbhash_integer_ops operator family).

This removes the pg_database.hashmethod field. The hash method is now
tracked on a per-table and per-column basis, using the opclasses, so it's
not needed anymore.

To help with upgrade from GPDB 5, this introduces a new GUC called
'gp_use_legacy_hashops'. If it's set, CREATE TABLE uses the legacy hash
opclasses, instead of the default hash opclasses, if the opclass is not
specified explicitly. pg_upgrade will set the new GUC, to force the use of
legacy hashops, when restoring the schema dump. It will also set the GUC
on all upgraded databases, as a per-database option, so any new tables
created after upgrade will also use the legacy opclasses. It seems better
to be consistent after upgrade, so that collocation between old and new
tables work for example. The idea is that some time after the upgrade, the
admin can reorganize all tables to use the default opclasses instead. At
that point, he should also clear the GUC on the converted databases. (Or
rather, the automated tool that hasn't been written yet, should do that.)

ORCA doesn't know about hash operator classes, or the possibility that we
might need to use a different hash function for two columns with the same
datatype. Therefore, it cannot produce correct plans for queries that mix
different distribution hash opclasses for the same datatype, in the same
query. There are checks in the Query->DXL translation, to detect that
case, and fall back to planner. As long as you stick to the default
opclasses in all tables, we let ORCA to create the plan without any regard
to them, and use the default opclasses when translating the DXL plan to a
Plan tree. We also allow the case that all tables in the query use the
"legacy" opclasses, so that ORCA works after pg_upgrade. But a mix of the
two, or using any non-default opclasses, forces ORCA to fall back.

One curiosity with this is the "int2vector" and "aclitem" datatypes. They
have a hash opclass, but no b-tree operators. GPDB 4 used to allow them
as DISTRIBUTED BY columns, but we forbid that in GPDB 5, in commit
56e7c16b. Now they are allowed again, so you can specify an int2vector
or aclitem column in DISTRIBUTED BY, but it's still pretty useless,
because the planner still can't form EquivalenceClasses on it, and will
treat it as "strewn" distribution, and won't co-locate joins.

Abstime, reltime, tinterval datatypes don't have default hash opclasses.
They are being removed completely on PostgreSQL v12, and users shouldn't
be using them in the first place, so instead of adding hash opclasses for
them now, we accept that they can't be used as distribution key columns
anymore. Add a check to pg_upgrade, to refuse upgrade if they are used
as distribution keys in the old cluster. Do the same for 'money' datatype
as well, although that's not being removed in upstream.

The legacy hashing code for anyarray in GPDB 5 was actually broken. It
could produce a different hash value for two arrays that are considered
equal, according to the = operator, if there were differences in e.g.
whether the null bitmap was stored or not. Add a check to pg_upgrade, to
reject the upgrade if array types were used as distribution keys. The
upstream hash opclass for anyarray works, though, so it is OK to use
arrays as distribution keys in new tables. We just don't support binary
upgrading them from GPDB 5. (See github issue
https://github.com/greenplum-db/gpdb/issues/5467). The legacy hashing of
'anyrange' had the same problem, but that was new in GPDB 6, so we don't
need a pg_upgrade check for that.

This also tightens the checks ALTER TABLE ALTER COLUMN and CREATE UNIQUE
INDEX, so that you can no longer create a situation where a non-hashable
column becomes the distribution key. (Fixes github issue
https://github.com/greenplum-db/gpdb/issues/6317)

Discussion: https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/4fZVeOpXllQCo-authored-by: NMel Kiyama <mkiyama@pivotal.io>
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NPengzhou Tang <ptang@pivotal.io>
Co-authored-by: NChris Hajas <chajas@pivotal.io>
Reviewed-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
Reviewed-by: NNing Yu <nyu@pivotal.io>
Reviewed-by: NSimon Gao <sgao@pivotal.io>
Reviewed-by: NJesse Zhang <jzhang@pivotal.io>
Reviewed-by: NZhenghua Lyu <zlv@pivotal.io>
Reviewed-by: NMelanie Plageman <mplageman@pivotal.io>
Reviewed-by: NYandong Yao <yyao@pivotal.io>

* docs - REPEATABLE READ xact mode is supported. SERIALIZABLE falls back to REPEATABLE READ.

* Note that GPDB doesn't implement PGSQL SSI transactions

* Review comments

* Docs - update docs to note that system columns are unavailable in
queries on replicated tables.

* Edits from reviewers

Discussion: https://github.com/greenplum-db/gpdb/pull/6410
Reviewed-by: Heikki Linnakangas

* Update DITA docs with Postgres 8.4 merge, interation 3 changes to SGML files.
(PR #3780)

* Update DITA docs with Postgres 8.4 merge, interation 3 changes to SGML files.
(PR #3780)

* Fix example syntax

* Add missing <p> codes and move into sectiondiv

* Update gp_create_table_random_default_distribution to describe new 6.x rules

* Update from Daniel

* Using same wording for behavior in CREATE TABLE

Allow specifying multiple UNIQUE constraints in the CREATE TABLE command,
or PRIMARY KEY and UNIQUE constraints, as long as they are all compatible
with the DISTRIBUTED BY clause.

Don't require the constraint's key columns to be a left-subset of the
DISTRIBUTED BY clause. It still needs to be a subset, but any subset will
do.

Likewise adjust the rules for deriving the distribution key from the
constraints, if no DISTRIBUTED BY was given, to choose the largest common
subset across all constraints.

Discussion: https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/OJzz8I6WuVI/tMB8GcXdAgAJ

* Modify SQL references for replicated tables

* Changes for edits and review comments

* docs - add foreign data wrapper-related ref pages

* remove CREATE SERVER example referencing default fdw

* edits from david, and his -> their

* docs - add distribution and skew topic, address random dist skew

* clarify single tuple assigned to first segment

* docs - type SET DEFAULT ENCODING

* edits requested by david

* add gp-specific clauses to summary of greenplum features page

* docs - review zstd doc additions, misc related edits

* modify ddl storage example 1 to use zstd

Add a new compression option for append-optimized tables, "zstd". It is
generally faster than zlib or quicklz, and compresses better. Or at least
it can be faster or compress better, if not both at the same time, by
adjusting the compression level. A major advantage of Zstandard is the wide
range of tuning, to choose the trade-off between compression speed and
ratio.

Update documentation to mention "zstd" alongside "zlib" and "quicklz". More
could be done; all the examples still use zlib or quicklz, for example, and
I think we want to emphasize Zstandard more in the docs, over those other
options. But this is the bare minimum to keep the docs factually correct.

Using the new option requires building the server with the libzstd library.
A new --with-zstd option is added for that. The default is to build without
libzstd, for now, but we should probably change the default to be on, after
we have had a chance to update all the buildfarm machines to have libzstd.

Patch by Ivan Leskin, Dmitriy Pavlov, Anton Chevychalov. Test case, docs
changes, and some minor editorialization by Heikki Linnakangas.

* docs: create table- fixed default storage options information
default storage options can be set for system, database, user.

* Small edit to keep it in line with the 4.3.x commit.

* rewording/conditionalizing more Pivotal-specific info

* conditionalizing/editing/or getting rid of more Pivotal-specific references and info

* GPDB DOCS - Fix issues found when validating files in gpdb-webhelp.ditamap.

* fix for GUC gp_connections_per_thread update to GPDB 5 description.

* add to release notes, update wording

* relnotes edits

* rebase and fix

* updating adminguide source with most recent 4.3.x work

* updating reference manual with most recent 4.3.x work

* updating utility guide with most recent 4.3.x changes

* updating client tools guide with most recent 4.3.x changes

* adding new file for client tools

* updating map files with most recent 4.3.x changes

* updating map files with most recent 4.3.x changes

* Revert "updating map files with most recent 4.3.x changes"

This reverts commit d7570343c17a126b4d11eaee3870ad6daa36966f.

* Revert "updating map files with most recent 4.3.x changes"

This reverts commit d7570343c17a126b4d11eaee3870ad6daa36966f.

* updating ditamaps with latest 4.3.x changes

* updating ditamaps with latest 4.3.x changes