To keep it consistent with the "Create table" syntax, CTAS should also
disallow duplicate distributed keys, otherwise backup and restore will
mess up.

In ALTER TABLE SET DISTRIBUTED BY, the "WITH(reorganize=...)" option
must be specified before the "DISTRIBUTED ..." clause.

Parse 1.8.2 is consistent with the other python libraries
defined in the python-dependencies.txt file
Co-authored-by: NTodd Sedano <tsedano@pivotal.io>
Co-authored-by: NChris Hajas <chajas@pivotal.io>

* docs - PL/Container - added note - domain object not supported.

* docs - PL/Container - updated note for non-support of domain object

Require the use of cc on OpenBSD as the gcc compiler isn't
supporting the compiler flags that setup.py invokes. Also fix
library paths.

OpenBSD require that linking with a library using sigwait(3) be
compiled with -pthread.  This adds a kludge in the relevant
makefiles which is less than elegant, but it seemed the least
intrusive change to make.

In order to use backtrace() in error reporting on OpenBSD we need
to link with libexecinfo from ports as backtrace() is a glibc only
addition.

There is no need to build gpmapreduce separately as it's automatically
built on "make install", and with recent changes to detect libyaml it's
not even supported. Fix by removing separate step and instead use the
--enable-mapreduce switch to autoconf.

Greenplum Mapreduce requires libyaml, but was lacking a specific
test for it in autoconf. This worked since gpfdist has the same
check but when building with --disable-gpfdist we need to ensure
we have libyaml to avoid late compilation failures.

Rather than hardcoding to require /bin/bash, move to using a lookup
via "/usr/bin/env bash" to allow for greater portability of the code.
This also changes the Bash test to checking if the current shell is
actually Bash, rather than looking if bash is available on the file
system (since we by the above mentioned changes no longer need that).

The NetCheckNIC() functionality was only used by FileRep, and is
not even compiling properly under all possible preprocessor flags
it once supported. Remove.

The -not syntax is not portable across all platforms, most notably
OpenBSD, so use the more portable ! operator instead.

Make sure to include all required header files to silence compilers
that are picky about that.

According to recent tests, this case now works fine, so there's no reason
to reject it anymore.  (Even if there are still some OpenBSD platforms
in the wild where it doesn't work, removing the check won't break any case
that worked before.)

We can actually remove the entire test that discovers whether libpython
is threaded, since without the OpenBSD case there's no need to know that
at all.

Per report from Davin Potts.  Back-patch to all active branches.
Co-authored-by: NDaniel Gustafsson <dgustafsson@pivotal.io>

Following the change in 8fcd3fdd
to cost-based enable GUCs, failing to find a way to construct an
N-way join should be an error rather than debug (as in upstream).
Reported-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>

Discussion: https://github.com/greenplum-db/gpdb/pull/5155Co-authored-by: NDaniel Gustafsson <dgustafsson@pivotal.io>

Instead of completely disabling the generation of Paths with disabled
plan types, add a high penalty to their cost estimates, like in the
upstream. This reduces our diff vs. upstream, making future merges more
straightforward.

Discussion: https://groups.google.com/a/greenplum.org/d/msg/gpdb-dev/Az2cDcqf73g/_tY6Yv1kBgAJCo-authored-by: NDaniel Gustafsson <dgustafsson@pivotal.io>
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>
Reviewed-by: NRichard Guo <riguo@pivotal.io>

As part of the Postgres 8.3 merge, all heap tables now automatically
create an array type. The array type will usually be created with
typname '_<heap_name>' since the automatically created composite type
already takes the typname '<heap_name>' first. If typname
'_<heap_name>' is taken, the logic will continue to prepend
underscores until no collision (truncating the end if typname gets
past NAMEDATALEN of 64). This might be an oversight in upstream
Postgres since certain scenarios involving creating a large number of
heap tables with similar names could result in a lot of typname
collisions until no heap tables with similar names can be
created. This is very noticable in Greenplum heap partition tables
because Greenplum has logic to automatically name child partitions
with similar names instead of having the user name each child
partition.

To prevent typname collision failures when creating a heap partition
table with a large number of child partitions, we will now stop
automatically creating the array type for child partitions.

References:
https://www.postgresql.org/message-id/flat/20070302234016.GF3665%40fetter.org
https://github.com/postgres/postgres/commit/bc8036fc666a8f846b1d4b2f935af7edd90eb5aa

The pg_get_partition_template_def and pg_get_partition_def functions
take accesss share locks, but do not release them until the end of the
transaction. If a transaction is long-running, this can conflict with
other user operations. Is it not necessary to hold the lock indefinitely
as the lock is only needed for the duration of the function call.
Co-authored-by: NChris Hajas <chajas@pivotal.io>
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>

If a segment exists in gp_segment_configuration but its ip address can
not be resolved we will run into a runtime error on gang creation:

    ERROR:  could not translate host name "segment-0a", port "40000" to
    address: Name or service not known (cdbutil.c:675)

This happens even if segment-0a is a mirror and is marked as down.  With
this error queries can not be executed, gpstart and gpstop will also
fail.

One way to trigger the issue:

- create a multiple segments cluster;
- remove sdw1's dns entry from /etc/hosts on mdw;
- kill postgres primary process on sdw1;

FTS can detect this error and automatically switch to mirror, but
queries can not be executed.

* docs - update system catalog maintenance information.

--Updated Admin. Guide and Best Practices for running REINDEX, VACUUM, and ANALYZE
--Added note to REINDEX reference about running ANALYZE after REINDEX.

* docs - edits for system catalog maintenance updates

* docs - update recommendation for running vacuum and analyze.

Update based on dev input.

* docs - add foreign data wrapper-related ref pages

* remove CREATE SERVER example referencing default fdw

* edits from david, and his -> their

We currently exit VACUUM early when there is a concurrent operation on
an AO relation. Instead of exiting early, go through the rest of the
AO segment files to see if they have crossed threshold for compaction.

TRUNCATE will rewrite the relation by creating a temporary table and
swapping it with the real relation. For AO, this includes the
auxiliary tables which is concerning for the AO relation's pg_aoseg
table which holds information that a AO segment file is available for
write or waiting to be compacted/dropped. Since we do not currently
invalidate the AppendOnlyHash cache entry, the entry could have
invisible leaks in its AOSegfileStatus array that will be stuck in
state AOSEG_STATE_AWAITING_DROP. These leaks will persist until the
user evicts the cache entry by not using the table to allow another AO
table to cache itself in that slot or by restarting the database. We
fix this issue by invalidating the cache entry at the end of TRUNCATE
on AO relations.

ALTER TABLE commands that are tagged as AT_SetDistributedBy require a
gather motion and does its own variation of creating a temporary table
for CTAS (basically bypassing the usual ATRewriteTable which actually
does do AppendOnlyHash cache entry invalidation). Without the
AppendOnlyHash cache entry invalidation, the entry could have
invisible leaks in its AOSegfileStatus array that will be stuck in
state AOSEG_STATE_AWAITING_DROP. These leaks will persist until the
user evicts the cache entry by not using the table to allow another AO
table to cache itself in that slot or by restarting the database. We
fix this issue by invalidating the cache entry at the end of
AT_SetDistributedBy ALTER TABLE cases.

The schema is named differently from the one being used in the
search_path so all the tables, views, functions, and etc. were
incorrectly being created in the public schema.

We had significant deduplication in hyperloglog extension and utility
library that we use in the analyze related code. This commit removes the
deduplication as well as significant amount dead code. It also fixes
some compiler warnings and some coverity issues.

This commit also puts the hyperloglog functions in a separate schema
which is non-modifiable by non superusers.
Signed-off-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>

Fixing some of the more obvious breaches of common style around
the code I just read for another patchset. There is no logical
changes introduced here, only rearrangement for clarity.

Discussion: https://github.com/greenplum-db/gpdb/pull/5216Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

The backup list is either a leftover debugging artefact or its
use was removed during the merge work and never made it into the
rewritten commit history. Either way, it serves no purpose so
remove from this hot code path.

Discussion: https://github.com/greenplum-db/gpdb/pull/5216Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Commit e0409357 moved to using default estimates rather
than interrogating the QEs for relations which lack statistics.
As an effect of this, the cdb_default_stats_used member was
hardcoded to false and the warnings for missing statistics did
never fire.

Rather than resurrecting the warnings, this removes the code
that attempts to figure out if the warnings at all apply since
it seems quite expensive to run that in the hot path of every
join query being planned.

Discussion: https://github.com/greenplum-db/gpdb/pull/5216Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

The map was missed by mistake, all AO loading actions need it.

Commit 4483b7d3 remove spclocation
from the tablespace catalog, but the \db command in psql wasn't
updated to match the corresponding Greenplum version as it was
backported prior to when it was introduced in upstream. This will
eventually go away as we merge with PostgreSQL, but that's not an
excuse for not fixing what is broken.

Discussion: https://github.com/greenplum-db/gpdb/pull/5238Reviewed-by: NAshwin Agrawal <aagrawal@pivotal.io>

Not sure why setting of guc gp_vmem_protect_limit has specific value for darwin.

Authored-by: NTodd Sedano <tsedano@pivotal.io>