There is no point in checking for the presence of a formatter, as
it's already tested for in the error path in transformFormatOpts()
when the external table is created.  Remove dead code and make it
fail on the error case should we ever manage to reach here.  This
resolves a FIXME added during the 9.1 merge which highlighted the
issue.
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Align a few error messages to match upstream coding convention for
error messages (which we are slowly adapting to), and fix a typo.
Spotted while fixing other things in the file.
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Commit 9b2d6688 removed check_pg_get_expr_args() as was done in
upstream, but Greenplum had an altered version with logic broken out
into the helper function check_pg_get_expr_arg().  This function is
now dead code so remove as well to get rid of compiler warning.

The is_split_part member is compared in _equalIndexStmt(), and is
copied in _copyIndexStmt(), so we should include it in the read
and write functions as well.
Reviewed-by: NJacob Champion <pchampion@pivotal.io>
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

The altconname member defines an alternative name for the index if
another name for a constraint was requested. Including it in the
comparison is mostly for correctness, as the index name is compared,
even if few (if any) situations would fail on this being omitted.
Reviewed-by: NJacob Champion <pchampion@pivotal.io>
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

IndexStmt is defined in src/include/nodes/parsenodes.h with indexOid
before the is_part_child member. Reorder the node functions to match
the definition.
Reviewed-by: NJacob Champion <pchampion@pivotal.io>
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

Translate nestparams passed from ORCA to create the nestparams node in Nested Loop joins.

This feature can be enabled by setting the trace flag EopttraceEnableNestLoopParams.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NEkta Khanna <ekhanna@pivotal.io>

Translate nestparams passed from ORCA to create the nestparams node in Nested Loop joins.

This feature can be enabled by setting the trace flag EopttraceEnableNestLoopParams.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NEkta Khanna <ekhanna@pivotal.io>

The segwalrep test uses a mock WAL receiver to validate what the WAL
sender is streaming out. The mock that we created did not handle AO
XLOG records that have been split across two pages with a continuation
record. We now properly construct the split AO record back to get the
AO information that is needed for test validation.

Also, do some trivial spelling fixes.
Co-authored-by: NAsim R Praveen <apraveen@pivotal.io>

* docs - update guc gp_enable_relsize_collection - ORCA support
Will be ported to 5X_STABLE.

Generated dev pipelines should be included in .gitignore. It's too
easy to accidentally add a generated dev pipeline in a commit.
Authored-by: NJason Vigil <jvigil@pivotal.io>

When constructing the connectionstring for connecting to the QEs,
ensure we don't overflow the assigned buffer.
Reviewed-by: NGang Xiong <gxiong@pivotal.io>

We are well past the 8.5devel version, so go back to using the
actual version from PG_VERSION rather than hardcoding. This
removes a FIXME that was added during the 9.0 merge cycle due
to 9.0 being labelled 8.5 in upstream for some time before it
was renamed.
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

When build depends by run "make", meet an error "pkg_resources.DistributionNotFound:
The 'pylint<1.9.0,>=1.8.1' distribution was not found and is required by conan".
Since we don't run pylint anywhere，it can be removed from dependencies file safely.
Reviewed-by: NChris Hajas <chajas@pivotal.io>
Reviewed-by: NDaniel Gustafsson <dgustafsson@pivotal.io>

Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

The commit 8ab6a6b4 from upstream removed the function
check_pg_get_expr_args and all the calls to it.
Nevertheless the merge brought it back and with it the
issue corrected on commit f223bb7aSigned-off-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>

gpload test case will run gpload with subprocess, read stdout
and stderr from it and wait for exit. sys.exit in gpload does some
cleanup may cause deadlock between test and gpload. os._exit will
exit immediately, but we need to flush stdout and stderr before
that.

* Remove reference to ability to use port/username

Port and username options do not work in current gpssh release.

* Removed example

* Added missing space

This commit moves hyperloglog aggregates and functions to catalog as
well as migrates any hyperloglog related functions into
`src/backend/utils`.

Between GPDB 5.8.0 and 5.9.0, the recommended Linux System Settings
changed for kernel.sem.
https://gpdb.docs.pivotal.io/580/install_guide/prep_os_install_gpdb.html#topic3
https://gpdb.docs.pivotal.io/590/install_guide/prep_os_install_gpdb.html#topic3

However, the gpdb repository had references to the kernel.sem parameter,
as well as a couple other parameters, which seem to be inconsistent.
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NTrevor Yacovone <tyacovone@pivotal.io>

The regression database has shown to be full of questionable
objects which we probably won't support for upgrades without
manual intervention before the upgrade. With all the changes
going into pg_upgrade it's time to start on a known minimal
dataset and expand from there to the full ICW leftover schema.
As a stop-gap for now, drop the regression databases eand only
upgrade a small subset.

Also disallow upgrades of orhpaned toast tables, and clean
up the invocation of the test_gpdb_pre.sql script to be less
hacky.

This is extracted from a larger patch in 5e92c436adcb32d295e77b3f3.
Co-authored-by: NPaul Guo <paulguo@gmail.com>
Co-authored-by: NMax Yang <myang@pivotal.io>

Make sure we use the psql client from the new bindir (the on in PATH
might well be an upstream postgres psql etc), and use a port different
from the standard postgres port since it's likely to be in use.

Also fix integer value comparison which caused a warning in my Bash,
while it might work in other versions of Bash.

binary-upgrade mode;  instead only skip dumping the current user.

This bug was introduced in during the removal of split_old_dump().  Bug
discovered during local testing.

Previously each QE would dump/restore the schema before upgrading
the datafiles, which consumes a lot of time on large databases.
Instead, use the datadir which was created from the dump/restore
on the QD for the QEs as well and bootstrap the segment upgrade
to jump straight to copying files instead.

This is a first stab at implementing this model of distribution,
follow-up commits will be required to finalize the patch.

This doesn't appear to accompish anything useful, and does make the
restore fail if the postgres database happens to have been dropped.

Previously, the Greenplum pg_upgrade was syncronizing the Oids across
upgrades using the backend Oid pre-allocation. All pre-allocations
were generated in an Oid file, which was executed before the schema
was restored. When doing parallel restores, this strategy becomes
problematic as the Oid preallocation may be done in a backend other
than the one where the Oid is required. This commit rips out the use
of Oid preallocation in favor of using the upstream Oid handling in
pg_upgrade where the Oids are set just before being required. A major
difference is that we use the Oid preassignment scaffolding under the
covers, but in a more upstream merge friendly way.

For now, a known regression is that toast tables which in the old
cluster doesn't match the naming convention in the new cluster fails
to restore. Drop before running test until we know how to handle these.

The previous commit backported the PostgreSQL 9.3 pg_upgrade code
base and overwrote the hacks to allow for upgrading a Greenplum
cluster. This commit brings back the patchsets that were lost, and
adapts them to the new pg_upgrade code as well as mildly refactors
them to take advantage of new opportunities.

This overwrites the Greenplum pg_upgrade version with the upstream
9.3.22 sources. No attempt is made to make this compile or at all
be even remotely useable. Follow-up commits will re-introduce all
functionality which was lost.

This attempts to minimize the diff to upstream in preparation of
merging with a later version of pg_upgrade. In some cases this
re-introduces quirks from small style cleanups we've performed,
but it will only be for a short period until the merge happens.

PostgreSQL have in recent versions moved to just having a version.c
file for version specific function. In this commit we move Greenplum
specific code to version_gp.c, since we would do that sooner or later
anyways.

No functionality is altered with this, this is limited to refactoring.

When there is no stats available for any table, ORCA was treating it as an
empty table while planning. On the other hand planner is utilizing a guc
`gp_enable_relsize_collection` to obtain the estimated size of the table, but
no other statistics. This commit enables ORCA to have the same behavior as
planner when the guc is set.
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

* Set fragment index for PXF fragments per segment

* Added X-GP-LAST-FRAGMENT header to the request

to indicate that this is the last fragment for the segment.
The header is only sent for the last fragment, with the value "true".
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NFrancisco Guerrero <aguerrero@pivotal.io>

This is not used by any other utilities and isn't documented anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>

This isn't used by anything and isn't referenced anywhere in
documentation.
Authored-by: NChris Hajas <chajas@pivotal.io>

This is not used or documented anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>