When constructing the connectionstring for connecting to the QEs,
ensure we don't overflow the assigned buffer.
Reviewed-by: NGang Xiong <gxiong@pivotal.io>

We are well past the 8.5devel version, so go back to using the
actual version from PG_VERSION rather than hardcoding. This
removes a FIXME that was added during the 9.0 merge cycle due
to 9.0 being labelled 8.5 in upstream for some time before it
was renamed.
Reviewed-by: NVenkatesh Raghavan <vraghavan@pivotal.io>

When build depends by run "make", meet an error "pkg_resources.DistributionNotFound:
The 'pylint<1.9.0,>=1.8.1' distribution was not found and is required by conan".
Since we don't run pylint anywhere，it can be removed from dependencies file safely.
Reviewed-by: NChris Hajas <chajas@pivotal.io>
Reviewed-by: NDaniel Gustafsson <dgustafsson@pivotal.io>

Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

The commit 8ab6a6b4 from upstream removed the function
check_pg_get_expr_args and all the calls to it.
Nevertheless the merge brought it back and with it the
issue corrected on commit f223bb7aSigned-off-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>

gpload test case will run gpload with subprocess, read stdout
and stderr from it and wait for exit. sys.exit in gpload does some
cleanup may cause deadlock between test and gpload. os._exit will
exit immediately, but we need to flush stdout and stderr before
that.

* Remove reference to ability to use port/username

Port and username options do not work in current gpssh release.

* Removed example

* Added missing space

This commit moves hyperloglog aggregates and functions to catalog as
well as migrates any hyperloglog related functions into
`src/backend/utils`.

Between GPDB 5.8.0 and 5.9.0, the recommended Linux System Settings
changed for kernel.sem.
https://gpdb.docs.pivotal.io/580/install_guide/prep_os_install_gpdb.html#topic3
https://gpdb.docs.pivotal.io/590/install_guide/prep_os_install_gpdb.html#topic3

However, the gpdb repository had references to the kernel.sem parameter,
as well as a couple other parameters, which seem to be inconsistent.
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NTrevor Yacovone <tyacovone@pivotal.io>

The regression database has shown to be full of questionable
objects which we probably won't support for upgrades without
manual intervention before the upgrade. With all the changes
going into pg_upgrade it's time to start on a known minimal
dataset and expand from there to the full ICW leftover schema.
As a stop-gap for now, drop the regression databases eand only
upgrade a small subset.

Also disallow upgrades of orhpaned toast tables, and clean
up the invocation of the test_gpdb_pre.sql script to be less
hacky.

This is extracted from a larger patch in 5e92c436adcb32d295e77b3f3.
Co-authored-by: NPaul Guo <paulguo@gmail.com>
Co-authored-by: NMax Yang <myang@pivotal.io>

Make sure we use the psql client from the new bindir (the on in PATH
might well be an upstream postgres psql etc), and use a port different
from the standard postgres port since it's likely to be in use.

Also fix integer value comparison which caused a warning in my Bash,
while it might work in other versions of Bash.

binary-upgrade mode;  instead only skip dumping the current user.

This bug was introduced in during the removal of split_old_dump().  Bug
discovered during local testing.

Previously each QE would dump/restore the schema before upgrading
the datafiles, which consumes a lot of time on large databases.
Instead, use the datadir which was created from the dump/restore
on the QD for the QEs as well and bootstrap the segment upgrade
to jump straight to copying files instead.

This is a first stab at implementing this model of distribution,
follow-up commits will be required to finalize the patch.

This doesn't appear to accompish anything useful, and does make the
restore fail if the postgres database happens to have been dropped.

Previously, the Greenplum pg_upgrade was syncronizing the Oids across
upgrades using the backend Oid pre-allocation. All pre-allocations
were generated in an Oid file, which was executed before the schema
was restored. When doing parallel restores, this strategy becomes
problematic as the Oid preallocation may be done in a backend other
than the one where the Oid is required. This commit rips out the use
of Oid preallocation in favor of using the upstream Oid handling in
pg_upgrade where the Oids are set just before being required. A major
difference is that we use the Oid preassignment scaffolding under the
covers, but in a more upstream merge friendly way.

For now, a known regression is that toast tables which in the old
cluster doesn't match the naming convention in the new cluster fails
to restore. Drop before running test until we know how to handle these.

The previous commit backported the PostgreSQL 9.3 pg_upgrade code
base and overwrote the hacks to allow for upgrading a Greenplum
cluster. This commit brings back the patchsets that were lost, and
adapts them to the new pg_upgrade code as well as mildly refactors
them to take advantage of new opportunities.

This overwrites the Greenplum pg_upgrade version with the upstream
9.3.22 sources. No attempt is made to make this compile or at all
be even remotely useable. Follow-up commits will re-introduce all
functionality which was lost.

This attempts to minimize the diff to upstream in preparation of
merging with a later version of pg_upgrade. In some cases this
re-introduces quirks from small style cleanups we've performed,
but it will only be for a short period until the merge happens.

PostgreSQL have in recent versions moved to just having a version.c
file for version specific function. In this commit we move Greenplum
specific code to version_gp.c, since we would do that sooner or later
anyways.

No functionality is altered with this, this is limited to refactoring.

When there is no stats available for any table, ORCA was treating it as an
empty table while planning. On the other hand planner is utilizing a guc
`gp_enable_relsize_collection` to obtain the estimated size of the table, but
no other statistics. This commit enables ORCA to have the same behavior as
planner when the guc is set.
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

* Set fragment index for PXF fragments per segment

* Added X-GP-LAST-FRAGMENT header to the request

to indicate that this is the last fragment for the segment.
The header is only sent for the last fragment, with the value "true".
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NFrancisco Guerrero <aguerrero@pivotal.io>

This is not used by any other utilities and isn't documented anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>

This isn't used by anything and isn't referenced anywhere in
documentation.
Authored-by: NChris Hajas <chajas@pivotal.io>

This is not used or documented anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>

This is a OS agnostic df utility that worked on both Solaris and Linux.
However, we no longer support solaris and this isn't referenced or
mentioned in the docs anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>

This utility was designed for DCAs with version prior to 1.0.2 and isn't
referenced or mentioned in the docs anywhere.
Authored-by: NChris Hajas <chajas@pivotal.io>

This isn't used by anything and isn't in the documentation.
Authored-by: NChris Hajas <chajas@pivotal.io>

This was added when we added the openbar gpcrondump tests in
c9770456. Since these tests no longer exist, we can remove this too.
Authored-by: NChris Hajas <chajas@pivotal.io>

This isn't used by any utilities.
Authored-by: NChris Hajas <chajas@pivotal.io>

This is not used by anything.
Authored-by: NChris Hajas <chajas@pivotal.io>

This was only used by gprepairmirrorseg, which was removed in a previous
commit.
Authored-by: NChris Hajas <chajas@pivotal.io>

Authored-by: NChris Hajas <chajas@pivotal.io>

The new optimized query introduced in commit 2ea6aec3 isn't as optimized
as we'd like if enable_nestloop is on -- released versions of GPDB craft
a plan that runs in polynomial time, and the dump grinds to a halt.

Disable enable_nestloop temporarily as a workaround. This code may need
to be somewhat rewritten to deal with external subpartition leaves
anyway, in which case we may be able to get rid of some of the
problematic JOINs.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>

* docs - fix some pivnet links

* add missing space

The AO VACUUM on crash_before_cleanup_phase table was supposed to skip
AO VACUUM drop phase because session 3 above it had acquired an
AccessShareLock. However, the VACUUM is executed in background and the
next command ends session 3's transaction and releases the
AccessShareLock. This creates a race condition where AO VACUUM drop
phase is not skipped when session 3's transaction ends too quickly. We
fix the issue by ending session 3's transaction only after the VACUUM
has hit the fault injector that tells us it is about to do its cleanup
phase.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>