The previous commit backported the PostgreSQL 9.3 pg_upgrade code
base and overwrote the hacks to allow for upgrading a Greenplum
cluster. This commit brings back the patchsets that were lost, and
adapts them to the new pg_upgrade code as well as mildly refactors
them to take advantage of new opportunities.

This overwrites the Greenplum pg_upgrade version with the upstream
9.3.22 sources. No attempt is made to make this compile or at all
be even remotely useable. Follow-up commits will re-introduce all
functionality which was lost.

When upgrading the cluster, allow the addition or removal of data
checksums on the data pages as they are copied across from the
old cluster. This allows for adding checksums without requiring a
full dump/restore cycle.

The test_gpdb.sh script is extended to support adding/removal of
checksums. In the process, also extend to remove gpdemo config
files generated when setting up the new cluster, and put them in
gitignore as well to keep git status happy during testing.

This also fixes a previously incorrect check for checksum version
and aligns variable names better with upstream.

Instead of meticulously recording the OIDs of each object in the pg_dump
output, dump and load all OIDs as a separate steps in pg_upgrade.

We now only preserve OIDs of types, relations and schemas from the old
cluster. Other objects are assigned new OIDs as part of the restore.
To ensure the OIDs are consistent between the QD and QEs, we dump the
(new) OIDs of all objects to a file, after upgrading the QD node, and use
those OIDs when restoring the QE nodes. We were already using a similar
mechanism for new array types, but we now do that for all objects.

In order for higher-level wrappers around pg_upgrade to know what
pg_upgrade does, this adds a trivial file based queue which reports
information on the operations performed. When started with -X, or
--progress, pg_upgrade will write progress as a set of operations
to a logfile. At intervals of either X number of operations, or Y
number of seconds, the file will be released and a new created such
that the released file can be consumed by a wrapper program. The
files will be named <sequence>.inprogress until released when they
are renamed to <sequence>.done. The sequence is gapless. Once the
file is renamed to <sequence>.done, pg_upgrade will never touch it
again and it can at that point be removed.

The format of the progress file is a simple semicolon delimited
line-based protocol:

  <timestamp>;<cluster>;<operation>;<message>;

timestamp is the number of microseconds since epoch; cluster defines
either "new", "old" or "none"; operation defines what actually did
happen and message contains the user defined message per report. The
timestamp is monotonically increasing for each operation such that
reading files in order is less important, operations can still be
individually sorted.

This commit includes a small set of progress report operations, while
not likely to be the final set it's enough to show how more can be
added.

Discussion:
  https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/CVgK36oU5Oo

This commit just brings in the upstream code for pg_upgrade and
pg_upgrade_support as is, with no changes. Subsequent work will change
it to work for Greenplum. The goal is to make it usable for upgrading
from Greenplum 4.3 to 5.0.

pg_upgrade_support is a crucial part of pg_upgrade, but lives in a
separate directory to make the build easier.

checks for PGHOST and PGHOSTADDR.

Also adjust some error message capitalization for consistency.

Don't insist on pg_dumpall and psql being present in the old cluster,
since they are not needed.  Do insist on pg_resetxlog being present
(in both old and new), since we need it.  Also check for pg_config,
but only in the new cluster.  Remove the useless attempt to call
pg_config in the old cluster; we don't need to know the old value of
--pkglibdir.  (In the case of a stripped-down migration installation
there might be nothing there to look at anyway, so any future change
that might reintroduce that need would have to be considered carefully.)

Per my attempts to build a minimal previous-version installation to support
pg_upgrade.

documentation.  (Many were left over from the old pg_migrator naming.)

than packing everything into 'ctx' and passing that to every function.

from Tom.

output to the same file, because it is impossible.

Also set user name for pg_dumpall in pg_upgrade.

Steve Singer

than in a subdirectory of the $HOME directory, or $TMP in Windows.

in their display of command-line options with other client applications.

proper default username.

Modify path separators for Win32.

Per ideas from Takahiro Itagaki

Add documentation.

Supports migration from PG 8.3 and 8.4.