The non-cosmetic changes involve teaching the "zic" tzdata compiler about
negative DST.  While I'm not currently intending that we start using
negative-DST data right away, it seems possible that somebody would try
to use our copy of zic with bleeding-edge IANA data.  So we'd better be
out in front of this change code-wise, even though it doesn't matter for
the data file we're shipping.

Discussion: https://postgr.es/m/30996.1525445902@sss.pgh.pa.us
(cherry picked from commit b45f6613)

DST law changes in Palestine and Antarctica (Casey Station).  Historical
corrections for Portugal and its colonies, as well as Enderbury, Jamaica,
Turks & Caicos Islands, and Uruguay.

(cherry picked from commit df629586)

This should have been part of commit
f590dc94 but we forgot. Now remove them
for good.
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>

`SHOW memory_spill_ratio` will always display 20 when it's the first
query in a connection (if you run this query in psql and pressed TAB
when entering the command then the implicit queries ran by the tab
completion function will be the first), the root cause is that SHOW
command will be bypassed in resgroup, so the bound resgroup will not be
assigned, and the resgroup's settings will not be loaded.

To display the proper value in this case we will also load the resgroup
settings even for bypassed queries.

Before:
     qp_functions             ... ok (76.24 sec)  (diff:0.06 sec)
     qp_gist_indexes4         ... ok (88.46 sec)  (diff:0.07 sec)
     qp_with_clause           ... ok (130.70 sec)  (diff:0.32 sec)

After:
     qp_functions             ... ok (4.49 sec)  (diff:0.06 sec)
     qp_gist_indexes4         ... ok (16.18 sec)  (diff:0.06 sec)
     qp_with_clause           ... ok (54.41 sec)  (diff:0.30 sec)

* docs - misc updates to gptransfer

- conref from best practices to admin guide
- qualify use for migration to diff number of segments
- misc edits

* conditionalize

* docs -draft update for migrating with gpcopy

* docs - migrating w/ gpcopy - updated ditamap

* add 'pivotal' condition

* some edits, additions, reorg

* removing ssh as a requirement (per comment on related PR)

* Clarifying that same-cluster copies aren't supported

* 4.3.16 -> 4.3.26

* replace --truncate with --truncate-source-after

* add note to start clusters in restricted mode

* add note about md5 validation when migrating between major version

* add pg_dump, pg_dumpall, psql dependencies in prerequisites section

* add info about actual free space needed for migration

* updating discussion of free space calculation

* updating to use new gpcopy option names

* more updates to clarify migration proc

* removing contentious statement about indexing (not present in main docs); adding resource groups to list of objects copied

* removing note about md5xor not working for version migration

* add note regarding validation with --append

* add note to install client package to get gpcopy dependencies

* removed --no-compression info as that's the default for same-host copies; relocated basic migration instructions

* replacing error section with reviewed info from ref page

* updating migration topics to use procedure format

* adding some post-migration steps for gpcopy

* address more feedback from brian; reorganize migration steps a bit; add general post-migration instructions from the relnotes

The return value of tzparse() has changed as of commit b749790a, but the
corresponding change to the tzparse() function in pg_load_tz() was never made.
As a result, under certain circumstances the server might pick a bogus
timezone.

For semi join queries if the constraints can eliminate the scanned relations,
the resulting relation should be marked as a dummy and the join using it should
be a dummy join.

The only change in the tarball is that we have removed the libraries and
header files for ddboost. These have already been removed for centos for
some time.
Co-authored-by: NChris Hajas <chajas@pivotal.io>
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>

We have added a unit test for gpaddmirrors when the mirror data directories are
provided interactively.
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>

Cpu usage of cpuset groups should also be displayed in
gp_toolkit.gp_resgroup_status.

Previously, for an interconnect connection, if no data are available at
sender peer, the sender sends a customized EOS packet to the receiver
and disables further send operations using shutdown(SHUT_WR), then
somehow, the sender closes the connection totally with close()
immediately and it counts on the kernel and TCP stack to guarantee the
data been transformed to the receiver. The problem is, on some platform,
if the connection is closed on one side, the TCP behave is undetermined,
the packets may be lost and receiver may report an unexpected error.

The correct way is sender blocks on the connection until receiver
getting the EOS packet and close its peer, then the sender can close
the connection safely.

For a result node with one-time filter, if it's outer plan is not
empty and contains a motion node, then it needs to squelch the outer
node explicitly if the one-time filter check is false. This is necessary
espically for motion node under it, ExecSquelchNode() force a stop
message so the interconnect sender don't stuck at recending or polling
ACKs.

This is a quick fix to make dispatch test pass, for a long term,
we need to redesign the dispatch test or make it a unit test.

* upgrade pgbouncer to 1.8.1

* support PAM/HBA auth type

* update submodule pgbouncer commit

* update pgbouncer's commit to support SSL connection

-change pgbouncer server_tls_ciphers default value

* docs - add gpcopy utility

* docs - gpcopy - review comment updates.

* docs - gpcopy reference - review comment updates and edits.

Also
--changed --dest-host to be a required option

* docs - gpcopy reference - command option changes.

-schema-only change to --metadata-only
--database change to --dbname
--batch-size change to --jobs

* docs - gpcopy ref. fix typos

* docs - discuss the partner connector (gppc) api

* address most of the edits requested by david

* add to requirements

* add the memory context functions

Dispatch tests don't expect backends created by other tests or auxiliary
processes like FTS and GDD, this commit disables GDD too to make dispatch
tests stable.

- Change strncpy to StrNCpy, make sure dest string be terminated
- Initilize some variables before use it.

Commit 1c1945fd9dbaf217062596062f73beac4934d7b6 broke compilation when
we use the trivial / dummy implementation of resource group. The fix for
that is trivial (this commit). But it begs the question: should we make
the build system less magical (switching the implementation based on the
platform), and instead just always exercise the dummy implementation (or
at least the building of it).

Previous algorithm scans entire directory to find specific relfilenode
extensions to be deleted. This is not optimal for large directory sizes. This
patch introduces extra logic based on the table extension pattern which helps
to avoid directory scan.

Algorithm is coded based on assumption that for CO tables a given concurrency
level either all columns have the file or none as well as the following file
table extension pattern:

  Heap Tables: contiguous extensions, no upper bound
  AO Tables: non contiguous extensions [.0 - .127]
  CO Tables: non contiguous extensions
         [  .0 - .127] for first column
         [.128 - .255] for second column
         [.256 - .283] for third column
         etc

  AO file format can be treated as a special case of CO tables with 1 column.

High level logic:
 1) Finds for which concurrency levels the table has files. This is
    calculated based off the first column. It performs 127
    (MAX_AOREL_CONCURRENCY) unlink().
 2) Iterates over the single column and deletes all concurrency level files.
    For AO tables this will exit fast.

This algorithm can be used for heap tables as well, however to prevent merge
conflicts it currently is only used for CO/AO tables.
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

Without this patch the strorage layout is not known in md and smgr layer. Due
to lack of this info sub-optimal operations need to be performed generically
for all table types. For example Heap specific functions like
ForgetRelationFsyncRequests(), DropRelFileNodeBuffers() gets called even for AO
and CO tables.

Adding new RelFileNodeWithStorageType struct to carry pass storage type to md
and smgr layer. XLOG_XACT_COMMIT and XLOG_XACT_ABORT wal records use the new
structure which has RelFileNode and storage type
Co-authored-by: NDavid Kimura <dkimura@pivotal.io>

* docs - pl/container new setting attribute roles

* docs - review comment updates for pl/container roles attribute

There was a recent change to fault injector framework that made simple
form "gp_inject_fault(faultname, type, db_id)" not work with
wait_until_triggered fault type. To get around this, we should
properly use "gp_wait_until_triggered_fault()" instead.

Reference:
https://github.com/greenplum-db/gpdb/commit/723e58481ad706d4c8f4f7af1325be2dcd36c985

This reverts commit 15493596.

For long running commands such as gpinitstandby with a large master data
directory, the server takes a long time. Therefore, there is no acitivity from
the client to the server. If the ClientAliveInterval is set, then the server
reports a timeout after ClientAliveInterval seconds.

Setting a ServerAliveInterval value less than the ClientAliveInterval interval
forces the client to send a Null message to the server.  Hence, avoiding the
timeout.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>
(cherry picked from commit 675aa8e3bd1d5bb187dc93d7ba494819cadb120e)

Alter tablespace needs to copy all underlying files of table from one tablespace
to other. For AO/CO tables this was implemented using full directory scan to
find files and copy when persistent tables were removed. This gets very
inefficient and varies in performance based on number of files present in the
directory. Instead use the same optimization logic used for `mdunlink_ao()`
leveraging known file layout for AO/CO tables.

Also, old logic had couple of bugs:
- missed coping the base or .0 file. Which means data loss if table was altered in past.
- xlogging even for temp tables

These are fixed as well with this patch. Additional tests added to cover for
those missing scenaiors. Also, moved the AO specific code to aomd.c, out of
tablecmds.c file to reduce conflicts to upstream.

Commit 07ee8008 added test section in
query_finish_pending.sql to validate case where if a query can be canceled when
cancel signal arrives fast than the query dispatched. For the same uses sleep
fault.

But the test was incorrect due to usage of "begin", as begin sleeps for 50 secs
instead of actual select query sleeping. Also, since the fault always trigger
the reset fault sleeps for additional 50 secs. Instead remove begin and just set
endoccurence to 1. Verified modified test fails/hangs without the fix and
passes/completes in couple secs with the fix.

bfv_partition tests fail if ICW is run n times after creating the cluster, as
the role is not dropped. With this commit now this test can be run n times
successfully without re-creating the cluster.

On the way also remove the suppression of warnings in role.sql.

Unit tests generate mock version of the .c files and its very annoying as they
get in TAGS file and always first visit lands in mocked implementation.