In assure_collocation_and_order, we check if the lower plan is already
distributed on the partitioning key, and insert a Motion node if not.
That check was broken. When a query with multiple windows is planned, we
create a chain of Group+Sort stages, to compute each aggregate. To do that,
we create a synthetic subquery for each stage, and each subquery will have
its own minimal range table and target list. The sort and distribution
path keys should use expressions that refer to the original plan and
PlannerInfo that we started with, but in assure_collocation_and_order(), we
were incorrectly building path keys based on the target lists in the
intermediate stages. Those are not comparable with path keys referring to
the original target list, and as a result.

Fixes github issue #2236.

Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>

This was disabled a long time ago, but even after digging into the git
history, I could not figure out why. Nothing seems to break with it, and
if there's a problem with handling composite types in ORCA, disabling
this transformation would just be plastering over it anyway, because there
are many other ways that you could end up with composite type constants
in a query.

Currently the windows clients and loaders are not packaging correctly.
This code will be used when the team responsible for the CLs is ready
to fix the issues, and integrate the changes into the full CI for gpdb.
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Signed-off-by: NJim Doty <jdoty@pivotal.io>

- Update license paths to fix client and loader build

  The license header from gpAux/client/install/src/ was moved to the gpaddon
  repository as it was introducing a non-commercial EULA into the open source repository.
  The makefile path for the clients and loaders build has been updated to fix the build.

- Make the host used for running WiX a variable, and set it to match the
  infrastructure.

  We set up a private hosted zone in the the gpdb5-pipeline concourse
  network, and gave the wix-packaging vm the hostname:
  `wix-packaging.gpdb5-pipeline-vpc.pivotal.io`
Signed-off-by: NJingyi Mei <jmei@pivotal.io>
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Fixes Windows clients and loaders build, which uses the default system python.

PYTHONHOME was being set to the empty string when BLD_ARCH was win32 (or
another arch without an override for PYTHONHOME).
Setting PYTHONHOME to the empty string causes python not to find its
home, resulting in an error like "distutils module not found".
Signed-off-by: NJim Doty <jdoty@pivotal.io>

This commit restores the old packaging code.  A separate commit will
update the code so that it runs on the the current infrastructure.

[#139296853](https://www.pivotaltracker.com/story/show/139296853)
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Upstream commit fc1adce4 (and follow-up commit 9e6dc137) tightened up
pg_get_expr() so that you cannot pass an arbitrary string as the argument.
It must come from one of the few system catalogs that actually store
serialized expressions.

GPDB has one more catalog table that stores expression, pg_partition_rule,
so list the appropriate pg_partition_rule columns as allowed exceptions,
too.

Fixes github issue #2000, reported by @jonasbu11

Only normal xids can have distributed xids, so no point trying to check
distributed log and lookout for its corresponding distributed xid. Not sure how
much performance this impacts but its more important to not go chasing down
non-existent file, corresponding to frozen xids. Though currently for
distributed log it treats that as local transaction but just lot of extra work
to get that answer.

When specifying exclusion items to pg_basebackup we need to quote
them properly to avoid the risk of breaking out of the SCONST rule
in the lexer with ./pg_basebackup -E "foo'NOWAIT EXCLUDE'bar" or
something equally silly.

1.Change the format of 'total_queue_duration' in 'gp_toolkit.gp_resgroup_status' to keep
it consistent with 'rsgqueueduration' in 'pg_stat_activity'.
2.Show resource group information for running queries in 'pg_stat_activity'.

Signed-off-by: Richard Guo<riguo@pivotal.io>

The 'sreh' test was disabled in the regression suite, looks useful,
re-enable it.

GPDB 5 removed the error table feature, rewrite these cases to use
gp_read_error_log().
Signed-off-by: NPeifeng Qiu <pqiu@pivotal.io>

It's good, but triggers an assertion with ORCA enabled, safe to ignore.

> FATAL:  Unexpected internal error (transform.c:54)
> DETAIL:  FailedAssertion("!(equal(qcopy, query) && "Preprocessing should not modify original query object")", File: "transform.c", Line: 54)
> ...
> connection to server was lost

Objective of the test is only to check if DDL/DML on partition tables lock only
the root table on QD.  Child partitions are locked on QE as needed.  The
intermittent failures are due to more locks acquired than expected.  In all
such cases, the additional locks were AccessShare locks on some catalog table.
This commit filters out locks on catalog tables from expected output.

The intermittent failures merit investigation but that can happen regardless of
this test fix.

The coverity build was failing on the gpperfmon `make distclean` step

Fixed on both sides:
- Don't call `make distclean` in the coverity build script -- not
  necessary
- Don't clean gpperfmon from `gpAux/Makefile`: this Makefile's clean
  targets are for enterprise-build related artifacts. General developers
  should be running `make clean distclean` from the top level directory of
  this repository.

Additional fix -- doesn't address the coverity red but a similar
cleanup:
- Add cleanup of gpperfmon to the top level makefile's `distclean`
  target. Not sure why it wasn't there; probably just an accidental
  omission.
- Add note explaining why the top level makefile has to circumvent
  `gpAux/Makefile`
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>

This fixes a bug introduced in e378d84b
where the partition selector may go in an infinite loop because it never
got a chance to actually _select_ the partitions.

Commit 7b6dc9bf was well-intentioned,
but wrong: in cases where `outerPlan` is NULL, inputSlot should be
`NULL`, but the execution of the partition selector should *NOT*
prematurely end there, instead, a static partition selection should
happen (once) before this operator was exhausted (returning NULL).

This is a pure revert commit. The `NULL` initialization will follow.

This reverts commit 7b6dc9bf.

My OCD prevented me from glossing over this one..

FileRepPrimary_IsResyncManagerOrWorker() is implicitly declared via
the MIRROREDLOCK_BUFMGR_LOCK macro unless cdb/cdbfilerepprimary.h is
included. Add to avoid warning.

This codepath triggered a misleading-indentation warning in GCC and
I can agree with that. Re-indent to make reading easier.

It's not crystal clear from the code that the numeric_rel_num
counter is guarding the pointer deref on numeric_rels, so init
the pointer and add a guard check in the if-statement.

statVal is uninitialized in the case of an unknown property, skip
using it in that case.

Functions only used in Assertions yields warnings on unused code
in non-cassert enabled builds.  Wrap the functions in the macro
defined in assertion builds to avoid warning and make the intent
with the code clearer.

We want to make pgcrypto part of the default set of contrib modules
shipped with Greenplum. Add to the default set building.

We had previously backported most of contrib/pgcrypto from
upstream PostgreSQL 8.4, but not all. This brings in the
remaining bits to get us in line with REL8_4_STABLE, as well
as removes a few local patches to minimize diff to upstream
(only affecting style and some unused includes).

Among others, this commit completely merges:

  commit df2e62603ebc59a8fff5b6cd21b844370a95b061
  Author: Bruce Momjian <bruce@momjian.us>
  Date:   Thu Apr 17 12:37:53 2014 -0400

    pgcrypto:  fix memset() calls that might be optimized away

    Specifically, on-stack memset() might be removed, so:

    	* Replace memset() with px_memset()
    	* Add px_memset to copy_crlf()
    	* Add px_memset to pgp-s2k.c

    Patch by Marko Kreen

    Report by PVS-Studio

    Backpatch through 8.4.

This commit partially merges:

  commit 99058cfc634ad59fa80f802f96c334be313faefb
  Author: Bruce Momjian <bruce@momjian.us>
  Date:   Tue May 6 11:26:24 2014 -0400

    Remove tabs after spaces in C comments

    This was not changed in HEAD, but will be done later as part of a
    pgindent run.  Future pgindent runs will also do this.

    Report by Tom Lane

    Backpatch through all supported branches, but not HEAD

With this, we are at parity with REL8_4_STABLE plus the below
commits which have been backported from higher up due to them
containing security and/or important bugfixes:

  commit 4a9710e6edd1a23f6a053eb4b58b2b784245dc7e
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Tue Nov 11 17:22:15 2014 -0500
  Loop when necessary in contrib/pgcrypto's pktreader_pull().

  commit d95ebe0ac39b946c207ed0c37bba0365132ba3c6
  Author: Noah Misch <noah@leadboat.com>
  Date:   Mon Feb 2 10:00:45 2015 -0500
  Fix buffer overrun after incomplete read in pullf_read_max().
  Security: CVE-2015-0243

  commit d1972da8ce5857cec09645c11d8c7ac81b87d887
  Author: Noah Misch <noah@leadboat.com>
  Date:   Mon Feb 2 10:00:45 2015 -0500
  Cherry-pick security-relevant fixes from upstream imath library.
  Security: CVE-2015-0243

  commit 0ba20043129280c6d076c80cc60e3f73880b9ea4
  Author: Noah Misch <noah@leadboat.com>
  Date:   Mon May 18 10:02:31 2015 -0400
  pgcrypto: Report errant decryption as "Wrong key or corrupt data".
  Security: CVE-2015-3167

Error table feature was removed, but `\h CREATE EXTERNAL TABLE` still
has the description. This commit corrects it.

- Remove Makefile definitions only used by gpcheckdb
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

We add one more udf that only allocates on one specific segment and modify the hit_vlim_crit_section test to use the new udf. The original generic UDF uses an union all which creates one extra slice on segment 0. This allowed existence of two QE processes on segment 0.

The test checks if we successfully ignore runaway termination as long as we are in critical section. The critical section is a per-process state and therefore a second runaway detector on the same segment can still mark the session as runaway. This causes the session to eventually trigger a runaway cleanup after it comes out of critical section. We still don't PANIC as runaway termination successfully ignores. But, the runaway detection couldn't ignore the critical section because of per-process critical section state and this eventually produces a diff in concourse occasionally.

An alternative to introducing a new UDF is to modify the test to trigger it on a different segment other than segment 0, where the extra QE process currently gets created. I am waiting someone from QO to explain why we need an extra slice on segment 0. Until then I would rather not patch the test to trigger it on a different segment without having a guarantee that in the future planner will not generate the extra slice on another segment.

## CID 130007 Resource leak (file descriptor) in XlogFillZero()

Coverity thinks that a file descriptor with value 0 may be leaked (no
matching close() for an open()).  But open() will never return a file
descriptor 0 in this case.  Change the code still to make Coverity happy.

## CID 129351 Dead code in XLogAddRecordsToChangeTracking()

This is a legitimate case.  The `while (1)` loop already scans xlog
until the end, so that (record == NULL) is true.  There is no point in
trying to traverse till the end of xlog yet another time aftre
breaking out of the `while (1)`.  So replace the code and add assert
instead.