1. Enable windows ccl pipeline job, fix configure options.
2. Remove C++ check when orca is disabled, add needed header check.
3. Remove non-exist client docs from msi bundle.
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

* Include instructions to build optimizer now that it's default [#145362679]

In addition to renaming the files, a new test is added to cover the
scenario when readers need to use subtransaction xid cache in writer's
PGPROC.

NOTE: Please use `git show -M10` to see the rename instead of separate remove
and add.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Now that subtransaction information is no longer maintained in shared
snapshot between readers and a writer, the tests are obsolete.  Faults
used by the tests are removed as well.

This commit contains a gpdb_master pipeline change - to remove
"subtransaction" job that used to run the tests being removed by this
commit.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Originally, the reader kept copies of subtransaction information in
two places.  First, it copied SharedLocalSnapshotSlot to share between
writer and reader.  Second, reader kept another copy in subxbuf for
better performance.  Due to lazy xid, subtransaction information can
change in the writer asynchronously with respect to the reader.  This
caused reader's subtransaction information out of date.

This fix removes those copies of subtransaction information in the
reader and adds a reference to the writer's PGPROC to
SharedLocalSnapshotSlot.  Reader should refer to subtransaction
information through writer's PGPROC and pg_subtrans.

Also added is a lwlock per shared snapshot slot.  The lock protects
shared snapshot information between a writer and readers belonging to
the same session.

Fixes github issues #2269 and #2284.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

When gpexpand is run with a specific duration or end-time using '-d' or
'-e' flag, there might be an off-chance that gpexpand reports that the
re-distribute was successful, however, that's not the case.
gpexpand status details reports as COMPLETED, but the data has not been
redistributed and is still distributed by RANDOM.
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>
Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>

* Add note why gpfdist should not run on the master

They are not used, so remove them.

* GPDB DOCS - Add ALTER/CREATE/DROP EXTENSION cmd and catalog updates.

[ci skip]

* GPDB DOCS - update DROP EXTENSION - emphasize extension files must be remove manually.

This makes it easier to generate html docs for greenplum.org/doc

appendonlywriter.c: CID 129803 Identical code for different branches. The
condition is redundant in AtEOXact_AppendOnly_StateTransition: The same code is
executed regardless of the condition.

cdbfilerep.c: CID 129285 Buffer not null terminated. In
FileRep_GetUnknownIdentifier: The string buffer may not have a null terminator
if the source string's length is equal to the buffer size. Replaced the call
with strlcpy.

cdbmirroredflatfile.c: CID 130180 Uninitialized pointer read. In MirrorFlatFile:
Reads an uninitialized pointer or its target for mirroredOpen. Hence
initializing it.

postmaster.c: CID 129834 Argument cannot be negative. In checkIODataDirectory:
Negative value used as argument `fd` to a function `write` expecting a positive
value. Hence fixing it to be called in else only when fd not negative.

faultinject.c: CID 149132 Uninitialized scalar variable. In gp_fault_inject_impl: Use of an
uninitialized variable hdr.is_segv_msg just initializing the same.

persistentutil.c: CID 129278 Buffer not null terminated. In
gp_persistent_relation_node_check: The string buffer `fdata->databaseDirName`
may not have a null terminator if the source string's length is equal to the
buffer size. Use safer alternative strlcpy.

pg_resetxlog.c: CID 130064 Resource leak. In WriteEmptyXLOG: Leak of memory or
pointers to system resources `fp`. Hence call fclose on it.

Refactor addDistributedBy() to use less clever coding and instead
write it with readable code. Also remove variable initializations
to allow the compiler to help us catch errors, and clean up some
whitespace issues.

Before parallelization on nodes in cdbparallelize if there are
any subplan nodes in the plan which refer to the same plan_id,
parallelization step breaks as a node must be processed only
once by it. This patch fixes the issue by generating a new
subplan node in glob subplans, and updating the plan_id of the
subplan to refer to the newly created node.

The condition containing subplans will be duplicated
as the partition selection key in the PartitionSelector node.
It is not OK to duplicate the expression, if it contains
SubPlans, because the code that adds motion nodes to a
subplan gets confused if there are multiple SubPlans
referring the same subplan ID.

Skip duplicating subplans clauses as multiple subplan node
referring to same plan_id of subplans breaks cdbparallize.
cdbparallize expects to apply motion only to a node once,
so if two subplan nodes refers to the same plan_id the assertion breaks.
Signed-off-by: NJemish Patel <jpatel@pivotal.io>

Before this commit, snapshot stored information of distributed in-progress
transactions populated during snapshot creation and its corresponding localXids
found during tuple visibility check later (used as cache) by reverse mapping
using single tightly coupled data structure DistributedSnapshotMapEntry. Storing
the information this way possed couple of problems:

1] Only one localXid can be cached for a distributedXid. For sub-transactions
same distribXid can be associated with multiple localXid, but since can cache
only one, for other local xids associated with distributedXid need to consult
the distributed_log.

2] While performing tuple visibility check, code must loop over full size of
distributed in-progress array always first to check if cached localXid can be
utilized to avoid reverse mapping.

Now, decoupled the distributed in-progress with local xids cache separately. So,
this allows us to store multiple xids per distributedXid. Also, allows to
optimize scanning localXid only if tuple xid is relevant to it and also scanning
size only equivalent to number of elements cached instead of size of distributed
in-progress always even if nothing was cached.

Along the way, refactored relevant code a bit as well to simplify further.

* GPDB DOCS - new GUCs from 4.3.x and mdcache

* Typo fixes

* typo

* updates based or review comments.

* Remove the notion that the server configuration can be changed using pgAdminIII

The "gpconfig" tool should be used instead

* Add explicit notion that the data is stored in GPDB

PL/Perl support is not shipped on SLES so skip including Perl, and
components which depend on Perl, in the test set. By invoking the
the ./configure step in the test-tree without --with-perl and
--enable-mapreduce on SLES the tests for those should be avoided.

GPDB DOC - remove unused topic referencing gp_dump, gp_restore. Utilities were removed in GPDB 4.3.3.0

* Deprecate gpsupport

* Remove gpsupport, add gpmt

Few minor updates in error messages and comments.
Signed-off-by: NDhanashree Kashid <dkashid@pivotal.io>

* Explain the "-M fast" option

This commit updates the `rangefuncs_cdb` tests to use `plpgsql`
instead of `plperlu`; since `plperlu` is not supported by SLES.

We do not need a separate answer file for ORCA because the results
produced by planner and ORCA are the same. Hence this commit also
removes `rangefuncs_cdb_optimizer.out` file.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

* Replace "template1" with "postgres" database

* Move ddboost cleanup step to be last to ensure all files in directory are removed from data domain server

Follow on to #2495

* Updated SLES targets to remove perl and mapreduce

* Removed unneded --disable-mapreduce

Add targets for gpnetbenchClient and gpnetbenchServer to top-level
Makefile so that gpcheckperf runs with the open source build

Remove targets for gpnetbenchClient and gpnetbenchServer from gpAux
Makefile

Add additional debugging print statement to gpcheckperf
Signed-off-by: NMelanie Plageman <mplageman@pivotal.io>