* Update readme instructions for using docker for building GPDB

It works fine without all those links in each and every directory in source code
so remove the extra work and annoyance.

The common situation was doing a big rebase of a branch: a conflicting
diff shows up in the template file as well as the generated pipeline.

The right way to handle that situation is resolve the diff in the
template and then regenerate the pipeline (rather than resolving the
diff in the generated file)

However, before this change, the pipeline validation was happening
before the new pipeline was generated, and thus the yaml parsing choked
on the git merge conflict indicators `======`

Also, only print out the message if it passes validation.

Author: C.J. Jameson <cjameson@pivotal.io>
Author: Jim Doty <jdoty@pivotal.io>

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Originaly, we use GUC gp_fts_probe_pause to skip FTS probe. However, in
order to change the FTS behavior, we need to signal the FTS process.
This will cause the test non-deterministic if the GUC value is used
before SIGHUP is handled by the backends to load the new value.

Using the fault injector, we directly set flag in the shared memory,
then immediately, FTS knows it will skip.

This will make the test more deterministic.

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

Replace rescan_requested and processing_fullscan with fullscan_requested
flag, and simpilfy the sleep logic in FtsLoop().

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

The pg_ctl response output for PQPING_MIRROR_READY was missing in
https://github.com/greenplum-db/gpdb/commit/b824fe8f269934987e30a4c177605b250703d20f. We
just need it to say the mirror has started.

Reported by Daniel Gustafsson in gpdb-dev mailing list:
https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/oNrxvVEG-UY

The commit that broke win32 compilation:
https://github.com/greenplum-db/gpdb/commit/f9c7df6914b63487a10b319e998e611986b0d7ce

To fix this, we put some win32 ifdefs around Greenplum-specific code
around threading. We don't really compile windows server so this
should be fine for now to compile the windows CCL.

This is a partial backport of the below commit from PostgreSQL 9.1,
the part left out is documentation:

  commit 35eb0958
  Author: Peter Eisentraut <peter_e@gmx.net>
  Date:   Thu Jan 13 09:13:12 2011 +0200

    Don't run regression tests in SQL_ASCII encoding by default

    Instead, run them in the encoding that the locale selects, which is
    more representative of real use.

    Also document how locale and encoding for regression test runs can be
    selected.

If we avoid calling pg_regress with an empty --multibyte we
can skip GPDB specific handling of the encoding parameter in
pg_regress and instead follow upstream. While the local diffs
to pg_regress weren't wrong per se, avoiding merge conflicts
is of more interest.

Also revert the commented out MULTIBYTE in the makefile to
match the upstream value. A follow-up commit will fix this
as well.

The errcode thrown in an ereport() on a segment was passed back to
the dispatcher, but then dropped and replaced with a default errcode
of ERRCODE_DATA_EXCEPTION. This works for most situations, but when
trapping errors the exact errcode must be propagated. This extends
the API to extract the errcode as well. The below case illustrates
the previous issue:

  CREATE TABLE test1(id int primary key);
  CREATE TABLE test2(id int primary key);
  INSERT INTO test1 VALUES(1);
  INSERT INTO test2 VALUES(1);
  CREATE OR REPLACE FUNCTION merge_table() RETURNS void AS $$
  DECLARE
	v_insert_sql varchar;
  BEGIN
	v_insert_sql :='INSERT INTO test1 SELECT * FROM test2';
	EXECUTE v_insert_sql;
	EXCEPTION WHEN unique_violation THEN
		RAISE NOTICE 'unique_violation';
	END;
  $$ LANGUAGE plpgsql volatile;
  SELECT merge_table();

This information can be easily derived from CDXLColRef member of
CDXLScalarIdent. This now mirrors what is done in the ORCA types
CScalarIdent and CColRef.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

* Use PXF docker image to run PXF smoke

* Remove unused singlecluster resource

Author: Xin Zhang <xzhang@pivotal.io>
Author: Asim R P <apraveen@pivotal.io>

- Add libpython
- Add libevent
- Add libgcc_s

This reverts commit fe30f20aacd78cfc1b9ba587152b8c73fbb2d045.

We can start mirror with -w flag now.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

Previously, the mirror would send a retry request in the FTS probe
response. This is unnecessary because the mirror's primary which would
now be considered the mirror is already assumed down by the FTS. This
prevents the retry loop from happening and allows the FTS to send
promotion signal faster.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

The gpstart utility does a begin/commit at the end of gpstart to
verify the cluster is ready to be handed off to the user.  To prevent
false errors or indefinite connection hang, we should have retry and
timeout in the connection string.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

These values are duplicated and prone to errors where they are updated
in errcodes.h but not here. We should just use the ERRCODE values in
errcodes.h instead.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

Previously, we did not use -w pg_ctl flag for mirror which waits until
segment is able to accept connections. We found that a promotion
scenario in which we run gpstart after FTS had updated the catalog and
before the promotion signal was sent would give a deadlock during
parallel segment start. To fix this, we add a new postmaster flag to
show walreceiver started at least once and respond to libpq calls
accordingly with CAC_MIRROR_READY value.

Author: Ashwin Agrawal <aagrawal@pivotal.io>
Author: Jimmy Yih <jyih@pivotal.io>

Pass tp to get_attstatsslot() instead of vardata->statsTuple to improve
readability.  This was changed while cherry-picking commit ae06d7b0 from
upstream.

The variable nvaluesp was being assigned the value of sslot.nvalues
after sslot was passed into free_attstatsslot. Since nvaluesp was not
being used in the function, just remove it to fix the issue.

Also get rid of the cdb version of mcv_selectivity() since the extra OUT
parameter `nvaluesp` is no longer there.

Fix suggested by Daniel Gustafsson

Author: Abhijit Subramanya <asubramanya@pivotal.io>
Author: Dhanashree Kashid <dkashid@pivotal.io>

- This includes how to install key library and gems on a mac.
- Improves formatting.

This is basically just work-around for broken functionality of current
CI. Ideally it should work with arbitary number of lines dumped. But since it
currently cannot and mostly fails to load if diff output is very long, adding
limit to number of lines printed. Don't know what the upper limit is but trying
with 1000. As if its broked beyond 1000 lines then its not helpful anyways
to look from CI and instead logging in container and checking is better.

Follow-up to commit 8922315e.

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

- Upgrades to version 10.1.12

Solves https://github.com/greenplum-db/gpdb/issues/4361

Filespaces are no more, remove comment and match schema with
upstream.

* Don't generate ssh keys for gpadmin if already present

* Let rest of the logic flow if the private key is present.

Entry DB process share snapshot with QD process, but it didn't update
the TransactionXmin. The assert in SubTransGetData() will fail in some
cases:
   Assert(TransactionIdFollowsOrEquals(xid, TransactionXmin));

1.QD takes a snapshot which contains a in-progress transaction A.
2.Transaction A commits.
3.QD creates a gang of entry DB. The entry DB process takes a snapshot
in InitPostgres and updates TransactionXmin.
4.The entry DB process scans the tuple inserted by the transaction A,
and find it's in the snapshot but its xid is larger than TransactionXmin.

* docs - add distribution and skew topic, address random dist skew

* clarify single tuple assigned to first segment