https://github.com/snaga/xlogdump.git, commit id
f534a9f6f70ffb691129a9e9ad0b9a85fdef403c

This was intentionally left broken by the merge of upstream B-tree split
WAL-logging changes. Should work now.

We explicitly checked if the subplan was projection-capable, and added
the hash expression ot the subplan's target list if it was. But projecting
the hash expression is not optional, you get a crash if you don't do it. So
instead of doing it only when the subplan is projection-capable, insert a
Result node if needed.

This version changes the spelling of the error message of violating a NOT
NULL-constraint, to match the message of the same error in the executor, so
those differences in regression test expected output can now be reverted.

Most of the patches in this merge were already cherry-picked into GPDB, the
changes merged are a fix for disallowing committing prepared transactions
cross database, followup fixes for the Makefile =/:= patches, an infinite
recursion fix in plan tree dumps and some documentation fixes.

 Conflicts:
	contrib/fuzzystrmatch/fuzzystrmatch.c
	doc/TODO
	doc/src/FAQ/TODO.html
	src/Makefile.shlib
	src/backend/libpq/pqcomm.c
	src/backend/optimizer/README
	src/backend/optimizer/geqo/geqo_eval.c
	src/backend/optimizer/path/joinrels.c
	src/backend/optimizer/plan/initsplan.c
	src/backend/postmaster/postmaster.c
	src/backend/utils/adt/formatting.c
	src/backend/utils/adt/xml.c
	src/backend/utils/error/elog.c
	src/bin/scripts/clusterdb.c
	src/bin/scripts/reindexdb.c
	src/bin/scripts/vacuumdb.c
	src/include/utils/xml.h
	src/interfaces/ecpg/ecpglib/execute.c
	src/port/open.c
	src/tools/msvc/Solution.pm
	src/tools/msvc/clean.bat
	src/tools/msvc/install.pl
	src/tools/msvc/mkvcbuild.pl
	src/tools/msvc/vcregress.bat

Remove the abs() call since ackTime is unsigned (newRTT is as well)
and thus can't be negative.

Also took the liberty of removing comments that doesn't provide any
value to the reader of the code, seems like dead code rather than
documentation.

oldPersistentSerialNum and segmentFileNum are only initalized in the
subsequent loop and printing them before the loop doesn't make any
sense.

This is preceeded by an elog(ERROR ..); call but the compiler will
warn for a potential unitialized use so initialize to InvalidOid
since it is the correct init value anyways.

This is a cherry-pick (with slight massage) of the commit referenced below
which would come eventually in the merge but was trivial enough to fix now.

  commit b2d80147d5a3d0a5815c5b8e63a6fd9cef17f4a7
  Author: Kevin Grittner <kgrittn@postgresql.org>
  Date:   Fri Dec 27 15:41:46 2013 -0600

      Fix misplaced right paren bugs in pgstatfuncs.c.

      The bug would only show up if the C sockaddr structure contained
      zero in the first byte for a valid address; otherwise it would
      fail to fail, which is probably why it went unnoticed for so long.

      Patch submitted by Joel Jacobson after seeing an article by Andrey
      Karpov in which he reports finding this through static code
      analysis using PVS-Studio.  While I was at it I moved a definition
      of a local variable referenced in the buggy code to a more local
      context.

      Backpatch to all supported branches.

This reverts commit 5dd2f280.

This reverts commit b46f781f.

relpages is not updated on master during create index and reindex, as
master need to have agreegated value from segments. As not updating the
relpages, avoid calling RelationGetNumberOfBlocks for master in this
function.

The gpssh utility can have issues during the initial ssh connection
step due to different network configurations that may cause delay.
This commit exposes two key pxssh variables, delaybeforesend and
sync_multiplier, into configurables so that users can change them
as needed to ensure their gpssh works as intended in their network.

Authors: Christopher Hajas and Jimmy Yih

This set of patches were largely already merged, the end result being some
strlcpy and Makefile cleanups being commited as well as an ordering fix for
a test. Bumping the catversion was obviously not required here but I will
keep bumping it to match the upstream changes to keep our master logically
in sync.

 Conflicts:
	GNUmakefile.in
	contrib/btree_gist/Makefile
	contrib/cube/Makefile
	contrib/gp_internal_tools/Makefile
	contrib/hstore/Makefile
	contrib/ltree/Makefile
	contrib/pg_trgm/Makefile
	contrib/seg/Makefile
	contrib/sslinfo/Makefile
	contrib/tablefunc/Makefile
	contrib/tsearch2/Makefile
	contrib/xml2/Makefile
	doc/FAQ
	doc/Makefile
	doc/TODO
	doc/src/FAQ/FAQ.html
	doc/src/FAQ/TODO.html
	doc/src/sgml/Makefile
	src/Makefile.global.in
	src/Makefile.shlib
	src/backend/Makefile
	src/backend/access/Makefile
	src/backend/access/common/heaptuple.c
	src/backend/access/heap/heapam.c
	src/backend/access/transam/xact.c
	src/backend/catalog/Makefile
	src/backend/commands/dbcommands.c
	src/backend/libpq/crypt.c
	src/backend/libpq/hba.c
	src/backend/libpq/ip.c
	src/backend/optimizer/Makefile
	src/backend/postmaster/pgstat.c
	src/backend/storage/Makefile
	src/backend/utils/Makefile
	src/backend/utils/adt/formatting.c
	src/backend/utils/fmgr/fmgr.c
	src/backend/utils/time/Makefile
	src/backend/utils/time/combocid.c
	src/bin/Makefile
	src/bin/pg_ctl/pg_ctl.c
	src/include/access/htup.h
	src/include/catalog/catversion.h
	src/include/pgstat.h
	src/include/storage/bufpage.h
	src/include/utils/combocid.h
	src/interfaces/Makefile
	src/interfaces/ecpg/test/Makefile
	src/interfaces/libpq/fe-auth.c
	src/interfaces/libpq/fe-connect.c
	src/pl/plperl/GNUmakefile
	src/pl/plperl/plperl.c
	src/pl/plpgsql/src/Makefile
	src/pl/plpython/Makefile
	src/pl/plpython/plpython.c
	src/port/Makefile
	src/test/regress/GNUmakefile
	src/test/regress/expected/combocid.out
	src/test/regress/expected/without_oid.out
	src/test/regress/parallel_schedule
	src/test/regress/sql/combocid.sql
	src/timezone/Makefile
	src/timezone/pgtz.c
	src/timezone/tznames/Makefile

This merge brings in a WAL optimization to the page split code to
avoid WAL logging the tuples that stay on the original page in a
page split.

Included in this merge pulls are also the following commits that fix
issues in, as well as tidies up, the split code: bc292937, 6f519ad0,
226a1005, a8d539f1 and 93190c30. See below for full commit messages for
each of these patches.

The pg_standby contrib module is merges as well but is disconnected
from the build since it doesn't make sense in a GPDB installation.

HEADS UP: The changes to cdbresynchronizechangetracking.c in this
commit are incomplete and will be fixed up in subsequent commits.

 Conflicts:
	contrib/Makefile
	doc/TODO
	doc/src/FAQ/TODO.html
	doc/src/sgml/biblio.sgml
	doc/src/sgml/config.sgml
	doc/src/sgml/monitoring.sgml
	doc/src/sgml/mvcc.sgml
	src/backend/access/nbtree/nbtinsert.c
	src/backend/access/nbtree/nbtxlog.c
	src/backend/access/transam/xact.c
	src/backend/libpq/auth.c
	src/backend/libpq/hba.c
	src/backend/postmaster/pgstat.c
	src/backend/utils/adt/formatting.c
	src/backend/utils/adt/oracle_compat.c
	src/backend/utils/adt/pgstatfuncs.c
	src/bin/pg_dump/pg_backup_files.c
	src/bin/psql/common.c
	src/bin/psql/copy.c
	src/bin/psql/prompt.c
	src/include/catalog/catversion.h
	src/interfaces/libpq/fe-connect.c
	src/interfaces/libpq/fe-secure.c
	src/pl/plpgsql/src/pl_exec.c
	src/test/regress/pg_regress.c
	src/test/regress/resultmap
	src/tools/entab/entab.c
	src/tools/msvc/Solution.pm
	src/tools/msvc/install.pl
	src/tools/msvc/mkvcbuild.pl

  commit 6f519ad0
  Author: Bruce Momjian <bruce@momjian.us>
  Date:   Wed Feb 21 20:02:17 2007 +0000

      btree source code cleanups:

      I refactored findsplitloc and checksplitloc so that the division of
      labor is more clear IMO. I pushed all the space calculation inside the
      loop to checksplitloc.

      I also fixed the off by 4 in free space calculation caused by
      PageGetFreeSpace subtracting sizeof(ItemIdData), even though it was
      harmless, because it was distracting and I felt it might come back to
      bite us in the future if we change the page layout or alignments.
      There's now a new function PageGetExactFreeSpace that doesn't do the
      subtraction.

      findsplitloc now tries the "just the new item to right page" split as
      well. If people don't like the refactoring, I can write a patch to just
      add that.

      Heikki Linnakangas

  commit bc292937
  Author: Bruce Momjian <bruce@momjian.us>
  Date:   Sat Mar 3 20:13:06 2007 +0000

      Split _bt_insertonpg to two functions.

      Heikki Linnakangas

  commit 226a1005
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Wed Apr 11 20:47:38 2007 +0000

      Code review for btree page split WAL reduction patch.  Make it actually work
      (original code *always* created a full-page image for the left page, thus
      leaving the intended savings unrealized), avoid risk of not having enough room
      on the page during xlog restore, squeeze out another couple bytes in the xlog
      record, clean up neglected comments.

  commit a8d539f1
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Sun May 20 21:08:19 2007 +0000

      To support external compression of archived WAL data, add a flag bit to
      WAL records that shows whether it is safe to remove full-page images
      (ie, whether or not an on-line backup was in progress when the WAL entry
      was made).  Also make provision for an XLOG_NOOP record type that can be
      used to fill in the extra space when decompressing the data for restore.

      This is the portion of Koichi Suzuki's "full page writes" patch that
      has to go into the core database.  The remainder of that work is two
      external compression and decompression programs, which for the time being
      will undergo separate development on pgfoundry.  Per discussion.

      Also, twiddle the handling of BTREE_SPLIT records to ensure it'll be
      possible to compress them (the previous coding caused essential info
      to be omitted).  The other commonly-used record types seem OK already,
      with the possible exception of GIN and GIST WAL records, which I don't
      understand well enough to opine on.

  commit 93190c30
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Fri Nov 16 19:53:50 2007 +0000

      Repair still another bug in the btree page split WAL reduction patch:
      it failed for splits of non-leaf pages because in such pages the first
      data key on a page is suppressed, and so we can't just copy the first
      key from the right page to reconstitute the left page's high key.
      Problem found by Koichi Suzuki, patch by Heikki.

to/from client;

This is intended for scenarios where QD hangs sending data to client(
e.g, client is buggy not receiving data), or hangs waiting for data
from client(e.g, COPY foo FROM stdin);

There is a send mutex to prevent from sending a message to client when
another message is in the middle of sending, however, this send mutex
can lead to self deadlock in some cases, including this hanging send()
case; this commit also resolve this kind of self deadlock, at the cost of
discarding some message, which is acceptable when the process is dying.

In some error cases, the resource queue lock is not released, hence
leading to lock leak and inconsistent resource queue status; this commit
overhauls the resource queue locking code to complete the cleanups if error
or signal happens during acquiring resource queue lock; meanwhile, some
unnecessary code blocks are removed and unreasonable code are fixed to make
it easier for reading and understanding;

Release ErrorData after logging to error table

If we dont' free these error data, it would cause OOM if there are a
lot of bad data imporetd from external table

Renaming optimizer_release_mdcache to a more intuitive name: optimizer_metadata_caching. Before this commit, we were caching metadata in optimizer side by setting optimizer_release_mdcache = OFF. Now, we cache metadata in optimizer side by setting optimizer_metadata_caching=ON.

table transfer, also updated the help doc with more details.

This set of patches were largely already cherry-picked, no new functionality
merged with just an updated test and some small fixes to match upstream.

 Conflicts:
	configure
	configure.in
	contrib/chkpass/chkpass.c
	contrib/dblink/dblink.c
	contrib/intarray/_int_bool.c
	contrib/tsearch2/query.c
	doc/FAQ
	doc/bug.template
	doc/src/FAQ/FAQ.html
	doc/src/sgml/release.sgml
	src/backend/access/transam/xlog.c
	src/backend/commands/portalcmds.c
	src/backend/libpq/be-secure.c
	src/backend/postmaster/postmaster.c
	src/backend/utils/fmgr/dfmgr.c
	src/bin/pg_config/pg_config.c
	src/bin/psql/tab-complete.c
	src/include/port.h
	src/interfaces/ecpg/preproc/descriptor.c
	src/port/strlcat.c
	src/test/regress/pg_regress.c
	src/tools/msvc/mkvcbuild.pl

closes #327

This set of patches adds support for cross-type hashing in hashed subplans
and moves NAMEDATALEN to pg_config_manual.h.

To better match upstream, this commit also renames ExecStoreMemTuple() to
ExecStoreMinimalTuple() since it caused a conflict here.

Conflicts:
	doc/TODO
	doc/src/FAQ/TODO.html
	doc/src/sgml/libpq.sgml
	doc/src/sgml/syntax.sgml
	doc/src/sgml/xindex.sgml
	doc/src/sgml/xoper.sgml
	src/backend/access/nbtree/nbtinsert.c
	src/backend/catalog/Makefile
	src/backend/catalog/genbki.sh
	src/backend/executor/execGrouping.c
	src/backend/executor/nodeSubplan.c
	src/backend/optimizer/plan/subselect.c
	src/backend/port/sysv_shmem.c
	src/include/catalog/catversion.h
	src/include/catalog/pg_operator.h
	src/include/pg_config_manual.h
	src/tools/msvc/genbki.pl

Merge fixes to check if a tuple is over the TOAST threshold for tuple
length and adjust the tuple threshold to avoid wasting space per tuple
and protect against potential data corruption (up until now there was
only wasted space as a consequence of the incorrect calculation).

This changes the on disk storage of toast tables.

 Conflicts:
	doc/TODO
	doc/src/FAQ/TODO.html
	doc/src/sgml/libpq.sgml
	src/backend/access/heap/heapam.c
	src/backend/access/heap/tuptoaster.c
	src/bin/psql/copy.c
	src/include/access/tuptoaster.h
	src/include/c.h
	src/include/catalog/catversion.h

This set of patches brings mainly the XMLSERIALIZE code which has been merged
already in the large XML merge previously done. This commit changes a few XML
related bits to better match upstream and be somewhat cleaner but doesnt alter
any functionality. On top of that there are a few ecpg related changes.

 Conflicts:
	configure
	doc/TODO
	doc/src/FAQ/TODO.html
	doc/src/sgml/features.sgml
	doc/src/sgml/install-win32.sgml
	doc/src/sgml/installation.sgml
	doc/src/sgml/libpq.sgml
	src/backend/catalog/information_schema.sql
	src/backend/catalog/sql_features.txt
	src/backend/commands/vacuumlazy.c
	src/backend/executor/execQual.c
	src/backend/nodes/copyfuncs.c
	src/backend/nodes/equalfuncs.c
	src/backend/nodes/outfuncs.c
	src/backend/nodes/readfuncs.c
	src/backend/parser/gram.y
	src/backend/parser/parse_expr.c
	src/backend/parser/parse_target.c
	src/backend/utils/adt/ruleutils.c
	src/backend/utils/adt/xml.c
	src/include/catalog/catversion.h
	src/include/catalog/pg_proc.h
	src/include/nodes/nodes.h
	src/include/nodes/parsenodes.h
	src/include/nodes/primnodes.h
	src/include/utils/errcodes.h
	src/include/utils/xml.h
	src/interfaces/ecpg/ChangeLog
	src/interfaces/ecpg/ecpglib/execute.c
	src/interfaces/ecpg/ecpglib/typename.c
	src/test/regress/expected/xml.out
	src/test/regress/expected/xml_1.out
	src/test/regress/sql/xml.sql
	src/tools/msvc/install.pl

* Add configuration option for gpfdist
use --disable-gpfdist to turn off gpfdist build

Enable to run extension regression tests from top level makefile

* Add installcheck gpfdist into extensions
* Add check for enterprise targeted builds
* Pass apr config to gpfdist
* Add JAVA_HOME check

Authors:
Marbin Tan and Shujie Shang

This merges two patches for security vulnerabilities that were already
applied in GPDB. The changes here are mainly superficial to bring GPDB
closer to upstream.

 Conflicts:
	doc/src/sgml/release.sgml
	src/backend/commands/tablecmds.c
	src/backend/executor/execMain.c
	src/backend/executor/execQual.c
	src/backend/executor/execScan.c
	src/backend/executor/execUtils.c
	src/backend/executor/functions.c
	src/backend/executor/nodeGroup.c
	src/backend/executor/nodeNestloop.c
	src/backend/executor/nodeResult.c
	src/backend/optimizer/util/clauses.c

This reverts commit f21b53f0.

1. Replace UNAME_P (uname -p) with UNAME_M (uname -m)
2. Remove ARCH_FLAGS usage from OSX build

ARCH_FLAGS is only used by "gpopt" to figure out
location and build architecture of ORCA/GPOS
components.

It's no longer useful to OSS process, since ORCA
and GPOS are already installed to default locations.

3. Copy libdxltranslators as part of 'make install'

Closes #316
Signed-off-by: NOmer Arap <oarap@pivotal.io>

This set of patches bring more spelling fixes to errormessages. The PL fix
had already been merged.

 Conflicts:
	contrib/earthdistance/expected/earthdistance.out
	doc/src/sgml/high-availability.sgml
	doc/src/sgml/sources.sgml
	src/pl/plpgsql/src/pl_exec.c
	src/test/regress/expected/domain.out

We count references to a CTE in parse analysis phase, but the way the
planner expands CTE references when gp_cte_sharing=off means that if there
is a references to a CTE, within another CTE, expanding references to the
outer CTE will also multiple the references to the inner CTE. So don't
assume that we can trust cterefercount in the planner. (It should be OK
for cterefcount's original purpose, to detect the case that there are no
references at all to a CTE, though.)

This fixes an assertion failure or crash, with the test query included
in the patch.

Refactor vacuum_appendonly_fill_stats so that it doesn't need to know
about those structs, and move them back to vacuum.c. This reduces the diff
vs. upstream, which will hopefully make merging slightly easier.