This corresponds to ORCA commit "Use prime number of buckets to avoid
hash collisions"
Authored-by: NChris Hajas <chajas@pivotal.io>

This is the equivalent of
https://github.com/greenplum-db/gporca/pull/462,
"Normalize expression before inferring predicates".

For ORCA commit "Fix crash when merging group containing CTE consumer"
Authored-by: NChris Hajas <chajas@pivotal.io>

Corresponding ORCA commit: "Fix crash during expression to DXL
translation in correlated NLJ"

Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>
Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

The default is a non OSS build across CI. Explict CI jobs that do build
OSS can use `--with-extra-version` to append `-oss`.
Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

* Build necessary extensions for release

* Add new  '--enable-debug-extensions' configure option, if provide the option,
  the extensions include:

      gp_distribution_policy
      gp_internal_tools
      gp_sparse_vector
      gp_replica_check
      gp_inject_fault
      gp_debug_numsegments

* Rename configure_flags to configure_flags_with_extensions

  The configure_flags_with_extensions SHOULD BE set to

      configure_flags_with_extensions: "--enable-cassert --enable-debug-extensions"

    in these secrets files

       secrets/gpdb_master-ci-secrets.dev.yml
       secrets/gpdb_master-ci-secrets.prod.yml
Co-authored-by: NBob Bao <bbao@pivotal.io>
Co-authored-by: NNing Fu <nfu@pivotal.io>

* Bump ORCA version to 3.29.0

* ORCA: Updating subquery plans in ICG expected files

This change is needed for ORCA PR https://github.com/greenplum-db/gporca/pull/449.
Some subquery plans changed in minor ways in the ICG test.
Co-authored-by: NChris Hajas <chajas@pivotal.io>

This reverts commit 848733b6.

This reverts commit bf7836f0.

Bump ORCA version to v3.27.0
Signed-off-by: NRahul Iyer <riyer@pivotal.io>

Co-authored-by: NChris Hajas <chajas@pivotal.io>
Co-authored-by: NShreedhar Hardikar <shardikar@pivotal.io>

Adds tests for ORCA commit 'Fix wrong results from LOJ reordering'

gphdfs is being deprecated for GPDB 6. This commit removes the
following:
- Remove gphdfs from pipelines
- Remove gphdfs from gpcontrib
- Remove gphdfs from Makefile
- Remove tests using gphdfs
- Remove GUCs, code, and comments used for gphdfs
- Remove gphdfs from GPDB docs
- Remove avro and parquet formats

Authored-by: NKaren Huddleston <khuddleston@pivotal.io>

We backported 128-bit integer support to speed up aggregates (commits
8122e143 and 959277a4) from upstream 9.6 into Greenplum (in
commits 9b164486 and 325e6fcd). However, we forgot to also port a
follow-up fix postgres/postgres@7518049980b, mostly because it's nuanced
and hard to reproduce.

There are two ways to tell the brokenness:

1. On a lucky day, tests would fail on my workstation, but not my laptop (or
   vice versa).

1. If you stare at the generated code for `int8_avg_combine` (and friends),
   you'll notice the compiler uses "aligned" instructions like `movaps` and
   `movdqa` (on AMD64).

Today's my lucky day.

Original commit message from postgres/postgres@7518049980b (by Tom Lane):

> Our initial work with int128 neglected alignment considerations, an
> oversight that came back to bite us in bug #14897 from Vincent Lachenal.
> It is unsurprising that int128 might have a 16-byte alignment requirement;
> what's slightly more surprising is that even notoriously lax Intel chips
> sometimes enforce that.

> Raising MAXALIGN seems out of the question: the costs in wasted disk and
> memory space would be significant, and there would also be an on-disk
> compatibility break.  Nor does it seem very practical to try to allow some
> data structures to have more-than-MAXALIGN alignment requirement, as we'd
> have to push knowledge of that throughout various code that copies data
> structures around.

> The only way out of the box is to make type int128 conform to the system's
> alignment assumptions.  Fortunately, gcc supports that via its
> __attribute__(aligned()) pragma; and since we don't currently support
> int128 on non-gcc-workalike compilers, we shouldn't be losing any platform
> support this way.

> Although we could have just done pg_attribute_aligned(MAXIMUM_ALIGNOF) and
> called it a day, I did a little bit of extra work to make the code more
> portable than that: it will also support int128 on compilers without
> __attribute__(aligned()), if the native alignment of their 128-bit-int
> type is no more than that of int64.

> Add a regression test case that exercises the one known instance of the
> problem, in parallel aggregation over a bigint column.

> This will need to be back-patched, along with the preparatory commit
> 91aec93e.  But let's see what the buildfarm makes of it first.

> Discussion: https://postgr.es/m/20171110185747.31519.28038@wrigleys.postgresql.org

(cherry picked from commit 75180499)

- Added with-quicklz configure flag
- Added quicklz gpcontrib directory with C wrapper functions and SQL installation file
- Added simple quicklz functional tests
- Added #undef HAVE_LIBQUICKLZ to pg_config.h.win32.
  This is to parallel the recent change in pg_config.h.in that adds
  quicklz. pg_config.h.win32 should be autogenerated, but isn't in
  practice.
Co-authored-by: NJimmy Yih <jyih@pivotal.io>
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NDavid Sharp <dsharp@pivotal.io>

Co-authored-by: NChris Hajas <chajas@pivotal.io>

This reverts commit dbece3da.

Performance regression observed for tpcds queries due to cardinality
misestimation. Impacted TPCDS queries 174, 111 and 104.
Signed-off-by: NSambitesh Dash <sdash@pivotal.io>

- set config flags to disable zstd in platforms that don't have the
  library baked in
  - currently only centos6 and centos7 have zstd provided in the build
    images
- delete the suse10 config flags, because it is no longer a supported
  platform
- add EXTRA_CONFIG_FLAGS to deb_create_package.bash
Co-authored-by: NTaylor Vesely <tvesely@pivotal.io>
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NKaren Huddleston <khuddleston@pivotal.io>

ORCA version to v3.17.0

This patch introduces a guc `optimizer_enable_agg_skew_avoidance` which
can be used to avoid plans which generate a lot of skew when aggregates
are used in the query. The default value of this guc is true.
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>
Co-authored-by: NHans Zeller <hzeller@pivotal.io>

Also bump ORCA's version to 3.16.0
Co-authored-by: NHans Zeller <hzeller@pivotal.io>

The support for sending alerts via Email or SNMP was quite a kludge,
and there are much better external tools for managing alerts than
what we can supply in core anyways so this retires the capability.

All references to alert sending in the docs are removed, but there
needs to be section written about how to migrate off this feature
in the release notes or a similar location.

Discussion: https://github.com/greenplum-db/gpdb/pull/6384

Co-authored-by: NChris Hajas <chajas@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>

When ZSTD compression is used for AO CO tables, insertion of data may cause an
error "Destination buffer is too small". This happens when compressed data is
larger than uncompressed input data.

This commit adds handling of this situation: do not change output buffer and
return size used equal to source size. The caller (e.g.,
'cdbappendonlystoragewrite.c') is able to handle such output; in this case, it
copies data from input to output itself.