[ci-skip]

The content is copied from an internal document, barring formatting changes.
Thank you Xin for digging out that document.  Let's bring the content close to
source code so that it's alive and useful.

[ci-skip]

* Ban split of list-partitioned sub-partitions

Currently, GPDB does not support splitting a multi-level partition when
the leaf partition is partitioned by list. However, in many cases, we
did not correctly error out and instead crashed. This commit detects a
query attempting to split a list partitioned sub-partition and correctly
errors out.
Signed-off-by: NSam Dash <sdash@pivotal.io>

* Ban split of list-partitioned sub-partitions

Currently, GPDB does not support splitting a multi-level partition when
the leaf partition is partitioned by list. However, in many cases, we
did not correctly error out and instead crashed. This commit detects a
query attempting to split a list partitioned sub-partition and correctly
errors out.
Signed-off-by: NSam Dash <sdash@pivotal.io>

* Ban split of list-partitioned sub-partitions

Currently, GPDB does not support splitting a multi-level partition when
the leaf partition is partitioned by list. However, in many cases, we
did not correctly error out and instead crashed. This commit detects a
query attempting to split a list partitioned sub-partition and correctly
errors out.
Signed-off-by: NSam Dash <sdash@pivotal.io>

CID 178114: Performance inefficiencies  (PASS_BY_VALUE)
The `ProbeConnectionInfo` parameter was being passed to the
`probeSegmentHelper()` function by value. This is very inefficient. Fixed it by
passing the parameter as a reference using a pointer.

CID 178115: Calling "pqGetInt" without checking return value (as is done
elsewhere 52 out of 57 times).
The return value of `pqGetInt()` function was not being checked in the
`processProbeResponse()` function. Fixed it by checking for the return value.
If EOF is returned, a message is logged and false is returned to the caller.
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

Signed-off-by: NChris Hajas <chajas@pivotal.io>
Signed-off-by: NJamie McAtamney <jmcatamney@pivotal.io>

* Fix crash for query with cube and having clause

We were not handling the case when a NULL outerslot was produced in agg_retrieve_direct.

* Adding test case for testing CUBE with HAVING clause
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

```
$ wget http://ftp.jaist.ac.jp/pub/apache/apr/${APR}.tar.gz
--2017-10-26 09:33:48--  http://ftp.jaist.ac.jp/pub/apache/apr/apr-1.6.2.tar.gz
Resolving ftp.jaist.ac.jp (ftp.jaist.ac.jp)... 150.65.7.130, 2001:df0:2ed:feed::feed
Connecting to ftp.jaist.ac.jp (ftp.jaist.ac.jp)|150.65.7.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-10-26 09:33:49 ERROR 404: Not Found.
```

I'm not sure we need all of these, but let's at least move them into the
main regression suite. I did leave out a few queries that were identical to
existing tests that I spotted, and merged a few that were almost identical.

When resource group is enabled, initialization of
SPIMemReserved should not depend on statement_mem.

Runaway system pick one session who consumed the most memories and
notify it to clean up once red-zone is hit, meanwhile, memories are
isolated between groups in resource group which means one query in
a resource group may cancel queries within another resource group.

To avoid this, we do not do runaway detection when resource group
is enabled, this means GUC runaway_detector_activation_percent takes
no effect under resource group.

Previously, GPDB use gp_vmem_protect_limit as the upper memory limit
that a segment can use, now in resource group, we calculate this
value automatically, so adjust the upper limit when resource group
is enabled.

Stat sender backend is used by gpperfmon to record dynamic memory usage
of segments, previously stat sender always report the memory usage as
zero because it's vmem tracker is not initialized. segmentVmemChunks is
shared by all backend, special backends like stat sender need to read it,
so it's unnecessary to check vmemTrackerInited for it.

This commit also removes the confused mem_alloc_max.

For example:

    alter tableA alter partition partiton1
    exchange partition partition1_subpartition1 with table external_table;

partition1 is the first level partition of tableA.
partition1_subpartition1 is a sub partition of partition1.

1) The routine ATPExecPartExchange in tablecmds.c searches
the target partition's table through parameters following
the key 'alter'. Then no matter what level the partition is, the
routine exchanges partition with the new table in the same way.
So exchanging sub partition with external table works well now.

2) Queries against partitioned tables that are altered to
use an external table as a leaf child partition fall back
to the legacy query optimizer. By calling routine
rel_has_external_partition in cdbpartition.c, GPORCA knows
if the table has an external partition. However, the routine
just searchs the first level partition child and it returns
false when the sub partition is an external table. Fix it in this
commit though searching every partition in table.
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

* Dump resource group related objects in pg_dumpall

This commit adds support for resource group in pg_dumpall to dump
created resource groups and the mapping info between the resource
group and roles. Meanwhile, add resource group related objects in
ICW for pg_dumpall, it just create resource groups and roles associated
with them to test if pg_dumpall and pg_upgrade can support resource
group well at the end of ICW. Not putting this in
contrib/pg_upgrade/test_gpdb_pre.sql is because any new resource
group created in ICW may make this script failed due to invalid total
cpu_rate_limit and memory_limit.

This should make the .gitignore file more aligned with the
src/test/regress/output directory.

Signed-off-by: NDivya Bhargov <dbhargov@pivotal.io>

* docs: gpreload - add notes about how utility works.

* docs - gpreload - fix typos.

After fix ec3370b0, we no longer need
different expected result files for ORCA due to the error message
difference.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Signed-off-by: NBen Christel <bchristel@pivotal.io>

Commit ec3370b0 converged the optimizer and planner behavior in terms
of constraint enforcement. This commit un-ignores the previously
generated files. We were scratching our heads when the "constraints" and
"sreh" tests were failing on our workstations because of a stale
optimizer answer file.
Signed-off-by: NTom Meyer <tmeyer@pivotal.io>

Signed-off-by: NJim Doty <jdoty@pivotal.io>

This reverts commit da4007b2 which
decreased the memory a SPI function can use, so lots of tests
failed with "insufficient memory reserved for statement".

Previously, mpp_resource_group_centos7 use bin_gpdb_centos7 directly
which may cause the inconsistence of bin_gpdb_centos7 and gpdb_src.
This commit pass through bin_gpdb_centos7 from gate_mm_misc_start to
avoid this even though it's not used by mm_misc jobs.

When resource group is enabled, initialization of
SPIMemReserved should not depend on statement_mem.

Some functions are only used in `assert()`, put these functions in
`#ifdef USE_ASSERT_CHECKING` to prevent compilation warnings.

MyProc stores resSlot instead of resSlotId now, but some merged PRs
still expect resSlotId. Updated.

* separate the ops on slot, group and self;
* use slot instead of slotId when possible;
* add slotId in slot struct;
* validate slot ownership;
* provide helper funcs for many typical operations;
* move DDL callback functions to resgroup.c;
* do not leak memory in DDL callback funcs;
* prevent wasting fixed quota due to rounding;
* remove unnecessary atomic ops;
* report original error messages in try catch block;
* begin with lower case in LOG_RESGROUP_DEBUG();
* clean up some uses of write_log();
Signed-off-by: NGang Xiong <gxiong@pivotal.io>

Dump runtime info of resgroup to help debug. Using `select value from pg_resgroup_get_status_kv('dump');` to trigger this function.

This fixes a flaky tinc test in which we fail over to a standby master, but
do not test to confirm the database is ready before querying it.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

By looking at the Concourse database, we were able to find 3 jobs in the
pr pipeline that have failed due to timing out.  We are doubling the
timeouts for those 3 jobs only.
Signed-off-by: NAlexandra Wang <lewang@pivotal.io>

In the below referenced commit, we added a new boolean
validate_reloptions to the QueryDispatchDesc. In UTILITY MODE, the
query descriptor does not have its QueryDispatchDesc initialized so
the segment would PANIC. To fix this, we default validate_reloptions
to true if the QueryDispatchDesc is not initialized as expected before
the change.

Reference:
https://github.com/greenplum-db/gpdb/commit/a3c07830258d62ae86ce9418b84ff0ca2e73e514

Authors: Jimmy Yih and Abhijit Subramanya