There were two problems with the existing NULL check. Firstly, it would
not catch expressions that returned NULL at runtime, only NULL constants.
You got a non-friendly "Unexpected internal error" error instead. Secondly,
it rejected expressions that contained NULL constants anywhere in the
expression, even if the expression as whole returned a non-NULL.

To fix both issues, add a NULL-check with a better error message to the
executor, and remove the parse-time check altogether. I copied the error
message texts from the upstream, so that you get the same error, even
though the implementation is quite different.

Add regression tests for those cases too, we apparently didn't have any.

Fixes github issue #2999.

The way ORCA was tied into the planner, running a planner_hook
was not supported in the intended way. This commit moves ORCA
into standard_planner() instead of planner() and leaves the hook
for extensions to make use of, with or without ORCA. Since the
intention with the optimizer GUC is to replace the planner in
postgres, while keeping the planning proess, this allows for
planner extensions to co-operate with that.

In order to reduce the Greenplum footprint in upstream postgres
source files for future merges, the ORCA functions are moved to
their own file.

Also adds a memaccounting class for planner hooks since they
otherwise ran in the planner scope, as well as a test for using
planner_hooks.

Both versions of the function, the one in readfast.c and readfuncs.c, were
identical. No need to define a special fast version then.

The alteration of memory_spill_ratio will take effect immediately.

In addition, remove 'SET statement_mem' from resource group
test cases, as this GUC will take no effect in resource group.

We make a copy of resource group capabilities in the shared memory, and
apply the change at the end of the ALTER transaction. In this way, we can
access shared memory content instead of catalog at the beginning of a
transaction as the visibility is the same.
Signed-off-by: NNing Yu <nyu@pivotal.io>

In some resource group test cases, the result will depend on the
number of segments, and current test cases are based on three
segments. This commit is to fix this issue.

In addition, in isolation2, gpdiff needs at least 3 dashes to identify 
the result of 'select' statement. Change the elements to be above 
100 in test case 'resgroup_memory_materialize_spill' to meet this 
requirement.

Hopefully reduces merge conflicts a bit.

Perhaps it's slightly useful, but prefer to stay in sync with upstream.

Hopefully reduce merge conflicts in the future.

gp_delete_persistent_*_node_entry() are currently only used for testing but with
asserts enabled fail as do not acquire PersistentObjLock. Since they are
modifying the persistent tables should be protected by `PersistentObjLock`.

When `gp_persistent_reset_all()` is called, gp_relation_node index is truncated
and while writing meta-page checksum calculation was missed. Ideally, following
`gp_persistent_build_all()` fixes the same and correctly builds the index and
calculates the checksum file. Better to get proper message like "ERROR: Did not
find gp_relation_node entry for relation name....." on operations after
reset_all than page verification failed.

We also refactor the code out of `CreateCheckPoint()` and make it into
its own function `GetXLogCleanUpTo()`.

Unit test added to verify the new logic works for both master and
segments.

This is hidden under #ifdef USE_SEGWALREP.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

This test checks PersistentObjLock is not acquired while holding io_in_progress
lock on a buffer. Tests will pass if PersistentObjLock is not acquired.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

Adds two new LW locks to protect filespace and tablespace hash tables
to disambiguate them from PersistentObjLock. Previously, PersistentObjLock was
overloaded to both protect these hash tables along with persistent heap tables.
If two backends try to flush the same dirty buffer, a deadlock could
potentially arise in which backend 1 holds PersistentObjLock and requests
io_in_progress lock of the buffer to be evicted.  Backend 2 holds
io_in_progress lock on the same buffer and attempts to obtain file path.
Because the file path is in hash tables protected by PersistentObjLock, backend
2 requests PersistentObjLock and blocks due to backend 1.
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Comments were copied from the add node code path and did not match their
intent. This commit repairs the language of these comments to match their
behavior.
Signed-off-by: NTaylor Vesely <tvesely@pivotal.io>

To match upstream more closely, hopefully avoiding some silly merge
conflicts in the future.

extent -> extend. And reformat.

Remove premature optimization in create_indexscan_plan, where we stripped
RestrictInfo decorations while we iterated through the list. That was
premature, because while it works, order_qual_clauses() can make use of the
information in the RestrictInfo, if it's still present. This reverts that
piece of code to the way it is in the upstream. I'm not sure if this makes
any observable difference, but let's be tidy in any case.

Also, reorder the extract_actual_clauses and order_qual_clauses calls
in create_ctescan_plan, so that order_qual_clauses is called first. For
the same reason.

I repurposed the existing psql_gpdb_du test for this. It was very small,
I think we can put tests for other \d commands in the same file. In the
passing fix a typo in a comment there, and move the expected output from
output/ to expected/, because it doesn't need any string replacements.

The \dx+ command requires pg_describe_object() function, which we don't
have in GPDB yet. Until then, treat \dx+ the same as \dx.

Note that this code is written such that \dx+ can still be used, when
connected to a PostgreSQL 9.1 or later server. Not that we particularly care
about that case, but might as well.

Per github issue #2574.

When there is \t between 'ON' and 'SEGMENT' , client can give
right error message.
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

resgroup: allow memory related alteration to take effect progressively.

memory_limit, memory_shared_quota and concurrency might be altered
concurrently when there're transactions running in the resource group,
these changes cannot take effect immediately.

Release memory 1)at the end of the 'ALTER' transaction 2)when a resource
group slot get freed.

Acquire memory at the beggning of a transaction if the resource group
dosen't have enough memory as configured.

- validate consistent checksum settings
    - Make sure checksum settings for all segments are same as master
    - Add logging proxy to allow logging to file to have different contents
      than stdout
    - Do heap checksum only for when starting up all segments.

- Add option --skip-heap-checksum-validation
    - If this option is provided to gpstart, the cluster will start up without
      checking for matching "data_checksums" GUC between master and segments.
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

We were passing an empty list and appending it instead of extending it.
using an empty list as a default value for a function parameter is an
anti-pattern.
Python behaves unintuitively, where it creates a single instance of the
variable and uses it again.
Feels like how a static variable would behave in c.
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

As we develop segment WAL replication, we need a green to green CI
pipeline that we can start small and add relative jobs from
gpdb_master as we add more functionality. This commit establishes that
pipeline and omits ICW tests that are currently not supported by
segment WAL replication.

Authors: Abhijit Subramanya and Jimmy Yih

- the params given to an invocation of gpconfig may be returned by
sys.argv as including a list, so parse appropriately.
Signed-off-by: NShoaib Lari <slari@pivotal.io>

Updating the log message to display the parameters gpconfig was called
with, both if the GUC was changed successfully or not.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>
Signed-off-by: NShoaib Lari <slari@pivotal.io>
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

This commit removes the duplicate line in the 'pg_proc.sql' which
causes OID 6065 to get used twice.
Signed-off-by: NAlex Wang <alex@awakesecurity.com>

If you have a simple insert, like "INSERT INTO foo VALUES ('bar')", we
evalute the target list (i.e. 'bar') in the master, and route the insert to
the correct partition and segment, based on the constants. However, there
was a mismatch between allConstantValuesClause(), and what its callers
assumed. The callers assumed that if allConstantValuesClause() returns
true, the target list contains only Const nodes. But in reality,
allConstantValuesClause() also returned true, if there were non-volatile
function expressions in the target list, that could be evaluated, and would
then produce a constant result.

Fix the mismatch, by making allConstantValuesClause() be more strict, so
so that it only returns true if all the entries are true Consts.

Fixes github issue #285, reported by @liruto.

It almost seems like when we merged with PostgreSQL 8.2 (sic), we missed
this one line from commit 986085a7. Before that, the rowMarks list was
a list of integers, but now it's a list of RowMarkClauses.

* Add table entries for rolcreaterexthdfs, rolcreatewexthdfs

* Change to gphdfs protocol

* typo

and its maximum value from 1000 to 10000.  ALTER TABLE SET STATISTICS
similarly now allows a value up to 10000.  Per discussion.

Cherry-pick from 65e3ea76

* Expand description of data_checksums GUC to include checksum behavior and scope.

* Mention that failed checksum verification prevents mirroring corrupted data.