Commit a7de4d60 added a test view, which is causing trouble for the
binary swap test. The binary swap test runs the regression tests using
the new binaries, but then swaps out an old binary, and runs pg_dump
against the regression database. That fails with this test view, because
the old version still has the bug that commit a7de4d60 fixed, and
cannot parse the view definition stored in the catalogs correctly.

So, the code is fine, and we are still binary-compatible. To silence the
false failure in the binary swap test, drop the test view after the test,
so that it won't be present in the regression database, when the binaries
are swapped.

Analysis by Jesse Zhang. See discssuon on gpdb-dev mailing list.

Discussion: https://groups.google.com/a/greenplum.org/forum/#!topic/gpdb-dev/UJ3U_yqs38A

Signed-off-by: NJohn Gaskin <Quikling@pivotal.io>

We had replaced the upstream code in readfuncs.c that checks what kind of
a Node we're reading, with a seemingly smarter binary search. However, that's
a premature optimization. Firstly, the linear search is pretty darn fast,
because after compiler optimizations, it will check for the string length
first. Secondly, the binary search implementation required an extra
palloc+pfree, which is expensive enough that it almost surely destroys any
performance gain from using a binary search. Thirdly, this isn't a very
performance-sensitive codepath anyway. This is used e.g. to read view
definitions from the catalog, which doesn't happen very often. The
serialization code used when dispatching a query from QD to QEs is a more
hot codepath, but that path uses the different method, in readfast.c.

So, revert the code the way it is in the upstream. This hopefully reduces
merge conflicts in the future.

Also, there was in fact a silly bug in the old implementation. It used
wrong identifier string for the RowCompare expression. Because of that, if
you tried to use a row comparison in a view, you got an error. Fix that,
and also add a regression test for it.

Commit 0be2e5b0 added a regression test to check that if a view contains
an ORDER BY, that ORDER BY is obeyed on selecting from the view. However,
it forgot to add the test to the schedule, so it was never run. Add it.

There is actually a problem with the test as it is written: gpdiff masks
out the differences in the row order, so this test won't catch the problem
that it was originally written for. Nevertheless, seems better to enable
the test and run, than not run it at all. But add a comment to point that
out.

The assertion didn't take into account that malloc() might return NULL.

When the new OID-dispatching mechanism was introduced (commit f9016da2),
a lot of FIXME comments were left in calls to CdbDispatchUtilityStatement.
These were places where I wasn't sure if the command that was executed
might create new objects, with new OIDs assigned to them, that would need
to be dispatched to the QE. I've now skimmed through the call sites, and
checked that they are all for ALTER or DROP commands that should not create
new OIDs, so remove the FIXME comments.

In theory, there could be an extension out there that uses it, but I doubt
it. And if there is, the fix is simple: just use the regular PG_MODULE_MAGIC
macro instead.

The duplicate was introduced by the 8.3 merge, because we had already
cherry-picked this in commit d481bd72. Harmless, but let's be tidy.

This is also in array.h.

I think this is some kind of a debugging function, but since it's not
exposed in pg_proc.h, there's no easy way to use it. Seems better to just
remove it, than try to expose it.

Also, mark transformIndexConstraints() function as static, as it is in
the upstream. It's only used within the same file.

This will slightly reduce the merge conflicts with 8.4.

In order to test this, modify the gpdiff tool to not suppress duplicate
WARNINGs, if the duplicates don't have a segment number attached to them.
The bug was easy to see by setting work_mem, which emits a WARNING in
GPDB, except that suppressing the duplicates hid the bug. Even though we
have a few tests that set work_mem, add one that does that explicitly for
the purpose of testing for this bug, with a comment explaining its purpose.

While we're at it, move the GPDB-specific test queries out of 'guc' test,
into a new 'guc_gp' test, to keep the upstream 'guc' test pristine.

Fixes github issue #3008, reported by @guofengrichard.

GUC 'memory_spill_ratio' can be set at multiple levels,
in particular at resource group level and session level,
and session would override resource group.

When setting GUC 'memory_spill_ratio' at session level, the
semantic validation will not be checked until it is referred
by further queries.

    commit c5d2bc31
    Author: Haozhou Wang <hawang@pivotal.io>
    Date:   Mon Jan 2 22:37:20 2017 -0500

        Support 'OPTIONS' keyword in GPDB external table

        In case user wanna specify parameters with OPTIONS other than config
        file in LOCATION, it provides flexible and friendly grammar.

        Usage:
        CREATE EXTERNAL TABLE xxx LOCATION xxx FORMAT xxx
        [OPTIONS ( [option_key 'option_value' [, ...]] )]
        xxx;

During gp_expand, the guc gp_before_filespace_setup is used to skip
mmxlog and ptas information from the checkpoint record. However, the
UnpackCheckPointRecord() was expecting additional information.

The fix is to make that part optional based on the remaining length of
checkpoint record.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Hopefully avoids a couple of merge conflicts in the future.

Setting auxiliary processes as GP_ROLE_UTILITY actually works fine
until a standby master is initialized and started.  The standby master
can go into certain redo paths such as redoDistributedCommitRecord
which will exit early if the Gp_role is GP_ROLE_UTILITY.  Having the
Gp_role check in these redo paths is not correct and will be revisited
for refactor some other day.

Commit context:
https://github.com/greenplum-db/gpdb/commit/6810843fe1c2565dd4f6225ca2838c182b03ca72

The "if (buf->len != field_size)" block in make_val_with_blanks() function
is unreachable. The code that precedes it has just checked that buf->len
isn't larger than field_size, and if it was smaller, it was padded. Remove
it, and refactor a bit for clarity.

Currently, auxiliary and backend processes are defaulted to
GP_ROLE_DISPATCH.  This was kind of harmless until we started enabling
segment WAL replication.  The WAL sender would be initialized as a
backend process that would have a gp_session_id set and mppIsWriter be
True.  This would intermittently cause issues with cursors.  The writer
would open a temporary file with its session id and process id but the
reader would construct the temporary filename incorrectly with the
right session id but the WAL sender's process id.  The reader would
then fail to open the correct temporary file.

To fix this, we set Gp_role to GP_ROLE_UTILITY for all auxiliary
processes and make exceptions for certain backend processes
(Autovacuum and WAL sender).

Add MM_gpinitstandby as part of release candidate

Follow up commit on 51a1cad7

Signed-off-by: NShoaib Lari <slari@pivotal.io>

The data_checksums GUC setting should be the same as the master.

The existing test for gpinitstandby is modified to run on a single host.
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

* Add log_temp_files guc

* Add subnav log_temp_files to ditamap

* Add mention of spill files and pg_log

The only user-visible effect is that the "non integer constant in %s" error
message now gets an error location.

This is from upstream commit 517ae403. We had already backported most of
that commit, so let's take it to completion.

Because Greenplum uses persistent tables (PT), if `gp_disable_tuple_hints` is
false, then reading PT tables during startup will cause hint bit
change, and attempt to mark page dirty, which will try to create a
Full Page Image in XLOG.

Because we are in init processing, write to XLOG is disallowed. We must
protect against the XLOG write.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>
Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>

* resource groups - sql ref pages - create new, updates to others

* david s requested edits

* more param description updates

* cannot submit RG DDL cmds in explicit transaction

* can now alter memory limit and shared; syntax corrections

* create RG - clarify MEMORY_SHARED_QUOTA and MEMORY_SPILL_RATIO descripts

* small edits requested by simon

* clarify default group usage

* remove reference to rg name NONE

* docs - add some resource group GUCs to reference guide

* incoporate david s requested edits

* remove resource_scheduler guc, edits from simon

Commit f51f2f57 backported upstream commit caddcb8f
but missed one destroyPQExpBuffer() call in the merge. Add to avoid
a tiny leak in the early exit path and to align us with upstream.

If src is null, or if strdup() fails, then srcdup will be null and
so will srcdup_start as it's a pointer to srcdup.  Regardless of
which, we can limit us to test srcdup to catch both error cases.
Also, avoid doublefree by not freeing srcdup_start.

getaddrinfo() returns a set of addrinfo structs, which needs to be
freed again with freeaddrinfo. Ensure to return resources before we
leave socket setup.

The loop failed to close the opened file descriptors causing a
small leak. Clean up as we go along.

We want to set toFile to point to the memory allocated by Safe_strdup,
not change toFile.