There are quite a few places in the planner that need that information.
Save a few system table lookups by keeping it in RelOptInfo. There is
no syscache for pg_exttable. I don't see any measurable performance
improvement from this, but you can see the number of index scans drop in
pg_stat_sys_tables, and I think it makes the code slightly simpler anyway.

This shaves a couple of minutes off the total runtime on my laptop.

Keeping upstream test files as close to upstream as possible helps with
diffing and merging with upstream. There are more GPDB-specific changes
there still, but this is a step in the right direction. The 'alter_table'
test runs for quite a long time, so by running it more in parallel, this
also shortens a installcheck-good run slightly.

The test runs in a schema of its own, bfv_aggregate. It is created at
the beginning of the test, so we can safely assume that it's empty.
Also, rather than dropping created objects after each test, just drop
the whole schema at the end.

In the passing, also add a DROP TABLE for the boolean test table. It
was missing. Not that we necessarily need to drop every table created
in a test case, but let's be consistent within the test file.

Vacuuming the single table ought to be enough.

It's pretty much identical to Form_pg_appendonly, which is now available
in the relcache. No need to pass this additional struct around.

This way, you don't need to always fetch it from the system catalogs,
which makes things simpler, and is marginally faster too.

To make all the fields in pg_appendonly accessible by direct access to
the Form_pg_appendonly struct, change 'compresstype' field from text to
name. "No compression" is now represented by an empty string, rather than
NULL. I hope there are no applications out there that will get confused
by this.

The GetAppendOnlyEntry() function used to take a Snapshot as argument,
but that seems unnecessary. The data in pg_appendonly doesn't change for
a table after it's been created. Except when it's ALTERed, or rewritten
by TRUNCATE or CLUSTER, but those operations invalidate the relcache, and
we're never interested in the old version.

There's not much need for the AppendOnlyEntry struct and the
GetAppendOnlyEntry() function anymore; you can just as easily just access
the Form_pg_appendonly struct directly. I'll remove that as a separate
commit, though, to keep this one more readable.

Many headers in src/include/catalog have gotten this treatment in the
upstream. This makes life easier for the next commit, which adds a
reference to Form_pg_appendonly to RelationData. With this separation,
we can avoid pulling in a lot of other headers into rel.h, and avoid a
circular dependency,

In the previous coding, if a "name" field in a struct, that was used for
direct access to a catalog tuple, i.e. Form_pg_*, was not accidentally
aligned to 4-byte boundary, the struct would not match what's stored on
disk. I was bit by this, when I tried changing the compresstype field
in pg_appendonly from text to name.

This seems like a good idea from an upgrade point of view too. This change
broke pg_upgrade for any tables that use the name datatype. That's not
a big deal, because it shouldn't be used in user tables at all anyway, but
might as well take the hit in 5.0 when the upgrade is expected to be more
painful anyway.

Original commit:

commit 5f6f840e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Tue Jun 24 17:58:27 2008 +0000

    Reduce the alignment requirement of type "name" from int to char, and arrange
    to suppress zero-padding of "name" entries in indexes.

    The alignment change is unlikely to save any space, but it is really needed
    anyway to make the world safe for our widespread practice of passing plain
    old C strings to functions that are declared as taking Name.  In the previous
    coding, the C compiler was entitled to assume that a Name pointer was
    word-aligned; but we were failing to guarantee that.  I think the reason
    we'd not seen failures is that usually the only thing that gets done with
    such a pointer is strcmp(), which is hard to optimize in a way that exploits
    word-alignment.  Still, some enterprising compiler guy will probably think
    of a way eventually, or we might change our code in a way that exposes
    more-obvious optimization opportunities.

    The padding change is accomplished in one-liner fashion by declaring the
    "name" index opclasses to use storage type "cstring" in pg_opclass.h.
    Normally btree and hash don't allow a nondefault storage type, because they
    don't have any provisions for converting the input datum to another type.
    However, because name and cstring are effectively the same thing except for
    padding, no conversion is needed --- we only need index_form_tuple() to treat
    the datum as being cstring not name, and this is sufficient.  This seems to
    make for about a one-third reduction in the typical sizes of system catalog
    indexes that involve "name" columns, of which we have many.

    These two changes are only weakly related, but the alignment change makes
    me feel safer that the padding change won't introduce problems, so I'm
    committing them together.

They were not used for anything. Look at segrelid and segidxid in
pg_appendonly instead.

Spotted while debugging a buggy patch of mine, that the relation names
in the ReadBuffer() error messages currently come out as "(null)". Would
segfault on with a different libc implementation.

Running gpcheckcat after constraints installcheck test shows trigger metadata
inconsistencies between the master and segments. This issue was introduced
in the 8.3 merge. If CreateTrigger Statement has a trigger oid already, we
should reuse it like before the 8.3 merge.

This patch series adds the JSON datatype, with all the fixes that went
into PostgreSQL 9.2 after the initial commit. See individual commits for
more details.

Author: Álvaro Hernández Tortosa <aht@8kdata.com>
Reviewed-by: Daniel Gustafsson
Reviewed-by: Andreas Scherbaum

Note: We don't alien node elimination when there is an explain codegen
or explain analyze codegen query in master.

Signed-off-by: NShreedhar Hardikar <shardikar@gmail.com>

This field was added to Const struct and the text out/read functions in
PostgreSQL 8.3 (upstream commit 0f4ff460). In the merge, we missed that
it also needs to be handled in the "fast" out/read functions.

I wasn't able to come up with a test case that would misbehave because of
this. When a Const is dispatched from the master to segments, the typmod
has already been "applied" to the value, e.g. a numeric has been rounded
to the desired precision. So it doesn't matter that the typmod is not
available in the segments. However, gpcheckcat complained about the
mismatch, in index expressions stored between the master and segments, as
well as in atttypmod stored for a view. And if we leave the typmod
uninitialized, it's left to 0, rather than -1 which would mean "no typmod",
so clearly this must be fixed, even if it doesn't have any user-visible
effects at the moment.

Reported by Jimmy.

Original implementation used array_ref for each element of the array. In case if array
contains nulls, each call to the function array_ref causes a cycle over nulls bitmap to
calculate the data offset. For array with 100'000 elements this causes 350x lower performance
This fix uses the code from array_out for effective array data traversal

Fixing the issue with PL/Python "subtransaction" regression tests - specifying order for table selects, and adding more verbose output of SPI errors

* We should ignore pg_temp_% during backup even if the user specifies
  the table. There is a potential error if we try to backup pg_temp_%,
  so we will prevent it.

* We noticed that when gpcrondump is given with a -S option (exclude schema)
  gpcrondump will also try to dump pg_temp schemas.
  This can be detected by the following issue:
  user creates a new schema
  user creates a temp table
  -- Keep the session alive

  On another session:
  Run gpcrondump -S <exclude schema> -x <dbname>

* We've filtered out pg_temp in our queries to prevent this issue.

Dumping pg_temp schemas doesn't make sense as gpdbrestore can't restore
those schemas and tables anyways, since pg_ and gp_ schemas are
restricted for the system.

Make sure to ignore pg_temp from backup.

* We want to make sure that we ignore pg_temp and not error out if we
  can't find the pg_temp tables during a backup

Authors: Marbin Tan, Karen Huddleston, & Chumki Roy

This GUC is disabled by default.

Currently, when users run an `UNION ALL` query, such as
```sql
SELECT a FROM foo UNION ALL SELECT b FROM bar
```,
GPDB will parallelize the execution across all the segments, but the
table scans on `foo` and `bar` are not executed parallel. With this GUC
enabled, we add redistribution motion node under APPEND(UNION) operator,
which makes all the children of APPEND(UNION) operator execute in
parallel in every segment.

If Agg has an Append as the outer child, group by cannot resolve the variable names of
the group by expression. This is because of incorrect assumption that it can take outerPlan
of any child to access the outer child of its child, which is not true for Append. For
Append we keep a list of children and lefttee and righttree are set to NULL.

This breaks bfv_partition test suite when optimizer is enabled. For repro:

```
drop table if exists t;
drop table if exists p1;
drop table if exists p2;
-- end_ignore

create table p1 (a int, b int) partition by range(b) (start (1) end(100) every (20));
create table p2 (a int, b int) partition by range(b) (start (1) end(100) every (20));
create table t(a int, b int);

insert into t select g, g*10 from generate_series(1,100) g;
insert into p1 select g, g%99 +1 from generate_series(1,10000) g;
insert into p2 select g, g%99 +1 from generate_series(1,10000) g;

analyze t;
analyze p1;
analyze p2;

explain select * from (select * from p1 union select * from p2) as p_all, t where p_all.b=t.b;
```

This commit fixes this by considering Append as special child and properly extracting the first
subplan under Append as its outer child.

Looks like it can also come under NestedLoop.

This came as part of the PR #980 comment to remove compiler warning to cast the parentPlan
of PartitionSelector to Sequence, and it failed an Assert check that parentPlan is not
always a Sequence.

In this PR we revert the earlier fix in #840, but keep some part of it. This
PR is similar in spirit to the earlier as it depends on parent of
PartitionSelector (which is expected to be Sequence) to deparse printablePredicate.
However, this PR resolves the parent from child, instead of resolving child from
parent and localizes the changes to just explain.c. We also keep the old behavior
of clearly isolating Sequence node from PartitionSelector node.

Besides this, we also pass the parent to resolve both outer and inner varno of a
PartitionSelector's printablePredicate, which is also necessary to make bfv_partition green.

These are minor fixes identified through static code analysis.

Add unit tests to gprestore filter and restore.

Authors: Chumki Roy and Chris Hajas

cdbdisp_dispatchUtilityStatement is designed to dispatch a utility statement synchronously used by
DefineIndex(), DefineExternalRelation(), DefineRelation() and createdb(). The formor three function
actually use it in asynchronous way, so replace it with CdbDispatchUtilityStatement.

createdb() use it synchronously, but the performance improvement is little. To make code clean, abandon
this function and replace it with asynchronous version.

Original cdbdisp_dispatchRMCommand() and CdbDoCommand() is easy confusing. This commit combine
them to one and meanwhile push down error handling to make coding easier.

refactor CdbDispatchUtilityStatement() to make it flexible for cdbCopyStart(),
dispatchVacuum() to call directly. Introduce flags like DF_NEED_TWO_SNAPSHOT,
DF_WITH_SNAPSHOT, DF_CANCEL_ON_ERROR to make function call much clearer

state.

There are two cases leading to this self-deadlock:
(1) SIGALRM for IdleSessionGangTimeout comes when QD is in malloc function call
of SSL code for example, and the handler HandleClientWaitTimeout would call
function free to destroy Gangs, hence we are calling free inside malloc, which
would produce a deadlock;
(2) If a SIGUSR1 come when we are inside HandleClientWaitTimeout and calling
function free, then we would be interrupted to process the Catchup event first,
in which we would possibly call malloc, hence we are calling malloc inside free,
which would cause a deadlock also.

To fix this issue, for case 1, we only enable SIGALRM handling of IdleSessionGangTimeout
exactly before recv call, which can protect malloc/free from being interrupted
by this kind of SIGALRM; for case 2, we prevent reentrant signal handling.

This fix is mainly borrowed from a patch of Tao Ma in Apache HAWK project.

The compiler rightly pointed out that this memset() call was not zeroing
the whole struct, as it was clearly intended to. 'dd_options' is a pointer,
so sizeof(dd_options) is only 4 or 8, depending on architecture. That was
harmless, because the caller just zeroed out the array, but let's silence
the compiler warning. Zeroing it twice is not necessary, but this is not
at all performance-critical, so I don't want to second-guess the intentions
of the original coder right now.

It was broken by the removal of int to text cast, as part of the PostgreSQL
8.3 merge.

The proprietary RSA BSAFE library was used for one thing only: for storing
the "lockbox" file containing the credentials for DDBoost. Replace with
a little hand-written key-value configuration file format, with password
obfuscation based on XOR, rand() and base64 encoding. This allows us to
finally remove the dependency to the BSAFE library.

The configuration file is now in a human-readable and -editable format
(except for the password), which is a bonus.