Use the standard initdb mechanism for it, instead of having bespoken code
in gpinitsystem.

Also remove the unnecessarily complicated logic to generate gp_toolkit.sql
from gp_toolkit.sql.in. All the JETPACK_* variables were constants, and
there is particular need in making the gp_toolkit functions relocatable to
different schemas or prefixes, so just hardcode them and remove the sed
magic.

Rewrite the file header of gp_toolkit.sql. The legal blurp is not needed,
make the header look more like that of information_schema.sql and
system_views.sql.

Remove compaction_info.sql. It was only needed for old in-place
upgrade from 4.3.x to 4.3.4.2, which is not relevant for master branch.

Fix the "jetpack" bugbuster regression test. It was originally written to
test the gp_toolkit views, but gp_toolkit was in fact not installed in
bugbuster, and the expected output just contained a bunch of "relation
does not exist" errors. Now it actually tests gp_toolkit again. Some of
the test queries didn't produce repeateable results, due to differing oids,
differences in table sizes, and other tables created by previous tests, so
fixed those too.

This has a small user-visible difference: gp_toolkit schema is now also
installed in template0. That caused the difference in the expected outputs:
pg_regress creates the "regression" database using template0 as the
template, so gp_toolkit used to not be installed in it, but it is now.

Just some random things I happened to spot while reading code.

Compiler warned about these. Commit 50e67f10 removed the only callers
of these functions.

* Add 'const' to arguments of some functions. While we're at it, remove the
duplicate extern declaration of FaultInjector_InjectFaultIfSet from
gpdbdefs.h.

* pstrdup() constants passed to makeString(). I think the lack of copy
was harmless, but makeString() explicitly says that the caller should make
a copy, and all other callers seem to obey that, so better safe than sorry.

The hack was put in place, because prodataaccess values for these functions
are wrong in the catalogs. They should be marked as READS SQL DATA, but
they're marked as NO SQL. However, there are many more functions that are
similarly marked incorrectly, so we really should go through the pg_proc
catalog and fix all of them. But I'll leave that for another patch.

ORCA doesn't actually pay any attention to the prodataaccess field. It used
to to use it for optimization purposes at some point, but the code was taken
out because most functions are indeed mislabeled, and it was causing
problems.

This hack was subtly wrong anyway: the functions were marked as set
returning functions in the hard-coded structs, but they're not in fact
set-returning.

While we're at it, remove hardcoded OID of count() aggregate function, and
include it from pg_proc.h instead.

securitypolicy needs to be aware of classpath
added test to pljava

This code and bug was in 1f4ad703, and it
caused an "operator XXX is not a valid ordering operator" error.

The 'sortop' field in SortClause and GroupClause needs to be an ordering
operator, i.e. the "<" operator, while we use the "=" operator to represent
grouping in other places. Need to be careful to convert between the two
in right places.

Add a few tests for this in the gp_dqa test case. This was reproducible by
the existing queries, when you coerce the planner to choose sort+group
aggregates instead of hash aggregates.

User could use VARIADIC to specify parameter list when defining UDF
if they want to use variadic parameters. It is easier for user to write
only one variadic function instead of same name function with different
parameters. An example for using variadic:
create function concat(text, variadic anyarray) returns text as $$
select array_to_string($2, $1);
$$ language sql immutable strict;

select concat('%', 1, 2, 3, 4, 5);

NOTE: The variadic change set is ported from upstream of PostgreSQL:
commit 517ae403
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Thu Dec 18 18:20:35 2008 +0000

Code review for function default parameters patch.  Fix numerous problems as
per recent discussions.  In passing this also fixes a couple of bugs in
the previous variadic-parameters patch.

commit 6563e9e2
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Wed Jul 16 16:55:24 2008 +0000

Add a "provariadic" column to pg_proc to eliminate the remarkably expensive
need to deconstruct proargmodes for each pg_proc entry inspected by
FuncnameGetCandidates().  Fixes function lookup performance regression
caused by yesterday's variadic-functions patch.

In passing, make pg_proc.probin be NULL, rather than a dummy value '-',
in cases where it is not actually used for the particular type of function.
This should buy back some of the space cost of the extra column.

commit d89737d3
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Wed Jul 16 01:30:23 2008 +0000

Support "variadic" functions, which can accept a variable number of arguments
so long as all the trailing arguments are of the same (non-array) type.
The function receives them as a single array argument (which is why they
have to all be the same type).

It might be useful to extend this facility to aggregates, but this patch
doesn't do that.

This patch imposes a noticeable slowdown on function lookup --- a follow-on
patch will fix that by adding a redundant column to pg_proc.

Conflicts:
	src/backend/gpopt/gpdbwrappers.cpp

SET command is session effective, all existed idle gangs should be set for later reuse,
but for busy gangs declared by cursors, errors occur if they receive a set command. Way
to fix it is marking busy gangs to no reuse so they can be destroyed after cursors been
closed.

Heikki suggested that we backport atomic operation from PostgreSQL 9.5 to
fix this

    commit b64d92f1
    Author: Andres Freund <andres@anarazel.de>
    Date:   Thu Sep 25 23:49:05 2014 +0200

    Add a basic atomic ops API abstracting away platform/architecture details.

    Several upcoming performance/scalability improvements require atomic
    operations. This new API avoids the need to splatter compiler and
    architecture dependent code over all the locations employing atomic
    ops.

    For several of the potential usages it'd be problematic to maintain
    both, a atomics using implementation and one using spinlocks or
    similar. In all likelihood one of the implementations would not get
    tested regularly under concurrency. To avoid that scenario the new API
    provides a automatic fallback of atomic operations to spinlocks. All
    properties of atomic operations are maintained. This fallback -
    obviously - isn't as fast as just using atomic ops, but it's not bad
    either. For one of the future users the atomics ontop spinlocks
    implementation was actually slightly faster than the old purely
    spinlock using implementation. That's important because it reduces the
    fear of regressing older platforms when improving the scalability for
    new ones.

    The API, loosely modeled after the C11 atomics support, currently
    provides 'atomic flags' and 32 bit unsigned integers. If the platform
    efficiently supports atomic 64 bit unsigned integers those are also
    provided.

    To implement atomics support for a platform/architecture/compiler for
    a type of atomics 32bit compare and exchange needs to be
    implemented. If available and more efficient native support for flags,
    32 bit atomic addition, and corresponding 64 bit operations may also
    be provided. Additional useful atomic operations are implemented
    generically ontop of these.

    The implementation for various versions of gcc, msvc and sun studio have
    been tested. Additional existing stub implementations for
    * Intel icc
    * HUPX acc
    * IBM xlc
    are included but have never been tested. These will likely require
    fixes based on buildfarm and user feedback.

    As atomic operations also require barriers for some operations the
    existing barrier support has been moved into the atomics code.

    Author: Andres Freund with contributions from Oskari Saarenmaa
    Reviewed-By: Amit Kapila, Robert Haas, Heikki Linnakangas and Álvaro Herrera
    Discussion: CA+TgmoYBW+ux5-8Ja=Mcyuy8=VXAnVRHp3Kess6Pn3DMXAPAEA@mail.gmail.com,
        20131015123303.GH5300@awork2.anarazel.de,
        20131028205522.GI20248@awork2.anarazel.de

This bug was introduced in merge commit 1f4ad703, by a mishap in merging
the upstream changes to convert_IN_to_join. The constructed InInfoClause
entry was accidentally added to the list of in-clauses twice. That was
usually harmless, but if the in-clause was later pulled up from a subquery,
the code adjusts the varnos while merging the range tables of the subquery
and its parent, ran twice. In doing so, it adjusted the varno twice,
pointing it to wrong or non-existing range table entry.

ORCA uses its own metadata cache to store information about relations,
operators etc. Currently, we always reset the cache when planning a query,
unless the optimizer_release_mdcache GUC is turned off, which is slow.

To make it safe to turn optimizer_release_mdcache off, use the catalog
cache invalidation mechanism to still reset the cache when there are
changes to the catalogs that affect the metadata cache.

The ORCA-facing interface of this is the same as in the previous attempt:
A function that returns true/false indicating whether there has been any
catalog changes whatsoever since last call.

This reverts commit 6c31a3b4. Per
discussion, we will implement the same functionality in a simpler way.

Fix the fake-tidycat definitions of OPFAMILYOID and OPFAMILYAMNAMENSP
syscaches. Remove reference to INHRELID - that syscache removed by one of
the merged upstream 8.3 patches. Re-run tidycat.pl, to correct the "last"
id number in syscache.h.

Added a generation-based Metadata Versioning mechanism
which will be used by ORCA to cache and invalidate catalog
data in its Metadata Cache.
Versioning is disabled by default at this point, until the
Metadata Cache eviction policy is completed.

This patch replaces all occurrences of pgsql-bugs@postgresql.org with
bugs@greenplum.org. Pointers to the pgsql-bugs mailinglist in the code
remain as they re. Regression and BugBuster tests are also patched.

These were renamed to 'typname' in GPDB a long time ago because 'typename'
is a reserved keyword in C++. That's a valid concern for header files that
need to be included in C++ code (like src/backend/gpopt), but it's not
necessary for local variables that appear in .c files, as long as we're
careful to compile those files as C code, not C++. So rename local
variables back to 'typename', to reduce diff vs. upstream and make merging
slightly easier.

In the upstream header files, the 'typename' fields were renamed to
'typeName', in a later PostgreSQL version. In GPDB, we've renamed them
to 'typname' instead. Would be good to adopt the upstream naming at some
point, but not in this patch.

This commit eliminates the global new/delete overrides that were causing
compatibility problems (the Allocators.(h/cpp/inl) files have been
completely removed). The GPOS `New()` macro is retained and works the
same way, but has been renamed `GPOS_NEW()` to avoid confusion and
possible name collisions. `GPOS_NEW()` works only for allocating
singleton objects. For allocating arrays, `GPOS_NEW_ARRAY()` is
provided. Because we no longer override the global delete,
objects/arrays allocated by `GPOS_NEW()` and `GPOS_NEW_ARRAY()` must now
be deleted by the new functions `GPOS_DELETE()` and
`GPOS_DELETE_ARRAY()` respectively. All code in GPOS has been
retrofitted for these changes, but Orca and other code that depends on
GPOS should also be changed.

Note that `GPOS_NEW()` and `GPOS_NEW_ARRAY()` should both be
exception-safe and not leak memory when a constructor throws.

Closes #166

As promised in previous commit. Upstream patch:

commit bd0a2609
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Fri Jun 1 19:38:07 2007 +0000

    Make CREATE/DROP/RENAME DATABASE wait a little bit to see if other backends
    will exit before failing because of conflicting DB usage.  Per discussion,
    this seems a good idea to help mask the fact that backend exit takes nonzero
    time.  Remove a couple of thereby-obsoleted sleeps in contrib and PL
    regression test sequences.

Instead of creating a mock version of every backend object file, only
create mock files as needed by individual tests. Speeds up the build, and
is nicer anyway.

This inverts the logic of the <testname>_REAL_OBJS mechanism. We now link
with the real version of each object file, *except* those listed in
<testname>_MOCK_OBJS variable.

Some tests were already doing this, but it seems nicer to default to using
the real memory manager for all tests. An individual test can still opt-out
by listing the relevant files in <testname>_MOCK_OBJS. Some tests had had
to jump through hoops to mock the memory manager, which is no longer
necessary.

There are a bunch of backend files that are never mocked, but always linked
into test program as is. List those in Makefile.mock, so that every Makefile
doesn't need to list them. Makes the makefiles simpler, and you need less
boilerplate code when adding a new test program.

Not all of these backend files were listed in the _REAL_OBJS variable of
every test, but in those tests that they were not, they were also not used,
so it doesn't matter if we use the real or mock version.

This way we don't need to link the whole backend into client mock programs.
We've gotten away with it so far, but it would cause clashes if one of the
client programs contained a function or global variable with same name as
in the backend. And of course it's completely unnecessary anyway.

Seems cleaner.

Rearrange code so that we can keep indentation etc. identical to upstream.
Makes merging easier.

auxiliary tables to not get shrunk and generate a notice to the user.

The AppendOnlyCompaction_IsRelationEmpty() function incorrectly assumed that
the column number for tupcount column was the same in pg_aoseg and pg_aocsseg
tables. This cause it to incorrectly return true even when the CO relation was
not empty. This method is used in vacuum to determine if the auxiliary
relations need to be vacuumed. Due to the bug, vacuum would update the
pg_aocsseg relation and vacuum it within the same transaction and hence
generate the NOTICE that it can't shrink the relation because transaction is
already in progress and would not shrink the relation.

Also make sure that we do a vacuum on the auxiliary relations only in two cases :-
1. Vacuum cleanup phase
2. Relation is empty and we are in prepare phase
Otherwise we will end up with the same issue above if some of the segments have
zero rows

pathnode_copy_node() was dead.

The freeze_outer_path argument was always passed as false. Remove it.

Make some functions that are only called within the same .c file static.

And revert some whitespace changes vs. upstream. All of these changes
reduce the diff against upstream slightly, reducing merge conflicts in the
future.

They belong in headers. Otherwise, if the function's signature changes, the
compiler will not warn you.

Makes merging easier.

It's no longer needed, as Params carry a typmod now.