Signed-off-by: NShreedhar Hardikar <shardikar@gmail.com>

This closes #836
Signed-off-by: NMarc Spehlmann <cramja@gmail.com>

reorder header file

This closes #858

Currently gp_connections_per_thread=0 means using main thread to
do dispatch instead of creating new threads. It's not working if
segments report error before sending back any data.
For example:
create table foo(id int);
insert into foo values(0);
set gp_connections_per_thread=0;
select 100/id from foo;

We disable it for now and will reenable it when we have the new
implementation of non-threaded dispatcher.

No functional changes are included in this diff, only trailing
whitespace cleanup.

The agebracket.yml mapred test case is testing the reduction with
a simple reducer summing the values. GPMapred has a builtin SUM
reducer however which will yield the same result, extend the test
with a copy of the agebracket.yml test using the builtin ensuring
that the output is identical.

I was alerted to this by a compiler warning, which this also fixes. Was
broken by the 8.3 merge.

We had backported similar changes to the other pg_*_is_visible()
functions earlier, so for consistency, do the same for the new functions
we got from the 8.3 merge.

The related upstream commit was:

commit 66bb74db
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Mon Dec 15 18:09:41 2008 +0000

    Arrange for the pg_foo_is_visible and has_foo_privilege families of functions
    to return NULL, instead of erroring out, if the target object is specified by
    OID and we can't find that OID in the catalogs.  Since these functions operate
    internally on SnapshotNow rules, there is a race condition when using them
    in user queries: the query's MVCC snapshot might "see" a catalog row that's
    already committed dead, leading to a failure when the inquiry function is
    applied.  Returning NULL should generally provide more convenient behavior.
    This issue has been complained of before, and in particular we are now seeing
    it in the regression tests due to another recent patch.

This test was introduced in commit c36ecd6f.
It does not actually test the changes of that commit, instead it was aiming to test some other scenarios.
It seems that the test does not have a deterministic behavior.
We remove it for now, and we will investigate if the scenarios that we were aiming to test are possible and write deterministic tests.

The recent hacks to make xlogdump work with the 8.3 merge had a
thinko where the initialization of the struct ended up inside the
guards for version checks effectively breaking it for < 9.0.

Spotted (and reviewed) by Heikki Linnakangas

It doesn't seem too useful, to hae a DEBUG1 of tuple counts. I doubt
anyone has taken advantage of that in years. Removing it allows us
to refactor the code inside GetPlannedStmtLogLevel() back into
GetCommandLogLevel(), like it is in the upstream.

There are no callers to it outside the file, so why not. Reduces the
diff vs. upstream.

This seems like a completely unnecessary change. Reduce the diff vs.
upstream.

Not sure what this was used for or when, but it's dead now.

Removed maxlen as a maximum size for the custom format supporting fixed
format descriptions. Allows for the creation of tables having up to
MaxHeapAttributeNumber of columns.

We replace unused init plans with dummy Result plans at end of planning.
Use an empty target list for the dummy Result, instead the target list of
the original Plan we're replacing. An empty target list is obviously better
from a performance point of view, but more importantly, if the original
target list contained an AggRef node, initializing the AggRef expression
at ExecutorStart() will fail, because there is a check that an AggRef
can only appear in an Agg node.

Per Nikhil Kak's report. gpcheckcat contains a query that tripped on this.

Now that PostgreSQL 8.3 has been merged we can enable the 8.3 specific
parts of xlogdump. There was a bug in the upstream code however that
included calls using struct members not available until 9.0 (coming in
commit 375369ac), disconnect this in the typical if/endif way. Since
xlogdump is abandonware due to the inclusion of pg_xlogdump in 9.3
theres not much use in submitting this upstream since we effectively
maintain this code.

The main effect of the readahead code was not readahead as such, but to
avoid trashing the buffer cache when doing a sequential scan of a large
table. We just got similar functionality from PostgreSQL 8.3: the "ring
buffer" feature, in commit d526575f. So we don't need the GPDB-specific
code anymore.

This was slightly mismerged: we were reading the same page twice in a row:
once with the new ReadBufferWithStrategy() call and second time with the
old KillAndReadBuffer() call. That is now fixed too.

The other FIXME comment that the removed comment refers to was reverted
earlier, before we pushed the 8.3 merge. Should've uncommented this back
then, but I missed the need for that because I've been building with
--enable-cassert. This fixes the regression failure on gpdtm_plpgsql test
case, when built without assertions.

Commit 5dc7dda9 added a forward declaration for PartitionSelector and
PartitionSelectorState, the redefenition of which is only legal in
C11 which we are yet to use.

+1 from hlinnakangas@

These queries were backported from later PostgreSQL releases earlier,
but had to be disabled when we started the 8.3 merge, because we didn't
have all the catalog changes from 8.3 yet. Now we do, so re-enable the
queries.

The PostgreSQL 8.3 merge included a band-aid fix for this, but since we
haven't yet made a GPDB 5.0 release, and are therefore free to still modify
the catalogs, let's apply the proper fix that went into PostgreSQL 9.3 at
the time.

commit 71627f3d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Wed Feb 13 16:20:01 2013 -0500

    Fix CVE-2013-0255 properly.

    Revert commit ab0f7b60 (in HEAD only)
    in favor of the proper solution, which is to declare enum_recv() correctly
    in the system catalogs.  It should be declared to take type "internal"
    not "cstring".

    Also improve the type_sanity regression test, which should have caught
    this typo, so that it actually would.  Most of the relevant checks on
    the signature of type I/O functions should not have been restricted to
    basetypes/pseudotypes, as they should apply to any type's I/O functions.

Everything in PostgreSQL 8.3 is now in Greenplum. PostgreSQL 8.3 has been
end-of-lifed in upstream, so there will be no new upstream minor releases
of this series anymore either.

This includes a lot of new features from PostgreSQL 8.3, and a ton of bug
fixes. See upstream release notes for details. One big user-visible change
included is the removal of implicit cast from other datatypes to text. When
this was done in PostgreSQL, it caused a lot of sloppily written
application queries to break. Which is a good thing in the long run, but it
caused a lot of pain on upgrade.

A few features work slightly differently in GPDB:

* Lazy XIDs feature in upstream reduces XID consumption, by not assigning
  XIDs to read-only transactions. That optimization has not been
  implemented for GPDB distributed transactions, however, so practically
  all queries in GPDB still consume XIDs, whether they're read-only or not.

* temp_tablespaces GUC was added in upstream, but it has been disabled in
  GPDB, in favor of GPDB-specific system to decide which filespace to use
  for temporary files.

* B-tree indexes can now be used to answer "col IS NULL" queries. That
  support has not been implemented for bitmap indexes.

* Support was added for "top N" sorting, speeding up queries with ORDER BY
  and LIMIT. That support was not implemented for tuplesort_mk.c, however,
  so you only get that benefit with enable_mk_sort=off.

* Multi-worker autovacuum support was merged, but autovacuum as whole is
  still disabled in GPDB.

* Planner hook was added. Nothing special there, but we should consider
  refactoring the ORCA glue code to use it.

* Plan cache invalidation was added upstream, which made the
  gp_plpgsql_clear_cache_always option obsolete. We also backported the
  further invalidation improvements from 8.4. See below for more
  information.

In addition to those, there were some big refactorings that were not that
interesting from user point of view, but caused a lot of code churn:
InitPlans are now initialized separately at executor startup, the executor
range table was changed to be "flat", and code related to parse analysis of
DDL commands was moved around. The way UPDATE WHERE CURRENT OF was
implemented in GPDB was refactored to match the new upstream support for
the same (see below for more information).

Plan cache invalidation
-----------------------

One notable feature included in this merge is the support for plan cache
invalidation. GPDB contained a hack to invalidate all plans in master
between transactions, for the same purpose, but the proper plan cache
invalidation is a much better solution. However, because the GPDB hack also
handled dropped/recreated functions in some cases, if we just leave it out
and replace with the 8.3 plan cache invalidation, there will be a
 regression in those cases. To fix, this merge commit includes a backport
the rest of the plan cache invalidation support from PostgreSQL 8.4, which
 handles functions and operators too. The upstream commit for that was:

commit ee33b95d
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Tue Sep 9 18:58:09 2008 +0000

    Improve the plan cache invalidation mechanism to make it invalidate plans
    when user-defined functions used in a plan are modified.  Also invalidate
    plans when schemas, operators, or operator classes are modified; but for these
    cases we just invalidate everything rather than tracking exact dependencies,
    since these types of objects seldom change in a production database.

    Tom Lane; loosely based on a patch by Martin Pihlak.

As a result of that, the gp_plpgsql_clear_cache_always GUC is removed, as
it's no longer needed. This also includes new test cases in the plpgsql
cache regression test, to demonstrate the improvements.

CURRENT OF code changes
-----------------------

This merge includes the upstream support for UPDATE WHERE CURRENT OF. GPDB
had support for that already, it's been refactored to use the upstream code
as much as possible (execCurrentOf()). Now that we have the upstream code
available, this also refactors the way current position of a cursor is
dispatched. Instead of modifying every CurrentOfExpr node in the plan tree
before dispatching it, store the current positions of each cursor mentioned
in a CurrentOfExpr in a separate CursorPosInfo node, and dispatch those
along with the query plan, in QueryDispatchDesc.

This was a joint effort between me (Heikki) and Daniel Gustafsson.

The printablePredicate of a static PartitionSelector node contains Var nodes
with varno=INNER. That's bogus, because the PartitionSelector node doesn't
actually have any child nodes, but works at execution time because the
printablePredicate is not only used by EXPLAIN. In most cases, it still
worked, because most Var nodes carry a varnoold field, which is used by
EXPLAIN for the lookup, but there was one case of "bogus varno" error even
memorized in the expected output of the regression suite. (PostgreSQL 8.3
changed the way EXPLAIN resolves the printable name so that varnoold no
longer saves the bacon, and you would get a lot more of those errors)

To fix, teach the EXPLAIN of a Sequence node to also reach into the
static PartitionSelector node, and print the printablePredicate as if that
qual was part of the Sequence node directly.

The user-visible effect of this is that the static Partition Selector
expression now appears in EXPLAIN output as a direct attribute of the
Sequence node, not as a separate child node. Also, if a static Partition
Selector doesn't have a "printablePredicate", i.e. it doesn't actually
do any selection, it's not printed at all.

This is mostly copy-pasted from CoerceToDomain.

Note: This requires an up-to-date version of ORCA to compile, older
versions of the ORCA library itself don't know about CoerceViaIO nodes
either.

This change in commit af5e4bcf was surely not intentional, and made the
test to fail.

    Port `dml/triggers`
    Port `olap/groupingfunction`
    Port `functions/builtin`
    Port `dml/functional`
    Port `dml/functional/sql_partiton`
    Port `functions/functionProperty`

    Update optfunctional_schedule with ported tests

    Update the Makefile for optimizer functional tests

When SharedScan requests a tuple from the underlying Sort, and the Sort has spilled to disk, the mk_sort code (also the regular sort) makes a copy of the memtuple, but places it in the sortcontext.
When hitting a QueryFinishPending (e.g., master received enough tuples, so stop the execution), EagerFree is called for the entire tree. When reaching the sort node, the sortcontext is deleted, even though the SharedScan above it still has a pointer to a memtuple allocated in this context.

The solution is: pass a memory context for the mk_sort and regular sort to use (change the API and implementation of the following two functions: tuplesort_gettupleslot_pos_mk and tuplesort_gettupleslot_pos). Those functions should use the context provided by the caller to make a copy of the memtuple.
Signed-off-by: NGeorge Caragea <gcaragea@pivotal.io>

Removed extraneous argument to _prompt_continue that would cause
gpcrondump to fail if run without -a.