A new version of libXML changes the output format, this patch
adapts the new "expected" file introduced in
PostgreSQL commit 085423e3.
Slight modifications by me.

This patch will not fail the current Pulse run in any way, because
the Pulse systems do not yet use the new libXML library, and therefore
are happy with xml.out and xml_1.out. I stripped down the regression
tests and this is the output without the patch:

parallel group (14 tests):  limit prepare conversion xml rowtypes copy2 polymorphism sequence without_oid temp truncate domain rangefuncs alter_table
     limit                ... ok (0,24 sec)  (diff:0,07 sec)
     copy2                ... ok (2,44 sec)  (diff:0,07 sec)
     temp                 ... ok (3,83 sec)  (diff:0,07 sec)
     domain               ... ok (7,12 sec)  (diff:0,10 sec)
     rangefuncs           ... ok (12,15 sec)  (diff:0,09 sec)
     prepare              ... ok (0,49 sec)  (diff:0,06 sec)
     without_oid          ... ok (3,49 sec)  (diff:0,07 sec)
     conversion           ... ok (1,19 sec)  (diff:0,17 sec)
     truncate             ... ok (4,03 sec)  (diff:0,06 sec)
     alter_table          ... ok (27,88 sec)  (diff:0,32 sec)
     sequence             ... ok (3,18 sec)  (diff:0,08 sec)
     polymorphism         ... ok (3,13 sec)  (diff:0,11 sec)
     rowtypes             ... ok (2,17 sec)  (diff:0,09 sec)
     xml                  ... FAILED (1,69 sec)  (diff:0,39 sec)
test variadic_parameters  ... ok (1,57 sec)  (diff:0,07 sec)
test default_parameters   ... ok (0,68 sec)  (diff:0,07 sec)

=======================
 1 of 97 tests failed.
=======================

This is the output with the patch:

parallel group (14 tests):  limit prepare conversion xml rowtypes copy2 polymorphism sequence without_oid temp truncate domain rangefuncs alter_table
     limit                ... ok (0,21 sec)  (diff:0,10 sec)
     copy2                ... ok (1,95 sec)  (diff:0,06 sec)
     temp                 ... ok (3,41 sec)  (diff:0,26 sec)
     domain               ... ok (6,67 sec)  (diff:0,19 sec)
     rangefuncs           ... ok (11,55 sec)  (diff:0,09 sec)
     prepare              ... ok (0,41 sec)  (diff:0,06 sec)
     without_oid          ... ok (2,91 sec)  (diff:0,06 sec)
     conversion           ... ok (0,96 sec)  (diff:0,15 sec)
     truncate             ... ok (3,62 sec)  (diff:0,09 sec)
     alter_table          ... ok (25,79 sec)  (diff:0,22 sec)
     sequence             ... ok (2,89 sec)  (diff:0,06 sec)
     polymorphism         ... ok (2,50 sec)  (diff:0,09 sec)
     rowtypes             ... ok (1,87 sec)  (diff:0,12 sec)
     xml                  ... ok (1,54 sec)  (diff:0,30 sec)
test variadic_parameters  ... ok (1,43 sec)  (diff:0,07 sec)
test default_parameters   ... ok (0,43 sec)  (diff:0,07 sec)

======================
 All 97 tests passed.
======================

Fix #550

It didn't do anything useful AFAICS.

* Support the aggregation transition datatype to be "internal"

Backport below commits from upstream:

    commit 22d9ddbb
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Sat Aug 14 15:47:13 2010 +0000

        Fix planner to make a reasonable assumption about the amount of memory space
        used by array_agg(), string_agg(), and similar aggregate functions that use
        "internal" as their transition datatype.  The previous coding thought this
        took *no* extra space, since "internal" is pass-by-value; but actually these
        aggregates typically consume a great deal of space.  Per bug #5608 from
        Itagaki Takahiro, and fix suggestion from Hitoshi Harada.

        Back-patch to 8.4, where array_agg was introduced.

    commit 9e0247ab
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Fri Nov 14 19:47:50 2008 +0000

        In CREATE AGGREGATE, allow the transition datatype to be "internal", but only
        if the user is superuser.  This makes available to extension modules the same
        sort of trick being practiced by array_agg().  The reason for the superuser
        restriction is that you could crash the system by connecting up an
        incompatible pair of internal-using functions as an aggregate.  It shouldn't
        interfere with any legitimate use, since you'd have to be superuser to create
        the internal-using transition and final functions anyway.

Besides, we cannot find regression tests in above commits, so we
add a set of tests for this feature, a part of them come from
legacy extension "intagg" regression tests.

Conflicts:
	src/test/regress/expected/arrays.out
	src/test/regress/sql/arrays.sql

* add tests to #588: run with role of (non-)superuser

    #588: Support the aggregate's transype to be "internal"

Normally, the aggregate's transtype can't be a pseudo-type.
However, we can allow "internal" for functions that want to
pass pointers to private data structures; but allow that only
to superusers. Here we add tests to verify this constraint.

This code was commented out without providing any useful information
so remove to reduce clutter.

This closes #606

This mirrors the check in vacuum.c, for VACUUMing external tables. This was
missed in commit f995c59f, which replaced most of the old ANALYZE with
the upstream code.

Fixes issue #395, reported by George Caragea.

Space for bit addition to nullbitmap was missed while estimating if
block has space for new delta item or not. Since adding a delta value,
adds a false bit to nullbitmap, it needs to be added in space
calculation before inserting new item to block. Missing same maybe endup
in failure during block write due to block size overflow.

The qp_query_execution test was added to the tree in commit 9d19664c
and then further updated in 29a0fec8 but it's not present in any test
schedule. Add to the greenplum_schedule to include it when running
installcheck-good.

Backport below commit from upstream:

    commit d5768dce
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Mon Feb 8 20:39:52 2010 +0000

        Create an official API function for C functions to use to check if they are
        being called as aggregates, and to get the aggregate transition state memory
        context if needed.  Use it instead of poking directly into AggState and
        WindowAggState in places that shouldn't know so much.

        We should have done this in 8.4, probably, but better late than never.

        Revised version of a patch by Hitoshi Harada.

modified:   src/backend/executor/nodeAgg.c
modified:   src/include/fmgr.h

Since we discover the apr-1-config binary during the configure stage
we might as well simplify the gpfdist Makefile by using it during
building. Also renames the APR_CONFIG variable to avoid colliding
with $APR_CONFIG using the gpAux makefile.

transaction block.

ReindexRelationList is a GPDB specific function that wraps each
reindex table operation within a separate transaction.  If the table
being reindexed is a partitioned table, each partition is reindexed in
a separate transaction.  Now that we have removed the assertion,
refactor this function to commit and start top level transcation,
rather than leaving this burden on callers.

Fixes #567.

This commit makes zlib to use palloc/pfree, by rewriting compress_zlib.c to utilize the API in gfile.c, and dispatcher to use palloc and pfree through a set of zlib wrapper functions. This enables our protection mechanisms (gp_vmem_protect_limit, runaway_detector_activation_percent) to keep the system stable by cleanly canceling a query which would otherwise cause us to run out of memory.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

Fixing memory leak because of reallocation of level state per partition for every new partition. Instead, we now reallocate level state for every level in the makeWindowState.

The following repro will consume gigabytes of memory and should run out of memory, given appropriate VMEM configuration:

drop table if exists wtest;
create table wtest
(
    id integer,
    dt date,
    name character varying(100),
    created_dt date
);

insert into wtest select a, current_date + a, 'xyz' || a, current_date + (a % 10) from (select a from generate_series(1, 10000000) as a) as temp;

select row_number() over (w) as row_num,
id,
dt,
name
from wtest
window w as (partition by dt,
name,
id
order by created_dt desc
);

After the fix the same repro should consume very little memory.

In some cases, the filerep processes get stuck waiting for the spin locks on an
in memory log and the postmaster PANICs if they can't acquire a lock after
timeout period. In order to avoid locking related issues we want to avoid
acquiring the locks in the first place. This patch introduces a new guc which
will cause filerep logs to be logged to server logs and will avoid logging to
the in memory log.

Add SPI_cursor_open_with_args/SPI_execute_with_args functions

This reverts to the upstream version of this function and removes the unit test that was testing the change.

Arrange for the "QEdetails" to be sent automatically from the QE at backend
startup, so that the QD doesn't need to request it and wait for the
response.

Makes merging easier, and is less code overall anyway. I don't think there
was any particular reason it was originally changed to differ from
upstream.

There is one slightly user-visible impact from this change: In a
full-database VACUUM ANALYZE command, the VACUUM and ANALYZE phases are
performed on one table, before moving to next, whereas before we performed
VACUUM on all tables first, and ANALYZE on all tables next. I couldn't
discern any reason from the code comments for that change in behaviour, so I
assume it was just coincidental.

Workfile caching has already been removed from master. This commit cleans up workfile manager and removes two functions (workfile_mgr_evict and acquire_entry_retry) related to workfile caching that we do not need them anymore.

Looks it falls back to legacy optimizer when running regression,
hence we temporally comment out three tests and create a story
on pivotaltracker #116312671 to solve it later.
-- comment out two tests in domain.sql
-- comment out one test in qp_misc_jiras.sql

  Thanks to Venkatesh Raghavan for helping

Backport below commits from upstream:

    commit 31edbadf
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Tue Jun 5 21:31:09 2007 +0000

        Downgrade implicit casts to text to be assignment-only, except for the ones
        from the other string-category types; this eliminates a lot of surprising
        interpretations that the parser could formerly make when there was no directly
        applicable operator.

        Create a general mechanism that supports casts to and from the standard string
        types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's
        I/O functions.  These new casts are assignment-only in the to-string direction,
        explicit-only in the other, and therefore should create no surprising behavior.
        Remove a bunch of thereby-obsoleted datatype-specific casting functions.

        The "general mechanism" is a new expression node type CoerceViaIO that can
        actually convert between *any* two datatypes if their external text
        representations are compatible.  This is more general than needed for the
        immediate feature, but might be useful in plpgsql or other places in future.

        This commit does nothing about the issue that applying the concatenation
        operator || to non-text types will now fail, often with strange error messages
        due to misinterpreting the operator as array concatenation.  Since it often
        (not always) worked before, we should either make it succeed or at least give
        a more user-friendly error; but details are still under debate.

        Peter Eisentraut and Tom Lane

    commit bf940763
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Tue Mar 27 23:21:12 2007 +0000

        Fix array coercion expressions to ensure that the correct volatility is
        seen by code inspecting the expression.  The best way to do this seems
        to be to drop the original representation as a function invocation, and
        instead make a special expression node type that represents applying
        the element-type coercion function to each array element.  In this way
        the element function is exposed and will be checked for volatility.
        Per report from Guillaume Smet.

Adding test changes in alter_table_optimizer.out for fixing tuple descriptor leak in split_rows during an ALTER TABLE SPLIT PARTITION.

The function updateAppendOnlyStats() was removed in 3fb6ca82 but the
prototype remained.

Signed-off-by: NXin Zhang <xzhang@pivotal.io>

The gp_default_storage_options GUC had two scenarios in which the value between
master and segments could be conflicting which would cause tables to possibly
have different storage parameters between master and segments.

First Scenario:
The below gpconfig example would give newly created user tables a default storage
parameter of AO on segments while master sets the table as heap. Fixed by not
allowing the user to make them different values.
gpconfig -c gp_default_storage_options -v "'appendonly=true'" -m "'appendonly=false'"

Second Scenario:
During a psql session not in transaction, a session-level gp_default_storage_options
GUC value can be set which will also be dispatched to segment processes. Those
segment processes when idle for a period of time will be stopped and the next SQL
(e.g. CREATE TABLE) will spawn new segment processes which will not have the
session-level GUC value set. Fixed by setting the value at cdbgang creation.

We already had a mostly up to date list in the directory specific
.gitignore files but the toplevel .gitignore contained a *.out
which took precedence (lower level files override higher level
files so removing *.out will allow for the src/test/regress/* files
to override the now removed .out file blocking). The reason for
not ignoring *.out is that we have quite a few tracked .out files
in the repository which we don't want to ignore changes to.

Also update the files to match the current set of tests since there
were a few missing.

Fixing the issue #482 with missing pg_proc references to the array manipulation
functions. This PR also back ports the regression tests for arrays from PG 8.4

- targeted_dispatch

If gpfdist is built with ssl and does not use --ssl
it will now print the correct message.

Authors: Marbin Tan & Larry Hamel