Since we discover the apr-1-config binary during the configure stage
we might as well simplify the gpfdist Makefile by using it during
building. Also renames the APR_CONFIG variable to avoid colliding
with $APR_CONFIG using the gpAux makefile.

transaction block.

ReindexRelationList is a GPDB specific function that wraps each
reindex table operation within a separate transaction.  If the table
being reindexed is a partitioned table, each partition is reindexed in
a separate transaction.  Now that we have removed the assertion,
refactor this function to commit and start top level transcation,
rather than leaving this burden on callers.

Fixes #567.

This commit makes zlib to use palloc/pfree, by rewriting compress_zlib.c to utilize the API in gfile.c, and dispatcher to use palloc and pfree through a set of zlib wrapper functions. This enables our protection mechanisms (gp_vmem_protect_limit, runaway_detector_activation_percent) to keep the system stable by cleanly canceling a query which would otherwise cause us to run out of memory.
Signed-off-by: NNikos Armenatzoglou <nikos.armenatzoglou@gmail.com>

Fixing memory leak because of reallocation of level state per partition for every new partition. Instead, we now reallocate level state for every level in the makeWindowState.

The following repro will consume gigabytes of memory and should run out of memory, given appropriate VMEM configuration:

drop table if exists wtest;
create table wtest
(
    id integer,
    dt date,
    name character varying(100),
    created_dt date
);

insert into wtest select a, current_date + a, 'xyz' || a, current_date + (a % 10) from (select a from generate_series(1, 10000000) as a) as temp;

select row_number() over (w) as row_num,
id,
dt,
name
from wtest
window w as (partition by dt,
name,
id
order by created_dt desc
);

After the fix the same repro should consume very little memory.

In some cases, the filerep processes get stuck waiting for the spin locks on an
in memory log and the postmaster PANICs if they can't acquire a lock after
timeout period. In order to avoid locking related issues we want to avoid
acquiring the locks in the first place. This patch introduces a new guc which
will cause filerep logs to be logged to server logs and will avoid logging to
the in memory log.

Add SPI_cursor_open_with_args/SPI_execute_with_args functions

w/o reuse_table.

If there is no reuse_table, there would be no way to check the format error. If resue_table is true,
gpload can show how to check the detail error message.

This reverts to the upstream version of this function and removes the unit test that was testing the change.

Arrange for the "QEdetails" to be sent automatically from the QE at backend
startup, so that the QD doesn't need to request it and wait for the
response.

Makes merging easier, and is less code overall anyway. I don't think there
was any particular reason it was originally changed to differ from
upstream.

There is one slightly user-visible impact from this change: In a
full-database VACUUM ANALYZE command, the VACUUM and ANALYZE phases are
performed on one table, before moving to next, whereas before we performed
VACUUM on all tables first, and ANALYZE on all tables next. I couldn't
discern any reason from the code comments for that change in behaviour, so I
assume it was just coincidental.

Workfile caching has already been removed from master. This commit cleans up workfile manager and removes two functions (workfile_mgr_evict and acquire_entry_retry) related to workfile caching that we do not need them anymore.

Looks it falls back to legacy optimizer when running regression,
hence we temporally comment out three tests and create a story
on pivotaltracker #116312671 to solve it later.
-- comment out two tests in domain.sql
-- comment out one test in qp_misc_jiras.sql

  Thanks to Venkatesh Raghavan for helping

Backport below commits from upstream:

    commit 31edbadf
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Tue Jun 5 21:31:09 2007 +0000

        Downgrade implicit casts to text to be assignment-only, except for the ones
        from the other string-category types; this eliminates a lot of surprising
        interpretations that the parser could formerly make when there was no directly
        applicable operator.

        Create a general mechanism that supports casts to and from the standard string
        types (text,varchar,bpchar) for *every* datatype, by invoking the datatype's
        I/O functions.  These new casts are assignment-only in the to-string direction,
        explicit-only in the other, and therefore should create no surprising behavior.
        Remove a bunch of thereby-obsoleted datatype-specific casting functions.

        The "general mechanism" is a new expression node type CoerceViaIO that can
        actually convert between *any* two datatypes if their external text
        representations are compatible.  This is more general than needed for the
        immediate feature, but might be useful in plpgsql or other places in future.

        This commit does nothing about the issue that applying the concatenation
        operator || to non-text types will now fail, often with strange error messages
        due to misinterpreting the operator as array concatenation.  Since it often
        (not always) worked before, we should either make it succeed or at least give
        a more user-friendly error; but details are still under debate.

        Peter Eisentraut and Tom Lane

    commit bf940763
    Author: Tom Lane <tgl@sss.pgh.pa.us>
    Date:   Tue Mar 27 23:21:12 2007 +0000

        Fix array coercion expressions to ensure that the correct volatility is
        seen by code inspecting the expression.  The best way to do this seems
        to be to drop the original representation as a function invocation, and
        instead make a special expression node type that represents applying
        the element-type coercion function to each array element.  In this way
        the element function is exposed and will be checked for volatility.
        Per report from Guillaume Smet.

Adding test changes in alter_table_optimizer.out for fixing tuple descriptor leak in split_rows during an ALTER TABLE SPLIT PARTITION.

The function updateAppendOnlyStats() was removed in 3fb6ca82 but the
prototype remained.

Signed-off-by: NXin Zhang <xzhang@pivotal.io>

The gp_default_storage_options GUC had two scenarios in which the value between
master and segments could be conflicting which would cause tables to possibly
have different storage parameters between master and segments.

First Scenario:
The below gpconfig example would give newly created user tables a default storage
parameter of AO on segments while master sets the table as heap. Fixed by not
allowing the user to make them different values.
gpconfig -c gp_default_storage_options -v "'appendonly=true'" -m "'appendonly=false'"

Second Scenario:
During a psql session not in transaction, a session-level gp_default_storage_options
GUC value can be set which will also be dispatched to segment processes. Those
segment processes when idle for a period of time will be stopped and the next SQL
(e.g. CREATE TABLE) will spawn new segment processes which will not have the
session-level GUC value set. Fixed by setting the value at cdbgang creation.

The new option spells out fully_qualified_domain_name since fqdn is
such a mouthful of an abbreviation. In order to retrieve the fqdn or
hostname in a portable manner the Python socket module seems the best
bet. It also avoids the overhead of opening a command interpreter and
pipe to get the information.

We already had a mostly up to date list in the directory specific
.gitignore files but the toplevel .gitignore contained a *.out
which took precedence (lower level files override higher level
files so removing *.out will allow for the src/test/regress/* files
to override the now removed .out file blocking). The reason for
not ignoring *.out is that we have quite a few tracked .out files
in the repository which we don't want to ignore changes to.

Also update the files to match the current set of tests since there
were a few missing.

Fixing the issue #482 with missing pg_proc references to the array manipulation
functions. This PR also back ports the regression tests for arrays from PG 8.4

The use of non-breaking space and horizontal ellipsis in the docs
cause problems for the generated docs in psql, replace with counter-
parts in the printable characters set. This fixes the "illegal
character encoding in string literal" warning from clang.

Due to the support of special characters, we made some changes in the format
of appendonly tables' state file, which inevitably create extra tables into the
dirty table lists for incremental backup.

Use the "template1" instead of the restoring database for connection url

Authors: Pengcheng Tang, Christopher Hajas

- targeted_dispatch

If gpfdist is built with ssl and does not use --ssl
it will now print the correct message.

Authors: Marbin Tan & Larry Hamel

the cleanup of EState

Below is the error condition that was being filtered by the cdbfast test
framework.  I have added an ignore section around the update that is
expected to fail in the planner scenario.

*** ./expected/DML_over_joins.out2016-03-27 16:44:49.000000000 -0700
--- ./results/DML_over_joins.out2016-03-27 16:44:49.000000000 -0700
***************
*** 1342,1348 ****
(2 rows)

update sales_par set region = 'new_region' where id in (select s.b from s, r where s.a = r.b) and day in (select a from r);
! ERROR:  moving tuple from partition "sales_par_1_prt_usa" to partition "sales_par_1_prt_other_regions" not supported
select sales_par.* from sales_par where id in (select s.b from s, r where s.a = r.b) and day in (select a from r);
id | year | month | day | region
----+------+-------+-----+--------
--- 1342,1348 ----
(2 rows)

update sales_par set region = 'new_region' where id in (select s.b from s, r where s.a = r.b) and day in (select a from r);
! ERROR:  moving tuple from partition "sales_par_1_prt_europe" to partition "sales_par_1_prt_other_regions" not supported
select sales_par.* from sales_par where id in (select s.b from s, r where s.a = r.b) and day in (select a from r);
id | year | month | day | region
----+------+-------+-----+--------

======================================================================

This enhances the existing "sirv_functions" ICG tests.  This is
accomplished by migrating the cdbfast regression "list_queries" test
schedule's "sirv_functions" test suite to ICG.

Here are some characteristics of the test suite:

  Schedule file: greenplum_schedule
  Schema: sirv_functions
  Set to run in parallel: yes
  Orca specific test output: no
  Test execution time: <1 minute
  Pivotal Tracker Story: #113967799