Since we have the documentation in the repo now, avoid linking to
the web version since the link is bound to do stale quickly.

gpmigrator and gpmigrator_mirror are older utilities
to help the upgrade from 4.2 -> 4.3.
This is no longer necessary; we are removing all
related gpmigrator calls and mentions.

* Remove gpmigrator from travis
* Remove upgrade test
Signed-off-by: NChumki Roy <croy@pivotal.io>

Previously, -a -m from gpstart still resulted into a prompt due to a
bug.
This actually made it such that the changed line did not run
properly -- probably hung on the prompt, since we don't validate if
gpstart actually runs properly.
6edeefec fixed the gpstart prompt issue and
caused this issue (actually running with master only) to finally show
up, which made the later tests to fail for unabling queries to the
database.. Ensure that we start fresh by doing
a gpstart -a instead of just doing master only.

--only show error for lacking MASTER_DATA_DIRECTORY for --file, not
other options

And older version of GPDB generates a bad plan. Changing the query to
generate the right plan for testing workfile which has Hash join instead
of nested loop.
Signed-off-by: NShreedhar Hardikar <shardikar@gmail.com>

- Compile python 2.7.12 on centos6
- Update ext tarball on ivy to remove python 2.6.2
- Update ivy dependencies for python to point to 2.7.12
Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>
Signed-off-by: NTom Meyer <tmeyer@pivotal.io>

* In some test cases, we would like to have the main process shutdown
  without waiting for the pool thread. This is possible with a pool
  thread being a daemon.

- Modified patch from Jimmy
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

In commit e8dd6b61 we
changed the way gprestore_filter.py filters languages
and functions.  The new functionality is tested in the
base backup.feature suite and does not need to be
separately tested in netbackup.feature.

* adding supported 9.1 xml functions to built-in list

* updates to SQL2008 support table for new XML functions

* adding expressions to function table

The old cdbfast test suite had these, move to ICW.
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

- gpdb supports anyelement, anyarray, anynonarray, and any enum types
- add polymorphic types section to data types page
- add polymorphic function example to create function page
- add polymorphic info and example to pl/pgsql page

removing duplicated hadoop compatibility table; clarifying that some hadoop versions no longer supported (#2075)

* GPDB DOCS - DO command - anonymous blocks

* GPDB DOCS - updates to support DO command.

* mention anonymous blocks in Admin Guide

* GPDB DOCS updates for DO command review

* GPDB DOCS fix typos in DO command

* GPDB DOCs Added cursor limitatation toPL/Python topic.

* GPDB DOCS - update CREATE LANGUAGE and pg_language for DO and 8.3 merge.

* Deprecation notices for gpmapreduce

* Fix typos, phrasing.

* Unneeded sentence.

* Fix typo

* Edits from pull request

* Edits from David Y.

This helps in predictably managing load on concourse.
Signed-off-by: NDavid Sharp <dsharp@pivotal.io>

Previously, any restore that filtered based on table or
schema name did not restore user-created CASTs because
they were not included in the filter. This commit adds
support for restoring casts with table and schema filters
and the --change-schema flag.

Gptransfer will hang at the end of the program if there are existing
threads when the --validate=md5 argument is passed on Python 2.7. This adds explicit exit calls.

Authors: Chris Hajas and Jamie McAtamney

Since we're already building pg_upgrade as part of the toplevel
GNUmakefile, add to the contrib Makefile as well to allow make
-C contrib/ <target> to include pg_upgrade. This adds the support
functions required by pg_upgrade as well.

Last call is never guaranteed for aborting transaction,
register resource owner callback to ensure proper cleanup.
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>

Most of the builtin analytical functions added in Greenplum have
been deprecated in favour of the corresponding functionality in
MADLib. The deprecation notice was committed in a61bf8b7, code as
well as tests are removed here. The sum(array[]) function require
the matrix_add() backend code and thus it remains.

This removes matrix_add(), matrix_multiply(), matrix_transpose(),
pinv(), mregr_coef(), mregr_r2(), mregr_pvalues(), mregr_tstats(),
nb_classify() and nb_probabilities().

functions.

Coverity complains that the function `cmp_deformed_tuple` access the
datum from expression eval as an array. But we don't have that use case due to the
following error. For e.g., the parser will complain that it can't have
more than 1 columns for sorting when we use range in window function as shown below.

```
select depname, empno, salary, LAST_VALUE(salary)
error.
over (partition by depname order by empno, salary range
between 5000 preceding and 3900 preceding) as
highest_salary from empsalary;
ERROR:  only one ORDER BY column may be specified when RANGE is used in
a window specification
```

- In the current system, greenplum-db-appliance-(.*) collides with
  greenplum-db-(.*)
Signed-off-by: NTom Meyer <tmeyer@pivotal.io>

* Multi-dimensional arrays can now be used as arguments to a PL/python function
(used to throw an error), and they can be returned as nested Python lists.

This makes a backwards-incompatible change to the handling of composite
types in arrays. Previously, you could return an array of composite types
as "[[col1, col2], [col1, col2]]", but now that is interpreted as a two-
dimensional array. Composite types in arrays must now be returned as
Python tuples, not lists, to resolve the ambiguity. I.e. "[(col1, col2),
(col1, col2)]".

To avoid breaking backwards-compatibility, when not necessary, () is still
accepted for arrays at the top-level, but it is always treated as a
single-dimensional array. Likewise, [] is still accepted for composite types,
when they are not in an array. Update the documentation to recommend using []
for arrays, and () for composite types, with a mention that those other things
are also accepted in some contexts.

The upstream patch is 94aceed3

* Give a hint, when [] is incorrectly used for a composite type in array.

    That used to be accepted, so let's try to give a hint to users on why
    their PL/python functions no longer work.

   see upstream 510e1b8e

* add tests for additional conversions.
move them into plpython_types
move the create function closer to the actual select
which is more like upstream

Since we have a boolean yes/no switch for coverage enabled builds
in Makefile.global from autoconf, use that rather than inspecting
the CFLAGS for ease of reading the code. Also use the top_builddir
variable in setting the libdir, while unlikely to move in the
hierarchy it's the right thing to do.

The old cdbfast test suite had tests for these, looks good, move to ICW.
Signed-off-by: NHaozhou Wang <hawang@pivotal.io>
Signed-off-by: NAdam Lee <ali@pivotal.io>

This backports commit 79d39420d in it's entirety and the required parts
of 3cba8240 (original commitmessages in full below). The difference to
upstream is GPDB using SFRM_Materialize instead of _Randomize as well as
the required pg_proc entry changes. Includes the upstream tests and bumps
the catalog.

  commit 79d39420d6cd60cab141b1f13185a2415edfa4a3
  Author: Andrew Dunstan <andrew@dunslane.net>
  Date:   Fri Mar 29 14:12:13 2013 -0400

    Add new JSON processing functions and parser API.

    The JSON parser is converted into a recursive descent parser, and
    exposed for use by other modules such as extensions. The API provides
    hooks for all the significant parser event such as the beginning and end
    of objects and arrays, and providing functions to handle these hooks
    allows for fairly simple construction of a wide variety of JSON
    processing functions. A set of new basic processing functions and
    operators is also added, which use this API, including operations to
    extract array elements, object fields, get the length of arrays and the
    set of keys of a field, deconstruct an object into a set of key/value
    pairs, and create records from JSON objects and arrays of objects.

    Catalog version bumped.

    Andrew Dunstan, with some documentation assistance from Merlin Moncure.

  commit 3cba8240
  Author: Itagaki Takahiro <itagaki.takahiro@gmail.com>
  Date:   Mon Feb 21 14:08:04 2011 +0900

    Add ENCODING option to COPY TO/FROM and file_fdw.
    File encodings can be specified separately from client encoding.
    If not specified, client encoding is used for backward compatibility.

    Cases when the encoding doesn't match client encoding are slower
    than matched cases because we don't have conversion procs for other
    encodings. Performance improvement would be be a future work.

    Original patch by Hitoshi Harada, and modified by me.

With commit abb9dd8c, gpstop -m now behaves
similar to without -m, hence add -a to tests so that it doesn't prompt.

PR github page currently doesn't show PR pipeline is running and incorrectly
shows passed if its a rebase. This fixes it so that correctly reflects it has
started running the job.