Verify tablespaces work for both a incremental and full recovery.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>

Co-authored-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
Co-authored-by: NAbhijit Subramanya <asubramanya@pivotal.io>

* gpbackup and gprestore will not run if expand in process

* Add backup step to overview and checklist

* update pl/r docs to use CREATE EXTENSION

* update pl/perl, /python, update deprecation notices, add CASCADE info

* Feedback from Chuck

* Update postgresql.org hrefs to 9.4

* Feedback from Lisa

* docs - address pxf jdbc ddl vs. server config

* misc edits requested by david

This corresponds to ORCA commit "Predicate pushdown in part tables with window func"
It includes ICG changes

Our goal with these pipelines is to find flakes, and the timeouts are getting in the way of that goal.

The answerfile was generated against 5X.  A lot has changed in
numeric.c since then, so update answer file accordingly.

Binary swap tests need to be enabled only on stable branches but not on master.
This commit makes it easy to enable / disable binary swap tests in a pipeline,
especially when gen_pipeline.py is used to generate it.

Compilation with --enable-testutils seems to has been broken long time
as `MaxDynamicLWLock` is no more defined. Given no one complained
seems this is unused config option and code. Hence, instead of fixing
the compilation ripping out this code.

* docs - note for pxf parquet date conversion

* localize to system time first

Full joins are sub-optimal in ORCA as they are implemented as a UNION of
Left Outer Join AND Left Anti-Semi Join. However, GPDB provides a full
outer join operator. Therefore, until ORCA implements a more optimal
FULL JOIN, it will fall back to the Postgres legacy query optimizer for
queries with FULL JOINs.

Co-authored by: Sambitesh Dash sdash@pivotal.io
Co-authored by: Ashuka Xue axue@pivotal.io

This is done to zero out the padding bytes and hence have consistent compression
ratios. Plus, zero bytes usually compress better.

Suggested by Heikki Linnakangas.

In Greenplum, data structure RelationData includes the related pg_appendonly
row for an ao table, when pg_appendonly changes for that ao table, we should
invalidate the relcache for it also to prevent possible inconsistency.

I can not find a real case to hit this after some tries, but this is an issue
we could hit in theory.

Reviewed-by: Hubert Zhang
Reviewed-by: Asim R P
Reviewed-by: Jimmy Yih

Currently, gpdb pipeline is fetching gpdb7 in master
pipeline but madlib hasn't supported gpdb7 yet.  We
are temporarily removing this section to unblock other
teams.
Co-authored-by: NJingyi Mei <jmei@pivotal.io>

Issue is encountered because XLogReaderState does not make any
guarantees to preserve the XLogRecord returned between calls to
ReadRecord. In this particular scenario we read the checkpoint and redo
records from the backup label. After reading the latter record we have
no guarantees that the former record is still pointing to unchanged
memory.

* docs - add optional procedure to cfg pxf host/port

* some edits requested by david

* docs - add hive filter pushdown note, correct xref

* just Disable

gpload.pyc is included in gpdb6 clients package. It is not necessary for
release and may conflict with gpdb6 server's package.
so we just remove it.

Rewrite test sql to verify partition numbers after drop partitions.
Co-authored-by: NWenlin Zhang <wzhang@pivotal.io>
Co-authored-by: NBing Xu <bxu@pivotal.io>

* Update pl/java docs for JVM version, shared library loc, CREATE EXTENSION command, .sql file deprecation

* Add default library location for JDK 8 and JDK 11

* format testdb; relocate uninstall topic

We've done the work to enable or disable specific combinations of
recursive CTEs. They are safe to use, and when they are not, there
will be an error message disallowing the query.

Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>

Noticed while doing contrib work that formatter_fixedwidth had a
wildly incorrectly linebroken errmsg, causing a very large section
of whitespace in the middle of the error message to the user.

While fixing this, also fixed all other ereport() calls to have
messages according to the style guide, and converted an internal
error to elog() instead.

Also fixed egregious whitespace offences in surrounding lines, but
left the bulk of the offences to a pgindent run.
Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>

Modified from camel to snake case.

This corresponds to ORCA commit "Use prime number of buckets to avoid
hash collisions"
Authored-by: NChris Hajas <chajas@pivotal.io>

In the case where the user wants to change the port of a mirror, but
leave it in the same directory on the same host, gpmovemirrors was
previously deleting the directory as part of the cleanup because it
did not recognize that the old and new directories were the same.
This commit adds a check to prevent that.

This commit also logs a warning if the move_config_file contains
identical attributes(host,port,data directory) for the old and new
mirror.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>

While actual logic relating to filespaces was already removed from gpmovemirrors
in a previous commit, there were still some comments and dead code referencing
filespaces, which this commit removes.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>

Mostly in code comments, but some in user facing messaging.
Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NJimmy Yih <jyih@pivotal.io>

Removes imports of modules once required, but for which usage has
been removed over the years in various refactorings.
Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NJimmy Yih <jyih@pivotal.io>

Each interconnect had its own (duplicate) function for converting a
sockaddr_storage struct into a human readable string. Replace these
with a shared implementation and convert it to use the upstream
library function for just this purpose. We were already relying on
this function in the case of an error on Solaris, a platform which
isn't supported by Greenplum anyways.
Reviewed-by: NHeikki Linnakangas <hlinnakangas@pivotal.io>
Reviewed-by: NNing Yu <nyu@pivotal.io>
Reviewed-by: NJimmy Yih <jyih@pivotal.io>

In commit e9d21573, the GPDBVersion is not updated, which causes gppkg
test failure. Because the gpversion of the demo gppkg mismatches the major
version of gpdb
Co-authored-by: NHaozhou Wang <hawang@pivotal.io>

Use case insensitive comparison when comparing headers in gpfdist
to ensure that X-Gp-Proto is treated equally to X-GP-PROTO. This
is mandated by RFC2616 which in subsection 4.2 writes regarding
headers: "Field names are case-insensitive".

Include apr_general.h to ensure that strcasecmp() is defined on
platforms where it isn't supplied in libc, like Windows which use
stricmp() instead.

Reported-by: Mike Wright in Github issue #5689
Reviewed-by: NJacob Champion <pchampion@pivotal.io>
Reviewed-by: NYandong Yao <yyao@pivotal.io>
Reviewed-by: NDaniel Gustafsson <dgustafsson@pivotal.io>

Fix two spotted occurrences of "Greemplum"

There are no callers of this, and there seems little use fot it, so
remove.
Reviewed-by: NBen Christel <bchristel@pivotal.io>
Reviewed-by: NBradford Boyle <bboyle@pivotal.io>

While it works for development and use, macOS is not a supported
platform for Greenplum release engineering. The release engineering
support for macOS was also stuck in 32-bit 10.6 land which is long
since EOL'd. This removes all leftover macOS bits, while ensuring
that building and running Greenplum on macOS still works.
Reviewed-by: NBen Christel <bchristel@pivotal.io>
Reviewed-by: NBradford Boyle <bboyle@pivotal.io>