* docs: S3 gpcheckcloud - add config parameter gpcheckcloud_newline

* docs: fix typo.

* docs: gpbackup/gprestore limitation for partitioned table with external table partition.

* docs: edit to fix a typo.

* docs: updated gpbackup/gprestore limitation. partitioned table w/ external table leaf part. not supported.

With the removal of installation of base packages in
setup_ssh_to_cluster in CCP 1.4.4, the gppkg job's assumption
on those base packages being installed, specifically openssl,
were broken.

Either the jobs task can install the package on the base centos 6 image
it was using on every execution, or we can just use the ccp image that
has them preinstalled.
Signed-off-by: NKris Macoskey <kmacoskey@pivotal.io>

* docs - add content for pxf writable profiles

* use consistent naming for PXF

* add writing to HDFS book subnav

* main pxf subnav title change

* edits from david

* use consistent terms when introducing pxf

* not writing is experimental

* style experimental warning on write page closer to rest of docs

* dir/file distinction, other edits from alex

* discuss distributed by

* rename read topic, more info to distributed by

Signed-off-by: NKris Macoskey <kmacoskey@pivotal.io>

- add regression test for gpcloud on ubuntu
- create pattern for reusing task across platforms
- Remove configure step since it is not necessary
- Remove unused files
Signed-off-by: NGoutam Tadi <gtadi@pivotal.io>

* Initial setup of building on Ubuntu with Conan

* Updated Pipeline to use new/less Ubuntu jobs

* Updated PR pipeline

* docs: gpbackup/gprestore limitation for partitioned table with external table partition.

* docs: edit to fix a typo.

* docs: gpload - YAML ERROR_TABLE element backward compatibility

In 5.x, ERROR_TABLE in the YAML file was removed and returns an error.

For backward compatibility issue a warning and use LOG_ERRORS and REUSE_TABLE set to true.

* docs: gpload - edits based on review.

* docs - add content for PXF HBase profile

* fix incorrect table name - indirect mapping now working w example

* use new page title in pxf overview xrefs

* incorporate edits from david

* note that hbase does not support predicate pushdown

The sequence xlog record generated at the end of DefineSequence
records the local tuple->t_data when the actual page item is
available. Not using the actual page item caused gp_replica_check
extension to fail for sequences because the tuple->t_data ctid would
be replicated as (0,0) instead of the actual page item ctid
(0,1). This is because tuple->t_data is used in the xlog record
creation but its t_ctid is not set even if tuple->t_self is fine.

To fix this, we cherry-pick and modify upstream fix.

Reference commit message:
commit 8d34f686
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:   Tue Apr 22 09:50:47 2014 +0300

    Avoid transient bogus page contents when creating a sequence.

    Don't use simple_heap_insert to insert the tuple to a sequence relation.
    simple_heap_insert creates a heap insertion WAL record, and replaying that
    will create a regular heap page without the special area containing the
    sequence magic constant, which is wrong for a sequence. That was not a bug
    because we always created a sequence WAL record after that, and replaying
    that overwrote the bogus heap page, and the transient state could never be
    seen by another backend because it was only done when creating a new
    sequence relation. But it's simpler and cleaner to avoid that in the first
    place.

We were using heap_mask which kinda worked but ideally we should be
using the intended mask for sequences which is seq_mask.

This test would only produce the LOG lines memorized in the expected output
if log_statement='all' was set. Remove the assumption, by temporarily
setting log_statement (and log_min_duration_statement), like in some
earlier tests in the same file.

Refactor resgroup cpu_rate_limit test to make it more stable.
Signed-off-by: NNing Yu <nyu@pivotal.io>

A query will be put in the waiting queue if it cannot get some per slot
memory quota. However per slot memory slot is just expected to be 0 when
memory_shared_quota is 100, so a query hangs infinitely in such a case.

To fix this we now distinguish the "no enough quota" and "zero quota"
cases.

gpload ERROR_TABLE configuration is forbidden in GPDB5 and use
LOG_ERRORS instead. That's why Informatica 9.x connector can't
load data into GPDB5.

So we accept ERROR_TABLE, if ERROR_TABLE is set, we will not
create it as before, but set LOG_ERRORS and REUSE_TABLE to true
instead.

Unfortunately, pg_upgrade still barfs on this. That was fixed in the
upstream in commit a2385cac13, in PostgreSQL 9.1, so we'll revisit this
once we merge up to that.

This reverts commit b8f8fccc. As failing some
crash recovery tests needs to be looked into and redone.

When configured with --enable-segwalrep, the tool will be enabled only for heap
relations. It has flagged some inconsistencies for other objects such as
sequences and appendonly tables. Once the issues are fixed, the tool will be
enabled to check for all objects created by ICW. This helps to consistently
check primary and mirrors.

Also add warning messages to gp_replica_check if files could not be opened.
Signed-off-by: NAshwin Agrawal <aagrawal@pivotal.io>

We should use XLogIsNeeded() instead of XLogArchivingActive() to decide whether
to generate XLog records or not. Otherwise the XLog records might not get
streamed to the standby.

Also fix the usage of XLogIsNeeded() in case of bulk loading using buffer pool
with ctas statements.

cdbappendonlystorage_test.c:
Test that calling appendonly_redo with XLOG_APPENDONLY_INSERT or
XLOG_APPENDONLY_TRUNCATE will call ao_insert_replay or
ao_truncate_replay respectively.

xlog_mm_test.c:
Test that an MMXLOG_REMOVE_FILE AO transaction log record will call
XLogAODropSegmentFile.

cdbmirroredappendonly_test.c:
Test that ao_insert_replay or ao_truncate_replay will call
XLogAOSegmentFile when we cannot find the AO segment file.

xlogutils_test.c:
Test that the invalid_page_tab hash table is properly used when
invalid AO segment files are encountered.

These four pieces combined will test the case when AO XLOG records are
replayed but the corresponding segment files are missing from
filesystem. The replay is entered into the invalid_page_tab hash table
until a corresponding MMXLOG_REMOVE_FILE XLOG record type comes along
to remove the entry.

Currently these unit tests are under --enable-segwalrep configure flag
in the Makefile but can be moved to the actual TARGET if non-segwalrep
tests are added and need to be tested.

Reference:
https://github.com/greenplum-db/gpdb/commit/b659d0479fa57c383e7dbe15043cd2ca9d46f77f

* initial import

- README.ubuntu.bash and README.centos.bash are created
for different dev-setup environments
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

The directory creation code here seems to be duplicated in multiple
places. Since tests are passing now, we're not sure if it's necessary.
It seems to only apply to a standby master, when using non-default
filespaces, and we haven't found a good test case to reproduce the
initial issue that it was supposed to solve -- it seems like originally,
there was a problem where the filespace hashtable was not initialized on
the standby.

Since things seem to be working as intended now, get rid of the dead
code.
Signed-off-by: NJacob Champion <pchampion@pivotal.io>
Signed-off-by: NAsim R P <apraveen@pivotal.io>

Add a FIXME to ao_insert_replay to check if it can work similar to
replay of heap insert xlog records.

This function had diverged a lot from upstream; post subselect merge.
One of the main reason is that upstream has lot of restrictive checks
which prevent pull-up of EXISTS/NOT EXISTS. GPDB handles them
differently; thus producing a join/initplan or a one-time filter.

The cases that GPDB handles and for which we have not ported the checks
from upstream are as follows:

- AGG with limit count with/without offset
- HAVING clause without AGG
- AGG without HAVING clause

For other conditions, we bail out as upstream. Hence we have added
checks differently for having and aggs inside simplify_EXISTS_query.
Rest of the code is similar to upstream.
Signed-off-by: NEkta Khanna <ekhanna@pivotal.io>

* removing docs for pgadmin (not included in build)

* remove pgadmin graphics

The error message was changed accidentally in commit 78b0a42e. Probably
a copy-paste mistake. Change it back.

* Moving module references to 'Additional Supplied Moduels' section to better match postregsql docs organization

* port postgresql passwordcheck docs

These tests are not stable enough, so remove them for now. We will add
them back after improvements.

This was in part already supported as the backend part of the below
commit was already backported. This brings in the frontend changes
as well the followup commit to remove backend support for SSLv3 as
it no longer make any sense to keep it around.

  commit 4dddf8552801ef013c40b22915928559a6fb22a0
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Thu May 21 20:41:55 2015 -0400

    Back-patch libpq support for TLS versions beyond v1.

    Since 7.3.2, libpq has been coded in such a way that the only SSL protocol
    it would allow was TLS v1.  That approach is looking increasingly obsolete.
    In commit 820f08ca we fixed it to allow TLS >= v1, but did not
    back-patch the change at the time, partly out of caution and partly because
    the question was confused by a contemporary server-side change to reject
    the now-obsolete SSL protocol v3.  9.4 has now been out long enough that
    it seems safe to assume the change is OK; hence, back-patch into 9.0-9.3.

    (I also chose to back-patch some relevant comments added by commit
    326e1d73, but did *not* change the server behavior; hence, pre-9.4
    servers will continue to allow SSL v3, even though no remotely modern
    client will request it.)

    Per gripe from Jan Bilek.

  commit 326e1d73
  Author: Tom Lane <tgl@sss.pgh.pa.us>
  Date:   Fri Jan 31 17:51:07 2014 -0500

    Disallow use of SSL v3 protocol in the server as well as in libpq.

    Commit 820f08ca claimed to make the server
    and libpq handle SSL protocol versions identically, but actually the server
    was still accepting SSL v3 protocol while libpq wasn't.  Per discussion,
    SSL v3 is obsolete, and there's no good reason to continue to accept it.
    So make the code really equivalent on both sides.  The behavior now is
    that we use the highest mutually-supported TLS protocol version.

    Marko Kreen, some comment-smithing by me