Add gpreload behave test

* gpreloaqd test where a table (public.t1) and
  view (gpreload_schema.t1) have the same name.

Add gpreload CI test jobs.
Co-authored-by: NEd Espino <eespino@vmware.com>

The four CM utilities gpinitsystem, gpinitstandby, gpaddmirrors, and
gpmovemirrors now have the relevant pg_hba.conf entries to allow WAL
replication to mirrors from their respective primaries across subnets.

There are two parts to this commit:
1). modify the CM utilities to add the pg_hba.conf entries to
allow WAL replication to mirrors across a subnet.
2). test the relevant CM utilities across subnets

The previous pg_hba.conf replication entry:
    'host replication $USER samenet trust'
does not allow WAL replication connections across subnets. We keep this entry
in order to support single-host development. We then add one replication line
for each primary and mirror interface address to new primaries and mirrors to
allow this. It looks like:
    'host replication $USER $IP_ADDRESS trust'
    or when HBA_HOSTNAMES=1
    'host replication $USER $HOSTNAME trust'
Further, if there is ever a failover and subsequent promotion,
replication connections can be made to the newly promoted primary from the host
on which the previous primary failed, because those addresses get copied over
to the new mirror during a pg_base_backup. We also add similar logic to support
cross-subnet replication between the master and standby. This behavior is
tested in the cross_subnet behave tests.

The cross_subnet behave tests assert that the replication connection is valid
by manually making the connection in addition to relying on segments being
synchronized, as a way to ensure that the pg_hba.conf file is being used.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Co-authored-by: NAdam Berlin <aberlin@pivotal.io>
Co-authored-by: NBhuvnesh Chaudhary <bchaudhary@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>

Exit with status 1 when dependencies cannot be sourced. Previously,
it was still returning 2 when dependencies could not be sourced.

Also updated the help text to document only exit codes 0 and 1.

Add tests for the user confirmation dialog that check empty, y, and no.

Add gpinitsystem to after_scenario tags_to_skip since gpstop fails,
in order to speed up our failure test cases.
Co-authored-by: NShoaib Lari <slari@pivotal.io>

Previously gpinitsystem returned 1 in case of warngings (2 in case of
errors). We have changed gpinitsystem to return 0 for warnings so that other
tools using gpinitsystem (such as gpupgrade) will not fail when gpinitsystem
generates warnings.

Also, gpinitsystem was exiting with non-zero return code if certain errors or
warnings had happened on any previous run of gpinitsystem that day. This has
also been fixed in this commit
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>

For commands called directly by the user, we provide the fix.

Since Behave and unit tests are supposed to behave as a normal user,
we do not provide the fix.  The fix is supposed to be done by the
commands themselves, and we want to test with an unmodified search_path
in the actual tests.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
Co-authored-by: NNikolaos Kalampalikis <nkalampalikis@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>

Use pg_ctl to allow background worker connections during smart shutdown
and remove application_name whitelist, which was checked to allow
smart shutdown to proceed. When pg_ctl timeouts or the user decides
to Control-C , prompt user to continue shutting down in smart mode
while waiting forever, or shutdown in fast or immediate mode.
We allow the user to send a SIGINT to interrupt a long-running smart
mode shutdown and resend either a fast or immediate shutdown.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
Co-authored-by: NShoai Lari <slari@pivotal.io>
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Co-authored-by: NNikolaos Kalampalikis <nkalampalikis@pivotal.io>

* Revert "CI: skip CLI behave tests that currently fail on ubuntu18.04"

This reverts commit 6a0cb2c6.
This commit is to re-enable behave test for centos/ubuntu

* Ouput the same error message format

gppkg for rpm and deb outputs error messages when consistency is broken, but
the message is not the same, which make a pipeline fail.

* Update gppkg for behave test and fix some minor error in gppkg

Now, the binary sample.gppkg is removed, instead we add sample-*.rpm and sample-*.deb.
Because sample.gppkg is platform specific and GP major version sensitive. The uploaded
rpm/deb files are available unless the rpm/deb file is incompatible on the specific
platform. GP_MAJORVERSION is dynamically retrieved from a makefile installed in the gpdb
folder, so the gp major version in the gppkg will be always correct.
sample.gppkg will only be generated when gppkg tag is provided

We skip the following tests for the following reasons:
   - gpinitsystem: TZ interpretation difference for TZ '' vs unset
   - gppkg: underlying package zip file contains wrong OS version
   - gpactivatestandby: cause unknown
   - gprecoverseg: bash pipe interpretation difference
   - gpconfig: demo cluster has LOCALE set to POSIX, not UTF-8

We plan to fix these in a subsequent commit.
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>
Co-authored-by: NNikolaos Kalampalikis <nkalampalikis@pivotal.io>

This commit adds some tests for gpssh-exkeys, as we want to be confident
in our test coverage before refactoring it to remove Paramiko.
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>

Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>

To run behave tests, behave now needs to be installed in the PATH. The
correct version of behave can be installed with pip using

    pip install -r gpMgmt/requirements-dev.txt

For the CI tests, use virtualenv, which allows behave to be installed
while still using greenplum PYTHONPATH for our vendored python modules.
This is more of a problem for platforms like centos6 where the system
python is a different version than the vendored version.
Co-authored-by: NKalen Krempely <kkrempely@pivotal.io>
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NDavid Krieger <dkrieger@pivotal.io>
Co-authored-by: NJacob Champion <pchampion@pivotal.io>

Skip gprecoverseg tablespace tests until tablespaces successfully work
with pg_rewind.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>

Verify tablespaces work for both a incremental and full recovery.
Co-authored-by: NJacob Champion <pchampion@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>

We add gpmovemirrors tests and add them to the pipeline as well.
Using behave tags, some of the tests can run locally on a demo
cluster.  We also introduce Behave tag logic to choose which
tests to run.

<Co-Authored-By> Kalen Krempely <kkrempely@pivotal.io>
<Co-Authored-By> Mark Sliva <msliva@pivotal.io>

Acceptance tests for gpstate were lacking, making it difficult to prove
that the previous commit didn't break anything. Start remedying that
here -- these tests are not exhaustive by any means, but they're a good
start.

The new Concourse job is called "gpstate".

The new (or modified) behave steps are as follows:

  - "user kills all {type} processes [with {signal}]"

    This step previously worked only with primary processes, and would
    only send SIGKILL. It now allows either primary or mirror to be
    specified, along with the specific signal to be sent. If no signal
    is specified, SIGTERM will be sent.

  - "a standard local demo cluster is created"

    This step creates a standard 3 content demo cluster with mirrors and
    master standby.

  - "a standard local demo cluster is running"

    This step checks that a standard demo cluster as defined above
    is currently running. If it is not, it creates one. This leads to a
    speed improvement in the tests that use this step.
Co-authored-by: NMark Sliva <msliva@pivotal.io>
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>

These scenarios come from the walrep_1 TINC Makefile target. They have
been designed to run on both singlenode and multinode environments.
The tests have not been removed from TINC yet because some walrep_2 TINC
tests have dependencies.
Co-authored-by: NJimmy Yih <jyih@pivotal.io>
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>

Prior to this commit, gpaddmirrors was missing two bits of work which
were previously done by gpinitsystem/gpcreateseg. When adding mirrors to
a cluster, the pg_hba.conf on primary segments has to be modified to
allow replication connections, e.g. for pgbasebackup. And after the
mirrors are built, the catalog has to be updated to reflect the new
cluster config.
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>
Co-authored-by: NJim Doty <jdoty@pivotal.io>

In addition to migrating the TiNC tests notable changes were made:
+ Made gpexpand tests for a cluster with mirrors work
+ Migrated Behave tests for duration and end time as behave tests
+ Add gpexpand behave tests to pipeline template
+ Cleanup some dead code in environment.py
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Co-authored-by: NJim Doty <jdoty@pivotal.io>
Co-authored-by: NKevin Yeap <kyeap@pivotal.io>
Co-authored-by: NNadeem Ghani <nghani@pivotal.io>
Co-authored-by: NShoaib Lari <slari@pivotal.io>

The data_checksums GUC setting should be the same as the master.

The existing test for gpinitstandby is modified to run on a single host.
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>
Signed-off-by: NC.J. Jameson <cjameson@pivotal.io>

Workaround a problem discovered by a client that noticed intermittent errors for gpssh when some nodes became very cpu-bound.

In particular, we override the way the ssh command prompt is validated
on a remote machine, within gpssh. The vendored module 'pexpect' tries to match 2 successive prompts
from an interactive bash shell.  However, if the target host is slow from
CPU loading or network loading, these prompts may return late.

In that case, the override retries several times, extending the timeout from the
default 1 second to up to 125 times that duration.

Experimentally, these added retries seem to tolerate about 1 second delay, testing with a 'tc' command that slows network traffic artificially.

The number of retries can be configured.

--add unit tests to verify happy path of ssh-ing to localhost
--add module for gpssh, for overriding pexpect (pxxssh)
--add readme to describe testing technique using 'tc' to delay network
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

* Enable to run behave tests with the installed binary instead of source
* code.

* This is an effort to enable us to run behave tests without relying on
* the source code and instead use the GPDB installed in the system as
* the behave tests are integration tests.

No need to create a whole new library for this. It's included in every
installation anyway, as part of gp_toolkit. Might as well just include
it in the backend, and make the build slightly simpler.

Usage: minirepro <database> [options]

Options:
  --version             show program's version number and exit
  -?, --help            Show this help message and exit
  -h HOST, --host=HOST  Specify a remote host
  -p PORT, --port=PORT  Specify a port other than 5432
  -U USER, --user=USER  Connect as someone other than current user
  -q QUERY_FILE         file name that contains the query
  -f OUTPUT_FILE        minirepro output file name

Closes #157.