Previously was using master.
Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>
Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io>

(cherry picked from commit 5e918bf6)

Make "demo_cluster" the default tag (ie: no tag). That is, when a
scenario does not specify a behave tag it uses the demo cluster
environment to run. Use the concourse_cluster tag if a scenario requires
a remote cluster. Concourse runs the scenarios with no tag that use a
demo cluster by filtering for --tags=~concourse_cluster.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 40028cf4)

Regenerate pipeline from CLI_BEHAVE_TESTS refactor.

(cherry picked from commit 5dc3bfaa)

Renamed to CLI_BEHAVE_TESTS for clarity

(cherry picked from commit 87c4a52c)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 735da14f)

Now that the previous commits have placed these tests in CLI_TESTS,
we can remove references to this symbol.

(cherry picked from commit 55add2c5)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit ffce7734)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit d67ecc3f)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit e3457667)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit a5130acb)

Now that the previous commits have placed these tests in CLI_TESTS,
we can remove references to this symbol.

(cherry picked from commit 8988618e)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit f676d2cc)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 37a09d64)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 8bb12f08)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.

(cherry picked from commit e7b5f462)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 642c4c8f)

Tag existing tests with @demo_cluster and @concourse_cluster
annotations. The @demo_cluster tag denotes the scenario can run
locally. The @concourse_cluster tag denotes the scenario that requires
a remote cluster.
Co-authored-by: NJamie McAtamney <jmcatamney@pivotal.io>
Authored-by: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 53aad9c3)

Templatize gpmovemirrors into a single concourse job to run both
concourse_cluster and demo_cluster tags separate tasks.
Co-Authored-By: NKalen Krempely <kkrempely@pivotal.io>
(cherry picked from commit 0df65394)

* Remove unused IVYREPO_ variables for CentOS 6/7

Ivy is no longer used for CentOS 6/7.
Authored-by: NAmil Khanzada <akhanzada@pivotal.io>

* Remove unused IVYREPO_ variables for Coverity

Ivy is no longer used for CentOS 6/7 and will go away.
The coverity pipeline is also puased at this point in time, but we'll
clean this up now.

(cherry-picked from c4c99bb0)
Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

* Fix bin_gpdb_centos6_icw_green versioned_file to store artifacts in gpdb_branch_6X_STABLE
* Generate pipeline

For GP 6 beta, the release engineering team is removing the apr-util
package from the list of bundled dependencies. Users will be asked
to provide their own apr-util package, which can differ in the version
on each platform. So it is necessary to verify gpperfmon is workable
with the platform provided apr-util on each supported platform.

Originally, gpperfmon test is done in CLI test suite, only covers centos6.
This commit changed it to a dedicated suite in order to test multiple
platforms.
Note: on SLES12 does not need to install libapr-util1 to run gpmmon

We add gpmovemirrors tests and add them to the pipeline as well.
Using behave tags, some of the tests can run locally on a demo
cluster.  We also introduce Behave tag logic to choose which
tests to run.

<Co-Authored-By> Kalen Krempely <kkrempely@pivotal.io>
<Co-Authored-By> Mark Sliva <msliva@pivotal.io>
(cherry-picked from fdf8b611)

It can handle defaults more correctly.
Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>
Co-authored-by: NKris Macoskey <kmacoskey@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>

ci: remove `-oss` from pattern used to parse version output

Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

Authored-by: NKris Macoskey <kmacoskey@pivotal.io>

This job is used to test online expand, it will create a cluster
with two segments, then expand to 3 and run all ICW to check
whether the cluster is OK after expansion.
As restart is forbidden in online expand test, so we exclude all
the case which contains restart opertaion. But if someone add a
new test with restart, the job may fail. Manual intervention to
exclude the test is needed.
So we move this job to our own dev pipeline to reduce the impact
on prod pipeline.

  * Add configure_flags_with_extensions to the pipeline template file
  * regenerate the gpdb_master pipeline and use fly to set it up.
Co-authored-by: NBob Bao <bbao@pivotal.io>
Co-authored-by: NNing Fu <nfu@pivotal.io>

* Build necessary extensions for release

* Add new  '--enable-debug-extensions' configure option, if provide the option,
  the extensions include:

      gp_distribution_policy
      gp_internal_tools
      gp_sparse_vector
      gp_replica_check
      gp_inject_fault
      gp_debug_numsegments

* Rename configure_flags to configure_flags_with_extensions

  The configure_flags_with_extensions SHOULD BE set to

      configure_flags_with_extensions: "--enable-cassert --enable-debug-extensions"

    in these secrets files

       secrets/gpdb_master-ci-secrets.dev.yml
       secrets/gpdb_master-ci-secrets.prod.yml
Co-authored-by: NBob Bao <bbao@pivotal.io>
Co-authored-by: NNing Fu <nfu@pivotal.io>

The gp-addon repo was used to provide the quick-lz artifacts but this
has been moved in to the Docker images used for building GPDB6.
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NBradford D. Boyle <bboyle@pivotal.io>

We are in the process of renaming the secrets files in
gp-continuous-integration to use the .prod suffix consistently.
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>

Existing code was globbing for libstdc++.so.* and copying it to gpdb
lib, we are now ignoring the .py since that should not be vendored.

* also cleaned up bash, no need for trailing '/.' on copy. Thanks d#!

[#164347190]
Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>
Co-authored-by: NNandish Jayaram <njayaram@pivotal.io>

- add a rename task to make the artifacts use the new naming convention
- keep pushing to the s3 resources so that other pipelines using those
  published artifacts don't break (although those s3 resources are now
  deprecated).
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NBen Christel <bchristel@pivotal.io>

The -D option was retired in commit
ec708f34, but we did not update the
tests accordingly.

Now we retire the -D option from gpexpand tests, too.

This commit moves all of the intermediate concourse resources (which are
only used internally in the pipeline) to now use GCP buckets as opposed
to AWS S3 storage.

The ((pipeline-name)) variable added to gen_pipeline.py is needed for
supporting the naming conventions that are used for new GCS objects,
which namespace artifacts by pipeline.
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>

gpload,gpfdist,createdb,dropdb,createuser,
dropuser,createlang,droplang,psql,pg_dump,
and pg_dumpall, these components are all
packaged into clients package.

1)gpAux/Makfile:
  * Remove loaders in gpAux/Makefile.
  * Now clients use some system libraries,
    so no need to copy these denpendencies
    into lib.
2)Modify compile_gpdb_clients.bash to export
  gpdb_clients bin package for building clients
  rpm.
3)Remove unused files under gpAux.

This reverts commit 203a0892.

This reverts commit 8758e2bf.

This commit moves all of the intermediate concourse resources (which are
only used internally in the pipeline) to now use GCP buckets as opposed
to AWS S3 storage.
Co-authored-by: NBen Christel <bchristel@pivotal.io>
Co-authored-by: NAmil Khanzada <akhanzada@pivotal.io>
Co-authored-by: NSambitesh Dash <sdash@pivotal.io>
Co-authored-by: NOliver Albertini <oalbertini@pivotal.io>