fix tests, add init_file to ignore GPDB segment output
use init_file from main regression tests, fix expected results
Requires GPDB to be configure with --with-openssl to pass

qp_olap_group2 did an elaborate dance with optimizer_log=on,
client_min_messages=on, and gpdiff rules to detect whether the queries
fell back to the traditional planner. Replace all that with the new simple
optimizer_trace_fallback GUC.

Also enable optimizer_trace_fallback in the 'gp_optimizer' test. Since this
test is all about testing ORCA, seems appropriate to memorize which queries
currently fall back and which do not, so that we detect regressions where
we start to fall back on queries that ORCA used to be able to plan. There
was one existing test that explicitly set client_min_messages=on, like the
tests in qp_olap_group2, to detect fallback. I kept the those extra logging
GUCs for that one case, so that we have some coverage for that too, although
I'm not sure how worthwhile it is anymore.

In the passing, in the one remaining test in gp_optimizer that sets
client_min_messages='log', don't assume that log_statement is set to 'all'.

Setting optimizer_trace_fallback=on for 'gp_optimizer' caught the issue
I fixed in previous commit, that one of the ANALYZE queries still used ORCA.

For the same reasons we disabled ORCA in all the other ANALYZE queries.
Missed this one.

I belive we don't want to use ORCA for any of the ANALYZE's internal
queries, current or future ones, so move the disabling of ORCA one level
up, into a wrapper around analyze_rel().

[ci skip]

* docs: fix broken links and permanent redirects

* fix doubled slashes in URLs

add_second_stage_agg() builds a SubQueryScan, and moves the "current" plan
underneath it. The SS_finalize_plan() call was misplaced, it needs to be
called before constructing the new upper-level range table, because any
references to the range table in the subplan refer to the original range
table, not the new dummy one that only contains an entry for the
SubQueryScan. It looks like the only thing in SS_finalize_plan() processing
that accesses the range table is processing a TableFunctionScan. That lead
to an assertion failure or crash, if a table function was used in a query
with an aggregate.

Fixes github issue #2033.

* added tests from upstream for oracle_compat

backport of upstream commit 607b2be7, from PostgreSQL 8.4? 

This removes uninteresting tests, and tests covering functionality
which is tested elsewhere. The superflous search_path setting is
removed, the amount of tables created reduced and the test data size
reduced. The resource_queue test was more or less duplicated already
but had enough interesting bits to warrant a move to the appropriate
suite.

This shaves roughly 20% of the runtime of tbe test suite.

Commit f004451e forgot to use @abs_srcdir@ in the expected output.

And remove the rest of old partition_exchange TINC test. We already had
tests in 'partition' test file, for validating that the values in the
exchanged table are valid for the partition (search for "-- validation").

There were also tests for exchanging a partition in a hash partitioned
table, but the validation hasn't apparently been implemented for hash
partitions, and they're not supported anyway, so that doesn't seem very
interesting to test.

The old cdbfast test suite had this, move to ICW.
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

The old cdbfast test suite had tests for these, looks good, move to ICW.
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

* GPDB DOCS Configuring clients w/ Kerberos or Active Directory

* GPDB DOCS updates to support client kerb/AD topics

[ci skip]

This commit brings the ps_status.c file closer to upstream and reverts all the
changes made to support gpkill utiltity which is obsolete now. Full removal of
GPKILL utility code will done as part of separate commit.

Seems 8.3 merge messed up the ps display logic, ends up missing to print useful
information for process like what command its running, waiting for lock,
etc. Also introduces newline causing the output tonot look tidy. It seem was
broken much more for MAC than other platforms, this commit fixes to correctly
report the useful process information.

I'm not sure if it's really worthwhile to test all of these with and
without CTE inlining, but kept it for now.

Clean up tuplesort code for 8.4 merge
  * Move private tuplesort_mk.h to tuplesort__mk_details.h
  * Move mksort API to tuplesort_mk.h
  * Move GPDB introduced tuplesort API to tuplesort_gp.h
  * Reorganize tuplesort.c to move all GPDB related functions to the end.
Signed-off-by: NKarthikeyan Jambu Rajaraman <karthi.jrk@gmail.com>

`-[ "$str" = "foo" ] && $(MAKE) -C bar` ignores errors while making bar,
better to use full `if-then` clause.

I modified the test slightly, to eliminate the sleep, making the test run
faster.

We have a test for this in the main test suite, in the 'partition' test.
This one:

create table rank_damage (i int, j int)  partition by range(j) (start(1)
end(10) every(1));
-- [other tests]
alter table rank_damage_1_prt_1 drop constraint rank_damage_1_prt_1_check;

> src/bin/gpfdist/ext is a submodule, from
> https://github.com/greenplum-db/gpfdist-ext. It contains tarballs of libapr,
> libevent, openssl and zlib, and a bunch of patches. Why do we build gpfdist
> like that? On my laptop, I never use those ancient, patched, versions, I
> link with the normal libraries that come with my distribution, and it works
> fine.
>
> I think that has something to do with Windows, because I also found this in
> src/interfaces/libpq/Makefile:
>
> ifeq ($(PORTNAME), win32)
> SHLIB_LINK += -L../../../../gpAux/ext/win32/kfw-3-2-2/lib
> -L../../../src/bin/gpfdist/ext/lib -lgssapi32 -lshfolder -lwsock32 -lws2_32
> -lsecur32 $(filter -leay32 -lssleay32 -lcomerr32 -lkrb5_32, $(LIBS))
> endif
>
> But if that's only for Windows builds, why do we try to build it on Linux?
>
> - Heikki

Thanks Heikki for bringing this up, we created a story to review the
gpfdist/ext submodule, this commit is to remove it from other OSes first.

It's safe actually because the functions we created have two arguments
at most. Fix it anyway in case three arguments functions are introduced
in future.

/tmp/build/0e1b53a0/gpdb_src/contrib/dblink/dblink.c: 241 in dblink_connect()

________________________________________________________________________________________________________
*** CID 163942:  Null pointer dereferences  (FORWARD_NULL)
/tmp/build/0e1b53a0/gpdb_src/contrib/dblink/dblink.c: 241 in dblink_connect()
235      * Create a persistent connection to another database
236      */
237     PG_FUNCTION_INFO_V1(dblink_connect);
238     Datum
239     dblink_connect(PG_FUNCTION_ARGS)
240     {
>>>     CID 163942:  Null pointer dereferences  (FORWARD_NULL)
>>>     Assigning: "connstr" = "NULL".
241             char       *connstr = NULL;
242             char       *connname = NULL;
243             char       *msg;
244             PGconn     *conn = NULL;
245             remoteConn *rconn = NULL;
246
Signed-off-by: NAdam Lee <ali@pivotal.io>
Signed-off-by: NYuan Zhao <yuzhao@pivotal.io>

	commit cae7ad90
	Author: Tom Lane <tgl@sss.pgh.pa.us>
	Date:   Mon Sep 22 13:55:14 2008 +0000

	    Fix dblink_connect() so that it verifies that a password is supplied in the
	    conninfo string *before* trying to connect to the remote server, not after.
	    As pointed out by Marko Kreen, in certain not-very-plausible situations
	    this could result in sending a password from the postgres user's .pgpass file,
	    or other places that non-superusers shouldn't have access to, to an
	    untrustworthy remote server.  The cleanest fix seems to be to expose libpq's
	    conninfo-string-parsing code so that dblink can check for a password option
	    without duplicating the parsing logic.

	    Joe Conway, with a little cleanup by Tom Lane

Some notable differences:

* For the last "appendonly_verify_block_checksums_co" test, don't use a
  compressed table. That seems fragile.

* In the "appendonly_verify_block_checksums_co" test, be more explicit in
  what is replaced. It used to scan backwards from end of file for the byte
  'a', but we didn't explicitly include that byte in the test data. What
  actually gets replaced depends heavily on how integers are encoded. (And
  the table was compressed too, which made it even more indeterministic.)
  In the rewritten test, we replace the string 'xyz', and we use a text
  field that contains that string as the table data.

* Don't restore the original table file after corrupting. That seemed
  uninteresting to test. Presumably the table was OK before we corrupted
  it, so surely it's still OK after restoring it back. In theory, there
  could be a problem if the file's corrupt contents were cached somewhere,
  but we don't cache AO tables, and I'm not sure what we'd try to prove by
  testing that, anway, because swapping the file while the system is active
  is surely not supported.

* The old script checked that the output when a corrupt table was SELECTed
  from contained the string "ERROR:  Header checksum does not match".
  However, half of the tests actually printed a different error, "Block
  checksum does not match". It turns out that the way the old select_table
  function asserted the result of the grep command was wrong. It should've
  done "assert(int(result) > 0), ..." rather than just "assert(result > 0),
  ...". As written, it always passed, even if there was no ERROR in the
  output. The rewritten test does not have that bug.

Tests stops standby and checks if walsender is gone along with other checks, but
walsender may take some time to go away, hence add retry for checking. This
makes test robust, as failurs were seen some times in CI.

Tests deleted the standby base directory and checked promote should fail after
it. It would be MAGIC if that succeeds and even if does its good then, ideally
no reason for it to go through hence deleting the test.

We rely on the catalog Oid being present to be able to search for
preassigned Oids to dispatch, so enforce the catalog relation Oid
and error out if missing.

These targets are not invoked directly anywhere so remove them.

The fault `twophase_transaction_commit_prepared` would cause the segment to
PANIC while writing a `COMMIT PREPARED` record on the segment. The master would
then retry the `COMMIT PREPARED` while the postmaster was still resetting on the
segment causing the master itself to `PANIC`. This patch introduces a new fault
injector `finish_prepared_before_commit`  which will not cause a `PANIC`, but
just error out since the intent of the test is to ensure that the retry works
correctly.

The `with time zone` functionality is tested in several other storage test
suites, such as those in access_methods, aoco_alter, aoco_compression,
pg_two_phase etc. Therefore, we are removing the `with time zone` columns form
the storage/walrepl/crash tests.

DDBoost 3.0 will be end of support soon so we want to use the newer
version.

Authors: Chris Hajas, Karen Huddleston, and Todd Sedano

[ci skip]

This test stage ran filerep_end_to_end test suite work-load and peformed crash
testing with the same. Takes around 2 hours for completion, It runs with
extremely heavy lifting things like gpfdist, resource queues, etc... which don't
seem of too much interest to crash testing. In long run really need much
simplfied crash recovery testing by creating wrk load to create all kinds of
xlog records in system and recover with it instead of piggy-backing for
work-loads on these already heavy test jobs.

This applies to tests that create a database with checkpoint skip fault
injected. Create database now requests checkpoint and if it is skipped, any
modifications to the template database pending to be written are potentially
lost. The database template0 is guaranteed to not have any modifications
because user connections are not allowed on it so use the same for testing.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Also, update some answer files.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

* Add note about updatable cursor restriction

* Update/delete a single row. Remove mystery <image> tag.

* Remove suggestion that cursors can be updated with a dynamic PL/pgSQL execute statement.

* Edit out passive voice.

The count(*) on exttab_cte_1 errorlog has failed without explanation,
and without being reproducible, both locally and on the CI pipeline.
Since the errorlog might hold insights into what went wrong, print it
in an ignored block to preserve it for debugging before it's lost.
This will include the errorlog output prefixed with GP_IGNORE in the
regression.diffs file.