* docs: Updatable cursors support

* fix PL/gpSQL typos

* Move updatable cursors release note to changed features and note that it is a clarification and doc change.

The test output with ORDER BY was not fully defined, because there were
some rows with the same key column, but different second column. Remove
ORDER BY from those queries, and let gpdiff.pl mask the difference in row
order instead.

Commit 20332d98 disabled the ALTER TABLE
CLUSTER ON syntax. This removes those tests from the gptransfer test
suite.

We have most of these exact same queries in
src/bin/gpfist/regress/input/exttab1.source already, and the rest are not
very interesting. Some of the other queries not in exttab1.source already
queries regular tables, rather than external tables, or were so similar
to existing queries that they don't provide any meaningful extra coverage.

The reason to attack test this right now is that commit 743b886a broke
the test. It'd be trivial to just fix the expected output, but let's rather
remove it altogether. (Most of the other query* files in this directory
could probably be removed too, or moved to src/bin/gpfdist/regress, but
I'll leave that for tomorrow.)

Make the test deterministics. Similar to
https://github.com/greenplum-db/gpdb/pull/1655

partition.

Analyze root partition should not generate statistics for the leaf or
midlevel partitions, immaterial of the value of the GUC
optimizer_analyze_midlevel_partition.

This fix makes it consistent with our docs.

* updates to language create/alter/drop allowing db owners

* identify pl/r and pl/python as untrusted

In order to have the PR status properly reported to github, we extracted
out the action of reporting "success" into a separate job. Each task
before that reports "failure"s themselves.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

- The "custom" compile typically takes a few minutes less than the main
  compile, so this won't delay feedback
- If the custom compile goes red, it will issue a failure to the github
  hook. But without this change, the ICW would still run and possibly
  issue a later "success" to github.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

There is no legitimate reason for these to be called with an invalid type.
Turn the checks into ERRORs.

Tuplesorts take care of cleaning up any memory and temporary files on
abort, no need to have PG_TRY()-PG_CATCH() blocks to do that. These blocks
were introduced a long time ago, in a patch related to workfile caching,
but we don't do workfile caching anymore anyway.

Commit 74a1efd7 missed including the _optimizer expected output for
the test ported from Bugbuster to ICW. This fixes by including the
same hunks remembered in the planner output file.

Since there is no GPDB specific trigger suite in ICW, and this bug
was originally part of the Rio sprint back in the day, I added it
to qp_misc_jiras. The suite already contained a version of the test
so extended it to to include the segment redistribution check as
well rather than duplicating for essentially the same test. This
test relies on the rows being allocated to segment id 0, which
should hold true for test setups with 2/3 segments. Should this be
a problem going forward the test should be rewritten to check for
all rows being co-located on a single segment but for now the more
readable version is kept.

It's worth noting that the expected output in the Bugbuster suite
has the incorrect behaviour remembered, but inside ignore blocks
so this bug has never actually been tested for.

Use a macro to encapsulate the pattern of calling curl_easy_setopt(), and
ereporting if it fails.

There was a rather ugly mechanism in fileam.c, to ensure that the
utl_fclose() function was called for any still-open external table, on
abort. That relied on a global variable and assumed that there can be only
one external table open at a time, which seems like a shaky assumption.

With this patch, utl_file() is no longer guaranteed to be called on abort.
Instead, make the UTL_FILE implementations responsible for cleaning up any
low-level resources, like file descriptors or handles to external libraries
(like libcurl), themselves. Use the ResourceOwner mechanism for that.

For curl- and EXECUTE-style external tables, register a new ResourceOwner
hook to close with the libcurl handles, and launched subprocesses, on
abort. For file external tables, modify fstream.c to open the underlying
file with OpenTransientFile(), so that it gets closed automatically on
abort.

This changes the url_fopen() API, so that it never returns a NULL. Instead
of returning NULL, and reporting an error in the *response_code/string
arguments, make url_fopen() always report errors directly with ereport().
That makes the callers simpler, and allows the implementations to flexibly
report whatever details of the error they have available.

OpenTransientFile() opens an arbitrary file, like open(), but will track
the file descriptor by the resource owner mechanism, so that it is
automatically closed on abort. We already had a similar mechanism for
temporary files, and for files opened with fopen(), this fills the gap of
when you want a plain file descriptor rather than a FILE *.

I need this for the next commit, which will use this to automatically close
files opened by the fstream.c on abort.

This is a backport of upstream commit 1f67078e. I only backported the
addition of the new function, not the changes to the various callers to
use the new function, to minimize merge conflicts.

This moves all the implementation-specific parts of the big UTL_FILE struct
out of the common url.h file, into the implementation-specific .c files.
Makes the code more readable, as you have all the implementation-specific
stuff in a single .c file now.

To avoid memory leaks. The callers better be in a sufficiently long-lived
memory context.

It's not necessary to clear the error buffer on every call. Also,
GPOS_NEW_ARRAY() is a pretty slow way of clearing memory. This greatly
reduces the overhead of planning simple queries with ORCA.

On my laptop, this reduces the time for planning "SELECT 123;" from
about 60 ms to 5 ms.

I'm not sure what exactly the purpose of each of these tests are, and they
probably need more culling and cleanup, but for now, just move them out of
bugbuster, so that we can get rid of bugbuster as a separate suite.

I don't see any reason to expect that this would behave differently with
an external table than a normal one.

Move the test from bugbuster to the normal regression suite.

This moves 'spi', 'spilltodisk', and 'wrkloadadmin' tests. The goal is to
get rid of bugbuster as a separate suite altogether, and this commit is one
step towards that goal.

Signed-off-by: NJingyi Mei <jmei@pivotal.io>

- /etc/init.d/sshd doesn't exist
- disable newer host key types in sshd_config
Signed-off-by: NTushar Dadlani <tdadlani@pivotal.io>

Also update all uses of the name in the code as well as pgindent
file.

It was not enabled, because there were a lot of errors in it, because of
features that have been disabled in Greenplum. Like enforcing foreign keys,
INSERT RETURNING, and doing SPI in functions in segments. Those limitations
make these tests a less interesting than in the upstream, but there are
still many good tests in there too. Like the test for anonymous DO blocks,
that are not tested by any other tests.

The .sql file was almost identical to the upstream version. Clean it up
to be even closer, by re-enabling some commented queries in it, removing
the duplicated tests for RETURN QUERY, removing the READS/CONTAINS/MODIFIES
SQL DATA noise words etc.

Memorize the expected output as it is. That's quite different from the
upstream, because of all the disabled functionality, but also because of
some cosmetic differences in error messages.

The 'schema_topology' test contained a copy of the for_vect(), so remove
that, now that we run the real thing.

There was no difference between the "vacuumed data" and "unvacuumed data"
tests, except that the first one performed a vacuum at the end. I think
these databases were meant to be used for some kind of follow-up tests,
comparing the behaviour of something with or without the vacuum. But we
have no such follow-up tests, so this is rather pointless.

It's questionable whether it makes sense to have even one copy of this,
but I kept it for now.

These tables and views were not used for any further testing. They are
uninteresting in themselves, we have a lot more interesting tests
elsewhere for tables, views, and comments on them.

In the test for upper case tables, one INSERT is enough to confirm that it
works.

We have more comprehensive testing of transactions in the 'transactions',
and elsewhere. The 'employee' case is quite unremarkable, so just remove
it.

That's where we have all the other tests for this feature, let's keep
them together.

Don't bother to create a separate table for each ALTER test.

And at the beginning, move it to a more logical place.

* Calling current_database() inside an ignore block doesn't do anything
  interesting. We have a test, which is not inside an ignore block, already
  in qp_functions.

* No need to switch to template1 before dropping "admin" role. We can drop
  it from within the regression database, like all the other test roles.

* Setting optimizer_disable_missing_stats_collection had no effect on any
  of the actual tests, because switching the connection with \c, on the
  very next command, resets all GUCs.

Most of these queries were already in the qp_correlated_query test. A few
were not, moved those instead of outright removing them.

This removed the last queries from schema_topology whose output was
different with ORCA than without ORCA, so remove the ORCA-specific expected
output file.

We have tests like this in the main test suite's 'external_table' test,
and in gpfdist's regression suite. Note that these external tables were
never queried, only created and dropped.

These roles were dropped just a few lines earlier already.