* porting hstore contrib module docs from postgres 8.4

* adding link to hstore doc from data type refernce

* adding placeholder install instructions

* removing info about GIN index support

Some unreentrant functions are invoked in signal handler.

To fix this bug: change signal handler to asynchronous modle.

using global variable "sig_flag" to store last signal state,every 1s polling or
after failed happen in block IO function(such as send/ receive) check "sig_flag".

  fix bug : gpload does not stop after informatica sends exit call

    Some unreentrant functions are invoked in signal handler.

    To fix this bug: change signal handler to asynchronous modle.

    using global variable "sig_flag" to store last signal state,every 1s polling or
    after failed happen in block IO function(such as send/ receive) check "sig_flag".

Introduced by commit 522c7c09, spotted by Coverity.

In PostgresQL, the DO block is after RemoveFuncStmt. It was slightly
misplaced when it was backported from PostgreSQL 9.0.

Any error that the user is expected to see should be using the
ereport() macro rather than its older cousin elog(). Also avoid
closing resources just before erroring out as the error cleanup
will be handled automatically, and move a long message to have
an errhint instead.

In binary swap test, new binary is replaced by old binary and then run
pg_dump, however, pg_dump will still try to load new regress.so, so if
regress.so contain new symbols only belong to new binary, it will report
an error. For resGroupPalloc() ifself, IsResGroupEnabled or
IsResGroupActivated make no much difference, so to make binary swap pass,
we still use IsResGroupEnabled.

This means that hstore will be compiled and installed by a top level "make",
and the regression tests are run as part of "make installcheck-world".

To make the regression tests pass on GPDB, where we have
standard_conforming_strings=on by default.

This is a cherry-pick of upstream commit:

commit 0e22e08f
Author: Robert Haas <rhaas@postgresql.org>
Date:   Tue Jul 20 01:22:52 2010 +0000

    Make hstore regression tests independent of standard_conforming_strings.

    Per buildfarm.

Our GIST version doesn't pass the 'recheck' argument to 'consistent'
functions yet. That will come in commit 9b5c8d45, as part of the PostgreSQL
merge.

I'm not sure if we broke this during the 8.3 merge, or if it was broken
even before that, but our version of heap_form_tuple was confused over
what the isnull argument is. The difference between heap_form_tuple()
and heap_formtuple() is the values used in the "isnull" array, but
our heap_form_tuple passed a 'char' array to the internal heap_fill_tuple()
function, which assumes a 'bool' arry.

That caused an assertion failure in hstore regression tests (and almost
certainly a crash or at least wrong results without assertions, though I
didn't check.)

To fix, take the version of heap_form_tuple() from PostgreSQL 8.4, which
converts the 'char' array to a 'bool' array, and then calls the new
heap_formtuple() function.

SPIMemReserved was initialized in CreateSharedMemoryAndSemaphores which
make no sense, this commit put it under InitResManager where the resource
memory policy is set.

As a pleasent side-effect, the window specification in a
"func_expr OVER (window_spec)" rule now gets its parse location set
correctly. Thanks to that, a few error messages now give a user-friendly
error location line.

GPDB 5 doesn't have the pg_get_functiondef server function needed by \ef,
so it doesn't work.

Fixes github issue #3076.

This is a backport of an upstream commit that conveniently fixes this:

commit 2463f5da3ccd67b266ee7a9b8c0906221a501b9e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Thu Nov 10 18:37:05 2011 -0500

    Throw nice error if server is too old to support psql's \ef or \sf command.

    Previously, you'd get "function pg_catalog.pg_get_functiondef(integer) does
    not exist", which is at best rather unprofessional-looking.  Back-patch
    to 8.4 where \ef was introduced.

    Josh Kupershmidt

The resource group is enabled but not initialized on auxiliary processes
and special backends like ftsprobe and filerep, previously we performed
resource group operations no matter resource group is initialized or not
which leads to some unexpected error.

Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>
Signed-off-by: NAdam Lee <ali@pivotal.io>

* Correct the maximum content id to be number of segments less 1

* Fix additional places that suggest content id are numbered from 1.

Previously, we were only checking localhost for standby heap checksum in
gpstart. This is not the case as the standby is usually on a remote host.
gpstart raises an error due to not finding the pg_control file of standby locally.

Fixed by always doing a remote host call for the standby master.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

Signed-off-by: NJemish Patel <jpatel@pivotal.io>

Use ereport(), with a proper error code, for errors that are "expected" to
happen sometimes, like missing configuration files, or network failure.

That's nicer in general, but the reason I bumped into this is that internal
error messages include a source file name and line number in GPDB, and if
those error messages are included in the expected output of regression
tests, those tests will fail any time you do change the file, so that the
elog() moves around and the line number changes.

An assertion with a side effect may alter the main codepath when
the tree is built with --enable-cassert, which in turn may lead
to subtle differences due compiler optimizations and/or straight
bugs in the side effect. Rewrite the assertions without side
effects to leave the main codepath intact.

This commit verified that connections in utility mode should not be
governed by resource group

This commit verified that cursors, prepare and statements within pl functions didn't
trigger self-deadlock by the concurrency control of resource group.

We used to use sequence scan on pg_resgroupcapability for functions that
need to do a full scan of pg_resgroupcapability. Problem is accessing
in this way will take a long time after pg_resgroupcapability table was
updated/deleted million times as our stress tests do, pg_resgroupcapability
was filled of invalid blocks and sequence scan wasted lots of time to bypass
those blocks. Using index scan on it can resolve this problem.

This commit contain all kinds of parallel tests include combination of CREATE,
DROP, ALTER and Queries, this commit depends on dblink component to run
queries concurrently.
Signed-off-by: NRichard Guo <riguo@pivotal.io>

Don't send nonsense '\n' characters just for counting, let segments
report how many rows are processed instead.
Signed-off-by: NMing LI <mli@apache.org>

GUC value is `gp_enable_segment_copy_checking`, its default value is true.
User can disable the distribution key check with with a GUC value.
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

When use command `COPY FROM ON SEGMENT`, we copy data from
local file to the table on the segment directly. When copying
data, we need to apply the distribution policy on the record to compute
the target segment. If the target segment ID isn't equal to
current segment ID, we will report error to keep the distribution
key restriction.

Because the segment has no meta data info about table distribution policy and
partition policy,we copy the distribution policy of main table from
master to segment in the query plan. When the parent table and
partitioned sub table has different distribution policy, it is difficult
to check all the distribution key restriction in all sub tables. In this
case , we will report error.

In case of the partitioned table's distribution policy is
RANDOMLY and different from the parent table, user can use GUC value
`gp_enable_segment_copy_checking` to disable this check.

Check the distribution key restriction as follows:

1) Table isn't partioned:
    Compute the data target segment.If the data doesn't belong the
    segment, will report error.

2) Table is partitioned and the distribution policy of partitioned table
as same as the main table:
    Compute the data target segment.If the data doesn't belong
    the segment, will report error.

3) Table is partitioned and the distribution policy of partitioned
table is different from main table:
    Not support to check ,report error.
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>
Signed-off-by: NMing LI <mli@apache.org>
Signed-off-by: NAdam Lee <ali@pivotal.io>

This allows setting one of the GPDB-added attributes on a line, without
modifying the original line. This reduces the diff of pg_proc.h from
upstream.

This has no effect on the resulting BKI file, except for whitespace. IOW,
there are no catalog changes in this commit. I checked that by diffing the
resulting BKI file, before and after this patch, with "diff -w".

I still left the TODO comment in pg_proc.h in place, which pointed out
that it'd be nice if we could automatically use prodataaccess = 'c' as the
default for SQL-language columns, and 'n' for others. I actually wrote a
more flexible prototype at first that could do that. In that prototype, you
could provide an arbitrary perl expression that was evaluated on every row,
and could compute a value based on other columns. But that was more
complicated, and at the same time, not as flexible, because you could still
not specify particular values for just one row. So I think this is better
in the end.

Also, I noticed that we haven't actually marked all SQL-language functions
with prodataaccess = 'c'. Tsk tsk. It's too late for catalog changes, so
not fixing that now. At some point, we should discuss whether we should
do something different with prodataaccess, like change the code so that
it's ignored for SQL language functions altogether. Or perhaps just remove
the column, the only useful value for it is the magic 's' value, which
can only be used in built-in functions because there's no DDL syntax for
it. But that's a whole different story.

It's harmless, as the genbki script ignores them, but seems a bit untidy.
It also makes it harder to diff between the re-constructed DATA lines and
the originals.

Calling elog(ERROR ..) will exit the current context, so the
cleanup function would never run. Shift around to ensure the
cleanup is called. This codepath was recently introduced in
commit 00ce2c14 where a surrounding try/catch block was removed.

Also change to ereport from elog since this is an error that
a user could run into.

This change adds a check to verify that the heap_checksum setting on
standby master matches master. If not, gpstart exits.
For completeness, we'll be checking all the segments.

Note: Heapchecksum (pg_control) can change if a user runs pg_resetxlog
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

On ARM64, 'char' is unsigned, unlike on most platforms including x86.

Per report from Dave Cramer.

Discussion: https://github.com/greenplum-db/gpdb/pull/1983#issuecomment-324955931

As part of the 8.3 merge, we backported the addition of the location field.
from 8.4, but accidentally left the copy support for it commented out.

Also, be more pedantic about using COPY/READ/WRITE_LOCATION_FIELD macros
to deal with location fields, rather than the *_INT_FIELD macros. This is
purely cosmetic, but hopefully reduces merge conflicts a bit. I didn't touch
the few places where we already had the location field in 8.3, those will
be changed as part of the 8.4 merge.

Currently, its possible to create a workerpool with 0 workers, which is
good for killing time but not much else. (hang forever).

Now we'll fail fast with the worker pool guard.
Signed-off-by: NShoaib Lari <slari@pivotal.io>
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

As part of the validation phase of gprecoverseg, before proceeding,
validate that the setting for GUC data_checksums is the same between
master and segments. The validation is done by comparing pg_control file content.
Fail fast if the settings are not the same.

If no segments are able to report their settings, then gprecoverseg
fails. (This failure to report would be unexpected since there is already a check for at least
one segment alive to progress to the validation phase.)
Signed-off-by: NMarbin Tan <mtan@pivotal.io>
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

Signed-off-by: NLarry Hamel <lhamel@pivotal.io>