Signed-off-by: NNadeem Ghani <nghani@pivotal.io>
Signed-off-by: NLarry Hamel <lhamel@pivotal.io>

Will hopefully reduce silly merge conflicts in the future.

They don't exist in the upstream. All but one of the callers actually just
needed list_member_*().

These functions are defined in cdbpartition.c, not tablecmds.c. The
prototypes should be in matching header file.

To match the upstream code.

Seems harmless, in this direction, but let's be tidy.

This gets rid of the GPDB-specific "QE details" message, that was only sent
once at QE backend startup, to notify the QD about the motion listener port
of the QE backend. Use a standard ParameterStatus message instead, pretending
that there is a GUC called "qe_listener_port". This reduces the difference
between the gp_libpq_fe copy of libpq, and libpq proper. I have a dream that
one day we will start using the standard libpq also for QD-QE communication,
and get rid of the special gp_libpq_fe copy altogether, and this is a small
step in that direction.

In the passing, change the type of Gp_listener_port variable from signed to
unsigned. Gp_listener_port actually holds two values: the TCP and UDP
listener ports, and there is bit-shifting code to store those two 16-bit
port numbers in the single 32-bit integer. But the bit-shifting was a bit
iffy, on a signed integer. Making it unsigned makes it more clear what's
happening.

Forgot to toupper() the first character of string to strstr().
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

The variable was removed in PostgreSQL 8.3 (commit 7c5e5439), but this
removal of the prototype was missed in the merge.

pcqCtx hasn't existed for some time, and we don't have prosrc as a variable
here either.

toupper() all characters, then strstr() instead.

strcasestr() needs string.h

StringInfo is more appropriate in backend code. (Unless the buffer needs to
be used in a thread.)

In the passing, rename the 'conn' static variable in cdbfilerepconnclient.c.
It seemed overly generic.

* Fix an assertion failure in resgroup.

Suppose a resgroup already has a session assigned but no query running.
In another session (not assigned to this resgroup) drop this resgroup
and recreate it, then an assertion failure would be triggered.

The cause is that the old resgroup's struct in shm is reused for the new
resgroup, so on a QE who only owns a weak ref to this struct may find the
struct's resgroup id different than its expection.

To fix it we add the checks for such cases.

* Changes according to PR reviews.

\copy command reads the file from the client filesystem
but 'on segment'  reads file from the segment system
So \copy command and 'on segment' option are incompatible
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

gpcheckmirrorseg.pl assumed GPDB is compiled as 32 bit on macs and hence doesn't
work correctly. Fixing the same as now we compile GPDB in 64 bits on MACs too.

Also, add errlog directory to ignore_list as its not supposed to be same on
primary and mirror. Plus, reflect missing relkinds for visimap, blockdirectory
and sequence tables.

CID 173910 Dereference after null check. In markDirty: Pointer is checked
against null but then dereferenced anyway. Removed the checking against and
added Assert instead as don't see why relation should be NULL in markDirty().

CID 173667 Printf arg type mismatch. A printf format string does not match the
types of one of the arguments. Fixed to make it only one %s and relpath() to get
the path.

Also, fixing the warning
```
guc.c:643:15: warning: incompatible pointer to integer conversion initializing 'bool' (aka 'char') with an expression of type 'void *' [-Wint-conversion]
                NULL, NULL, NULL
```

Clean up after commits e40e78fa, 038aa959, 859d6592 and a21da89e.
Tut tut.

In commit 06f56fe8, we assigned a random segment to server as singleton
QE for load blance. An assertion failure "outputSegIdx[0] == gp_singleton_segindex"
occurs when plan contains initplans because apply_motion_mutator() did
not set segindex of FLOW_SINGLETON to gp_singleton_segindex. Although
segindex field did not affect the creation of singleton gang and also
did not affect QE sending data to the right upstream, we still need
making them consistent to silent the assertion and avoid potential issue.

missing "-y" for yum in commit 80e931e8

This reverts commit 64d150e9.

Error message on Travis was:

```
$ wget http://ftp.jaist.ac.jp/pub/apache/apr/${APR_TAR}
--2017-08-02 06:45:44--  http://ftp.jaist.ac.jp/pub/apache/apr/apr-1.5.2.tar.gz
Resolving ftp.jaist.ac.jp (ftp.jaist.ac.jp)... 150.65.7.130, 2001:df0:2ed:feed::feed
Connecting to ftp.jaist.ac.jp (ftp.jaist.ac.jp)|150.65.7.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-08-02 06:45:45 ERROR 404: Not Found.
The command "wget http://ftp.jaist.ac.jp/pub/apache/apr/${APR_TAR}" failed and exited with 8 during .
```

Signed-off-by: NJim Doty <jdoty@pivotal.io>

In order to protect users of the concourse cluster provisioner service from
some pending breaking changes, we are pinning the master pipeline to the latest
tag for the concourse cluster provisioner repo.  We will in the future facilitate
upgrades on the master pipeline to take advantage of the next release.
Signed-off-by: NJim Doty <jdoty@pivotal.io>

[ci skip]

It's easy to lose log/error information if we print the usage when there is an
error with the cluster config and/or input config option.
Make it more direct by just showing the error to the user.

We add the PT information in addition to the backup block for proper recovery.
In this case, during XLog redo, the blocks of the dropped tables will not be
restored after consulting PT.

In order to fetch the PT information, we have to pass Relation to
MarkBufferDirtyHint(). For that interface change, we refactored
MarkBufferDirtyHint() with additional Relation parameter. The relation
information is eventually passed to XLogSaveBufferForHint() to fetch the PT
information when preparing the XLOG_HINT record.

pg_filedump and other external utility programs are likely to want to be
able to check Postgres page checksums.  To avoid messy duplication of code,
move the checksumming functionality into an exported header file, much as
we did awhile back for the CRC code.

In passing, get rid of an unportable assumption that a static char[] array
will be word-aligned, and do some other minor code beautification.

(cherry picked from commit f0421634)

Ants Aasma and Jeff Davis

(cherry picked from commit fdea2530)

Isolate checksum calculation to its own module, so that bufpage
knows little if anything about the details of the calculation.

This implementation is a modified FNV-1a hash checksum, details
of which are given in the new checksum.c header comments.

Basic implementation only, so we fix the output value.

Later related commits will add version numbers to pg_control,
compiler optimization flags and memory barriers.

Ants Aasma, reviewed by Jeff Davis and Simon Riggs

(cherry picked from commit 43e7a668)

Heikki reported comment was wrong, so fixed
code to match the comment: we only need to
take additional locking precautions when we
have a shared lock on the buffer.

(cherry picked from commit 5787c673)

We copy the buffer before inserting an XLOG_HINT to avoid WAL CRC errors
caused by concurrent hint writes to buffer while share locked. To make this work
we refactor RestoreBackupBlock() to allow an XLOG_HINT to avoid the normal
path for backup blocks, which assumes the underlying buffer is exclusive locked.
Resulting code completely changes layout of XLOG_HINT WAL records, but
this isn't even beta code, so this is a low impact change.
In passing, avoid taking WALInsertLock for full page writes on checksummed
hints, remove related cruft from XLogInsert() and improve xlog_desc record for
XLOG_HINT.

Andres Freund

Bug report by Fujii Masao, testing by Jeff Janes and Jaime Casanova,
review by Jeff Davis and Simon Riggs. Applied with changes from review
and some comment editing.

(cherry picked from commit 47c43331)