strcasestr() needs string.h

StringInfo is more appropriate in backend code. (Unless the buffer needs to
be used in a thread.)

In the passing, rename the 'conn' static variable in cdbfilerepconnclient.c.
It seemed overly generic.

* Fix an assertion failure in resgroup.

Suppose a resgroup already has a session assigned but no query running.
In another session (not assigned to this resgroup) drop this resgroup
and recreate it, then an assertion failure would be triggered.

The cause is that the old resgroup's struct in shm is reused for the new
resgroup, so on a QE who only owns a weak ref to this struct may find the
struct's resgroup id different than its expection.

To fix it we add the checks for such cases.

* Changes according to PR reviews.

\copy command reads the file from the client filesystem
but 'on segment'  reads file from the segment system
So \copy command and 'on segment' option are incompatible
Signed-off-by: NXiaoran Wang <xiwang@pivotal.io>

gpcheckmirrorseg.pl assumed GPDB is compiled as 32 bit on macs and hence doesn't
work correctly. Fixing the same as now we compile GPDB in 64 bits on MACs too.

Also, add errlog directory to ignore_list as its not supposed to be same on
primary and mirror. Plus, reflect missing relkinds for visimap, blockdirectory
and sequence tables.

CID 173910 Dereference after null check. In markDirty: Pointer is checked
against null but then dereferenced anyway. Removed the checking against and
added Assert instead as don't see why relation should be NULL in markDirty().

CID 173667 Printf arg type mismatch. A printf format string does not match the
types of one of the arguments. Fixed to make it only one %s and relpath() to get
the path.

Also, fixing the warning
```
guc.c:643:15: warning: incompatible pointer to integer conversion initializing 'bool' (aka 'char') with an expression of type 'void *' [-Wint-conversion]
                NULL, NULL, NULL
```

Clean up after commits e40e78fa, 038aa959, 859d6592 and a21da89e.
Tut tut.

In commit 06f56fe8, we assigned a random segment to server as singleton
QE for load blance. An assertion failure "outputSegIdx[0] == gp_singleton_segindex"
occurs when plan contains initplans because apply_motion_mutator() did
not set segindex of FLOW_SINGLETON to gp_singleton_segindex. Although
segindex field did not affect the creation of singleton gang and also
did not affect QE sending data to the right upstream, we still need
making them consistent to silent the assertion and avoid potential issue.

missing "-y" for yum in commit 80e931e8

This reverts commit 64d150e9.

Error message on Travis was:

```
$ wget http://ftp.jaist.ac.jp/pub/apache/apr/${APR_TAR}
--2017-08-02 06:45:44--  http://ftp.jaist.ac.jp/pub/apache/apr/apr-1.5.2.tar.gz
Resolving ftp.jaist.ac.jp (ftp.jaist.ac.jp)... 150.65.7.130, 2001:df0:2ed:feed::feed
Connecting to ftp.jaist.ac.jp (ftp.jaist.ac.jp)|150.65.7.130|:80... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-08-02 06:45:45 ERROR 404: Not Found.
The command "wget http://ftp.jaist.ac.jp/pub/apache/apr/${APR_TAR}" failed and exited with 8 during .
```

Signed-off-by: NJim Doty <jdoty@pivotal.io>

In order to protect users of the concourse cluster provisioner service from
some pending breaking changes, we are pinning the master pipeline to the latest
tag for the concourse cluster provisioner repo.  We will in the future facilitate
upgrades on the master pipeline to take advantage of the next release.
Signed-off-by: NJim Doty <jdoty@pivotal.io>

[ci skip]

It's easy to lose log/error information if we print the usage when there is an
error with the cluster config and/or input config option.
Make it more direct by just showing the error to the user.

We add the PT information in addition to the backup block for proper recovery.
In this case, during XLog redo, the blocks of the dropped tables will not be
restored after consulting PT.

In order to fetch the PT information, we have to pass Relation to
MarkBufferDirtyHint(). For that interface change, we refactored
MarkBufferDirtyHint() with additional Relation parameter. The relation
information is eventually passed to XLogSaveBufferForHint() to fetch the PT
information when preparing the XLOG_HINT record.

pg_filedump and other external utility programs are likely to want to be
able to check Postgres page checksums.  To avoid messy duplication of code,
move the checksumming functionality into an exported header file, much as
we did awhile back for the CRC code.

In passing, get rid of an unportable assumption that a static char[] array
will be word-aligned, and do some other minor code beautification.

(cherry picked from commit f0421634)

Ants Aasma and Jeff Davis

(cherry picked from commit fdea2530)

Isolate checksum calculation to its own module, so that bufpage
knows little if anything about the details of the calculation.

This implementation is a modified FNV-1a hash checksum, details
of which are given in the new checksum.c header comments.

Basic implementation only, so we fix the output value.

Later related commits will add version numbers to pg_control,
compiler optimization flags and memory barriers.

Ants Aasma, reviewed by Jeff Davis and Simon Riggs

(cherry picked from commit 43e7a668)

Heikki reported comment was wrong, so fixed
code to match the comment: we only need to
take additional locking precautions when we
have a shared lock on the buffer.

(cherry picked from commit 5787c673)

We copy the buffer before inserting an XLOG_HINT to avoid WAL CRC errors
caused by concurrent hint writes to buffer while share locked. To make this work
we refactor RestoreBackupBlock() to allow an XLOG_HINT to avoid the normal
path for backup blocks, which assumes the underlying buffer is exclusive locked.
Resulting code completely changes layout of XLOG_HINT WAL records, but
this isn't even beta code, so this is a low impact change.
In passing, avoid taking WALInsertLock for full page writes on checksummed
hints, remove related cruft from XLogInsert() and improve xlog_desc record for
XLOG_HINT.

Andres Freund

Bug report by Fujii Masao, testing by Jeff Janes and Jaime Casanova,
review by Jeff Davis and Simon Riggs. Applied with changes from review
and some comment editing.

(cherry picked from commit 47c43331)

(cherry picked from commit a4b94b85)

From performance analysis by Heikki Linnakangas

(cherry picked from commit 1be20351)

Also exclude the block backup for temp relations
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

With `gp_disable_tuple_hints` off, buffer will be marked dirty for hint
bit changes. The XLOG should capture the full page image if the hint bit
is the first change after checkpoint.

During recovery, the full page image should be replayed to override the
page, even if the page is corrupted on disk.

This test verified the table with corrupted page full fixed after
recovery if the `gp_disable_tuple_hints` off.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Originally, we didn't verify the correct error level in our helper method.

Using errcode 0 will cause ereport() to treat it as an internal
error and print the filename/line. Since this is a userfacing
error it should have a proper errcode to avoid this. This also
allows the gpdiff rule to be removed.

Commit 82329ca1 introduced this var
but never ended up using it. Remove to avoid compiler warnings on
unused var.

1)add pipeline job for resource group test
2)make test cases independent of segment count and hosts count

This test was asserting on a higher level mock which wasn't being used.
This commit uses the correct mock and the tests are passing.
Signed-off-by: NMarbin Tan <mtan@pivotal.io>

It looks like this is passing in concourse, however, this test suite is
having issues running properly. One of the mocks is not returning the
proper behavior causing the test to fail.
Signed-off-by: NNadeem Ghani <nghani@pivotal.io>

Refactor similar usage to share code with gpperfmon behave tests
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Resource group may have memory overuse in below case:

	CREATE RESOURCE GROUP rg_concurrency_test WITH
	(concurrency=1, cpu_rate_limit=20, memory_limit=60,
	 memory_shared_quota=0, memory_spill_ratio=10);
	CREATE ROLE role_concurrency_test RESOURCE GROUP rg_concurrency_test;

	11:SET ROLE role_concurrency_test;
	11:BEGIN;

	21:SET ROLE role_concurrency_test;
	22:SET ROLE role_concurrency_test;
	21&:BEGIN;
	22&:BEGIN;

	ALTER RESOURCE GROUP rg_concurrency_test SET CONCURRENCY 2;

	11:END;

The cause is that we didn't check overall memory quota usage in the
past, so pending queries can be waken up as long as the concurrency
limit is not reached, in such a case if the currently running tranctions
have used all the memory quota in the resource group then the overall
memory usage will be exceeded.

To fix this issue we now checks both concurrency limit and memory quota
usage to decide whether to wake up pending queries.
Signed-off-by: NZhenghua Lyu <zlv@pivotal.io>