Upstream doesn't have it and not used anymore in Greenplum, so loose it.

This reverts commit a21da89e. These
changes are obsolete now that persistent tables have been removed.
Co-authored-by: NAsim R P <apraveen@pivotal.io>

This is essentially a use-after-free bug. Found by running regression
tests with CLOBBER_CACHE_ALWAYS and RELCACHE_FORCE_RELEASE.

SEQ_LOG_VALS are pre-logged to avoid logging every fetch from a
sequence, so we need to mask out last_value and log_cnt for sequence
pages before running consistency checks.

* Revert almost all the changes in smgr.c / md.c, to not go through
  the Mirrored* APIs.

* Remove mmxlog stuff. Use upstream "pending relation deletion" code
  instead.

* Get rid of multiple startup passes. Now it's just a single pass like
  in the upstream.

* Revert the way database drop/create are handled to the way it is in
  upstream. Doesn't use PT anymore, but accesses file system directly,
  and WAL-logs a single CREATE/DROP DATABASE WAL record.

* Get rid of MirroredLock

* Remove a few tests that were specific to persistent tables.

* Plus a lot of little removals and reverts to upstream code.

The sequence xlog record generated at the end of DefineSequence
records the local tuple->t_data when the actual page item is
available. Not using the actual page item caused gp_replica_check
extension to fail for sequences because the tuple->t_data ctid would
be replicated as (0,0) instead of the actual page item ctid
(0,1). This is because tuple->t_data is used in the xlog record
creation but its t_ctid is not set even if tuple->t_self is fine.

To fix this, we cherry-pick and modify upstream fix.

Reference commit message:
commit 8d34f686
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:   Tue Apr 22 09:50:47 2014 +0300

    Avoid transient bogus page contents when creating a sequence.

    Don't use simple_heap_insert to insert the tuple to a sequence relation.
    simple_heap_insert creates a heap insertion WAL record, and replaying that
    will create a regular heap page without the special area containing the
    sequence magic constant, which is wrong for a sequence. That was not a bug
    because we always created a sequence WAL record after that, and replaying
    that overwrote the bogus heap page, and the transient state could never be
    seen by another backend because it was only done when creating a new
    sequence relation. But it's simpler and cleaner to avoid that in the first
    place.

This is majorly backport of Postgres commit.
------------------------------------------------
commit a507b869
Author: Robert Haas <rhaas@postgresql.org>
Date:   Wed Feb 8 15:45:30 2017 -0500

    Add WAL consistency checking facility.

    When the new GUC wal_consistency_checking is set to a non-empty value,
    it triggers recording of additional full-page images, which are
    compared on the standby against the results of applying the WAL record
    (without regard to those full-page images).  Allowable differences
    such as hints are masked out, and the resulting pages are compared;
    any difference results in a FATAL error on the standby.

    Kuntal Ghosh, based on earlier patches by Michael Paquier and Heikki
    Linnakangas.  Extensively reviewed and revised by Michael Paquier and
    by me, with additional reviews and comments from Amit Kapila, Álvaro
    Herrera, Simon Riggs, and Peter Eisentraut.
------------------------------------------------

Its modified to work with current xlog format of Greenplum, which is different
from Postgres code when this patch was committed. Main changes are to fit
current backup block format in xlog records. Also, some masking routines differ
compared to upstream as some of the masked flags would come once Greenplum
catches up to latest versions of Postgres.

This bumps the copyright years to the appropriate years after not
having been updated for some time.  Also reformats existing code
headers to match the upstream style to ensure consistency.

When the new OID-dispatching mechanism was introduced (commit f9016da2),
a lot of FIXME comments were left in calls to CdbDispatchUtilityStatement.
These were places where I wasn't sure if the command that was executed
might create new objects, with new OIDs assigned to them, that would need
to be dispatched to the QE. I've now skimmed through the call sites, and
checked that they are all for ALTER or DROP commands that should not create
new OIDs, so remove the FIXME comments.

We add the PT information in addition to the backup block for proper recovery.
In this case, during XLog redo, the blocks of the dropped tables will not be
restored after consulting PT.

In order to fetch the PT information, we have to pass Relation to
MarkBufferDirtyHint(). For that interface change, we refactored
MarkBufferDirtyHint() with additional Relation parameter. The relation
information is eventually passed to XLogSaveBufferForHint() to fetch the PT
information when preparing the XLOG_HINT record.

Checksums are set immediately prior to flush out of shared buffers
and checked when pages are read in again. Hint bit setting will
require full page write when block is dirtied, which causes various
infrastructure changes. Extensive comments, docs and README.

WARNING message thrown if checksum fails on non-all zeroes page;
ERROR thrown but can be disabled with ignore_checksum_failure = on.

Feature enabled by an initdb option, since transition from option off
to option on is long and complex and has not yet been implemented.
Default is not to use checksums.

Checksum used is WAL CRC-32 truncated to 16-bits.

Simon Riggs, Jeff Davis, Greg Smith
Wide input and assistance from many community members. Thank you.

(cherry picked from commit 96ef3b8f)

Remove use of PageSetTLI() from all page manipulation functions
and adjust README to indicate change in the way we make changes
to pages. Repurpose those bytes into the pd_checksum field and
explain how that works in comments about page header.

Refactoring ahead of actual feature patch which would make use
of the checksum field, arriving later.

Jeff Davis, with comments and doc changes by Simon Riggs
Direction suggested by Robert Haas; many others providing
review comments.

(cherry picked from bb7cc262)

This way we are consistent now with the upstream Postgres.

Postgres merge introduced init_sequence() which can optionally lock sequence
relation.

In GPDB, single sequence server instance is used to generate sequence values
for all requests coming from segments, hence it doesn't require a lock on
sequence relation.

There are three concurrent scenarios when using sequence:
Scenario A: concurrent requests from segments:
create table t1 (c int, d serial) distributed by (c);
insert into t1 select i from generate_series(1, 100) i;

Scenario B: concurrent requests from master:
tx1: select nextval('t1_c_seq'::regclass);
tx2: select nextval('t1_c_seq'::regclass);

Scenario C: concurrent requests from both master and segments
tx1: select nextval('t1_c_seq'::regclass);
tx2: insert into t1 values (200, default);

Scenario A is protected by the single instance of sequence server.
Scenario B and C are protected by the BUFFER_LOCK_EXCLUSIVE on shared
buffer of the sequence relation.

With that said, we don't need to hold additional lock on sequence relation.
Signed-off-by: NXin Zhang <xzhang@pivotal.io>

Instead of carrying a "new OID" field in all the structs that represent
CREATE statements, introduce a generic mechanism for capturing the OIDs
of all created objects, dispatching them to the QEs, and using those same
OIDs when the corresponding objects are created in the QEs. This allows
removing a lot of scattered changes in DDL command handling, that was
previously needed to ensure that objects are assigned the same OIDs in
all the nodes.

This also provides the groundwork for pg_upgrade to dictate the OIDs to use
for upgraded objects. The upstream has mechanisms for pg_upgrade to dictate
the OIDs for a few objects (relations and types, at least), but in GPDB,
we need to preserve the OIDs of almost all object types.

In many places where we had used a mixture of spaces and tabs for
indentation, new versions of gcc complained about misleading indentation,
because gcc doesn't know we're using tab width of 4. To fix, make the
indentation consistent in all the places where gcc gave a warning. Would
be nice to fix it all around, but that's a lot of work, so let's do it
in a piecemeal fashion whenever we run into issues or need to modify a
piece of code anyway.

For some files, especially the GPDB-specific ones, I ran pgindent over
the whole file. I used the pgindent from PostgreSQL master, which is
slightly different from what was used back 8.3 days, but that's what I had
easily available, and that's what we're heading to in the future anyway.
In some cases, I didn't commit the pgindented result if there were
funnily formatted code or comments that would need special treatment.

For other places, I fixed the indentation locally, just enough to make the
warnings go away.

I also did a tiny bit of other trivial cleanup, that I happened to spot
while working on this, although I tried to refrain from anything more
extensive.

The pg_aoseg and pg_aocsseg index scans were replaced with heap
scans. Therefore, we no longer need the pg_aoseg and pg_aocsseg
indexes. This commit removes all references to them in the code and
catalog.

Authors: Jimmy Yih, Asim Praveen, and Abhijit Subramanya

Refactor code to use common routine to fetch PT info for xlogging. Check can
be easliy added at this common place to validate persistent info is
available. Plus still add check during recovery for persistentTID zero. As
with postgres upstream merges possible the function to populate persistent info is
not called at all, so this check will not hit during xlog record construction
but atleast gives clear clue during recovery.

To reduce our diff vs upstream, to make diffing and merging easier.

refactor CdbDispatchUtilityStatement() to make it flexible for cdbCopyStart(),
dispatchVacuum() to call directly. Introduce flags like DF_NEED_TWO_SNAPSHOT,
DF_WITH_SNAPSHOT, DF_CANCEL_ON_ERROR to make function call much clearer

The checks for invalid TIDs are very cheap, a few CPU instructions. It's
better to catch bugs involving invalid TID early, so let's always check
for them.

The LOGs in storeAttrDefault() that were also tied to this GUC seemed
oddly specific. They were probably added long time ago to hunt for some
particular bug, and don't seem generally useful, so I just removed them.

dispatcher/ directory

This commit has no logic change, it just contains movement of code across
files, to make dispatcher code clearer, and easier for unit testing.

Signed-off-by: Kenan Yao

Remove unnecessary #includes, add #includes that are actually needed by
some headers.

In cdbcat.h, include only the header files that are actually needed for
the single function prototype in that file. And don't include cdbcat.h
unnecessarily. A couple of .c files were including cdbcat.h to get
GpPolicy, but that's actually defined in catalog/gp_policy.h, so #include
that directly instead where needed.

In a sequence, if NO CYCLE is specified, any calls to nextval after the sequence has reached its maximum value will return an error.

If a crash occurred immediately after the first nextval() call for a serial
column, WAL replay would restore the sequence to a state in which it
appeared that no nextval() had been done, thus allowing the first sequence
value to be returned again by the next nextval() call; as reported in
bug #6748 from Xiangming Mei.

More generally, the problem would occur if an ALTER SEQUENCE was executed
on a freshly created or reset sequence.  (The manifestation with serial
columns was introduced in 8.2 when we added an ALTER SEQUENCE OWNED BY step
to serial column creation.)  The cause is that sequence creation attempted
to save one WAL entry by writing out a WAL record that made it appear that
the first nextval() had already happened (viz, with is_called = true),
while marking the sequence's in-database state with log_cnt = 1 to show
that the first nextval() need not emit a WAL record.  However, ALTER
SEQUENCE would emit a new WAL entry reflecting the actual in-database state
(with is_called = false).  Then, nextval would allocate the first sequence
value and set is_called = true, but it would trust the log_cnt value and
not emit any WAL record.  A crash at this point would thus restore the
sequence to its post-ALTER state, causing the next nextval() call to return
the first sequence value again.

To fix, get rid of the idea of logging an is_called status different from
reality.  This means that the first nextval-driven WAL record will happen
at the first nextval call not the second, but the marginal cost of that is
pretty negligible.  In addition, make sure that ALTER SEQUENCE resets
log_cnt to zero in any case where it touches sequence parameters that
affect future nextval results.  This will result in some user-visible
changes in the contents of a sequence's log_cnt column, as reflected in the
patch's regression test changes; but no application should be depending on
that anyway, since it was already true that log_cnt changes rather
unpredictably depending on checkpoint timing.

In addition, make some basically-cosmetic improvements to get rid of
sequence.c's undesirable intimacy with page layout details.  It was always
really trying to WAL-log the contents of the sequence tuple, so we should
have it do that directly using a HeapTuple's t_data and t_len, rather than
backing into it with some magic assumptions about where the tuple would be
on the sequence's page.

Back-patch to all supported branches.

My previous commit disallowed this operation, but did nothing about
cleaning up the damage if one had already been done.  With the operation
disallowed, it's okay to just forcibly clear xmax in a sequence's tuple,
since any value seen there could not represent a live transaction's lock.
So, any sequence-specific operation will repair the problem automatically,
whether or not the user has already seen "could not access status of
transaction" failures.

My previous commit disallowed this operation, but did nothing about
cleaning up the damage if one had already been done.  With the operation
disallowed, it's okay to just forcibly clear xmax in a sequence's tuple,
since any value seen there could not represent a live transaction's lock.
So, any sequence-specific operation will repair the problem automatically,
whether or not the user has already seen "could not access status of
transaction" failures.

I found actual bugs in GiST and plpgsql; the rest of this is cosmetic
but meant to decrease the odds of future bugs of omission.

The initial collations patch treated a COLLATE spec as part of a TypeName,
following what can only be described as brain fade on the part of the SQL
committee.  It's a lot more reasonable to treat COLLATE as a syntactically
separate object, so that it can be added in only the productions where it
actually belongs, rather than needing to reject it in a boatload of places
where it doesn't belong (something the original patch mostly failed to do).
In addition this change lets us meet the spec's requirement to allow
COLLATE anywhere in the clauses of a ColumnDef, and it avoids unfriendly
behavior for constructs such as "foo::type COLLATE collation".

To do this, pull collation information out of TypeName and put it in
ColumnDef instead, thus reverting most of the collation-related changes in
parse_type.c's API.  I made one additional structural change, which was to
use a ColumnDef as an intermediate node in AT_AlterColumnType AlterTableCmd
nodes.  This provides enough room to get rid of the "transform" wart in
AlterTableCmd too, since the ColumnDef can carry the USING expression
easily enough.

Also fix some other minor bugs that have crept in in the same areas,
like failure to copy recently-added fields of ColumnDef in copyfuncs.c.

While at it, document the formerly secret ability to specify a collation
in ALTER TABLE ALTER COLUMN TYPE, ALTER TYPE ADD ATTRIBUTE, and
ALTER TYPE ALTER ATTRIBUTE TYPE; and correct some misstatements about
what the default collation selection will be when COLLATE is omitted.

BTW, the three-parameter form of format_type() should go away too,
since it just contributes to the confusion in this area; but I'll do
that in a separate patch.

"SELECT ... INTO UNLOGGED tabname" works, but wasn't documented; CREATE
UNLOGGED SEQUENCE and CREATE UNLOGGED VIEW failed an assertion, instead
of throwing a sensible error.

Latter issue reported by Itagaki Takahiro; patch review by Tom Lane.

This adds collation support for columns and domains, a COLLATE clause
to override it per expression, and B-tree index support.

Peter Eisentraut
reviewed by Pavel Stehule, Itagaki Takahiro, Robert Haas, Noah Misch

Add new function pg_sequence_parameters that returns a sequence's start,
minimum, maximum, increment, and cycle values, and use that in the view.
(bug #5662; design suggestion by Tom Lane)

Also slightly adjust the view's column order and permissions after review of
SQL standard.

This commit replaces pg_class.relistemp with pg_class.relpersistence;
and also modifies the RangeVar node type to carry relpersistence rather
than istemp.  It also removes removes rd_istemp from RelationData and
instead performs the correct computation based on relpersistence.

For clarity, we add three new macros: RelationNeedsWAL(),
RelationUsesLocalBuffers(), and RelationUsesTempNamespace(), so that we
can clarify the purpose of each check that previous depended on
rd_istemp.

This is intended as infrastructure for the upcoming unlogged tables
patch, as well as for future possible work on global temporary tables.

In the previous coding, we simply issued ALTER SEQUENCE RESTART commands,
which do not roll back on error.  This meant that an error between
truncating and committing left the sequences out of sync with the table
contents, with potentially bad consequences as were noted in a Warning on
the TRUNCATE man page.

To fix, create a new storage file (relfilenode) for a sequence that is to
be reset due to RESTART IDENTITY.  If the transaction aborts, we'll
automatically revert to the old storage file.  This acts just like a
rewriting ALTER TABLE operation.  A penalty is that we have to take
exclusive lock on the sequence, but since we've already got exclusive lock
on its owning table, that seems unlikely to be much of a problem.

The interaction of this with usual nontransactional behaviors of sequence
operations is a bit weird, but it's hard to see what would be completely
consistent.  Our choice is to discard cached-but-unissued sequence values
both when the RESTART is executed, and at rollback if any; but to not touch
the currval() state either time.

In passing, move the sequence reset operations to happen before not after
any AFTER TRUNCATE triggers are fired.  The previous ordering was not
logically sensible, but was forced by the need to minimize inconsistency
if the triggers caused an error.  Transactional rollback is a much better
solution to that.

Patch by Steve Singer, rather heavily adjusted by me.

Itagaki Takahiro, with slight modifications.