This patch pulls in the addition of checksum version information to pg_control
and a GUC to report the checksum version.  Heap data checksum feature will be
pulled in its entirety as subsequent patches.

Upstream commit that this patch pulls from:

commit 96ef3b8f
Author: Simon Riggs <simon@2ndQuadrant.com>
Date:   Fri Mar 22 13:54:07 2013 +0000

    Allow I/O reliability checks using 16-bit checksums

commit 44395174
Author: Simon Riggs <simon@2ndQuadrant.com>
Date:   Tue Apr 30 12:27:12 2013 +0100

    Record data_checksum_version in control file.
commit 5a7e75849cb595943fc605c4532716e9dd69f8a0
Author: Heikki Linnakangas <heikki.linnakangas@iki.fi>
Date:   Mon Sep 16 14:36:01 2013 +0300

    Add a GUC to report whether data page checksums are enabled.

Rename checkpoint.c to checkpointer.c. And move the code from bgwriter.c to
checkpointer.c and also renames most of corresponding data structures to refect
the clear ownership and association. This commit brings it as close as possible
to PostgreSQL 9.2.

Reference to PostgreSQL related commits:
commit 806a2aee
    Split work of bgwriter between 2 processes: bgwriter and checkpointer.
commit bf405ba8
    Add new file for checkpointer.c
commit 8f28789b
    Rename BgWriterShmem/Request to CheckpointerShmem/Request
commit d843589e5ab361dd4738dab5c9016e704faf4153
    Fix management of pendingOpsTable in auxiliary processes.

System catalogs are now defined like in the upstream, without any special
tidycat headers.

Note: This doesn't change the way pg_proc_gp.h is generated from
pg_proc.sql, by catullus.pl.

This comes with a replacement for generating 4.3.json:
src/backend/catalog/process_foreign_keys.pl. The foreign keys were the
only thing that the json file was used for. AFAIK gpcheckcat is the only
tool that reads that file, and it only paid attention to the foreign key
information.

While working on this, I noticed that a few tables were missing foreign
key declarations. I added FIXME comments on them. Also, a few tables used
to have "vector" references, e.g. pg_index.indclass was an oidvector, where
each element of the array points to pg_class.oid. AFAICS, those
declarations were actually not used for anything. I left those in place as
comments, in case we want to add support in gpcheckcat and the new
process_foreign_keys.pl tool for them, but for now they're just
documentation.

This also removes pablopcatso.pl. It read the json file (Ok, so there was
one more user besides gpcheckcat for it), which is now gone. We could
create a tool that reads the same information straight from the header
files, or from a live database, but it was just a developer aid, and I
don't think anyone's used it for quite a while, so I don't think we need a
replacement.

In the previous coding, if a "name" field in a struct, that was used for
direct access to a catalog tuple, i.e. Form_pg_*, was not accidentally
aligned to 4-byte boundary, the struct would not match what's stored on
disk. I was bit by this, when I tried changing the compresstype field
in pg_appendonly from text to name.

This seems like a good idea from an upgrade point of view too. This change
broke pg_upgrade for any tables that use the name datatype. That's not
a big deal, because it shouldn't be used in user tables at all anyway, but
might as well take the hit in 5.0 when the upgrade is expected to be more
painful anyway.

Original commit:

commit 5f6f840e
Author: Tom Lane <tgl@sss.pgh.pa.us>
Date:   Tue Jun 24 17:58:27 2008 +0000

    Reduce the alignment requirement of type "name" from int to char, and arrange
    to suppress zero-padding of "name" entries in indexes.

    The alignment change is unlikely to save any space, but it is really needed
    anyway to make the world safe for our widespread practice of passing plain
    old C strings to functions that are declared as taking Name.  In the previous
    coding, the C compiler was entitled to assume that a Name pointer was
    word-aligned; but we were failing to guarantee that.  I think the reason
    we'd not seen failures is that usually the only thing that gets done with
    such a pointer is strcmp(), which is hard to optimize in a way that exploits
    word-alignment.  Still, some enterprising compiler guy will probably think
    of a way eventually, or we might change our code in a way that exposes
    more-obvious optimization opportunities.

    The padding change is accomplished in one-liner fashion by declaring the
    "name" index opclasses to use storage type "cstring" in pg_opclass.h.
    Normally btree and hash don't allow a nondefault storage type, because they
    don't have any provisions for converting the input datum to another type.
    However, because name and cstring are effectively the same thing except for
    padding, no conversion is needed --- we only need index_form_tuple() to treat
    the datum as being cstring not name, and this is sufficient.  This seems to
    make for about a one-third reduction in the typical sizes of system catalog
    indexes that involve "name" columns, of which we have many.

    These two changes are only weakly related, but the alignment change makes
    me feel safer that the padding change won't introduce problems, so I'm
    committing them together.

Having a "checkpoint.h", corresponding to "checkpoint.c", makes perfect
sense, but those function definitions are in bgwriter.h in PostgreSQL, and
keeping the code as close to upstream as possible trumps the consistency of
keeping definitions for "foo.c" in header file called "foo.h". Keeping
things close to upstream makes merging easier.

Revert the change to use binary search instead of linear search, for
searching the type array. Bootstrapping is hardly performance critical, and
the array isn't that large anyway.

Remove "bigint" from the array. I'm not sure what it was added for originally,
but it's not present in the upstream, and initdb seems to work without it.

There can never be a need to push the indcheckxmin horizon forward, since
any HOT chains that are actually broken with respect to the index must
pre-date its original creation.  So we can just avoid changing pg_index
altogether during a REINDEX operation.

This offers a cleaner solution than my previous patch for the problem
found a few days ago that we mustn't try to update pg_index while we are
reindexing it.  System catalog indexes will always be created with
indcheckxmin = false during initdb, and with this modified code we should
never try to change their pg_index entries.  This avoids special-casing
system catalogs as the former patch did, and should provide a performance
benefit for many cases where REINDEX formerly caused an index to be
considered unusable for a short time.

Back-patch to 8.3 to cover all versions containing HOT.  Note that this
patch changes the API for index_build(), but I believe it is unlikely that
any add-on code is calling that directly.

its usual buffer cleaning duties during archive recovery, and it's responsible
for performing restartpoints.

This requires some changes in postmaster. When the startup process has done
all the initialization and is ready to start WAL redo, it signals the
postmaster to launch the background writer. The postmaster is signaled again
when the point in recovery is reached where we know that the database is in
consistent state. Postmaster isn't interested in that at the moment, but
that's the point where we could let other backends in to perform read-only
queries. The postmaster is signaled third time when the recovery has ended,
so that postmaster knows that it's safe to start accepting connections.

The startup process now traps SIGTERM, and performs a "clean" shutdown. If
you do a fast shutdown during recovery, a shutdown restartpoint is performed,
like a shutdown checkpoint, and postmaster kills the processes cleanly. You
still have to continue the recovery at next startup, though.

Currently, the background writer is only launched during archive recovery.
We could launch it during crash recovery as well, but it seems better to keep
that codepath as simple as possible, for the sake of robustness. And it
couldn't do any restartpoints during crash recovery anyway, so it wouldn't be
that useful.

log_restartpoints is gone. Use log_checkpoints instead. This is yet to be
documented.

This whole operation is a pre-requisite for Hot Standby, but has some value of
its own whether the hot standby patch makes 8.4 or not.

Simon Riggs, with lots of modifications by me.

Stephen Frost, with help from KaiGai Kohei and others

and heap_deformtuple in favor of the newer functions heap_form_tuple et al
(which do the same things but use bool control flags instead of arbitrary
char values).  Eliminate the former duplicate coding of these functions,
reducing the deprecated functions to mere wrappers around the newer ones.
We can't get rid of them entirely because add-on modules probably still
contain many instances of the old coding style.

Kris Jurka

free space information is stored in a dedicated FSM relation fork, with each
relation (except for hash indexes; they don't use FSM).

This eliminates the max_fsm_relations and max_fsm_pages GUC options; remove any
trace of them from the backend, initdb, and documentation.

Rewrite contrib/pg_freespacemap to match the new FSM implementation. Also
introduce a new variant of the get_raw_page(regclass, int4, int4) function in
contrib/pageinspect that let's you to return pages from any relation fork, and
a new fsm_page_contents() function to inspect the new FSM pages.

There are still some weak spots around JOIN USING and relation alias lists,
but most errors reported within backend/parser/ now have locations.

to suppress zero-padding of "name" entries in indexes.

The alignment change is unlikely to save any space, but it is really needed
anyway to make the world safe for our widespread practice of passing plain
old C strings to functions that are declared as taking Name.  In the previous
coding, the C compiler was entitled to assume that a Name pointer was
word-aligned; but we were failing to guarantee that.  I think the reason
we'd not seen failures is that usually the only thing that gets done with
such a pointer is strcmp(), which is hard to optimize in a way that exploits
word-alignment.  Still, some enterprising compiler guy will probably think
of a way eventually, or we might change our code in a way that exposes
more-obvious optimization opportunities.

The padding change is accomplished in one-liner fashion by declaring the
"name" index opclasses to use storage type "cstring" in pg_opclass.h.
Normally btree and hash don't allow a nondefault storage type, because they
don't have any provisions for converting the input datum to another type.
However, because name and cstring are effectively the same thing except for
padding, no conversion is needed --- we only need index_form_tuple() to treat
the datum as being cstring not name, and this is sufficient.  This seems to
make for about a one-third reduction in the typical sizes of system catalog
indexes that involve "name" columns, of which we have many.

These two changes are only weakly related, but the alignment change makes
me feel safer that the padding change won't introduce problems, so I'm
committing them together.

unnecessary #include lines in it.  Also, move some tuple routine prototypes and
macros to htup.h, which allows removal of heapam.h inclusion from some .c
files.

For this to work, a new header file access/sysattr.h needed to be created,
initially containing attribute numbers of system columns, for pg_dump usage.

While at it, make contrib ltree, intarray and hstore header files more
consistent with our header style.

where Datum is 8 bytes wide.  Since this will break old-style C functions
(those still using version 0 calling convention) that have arguments or
results of these types, provide a configure option to disable it and retain
the old pass-by-reference behavior.  Likewise, provide a configure option
to disable the recently-committed float4 pass-by-value change.

Zoltan Boszormenyi, plus configurability stuff by me.

uses of the long-deprecated float32 in contrib/seg; the definitions themselves
are still there, but no longer used.  fmgr/README updated to match.

I added a CREATE FUNCTION to account for existing seg_center() code in seg.c
too, and some tests for it and the neighbor functions.  At the same time,
remove checks for NULL which are not needed (because the functions are declared
STRICT).

I had to do some adjustments to contrib's btree_gist too.  The choices for
representation there are not ideal for changing the underlying types :-(

Original patch by Zoltan Boszormenyi, with some adjustments by me.

tqual.h into heapam.h.  This makes all inclusion of tqual.h explicit.

I also sorted alphabetically the includes on some source files.

on Linux.

so that we will be able to create a cookie for all processes for CSVlogs.
It is set wherever MyProcPid is set. Take the opportunity to remove the now
unnecessary session-only restriction on the %s and %c escapes in log_line_prefix.

and fsync WAL at convenient intervals.  For the moment it just tries to
offload this work from backends, but soon it will be responsible for
guaranteeing a maximum delay before asynchronously-committed transactions
will be flushed to disk.

This is a portion of Simon Riggs' async-commit patch, committed to CVS
separately because a background WAL writer seems like it might be a good idea
independently of the async-commit feature.  I rebased walwriter.c on
bgwriter.c because it seemed like a more appropriate way of handling signals;
while the startup/shutdown logic in postmaster.c is more like autovac because
we want walwriter to quit before we start the shutdown checkpoint.

over a fairly long period of time, rather than being spat out in a burst.
This happens only for background checkpoints carried out by the bgwriter;
other cases, such as a shutdown checkpoint, are still done at full speed.

Remove the "all buffers" scan in the bgwriter, and associated stats
infrastructure, since this seems no longer very useful when the checkpoint
itself is properly throttled.

Original patch by Itagaki Takahiro, reworked by Heikki Linnakangas,
and some minor API editorialization by me.

comments and variables to "auxiliary proc", per Heikki's request.

startup and bgwriter processes), and the -y flag.  It's not used anywhere.

continuously, and requests vacuum runs of "autovacuum workers" to postmaster.
The workers do the actual vacuum work.  This allows for future improvements,
like allowing multiple autovacuum jobs running in parallel.

For now, the code keeps the original behavior of having a single autovac
process at any time by sleeping until the previous worker has finished.

columns procost and prorows, to allow simple user adjustment of the estimated
cost of a function call, as well as control of the estimated number of rows
returned by a set-returning function.  We might eventually wish to extend this
to allow function-specific estimation routines, but there seems to be
consensus that we should try a simple constant estimate first.  In particular
this provides a relatively simple way to control the order in which different
WHERE clauses are applied in a plan node, which is a Good Thing in view of the
fact that the recent EquivalenceClass planner rewrite made that much less
predictable than before.

back-stamped for this.

Windows), arrange for each postmaster child process to be its own process
group leader, and deliver signals SIGINT, SIGTERM, SIGQUIT to the whole
process group not only the direct child process.  This provides saner behavior
for archive and recovery scripts; in particular, it's possible to shut down a
warm-standby recovery server using "pg_ctl stop -m immediate", since delivery
of SIGQUIT to the startup subprocess will result in killing the waiting
recovery_command.  Also, this makes Query Cancel and statement_timeout apply
to scripts being run from backends via system().  (There is no support in the
core backend for that, but it's widely done using untrusted PLs.)  Per gripe
from Stephen Harris and subsequent discussion.

promoted to FATAL) end in exit(1) not exit(0).  Then change the postmaster to
allow exit(1) without a system-wide panic, but not for the startup subprocess
or the bgwriter.  There were a couple of places that were using exit(1) to
deliberately force a system-wide panic; adjust these to be exit(2) instead.
This fixes the problem noted back in July that if the startup process exits
with elog(ERROR), the postmaster would think everything is hunky-dory and
proceed to start up.  Alternative solutions such as trying to run the entire
startup process as a critical section seem less clean, primarily because of
the fact that a fair amount of startup code is shared by all postmaster
children in the EXEC_BACKEND case.  We'd need an ugly special case somewhere
near the head of main.c to make it work if it's the child process's
responsibility to determine what happens; and what's the point when the
postmaster already treats different children differently?

internal TypInfo table in bootstrap mode.  This allows array_in and
array_out to be used during early bootstrap, which eliminates the
former obstacle to giving OUT parameters to built-in functions.

(table or index) before trying to open its relcache entry.  This fixes
race conditions in which someone else commits a change to the relation's
catalog entries while we are in process of doing relcache load.  Problems
of that ilk have been reported sporadically for years, but it was not
really practical to fix until recently --- for instance, the recent
addition of WAL-log support for in-place updates helped.

Along the way, remove pg_am.amconcurrent: all AMs are now expected to support
concurrent update.

created in the bootstrap phase proper, rather than added after-the-fact
by initdb.  This is cleaner than before because it allows us to retire the
undocumented ALTER TABLE ... CREATE TOAST TABLE command, but the real reason
I'm doing it is so that toast tables of shared catalogs will now have
predetermined OIDs.  This will allow a reasonably clean solution to the
problem of locking tables before we load their relcache entries, to appear
in a forthcoming patch.

it's handled just about like timezone; in particular, don't try
to read anything during InitializeGUCOptions.  Should solve current
startup failure on Windows, and avoid wasted cycles if a nondefault
setting is specified in postgresql.conf too.  Possibly we need to
think about a more general solution for handling 'expensive to set'
GUC options.