The SQL standard does not have general functions-in-FROM, but it does
allow UNNEST() there (see the <collection derived table> production),
and the semantics of that are defined to include lateral references.
So spec compliance requires allowing lateral references within UNNEST()
even without an explicit LATERAL keyword.  Rather than making UNNEST()
a special case, it seems best to extend this flexibility to any
function-in-FROM.  We'll still allow LATERAL to be written explicitly
for clarity's sake, but it's now a noise word in this context.

In theory this change could result in a change in behavior of existing
queries, by allowing what had been an outer reference in a function-in-FROM
to be captured by an earlier FROM-item at the same level.  However, all
pre-9.3 PG releases have a bug that causes them to match variable
references to earlier FROM-items in preference to outer references (and
then throw an error).  So no previously-working query could contain the
type of ambiguity that would risk a change of behavior.

Per a suggestion from Andrew Gierth, though I didn't use his patch.

This patch implements the standard syntax of LATERAL attached to a
sub-SELECT in FROM, and also allows LATERAL attached to a function in FROM,
since set-returning function calls are expected to be one of the principal
use-cases.

The main change here is a rewrite of the mechanism for keeping track of
which relations are visible for column references while the FROM clause is
being scanned.  The parser "namespace" lists are no longer lists of bare
RTEs, but are lists of ParseNamespaceItem structs, which carry an RTE
pointer as well as some visibility-controlling flags.  Aside from
supporting LATERAL correctly, this lets us get rid of the ancient hacks
that required rechecking subqueries and JOIN/ON and function-in-FROM
expressions for invalid references after they were initially parsed.
Invalid column references are now always correctly detected on sight.

In passing, remove assorted parser error checks that are now dead code by
virtue of our having gotten rid of add_missing_from, as well as some
comments that are obsolete for the same reason.  (It was mainly
add_missing_from that caused so much fudging here in the first place.)

The planner support for this feature is very minimal, and will be improved
in future patches.  It works well enough for testing purposes, though.

catversion bump forced due to new field in RangeTblEntry.

Per recent discussion on pgsql-hackers, these messages are too
chatty for most users.

If you decide you want to take the hint, this gives you something you
can paste right back to the server.

Dean Rasheed

When a btree index contains all columns required by the query, and the
visibility map shows that all tuples on a target heap page are
visible-to-all, we don't need to fetch that heap page.  This patch depends
on the previous patches that made the visibility map reliable.

There's a fair amount left to do here, notably trying to figure out a less
chintzy way of estimating the cost of an index-only scan, but the core
functionality seems ready to commit.

Robert Haas and Ibrar Ahmed, with some previous work by Heikki Linnakangas.

There were corner cases in which the planner would attempt to inline such
a function, which would result in a failure at runtime due to loss of
information about exactly what the result record type is.  Fix by disabling
inlining when the function's recorded result type is RECORD.  There might
be some sub-cases where inlining could still be allowed, but this is a
simple and backpatchable fix, so leave refinements for another day.
Per bug #5777 from Nate Carson.

Back-patch to all supported branches.  8.1 happens to avoid a core-dump
here, but it still does the wrong thing.

relation using the general PARAM_EXEC executor parameter mechanism, rather
than the ad-hoc kluge of passing the outer tuple down through ExecReScan.
The previous method was hard to understand and could never be extended to
handle parameters coming from multiple join levels.  This patch doesn't
change the set of possible plans nor have any significant performance effect,
but it's necessary infrastructure for future generalization of the concept
of an inner indexscan plan.

ExecReScan's second parameter is now unused, so it's removed.

The logic for determining whether to materialize has been significantly
overhauled for 9.0.  In case there should be any doubt about whether
materialization is a win in any particular case, this should provide a
convenient way of seeing what happens without it; but even with enable_material
turned off, we still materialize in cases where it is required for
correctness.

Thanks to Tom Lane for the review.

we have to cope with the possibility that the declared result rowtype contains
dropped columns.  This fails in 8.4, as per bug #5240.

While at it, be more paranoid about inserting binary coercions when inlining.
The pre-8.4 code did not really need to worry about that because it could not
inline at all in any case where an added coercion could change the behavior
of the function's statement.  However, when inlining a SRF we allow sorting,
grouping, and set-ops such as UNION.  In these cases, modifying one of the
targetlist entries that the sort/group/setop depends on could conceivably
change the behavior of the function's statement --- so don't inline when
such a case applies.

match which function parameters.  The syntax uses AS, for example
	funcname(value AS arg1, anothervalue AS arg2)

Pavel Stehule

function returning setof record.  This used to work, more or less
accidentally, but I had broken it while extending the code to allow
materialize-mode functions to be called in select lists.  Add a regression
test case so it doesn't get broken again.  Per gripe from Greg Davidson.

TupleTableSlots.  We have functions for retrieving a minimal tuple from a slot
after storing a regular tuple in it, or vice versa; but these were implemented
by converting the internal storage from one format to the other.  The problem
with that is it invalidates any pass-by-reference Datums that were already
fetched from the slot, since they'll be pointing into the just-freed version
of the tuple.  The known problem cases involve fetching both a whole-row
variable and a pass-by-reference value from a slot that is fed from a
tuplestore or tuplesort object.  The added regression tests illustrate some
simple cases, but there may be other failure scenarios traceable to the same
bug.  Note that the added tests probably only fail on unpatched code if it's
built with --enable-cassert; otherwise the bug leads to fetching from freed
memory, which will not have been overwritten without additional conditions.

Fix by allowing a slot to contain both formats simultaneously; which turns out
not to complicate the logic much at all, if anything it seems less contorted
than before.

Back-patch to 8.2, where minimal tuples were introduced.

RETURNING clause, not just a SELECT as formerly.

A side effect of this patch is that when a set-returning SQL function is used
in a FROM clause, performance is improved because the output is collected into
a tuplestore within the function, rather than using the less efficient
value-per-call mechanism.

There are still some weak spots around JOIN USING and relation alias lists,
but most errors reported within backend/parser/ now have locations.

(Unlike the original submission, this patch treats TABLE output parameters
as being entirely equivalent to OUT parameters -- tgl)

Pavel Stehule

pg_type.typtype whereever practical.  Tom Dunstan, with some kibitzing
from Tom Lane.

Standard English uses "may", "can", and "might" in different ways:

        may - permission, "You may borrow my rake."

        can - ability, "I can lift that log."

        might - possibility, "It might rain today."

Unfortunately, in conversational English, their use is often mixed, as
in, "You may use this variable to do X", when in fact, "can" is a better
choice.  Similarly, "It may crash" is better stated, "It might crash".

particular get rid of single quotes around language names and old WITH ()
construct.

the parameter's name (if any) as the default column name for SELECT FROM
the function, rather than the function name as previously.  I still think
this is a bad idea, but I lost the argument.  Force decompilation of
function RTEs to specify full aliases always, to reduce the odds of this
decision breaking dumped views.

Also improve wording.

scans are candidates for exclusion; this should be fixed eventually.
Simon Riggs, with some help from Tom Lane.

logic operations during planning.  Seems cleaner to create two new Path
node types, instead --- this avoids duplication of cost-estimation code.
Also, create an enable_bitmapscan GUC parameter to control use of bitmap
plans.

functions with OUT parameters.  The various PLs still need work, as does
pg_dump.  Rudimentary docs and regression tests included.

>>ISTM that "source" is worth knowing.
>
> Hm, possibly.  Any other opinions?

This version has the seven fields I proposed, including "source". Here's
an example that shows why I think it's valuable:

regression=# \x
Expanded display is on.
regression=# select * from pg_settings where name = 'enable_seqscan';
-[ RECORD 1 ]-----------
name    | enable_seqscan
setting | on
context | user
vartype | bool
source  | default
min_val |
max_val |

regression=# update pg_settings set setting = 'off' where name =
'enable_seqscan';
-[ RECORD 1 ]---
set_config | off

regression=# select * from pg_settings where name = 'enable_seqscan';
-[ RECORD 1 ]-----------
name    | enable_seqscan
setting | off
context | user
vartype | bool
source  | session
min_val |
max_val |

regression=# alter user postgres set enable_seqscan to 'off';
ALTER USER

(log out and then back in again)

regression=# \x
Expanded display is on.
regression=# select * from pg_settings where name = 'enable_seqscan';
-[ RECORD 1 ]-----------
name    | enable_seqscan
setting | off
context | user
vartype | bool
source  | user
min_val |
max_val |

In the first case, enable_seqscan is set to its default value. After
setting it to off, it is obvious that the value has been changed for the
session only. In the third case, you can see that the value has been set
specifically for the user.

Joe Conway

of the containing query (which really can only happen in a rule context).
Per example from Brandon Craig Rhodes.  Also, make the error message
more specific for the similar case with sub-select in FROM.  The revised
coding should be easier to adapt to SQL99's LATERAL(), when we get around
to supporting that.

parameter to allow it to be forced off for comparison purposes.
Add ORDER BY clauses to a bunch of regression test queries that will
otherwise produce randomly-ordered output in the new regime.

types, SRFs.  Not happy with memory management yet, but I'll commit these
other changes.

rangefunc regression test for the new behavior.

Joe Conway

to build dependencies for rules, constraint expressions, and default
expressions.  Repair some problems in the original design of
recursiveDeletion() exposed by more complex dependency sets.  Fix
regression tests that were deleting things in illegal sequences.