fly.  Fix problem with incompletely duplicated setup code.  Andrew Dunstan,
from an idea of Michael Fuhr's.

idea on consistency grounds, whether or not it really fixes bug #1831.
Michael Fuhr

> position. Performing the check in the existing position allows the call
> to go through to perl first, possibly resulting in a SEGV.

Andrew Dunstan

Reported by Michael Fuhr, fixed by Andrew Dunstan.

Add suitable regression tests.  Andrew Dunstan

Add missing plperl include.

return arays nicely without having to make the plperl programmer aware
of anything. The attached patch allows plperl to return an arrayref
where the function returns an array type. It silently calls a perl
function to stringify the array before passing it to the pg array
parser. Non-array returns are handled as before (i.e. passed through
this process) so it is backwards compatible. I will presently submit
regression tests and docs.

example:

andrew=# create or replace function blah() returns text[][] language
plperl as $$ return [['a"b','c,d'],['e\\f','g']]; $$;
CREATE FUNCTION
andrew=# select blah();
            blah
-----------------------------
 {{"a\"b","c,d"},{"e\\f",g}}


This would complete half of the TODO item:

  . Pass arrays natively instead of as text between plperl and postgres

(The other half is translating pg array arguments to perl arrays - that
will have to wait for 8.1).

Some of this patch is adapted from a previously submitted patch from
Sergej Sergeev. Both he and Abhijit Menon-Sen have looked it over
briefly and tentatively said it looks ok.

Andrew Dunstan

for PL/Perl, to avoid loading the entire result set into memory as the
existing spi_exec_query() function does.

Here's how one might use the new functions:

    $x = spi_query("select ...");
    while (defined ($y = spi_fetchrow($x))) {
        ...
        return_next(...);
    }

The changes do not affect the spi_exec_query() interface in any way.

Abhijit Menon-Sen

plperl - the attached small patch remedies that omission, and adds a
small regression test for error and warning output - the new regression
input and expected output are in separate attached files.

Andrew Dunstan

plperl - the attached small patch remedies that omission.

Andrew Dunstan

> against rc1. It simply checks with GetDatabaseEncoding() if the current
> database is in UTF-8, and if so, sets the UTF-8 flag on the arguments
> that are passed to perl. This means that it isn't necessary to
> utf8::upgrade() every string, as perl has no way of knowing offhand
> that a string is UTF-8 -- but postgres does, because the database
> encoding is specified, so it makes sense to turn the flag on. You
> should also be able to properly manipulate UTF-8 strings now from
> plperl as opposed to plperlu, because otherwise you'd have to use
> encoding 'utf8' which was not allowed. It could also eliminate some
> unexpected bugs if you assume that perl knows the string is unicode.
It
> is enabled only for perl 5.6 and higher, so earlier versions will not
> be affected.
>
> I have been assured by crab that the patch is quite harmless and will
> not break anything. It would be great to see it in 8 final! :-)

David Kamholz

1. Rename spi_return_next to return_next.
2. Add a new test for return_next.
3. Update the expected output.
4. Update the documentation.

Abhijit Menon-Sen

>
> > The second issue is where plperl returns a large result set.

I have attached the following seven patches to address this problem:

1. Trivial. Replaces some errant spaces with tabs.

2. Trivial. Fixes the spelling of Jan's name, and gets rid of many
   inane, useless, annoying, and often misleading comments. Here's
   a sample: "plperl_init_all() - Initialize all".

   (I have tried to add some useful comments here and there, and will
   continue to do so now and again.)

3. Trivial. Splits up some long lines.

4. Converts SRFs in PL/Perl to use a Tuplestore and SFRM_Materialize
   to return the result set, based on the PL/PgSQL model.

   There are two major consequences: result sets will spill to disk when
   they can no longer fit in work_mem; and "select foo_srf()" no longer
   works. (I didn't lose sleep over the latter, since that form is not
   valid in PL/PgSQL, and it's not documented in PL/Perl.)

5. Trivial, but important. Fixes use of "undef" instead of undef. This
   would cause empty functions to fail in bizarre ways. I suspect that
   there's still another (old) bug here. I'll investigate further.

6. Moves the majority of (4) out into a new plperl_return_next()
   function, to make it possible to expose the functionality to
   Perl; cleans up some of the code besides.

7. Add an spi_return_next function for use in Perl code.

If you want to apply the patches and try them out, 8-composite.diff is
what you should use. (Note: my patches depend upon Andrew's use-strict
and %_SHARED patches being applied.)

Here's something to try:

    create or replace function foo() returns setof record as $$
    $i = 0;
    for ("World", "PostgreSQL", "PL/Perl") {
        spi_return_next({f1=>++$i, f2=>'Hello', f3=>$_});
    }
    return;
    $$ language plperl;
    select * from foo() as (f1 integer, f2 text, f3 text);

(Many thanks to Andrews Dunstan and Supernews for their help.)

Abhijit Menon-Sen

sharing of %_SHARED. From Andrew Dunstan.

which is neither needed by nor related to that header.  Remove the bogus
inclusion and instead include the header in those C files that actually
need it.  Also fix unnecessary inclusions and bad inclusion order in
tsearch2 files.

only one argument.  (Per recent discussion, the option to accept multiple
arguments is pretty useless for user-defined types, and would be a likely
source of security holes if it was used.)  Simplify call sites of
output/send functions to not bother passing more than one argument.

change saves a great deal of space in pg_proc and its primary index,
and it eliminates the former requirement that INDEX_MAX_KEYS and
FUNC_MAX_ARGS have the same value.  INDEX_MAX_KEYS is still embedded
in the on-disk representation (because it affects index tuple header
size), but FUNC_MAX_ARGS is not.  I believe it would now be possible
to increase FUNC_MAX_ARGS at little cost, but haven't experimented yet.
There are still a lot of vestigial references to FUNC_MAX_ARGS, which
I will clean up in a separate pass.  However, getting rid of it
altogether would require changing the FunctionCallInfoData struct,
and I'm not sure I want to buy into that.

macros around strings that were missing them.

as a statement trigger :-(.  Per report from Sokolov Yura.

returned by Perl.  Per report from Nicolas Addington.

so that they will be translatable.  Give messages some semblance of
conformance to the style guide.

Per report from Mark Kirkwood.

data returned from Perl.  Consolidate multiple bits of code to convert
a Perl hash to a tuple, and drive the conversion off the keys present
in the hash rather than the tuple column names, so we detect error if
the hash contains keys it shouldn't.  (This means keys not in the hash
will silently default to NULL, which seems ok to me.)  Fix a bunch of
reference-count leaks too.

Get rid of static variables for SETOF result, don't crash when called
from non-FROM context, eliminate dead code, etc.

it's not really broken.  Andrew Dunstan

operations are now run as subtransactions, so that errors in them
can be reported as ordinary Perl or Tcl errors and caught by the
normal error handling convention of those languages.  Also do some
minor code cleanup in pltcl.c: extract a large chunk of duplicated
code in pltcl_SPI_execute and pltcl_SPI_execute_plan into a shared
subroutine.

rather than longjmp'ing clear out of Perl and thereby leaving Perl in
a broken state.  Also some minor prettification of error messages.
Still need to do something with spi_exec_query() error handling.

may expand the Perl stack, therefore we must SPAGAIN to reload the local
stack pointer after calling it.  Also a couple other marginal readability
improvements.

some of the bugs exposed thereby.  The remaining 'might be used uninitialized'
warnings look like live bugs, but I am not familiar enough with Perl/C hacking
to tell how to fix them.

patch, but allow srand and disallow sprintf as per subsequent discussion.

1. Two minor cleanups:

    - We don't need to call hv_exists+hv_fetch; we should just check the
      return value of hv_fetch.
    - newSVpv("undef",0) is the string "undef", not a real undef.

2. This should fix the bug Andrew Dunstan described in a recent -hackers
   post. It replaces three bogus "eval_pv(key, 0)" calls with newSVpv,
   and eliminates another redundant hv_exists+hv_fetch pair.

3. plperl_build_tuple_argument builds up a string of Perl code to create
   a hash representing the tuple. This patch creates the hash directly.

4. Another minor cleanup: replace a couple of av_store()s with av_push.

5. Analogous to #3 for plperl_trigger_build_args. This patch removes the
   static sv_add_tuple_value function, which does much the same as two
   other utility functions defined later, and merges the functionality
   into plperl_hash_from_tuple.

I have tested the patches to the best of my limited ability, but I would
appreciate it very much if someone else could review and test them too.

(Thanks to Andrew and David Fetter for their help with some testing.)

Abhijit Menon-Sen