problems in applications that may have a large number of files open,
such that libpq's socket number exceeds the range supported by fd_set.
From Chris Brown.

much enthusiasm.

that tv_sec is signed; return a useful error message on timeout failure;
honor PGCONNECT_TIMEOUT environment variable in PQsetdbLogin; make code
obey documentation statement that timeout=0 means no timeout.

Code cleanup.

clear on that point.

client
utilities (libpq.dll and psql.exe) for win32 (missing defines,
adjustments to
includes, pedantic casting, non-existent functions) per:
   http://developer.postgresql.org/docs/postgres/install-win32.html.

It compiles cleanly under Windows 2000 using Visual Studio .net. Also
compiles clean and passes all regression tests (regular and contrib)
under Linux.

In addition to a review by the usual suspects, it would be very
desirable for  someone well versed in the peculiarities of win32 to take
a look.

Joe Conway

Denis A Ustimenko

files rather than a header file where they belong.  Pay some modicum
of attention to picking global routine names that aren't likely to
conflict with surrounding applications.

I am no longer pursuing a total non-blocking implementation.  I haven't
found a good way to test it with the type of work that I do with
PostgreSQL.  I do use blocking SSL sockets with this mod and have had no
problem whatsoever.  The bug that I fixed in this patch is exceptionally
hard to reproduce reliably.

Jack Bates

Attached are a revised set of SSL patches.  Many of these patches
are motivated by security concerns, it's not just bug fixes.  The key
differences (from stock 7.2.1) are:

*) almost all code that directly uses the OpenSSL library is in two
   new files,

     src/interfaces/libpq/fe-ssl.c
     src/backend/postmaster/be-ssl.c

   in the long run, it would be nice to merge these two files.

*) the legacy code to read and write network data have been
   encapsulated into read_SSL() and write_SSL().  These functions
   should probably be renamed - they handle both SSL and non-SSL
   cases.

   the remaining code should eliminate the problems identified
   earlier, albeit not very cleanly.

*) both front- and back-ends will send a SSL shutdown via the
   new close_SSL() function.  This is necessary for sessions to
   work properly.

   (Sessions are not yet fully supported, but by cleanly closing
   the SSL connection instead of just sending a TCP FIN packet
   other SSL tools will be much happier.)

*) The client certificate and key are now expected in a subdirectory
   of the user's home directory.  Specifically,

	- the directory .postgresql must be owned by the user, and
	  allow no access by 'group' or 'other.'

	- the file .postgresql/postgresql.crt must be a regular file
	  owned by the user.

	- the file .postgresql/postgresql.key must be a regular file
	  owned by the user, and allow no access by 'group' or 'other'.

   At the current time encrypted private keys are not supported.
   There should also be a way to support multiple client certs/keys.

*) the front-end performs minimal validation of the back-end cert.
   Self-signed certs are permitted, but the common name *must*
   match the hostname used by the front-end.  (The cert itself
   should always use a fully qualified domain name (FDQN) in its
   common name field.)

   This means that

	  psql -h eris db

   will fail, but

	  psql -h eris.example.com db

   will succeed.  At the current time this must be an exact match;
   future patches may support any FQDN that resolves to the address
   returned by getpeername(2).

   Another common "problem" is expiring certs.  For now, it may be
   a good idea to use a very-long-lived self-signed cert.

   As a compile-time option, the front-end can specify a file
   containing valid root certificates, but it is not yet required.

*) the back-end performs minimal validation of the client cert.
   It allows self-signed certs.  It checks for expiration.  It
   supports a compile-time option specifying a file containing
   valid root certificates.

*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.

*) both front- and back-ends support DSA keys.  DSA keys are
   moderately more expensive on startup, but many people consider
   them preferable than RSA keys.  (E.g., SSH2 prefers DSA keys.)

*) if /dev/urandom exists, both client and server will read 16k
   of randomization data from it.

*) the server can read empheral DH parameters from the files

     $DataDir/dh512.pem
     $DataDir/dh1024.pem
     $DataDir/dh2048.pem
     $DataDir/dh4096.pem

   if none are provided, the server will default to hardcoded
   parameter files provided by the OpenSSL project.

Remaining tasks:

*) the select() clauses need to be revisited - the SSL abstraction
   layer may need to absorb more of the current code to avoid rare
   deadlock conditions.  This also touches on a true solution to
   the pg_eof() problem.

*) the SIGPIPE signal handler may need to be revisited.

*) support encrypted private keys.

*) sessions are not yet fully supported.  (SSL sessions can span
   multiple "connections," and allow the client and server to avoid
   costly renegotiations.)

*) makecert - a script that creates back-end certs.

*) pgkeygen - a tool that creates front-end certs.

*) the whole protocol issue, SASL, etc.

 *) certs are fully validated - valid root certs must be available.
    This is a hassle, but it means that you *can* trust the identity
    of the server.

 *) the client library can handle hardcoded root certificates, to
    avoid the need to copy these files.

 *) host name of server cert must resolve to IP address, or be a
    recognized alias.  This is more liberal than the previous
    iteration.

 *) the number of bytes transferred is tracked, and the session
    key is periodically renegotiated.

 *) basic cert generation scripts (mkcert.sh, pgkeygen.sh).  The
    configuration files have reasonable defaults for each type
    of use.

Bear Giles

are motivated by security concerns, it's not just bug fixes.  The key
differences (from stock 7.2.1) are:

*) almost all code that directly uses the OpenSSL library is in two
   new files,

     src/interfaces/libpq/fe-ssl.c
     src/backend/postmaster/be-ssl.c

   in the long run, it would be nice to merge these two files.

*) the legacy code to read and write network data have been
   encapsulated into read_SSL() and write_SSL().  These functions
   should probably be renamed - they handle both SSL and non-SSL
   cases.

   the remaining code should eliminate the problems identified
   earlier, albeit not very cleanly.

*) both front- and back-ends will send a SSL shutdown via the
   new close_SSL() function.  This is necessary for sessions to
   work properly.

   (Sessions are not yet fully supported, but by cleanly closing
   the SSL connection instead of just sending a TCP FIN packet
   other SSL tools will be much happier.)

*) The client certificate and key are now expected in a subdirectory
   of the user's home directory.  Specifically,

	- the directory .postgresql must be owned by the user, and
	  allow no access by 'group' or 'other.'

	- the file .postgresql/postgresql.crt must be a regular file
	  owned by the user.

	- the file .postgresql/postgresql.key must be a regular file
	  owned by the user, and allow no access by 'group' or 'other'.

   At the current time encrypted private keys are not supported.
   There should also be a way to support multiple client certs/keys.

*) the front-end performs minimal validation of the back-end cert.
   Self-signed certs are permitted, but the common name *must*
   match the hostname used by the front-end.  (The cert itself
   should always use a fully qualified domain name (FDQN) in its
   common name field.)

   This means that

	  psql -h eris db

   will fail, but

	  psql -h eris.example.com db

   will succeed.  At the current time this must be an exact match;
   future patches may support any FQDN that resolves to the address
   returned by getpeername(2).

   Another common "problem" is expiring certs.  For now, it may be
   a good idea to use a very-long-lived self-signed cert.

   As a compile-time option, the front-end can specify a file
   containing valid root certificates, but it is not yet required.

*) the back-end performs minimal validation of the client cert.
   It allows self-signed certs.  It checks for expiration.  It
   supports a compile-time option specifying a file containing
   valid root certificates.

*) both front- and back-ends default to TLSv1, not SSLv3/SSLv2.

*) both front- and back-ends support DSA keys.  DSA keys are
   moderately more expensive on startup, but many people consider
   them preferable than RSA keys.  (E.g., SSH2 prefers DSA keys.)

*) if /dev/urandom exists, both client and server will read 16k
   of randomization data from it.

*) the server can read empheral DH parameters from the files

     $DataDir/dh512.pem
     $DataDir/dh1024.pem
     $DataDir/dh2048.pem
     $DataDir/dh4096.pem

   if none are provided, the server will default to hardcoded
   parameter files provided by the OpenSSL project.

Remaining tasks:

*) the select() clauses need to be revisited - the SSL abstraction
   layer may need to absorb more of the current code to avoid rare
   deadlock conditions.  This also touches on a true solution to
   the pg_eof() problem.

*) the SIGPIPE signal handler may need to be revisited.

*) support encrypted private keys.

*) sessions are not yet fully supported.  (SSL sessions can span
   multiple "connections," and allow the client and server to avoid
   costly renegotiations.)

*) makecert - a script that creates back-end certs.

*) pgkeygen - a tool that creates front-end certs.

*) the whole protocol issue, SASL, etc.

 *) certs are fully validated - valid root certs must be available.
    This is a hassle, but it means that you *can* trust the identity
    of the server.

 *) the client library can handle hardcoded root certificates, to
    avoid the need to copy these files.

 *) host name of server cert must resolve to IP address, or be a
    recognized alias.  This is more liberal than the previous
    iteration.

 *) the number of bytes transferred is tracked, and the session
    key is periodically renegotiated.

 *) basic cert generation scripts (mkcert.sh, pgkeygen.sh).  The
    configuration files have reasonable defaults for each type
    of use.

Bear Giles

work on all win9x machines, so i made it go thru a l ookup table
instead, using the DLL as last resort.  I also moved this out of the
fe-misc.c file because of the size of the lookup ta ble. Who knows, we
might add more other win32 specific code there in the future.

I also fixed a small typo in the pg_config.h.win32 that made the
compiler compla in about the gnu snprintf declaration.

I tried to make this patch with psql coding style. I've successfully
tested this on win2k and win98 and it works fine (i.e. the mes sage
shows on win98 too, it didn't with the old implementation).

Magnus Naeslund

In summary, if a software writer implements timer events or other events
 which generate a signal with a timing fast enough to occur while libpq
is inside connect(), then connect returns -EINTR.  The code following
the connect call does not handle this and generates an error message.
The sum result is that the pg_connect() fails.  If the timer or other
event is right on the window of the connect() completion time, the
pg_connect() may appear to work sporadically.  If the event is too slow,
 pg_connect() will appear to always work and if the event is too fast,
pg_connect() will always fail.

David Ford

o  Change all current CVS messages of NOTICE to WARNING.  We were going
to do this just before 7.3 beta but it has to be done now, as you will
see below.

o Change current INFO messages that should be controlled by
client_min_messages to NOTICE.

o Force remaining INFO messages, like from EXPLAIN, VACUUM VERBOSE, etc.
to always go to the client.

o Remove INFO from the client_min_messages options and add NOTICE.

Seems we do need three non-ERROR elog levels to handle the various
behaviors we need for these messages.

Regression passed.

	Fix for non-blocking connections in libpq

Bernhard Herzog

queries over non-blocking connections with libpq. "Larger" here
basically means that it doesn't fit into the output buffer.

The basic strategy is to fix pqFlush and pqPutBytes.

The problem with pqFlush as it stands now is that it returns EOF when an
error occurs or when not all data could be sent. The latter case is
clearly not an error for a non-blocking connection but the caller can't
distringuish it from an error very well.

The first part of the fix is therefore to fix pqFlush. This is done by
to renaming it to pqSendSome which only differs from pqFlush in its
return values to allow the caller to make the above distinction and a
new pqFlush which is implemented in terms of pqSendSome and behaves
exactly like the old pqFlush.

The second part of the fix modifies pqPutBytes to use pqSendSome instead
of pqFlush and to either send all the data or if not all data can be
sent on a non-blocking connection to at least put all data into the
output buffer, enlarging it if necessary. The callers of pqPutBytes
don't have to be changed because from their point of view pqPutBytes
behaves like before. It either succeeds in queueing all output data or
fails with an error.

I've also added a new API function PQsendSome which analogously to
PQflush just calls pqSendSome. Programs using non-blocking queries
should use this new function. The main difference is that this function
will have to be called repeatedly (calling select() properly in between)
until all data has been written.

AFAICT, the code in CVS HEAD hasn't changed with respect to non-blocking
queries and this fix should work there, too, but I haven't tested that
yet.

Bernhard Herzog

Magnus Hagander that DLL only contains error strings for the Net***
functions, *not* WinSock.  We need to look for a workable solution for
older Windows flavors ... but it won't happen for PG 7.2.

potential problems discussed in pgsql-interfaces.

(surprised no one has reported these yet...)

initdb/regression tests pass.

spacing.  Also adds space for one-line comments.

tests pass.

cleverer than it actually is ;-) Braces are good for you :-)

Gerhard HÌring

-------------------------------------------------------------------
Subject: Re: [PATCHES] encoding names
From: Karel Zak <zakkr@zf.jcu.cz>
To: Peter Eisentraut <peter_e@gmx.net>
Cc: pgsql-patches <pgsql-patches@postgresql.org>
Date: Fri, 31 Aug 2001 17:24:38 +0200

On Thu, Aug 30, 2001 at 01:30:40AM +0200, Peter Eisentraut wrote:
> > 		- convert encoding 'name' to 'id'
>
> I thought we decided not to add functions returning "new" names until we
> know exactly what the new names should be, and pending schema

 Ok, the patch not to add functions.

> better
>
>     ...(): encoding name too long

 Fixed.

 I found new bug in command/variable.c in parse_client_encoding(), nobody
probably never see this error:

if (pg_set_client_encoding(encoding))
{
	elog(ERROR, "Conversion between %s and %s is not supported",
                     value, GetDatabaseEncodingName());
}

because pg_set_client_encoding() returns -1 for error and 0 as true.
It's fixed too.

 IMHO it can be apply.

		Karel
PS:

    * following files are renamed:

src/utils/mb/Unicode/KOI8_to_utf8.map  -->
        src/utils/mb/Unicode/koi8r_to_utf8.map

src/utils/mb/Unicode/WIN_to_utf8.map  -->
        src/utils/mb/Unicode/win1251_to_utf8.map

src/utils/mb/Unicode/utf8_to_KOI8.map -->
        src/utils/mb/Unicode/utf8_to_koi8r.map

src/utils/mb/Unicode/utf8_to_WIN.map -->
        src/utils/mb/Unicode/utf8_to_win1251.map

   * new file:

src/utils/mb/encname.c

   * removed file:

src/utils/mb/common.c

--
 Karel Zak  <zakkr@zf.jcu.cz>
 http://home.zf.jcu.cz/~zakkr/

 C, PostgreSQL, PHP, WWW, http://docs.linux.cz, http://mape.jcu.cz

really played it totally safe in my last suggestion, the system table might
pick up the msg but not the netmsg.dll, so better try both.
I also added a hex printout of the "errno" appended to all messages, that's
nicer.

If anyone hate my coding style, or that i'm using goto constructs, just tell
me, and i'll rework it into a nested if () thing.

Magnus Naeslund(f)

> It seems that win9x doesn't have the "netmsg.dll" so it defaults to "normal"
> FormatMessage.
> I wonder if one could load wsock32.dll or winsock.dll on those systems
> instead of netmsg.dll.
>
> Mikhail, could you please test this code on your nt4 system?
> Could someone else test this code on a win98/95 system?
>
> It works on win2k over here.

It works on win2k here too but not on win98/95 or winNT.
Anyway, attached is the patch which uses Magnus's my_sock_strerror
function (renamed to winsock_strerror). The only difference is that
I put the code to load and unload netmsg.dll in the libpqdll.c
(is this OK Magnus?).

Mikhail Terekhov

functions do not set errno, so some normal conditions are treated as
fatal errors. e.g. fetching large tuples fails, as at some point recv()
returns EWOULDBLOCK. here's a patch, which replaces errno with
WSAGetLastError(). i've tried to to affect non-win32 code.

Dmitry Yurtaev

accordingly.

the startup exchange is complete.  Also make sure that packets defined as
single bytes aren't sent with a trailing '\0'.

data.

ready.  It appears that most (all?) Unixen will consider a socket to
be read or write ready if it has an error condition, but of course
Microsoft does things differently.