compiler warnings.  Marko Kreen and Kris Jurka.

Marko Kreen

Marko Kreen

be more wary about having a value for BYTE_ORDER, clean up randomly-
chosen ways of including Postgres core headers.
Marko Kreen and Tom Lane

Marko Kreen

>
> I think this is because we don't have -lz in SHLIB_LINK.
> Following patch fixes it.

Marko Kreen

Marko Kreen

OpenSSL 0.9.6x.  The DES functions use the older 'des_'
API, but the newer 3DES functions use the 0.9.7x-only
'DES_' API.

I think I just used /usr/include/openssl/des.h for reference
when implementing them, and had upgraded OpenSSL in the
meantime.

Following patch converts DES also to newer API and provides
compatibility functions for OpenSSL < 0.9.7.

I chose this route because:

- openssl.c uses few DES functions.
- compatibility for old 'des_' API is going away at some point
  of time from OpenSSL.
- as seen from macros, new API is saner
- Thus pgcrypto supports any OpenSSL version from 0.9.5 to 1.0

Tested with OpenSSL 0.9.6c and 0.9.7e.

Marko Kreen

of password-based encryption from RFC2440 (OpenPGP).

The goal of this code is to be more featureful encryption solution
than current encrypt(), which only functionality is running cipher
over data.

Compared to encrypt(), pgp_encrypt() does following:

* It uses the equvialent of random Inital Vector to get cipher
  into random state before it processes user data
* Stores SHA-1 of the data into result so any modification
  will be detected.
* Remembers if data was text or binary - thus it can decrypt
  to/from text data.  This was a major nuisance for encrypt().
* Stores info about used algorithms with result, so user needs
  not remember them - more user friendly!
* Uses String2Key algorithms (similar to crypt()) with random salt
  to generate full-length binary key to be used for encrypting.
* Uses standard format for data - you can feed it to GnuPG, if needed.

Optional features (off by default):

* Can use separate session key - user data will be encrypted
  with totally random key, which will be encrypted with S2K
  generated key and attached to result.
* Data compression with zlib.
* Can convert between CRLF<->LF line-endings - to get fully
  RFC2440-compliant behaviour.  This is off by default as
  pgcrypto does not know the line-endings of user data.

Interface is simple:


    pgp_encrypt(data text, key text) returns bytea
    pgp_decrypt(data text, key text) returns text
    pgp_encrypt_bytea(data bytea, key text) returns bytea
    pgp_decrypt_bytea(data bytea, key text) returns bytea

To change parameters (cipher, compression, mdc):

    pgp_encrypt(data text, key text, parms text) returns bytea
    pgp_decrypt(data text, key text, parms text) returns text
    pgp_encrypt_bytea(data bytea, key text, parms text) returns bytea
    pgp_decrypt_bytea(data bytea, key text, parms text) returns bytea

Parameter names I lifted from gpg:

   pgp_encrypt('message', 'key', 'compress-algo=1,cipher-algo=aes256')

For text data, pgp_encrypt simply encrypts the PostgreSQL internal data.

This maps to RFC2440 data type 't' - 'extenally specified encoding'.
But this may cause problems if data is dumped and reloaded into database
which as different internal encoding.  My next goal is to implement data
type 'u' - which means data is in UTF-8 encoding by converting internal
encoding to UTF-8 and back.  And there wont be any compatibility
problems with current code, I think its ok to submit this without UTF-8
encoding by converting internal encoding to UTF-8 and back.  And there
wont be any compatibility problems with current code, I think its ok to
submit this without UTF-8 support.


Here is v4 of PGP encrypt.  This depends on previously sent
Fortuna-patch, as it uses the px_add_entropy function.

- New function: pgp_key_id() for finding key id's.
- Add SHA1 of user data and key into RNG pools.  We need to get
  randomness from somewhere, and it is in user best interests
  to contribute.
- Regenerate pgp-armor test for SQL_ASCII database.
- Cleanup the key handling so that the pubkey support is less
  hackish.

Marko Kreen

- Move openssl random provider to openssl.c and builtin provider
  to internal.c
- Make px_random_bytes use Fortuna, instead of giving error.
- Retarget random.c to aquiring system randomness, for initial seeding
  of Fortuna.  There is ATM 2 functions for Windows,
  reader from /dev/urandom and the regular time()/getpid() silliness.

Marko Kreen

New hashes: SHA256, SHA384, SHA512.

Marko Kreen

functions as STRICT, and all functions except gen_salt() as IMMUTABLE.
gen_salt() is VOLATILE.

Although the functions are now STRICT, I left their PG_ARGISNULL()
checks in place as a protective measure for users who install the
new code but use old (non-STRICT) catalog entries (e.g., restored
from a dump).  Per recent discussion in pgsql-hackers.

Patch from Michael Fuhr.

to make.  We ship the table file in the tarball and so this dependency
just opens file timestamp skew problems without doing anything useful.
(Not that it should hurt, either ... except for cross-compile builds.)

is used in the toplevel configure.  Per Marko Kreen.

auto-detection of AES.

Now openssl.c just checks OpenSSL version.  Whoever compiles
newer OpenSSL without AES is on his own.

Marko Kreen

(currently in beta) when cryptolib = openssl.  According to the
following checkin message from several years ago, OpenSSL application
developers should no longer rely on <openssl/evp.h> to include
everything they need:

http://cvs.openssl.org/chngview?cn=9888

This patch adds the necessary header files.  It doesn't appear to
break anything when building against OpenSSL 0.9.7.

BTW, core appears to build and work fine with OpenSSL 0.9.8.  I've
built 7.3 through HEAD against 0.9.8-beta6 without noticing any
problems.

Michael Fuhr

them, the executation behavior could be unexpected.

* test error handling
* add tests for des, 3des, cast5
* add some tests to blowfish, rijndael
* Makefile: ability to specify different tests for different crypto
  libraries, so we can skip des, 3des and cast5 for builtin.

Marko Kreen

Reserve px_get_random_bytes() for strong randomness,
add new function px_get_pseudo_random_bytes() for
weak randomness and use it in gen_salt().

On openssl case, use RAND_pseudo_bytes() for
px_get_pseudo_random_bytes().

Final result is that is user has not configured random
souce but kept the 'silly' one, gen_salt() keeps
working, but pgp_encrypt() will throw error.

Marko Kreen

* openssl.c: Add 3des and AES support
* README.pgcrypto: list only supported ciphers for openssl

OpenSSL has pre-processor symbol OPENSSL_NO_AES, which
isn't that helpful for detecting if it _does_ exist.
Thus the hack with AES_ENCRYPT.

Marko Kreen

* Use error codes instead of -1
* px_strerror for new error codes
* calling convention change for px_gen_salt - return error code
* use px_strerror in pgcrypto.c

Marko Kreen

  It was a bad style to begin with, and now several loops can be clearer.
* pgcrypto.c: Fix function comments
* crypt-gensalt.c, crypt-blowfish.c: stop messing with errno
* openssl.c: use px_free instead pfree
* px.h: make redefining px_alloc/px_realloc/px_free easier

Marko Kreen

libmcrypt seems to dead, maintainer address bounces,
and cast-128 fails on 2 of the 3 test vectors from RFC2144.

So I see no reason to keep around stuff I don't trust
anymore.

Support for several crypto libraries is probably only
confusing to users, although it was good for initial
developing - it helped to find hidden assumptions and
forced me to create regression tests for all functionality.

Marko Kreen

0.9.7x have EVP_DigestFinal function which which clears all of
EVP_MD_CTX.  This makes pgcrypto crash in functions which
re-use one digest context several times: hmac() and crypt()
with md5 algorithm.

Following patch fixes it by carring the digest info around
EVP_DigestFinal and re-initializing cipher.

Marko Kreen.

has been defined. Previously, pgcrypto would compile but would be
unusable.

writing 'foo()' rather than 'foo(void)'.

"static" to reduce size of generated code slightly.

Claudio Natoli

>
> The patch adds missing the "libpgport.a" file to the installation under
> "install-all-headers". It is needed by some contribs. I install the
> library in "pkglibdir", but I was wondering whether it should be "libdir"?
> I was wondering also whether it would make sense to have a "libpgport.so"?
>
> It fixes various macros which are used by contrib makefiles, especially
> libpq_*dir and LDFLAGS when used under PGXS. It seems to me that they are
> needed to
>
> It adds the ability to test and use PGXS with contribs, with "make
> USE_PGXS=1". Without the macro, this is exactly as before, there should be
> no difference, esp. wrt the vpath feature that seemed broken by previous
> submission. So it should not harm anybody, and it is useful at least to me.
>
> It fixes some inconsistencies in various contrib makefiles
> (useless override, ":=" instead of "=").

Fabien COELHO

conversion of basic ASCII letters.  Remove all uses of strcasecmp and
strncasecmp in favor of new functions pg_strcasecmp and pg_strncasecmp;
remove most but not all direct uses of toupper and tolower in favor of
pg_toupper and pg_tolower.  These functions use the same notions of
case folding already developed for identifier case conversion.  I left
the straight locale-based folding in place for situations where we are
just manipulating user data and not trying to match it to built-in
strings --- for example, the SQL upper() function is still locale
dependent.  Perhaps this will prove not to be what's wanted, but at
the moment we can initdb and pass regression tests in Turkish locale.

make sure the $Id tags are converted to $PostgreSQL as well ...

$Header: -> $PostgreSQL Changes ...