"Howard Chu" <hyc@highlandsun.com>, it may be general enough
to work on any Unixly system.

turned on, and (b) left a somewhat curious debugging string in the output.

few statements equivalent to "ENGINE_add(ENGINE_openssl())" etc. The inner
call to ENGINE_openssl() (as with other functions like it) orphans a
structural reference count. Second, the ENGINE_cleanup() function also
needs to clean up the functional reference counts held internally as the
list of "defaults" (ie. as used when RSA_new() requires an appropriate
ENGINE reference). So ENGINE_clear_defaults() was created and is called
from within ENGINE_cleanup(). Third, some of the existing code was
logically broken in its treatment of reference counts and locking (my
fault), so the necessary bits have been restructured and tidied up.

To test this stuff, compiling with ENGINE_REF_COUNT_DEBUG will cause every
reference count change (both structural and functional) to log a message to
'stderr'. Using with "openssl engine" for example shows this in action
quite well as the 'engine' sub-command cleans up after itself properly.

Also replaced some spaces with tabs.

Tru64 shared libraries can be linked with static libraries.

 * "ex_data" - a CRYPTO_EX_DATA structure in the ENGINE structure itself
   that allows an ENGINE to store its own information there rather than in
   global variables. It follows the declarations and implementations used
   in RSA code, for better or worse. However there's a problem when storing
   state with ENGINEs because, unlike related structure types in OpenSSL,
   there is no ENGINE-vs-ENGINE_METHOD separation. Because of what ENGINE
   is, it has method pointers as its structure elements ...  which leads
   to;

 * ENGINE_FLAGS_BY_ID_COPY - if an ENGINE should not be used just as a
   reference to an "implementation" (eg. to get to a hardware device), but
   should also be able to maintain state, then this flag can be set by the
   ENGINE implementation. The result is that any call to ENGINE_by_id()
   will not result in the existing ENGINE being returned (with its
   structural reference count incremented) but instead a new copy of the
   ENGINE will be returned that can maintain its own state independantly of
   any other copies returned in the past or future. Eg. key-generation
   might involve a series of ENGINE-specific control commands to set
   algorithms, sizes, module-keys, ids, ACLs, etc. A final command could
   generate the key. An ENGINE doing this would *have* to declare
   ENGINE_FLAGS_BY_ID_COPY so that the state of that process can be
   maintained "per-handle" and unaffected by other code having a reference
   to the same ENGINE structure.

takes care of what would otherwise be seen as a memory leak.

unless there's a default clause.

appropriate code if any of them is defined.

here.

We shouldn't skip over header files to avoid functions of disabled algorithms.  The selection is done in a different way

Spotted by Mark Crispin <MRC@Panda.COM>

or symbol name to the error stack in the event a load or bind operation
failed.

Fix ASN1 bug when decoding OTHER type.

Various S/MIME DSA related fixes.

on VMS.

ENGINE.

 * Extra verbosity can be added with more "v"'s, eg. '-vvv' gives
   information about input flags and descriptions for each control command
   in each ENGINE. Check the output of "openssl engine -vvv" for example.

 * '-pre <cmd>' and '-post <cmd>' can be used to invoke control commands on
   the specified ENGINE (or on all of them if no engine id is specified,
   although that usually gets pretty ugly). '-post' commands are only
   attempted if '-t' is specified and the engine successfully initialises.
   '-pre' commands are always attempted whether or not '-t' causes an
   initialisation to be tried afterwards. Multiple '-pre' and/or '-post'
   commands can be specified and they will be called in the order they
   occur on the command line.

Parameterised commands (the normal case, there are currently no
unparameterised ones) are split into command and argument via a separating
colon. Eg. "openssl engine -pre SO_PATH:/lib/libdriver.so <id>" results in
the call;
    ENGINE_ctrl_cmd_string(e, "SO_PATH", "/lib/libdriver.so", 0);

Application code should similarly allow arbitrary name-value string pairs
to be passed into ENGINEs in a manner matching that in apps/engine.c,
either using the same colon-separated format, or entered as two distinct
strings. Eg. as stored in a registry. The last parameter of
ENGINE_ctrl_cmd_string can be changed from 0 to 1 if the command should
only be attempted if it's supported by the specified ENGINE (eg. for
commands like "FORK_CHECK:1" that may or may not apply to the run-time
ENGINE).

This change adds some basic control commands to the existing ENGINEs
(except the software 'openssl' engine). All these engines currently load
shared-libraries for hardware APIs, so they've all been given "SO_PATH"
commands that will configure the chosen ENGINE to load its shared library
from the given path. Eg. by calling;
    ENGINE_ctrl_cmd_string(e, "SO_PATH", <path>, 0).

The nCipher 'chil' ENGINE has also had "FORK_CHECK" and "THREAD_LOCKING"
commands added so these settings could be handled via application-level
configuration rather than in application source code.

Changes to "openssl engine" to test and examine these control commands will
be made shortly. It will also provide the necessary tips to application
programs wanting to support these dynamic control commands.

This change adds some new functionality to the ENGINE code and API to
make it possible for ENGINEs to describe and implement their own control
commands that can be interrogated and used by calling applications at
run-time. The source code includes numerous comments explaining how it all
works and some of the finer details. But basically, an ENGINE will normally
declare an array of ENGINE_CMD_DEFN entries in its ENGINE - and the various
new ENGINE_CTRL_*** command types take care of iterating through this list
of definitions, converting command numbers to names, command names to
numbers, getting descriptions, getting input flags, etc. These
administrative commands are handled directly in the base ENGINE code rather
than in each ENGINE's ctrl() handler, unless they specify the
ENGINE_FLAGS_MANUAL_CMD_CTRL flag (ie. if they're doing something clever or
dynamic with the command definitions).

There is also a new function, ENGINE_cmd_is_executable(), that will
determine if an ENGINE control command is of an "executable" type that
can be used in another new function, ENGINE_ctrl_cmd_string(). If not, the
control command is not supposed to be exposed out to user/config level
access - eg. it could involve the exchange of binary data, returning
results to calling code, etc etc. If the command is executable then
ENGINE_ctrl_cmd_string() can be called using a name/arg string pair. The
control command's input flags will be used to determine necessary
conversions before the control command is called, and commands of this
form will always return zero or one (failure or success, respectively).
This is set up so that arbitrary applications can support control commands
in a consistent way so that tweaking particular ENGINE behaviour is
specific to the ENGINE and the host environment, and independant of the
application or OpenSSL.

Some code demonstrating this stuff in action will applied shortly to the
various ENGINE implementations, as well as "openssl engine" support for
executing arbitrary control commands before and/or after initialising
various ENGINEs.

The existing ENGINEs (including the default 'openssl' software engine) were
static, declared inside the source file for each engine implementation. The
reason this was not going boom was that all the ENGINEs had reference
counts that never hit zero (once linked into the internal list, each would
always have at least 1 lasting structural reference).

To fix this so it will stay standing when an "unload" function is added to
match ENGINE_load_builtin_engines(), the "constructor" functions for each
ENGINE implementation have been changed to dynamically allocate and
construct their own ENGINEs using API functions. The other benefit of this
is that no ENGINE implementation has to include the internal "engine_int.h"
header file any more.

static data where they could be parameterised by ctrl() commands.

Previously RAND_get_rand_method was returning a non-const pointer, but it
should be const. As with all other such cases, METHOD pointers are stored and
returned as "const". The only methods one should be able to alter are methods
"local" to the relevant code, in which case a non-const handle to the methods
should already exist.

This change has been forced by the constifying of the ENGINE code (before
which RAND_METHOD was the only method pointer in an ENGINE structure that was
not constant).

ENGINE handler functions should take the ENGINE structure as a parameter -
this is because ENGINE structures can be copied, and like other
structure/method setups in OpenSSL, it should be possible for init(),
finish(), ctrl(), etc to adjust state inside the ENGINE structures rather
than globally. This commit includes the dependant changes in the ENGINE
implementations.

Previous changes permanently removed the commented-out old code for where
it was possible to create and use an ENGINE statically, and this code gets
rid of the ENGINE_FLAGS_MALLOCED flag that supported the distinction with
dynamically allocated ENGINEs. It also moves the area for ENGINE_FLAGS_***
values from engine_int.h to engine.h - because it should be possible to
declare ENGINEs just from declarations in exported headers.

* Constify the get/set functions, and add some that functions were missing.

* Add a new 'ENGINE_cpy()' function that will produce a new ENGINE based
  copied from an original (except for the references, ie. the new copy will
  be like an ENGINE returned from 'ENGINE_new()' - a structural reference).

* Removed the "null parameter" checking in the get/set functions - it is
  legitimate to set NULL values as a way of *changing* an ENGINE (ie.
  removing a handler that previously existed). Also, passing a NULL pointer
  for an ENGINE is obviously wrong for these functions, so don't bother
  checking for it. The result is a number of error codes and strings could
  be removed.

to ENGINE_free(). Also, remove "#if 0" code that has no useful future.