#### 6.4.4.10 Using the HashiCorp Vault Keyring Plugin

[]()[]()

Note

 The `keyring_hashicorp` plugin is an extension included in MySQL Enterprise Edition, a commercial product. To learn more about commercial products, see [https://www.mysql.com/products/](https://www.mysql.com/products/).

 The `keyring_hashicorp` keyring plugin communicates with HashiCorp Vault for back end storage. The plugin supports HashiCorp Vault AppRole authentication. No key information is permanently stored in MySQL server local storage. (An optional in-memory key cache may be used as intermediate storage.) Random key generation is performed on the MySQL server side, with the keys subsequently stored to Hashicorp Vault.

 The `keyring_hashicorp` plugin supports the functions that comprise the standard MySQL Keyring service interface. Keyring operations performed by those functions are accessible at two levels:

* SQL interface: In SQL statements, call the functions described in [Section 6.4.4.14, “General-Purpose Keyring Key-Management Functions”](keyring-functions-general-purpose.html).

* C interface: In C-language code, call the keyring service functions described in [Section 5.6.9.2, “The Keyring Service”](keyring-service.html).

 Example (using the SQL interface):

```
SELECT keyring_key_generate('MyKey', 'AES', 32);
SELECT keyring_key_remove('MyKey');
```

 For information about the characteristics of key values permitted by `keyring_hashicorp`, see [Section 6.4.4.12, “Supported Keyring Key Types and Lengths”](keyring-key-types.html).

 To install `keyring_hashicorp`, use the general instructions found in [Section 6.4.4.3, “Keyring Plugin Installation”](keyring-plugin-installation.html), together with the configuration information specific to `keyring_hashicorp` found here. Plugin-specific configuration includes preparation of the certificate and key files needed for connecting to HashiCorp Vault, as well as configuring HashiCorp Vault itself. The following sections provide the necessary instructions.

* [Certificate and Key Preparation](keyring-hashicorp-plugin.html#keyring-hashicorp-certificate-configuration)

* [HashiCorp Vault Setup](keyring-hashicorp-plugin.html#keyring-hashicorp-vault-configuration)

* [keyring\_hashicorp Configuration](keyring-hashicorp-plugin.html#keyring-hashicorp-plugin-configuration)

##### Certificate and Key Preparation

[]()

 The `keyring_hashicorp` plugin requires a secure connection to the HashiCorp Vault server, employing the HTTPS protocol. A typical setup includes a set of certificate and key files:

* `company.crt`: A custom CA certificate belonging to the organization. This file is used both by HashiCorp Vault server and the `keyring_hashicorp` plugin.

* `vault.key`: The private key of the HashiCorp Vault server instance. This file is used by HashiCorp Vault server.

* `vault.crt`: The certificate of the HashiCorp Vault server instance. This file must be signed by the organization CA certificate.

 The following instructions describe how to create the certificate and key files using OpenSSL. (If you already have those files, proceeed to [HashiCorp Vault Setup](keyring-hashicorp-plugin.html#keyring-hashicorp-vault-configuration).) The instructions as shown apply to Linux platforms and may require adjustment for other platforms.

Important

 Certificates generated by these instructions are self-signed, which may not be very secure. After you gain experience using such files, consider obtaining certificate/key material from a registered certificate authority.

1. Prepare the company and HashiCorp Vault server keys.

    Use the following commands to generate the key files:

   ```
   openssl genrsa -aes256 -out company.key 4096
   openssl genrsa -aes256 -out vault.key 2048
   ```

    The commands produce files holding the company private key (`company.key`) and the Vault server private key (`vault.key`). The keys are randomly generated RSA keys of 4,096 and 2,048 bits, respectively.

    Each command prompts for a password. For testing purposes, the password is not required. To disable it, omit the `-aes256` argument.

    The key files hold sensitive information and should be stored in a secure location. The password (also sensitive) is required later, so write it down and store it in a secure location.

    (Optional) To check key file content and validity, use the following commands:

   ```
   openssl rsa -in company.key -check
   openssl rsa -in vault.key -check
   ```

2. Create the company CA certificate.

    Use the following command to create a company CA certificate file named `company.crt` that is valid for 365 days (enter the command on a single line):

   ```
   openssl req -x509 -new -nodes -key company.key
     -sha256 -days 365 -out company.crt
   ```

    If you used the `-aes256` argument to perform key encryption during key generation, you are prompted for the company key password during CA certificate creation. You are also prompted for information about the certificate holder (that is, you or your company), as shown here:

   ```
   Country Name (2 letter code) [AU]:
   State or Province Name (full name) [Some-State]:
   Locality Name (eg, city) []:
   Organization Name (eg, company) [Internet Widgits Pty Ltd]:
   Organizational Unit Name (eg, section) []:
   Common Name (e.g. server FQDN or YOUR name) []:
   Email Address []:
   ```

    Answer the prompts with appropriate values.

3. Create a certificate signing request.

    To create a HashiCorp Vault server certificate, a Certificate Signing Request (CSR) must be prepared for the newly created server key. Create a configuration file named `request.conf` containing the following lines. If the HashiCorp Vault server does not run on the local host, substitute appropriate CN and IP values, and make any other changes required.

   ```
   [req]
   distinguished_name = vault
   x509_entensions = v3_req
   prompt = no

   [vault]
   C = US
   ST = CA
   L = RWC
   O = Company
   CN = 127.0.0.1

   [v3_req]
   subjectAltName = @alternatives
   authorityKeyIdentifier = keyid,issuer
   basicConstraints = CA:TRUE

   [alternatives]
   IP = 127.0.0.1
   ```

    Use this command to create the signing request:

   ```
   openssl req -new -key vault.key -config request.conf -out request.csr
   ```

    The output file (`request.csr`) is an intermediate file that serves as input for creation of the server certificate.

4. Create the HashiCorp Vault server certificate.

    Sign the combined information from the HashiCorp Vault server key (`vault.key`) and the CSR (`request.csr`) with the company certificate (`company.crt`) to create the HashiCorp Vault server certificate (`vault.crt`). Use the following command to do this (enter the command on a single line):

   ```
   openssl x509 -req -in request.csr
     -CA company.crt -CAkey company.key -CAcreateserial
     -out vault.crt -days 365 -sha256
   ```

    To make the `vault.crt` server certificate useful, append the contents of the `company.crt` company certificate to it. This is required so that the company certificate is delivered along with the server certificate in requests.

   ```
   cat company.crt >> vault.crt
   ```

    If you display the contents of the `vault.crt` file, it should look like this:

   ```
   -----BEGIN CERTIFICATE-----
   ... content of HashiCorp Vault server certificate ...
   -----END CERTIFICATE-----
   -----BEGIN CERTIFICATE-----
   ... content of company certificate ...
   -----END CERTIFICATE-----
   ```

##### HashiCorp Vault Setup

[]()

 The following instructions describe how to create a HashiCorp Vault setup that facilitates testing the `keyring_hashicorp` plugin.

Important

 A test setup is similar to a production setup, but production use of HashiCorp Vault entails additional security considerations such as use of non-self-signed certificates and storing the company certificate in the system trust store. You must implement whatever additional security steps are needed to satisfy your operational requirements.

 These instructions assume availability of the certificate and key files created in [Certificate and Key Preparation](keyring-hashicorp-plugin.html#keyring-hashicorp-certificate-configuration). See that section if you do not have those files.

1. Fetch the HashiCorp Vault binary.

    Download the HashiCorp Vault binary appropriate for your platform from [https://www.vaultproject.io/downloads.html](https://www.vaultproject.io/downloads.html).

    Extract the content of the archive to produce the executable **vault** command, which is used to perform HashiCorp Vault operations. If necessary, add the directory where you install the command to the system path.

    (Optional) HashiCorp Vault supports autocomplete options that make it easier to use. For more information, see [https://learn.hashicorp.com/vault/getting-started/install#command-completion](https://learn.hashicorp.com/vault/getting-started/install#command-completion).

2. Create the HashiCorp Vault server configuration file.

    Prepare a configuration file named `config.hcl` with the following content. For the `tls_cert_file`, `tls_key_file`, and `path` values, substitute path names appropriate for your system.

   ```
   listener "tcp" {
     address="127.0.0.1:8200"
     tls_cert_file="/home/username/certificates/vault.crt"
     tls_key_file="/home/username/certificates/vault.key"
   }

   storage "file" {
     path = "/home/username/vaultstorage/storage"
   }

   ui = true
   ```

3. Start the HashiCorp Vault server.

    To start the Vault server, use the following command, where the `-config` option specifies the path to the configuration file just created:

   ```
   vault server -config=config.hcl
   ```

    During this step, you may be prompted for a password for the Vault server private key stored in the `vault.key` file.

    The server should start, displaying some information on the console (IP, port, and so forth).

    So that you can enter the remaining commands, put the **vault server** command in the background or open another terminal before continuing.

4. Initialize the HashiCorp Vault server.

   Note

    The operations described in this step are required only when starting Vault the first time, to obtain the unseal key and root token. Subsequent Vault instance restarts require only unsealing using the unseal key.

    Issue the following commands (assuming Bourne shell syntax):

   ```
   export VAULT_SKIP_VERIFY=1
   vault operator init -n 1 -t 1
   ```

    The first command enables the **vault** command to temporarily ignore the fact that no company certificate has been added to the system trust store. It compensates for the fact that our self-signed CA is not added to that store. (For production use, such a certificate should be added.)

    The second command creates a single unseal key with a requirement for a single unseal key to be present for unsealing. (For production use, an instance would have multiple unseal keys with up to that many keys required to be entered to unseal it. The unseal keys should be delivered to key custodians within the company. Use of a single key might be considered a security issue because that permits the vault to be unsealed by a single key custodian.)

    Vault should reply with information about the unseal key and root token, plus some additional text (the actual unseal key and root token values differ from those shown here):

   ```
   ...
   Unseal Key 1: I2xwcFQc892O0Nt2pBiRNlnkHzTUrWS+JybL39BjcOE=
   Initial Root Token: s.vTvXeo3tPEYehfcd9WH7oUKz
   ...
   ```

    Store the unseal key and root token in a secure location.

5. Unseal the HashiCorp Vault server.

    Use this command to unseal the Vault server:

   ```
   vault operator unseal
   ```

    When prompted to enter the unseal key, use the key obtained previously during Vault initialization.

    Vault should produce output indicating that setup is complete and the vault is unsealed.

6. Log in to the HashiCorp Vault server and verify its status.

    Prepare the environment variables required for logging in as root:

   ```
   vault login s.vTvXeo3tPEYehfcd9WH7oUKz
   ```

    For the token value in that command, substitute the content of the root token obtained previously during Vault initialization.

    Verify the Vault server status:

   ```
   vault status
   ```

    The output should contain these lines (among others):

   ```
   ...
   Initialized     true
   Sealed          false
   ...
   ```

7. Set up HashiCorp Vault authentication and storage.

   Note

    The operations described in this step are needed only the first time the Vault instance is run. They need not be repeated afterward.

    Enable the AppRole authentication method and verify that it is in the authentication method list:

   ```
   vault auth enable approle
   vault auth list
   ```

    Enable the Vault KeyValue storage engine:

   ```
   vault secrets enable -version=1 kv
   ```

    Create and set up a role for use with the `keyring_hashicorp` plugin (enter the command on a single line):

   ```
   vault write auth/approle/role/mysql token_num_uses=0
     token_ttl=20m token_max_ttl=30m secret_id_num_uses=0
   ```

8. Add an AppRole security policy.

   Note

    The operations described in this step are needed only the first time the Vault instance is run. They need not be repeated afterward.

    Prepare a policy that to permit the previously created role to access appropriate secrets. Create a new file named `mysql.hcl` with the following content:

   ```
   path "kv/mysql/*" {
     capabilities = ["create", "read", "update", "delete", "list"]
   }
   ```

   Note

   `kv/mysql/` in this example may need adjustment per your local installation policies and security requirements. If so, make the same adjustment wherever else `kv/mysql/` appears in these instructions.

    Import the policy file to the Vault server to create a policy named `mysql-policy`, then assign the policy to the new role:

   ```
   vault policy write mysql-policy mysql.hcl
   vault write auth/approle/role/mysql policies=mysql-policy
   ```

    Obtain the ID of the newly created role and store it in a secure location:

   ```
   vault read auth/approle/role/mysql/role-id
   ```

    Generate a secret ID for the role and store it in a secure location:

   ```
   vault write -f auth/approle/role/mysql/secret-id
   ```

    After these AppRole role ID and secret ID credentials are generated, they are expected to remain valid indefinitely. They need not be generated again and the `keyring_hashicorp` plugin can be configured with them for use on an ongoing basis. For more information about AuthRole authentication, visit [https://www.vaultproject.io/docs/auth/approle.html](https://www.vaultproject.io/docs/auth/approle.html).

##### keyring\_hashicorp Configuration

[]()

 The plugin library file contains the `keyring_hashicorp` plugin and a loadable function, [`keyring_hashicorp_update_config()`](keyring-functions-plugin-specific.html#function_keyring-hashicorp-update-config). When the plugin initializes and terminates, it automatically loads and unloads the function. There is no need to load and unload the function manually.

 The `keyring_hashicorp` plugin supports the configuration parameters shown in the following table. To specify these parameters, assign values to the corresponding system variables.

|Configuration Parameter |                                          System Variable                                          |Mandatory|
|------------------------|---------------------------------------------------------------------------------------------------|---------|
|  HashiCorp Server URL  |[`keyring_hashicorp_server_url`](keyring-system-variables.html#sysvar_keyring_hashicorp_server_url)|   No    |
|    AppRole role ID     |   [`keyring_hashicorp_role_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_role_id)   |   Yes   |
|   AppRole secret ID    | [`keyring_hashicorp_secret_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_secret_id) |   Yes   |
|       Store path       |[`keyring_hashicorp_store_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_store_path)|   Yes   |
|   Authorization Path   | [`keyring_hashicorp_auth_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_auth_path) |   No    |
|CA certificate file path|   [`keyring_hashicorp_ca_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_ca_path)   |   No    |
|     Cache control      |   [`keyring_hashicorp_caching`](keyring-system-variables.html#sysvar_keyring_hashicorp_caching)   |   No    |

 To be usable during the server startup process, `keyring_hashicorp` must be loaded using the [`--early-plugin-load`](server-options.html#option_mysqld_early-plugin-load) option. As indicated by the preceding table, several plugin-related system variables are mandatory and must also be set. For example, use these lines in the server `my.cnf` file, adjusting the `.so` suffix and file locations for your platform as necessary:

```
[mysqld]
early-plugin-load=keyring_hashicorp.so
keyring_hashicorp_role_id='ee3b495c-d0c9-11e9-8881-8444c71c32aa'
keyring_hashicorp_secret_id='0512af29-d0ca-11e9-95ee-0010e00dd718'
keyring_hashicorp_store_path='/v1/kv/mysql'
keyring_hashicorp_auth_path='/v1/auth/approle/login'
```

Note

 Per the [HashiCorp documentation](https://www.vaultproject.io/api-docs), all API routes are prefixed with a protocol version (which you can see in the preceding example as `/v1/` in the [`keyring_hashicorp_store_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_store_path) and [`keyring_hashicorp_auth_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_auth_path) values). If HashiCorp develops new protocol versions, it may be necessary to change `/v1/` to something else in your configuration.

 MySQL Server authenticates against HashiCorp Vault using AppRole authentication. Successful authentication requires that two secrets be provided to Vault, a role ID and a secret ID, which are similar in concept to user name and password. The role ID and secret ID values to use are those obtained during the HashiCorp Vault setup procedure performed previously. To specify the two IDs, assign their respective values to the [`keyring_hashicorp_role_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_role_id) and [`keyring_hashicorp_secret_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_secret_id) system variables. The setup procedure also results in a store path of `/v1/kv/mysql`, which is the value to assign to [`keyring_hashicorp_commit_store_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_commit_store_path).

 At plugin initialization time, `keyring_hashicorp` attempts to connect to the HashiCorp Vault server using the configuration values. If the connection is successful, the plugin stores the values in corresponding system variables that have `_commit_` in their name. For example, upon successful connection, the plugin stores the values of [`keyring_hashicorp_role_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_role_id) and [`keyring_hashicorp_store_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_store_path) in [`keyring_hashicorp_commit_role_id`](keyring-system-variables.html#sysvar_keyring_hashicorp_commit_role_id) and [`keyring_hashicorp_commit_store_path`](keyring-system-variables.html#sysvar_keyring_hashicorp_commit_store_path).

 Reconfiguration at runtime can be performed with the assistance of the [`keyring_hashicorp_update_config()`](keyring-functions-plugin-specific.html#function_keyring-hashicorp-update-config) function:

1. Use [`SET`](set-variable.html) statements to assign the desired new values to the configuration system variables shown in the preceding table. These assignments in themselves have no effect on ongoing plugin operation.

2. Invoke [`keyring_hashicorp_update_config()`](keyring-functions-plugin-specific.html#function_keyring-hashicorp-update-config) to cause the plugin to reconfigure and reconnect to the HashiCorp Vault server using the new variable values.

3. If the connection is successful, the plugin stores the updated configuration values in corresponding system variables that have `_commit_` in their name.

 For example, if you have reconfigured HashiCorp Vault to listen on port 8201 rather than the default 8200, reconfigure `keyring_hashicorp` like this:

```
mysql> SET GLOBAL keyring_hashicorp_server_url = 'https://127.0.0.1:8201';
Query OK, 0 rows affected (0.00 sec)

mysql> SELECT keyring_hashicorp_update_config();
+--------------------------------------+
| keyring_hashicorp_update_config()    |
+--------------------------------------+
| Configuration update was successful. |
+--------------------------------------+
1 row in set (0.03 sec)
```

 If the plugin is not able to connect to HashiCorp Vault during initialization or reconfiguration and there was no existing connection, the `_commit_` system variables are set to `'Not committed'` for string-valued variables, and `OFF` for Boolean-valued variables. If the plugin is not able to connect but there was an existing connection, that connection remains active and the `_commit_` variables reflect the values used for it.

Note

 If you do not set the mandatory system variables at server startup, or if some other plugin initialization error occurs, initialization fails. In this case, you can use the runtime reconfiguration procedure to initialize the plugin without restarting the server.

 For additional information about the `keyring_hashicorp` plugin-specific system variables and function, see [Section 6.4.4.18, “Keyring System Variables”](keyring-system-variables.html), and [Section 6.4.4.15, “Plugin-Specific Keyring Key-Management Functions”](keyring-functions-plugin-specific.html).

