#### 8.12.3.1 How MySQL Uses Memory

[]()

 MySQL allocates buffers and caches to improve performance of database operations. The default configuration is designed to permit a MySQL server to start on a virtual machine that has approximately 512MB of RAM. You can improve MySQL performance by increasing the values of certain cache and buffer-related system variables. You can also modify the default configuration to run MySQL on systems with limited memory.

 The following list describes some of the ways that MySQL uses memory. Where applicable, relevant system variables are referenced. Some items are storage engine or feature specific.

* The `InnoDB` buffer pool is a memory area that holds cached `InnoDB` data for tables, indexes, and other auxiliary buffers. For efficiency of high-volume read operations, the buffer pool is divided into [pages](glossary.html#glos_page) that can potentially hold multiple rows. For efficiency of cache management, the buffer pool is implemented as a linked list of pages; data that is rarely used is aged out of the cache, using a variation of the [LRU](glossary.html#glos_lru) algorithm. For more information, see [Section 15.5.1, “Buffer Pool”](innodb-buffer-pool.html).

   The size of the buffer pool is important for system performance:

  * `InnoDB` allocates memory for the entire buffer pool at server startup, using `malloc()` operations. The [`innodb_buffer_pool_size`](innodb-parameters.html#sysvar_innodb_buffer_pool_size) system variable defines the buffer pool size. Typically, a recommended [`innodb_buffer_pool_size`](innodb-parameters.html#sysvar_innodb_buffer_pool_size) value is 50 to 75 percent of system memory. [`innodb_buffer_pool_size`](innodb-parameters.html#sysvar_innodb_buffer_pool_size) can be configured dynamically, while the server is running. For more information, see [Section 15.8.3.1, “Configuring InnoDB Buffer Pool Size”](innodb-buffer-pool-resize.html).

  * On systems with a large amount of memory, you can improve concurrency by dividing the buffer pool into multiple [buffer pool instances](glossary.html#glos_buffer_pool_instance). The [`innodb_buffer_pool_instances`](innodb-parameters.html#sysvar_innodb_buffer_pool_instances) system variable defines the number of buffer pool instances.

  * A buffer pool that is too small may cause excessive churning as pages are flushed from the buffer pool only to be required again a short time later.

  * A buffer pool that is too large may cause swapping due to competition for memory.

* The storage engine interface enables the optimizer to provide information about the size of the record buffer to be used for scans that the optimizer estimates are likely to read multiple rows. The buffer size can vary based on the size of the estimate. `InnoDB` uses this variable-size buffering capability to take advantage of row prefetching, and to reduce the overhead of latching and B-tree navigation.

* All threads share the [`MyISAM`](myisam-storage-engine.html) key buffer. The [`key_buffer_size`](server-system-variables.html#sysvar_key_buffer_size) system variable determines its size.

   For each `MyISAM` table the server opens, the index file is opened once; the data file is opened once for each concurrently running thread that accesses the table. For each concurrent thread, a table structure, column structures for each column, and a buffer of size `3 * *`N`*` are allocated (where *`N`* is the maximum row length, not counting [`BLOB`](blob.html) columns). A [`BLOB`](blob.html) column requires five to eight bytes plus the length of the [`BLOB`](blob.html) data. The `MyISAM` storage engine maintains one extra row buffer for internal use.

* The [`myisam_use_mmap`](server-system-variables.html#sysvar_myisam_use_mmap) system variable can be set to 1 to enable memory-mapping for all `MyISAM` tables.

* If an internal in-memory temporary table becomes too large (as determined using the [`tmp_table_size`](server-system-variables.html#sysvar_tmp_table_size) and [`max_heap_table_size`](server-system-variables.html#sysvar_max_heap_table_size) system variables), MySQL automatically converts the table from in-memory to on-disk format. As of MySQL 8.0.16, on-disk temporary tables always use the InnoDB storage engine. (Previously, the storage engine employed for this purpose was determined by the [`internal_tmp_disk_storage_engine`](server-system-variables.html#sysvar_internal_tmp_disk_storage_engine) system variable, which is no longer supported.) You can increase the permissible temporary table size as described in [Section 8.4.4, “Internal Temporary Table Use in MySQL”](internal-temporary-tables.html).

   For [`MEMORY`](memory-storage-engine.html) tables explicitly created with [`CREATE TABLE`](create-table.html), only the [`max_heap_table_size`](server-system-variables.html#sysvar_max_heap_table_size) system variable determines how large a table can grow, and there is no conversion to on-disk format.

* The [MySQL Performance Schema](performance-schema.html) is a feature for monitoring MySQL server execution at a low level. The Performance Schema dynamically allocates memory incrementally, scaling its memory use to actual server load, instead of allocating required memory during server startup. Once memory is allocated, it is not freed until the server is restarted. For more information, see [Section 27.17, “The Performance Schema Memory-Allocation Model”](performance-schema-memory-model.html).

* Each thread that the server uses to manage client connections requires some thread-specific space. The following list indicates these and which system variables control their size:

  * A stack ([`thread_stack`](server-system-variables.html#sysvar_thread_stack))

  * A connection buffer ([`net_buffer_length`](server-system-variables.html#sysvar_net_buffer_length))

  * A result buffer ([`net_buffer_length`](server-system-variables.html#sysvar_net_buffer_length))

   The connection buffer and result buffer each begin with a size equal to [`net_buffer_length`](server-system-variables.html#sysvar_net_buffer_length) bytes, but are dynamically enlarged up to [`max_allowed_packet`](server-system-variables.html#sysvar_max_allowed_packet) bytes as needed. The result buffer shrinks to [`net_buffer_length`](server-system-variables.html#sysvar_net_buffer_length) bytes after each SQL statement. While a statement is running, a copy of the current statement string is also allocated.

   Each connection thread uses memory for computing statement digests. The server allocates [`max_digest_length`](server-system-variables.html#sysvar_max_digest_length) bytes per session. See [Section 27.10, “Performance Schema Statement Digests and Sampling”](performance-schema-statement-digests.html).

* All threads share the same base memory.

* When a thread is no longer needed, the memory allocated to it is released and returned to the system unless the thread goes back into the thread cache. In that case, the memory remains allocated.

* Each request that performs a sequential scan of a table allocates a read buffer. The [`read_buffer_size`](server-system-variables.html#sysvar_read_buffer_size) system variable determines the buffer size.

* When reading rows in an arbitrary sequence (for example, following a sort), a random-read buffer may be allocated to avoid disk seeks. The [`read_rnd_buffer_size`](server-system-variables.html#sysvar_read_rnd_buffer_size) system variable determines the buffer size.

* All joins are executed in a single pass, and most joins can be done without even using a temporary table. Most temporary tables are memory-based hash tables. Temporary tables with a large row length (calculated as the sum of all column lengths) or that contain [`BLOB`](blob.html) columns are stored on disk.

* Most requests that perform a sort allocate a sort buffer and zero to two temporary files depending on the result set size. See [Section B.3.3.5, “Where MySQL Stores Temporary Files”](temporary-files.html).

* Almost all parsing and calculating is done in thread-local and reusable memory pools. No memory overhead is needed for small items, thus avoiding the normal slow memory allocation and freeing. Memory is allocated only for unexpectedly large strings.

* For each table having [`BLOB`](blob.html) columns, a buffer is enlarged dynamically to read in larger [`BLOB`](blob.html) values. If you scan a table, the buffer grows as large as the largest [`BLOB`](blob.html) value.

* MySQL requires memory and descriptors for the table cache. Handler structures for all in-use tables are saved in the table cache and managed as “First In, First Out” (FIFO). The [`table_open_cache`](server-system-variables.html#sysvar_table_open_cache) system variable defines the initial table cache size; see [Section 8.4.3.1, “How MySQL Opens and Closes Tables”](table-cache.html).

   MySQL also requires memory for the table definition cache. The [`table_definition_cache`](server-system-variables.html#sysvar_table_definition_cache) system variable defines the number of table definitions that can be stored in the table definition cache. If you use a large number of tables, you can create a large table definition cache to speed up the opening of tables. The table definition cache takes less space and does not use file descriptors, unlike the table cache.

* A [`FLUSH TABLES`](flush.html#flush-tables) statement or [**mysqladmin flush-tables**](mysqladmin.html) command closes all tables that are not in use at once and marks all in-use tables to be closed when the currently executing thread finishes. This effectively frees most in-use memory. [`FLUSH TABLES`](flush.html#flush-tables) does not return until all tables have been closed.

* The server caches information in memory as a result of [`GRANT`](grant.html), [`CREATE USER`](create-user.html), [`CREATE SERVER`](create-server.html), and [`INSTALL PLUGIN`](install-plugin.html) statements. This memory is not released by the corresponding [`REVOKE`](revoke.html), [`DROP USER`](drop-user.html), [`DROP SERVER`](drop-server.html), and [`UNINSTALL PLUGIN`](uninstall-plugin.html) statements, so for a server that executes many instances of the statements that cause caching, there is an increase in cached memory use unless it is freed with [`FLUSH PRIVILEGES`](flush.html#flush-privileges).

* In a replication topology, the following settings affect memory usage, and can be adjusted as required:

  * The [`max_allowed_packet`](server-system-variables.html#sysvar_max_allowed_packet) system variable on a replication source limits the maximum message size that the source sends to its replicas for processing. This setting defaults to 64M.

  * The system variable [`replica_pending_jobs_size_max`](replication-options-replica.html#sysvar_replica_pending_jobs_size_max) (from MySQL 8.0.26) or [`slave_pending_jobs_size_max`](replication-options-replica.html#sysvar_slave_pending_jobs_size_max) (before MySQL 8.0.26) on a multithreaded replica sets the maximum amount of memory that is made available for holding messages awaiting processing. This setting defaults to 128M. The memory is only allocated when needed, but it might be used if your replication topology handles large transactions sometimes. It is a soft limit, and larger transactions can be processed.

  * The [`rpl_read_size`](replication-options-replica.html#sysvar_rpl_read_size) system variable on a replication source or replica controls the minimum amount of data in bytes that is read from the binary log files and relay log files. The default is 8192 bytes. A buffer the size of this value is allocated for each thread that reads from the binary log and relay log files, including dump threads on sources and coordinator threads on replicas.

  * The [`binlog_transaction_dependency_history_size`](replication-options-binary-log.html#sysvar_binlog_transaction_dependency_history_size) system variable limits the number of row hashes held as an in-memory history.

  * The [`max_binlog_cache_size`](replication-options-binary-log.html#sysvar_max_binlog_cache_size) system variable specifies the upper limit of memory usage by an individual transaction.

  * The [`max_binlog_stmt_cache_size`](replication-options-binary-log.html#sysvar_max_binlog_stmt_cache_size) system variable specifies the upper limit of memory usage by the statement cache.

**ps** and other system status programs may report that [**mysqld**](mysqld.html) uses a lot of memory. This may be caused by thread stacks on different memory addresses. For example, the Solaris version of **ps** counts the unused memory between stacks as used memory. To verify this, check available swap with `swap -s`. We test [**mysqld**](mysqld.html) with several memory-leakage detectors (both commercial and Open Source), so there should be no memory leaks.

##### Monitoring MySQL Memory Usage

[]()[]()

 The following example demonstrates how to use [Performance Schema](performance-schema.html) and [sys schema](sys-schema.html) to monitor MySQL memory usage.

 Most Performance Schema memory instrumentation is disabled by default. Instruments can be enabled by updating the `ENABLED` column of the Performance Schema [`setup_instruments`](performance-schema-setup-instruments-table.html) table. Memory instruments have names in the form of `memory/*`code_area`*/*`instrument_name`*`, where *`code_area`* is a value such as `sql` or `innodb`, and *`instrument_name`* is the instrument detail.

1. To view available MySQL memory instruments, query the Performance Schema [`setup_instruments`](performance-schema-setup-instruments-table.html) table. The following query returns hundreds of memory instruments for all code areas.

   ```
   mysql> SELECT * FROM performance_schema.setup_instruments
          WHERE NAME LIKE '%memory%';
   ```

    You can narrow results by specifying a code area. For example, you can limit results to `InnoDB` memory instruments by specifying `innodb` as the code area.

   ```
   mysql> SELECT * FROM performance_schema.setup_instruments
          WHERE NAME LIKE '%memory/innodb%';
   +-------------------------------------------+---------+-------+
   | NAME                                      | ENABLED | TIMED |
   +-------------------------------------------+---------+-------+
   | memory/innodb/adaptive hash index         | NO      | NO    |
   | memory/innodb/buf_buf_pool                | NO      | NO    |
   | memory/innodb/dict_stats_bg_recalc_pool_t | NO      | NO    |
   | memory/innodb/dict_stats_index_map_t      | NO      | NO    |
   | memory/innodb/dict_stats_n_diff_on_level  | NO      | NO    |
   | memory/innodb/other                       | NO      | NO    |
   | memory/innodb/row_log_buf                 | NO      | NO    |
   | memory/innodb/row_merge_sort              | NO      | NO    |
   | memory/innodb/std                         | NO      | NO    |
   | memory/innodb/trx_sys_t::rw_trx_ids       | NO      | NO    |
   ...
   ```

    Depending on your MySQL installation, code areas may include `performance_schema`, `sql`, `client`, `innodb`, `myisam`, `csv`, `memory`, `blackhole`, `archive`, `partition`, and others.

2. To enable memory instruments, add a `performance-schema-instrument` rule to your MySQL configuration file. For example, to enable all memory instruments, add this rule to your configuration file and restart the server:

   ```
   performance-schema-instrument='memory/%=COUNTED'
   ```

   Note

    Enabling memory instruments at startup ensures that memory allocations that occur at startup are counted.

    After restarting the server, the `ENABLED` column of the Performance Schema [`setup_instruments`](performance-schema-setup-instruments-table.html) table should report `YES` for memory instruments that you enabled. The `TIMED` column in the [`setup_instruments`](performance-schema-setup-instruments-table.html) table is ignored for memory instruments because memory operations are not timed.

   ```
   mysql> SELECT * FROM performance_schema.setup_instruments
          WHERE NAME LIKE '%memory/innodb%';
   +-------------------------------------------+---------+-------+
   | NAME                                      | ENABLED | TIMED |
   +-------------------------------------------+---------+-------+
   | memory/innodb/adaptive hash index         | NO      | NO    |
   | memory/innodb/buf_buf_pool                | NO      | NO    |
   | memory/innodb/dict_stats_bg_recalc_pool_t | NO      | NO    |
   | memory/innodb/dict_stats_index_map_t      | NO      | NO    |
   | memory/innodb/dict_stats_n_diff_on_level  | NO      | NO    |
   | memory/innodb/other                       | NO      | NO    |
   | memory/innodb/row_log_buf                 | NO      | NO    |
   | memory/innodb/row_merge_sort              | NO      | NO    |
   | memory/innodb/std                         | NO      | NO    |
   | memory/innodb/trx_sys_t::rw_trx_ids       | NO      | NO    |
   ...
   ```

3. Query memory instrument data. In this example, memory instrument data is queried in the Performance Schema [`memory_summary_global_by_event_name`](performance-schema-memory-summary-tables.html) table, which summarizes data by `EVENT_NAME`. The `EVENT_NAME` is the name of the instrument.

    The following query returns memory data for the `InnoDB` buffer pool. For column descriptions, see [Section 27.12.20.10, “Memory Summary Tables”](performance-schema-memory-summary-tables.html).

   ```
   mysql> SELECT * FROM performance_schema.memory_summary_global_by_event_name
          WHERE EVENT_NAME LIKE 'memory/innodb/buf_buf_pool'\G
                     EVENT_NAME: memory/innodb/buf_buf_pool
                    COUNT_ALLOC: 1
                     COUNT_FREE: 0
      SUM_NUMBER_OF_BYTES_ALLOC: 137428992
       SUM_NUMBER_OF_BYTES_FREE: 0
                 LOW_COUNT_USED: 0
             CURRENT_COUNT_USED: 1
                HIGH_COUNT_USED: 1
       LOW_NUMBER_OF_BYTES_USED: 0
   CURRENT_NUMBER_OF_BYTES_USED: 137428992
      HIGH_NUMBER_OF_BYTES_USED: 137428992
   ```

    The same underlying data can be queried using the [`sys`](sys-schema.html) schema [`memory_global_by_current_bytes`](sys-memory-global-by-current-bytes.html) table, which shows current memory usage within the server globally, broken down by allocation type.

   ```
   mysql> SELECT * FROM sys.memory_global_by_current_bytes
          WHERE event_name LIKE 'memory/innodb/buf_buf_pool'\G
   *************************** 1. row ***************************
          event_name: memory/innodb/buf_buf_pool
       current_count: 1
       current_alloc: 131.06 MiB
   current_avg_alloc: 131.06 MiB
          high_count: 1
          high_alloc: 131.06 MiB
      high_avg_alloc: 131.06 MiB
   ```

    This [`sys`](sys-schema.html) schema query aggregates currently allocated memory (`current_alloc`) by code area:

   ```
   mysql> SELECT SUBSTRING_INDEX(event_name,'/',2) AS
          code_area, FORMAT_BYTES(SUM(current_alloc))
          AS current_alloc
          FROM sys.x$memory_global_by_current_bytes
          GROUP BY SUBSTRING_INDEX(event_name,'/',2)
          ORDER BY SUM(current_alloc) DESC;
   +---------------------------+---------------+
   | code_area                 | current_alloc |
   +---------------------------+---------------+
   | memory/innodb             | 843.24 MiB    |
   | memory/performance_schema | 81.29 MiB     |
   | memory/mysys              | 8.20 MiB      |
   | memory/sql                | 2.47 MiB      |
   | memory/memory             | 174.01 KiB    |
   | memory/myisam             | 46.53 KiB     |
   | memory/blackhole          | 512 bytes     |
   | memory/federated          | 512 bytes     |
   | memory/csv                | 512 bytes     |
   | memory/vio                | 496 bytes     |
   +---------------------------+---------------+
   ```

   Note

    Prior to MySQL 8.0.16, `sys.format_bytes()` was used for [`FORMAT_BYTES()`](performance-schema-functions.html#function_format-bytes).

    For more information about [`sys`](sys-schema.html) schema, see [Chapter 28, *MySQL sys Schema*](sys-schema.html).

