diff --git a/blog/mysq-advance/img/innodb-store.png b/blog/mysq-advance/img/innodb-store.png new file mode 100644 index 0000000000000000000000000000000000000000..729cbd3521667fe0a4ea5cf1c0abd7bde2e68292 Binary files /dev/null and b/blog/mysq-advance/img/innodb-store.png differ diff --git a/blog/mysq-advance/img/page-merge-1.png b/blog/mysq-advance/img/page-merge-1.png new file mode 100644 index 0000000000000000000000000000000000000000..68e9a56f47cade859a5280aabfd49aa4c8ad6b20 Binary files /dev/null and b/blog/mysq-advance/img/page-merge-1.png differ diff --git a/blog/mysq-advance/img/page-merge-2.png b/blog/mysq-advance/img/page-merge-2.png new file mode 100644 index 0000000000000000000000000000000000000000..b627276c444484bc74374a72f6eca55f19acc554 Binary files /dev/null and b/blog/mysq-advance/img/page-merge-2.png differ diff --git a/blog/mysq-advance/img/page-split-1.png b/blog/mysq-advance/img/page-split-1.png new file mode 100644 index 0000000000000000000000000000000000000000..3989c67e80490326f746e54c33b999142d53ae63 Binary files /dev/null and b/blog/mysq-advance/img/page-split-1.png differ diff --git a/blog/mysq-advance/img/page-split-2.png b/blog/mysq-advance/img/page-split-2.png new file mode 100644 index 0000000000000000000000000000000000000000..39eca06390307be4623d266639dbdd1a644a7e5a Binary files /dev/null and b/blog/mysq-advance/img/page-split-2.png differ diff --git a/blog/mysq-advance/index.md b/blog/mysq-advance/index.md index 28ae586e6a0786b7e43429711cb2b80295a96cf7..efe7a9fb78f10534c2c33e163afc91998639bf1e 100644 --- a/blog/mysq-advance/index.md +++ b/blog/mysq-advance/index.md @@ -36,4 +36,4 @@ MySQL 运维篇 - 3. [https://www.yuque.com/heychu/akbndd/zhuxqd](https://www.yuque.com/heychu/akbndd/zhuxqd) -https://www.bilibili.com/video/BV1Kr4y1i7ru?p=90&spm_id_from=pageDriver \ No newline at end of file +https://www.bilibili.com/video/BV1Kr4y1i7ru?p=92&spm_id_from=pageDriver \ No newline at end of file diff --git a/blog/mysq-advance/sql-optimization.md b/blog/mysq-advance/sql-optimization.md index b4c7524bc6d08255fb643a0c0ab8659fdde648a5..0bbd47c0492316195bd67993008edade50686307 100644 --- a/blog/mysq-advance/sql-optimization.md +++ b/blog/mysq-advance/sql-optimization.md @@ -205,3 +205,252 @@ mysql> select count(*) from tb_user; ``` +## 2、主键优化 + +### 2.1、数据组织方式 + +在InnoDB存储引擎中，表数据都是根据主键顺序组织存放的， + +这种存储方式的表称为`索引组织表`（index organized table IOT） + +![](img/mysql-b+tree.png) + +![](img/innodb-store.png) + + +### 2.2、页分裂 + +页可以为空，也可以填充一半，也可以填充100%。 + +每个页包含了2-N行数据（如果一行数据太大，会行溢出），根据主键排列 + +如果插入的数据主键不是顺序的，有可能发生页分裂 + +![](img/page-split-1.png) + +![](img/page-split-2.png) + +### 2.3、页合并 + +当删除一行记录的时候，实际上记录并没有被物理删除，只是记录被标记（flaged）为删除。 + +并且它的空间变得允许被其他记录声明使用。 + +当页中删除的记录达到 MERGE_THRESHOLD (默认为页的50%)， + +InnoDB会开始寻找最靠近的页（前或后）看看是否可以将两个页合并优化空间使用。 + +MERGE_THRESHOLD ： 合并也的阈值，可以自己设置，在创建表或者创建索引时指定 + +![](img/page-merge-1.png) + +![](img/page-merge-2.png) + +### 2.4、主键设计原则 + +1. 满足业务需求的情况下，尽量降低主键的长度（二级索引叶子节点存放主键，主键过长会占用大量磁盘空间） + +2. 插入数据时，尽量选择顺序插入，选择使用 `auto_increment` 自增主键（不是顺序插入，会造成页分裂） + +3. 尽量不要使用UUID做主键或者是其他自然主键，如身份证号（长度较长，耗费磁盘io） + +4. 业务操作时，避免对主键的修改 + + + + + + + + + + + + + + + + + + + + + + +## 3、order by 优化 + +- using filesort：通过表的索引或全表扫描，读取满足条件的数据行，然后在`排序缓冲区sort buffer`中完成排序操作，所有不是通过索引直接返回排序结果的排序都叫FileSort排序 + +- using index: 通过有序索引顺序扫描直接返回有序数据，这种情况即为using index,不需要额外排序，操作效率高 + + +测试数据 + +```sql +-- 查看表数据 +mysql> select * from tb_user limit 3; ++----+-----------+-------------+---------------------+------+--------+----------------+ +| id | name | phone | profession | age | status | email | ++----+-----------+-------------+---------------------+------+--------+----------------+ +| 1 | 费阳 | 13777763170 | 法务经理 | 27 | 1 | wyao@gmail.com | +| 2 | 祁海燕 | 13400806360 | 日式厨师 | 23 | 0 | jwan@jin.cn | +| 3 | 姬秀英 | 18281241586 | 食品/饮料研发 | 29 | 0 | li97@wang.cn | ++----+-----------+-------------+---------------------+------+--------+----------------+ +3 rows in set (0.01 sec) + +-- 查看索引 +mysql> show index from tb_user; ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| tb_user | 0 | PRIMARY | 1 | id | A | 9804 | NULL | NULL | | BTREE | | | YES | NULL | +| tb_user | 0 | idx_user_phone | 1 | phone | A | 9804 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 0 | idx_user_phone_name | 1 | phone | A | 9804 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 0 | idx_user_phone_name | 2 | name | A | 9804 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_name | 1 | name | A | 9130 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 1 | profession | A | 948 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 2 | age | A | 6232 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 3 | status | A | 7596 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_email_5 | 1 | email | A | 3955 | 5 | NULL | YES | BTREE | | | YES | NULL | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +9 rows in set (0.03 sec) + +-- 删除索引 +drop index idx_user_phone on tb_user; + +drop index idx_user_phone_name on tb_user; + +drop index idx_user_name on tb_user; + +show index from tb_user; ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| tb_user | 0 | PRIMARY | 1 | id | A | 9804 | NULL | NULL | | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 1 | profession | A | 948 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 2 | age | A | 6232 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 3 | status | A | 7596 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_email_5 | 1 | email | A | 3955 | 5 | NULL | YES | BTREE | | | YES | NULL | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +5 rows in set (0.00 sec) +``` + +索引测试 + +```sql +-- 没有创建索引时，根据age, phone进行排序 Using filesort +mysql> explain select id, age, phone from tb_user order by age, phone; ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +| 1 | SIMPLE | tb_user | NULL | ALL | NULL | NULL | NULL | NULL | 9804 | 100.00 | Using filesort | ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +1 row in set, 1 warning (0.00 sec) + +-- 创建索引 +create index idx_user_age_phone on tb_user(age, phone); + +-- 创建索引后，根据age进行升序排序 +mysql> explain select id, age, phone from tb_user order by age; ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone | 52 | NULL | 9804 | 100.00 | Using index | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +1 row in set, 1 warning (0.00 sec) + +-- 创建索引后，根据age, phone进行升序排序 +mysql> explain select id, age, phone from tb_user order by age, phone; ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone | 52 | NULL | 9804 | 100.00 | Using index | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-------------+ +1 row in set, 1 warning (0.01 sec) + +-- 创建索引后，根据age, phone进行降序排序 +mysql> explain select id, age, phone from tb_user order by age desc, phone desc; ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+----------------------------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+----------------------------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone | 52 | NULL | 9804 | 100.00 | Backward index scan; Using index | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+----------------------------------+ +1 row in set, 1 warning (0.00 sec) + +-- 交换age，phone的先后位置 出现Using filesort +mysql> explain select id, age, phone from tb_user order by phone, age; ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone | 52 | NULL | 9804 | 100.00 | Using index; Using filesort | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +1 row in set, 1 warning (0.01 sec) + +-- age升序排列，phone降序排列，出现Using filesort +mysql> explain select id, age, phone from tb_user order by age asc, phone desc; ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone | 52 | NULL | 9804 | 100.00 | Using index; Using filesort | ++----+-------------+---------+------------+-------+---------------+--------------------+---------+------+------+----------+-----------------------------+ +1 row in set, 1 warning (0.00 sec) + +-- 创建age升序，phone降序的索引 +create index idx_user_age_phone_ad on tb_user(age asc, phone desc); + +-- 查看索引 （Collation： A=asc, D=desc） +mysql> show index from tb_user; ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| Table | Non_unique | Key_name | Seq_in_index | Column_name | Collation | Cardinality | Sub_part | Packed | Null | Index_type | Comment | Index_comment | Visible | Expression | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +| tb_user | 0 | PRIMARY | 1 | id | A | 9804 | NULL | NULL | | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 1 | profession | A | 948 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 2 | age | A | 6232 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_profession_age_status | 3 | status | A | 7596 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_email_5 | 1 | email | A | 3955 | 5 | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_age_phone | 1 | age | A | 11 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_age_phone | 2 | phone | A | 9804 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_age_phone_ad | 1 | age | A | 11 | NULL | NULL | YES | BTREE | | | YES | NULL | +| tb_user | 1 | idx_user_age_phone_ad | 2 | phone | D | 9804 | NULL | NULL | YES | BTREE | | | YES | NULL | ++---------+------------+--------------------------------+--------------+-------------+-----------+-------------+----------+--------+------+------------+---------+---------------+---------+------------+ +9 rows in set (0.02 sec) + +-- 创建索引后，age升序排列，phone降序排列 +mysql> explain select id, age, phone from tb_user order by age asc, phone desc; ++----+-------------+---------+------------+-------+---------------+-----------------------+---------+------+------+----------+-------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+-------+---------------+-----------------------+---------+------+------+----------+-------------+ +| 1 | SIMPLE | tb_user | NULL | index | NULL | idx_user_age_phone_ad | 52 | NULL | 9804 | 100.00 | Using index | ++----+-------------+---------+------------+-------+---------------+-----------------------+---------+------+------+----------+-------------+ +1 row in set, 1 warning (0.00 sec) + + +-- 如果没有用到覆盖索引，会出现：Using filesort +mysql> explain select * from tb_user order by age asc, phone desc; ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +| id | select_type | table | partitions | type | possible_keys | key | key_len | ref | rows | filtered | Extra | ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +| 1 | SIMPLE | tb_user | NULL | ALL | NULL | NULL | NULL | NULL | 9804 | 100.00 | Using filesort | ++----+-------------+---------+------------+------+---------------+------+---------+------+------+----------+----------------+ +1 row in set, 1 warning (0.00 sec) +``` + +### 3.1、优化原则 + +1. 根据排序字段建立合适的索引，多字段排序时，也遵循最左前缀法则 + +2. 尽量使用覆盖索引 + +3. 多字段排序，一个升序一个降序，此时需要注意联合索引在创建时的规则（asc/desc） + +4. 如果不可避免的出现filesort, 大数据量排序时，可以适当增大排序缓冲区的大小 sort_buffer_size(默认值256k) + +```sql +mysql> show variables like 'sort_buffer_size'; ++------------------+--------+ +| Variable_name | Value | ++------------------+--------+ +| sort_buffer_size | 262144 | ++------------------+--------+ +1 row in set (0.03 sec) +```