Version information

Items	Description
Release date	2022-7-18
Release number	V3.1.4
Commit number	b4bfa011

About this version

To provide better services to community users, OceanBase Community Edition V3.1.4 optimizes the database to make it more user-friendly and stable based on the feedback on applications of the community project.

• Database improvements: OceanBase Community Edition V3.1.4 supports dynamic allocation of data files as needed. It also supports statistics collection and monitoring in the form of histograms.
• Stability improvements: Multimodel OBKV supports cleanup of expired historical data, which increases the capacity of processing large transactions in a single table.
• Improved ease of use: OceanBase Community Edition V3.1.4 supports the deletion of a single cache plan and optimizes the display of logs and error codes. ob-operator supports the deployment of OBProxy. The latest testing framework is open sourced.

Updated features

Dynamic allocation of data files as needed

OceanBase Database is a distributed disk-resident database. It organizes and manages data in a logical architecture of macroblocks and microblocks. In terms of physical storage, OceanBase Database uses a ssblock file to store data on each OBServer. At present, the disk space is fully pre-allocated and occupied by the ssblock file on each OBServer. This allocation mode ensures the availability of disk resources during the operation of OceanBase Database. It avoids data persistence failures due to insufficient disk space, which reduces the system runtime risk. This new version of OceanBase Database supports dynamic allocation of data files as needed. You need to configure only the datafile_next parameter that specifies the step size of automatic disk usage increase and the datafile_maxsize parameter that specifies the maximum disk space that can be automatically allocated. OceanBase Database regulates disk space by using scheduled tasks based on the following rules:

• OceanBase Database calculates the disk usage by using the background statistics of scheduled tasks, and automatically increases the disk space when the disk usage reaches 95% of the initial value of the ssblock disk space. The percentage threshold can be modified.
• When the allocated disk space is insufficient, the system automatically increases the disk space based on the configuration.
• You cannot reclaim the disc space occupied by the data files on OBServer nodes. To reduce and release the currently occupied disk space, you can scale in the cluster and remove servers from the cluster.

Collection and monitoring of statistics in the form of histograms

OceanBase Community Edition V3.1.4 supports the collection and monitoring of statistics in the form of histograms. You can use OceanBase Database in combination with the histogram monitoring metrics of Prometheus. Specifically, you can define and generate a new statistics table that records statistical data of the specified percentiles, such as the 95th and 99th percentiles, in OceanBase Database. Then, the open source component OBAgent exports the statistics to and displays the statistics in Prometheus. This makes OceanBase Database easier to use. When this feature is enabled, the write performance can be somewhat affected. Histogram statistics of OceanBase Database supports the following features:

• The QUERY_RESPONSE_TIME view, which is defined in INFORMATION_SCHEMA.
• The query_response_time_stats variable, which is the system variable that enables or disables the statistics feature.
• The query_response_time_range_base variable, which is the system variable that specifies the statistics collection interval.
• The query_response_time_flush variable, which is the system variable that forces the refresh and re-retrieval of statistics.
• The open source component OBAgent, which reads the QUERY_RESPONSE_TIME view for Prometheus to generate histograms.

Cleanup of expired historical data

OBKV is supported to clean up the expired historical data. OceanBase Database maintains expired data on OBServers and supports the recycle of the expired data based on the timestamp and version respectively specified by the TIMESTAMP and MAXVERSION parameters. You can specify the expiration time and the maximum number of versions of table data in the comment for the column family table upon table creation. OceanBase Database runs periodic background tasks to launch partition-level transaction scanning and clean up the expired data at the Partition Service layer. Key updates:

• The tenant-level system tables __all_kv_ttl_task** and ** __all_kv_ttl_task_history are provided to record the status and the historical operation records of time-to-live (TTL) tasks. Garbage collection (GC) is performed on the __all_kv_ttl_task_history table to maintain the number of records in the table, which is similar to the GC strategy for other history tables of historical records.
• The DBA_OB_KV_TTL_TASKS and DBA_OB_KV_TTL_TASK_HISTORY views are provided for you to view the current and historical TTL tasks in the tenant.
• The virtual __all_virtual_kv_ttl_task** and **__all_virtual_kv_ttl_task_history tables and the corresponding CDB_OB_KV_TTL_TASKS and CDB_OB_KV_TTL_TASK_HISTORY views are provided for you to view the current and historical TTL tasks of all tenants in the SYS tenant.
• The kv_ttl_duty_duration parameter is provided to specify the background execution period for periodic TTL tasks. If a TTL task is unfinished when the execution period expires, this task and tasks following it are suspended and then resumed in the same execution period on the next day. You can set the execution period for background TTL tasks based on the characteristics of your application system.
• The enable_kv_ttl parameter is provided to specify whether to start the background TTL tasks. The kv_ttl_history_recycle_interval parameter specifies the period of time to retain the history of TTL tasks.
• Memory usage and throttling is supported. The progress of background TTL tasks can be dynamically adjusted based on the memory usage of MemStore.
• Transaction usage and throttling is supported. The number of table records that can be queried and deleted in each background TTL task is limited. This way, transactions can be committed in batches to avoid the generation of large transactions. In addition, if lock contention occurs to a transaction, the transaction can be rolled back.

Invalidate the execution plan of a single SQL statement

When an irrational SQL execution plan is found, the DBA can invalidate the plan based on the SQL ID. This avoids the invalidation of the entire plan cache, which can affect the system stability. Usage notes:

• The SYS tenant can invalidate a specific or all SQL execution plans of a user tenant by executing the following statement: ALTER SYSTEM FLUSH PLAN CACHE [ [SQL_identifier] [db_list] tenant_list ][global];.
• A MySQL tenant can invalidate a specific or all SQL execution plans of the tenant by executing the following statement: ALTER SYSTEM FLUSH PLAN CACHE [ [ SQL_identifier ] [db_list] ] [global];.

Improved ease of use

• ob-operator supports the deployment of OBProxy, which makes it easier to operate on multiple nodes in the cluster and improves the routing accuracy.
• To be compatible with MySQL error codes, OceanBase Database of earlier versions converted internal error codes to external error codes. As a result, the error codes displayed on the client are inconsistent with those recorded in the log, making it difficult for users to troubleshoot the errors. To address this issue, in OceanBase Community Edition V3.1.4, internal and external error codes are both recorded in the logs to help you locate the errors.
• If the execution of the update [table_name] set [column_name] statement fails due to the Data too long for column error, the error message specifies the column that causes the error.
• The latest OBClient and the testing framework are open sourced, making it easier for community developers to participate in code development and testing.

Other changes

• The lower_case_table_names variable cannot be modified. The lower_case_table_names variable is a global read-only variable. You cannot modify it by using the set global lower_case_table_names; statement or the alter tenant [tenant_name] set variables lower_case_table_names; statement.
• When automatic scaling is enabled, the total_size field of the __all_virtual_disk_stat table indicates the maximum disk space that is available for data storage.

Fixed issues

• The content of the datadir variable is not displayed because it is a read-only system variable. In OceanBase Community Edition V3.1.4, the content of the cluster variable data_dir is displayed.
• The internal limits on large transactions and on the size of the log_id of transaction groups are removed. The _max_trx_size parameter is provided to prevent the generation of large transactions, which can cause adverse impact on the database.
• The restriction that table names cannot exceed 64 characters in length is removed.
• The execution of a query with the /*+ LEADING */ comment will fail with the Invalid argument error.
• The 4016 error is returned if you use the create table t select ~1; syntax to create a table.
• In earlier versions, you can only create an SQL comment on a single line in your SQL statement. In OceanBase Community Edition V3.1.4, your SQL comment can span multiple lines.
• The virtual table definitions corresponding to the show full columns command are modified to improve the compatibility with MySQL databases.

版本信息

信息	描述
发布时间	2022-7-18
版本号	V3.1.4
提交号	b4bfa011

概要说明

在 V3.1.4 版本，OceanBase数据库针对社区项目应用反馈，优化提升内核易用性与稳定性，服务广大社区用户。

• 内核能力提升：支持数据文件动态按需分配，支持直方图类型统计信息采集与监控。
• 稳定性提升：扩展单表大事务能力到2TB，多模OBKV支持历史数据过期清理。
• 易用性提升：支持删除单条缓存计划，优化日志与错误码展示等，ob-operator支持部署OBProxy，开源最新测试套框架。

特性更新

支持数据文件动态按需分配

OceanBase数据库是一款分布式磁盘数据库，在系统逻辑架构上使用宏块（Macro Block）与微块（Micro Block）对数据进行组织管理，在物理存储上则使用ssblock文件进行数据存储。当前OceanBase的ssblock文件采用预先全额分配并占用的磁盘空间管理方式，该分配方式保障了OceanBase数据库在使用过程中的磁盘资源可获取性，可避免因磁盘空间不足导致数据持久化失败，减少系统运行时的风险。同时本次版本迭代OceanBase数据库新增支持数据文件动态按需分配， 用户只需要配置其关心的一次磁盘自动分配大小datafile_next和磁盘空间自动分配上限值datafile_maxsize即可。 OceanBase数据库主要采用定时任务的方式进行磁盘空间调整，核心能力如下：

• 依据ssblock磁盘空间的初始值设定，使用定时任务后台统计与计算，当磁盘使用率达到95%（可配置）时，自动触发磁盘空间扩展。
• 在用户分配的磁盘报空间不足时，系统根据用户配置自动进行磁盘空间扩展。
• 不支持单OBServer节点数据文件空间回收，若用户希望减少并释放当前磁盘空间使用量，可通过对集群加减机器的方式（集群扩缩容）进行。

支持直方图类型统计信息采集与监控。

新增支持直方图类型统计信息采集与监控。结合Prometheus的直方图类型(Histogram)监控指标，在OceanBase数据库内部定义和生成新的统计信息表，记录99、95等分位的统计数据，通过obagent开源组件输出到Prometheus进行展示，提升产品易用性。但需要注意的是，该功能启用时会带来少量写性能损耗。 OceanBase数据库的直方图类型统计信息提供如下能力：

• 支持视图QUERY_RESPONSE_TIME，定义在INFORMATION_SCHEMA中。
• 支持系统变量query_response_time_stats， 设置统计开发启动。
• 支持系统变量query_response_time_range_base ，设置统计间隔时间。
• 支持系统变量query_response_time_flush，设置刷新统计数据且重新读取数据。
• 支持obagent组件读取该视图，在Prometheus中生成直方图。

多模数据库（OBKV）支持历史数据过期清理。

新增支持多模数据库（OBKV）支持历史数据过期清理。OceanBase数据库采用服务端维护过期数据的方式，支持基于TIMESTAMP的时间过期回收和MAXVERSION版本过期的回收，用户可在 column family 表的 comment 中设置表数据的过期时间和多版本数量，OceanBase内部采用周期性后台任务， 在Partition Service层基于分区粒度开启事务扫描并清理过期数据。核心能力如下：

• 新增租户级系统表__all_kv_ttl_task 和 __all_kv_ttl_task_history，用以记录内部周期TTL任务状态和TTL任务的历史操作记录。该历史记录表需要进行垃圾回收，以维护该表记录数的稳定，类似于其他History表的GC策略。
• 新增视图 DBA_OB_KV_TTL_TASKS 和 DBA_OB_KV_TTL_TASK_HISTORY，便于用户查看本租户下的TTL当前 TTL 任务和历史 TTL 任务。
• 新增虚拟表 __all_virtual_kv_ttl_task 和 __all_virtual_kv_ttl_task_history，以及对应的两个视图 CDB_OB_KV_TTL_TASKS 和 CDB_OB_KV_TTL_TASK_HISTORY，便于用户在系统租户下查看所有租户当前正在执行的 TTL 任务和历史 TTL 任务。
• TTL是后台运行的周期性任务，该任务会对设置TTL属性的表进行周期性的巡检和删除操作，会额外占用小部分系统的内存、CPU等资源。增加配置项kv_ttl_duty_duration用以限制TTL周期任务的后台运行时间，如果没有执行完，则内部把没有执行完的任务挂起来，后面再次进入到任务运行时间段的时候，继续上次挂起来的任务。用户可以根据业务运行特征，设置TTL后台任务的执行时间。
• 新增配置项enable_kv_ttl用于表示是否启动TTL后台任务，kv_ttl_history_recycle_interval用以描述TTL任务历史记录保留的时间段。
• 内存使用与限流，后台TTL任务根据memstore的内存使用量动态调整任务进度。
• 事务使用与限流，后台TTL任务限制每次查询和删除的表记录条数，分批次提交事务，控制大事务的产生，同时支持锁冲突回滚当前事务。

支持失效单条SQL缓存计划。

支持失效单条 SQL 的缓存计划，当 DBA 诊断出某条SQL 的执行计划不合理时，可以根据 SQL ID 做细粒度计划缓存的淘汰，避免失效整个Plan Cache影响系统性能稳定。使用方法如下：

• 系统租户通过 ALTER SYSTEM FLUSH PLAN CACHE [ [SQL_identifier] [db_list] tenant_list ][global] 语句指定失效某租户下特定或全部 SQL 的计划。
• MySQL 租户通过增加命令 ALTER SYSTEM FLUSH PLAN CACHE [ [ SQL_identifier ] [db_list] ] [global]; 失效特定或全部 SQL 的计划。

产品易用性增强

• ob-operator支持部署OBProxy，便于用户使用多节点的集群，同时提升集群路由准确度。
• OceanBase数据库因需要MySQL错误码兼容，在生产日志阶段做了内外部错误码转换，用户在客户端看到的错误码和日志中记录的错误码不一致，无法通过错误码在日志中直接检错到相关信息，给排查问题带来不少困惑。为此，OceanBase数据库在日志生成阶段，同时记录输出用户客户端看到的错误码（user_error_code）和内部错误码（internal_error_code），方便根据错误码查找定位问题。
• 针对update [table_name] set [column_name]语句，优化执行失败信息返回，在报错Data too long for column时明确对应出错的列信息。
• 开源最新版本的obclient客户端以及测试驱动框架，帮助社区开发者快速参与到代码开发和测试工作，降低入门门槛。

行为变更

• 禁止修改 lower_case_table_names变量

lower_case_table_names变量是全局只读变量，因此我们禁止通过set global lower_case_table_names和alter tenant [tenant_name] set variables lower_case_table_names语法对该变量进行修改。

• 在自动扩容开启的情况下，__all_virtual_disk_stat的total_size字段统计值是可存储磁盘空间的最大值。

问题修复

• 修复系统变量datadir的内容显示，采用集群配置项data_dir的内容，该系统变量为只读变量不支持修改。
• 修复去除大事务内部限制，取消事务组log_id生成大小的限制，同时增加配置项_max_trx_size避免生成事务过大给数据库带来负面影响。
• 修复表名称无法存储超过64个字符的问题。
• 修复查询带/*+ LEADING */ 功能时，执行是报错Invalid argument的问题。
• 修复创建表使用create table t select ~1语法报错4016的问题。
• 修复sql多行注释的问题
• 修改show full columns命令对应的虚拟表定义，完善mysql兼容性

OCP 3.3.0-CE

• Release Note
• 支持标准部署的社区版 OBProxy 接管
• 兼容支持社区版 OBProxy 部署
• 新增备份恢复功能，支持 NFS 介质的备份与恢复。
• 容量管理, 增加以副本的⻆度查看资源的使用情况。
• 增加日志查询功能，可以直接在 OCP 上查询 OceanBase， OBProxy 和主机的日志，方便快速 定位和分析问题

OMS 3.3.0-CE

• Release Note
• 新增支持 OceanBase-CE 至 OceanBase-CE 逻辑升级。
• 新增支持 OB_MySQL_CE -> Kafka 结构同步，全量同步，增量同步。 ⚫ 新增支持 OB_MySQL_CE ->RocetMQ 增量同步。
• 支持非唯一索引后置创建功能，全面提升全量迁移效能。
• 支持 JSON 数据类型。
• 支持 MySQL 8.0 版本。
• 去除 OceanBase-CE 数据源关联 OCP 硬限制，支持更多业务场景。

OBDeploy

• Release Note
• 新增obd test tpcc命令
• 新增/usr/obd/example目录，内容与代码库example目录一致 (https://github.com/oceanbase/obdeploy/issues/123)
• mysqltest命令执行record选项，开启后将只生成结果文件，不进行对比