提交 c434b086 编写于 作者: T tristaZero

orchestration

上级 ed31ef4c
+++
pre = "<b>3.3. </b>"
title = "数据治理"
title = "编排治理"
weight = 3
chapter = true
+++
......
......@@ -6,7 +6,7 @@ weight = 5
+++
## 背景
安全控制一直是数据治理的重要环节,数据脱敏属于安全控制的范畴。对互联网公司、传统行业来说,数据安全一直是极为重视和敏感的话题。数据脱敏是指对某些敏感信息通过脱敏规则进行数据的变形,实现敏感隐私数据的可靠保护。涉及客户安全数据或者一些商业性敏感数据,如身份证号、手机号、卡号、客户号等个人信息按照相关部门规定,都需要进行数据脱敏。
安全控制一直是治理的重要环节,数据脱敏属于安全控制的范畴。对互联网公司、传统行业来说,数据安全一直是极为重视和敏感的话题。数据脱敏是指对某些敏感信息通过脱敏规则进行数据的变形,实现敏感隐私数据的可靠保护。涉及客户安全数据或者一些商业性敏感数据,如身份证号、手机号、卡号、客户号等个人信息按照相关部门规定,都需要进行数据脱敏。
ShardingSphere为此提供了数据脱敏的功能,它将用户的敏感信息经过加密存储到数据库中。当用户查询敏感信息的时候,再对敏感信息进行自动解密,为用户返回原始数据。
......
......@@ -7,7 +7,7 @@ weight = 5
## Background
Security control has always been a crucial link of data orchestration; data masking falls into this category. For both Internet enterprises and traditional sectors, data security has always been a highly focused and sensitive topic. Data masking refers to transforming some sensitive information through masking rules to safely protect the private data. Data that involves client security or business sensibility, such as ID number, phone number, card number, client number and other personal information, is required of data masking according to relevant regulations.
Security control has always been a crucial link of orchestration; data masking falls into this category. For both Internet enterprises and traditional sectors, data security has always been a highly focused and sensitive topic. Data masking refers to transforming some sensitive information through masking rules to safely protect the private data. Data that involves client security or business sensibility, such as ID number, phone number, card number, client number and other personal information, is required of data masking according to relevant regulations.
Because of that, ShardingSphere has provided the function of data masking, which stores users' sensitive information in the database after encryption. When users search for them, they will be decrypted and returned to users as the original data. It has the encryption and decryption processes totally transparent to users, who can store desensitized data and acquire original data without any awareness. In addition, ShardingSphere has provided internal masking algorithm, which can directly used by users. In the same time, we have also provided masking algorithm related interfaces, which can be implemented by users themselves. Then, after simple configurations, ShardingSphere can use algorithms provided by users to perform encryption, decryption and masking operations.
......
......@@ -7,7 +7,7 @@ weight = 2
## 实现动机
通过注册中心,提供熔断数据库访问程序对数据库的访问和禁用从库的访问的能力。数据治理仍然有大量未完成的功能。
通过注册中心,提供熔断数据库访问程序对数据库的访问和禁用从库的访问的能力。治理仍然有大量未完成的功能。
## 注册中心数据结构
......
......@@ -8,7 +8,7 @@ weight = 2
## Motivation
Registry center provides the ability to disable the access of application to database and the access to slave database.
Data orchestration still has many functions to be developed.
Orchestration still has many functions to be developed.
## Data Structure in Registry Center
......
......@@ -170,7 +170,7 @@ weight = 1
}
```
### 数据治理
### 治理
```java
DataSource getDataSource() throws SQLException {
......@@ -333,37 +333,37 @@ ShardingStrategyConfiguration的实现类,用于配置不分片的策略。
| check.table.metadata.enabled (?) | boolean | 是否在启动时检查分表元数据一致性,默认值: false |
### 数据治理
### 治理
#### OrchestrationShardingDataSourceFactory
数据分片 + 数据治理的数据源工厂。
数据分片 + 治理的数据源工厂。
| *名称* | *数据类型* | *说明* |
| ------------------- | ------------------------- | --------------------------- |
| dataSourceMap | Map\<String, DataSource\> | 同ShardingDataSourceFactory |
| shardingRuleConfig | ShardingRuleConfiguration | 同ShardingDataSourceFactory |
| props (?) | Properties | 同ShardingDataSourceFactory |
| orchestrationConfig | OrchestrationConfiguration | 数据治理规则配置 |
| orchestrationConfig | OrchestrationConfiguration | 治理规则配置 |
#### OrchestrationMasterSlaveDataSourceFactory
读写分离 + 数据治理的数据源工厂。
读写分离 + 治理的数据源工厂。
| *名称* | *数据类型* | *说明* |
| --------------------- | ---------------------------- | ------------------------------ |
| dataSourceMap | Map\<String, DataSource\> | 同MasterSlaveDataSourceFactory |
| masterSlaveRuleConfig | MasterSlaveRuleConfiguration | 同MasterSlaveDataSourceFactory |
| props (?) | Properties | 同ShardingDataSourceFactory |
| orchestrationConfig | OrchestrationConfiguration | 数据治理规则配置 |
| orchestrationConfig | OrchestrationConfiguration | 治理规则配置 |
#### OrchestrationConfiguration
数据治理规则配置对象。
治理规则配置对象。
| *名称* | *数据类型* | *说明* |
| --------------- | --------------------------- | ---------------------------------------------------------- |
| name | String | 数据治理实例名称 |
| name | String | 治理实例名称 |
| overwrite | boolean | 本地配置是否覆盖注册中心配置,如果可覆盖,每次启动都以本地配置为准 |
| regCenterConfig | RegistryCenterConfiguration | 注册中心配置 |
......
......@@ -169,7 +169,7 @@ weight = 1
}
```
### Data Orchestration
### Orchestration
```java
DataSource getDataSource() throws SQLException {
......@@ -323,7 +323,7 @@ Property configuration items, can be of the following properties.
| max.connections.size.per.query (?) | int | The maximum connection number allocated by each query of each physical database, default value: 1 |
| check.table.metadata.enabled (?) | boolean | Check meta-data consistency or not in initialization, default value: false |
### Data Orchestration
### Orchestration
#### OrchestrationShardingDataSourceFactory
......@@ -332,7 +332,7 @@ Property configuration items, can be of the following properties.
| dataSourceMap | Map\<String, DataSource\> | Same as `ShardingDataSourceFactory` |
| shardingRuleConfig | ShardingRuleConfiguration | Same as `ShardingDataSourceFactory` |
| props (?) | Properties | Same as `ShardingDataSourceFactory` |
| orchestrationConfig | OrchestrationConfiguration | Data orchestration rule configurations |
| orchestrationConfig | OrchestrationConfiguration | Orchestration rule configurations |
#### OrchestrationMasterSlaveDataSourceFactory
......@@ -342,13 +342,13 @@ Property configuration items, can be of the following properties.
| masterSlaveRuleConfig | MasterSlaveRuleConfiguration | Same as `MasterSlaveDataSourceFactory` |
| configMap (?) | Map<String, Object> | Same as `MasterSlaveDataSourceFactory` |
| props (?) | Properties | Same as `ShardingDataSourceFactory` |
| orchestrationConfig | OrchestrationConfiguration | Data orchestration rule configurations |
| orchestrationConfig | OrchestrationConfiguration | Orchestration rule configurations |
#### OrchestrationConfiguration
| *Name* | *Data Type* | *Explanation* |
| --------------- | --------------------------- | ------------------------------------------------------------ |
| name | String | Data orchestration example name |
| name | String | Orchestration example name |
| overwrite | boolean | Local configurations overwrite registry center configurations or not; if they overwrite, each start takes reference of local configurations |
| regCenterConfig | RegistryCenterConfiguration | Registry center configurations |
......
......@@ -167,7 +167,7 @@ sharding.jdbc.config.sharding.encryptRule.encryptors.order_encryptor.type=AES
sharding.jdbc.config.sharding.encryptRule.encryptors.order_encryptor.props.aes.key.value=123456
```
### 数据治理
### 治理
```properties
sharding.jdbc.datasource.names=ds,ds0,ds1
......@@ -304,12 +304,12 @@ sharding.jdbc.config.sharding.encryptRule.encryptors.<encryptor-name>.qualifiedC
sharding.jdbc.config.sharding.encryptRule.encryptors.<encryptor-name>.assistedQueryColumns= #辅助查询字段,针对ShardingQueryAssistedEncryptor类型的加解密器进行辅助查询
sharding.jdbc.config.sharding.encryptRule.encryptors.<encryptor-name>.props.<property-name>= #属性配置, 比如AES算法的KEY属性:aes.key.value
```
### 数据治理
### 治理
```properties
#省略数据源、数据分片和读写分离配置
sharding.jdbc.config.sharding.orchestration.name= #数据治理实例名称
sharding.jdbc.config.sharding.orchestration.name= #治理实例名称
sharding.jdbc.config.sharding.orchestration.overwrite= #本地配置是否覆盖注册中心配置。如果可覆盖,每次启动都以本地配置为准
sharding.jdbc.config.sharding.orchestration.registry.type= #配置中心类型。如:zookeeper
sharding.jdbc.config.sharding.orchestration.registry.server-lists= #连接注册中心服务器的列表。包括IP地址和端口号。多个地址用逗号分隔。如: host1:2181,host2:2181
......
......@@ -167,7 +167,7 @@ sharding.jdbc.config.sharding.encryptRule.encryptors.order_encryptor.type=AES
sharding.jdbc.config.sharding.encryptRule.encryptors.order_encryptor.props.aes.key.value=123456
```
### Data Orchestration
### Orchestration
```properties
sharding.jdbc.datasource.names=ds,ds0,ds1
......@@ -306,12 +306,12 @@ sharding.jdbc.config.sharding.encryptRule.encryptors.<encryptor-name>.assistedQu
sharding.jdbc.config.sharding.encryptRule.encryptors.<encryptor-name>.props.<property-name>= #Properties, e.g. `aes.key.value` for AES encryptor
```
### Data Orchestration
### Orchestration
```properties
#Omit data source, data sharding and read-write split configurations
sharding.jdbc.config.sharding.orchestration.name= #Data orchestration instance name
sharding.jdbc.config.sharding.orchestration.name= #Orchestration instance name
sharding.jdbc.config.sharding.orchestration.overwrite= #Whether to overwrite local configurations with registry center configurations; if it can, each initialization should refer to local configurations
sharding.jdbc.config.sharding.orchestration.registry.type= #Registry center type. Example:zookeeper
sharding.jdbc.config.sharding.orchestration.registry.server-lists= #The list of servers that connect to registry center, including IP and port number; use commas to separate
......
......@@ -351,7 +351,7 @@ weight = 4
</beans>
```
### 数据治理
### 治理
```xml
<?xml version="1.0" encoding="UTF-8"?>
......@@ -523,7 +523,7 @@ weight = 4
| max.connections.size.per.query (?) | 属性 | 每个物理数据库为每次查询分配的最大连接数量。默认值: 1 |
| check.table.metadata.enabled (?) | 属性 | 是否在启动时检查分表元数据一致性,默认值: false |
### 数据分片 + 数据治理
### 数据分片 + 治理
命名空间:http://shardingsphere.apache.org/schema/shardingsphere/orchestration/orchestration.xsd
......@@ -536,7 +536,7 @@ weight = 4
| registry-center-ref | 属性 | 注册中心id |
| overwrite | 属性 | 本地配置是否覆盖注册中心配置。如果可覆盖,每次启动都以本地配置为准。 缺省为不覆盖 |
### 读写分离 + 数据治理
### 读写分离 + 治理
命名空间:http://shardingsphere.apache.org/schema/shardingsphere/orchestration/orchestration.xsd
......@@ -549,7 +549,7 @@ weight = 4
| registry-center-ref | 属性 | 注册中心id |
| overwrite | 属性 | 本地配置是否覆盖注册中心配置。如果可覆盖,每次启动都以本地配置为准。缺省为不覆盖 |
### 数据治理注册中心
### 治理注册中心
命名空间:http://shardingsphere.apache.org/schema/shardingsphere/orchestration/orchestration.xsd
......
......@@ -225,7 +225,7 @@ props:
sql.show: true
```
### 数据治理
### 治理
```yaml
#省略数据分片和读写分离配置
......@@ -384,7 +384,7 @@ shardingRule: #省略分片规则配
aes.key.value:
```
### 数据治理
### 治理
```yaml
dataSources: #省略数据源配置
......@@ -392,7 +392,7 @@ shardingRule: #省略分片规则配置
masterSlaveRule: #省略读写分离规则配置
orchestration:
name: #数据治理实例名称
name: #治理实例名称
overwrite: #本地配置是否覆盖注册中心配置。如果可覆盖,每次启动都以本地配置为准
registry: #注册中心配置
serverLists: #连接注册中心服务器的列表。包括IP地址和端口号。多个地址用逗号分隔。如: host1:2181,host2:2181
......
......@@ -349,7 +349,7 @@ shardingRule: #Omit sharding rule configurations
masterSlaveRule: #Omit read-write split rule configurations
orchestration:
name: #Data orchestration instance name
name: #Orchestration instance name
overwrite: #Whether to overwrite local configurations with registry center configurations; if it can, each initialization should refer to local configurations
registry: #Registry center configuration
serverLists: #The list of servers that connect to registry center, including IP and port number; use commas to seperate addresses, such as: host1:2181,host2:2181
......
+++
toc = true
title = "数据治理"
title = "编排治理"
weight = 4
+++
使用数据治理功能需要指定一个注册中心。配置将全部存入注册中心,可以在每次启动时使用本地配置覆盖注册中心配置,也可以只通过注册中心读取配置。
使用治理功能需要指定一个注册中心。配置将全部存入注册中心,可以在每次启动时使用本地配置覆盖注册中心配置,也可以只通过注册中心读取配置。
## 不使用Spring
......@@ -41,7 +41,7 @@ weight = 4
regConfig.setServerLists("localhost:2181");
regConfig.setNamespace("sharding-sphere-orchestration");
// 配置数据治理
// 配置治理
OrchestrationConfiguration orchConfig = new OrchestrationConfiguration("orchestration-sharding-data-source", regConfig, false);
// 获取数据源对象
......
......@@ -4,7 +4,7 @@ title = "Orchestration"
weight = 4
+++
The use of data orchestration function requires to designate a registry center, in which all the configurations are saved.
The use of orchestration function requires to designate a registry center, in which all the configurations are saved.
Users can either use local configurations to cover registry center configurations or read configurations from registry center only.
## Not Use Spring
......
......@@ -228,7 +228,7 @@ shardingRule:
Sharding-Proxy使用conf/server.yaml配置注册中心、认证信息以及公用属性。
### 数据治理
### 治理
```yaml
#省略数据分片和读写分离配置
......@@ -291,7 +291,7 @@ masterSlaveRule: #省略读写分离配置,与Sharding-JDBC配置一致
## 全局配置项说明
### 数据治理
### 治理
与Sharding-JDBC配置一致。
......
......@@ -227,7 +227,7 @@ shardingRule:
Sharding-Proxy uses `conf/server.yaml` to configure the registry center, authentication information and common properties.
### Data Orchestration
### Orchestration
```yaml
#Omit data sharding and read-write split configurations
......@@ -290,7 +290,7 @@ masterSlaveRule: #Omit data source configurations; keep it consistent with Shard
## Overall Configuration Explanation
### Data Orchestration
### Orchestration
It is the same with Sharding-JDBC configuration.
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册