未验证 提交 94423c84 编写于 作者: L Liang Zhang 提交者: GitHub

Change properties mod sharding algorithm config from partition to sharding (#6054)

* partition.volume --> sharding.volume

* partition.ranges --> sharding.ranges

* partition.seconds --> sharding.seconds
上级 16a2e81f
......@@ -100,11 +100,11 @@ Apache ShardingSphere 内置的标准分片算法实现类包括:
可配置属性:
| *属性名称* | *数据类型* | *说明* |
| ---------------- | --------- | -------------------------- |
| range.lower | long | 范围下界,超过边界的数据会报错 |
| range.upper | long | 范围上界,超过边界的数据会报错 |
| partition.volume | long | 分片容量 |
| *属性名称* | *数据类型* | *说明* |
| --------------- | --------- | -------------------------- |
| range.lower | long | 范围下界,超过边界的数据会报错 |
| range.upper | long | 范围上界,超过边界的数据会报错 |
| sharding.volume | long | 分片容量 |
#### 自定义边界范围分片算法
......@@ -112,9 +112,9 @@ Apache ShardingSphere 内置的标准分片算法实现类包括:
可配置属性:
| *属性名称* | *数据类型* | *说明* |
| ---------------- | --------- | --------------------------------- |
| partition.ranges | String | 分片的范围边界,多个范围边界以逗号分隔 |
| *属性名称* | *数据类型* | *说明* |
| --------------- | --------- | --------------------------------- |
| sharding.ranges | String | 分片的范围边界,多个范围边界以逗号分隔 |
#### 定长时间段分片算法
......@@ -122,11 +122,11 @@ Apache ShardingSphere 内置的标准分片算法实现类包括:
可配置属性:
| *属性名称* | *数据类型* | *说明* |
| ----------------- | --------- | ----------------------------------------------- |
| datetime.lower | String | 分片的起始时间范围,时间戳格式:yyyy-MM-dd HH:mm:ss |
| datetime.upper | String | 分片的结束时间范围,时间戳格式:yyyy-MM-dd HH:mm:ss |
| partition.seconds | long | 单一分片所能承载的最大时间,单位:秒 |
| *属性名称* | *数据类型* | *说明* |
| ---------------- | --------- | ----------------------------------------------- |
| datetime.lower | String | 分片的起始时间范围,时间戳格式:yyyy-MM-dd HH:mm:ss |
| datetime.upper | String | 分片的结束时间范围,时间戳格式:yyyy-MM-dd HH:mm:ss |
| sharding.seconds | long | 单一分片所能承载的最大时间,单位:秒 |
#### 自定义时间边界分片算法
......
......@@ -100,11 +100,11 @@ Class name: org.apache.shardingsphere.sharding.strategy.algorithm.sharding.range
Attributes:
| *Name* | *DataType* | *Description* |
| ---------------- | ---------- | -------------------------------------------------------- |
| range.lower | long | Range lower bound, throw exception if lower than bound |
| range.upper | long | Range upper bound, throw exception if upper than bound |
| partition.volume | long | Sharding volume |
| *Name* | *DataType* | *Description* |
| --------------- | ---------- | ------------------------------------------------------ |
| range.lower | long | Range lower bound, throw exception if lower than bound |
| range.upper | long | Range upper bound, throw exception if upper than bound |
| sharding.volume | long | Sharding volume |
#### Customized Range Sharding Algorithm
......@@ -112,19 +112,9 @@ Class name: org.apache.shardingsphere.sharding.strategy.algorithm.sharding.range
Attributes:
| *Name* | *DataType* | *Description* |
| ---------------- | ---------- | ----------------------------------------------------------------- |
| partition.ranges | String | Range of sharding border, multiple boundaries separated by commas |
#### Custom Range Bound Sharding Algorithm
Class name: org.apache.shardingsphere.sharding.strategy.algorithm.sharding.range.CustomRangeShardingAlgorithm
Attributes:
| *Name* | *DataType* | *Description* |
| ---------------- | ---------- | ------------------------------------------------------------------ |
| partition.ranges | String | Sharding range boundaries, multiple boundaries separated by commas |
| *Name* | *DataType* | *Description* |
| --------------- | ---------- | ----------------------------------------------------------------- |
| sharding.ranges | String | Range of sharding border, multiple boundaries separated by commas |
#### Fixed Time Range Sharding Algorithm
......@@ -132,11 +122,11 @@ Class name: org.apache.shardingsphere.sharding.strategy.algorithm.sharding.Datet
Attributes:
| *Name* | *DataType* | *Description* |
| ----------------- | ---------- | ----------------------------------------------------------- |
| datetime.lower | String | Shard datetime begin boundary, pattern: yyyy-MM-dd HH:mm:ss |
| datetime.upper | String | Shard datetime end boundary, pattern: yyyy-MM-dd HH:mm:ss |
| partition.seconds | long | Max seconds for the data in one shard |
| *Name* | *DataType* | *Description* |
| ---------------- | ---------- | ----------------------------------------------------------- |
| datetime.lower | String | Shard datetime begin boundary, pattern: yyyy-MM-dd HH:mm:ss |
| datetime.upper | String | Shard datetime end boundary, pattern: yyyy-MM-dd HH:mm:ss |
| sharding.seconds | long | Max seconds for the data in one shard |
#### Custom Datetime Bound Sharding Algorithm
......
......@@ -39,18 +39,18 @@ import java.util.Properties;
* Datetime sharding algorithm.
*
* <p>Shard by `y = floor(x/v)` algorithm, which means y begins from 0.
* v is `PARTITION_SECONDS`, and the minimum time unit is 1 sec.
* `DATETIME_LOWER` decides the beginning datetime to shard. On the other hand, `DATETIME_UPPER` decides the end datetime to shard.</p>
* <p>Notice: Anytime less then `DATETIME_LOWER` will route to the first partition, and anytime great than `DATETIME_UPPER` will route to the last partition.</p>
* v is `sharding.seconds`, and the minimum time unit is 1 sec.
* `datetime.lower` decides the beginning datetime to shard. On the other hand, `datetime.upper` decides the end datetime to shard.</p>
* <p>Notice: Anytime less then `datetime.lower` will route to the first partition, and anytime great than `datetime.upper` will route to the last sharding.</p>
*/
@Getter
public final class DatetimeShardingAlgorithm implements StandardShardingAlgorithm<Comparable<?>>, ShardingAutoTableAlgorithm {
private static final String DATETIME_LOWER = "datetime.lower";
public static final String DATETIME_LOWER = "datetime.lower";
private static final String DATETIME_UPPER = "datetime.upper";
public static final String DATETIME_UPPER = "datetime.upper";
private static final String PARTITION_SECONDS = "partition.seconds";
public static final String SHARDING_SECONDS_KEY = "sharding.seconds";
private static final String DATETIME_PATTERN = "yyyy-MM-dd HH:mm:ss";
......@@ -63,7 +63,7 @@ public final class DatetimeShardingAlgorithm implements StandardShardingAlgorith
@Override
public void init() {
Preconditions.checkNotNull(props.get(PARTITION_SECONDS), "Sharding partition volume cannot be null.");
Preconditions.checkNotNull(props.get(SHARDING_SECONDS_KEY), "Sharding partition volume cannot be null.");
Preconditions.checkState(null != props.get(DATETIME_LOWER) && checkDatetimePattern(props.get(DATETIME_LOWER).toString()), "%s pattern is required.", DATETIME_PATTERN);
Preconditions.checkState(null != props.get(DATETIME_UPPER) && checkDatetimePattern(props.get(DATETIME_UPPER).toString()),
"%s pattern is required.", DATETIME_PATTERN);
......@@ -128,7 +128,7 @@ public final class DatetimeShardingAlgorithm implements StandardShardingAlgorith
}
private long getPartitionValue() {
return Long.parseLong(props.get(PARTITION_SECONDS).toString());
return Long.parseLong(props.get(SHARDING_SECONDS_KEY).toString());
}
@Override
......
......@@ -33,11 +33,11 @@ import java.util.stream.Collectors;
* Custom range sharding algorithm.
* <p>
* Custom range sharding algorithm is similar to the rule of partition table.
* User can specify the range by setting the `partition.ranges` parameter.
* The `partition.ranges` parameter is an ordered list of numbers, separated by commas.
* User can specify the range by setting the `sharding.ranges` parameter.
* The `sharding.ranges` parameter is an ordered list of numbers, separated by commas.
* </p>
* <p>
* For example: If the `partition.ranges` parameter is set to `1,5,10`,
* For example: If the `sharding.ranges` parameter is set to `1,5,10`,
* the parameter will split all values into four intervals——(-∞, 1), [1,5), [5,10), [10, +∞),
* which corresponding to partition_0, partition_1, partition_2, partition_3.
* The sharding values will be divided into different partition by its value.
......@@ -45,12 +45,12 @@ import java.util.stream.Collectors;
*/
public final class CustomRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
private static final String PARTITION_RANGES = "partition.ranges";
public static final String SHARDING_RANGES_KEY = "sharding.ranges";
@Override
public Map<Integer, Range<Long>> createPartitionRangeMap(final Properties props) {
Preconditions.checkNotNull(props.get(PARTITION_RANGES), "Custom range sharding algorithm partition ranges cannot be null.");
List<Long> partitionRanges = Splitter.on(",").trimResults().splitToList(props.get(PARTITION_RANGES).toString())
Preconditions.checkNotNull(props.get(SHARDING_RANGES_KEY), "Custom range sharding algorithm partition ranges cannot be null.");
List<Long> partitionRanges = Splitter.on(",").trimResults().splitToList(props.get(SHARDING_RANGES_KEY).toString())
.stream().map(Longs::tryParse).filter(Objects::nonNull).sorted().collect(Collectors.toList());
Preconditions.checkArgument(CollectionUtils.isNotEmpty(partitionRanges), "Custom range sharding algorithm partition ranges is not valid.");
Map<Integer, Range<Long>> result = Maps.newHashMapWithExpectedSize(partitionRanges.size() + 1);
......
......@@ -31,31 +31,31 @@ import java.util.Properties;
*
* <p>
* Standard range sharding algorithm is similar to the rule of partition table, but it can only be split by the same size.
* User can specify the range by setting `range.lower`, `range.upper` and `partition.volume` parameters.
* The `partition.volume` parameter determines the size of each partition.
* User can specify the range by setting `range.lower`, `range.upper` and `sharding.volume` parameters.
* The `sharding.volume` parameter determines the size of each sharding.
* </p>
* <p>
* For example: If the `range.lower` parameter is set to `10`, the `range.upper` parameter is set to `45`,
* and the `partition.volume` parameter is set to `10`. The values in range [10,45] will be split to different partitions
* and the `sharding.volume` parameter is set to `10`. The values in range [10,45] will be split to different partitions
* ——[10,20), [20, 30), [30, 40), [40, 45), and other values will be split to (-∞, 10) and [45, +∞).
* </p>
*/
public final class StandardRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
private static final String RANGE_LOWER = "range.lower";
public static final String RANGE_LOWER_KEY = "range.lower";
private static final String RANGE_UPPER = "range.upper";
public static final String RANGE_UPPER_KEY = "range.upper";
private static final String PARTITION_VOLUME = "partition.volume";
public static final String SHARDING_VOLUME_KEY = "sharding.volume";
@Override
public Map<Integer, Range<Long>> createPartitionRangeMap(final Properties props) {
Preconditions.checkNotNull(props.get(RANGE_LOWER), "Standard range sharding algorithm partition lower cannot be null.");
Preconditions.checkNotNull(props.get(RANGE_UPPER), "Standard range sharding algorithm partition upper cannot be null.");
Preconditions.checkNotNull(props.get(PARTITION_VOLUME), "Standard range sharding algorithm partition volume cannot be null.");
long lower = Long.parseLong(props.get(RANGE_LOWER).toString());
long upper = Long.parseLong(props.get(RANGE_UPPER).toString());
long volume = Long.parseLong(props.get(PARTITION_VOLUME).toString());
Preconditions.checkNotNull(props.get(RANGE_LOWER_KEY), "Standard range sharding algorithm partition lower cannot be null.");
Preconditions.checkNotNull(props.get(RANGE_UPPER_KEY), "Standard range sharding algorithm partition upper cannot be null.");
Preconditions.checkNotNull(props.get(SHARDING_VOLUME_KEY), "Standard range sharding algorithm partition volume cannot be null.");
long lower = Long.parseLong(props.get(RANGE_LOWER_KEY).toString());
long upper = Long.parseLong(props.get(RANGE_UPPER_KEY).toString());
long volume = Long.parseLong(props.get(SHARDING_VOLUME_KEY).toString());
Preconditions.checkArgument(upper - lower >= volume, "Standard range sharding algorithm partition range can not be smaller than volume.");
int partitionSize = Math.toIntExact(LongMath.divide(upper - lower, volume, RoundingMode.CEILING));
Map<Integer, Range<Long>> result = Maps.newHashMapWithExpectedSize(partitionSize + 2);
......
......@@ -42,9 +42,9 @@ public final class DatetimeShardingAlgorithmTest {
@Before
public void setup() {
DatetimeShardingAlgorithm shardingAlgorithm = new DatetimeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("partition.seconds", "4");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2020-01-01 00:00:16");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.DATETIME_LOWER, "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.DATETIME_UPPER, "2020-01-01 00:00:16");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.SHARDING_SECONDS_KEY, "4");
shardingAlgorithm.init();
shardingStrategy = new StandardShardingStrategy("create_time", shardingAlgorithm);
}
......@@ -128,9 +128,9 @@ public final class DatetimeShardingAlgorithmTest {
@Test
public void assertGetAutoTablesAmount() {
DatetimeShardingAlgorithm shardingAlgorithm = new DatetimeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("partition.seconds", "86400");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.DATETIME_LOWER, "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.DATETIME_UPPER, "2021-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty(DatetimeShardingAlgorithm.SHARDING_SECONDS_KEY, "86400");
shardingAlgorithm.init();
assertThat(shardingAlgorithm.getAutoTablesAmount(), is(368));
}
......
......@@ -42,7 +42,7 @@ public final class CustomRangeShardingAlgorithmTest {
@Before
public void setUp() {
CustomRangeShardingAlgorithm shardingAlgorithm = new CustomRangeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("partition.ranges", "1,5,10");
shardingAlgorithm.getProps().setProperty(CustomRangeShardingAlgorithm.SHARDING_RANGES_KEY, "1,5,10");
shardingAlgorithm.init();
shardingStrategy = new StandardShardingStrategy("order_id", shardingAlgorithm);
}
......@@ -72,7 +72,7 @@ public final class CustomRangeShardingAlgorithmTest {
@Test
public void assertGetAutoTablesAmount() {
CustomRangeShardingAlgorithm shardingAlgorithm = new CustomRangeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("partition.ranges", "1,5,10");
shardingAlgorithm.getProps().setProperty(CustomRangeShardingAlgorithm.SHARDING_RANGES_KEY, "1,5,10");
shardingAlgorithm.init();
assertThat(shardingAlgorithm.getAutoTablesAmount(), is(4));
}
......
......@@ -42,9 +42,9 @@ public final class StandardRangeShardingAlgorithmTest {
@Before
public void setUp() {
StandardRangeShardingAlgorithm shardingAlgorithm = new StandardRangeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("range.lower", "10");
shardingAlgorithm.getProps().setProperty("range.upper", "45");
shardingAlgorithm.getProps().setProperty("partition.volume", "10");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.RANGE_LOWER_KEY, "10");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.RANGE_UPPER_KEY, "45");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.SHARDING_VOLUME_KEY, "10");
shardingAlgorithm.init();
shardingStrategy = new StandardShardingStrategy("order_id", shardingAlgorithm);
}
......@@ -97,9 +97,9 @@ public final class StandardRangeShardingAlgorithmTest {
@Test
public void assertGetAutoTablesAmount() {
StandardRangeShardingAlgorithm shardingAlgorithm = new StandardRangeShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("range.lower", "10");
shardingAlgorithm.getProps().setProperty("range.upper", "45");
shardingAlgorithm.getProps().setProperty("partition.volume", "10");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.RANGE_LOWER_KEY, "10");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.RANGE_UPPER_KEY, "45");
shardingAlgorithm.getProps().setProperty(StandardRangeShardingAlgorithm.SHARDING_VOLUME_KEY, "10");
shardingAlgorithm.init();
assertThat(shardingAlgorithm.getAutoTablesAmount(), is(6));
}
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册