未验证 提交 93181172 编写于 作者: L Liang Zhang 提交者: GitHub

Revise built-in sharding algorithm (#6082)

* refactor FixedIntervalShardingAlgorithm

* refactor MutableIntervalShardingAlgorithm

* refactor MutableIntervalShardingAlgorithm

* Update MutableIntervalShardingAlgorithm docs

* clean javadoc

* fix test cases

* rename ShardingAlgorithmTest

* Update docs
上级 010567d3
......@@ -103,20 +103,7 @@ weight = 1
类名称:org.apache.shardingsphere.infra.config.algorithm.ShardingSphereAlgorithmConfiguration
### 标准分片算法配置
Apache ShardingSphere 内置的标准分片算法实现类包括:
#### 行表达式分片算法
类型:INLINE
可配置属性:
| *属性名称* | *数据类型* | *说明* | *默认值* |
| ----------------------------------------- | --------- | --------------------------------------------------- | ------- |
| algorithm.expression | String | 分片算法的行表达式 | - |
| allow.range.query.with.inline.sharding (?)| boolean | 是否允许范围查询。注意:范围查询会无视分片策略,进行全路由 | false |
### 自动分片算法配置
#### 取模分片算法
......@@ -160,9 +147,9 @@ Apache ShardingSphere 内置的标准分片算法实现类包括:
| --------------- | --------- | --------------------------------- |
| sharding.ranges | String | 分片的范围边界,多个范围边界以逗号分隔 |
#### 定长时间段分片算法
#### 自动时间段分片算法
类型:FIXED_INTERVAL
类型:AUTO_INTERVAL
可配置属性:
......@@ -172,20 +159,35 @@ Apache ShardingSphere 内置的标准分片算法实现类包括:
| datetime.upper | String | 分片的结束时间范围,时间戳格式:yyyy-MM-dd HH:mm:ss |
| sharding.seconds | long | 单一分片所能承载的最大时间,单位:秒 |
#### 基于可变时间范围的分片算法
### 标准分片算法配置
Apache ShardingSphere 内置的标准分片算法实现类包括:
#### 行表达式分片算法
类型:INLINE
可配置属性:
| *属性名称* | *数据类型* | *说明* | *默认值* |
| ----------------------------------------- | --------- | --------------------------------------------------- | ------- |
| algorithm.expression | String | 分片算法的行表达式 | - |
| allow.range.query.with.inline.sharding (?)| boolean | 是否允许范围查询。注意:范围查询会无视分片策略,进行全路由 | false |
#### 时间范围分片算法
类型:MUTABLE_INTERVAL
类型:INTERVAL
可配置属性:
| *属性名称* | *数据类型* | *说明* |
| -------------------- | --------- | ----------------------------------- |
| datetime.format | String | 时间戳格式,例如:yyyy-MM-dd HH:mm:ss |
| table.suffix.format | String | TODO |
| datetime.lower | String | TODO |
| datetime.upper | String | TODO |
| datetime.step.unit | String | TODO |
| datetime.step.amount | String | TODO |
| *属性名称* | *数据类型* | *说明* | *默认值* |
| ---------------------------- | --------- | -------------------------------------------------------------------------------- | ------- |
| datetime.pattern | String | 分片键的时间戳格式,必须遵循 Java DateTimeFormatter 的格式。例如:yyyy-MM-dd HH:mm:ss | - |
| datetime.lower | String | 时间分片下界值,格式与 `datetime.pattern` 定义的时间戳格式一致 | - |
| datetime.upper (?) | String | 时间分片上界值,格式与 `datetime.pattern` 定义的时间戳格式一致 | 当前时间 |
| sharding.suffix.pattern | String | 分片数据源或真实表的后缀格式,必须遵循 Java DateTimeFormatter 的格式。例如:yyyyMM | - |
| datetime.interval.amount (?) | int | 分片键时间间隔,超过该时间间隔将进入下一分片 | 1 |
| datetime.interval.unit (?) | String | 分片键时间间隔单位,必须遵循 Java ChronoUnit 的枚举值。例如:MONTHS | DAYS |
### 复合分片算法配置
......
......@@ -103,20 +103,7 @@ Attributes:
Class name: org.apache.shardingsphere.infra.config.algorithm.ShardingSphereAlgorithmConfiguration
### Standard Sharding Algorithm Configuration
Apache ShardingSphere built-in standard sharding algorithm are:
#### Inline Sharding Algorithm
Type: INLINE
Attributes:
| *Name* | *DataType* | *Description* | *Default Value* |
| ----------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------- | --------------- |
| algorithm.expression | String | Inline expression sharding algorithm | - |
| allow.range.query.with.inline.sharding (?)| boolean | Whether range query is allowed. Note: range query will ignore sharding strategy and conduct full routing | false |
### Auto Sharding Algorithm Configuration
#### Modulo Sharding Algorithm
......@@ -160,9 +147,9 @@ Attributes:
| --------------- | ---------- | ----------------------------------------------------------------- |
| sharding.ranges | String | Range of sharding border, multiple boundaries separated by commas |
#### Fixed Time Range Sharding Algorithm
#### Auto Interval Sharding Algorithm
Type: FIXED_INTERVAL
Type: AUTO_INTERVAL
Attributes:
......@@ -172,20 +159,35 @@ Attributes:
| datetime.upper | String | Shard datetime end boundary, pattern: yyyy-MM-dd HH:mm:ss |
| sharding.seconds | long | Max seconds for the data in one shard |
#### Mutable Interval Sharding Algorithm
### Standard Sharding Algorithm Configuration
Apache ShardingSphere built-in standard sharding algorithm are:
#### Inline Sharding Algorithm
Type: INLINE
Attributes:
| *Name* | *DataType* | *Description* | *Default Value* |
| ----------------------------------------- | ---------- | -------------------------------------------------------------------------------------------------------- | --------------- |
| algorithm.expression | String | Inline expression sharding algorithm | - |
| allow.range.query.with.inline.sharding (?)| boolean | Whether range query is allowed. Note: range query will ignore sharding strategy and conduct full routing | false |
#### Interval Sharding Algorithm
Type: MUTABLE_INTERVAL
Type: INTERVAL
Attributes:
| *Name* | *DataType* | *Description* |
| -------------------- | ---------- | ---------------------------------------------- |
| datetime.format | String | Datetime pattern, example: yyyy-MM-dd HH:mm:ss |
| table.suffix.format | String | TODO |
| datetime.lower | String | TODO |
| datetime.upper | String | TODO |
| datetime.step.unit | String | TODO |
| datetime.step.amount | String | TODO |
| *Name* | *DataType* | *Description* | *Default Value* |
| ---------------------------- | ---------- | --------------------------------------------------------------------------------------------------------------------- | --------------- |
| datetime.pattern | String | Timestamp pattern of sharding value, must can be transformed to Java LocalDateTime. For example: yyyy-MM-dd HH:mm:ss | - |
| datetime.lower | String | Datetime sharding lower boundary, pattern is defined `datetime.pattern` | - |
| datetime.upper (?) | String | Datetime sharding upper boundary, pattern is defined `datetime.pattern` | Now |
| sharding.suffix.pattern | String | Suffix pattern of sharding data sources or tables, must can be transformed to Java LocalDateTime. For example: yyyyMM | - |
| datetime.interval.amount (?) | int | Interval of sharding value | 1 |
| datetime.interval.unit (?) | String | Unit of sharding value interval, must can be transformed to Java ChronoUnit's Enum value. For example: MONTHS | DAYS |
### Complex Sharding Algorithm Configuration
......
......@@ -21,6 +21,7 @@ import com.google.common.base.Preconditions;
import com.google.common.collect.Range;
import lombok.Getter;
import lombok.Setter;
import org.apache.shardingsphere.infra.config.exception.ShardingSphereConfigurationException;
import org.apache.shardingsphere.sharding.api.sharding.ShardingAutoTableAlgorithm;
import org.apache.shardingsphere.sharding.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.RangeShardingValue;
......@@ -36,17 +37,10 @@ import java.util.LinkedHashSet;
import java.util.Properties;
/**
* Fixed interval sharding algorithm.
*
* <p>
* Shard by `y = floor(x/v)` algorithm, which means y begins from 0. v is `sharding.seconds`, and the minimum time unit is 1 sec.
* `datetime.lower` decides the beginning datetime to shard. On the other hand, `datetime.upper` decides the end datetime to shard.
*
* Notice: Anytime less then `datetime.lower` will route to the first partition, and anytime great than `datetime.upper` will route to the last sharding.
* </p>
* Auto interval sharding algorithm.
*/
@Getter
public final class FixedIntervalShardingAlgorithm implements StandardShardingAlgorithm<Comparable<?>>, ShardingAutoTableAlgorithm {
public final class AutoIntervalShardingAlgorithm implements StandardShardingAlgorithm<Comparable<?>>, ShardingAutoTableAlgorithm {
private static final String DATE_TIME_LOWER_KEY = "datetime.lower";
......@@ -54,9 +48,7 @@ public final class FixedIntervalShardingAlgorithm implements StandardShardingAlg
private static final String SHARDING_SECONDS_KEY = "sharding.seconds";
private static final String DATE_TIME_PATTERN = "yyyy-MM-dd HH:mm:ss";
private static final DateTimeFormatter DATE_FORMAT = DateTimeFormatter.ofPattern(DATE_TIME_PATTERN);
private static final DateTimeFormatter DATE_TIME_FORMAT = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
@Setter
private Properties props = new Properties();
......@@ -71,24 +63,24 @@ public final class FixedIntervalShardingAlgorithm implements StandardShardingAlg
@Override
public void init() {
dateTimeLower = getDateTimeLower();
dateTimeUpper = getDateTimeUpper();
dateTimeLower = getDateTime(DATE_TIME_LOWER_KEY);
dateTimeUpper = getDateTime(DATE_TIME_UPPER_KEY);
shardingSeconds = getShardingSeconds();
autoTablesAmount = (int) (Math.ceil(parseDate(props.getProperty(DATE_TIME_UPPER_KEY)) / shardingSeconds) + 2);
}
private LocalDateTime getDateTimeLower() {
Preconditions.checkState(props.containsKey(DATE_TIME_LOWER_KEY) && checkDatetimePattern(props.getProperty(DATE_TIME_LOWER_KEY)), "%s pattern is required.", DATE_TIME_PATTERN);
return LocalDateTime.parse(props.getProperty(DATE_TIME_LOWER_KEY), DATE_FORMAT);
}
private LocalDateTime getDateTimeUpper() {
Preconditions.checkState(props.containsKey(DATE_TIME_UPPER_KEY) && checkDatetimePattern(props.getProperty(DATE_TIME_UPPER_KEY)), "%s pattern is required.", DATE_TIME_PATTERN);
return LocalDateTime.parse(props.getProperty(DATE_TIME_UPPER_KEY), DATE_FORMAT);
private LocalDateTime getDateTime(final String dateTimeKey) {
String value = props.getProperty(dateTimeKey);
Preconditions.checkNotNull(value, "%s cannot be null.", dateTimeKey);
try {
return LocalDateTime.parse(value, DATE_TIME_FORMAT);
} catch (final DateTimeParseException ex) {
throw new ShardingSphereConfigurationException("Invalid %s, datetime pattern should be `yyyy-MM-dd HH:mm:ss`, value is `%s`", dateTimeKey, value);
}
}
private long getShardingSeconds() {
Preconditions.checkArgument(props.containsKey(SHARDING_SECONDS_KEY), "Sharding seconds cannot be null.");
Preconditions.checkArgument(props.containsKey(SHARDING_SECONDS_KEY), "%s cannot be null.", SHARDING_SECONDS_KEY);
return Long.parseLong(props.get(SHARDING_SECONDS_KEY).toString());
}
......@@ -123,8 +115,7 @@ public final class FixedIntervalShardingAlgorithm implements StandardShardingAlg
}
private int doSharding(final long shardingValue) {
DecimalFormat decimalFormat = new DecimalFormat("0.00");
String position = decimalFormat.format((float) shardingValue / shardingSeconds);
String position = new DecimalFormat("0.00").format((float) shardingValue / shardingSeconds);
return Math.min(Math.max(0, (int) Math.ceil(Float.parseFloat(position))), autoTablesAmount - 1);
}
......@@ -136,22 +127,13 @@ public final class FixedIntervalShardingAlgorithm implements StandardShardingAlg
return valueRange.hasUpperBound() ? doSharding(parseDate(valueRange.upperEndpoint())) : autoTablesAmount - 1;
}
private boolean checkDatetimePattern(final String datetime) {
try {
DATE_FORMAT.parse(datetime);
return true;
} catch (final DateTimeParseException ex) {
return false;
}
}
private long parseDate(final Comparable<?> shardingValue) {
LocalDateTime dateValue = LocalDateTime.parse(shardingValue.toString(), DATE_FORMAT);
LocalDateTime dateValue = LocalDateTime.parse(shardingValue.toString(), DATE_TIME_FORMAT);
return Duration.between(dateTimeLower, dateValue).toMillis() / 1000;
}
@Override
public String getType() {
return "FIXED_INTERVAL";
return "AUTO_INTERVAL";
}
}
......@@ -20,6 +20,8 @@ package org.apache.shardingsphere.sharding.algorithm.sharding.datetime;
import com.google.common.base.Preconditions;
import lombok.Getter;
import lombok.Setter;
import org.apache.shardingsphere.infra.config.exception.ShardingSphereConfigurationException;
import org.apache.shardingsphere.sharding.algorithm.sharding.ShardingAlgorithmException;
import org.apache.shardingsphere.sharding.api.sharding.standard.PreciseShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.RangeShardingValue;
import org.apache.shardingsphere.sharding.api.sharding.standard.StandardShardingAlgorithm;
......@@ -27,161 +29,133 @@ import org.apache.shardingsphere.sharding.api.sharding.standard.StandardSharding
import java.time.LocalDateTime;
import java.time.format.DateTimeFormatter;
import java.time.format.DateTimeParseException;
import java.time.temporal.ChronoField;
import java.time.temporal.ChronoUnit;
import java.time.temporal.IsoFields;
import java.time.temporal.TemporalField;
import java.util.Collection;
import java.util.HashSet;
import java.util.Properties;
import java.util.Set;
import java.util.stream.Collectors;
/**
* Mutable interval sharding algorithm that adapt various shard method by define properties below.
*
* <p>properties defined here:
*
* <p>datetime.format: the datetime format used by applications, must can be transformed to {@link LocalDateTime},
* used by {@link LocalDateTime#parse(CharSequence, DateTimeFormatter)}.
*
* <p>table.suffix.format: suffix for sharded tables, used by {@link LocalDateTime#format(DateTimeFormatter)},
* examples:
* suffix=yyyyQQ means shard by {@link IsoFields#QUARTER_OF_YEAR};
* suffix=yyyyMM means shard by {@link ChronoUnit#MONTHS};
* suffix=yyyyMMdd means shard by {@link ChronoField#DAY_OF_YEAR}.
*
* <p>detail explain for each char in datetime.format and table.suffix.format can refer {@link TemporalField}.
*
* <p>datetime.lower and datetime.upper: if app query with only half bound, lower and upper helps to build other half bound,
* datetime.lower must be specified and datetime.upper has a default value to {@link LocalDateTime#now}
* (default value of datetime.upper could only be used when query sql needn't get result that time larger than query time).
*
* <p>datetime.step.unit and datetime.step.amount used for calculate tables for range shard, datetime.step.unit is name of
* {@link ChronoUnit}, default unit is Days and amount is 1, amount + unit should not be larger than but close to your shard range.
*
* <p>examples: when shard by {@link IsoFields#QUARTER_OF_YEAR}, datetime.step.unit = Months and datetime.step.amount = 3 is a better choice.
* Interval sharding algorithm.
*/
public final class MutableIntervalShardingAlgorithm implements StandardShardingAlgorithm<Comparable<?>> {
private static final String DATE_TIME_FORMAT_KEY = "datetime.format";
public final class IntervalShardingAlgorithm implements StandardShardingAlgorithm<Comparable<?>> {
private static final String TABLE_SUFFIX_FORMAT_KEY = "table.suffix.format";
private static final String DATE_TIME_PATTERN_KEY = "datetime.pattern";
private static final String DATE_TIME_LOWER_KEY = "datetime.lower";
private static final String DATE_TIME_UPPER_KEY = "datetime.upper";
private static final String STEP_UNIT_KEY = "datetime.step.unit";
private static final String SHARDING_SUFFIX_FORMAT_KEY = "sharding.suffix.pattern";
private static final String STEP_AMOUNT_KEY = "datetime.step.amount";
private static final String INTERVAL_AMOUNT_KEY = "datetime.interval.amount";
private static final String INTERVAL_UNIT_KEY = "datetime.interval.unit";
@Getter
@Setter
private Properties props = new Properties();
private String datetimeFormat;
private DateTimeFormatter dateTimeFormatter;
private DateTimeFormatter tableSuffixFormat;
private int dateTimePatternLength;
private String dateTimeLower;
private LocalDateTime dateTimeLower;
private String dateTimeUpper;
private LocalDateTime dateTimeUpper;
private ChronoUnit stepUnit;
private DateTimeFormatter tableSuffixPattern;
private int stepAmount;
private DateTimeFormatter datetimeFormatter;
private ChronoUnit stepUnit;
@Override
public void init() {
datetimeFormat = getDatetimeFormat();
tableSuffixFormat = getTableSuffixFormat();
dateTimeLower = getDateTimeLower();
dateTimeUpper = getDateTimeUpper();
stepUnit = null == props.getProperty(STEP_UNIT_KEY) ? ChronoUnit.DAYS : generateStepUnit(props.getProperty(STEP_UNIT_KEY));
stepAmount = Integer.parseInt(props.getProperty(STEP_AMOUNT_KEY, "1"));
datetimeFormatter = DateTimeFormatter.ofPattern(datetimeFormat);
try {
parseDateTimeForValue(dateTimeLower);
if (null != dateTimeUpper) {
parseDateTimeForValue(dateTimeUpper);
}
} catch (final DateTimeParseException ex) {
throw new UnsupportedOperationException("Cannot apply shard value for default lower/upper values", ex);
}
String dateTimePattern = getDateTimePattern();
dateTimeFormatter = DateTimeFormatter.ofPattern(dateTimePattern);
dateTimePatternLength = dateTimePattern.length();
dateTimeLower = getDateTimeLower(dateTimePattern);
dateTimeUpper = getDateTimeUpper(dateTimePattern);
tableSuffixPattern = getTableSuffixPattern();
stepAmount = Integer.parseInt(props.getOrDefault(INTERVAL_AMOUNT_KEY, 1).toString());
stepUnit = props.containsKey(INTERVAL_UNIT_KEY) ? getStepUnit(props.getProperty(INTERVAL_UNIT_KEY)) : ChronoUnit.DAYS;
}
private String getDatetimeFormat() {
Preconditions.checkArgument(props.containsKey(DATE_TIME_FORMAT_KEY));
return props.getProperty(DATE_TIME_FORMAT_KEY);
private String getDateTimePattern() {
Preconditions.checkArgument(props.containsKey(DATE_TIME_PATTERN_KEY), "% can not be null.", DATE_TIME_PATTERN_KEY);
return props.getProperty(DATE_TIME_PATTERN_KEY);
}
private DateTimeFormatter getTableSuffixFormat() {
Preconditions.checkArgument(props.containsKey(TABLE_SUFFIX_FORMAT_KEY));
return DateTimeFormatter.ofPattern(props.getProperty(TABLE_SUFFIX_FORMAT_KEY));
private LocalDateTime getDateTimeLower(final String dateTimePattern) {
Preconditions.checkArgument(props.containsKey(DATE_TIME_LOWER_KEY), "% can not be null.", DATE_TIME_LOWER_KEY);
return getDateTime(DATE_TIME_LOWER_KEY, props.getProperty(DATE_TIME_LOWER_KEY), dateTimePattern);
}
private String getDateTimeLower() {
Preconditions.checkArgument(props.containsKey(DATE_TIME_LOWER_KEY));
return props.getProperty(DATE_TIME_LOWER_KEY);
private LocalDateTime getDateTimeUpper(final String dateTimePattern) {
return props.containsKey(DATE_TIME_UPPER_KEY) ? getDateTime(DATE_TIME_UPPER_KEY, props.getProperty(DATE_TIME_UPPER_KEY), dateTimePattern) : LocalDateTime.now();
}
private LocalDateTime getDateTime(final String dateTimeKey, final String dateTimeValue, final String dateTimePattern) {
try {
return LocalDateTime.parse(dateTimeValue, dateTimeFormatter);
} catch (final DateTimeParseException ex) {
throw new ShardingSphereConfigurationException("Invalid %s, datetime pattern should be `%s`, value is `%s`", dateTimeKey, dateTimePattern, dateTimeValue);
}
}
private String getDateTimeUpper() {
return props.getProperty(DATE_TIME_UPPER_KEY);
private DateTimeFormatter getTableSuffixPattern() {
Preconditions.checkArgument(props.containsKey(SHARDING_SUFFIX_FORMAT_KEY), "% can not be null.", SHARDING_SUFFIX_FORMAT_KEY);
return DateTimeFormatter.ofPattern(props.getProperty(SHARDING_SUFFIX_FORMAT_KEY));
}
private ChronoUnit generateStepUnit(final String stepUnit) {
for (ChronoUnit unit : ChronoUnit.values()) {
if (unit.toString().equalsIgnoreCase(stepUnit)) {
return unit;
private ChronoUnit getStepUnit(final String stepUnit) {
for (ChronoUnit each : ChronoUnit.values()) {
if (each.toString().equalsIgnoreCase(stepUnit)) {
return each;
}
}
throw new UnsupportedOperationException(String.format("Cannot find step unit for specified datetime.step.unit prop: `%s`", stepUnit));
throw new UnsupportedOperationException(String.format("Cannot find step unit for specified %s property: `%s`", INTERVAL_UNIT_KEY, stepUnit));
}
@Override
public String doSharding(final Collection<String> availableTargetNames, final PreciseShardingValue<Comparable<?>> shardingValue) {
return availableTargetNames.stream()
.filter(tableName -> tableName.endsWith(parseDateTimeForValue(shardingValue.getValue().toString()).format(tableSuffixFormat)))
.findFirst().orElseThrow(() -> new UnsupportedOperationException(
String.format("failed to shard value %s, and availableTables %s", shardingValue, availableTargetNames)));
.filter(each -> each.endsWith(parseDateTime(shardingValue.getValue().toString()).format(tableSuffixPattern)))
.findFirst().orElseThrow(() -> new ShardingAlgorithmException(String.format("failed to shard value %s, and availableTables %s", shardingValue, availableTargetNames)));
}
@Override
public Collection<String> doSharding(final Collection<String> availableTargetNames, final RangeShardingValue<Comparable<?>> shardingValue) {
boolean hasStart = shardingValue.getValueRange().hasLowerBound();
boolean hasEnd = shardingValue.getValueRange().hasUpperBound();
Set<String> tables = new HashSet<>();
if (!hasStart && !hasEnd) {
boolean hasStartTime = shardingValue.getValueRange().hasLowerBound();
boolean hasEndTime = shardingValue.getValueRange().hasUpperBound();
if (!hasStartTime && !hasEndTime) {
return availableTargetNames;
}
LocalDateTime start = hasStart ? parseDateTimeForValue(shardingValue.getValueRange().lowerEndpoint().toString()) : parseDateTimeForValue(dateTimeLower);
LocalDateTime end = hasEnd
? parseDateTimeForValue(shardingValue.getValueRange().upperEndpoint().toString())
: dateTimeUpper == null
? LocalDateTime.now()
: parseDateTimeForValue(dateTimeUpper);
LocalDateTime tmp = start;
while (!tmp.isAfter(end)) {
mergeTableIfMatch(tmp, tables, availableTargetNames);
tmp = tmp.plus(stepAmount, stepUnit);
LocalDateTime startTime = hasStartTime ? parseDateTime(shardingValue.getValueRange().lowerEndpoint().toString()) : dateTimeLower;
LocalDateTime endTime = hasEndTime ? parseDateTime(shardingValue.getValueRange().upperEndpoint().toString()) : dateTimeUpper;
LocalDateTime calculateTime = startTime;
Set<String> result = new HashSet<>();
while (!calculateTime.isAfter(endTime)) {
result.addAll(getMatchedTables(calculateTime, availableTargetNames));
calculateTime = calculateTime.plus(stepAmount, stepUnit);
}
mergeTableIfMatch(end, tables, availableTargetNames);
return tables;
result.addAll(getMatchedTables(endTime, availableTargetNames));
return result;
}
private LocalDateTime parseDateTimeForValue(final String value) {
return LocalDateTime.parse(value.substring(0, datetimeFormat.length()), datetimeFormatter);
private LocalDateTime parseDateTime(final String value) {
return LocalDateTime.parse(value.substring(0, dateTimePatternLength), dateTimeFormatter);
}
private void mergeTableIfMatch(final LocalDateTime dateTime, final Collection<String> tables, final Collection<String> availableTargetNames) {
String suffix = dateTime.format(tableSuffixFormat);
availableTargetNames.parallelStream().filter(tableName -> tableName.endsWith(suffix)).findAny().map(tables::add);
private Collection<String> getMatchedTables(final LocalDateTime dateTime, final Collection<String> availableTargetNames) {
String tableSuffix = dateTime.format(tableSuffixPattern);
return availableTargetNames.parallelStream().filter(each -> each.endsWith(tableSuffix)).collect(Collectors.toSet());
}
@Override
public String getType() {
return "MUTABLE_INTERVAL";
return "INTERVAL";
}
}
......@@ -30,11 +30,6 @@ import java.util.Properties;
/**
* Hash sharding algorithm.
*
* <p>
* Shard by `y = z mod v` algorithm with z = hash(x), v is sharding count.
* All available targets will be returned if sharding value is {@code RangeShardingValue}.
* </p>
*/
@Getter
@Setter
......
......@@ -31,10 +31,6 @@ import java.util.Properties;
/**
* Modulo sharding algorithm.
*
* <p>
* Shard by `y = x mod v` algorithm, v is sharding count.
* </p>
*/
@Getter
@Setter
......
......@@ -31,14 +31,6 @@ import java.util.stream.Collectors;
/**
* Boundary based range sharding algorithm.
*
* <p>
* This algorithm is similar to the rule of partition table.
*
* For example: If the `sharding.ranges` parameter is set to `1, 5, 10`, the parameter will split all values into four intervals with (-∞, 1), [1, 5), [5, 10), [10, +∞),
* which corresponding to partition_0, partition_1, partition_2, partition_3.
* The sharding values will be divided into different sharding by its value.
* </p>
*/
public final class BoundaryBasedRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
......
......@@ -28,13 +28,6 @@ import java.util.Properties;
/**
* Volume based range sharding algorithm.
*
* <p>
* This algorithm is similar to the rule of partition table, but it can only be split by the same size.
*
* For example: If the `range.lower` parameter is set to `10`, the `range.upper` parameter is set to `45`, and the `sharding.volume` parameter is set to `10`.
* The values in range [10, 45] will be split to different partitions with [10, 20), [20, 30), [30, 40), [40, 45), and other values will throw exception.
* </p>
*/
public final class VolumeBasedRangeShardingAlgorithm extends AbstractRangeShardingAlgorithm {
......
......@@ -20,5 +20,5 @@ org.apache.shardingsphere.sharding.algorithm.sharding.mod.ModShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.mod.HashModShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.range.VolumeBasedRangeShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.range.BoundaryBasedRangeShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.datetime.FixedIntervalShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.datetime.MutableIntervalShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.datetime.AutoIntervalShardingAlgorithm
org.apache.shardingsphere.sharding.algorithm.sharding.datetime.IntervalShardingAlgorithm
......@@ -36,13 +36,13 @@ import static org.hamcrest.CoreMatchers.is;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
public final class FixedIntervalShardingAlgorithmTest {
public final class AutoIntervalShardingAlgorithmTest {
private StandardShardingStrategy shardingStrategy;
@Before
public void setup() {
FixedIntervalShardingAlgorithm shardingAlgorithm = new FixedIntervalShardingAlgorithm();
AutoIntervalShardingAlgorithm shardingAlgorithm = new AutoIntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.lower", "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2020-01-01 00:00:16");
shardingAlgorithm.getProps().setProperty("sharding.seconds", "4");
......@@ -126,7 +126,7 @@ public final class FixedIntervalShardingAlgorithmTest {
@Test
public void assertGetAutoTablesAmount() {
FixedIntervalShardingAlgorithm shardingAlgorithm = new FixedIntervalShardingAlgorithm();
AutoIntervalShardingAlgorithm shardingAlgorithm = new AutoIntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.lower", "2020-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("sharding.seconds", "86400");
......
......@@ -40,7 +40,7 @@ import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertThat;
import static org.junit.Assert.assertTrue;
public final class MutableIntervalShardingAlgorithmTest {
public final class IntervalShardingAlgorithmTest {
private final List<String> availableTablesForMonthStrategy = new ArrayList<>();
......@@ -57,13 +57,13 @@ public final class MutableIntervalShardingAlgorithmTest {
}
private void initShardStrategyByQuarter() {
MutableIntervalShardingAlgorithm shardingAlgorithm = new MutableIntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.format", "yyyy-MM-dd HH:mm:ss");
shardingAlgorithm.getProps().setProperty("table.suffix.format", "yyyyQQ");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2016-01-01 00:00:00.000");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-12-31 00:00:00.000");
shardingAlgorithm.getProps().setProperty("datetime.step.unit", "Months");
shardingAlgorithm.getProps().setProperty("datetime.step.amount", "3");
IntervalShardingAlgorithm shardingAlgorithm = new IntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.pattern", "yyyy-MM-dd HH:mm:ss");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2016-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-12-31 00:00:00");
shardingAlgorithm.getProps().setProperty("sharding.suffix.pattern", "yyyyQQ");
shardingAlgorithm.getProps().setProperty("datetime.interval.amount", "3");
shardingAlgorithm.getProps().setProperty("datetime.interval.unit", "Months");
shardingAlgorithm.init();
this.shardingStrategyByQuarter = new StandardShardingStrategy("create_time", shardingAlgorithm);
for (int i = 2016; i <= 2020; i++) {
......@@ -74,13 +74,13 @@ public final class MutableIntervalShardingAlgorithmTest {
}
private void initShardStrategyByMonth() {
MutableIntervalShardingAlgorithm shardingAlgorithm = new MutableIntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.format", "yyyy-MM-dd HH:mm:ss");
shardingAlgorithm.getProps().setProperty("table.suffix.format", "yyyyMM");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2016-01-01 00:00:00.000");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-12-31 00:00:00.000");
shardingAlgorithm.getProps().setProperty("datetime.step.unit", "Months");
shardingAlgorithm.getProps().setProperty("datetime.step.amount", "1");
IntervalShardingAlgorithm shardingAlgorithm = new IntervalShardingAlgorithm();
shardingAlgorithm.getProps().setProperty("datetime.pattern", "yyyy-MM-dd HH:mm:ss");
shardingAlgorithm.getProps().setProperty("datetime.lower", "2016-01-01 00:00:00");
shardingAlgorithm.getProps().setProperty("datetime.upper", "2021-12-31 00:00:00");
shardingAlgorithm.getProps().setProperty("sharding.suffix.pattern", "yyyyMM");
shardingAlgorithm.getProps().setProperty("datetime.interval.amount", "1");
shardingAlgorithm.getProps().setProperty("datetime.interval.unit", "Months");
shardingAlgorithm.init();
this.shardingStrategyByMonth = new StandardShardingStrategy("create_time", shardingAlgorithm);
for (int i = 2016; i <= 2020; i++) {
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册