Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
DiDi
kafka-manager
提交
4543a339
K
kafka-manager
项目概览
DiDi
/
kafka-manager
10 个月 前同步成功
通知
58
Star
6372
Fork
1229
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
K
kafka-manager
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
4543a339
编写于
12月 09, 2022
作者:
W
wyb
提交者:
EricZeng
12月 14, 2022
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Bugfix]修复job更新中的数组越界报错(#744)
上级
1c4fbef9
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
50 addition
and
17 deletion
+50
-17
km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/reassign/ReassignResult.java
...eaming/km/common/bean/entity/reassign/ReassignResult.java
+0
-7
km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/utils/CommonUtils.java
...iaojukeji/know/streaming/km/common/utils/CommonUtils.java
+16
-0
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/reassign/impl/ReassignJobServiceImpl.java
...km/core/service/reassign/impl/ReassignJobServiceImpl.java
+34
-10
未找到文件。
km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/bean/entity/reassign/ReassignResult.java
浏览文件 @
4543a339
package
com.xiaojukeji.know.streaming.km.common.bean.entity.reassign
;
import
com.xiaojukeji.know.streaming.km.common.utils.CommonUtils
;
import
lombok.Data
;
import
org.apache.kafka.common.TopicPartition
;
...
...
@@ -20,10 +19,4 @@ public class ReassignResult {
return
state
.
isDone
();
}
public
boolean
checkPreferredReplicaElectionUnNeed
(
String
reassignBrokerIds
,
String
originalBrokerIds
)
{
Integer
targetLeader
=
CommonUtils
.
string2IntList
(
reassignBrokerIds
).
get
(
0
);
Integer
originalLeader
=
CommonUtils
.
string2IntList
(
originalBrokerIds
).
get
(
0
);
return
originalLeader
.
equals
(
targetLeader
);
}
}
km-common/src/main/java/com/xiaojukeji/know/streaming/km/common/utils/CommonUtils.java
浏览文件 @
4543a339
...
...
@@ -261,4 +261,20 @@ public class CommonUtils {
return
null
;
}
}
/**
* 校验两个list的第一个元素是否相等,以","分隔元素。
* @param str1
* @param str2
* @return
*/
public
static
boolean
checkFirstElementIsEquals
(
String
str1
,
String
str2
)
{
if
(
ValidateUtils
.
anyBlank
(
str1
,
str2
))
{
return
false
;
}
Integer
targetLeader
=
CommonUtils
.
string2IntList
(
str1
).
get
(
0
);
Integer
originalLeader
=
CommonUtils
.
string2IntList
(
str2
).
get
(
0
);
return
originalLeader
.
equals
(
targetLeader
);
}
}
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/service/reassign/impl/ReassignJobServiceImpl.java
浏览文件 @
4543a339
...
...
@@ -28,6 +28,7 @@ import com.xiaojukeji.know.streaming.km.common.enums.operaterecord.OperationEnum
import
com.xiaojukeji.know.streaming.km.common.utils.CommonUtils
;
import
com.xiaojukeji.know.streaming.km.common.utils.ConvertUtil
;
import
com.xiaojukeji.know.streaming.km.common.utils.ValidateUtils
;
import
com.xiaojukeji.know.streaming.km.common.utils.kafka.KafkaReassignUtil
;
import
com.xiaojukeji.know.streaming.km.core.service.broker.BrokerService
;
import
com.xiaojukeji.know.streaming.km.core.service.cluster.ClusterPhyService
;
import
com.xiaojukeji.know.streaming.km.core.service.oprecord.OpLogWrapService
;
...
...
@@ -385,11 +386,13 @@ public class ReassignJobServiceImpl implements ReassignJobService {
// 更新任务状态
rv
=
this
.
checkAndSetSuccessIfFinished
(
jobPO
,
rrr
.
getData
());
if
(
rv
.
successful
()){
//如果任务还未完成,先返回,不必考虑优先副本的重新选举。
if
(!
rv
.
successful
())
{
return
Result
.
buildFromIgnoreData
(
rv
);
}
//
已完成
//
任务已完成,检查是否需要重新选举,并进行选举。
rv
=
this
.
preferredReplicaElection
(
jobId
);
...
...
@@ -500,10 +503,8 @@ public class ReassignJobServiceImpl implements ReassignJobService {
List
<
ReassignSubJobPO
>
subJobPOList
=
this
.
getSubJobsByJobId
(
jobId
);
List
<
TopicPartition
>
topicPartitions
=
new
ArrayList
<>();
subJobPOList
.
stream
().
forEach
(
reassignPO
->
{
Integer
targetLeader
=
CommonUtils
.
string2IntList
(
reassignPO
.
getReassignBrokerIds
()).
get
(
0
);
Integer
originalLeader
=
CommonUtils
.
string2IntList
(
reassignPO
.
getOriginalBrokerIds
()).
get
(
0
);
//替换过leader的添加到优先副本选举任务列表
if
(!
originalLeader
.
equals
(
targetLeader
))
{
if
(!
CommonUtils
.
checkFirstElementIsEquals
(
reassignPO
.
getReassignBrokerIds
(),
reassignPO
.
getOriginalBrokerIds
()))
{
topicPartitions
.
add
(
new
TopicPartition
(
reassignPO
.
getTopicName
(),
reassignPO
.
getPartitionId
()));
}
});
...
...
@@ -534,8 +535,12 @@ public class ReassignJobServiceImpl implements ReassignJobService {
if
(
dbSubPO
==
null
)
{
// DB中不存在
reassignSubJobDAO
.
insert
(
elem
);
return
;
}
//补全缺失信息
this
.
completeInfo
(
elem
,
dbSubPO
);
// 已存在则进行更新
elem
.
setId
(
dbSubPO
.
getId
());
reassignSubJobDAO
.
updateById
(
elem
);
...
...
@@ -565,13 +570,10 @@ public class ReassignJobServiceImpl implements ReassignJobService {
long
now
=
System
.
currentTimeMillis
();
boolean
existNotFinished
=
false
;
boolean
unNeedPreferredReplicaElection
=
tru
e
;
boolean
jobSucceed
=
fals
e
;
List
<
ReassignSubJobPO
>
subJobPOList
=
this
.
getSubJobsByJobId
(
jobPO
.
getId
());
for
(
ReassignSubJobPO
subJobPO:
subJobPOList
)
{
if
(!
reassignmentResult
.
checkPreferredReplicaElectionUnNeed
(
subJobPO
.
getReassignBrokerIds
(),
subJobPO
.
getOriginalBrokerIds
()))
{
unNeedPreferredReplicaElection
=
false
;
}
if
(!
reassignmentResult
.
checkPartitionFinished
(
subJobPO
.
getTopicName
(),
subJobPO
.
getPartitionId
()))
{
existNotFinished
=
true
;
...
...
@@ -591,12 +593,13 @@ public class ReassignJobServiceImpl implements ReassignJobService {
// 当前没有分区处于迁移中, 并且没有任务并不处于执行中
ReassignJobPO
newJobPO
=
new
ReassignJobPO
();
newJobPO
.
setId
(
jobPO
.
getId
());
jobSucceed
=
true
;
newJobPO
.
setStatus
(
JobStatusEnum
.
SUCCESS
.
getStatus
());
newJobPO
.
setFinishedTime
(
new
Date
(
now
));
reassignJobDAO
.
updateById
(
newJobPO
);
}
return
Result
.
build
(
unNeedPreferredReplicaElection
);
return
Result
.
build
(
jobSucceed
);
}
private
Result
<
List
<
ReassignSubJobPO
>>
setJobInRunning
(
ReassignJobPO
jobPO
)
{
...
...
@@ -861,4 +864,25 @@ public class ReassignJobServiceImpl implements ReassignJobService {
return
returnRV
;
}
private
void
completeInfo
(
ReassignSubJobPO
newPO
,
ReassignSubJobPO
dbPO
)
{
if
(
newPO
.
getJobId
()
==
null
)
{
newPO
.
setJobId
(
dbPO
.
getJobId
());
}
if
(
newPO
.
getTopicName
()
==
null
)
{
newPO
.
setTopicName
(
dbPO
.
getTopicName
());
}
if
(
newPO
.
getClusterPhyId
()
==
null
)
{
newPO
.
setClusterPhyId
(
dbPO
.
getClusterPhyId
());
}
if
(
newPO
.
getPartitionId
()
==
null
)
{
newPO
.
setPartitionId
(
dbPO
.
getPartitionId
());
}
if
(
newPO
.
getOriginalBrokerIds
()
==
null
||
newPO
.
getOriginalBrokerIds
().
isEmpty
())
{
newPO
.
setOriginalBrokerIds
(
dbPO
.
getOriginalBrokerIds
());
}
if
(
newPO
.
getReassignBrokerIds
()
==
null
||
newPO
.
getReassignBrokerIds
().
isEmpty
())
{
newPO
.
setReassignBrokerIds
(
dbPO
.
getReassignBrokerIds
());
}
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录