Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
DiDi
kafka-manager
提交
793c7800
K
kafka-manager
项目概览
DiDi
/
kafka-manager
9 个月 前同步成功
通知
58
Star
6372
Fork
1229
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
K
kafka-manager
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
提交
793c7800
编写于
2月 22, 2023
作者:
W
wyb
提交者:
zengqiao
2月 23, 2023
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Bugfix]修复mm2列表请求超时(#949)
调整代码结构
上级
ec6f0634
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
72 addition
and
9 deletion
+72
-9
km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/connect/mm2/impl/MirrorMakerManagerImpl.java
...aming/km/biz/connect/mm2/impl/MirrorMakerManagerImpl.java
+29
-9
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/utils/ApiCallThreadPoolService.java
...now/streaming/km/core/utils/ApiCallThreadPoolService.java
+43
-0
未找到文件。
km-biz/src/main/java/com/xiaojukeji/know/streaming/km/biz/connect/mm2/impl/MirrorMakerManagerImpl.java
浏览文件 @
793c7800
...
...
@@ -41,6 +41,7 @@ import com.xiaojukeji.know.streaming.km.core.service.connect.mm2.MirrorMakerMetr
import
com.xiaojukeji.know.streaming.km.core.service.connect.plugin.PluginService
;
import
com.xiaojukeji.know.streaming.km.core.service.connect.worker.WorkerConnectorService
;
import
com.xiaojukeji.know.streaming.km.core.service.connect.worker.WorkerService
;
import
com.xiaojukeji.know.streaming.km.core.utils.ApiCallThreadPoolService
;
import
com.xiaojukeji.know.streaming.km.persistence.cache.LoadedClusterPhyCache
;
import
org.apache.commons.lang.StringUtils
;
import
org.springframework.beans.factory.annotation.Autowired
;
...
...
@@ -296,7 +297,9 @@ public class MirrorMakerManagerImpl implements MirrorMakerManager {
List
<
ClusterMirrorMakerOverviewVO
>
mirrorMakerOverviewVOList
=
this
.
convert2ClusterMirrorMakerOverviewVO
(
mirrorMakerList
,
connectClusterList
,
latestMetricsResult
.
getData
());
PaginationResult
<
ClusterMirrorMakerOverviewVO
>
voPaginationResult
=
this
.
pagingMirrorMakerInLocal
(
mirrorMakerOverviewVOList
,
dto
);
List
<
ClusterMirrorMakerOverviewVO
>
mirrorMakerVOList
=
this
.
completeClusterInfo
(
mirrorMakerOverviewVOList
);
PaginationResult
<
ClusterMirrorMakerOverviewVO
>
voPaginationResult
=
this
.
pagingMirrorMakerInLocal
(
mirrorMakerVOList
,
dto
);
if
(
voPaginationResult
.
failed
())
{
LOGGER
.
error
(
"method=ClusterMirrorMakerOverviewVO||clusterPhyId={}||result={}||errMsg=pagination in local failed"
,
clusterPhyId
,
voPaginationResult
);
...
...
@@ -304,10 +307,6 @@ public class MirrorMakerManagerImpl implements MirrorMakerManager {
return
PaginationResult
.
buildFailure
(
voPaginationResult
,
dto
);
}
//这里再补充源集群和目的集群信息,减少网络请求。
this
.
completeClusterInfo
(
voPaginationResult
.
getData
().
getBizData
());
// 查询历史指标
Result
<
List
<
MetricMultiLinesVO
>>
lineMetricsResult
=
mirrorMakerMetricService
.
listMirrorMakerClusterMetricsFromES
(
clusterPhyId
,
...
...
@@ -596,14 +595,31 @@ public class MirrorMakerManagerImpl implements MirrorMakerManager {
return
voList
;
}
private
void
completeClusterInfo
(
List
<
ClusterMirrorMakerOverviewVO
>
mirrorMakerVOList
)
{
private
List
<
ClusterMirrorMakerOverviewVO
>
completeClusterInfo
(
List
<
ClusterMirrorMakerOverviewVO
>
mirrorMakerVOList
)
{
Map
<
String
,
KSConnectorInfo
>
connectorInfoMap
=
new
HashMap
<>();
for
(
ClusterMirrorMakerOverviewVO
mirrorMakerVO
:
mirrorMakerVOList
)
{
Result
<
KSConnectorInfo
>
connectorInfoRet
=
connectorService
.
getConnectorInfoFromCluster
(
mirrorMakerVO
.
getConnectClusterId
(),
mirrorMakerVO
.
getConnectorName
());
if
(!
connectorInfoRet
.
hasData
())
{
ApiCallThreadPoolService
.
runnableTask
(
String
.
format
(
"method=completeClusterInfo||connectClusterId=%d||connectorName=%s||getMirrorMakerInfo"
,
mirrorMakerVO
.
getConnectClusterId
(),
mirrorMakerVO
.
getConnectorName
()),
3000
,
()
->
{
Result
<
KSConnectorInfo
>
connectorInfoRet
=
connectorService
.
getConnectorInfoFromCluster
(
mirrorMakerVO
.
getConnectClusterId
(),
mirrorMakerVO
.
getConnectorName
());
if
(
connectorInfoRet
.
hasData
())
{
connectorInfoMap
.
put
(
mirrorMakerVO
.
getConnectClusterId
()
+
mirrorMakerVO
.
getConnectorName
(),
connectorInfoRet
.
getData
());
}
return
connectorInfoRet
.
getData
();
});
}
ApiCallThreadPoolService
.
waitResult
(
1000
);
List
<
ClusterMirrorMakerOverviewVO
>
newMirrorMakerVOList
=
new
ArrayList
<>();
for
(
ClusterMirrorMakerOverviewVO
mirrorMakerVO
:
mirrorMakerVOList
)
{
KSConnectorInfo
connectorInfo
=
connectorInfoMap
.
get
(
mirrorMakerVO
.
getConnectClusterId
()
+
mirrorMakerVO
.
getConnectorName
());
if
(
connectorInfo
==
null
)
{
continue
;
}
KSConnectorInfo
connectorInfo
=
connectorInfoRet
.
getData
();
String
sourceClusterAlias
=
connectorInfo
.
getConfig
().
get
(
MIRROR_MAKER_SOURCE_CLUSTER_ALIAS_FIELD_NAME
);
String
targetClusterAlias
=
connectorInfo
.
getConfig
().
get
(
MIRROR_MAKER_TARGET_CLUSTER_ALIAS_FIELD_NAME
);
...
...
@@ -627,6 +643,10 @@ public class MirrorMakerManagerImpl implements MirrorMakerManager {
}
}
newMirrorMakerVOList
.
add
(
mirrorMakerVO
);
}
return
newMirrorMakerVOList
;
}
}
km-core/src/main/java/com/xiaojukeji/know/streaming/km/core/utils/ApiCallThreadPoolService.java
0 → 100644
浏览文件 @
793c7800
package
com.xiaojukeji.know.streaming.km.core.utils
;
import
com.xiaojukeji.know.streaming.km.common.utils.FutureWaitUtil
;
import
lombok.NoArgsConstructor
;
import
org.springframework.beans.factory.annotation.Value
;
import
org.springframework.stereotype.Service
;
import
javax.annotation.PostConstruct
;
import
java.util.concurrent.Callable
;
/**
* @author wyb
* @date 2023/2/22
*/
@Service
@NoArgsConstructor
public
class
ApiCallThreadPoolService
{
@Value
(
value
=
"${thread-pool.api.thread-num:2}"
)
private
Integer
threadNum
;
@Value
(
value
=
"${thread-pool.api.queue-size:500}"
)
private
Integer
queueSize
;
private
static
FutureWaitUtil
<
Object
>
apiFutureUtil
;
@PostConstruct
private
void
init
()
{
apiFutureUtil
=
FutureWaitUtil
.
init
(
"ApiCallTP"
,
threadNum
,
threadNum
,
queueSize
);
}
public
static
void
runnableTask
(
String
taskName
,
Integer
timeoutUnisMs
,
Callable
<
Object
>
callable
)
{
apiFutureUtil
.
runnableTask
(
taskName
,
timeoutUnisMs
,
callable
);
}
public
static
void
waitResult
(
Integer
stepWaitTimeUnitMs
)
{
apiFutureUtil
.
waitResult
(
stepWaitTimeUnitMs
);
}
}
\ No newline at end of file
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录