Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
s920243400
Rocketmq
提交
3d685903
R
Rocketmq
项目概览
s920243400
/
Rocketmq
与 Fork 源项目一致
Fork自
Apache RocketMQ / Rocketmq
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
R
Rocketmq
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
提交
3d685903
编写于
11月 23, 2021
作者:
D
dongeforever
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Polish the code
上级
daf47490
变更
6
隐藏空白更改
内联
并排
Showing
6 changed file
with
122 addition
and
1 deletion
+122
-1
common/src/main/java/org/apache/rocketmq/common/rpc/ClientMetadata.java
...n/java/org/apache/rocketmq/common/rpc/ClientMetadata.java
+1
-1
common/src/main/java/org/apache/rocketmq/common/statictopic/TopicQueueMappingUtils.java
...e/rocketmq/common/statictopic/TopicQueueMappingUtils.java
+74
-0
remoting/src/main/java/org/apache/rocketmq/remoting/netty/NettyRemotingClient.java
...g/apache/rocketmq/remoting/netty/NettyRemotingClient.java
+1
-0
tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java
...va/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java
+10
-0
tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java
...rg/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java
+31
-0
tools/src/main/java/org/apache/rocketmq/tools/admin/MQAdminExt.java
...main/java/org/apache/rocketmq/tools/admin/MQAdminExt.java
+5
-0
未找到文件。
common/src/main/java/org/apache/rocketmq/common/rpc/ClientMetadata.java
浏览文件 @
3d685903
...
...
@@ -112,7 +112,7 @@ public class ClientMetadata {
for
(
Map
.
Entry
<
String
,
TopicQueueMappingInfo
>
entry
:
mappingInfos
)
{
TopicQueueMappingInfo
info
=
entry
.
getValue
();
if
(
info
.
getEpoch
()
>=
maxTotalNumOfEpoch
&&
info
.
getTotalQueues
()
>
maxTotalNums
)
{
maxTotalNums
=
entry
.
getValue
()
.
getTotalQueues
();
maxTotalNums
=
info
.
getTotalQueues
();
}
for
(
Map
.
Entry
<
Integer
,
Integer
>
idEntry
:
entry
.
getValue
().
getCurrIdMap
().
entrySet
())
{
int
globalId
=
idEntry
.
getKey
();
...
...
common/src/main/java/org/apache/rocketmq/common/statictopic/TopicQueueMappingUtils.java
浏览文件 @
3d685903
...
...
@@ -18,6 +18,9 @@ package org.apache.rocketmq.common.statictopic;
import
com.google.common.collect.ImmutableList
;
import
org.apache.rocketmq.common.MixAll
;
import
org.apache.rocketmq.common.TopicConfig
;
import
org.apache.rocketmq.common.rpc.ClientMetadata
;
import
org.apache.rocketmq.remoting.exception.RemotingException
;
import
java.io.File
;
import
java.util.AbstractMap
;
...
...
@@ -29,6 +32,8 @@ import java.util.HashMap;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Random
;
import
java.util.Set
;
import
java.util.stream.Collectors
;
public
class
TopicQueueMappingUtils
{
...
...
@@ -306,6 +311,75 @@ public class TopicQueueMappingUtils {
}
}
public
Map
<
String
,
TopicConfigAndQueueMapping
>
createTopicConfigMapping
(
String
topic
,
int
queueNum
,
Set
<
String
>
targetBrokers
,
Map
<
String
,
TopicConfigAndQueueMapping
>
brokerConfigMap
)
{
Map
<
Integer
,
TopicQueueMappingOne
>
globalIdMap
=
new
HashMap
<
Integer
,
TopicQueueMappingOne
>();
Map
.
Entry
<
Long
,
Integer
>
maxEpochAndNum
=
new
AbstractMap
.
SimpleImmutableEntry
<
Long
,
Integer
>(
System
.
currentTimeMillis
(),
queueNum
);
if
(!
brokerConfigMap
.
isEmpty
())
{
maxEpochAndNum
=
TopicQueueMappingUtils
.
checkConsistenceOfTopicConfigAndQueueMapping
(
topic
,
brokerConfigMap
);
globalIdMap
=
TopicQueueMappingUtils
.
checkAndBuildMappingItems
(
new
ArrayList
<
TopicQueueMappingDetail
>(
TopicQueueMappingUtils
.
getMappingDetailFromConfig
(
brokerConfigMap
.
values
())),
false
,
true
);
}
if
(
queueNum
<
globalIdMap
.
size
())
{
throw
new
RuntimeException
(
String
.
format
(
"Cannot decrease the queue num for static topic %d < %d"
,
queueNum
,
globalIdMap
.
size
()));
}
//check the queue number
if
(
queueNum
==
globalIdMap
.
size
())
{
throw
new
RuntimeException
(
"The topic queue num is equal the existed queue num, do nothing"
);
}
//the check is ok, now do the mapping allocation
Map
<
String
,
Integer
>
brokerNumMap
=
new
HashMap
<
String
,
Integer
>();
for
(
String
broker:
targetBrokers
)
{
brokerNumMap
.
put
(
broker
,
0
);
}
final
Map
<
Integer
,
String
>
oldIdToBroker
=
new
HashMap
<
Integer
,
String
>();
for
(
Map
.
Entry
<
Integer
,
TopicQueueMappingOne
>
entry
:
globalIdMap
.
entrySet
())
{
String
leaderbroker
=
entry
.
getValue
().
getBname
();
oldIdToBroker
.
put
(
entry
.
getKey
(),
leaderbroker
);
if
(!
brokerNumMap
.
containsKey
(
leaderbroker
))
{
brokerNumMap
.
put
(
leaderbroker
,
1
);
}
else
{
brokerNumMap
.
put
(
leaderbroker
,
brokerNumMap
.
get
(
leaderbroker
)
+
1
);
}
}
TopicQueueMappingUtils
.
MappingAllocator
allocator
=
TopicQueueMappingUtils
.
buildMappingAllocator
(
oldIdToBroker
,
brokerNumMap
);
allocator
.
upToNum
(
queueNum
);
Map
<
Integer
,
String
>
newIdToBroker
=
allocator
.
getIdToBroker
();
//construct the topic configAndMapping
long
newEpoch
=
Math
.
max
(
maxEpochAndNum
.
getKey
()
+
1000
,
System
.
currentTimeMillis
());
for
(
Map
.
Entry
<
Integer
,
String
>
e
:
newIdToBroker
.
entrySet
())
{
Integer
queueId
=
e
.
getKey
();
String
broker
=
e
.
getValue
();
if
(
globalIdMap
.
containsKey
(
queueId
))
{
//ignore the exited
continue
;
}
TopicConfigAndQueueMapping
configMapping
;
if
(!
brokerConfigMap
.
containsKey
(
broker
))
{
configMapping
=
new
TopicConfigAndQueueMapping
(
new
TopicConfig
(
topic
),
new
TopicQueueMappingDetail
(
topic
,
0
,
broker
,
-
1
));
configMapping
.
setWriteQueueNums
(
1
);
configMapping
.
setReadQueueNums
(
1
);
brokerConfigMap
.
put
(
broker
,
configMapping
);
}
else
{
configMapping
=
brokerConfigMap
.
get
(
broker
);
configMapping
.
setWriteQueueNums
(
configMapping
.
getWriteQueueNums
()
+
1
);
configMapping
.
setReadQueueNums
(
configMapping
.
getReadQueueNums
()
+
1
);
}
LogicQueueMappingItem
mappingItem
=
new
LogicQueueMappingItem
(
0
,
configMapping
.
getWriteQueueNums
()
-
1
,
broker
,
0
,
0
,
-
1
,
-
1
,
-
1
);
configMapping
.
getMappingDetail
().
putMappingInfo
(
queueId
,
ImmutableList
.
of
(
mappingItem
));
}
// set the topic config
for
(
Map
.
Entry
<
String
,
TopicConfigAndQueueMapping
>
entry
:
brokerConfigMap
.
entrySet
())
{
TopicConfigAndQueueMapping
configMapping
=
entry
.
getValue
();
configMapping
.
getMappingDetail
().
setEpoch
(
newEpoch
);
configMapping
.
getMappingDetail
().
setTotalQueues
(
queueNum
);
}
//double check the config
TopicQueueMappingUtils
.
checkConsistenceOfTopicConfigAndQueueMapping
(
topic
,
brokerConfigMap
);
TopicQueueMappingUtils
.
checkAndBuildMappingItems
(
new
ArrayList
<
TopicQueueMappingDetail
>(
TopicQueueMappingUtils
.
getMappingDetailFromConfig
(
brokerConfigMap
.
values
())),
false
,
true
);
return
brokerConfigMap
;
}
}
remoting/src/main/java/org/apache/rocketmq/remoting/netty/NettyRemotingClient.java
浏览文件 @
3d685903
...
...
@@ -161,6 +161,7 @@ public class NettyRemotingClient extends NettyRemotingAbstract implements Remoti
}
});
Bootstrap
handler
=
this
.
bootstrap
.
group
(
this
.
eventLoopGroupWorker
).
channel
(
NioSocketChannel
.
class
)
.
option
(
ChannelOption
.
TCP_NODELAY
,
true
)
.
option
(
ChannelOption
.
SO_KEEPALIVE
,
false
)
...
...
tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExt.java
浏览文件 @
3d685903
...
...
@@ -17,6 +17,7 @@
package
org.apache.rocketmq.tools.admin
;
import
java.io.UnsupportedEncodingException
;
import
java.util.HashMap
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.Properties
;
...
...
@@ -28,6 +29,9 @@ import org.apache.rocketmq.client.exception.MQClientException;
import
org.apache.rocketmq.common.AclConfig
;
import
org.apache.rocketmq.common.PlainAccessConfig
;
import
org.apache.rocketmq.common.TopicConfig
;
import
org.apache.rocketmq.common.protocol.route.QueueData
;
import
org.apache.rocketmq.common.rpc.ClientMetadata
;
import
org.apache.rocketmq.common.statictopic.TopicConfigAndQueueMapping
;
import
org.apache.rocketmq.common.statictopic.TopicQueueMappingDetail
;
import
org.apache.rocketmq.common.admin.ConsumeStats
;
import
org.apache.rocketmq.common.admin.RollbackStats
;
...
...
@@ -220,6 +224,11 @@ public class DefaultMQAdminExt extends ClientConfig implements MQAdminExt {
return
defaultMQAdminExtImpl
.
examineTopicConfig
(
addr
,
topic
);
}
@Override
public
Map
<
String
,
TopicConfigAndQueueMapping
>
getTopicConfigMap
(
ClientMetadata
clientMetadata
,
String
topic
)
throws
RemotingException
,
MQBrokerException
,
InterruptedException
,
MQClientException
{
return
this
.
defaultMQAdminExtImpl
.
getTopicConfigMap
(
clientMetadata
,
topic
);
}
@Override
public
TopicStatsTable
examineTopicStats
(
String
topic
)
throws
RemotingException
,
MQClientException
,
InterruptedException
,
...
...
@@ -665,6 +674,7 @@ public class DefaultMQAdminExt extends ClientConfig implements MQAdminExt {
this
.
defaultMQAdminExtImpl
.
createStaticTopic
(
addr
,
defaultTopic
,
topicConfig
,
mappingDetail
,
force
);
}
@Override
public
void
migrateTopicLogicalQueueNotify
(
String
brokerAddr
,
LogicalQueueRouteData
fromQueueRouteData
,
LogicalQueueRouteData
toQueueRouteData
)
throws
InterruptedException
,
MQBrokerException
,
RemotingTimeoutException
,
RemotingSendRequestException
,
RemotingConnectException
{
...
...
tools/src/main/java/org/apache/rocketmq/tools/admin/DefaultMQAdminExtImpl.java
浏览文件 @
3d685903
...
...
@@ -17,6 +17,7 @@
package
org.apache.rocketmq.tools.admin
;
import
java.io.UnsupportedEncodingException
;
import
java.util.AbstractMap
;
import
java.util.ArrayList
;
import
java.util.Arrays
;
import
java.util.Collections
;
...
...
@@ -29,6 +30,9 @@ import java.util.Map.Entry;
import
java.util.Properties
;
import
java.util.Random
;
import
java.util.Set
;
import
java.util.stream.Collectors
;
import
com.google.common.collect.ImmutableList
;
import
org.apache.commons.lang3.StringUtils
;
import
org.apache.rocketmq.client.QueryResult
;
import
org.apache.rocketmq.client.admin.MQAdminExtInner
;
...
...
@@ -42,6 +46,9 @@ import org.apache.rocketmq.common.MixAll;
import
org.apache.rocketmq.common.PlainAccessConfig
;
import
org.apache.rocketmq.common.ServiceState
;
import
org.apache.rocketmq.common.TopicConfig
;
import
org.apache.rocketmq.common.rpc.ClientMetadata
;
import
org.apache.rocketmq.common.statictopic.LogicQueueMappingItem
;
import
org.apache.rocketmq.common.statictopic.TopicConfigAndQueueMapping
;
import
org.apache.rocketmq.common.statictopic.TopicQueueMappingDetail
;
import
org.apache.rocketmq.common.UtilAll
;
import
org.apache.rocketmq.common.admin.ConsumeStats
;
...
...
@@ -82,6 +89,9 @@ import org.apache.rocketmq.common.protocol.route.LogicalQueuesInfo;
import
org.apache.rocketmq.common.protocol.route.MessageQueueRouteState
;
import
org.apache.rocketmq.common.protocol.route.QueueData
;
import
org.apache.rocketmq.common.protocol.route.TopicRouteData
;
import
org.apache.rocketmq.common.statictopic.TopicQueueMappingOne
;
import
org.apache.rocketmq.common.statictopic.TopicQueueMappingUtils
;
import
org.apache.rocketmq.common.statictopic.TopicRemappingDetailWrapper
;
import
org.apache.rocketmq.common.subscription.SubscriptionGroupConfig
;
import
org.apache.rocketmq.logging.InternalLogger
;
import
org.apache.rocketmq.remoting.RPCHook
;
...
...
@@ -1100,6 +1110,27 @@ public class DefaultMQAdminExtImpl implements MQAdminExt, MQAdminExtInner {
this
.
mqClientInstance
.
getMQAdminImpl
().
createStaticTopic
(
addr
,
defaultTopic
,
topicConfig
,
mappingDetail
,
force
);
}
@Override
public
Map
<
String
,
TopicConfigAndQueueMapping
>
getTopicConfigMap
(
ClientMetadata
clientMetadata
,
String
topic
)
throws
RemotingException
,
MQBrokerException
,
InterruptedException
,
MQClientException
{
TopicRouteData
routeData
=
examineTopicRouteInfo
(
topic
);
clientMetadata
.
freshTopicRoute
(
topic
,
routeData
);
Map
<
String
,
TopicConfigAndQueueMapping
>
brokerConfigMap
=
new
HashMap
<>();
if
(
routeData
!=
null
&&
!
routeData
.
getQueueDatas
().
isEmpty
())
{
for
(
QueueData
queueData:
routeData
.
getQueueDatas
())
{
String
bname
=
queueData
.
getBrokerName
();
String
addr
=
clientMetadata
.
findMasterBrokerAddr
(
bname
);
TopicConfigAndQueueMapping
mapping
=
(
TopicConfigAndQueueMapping
)
examineTopicConfig
(
addr
,
topic
);
//allow the config is null
if
(
mapping
!=
null
)
{
brokerConfigMap
.
put
(
bname
,
mapping
);
}
}
}
return
brokerConfigMap
;
}
@Override
public
long
searchOffset
(
MessageQueue
mq
,
long
timestamp
)
throws
MQClientException
{
return
this
.
mqClientInstance
.
getMQAdminImpl
().
searchOffset
(
mq
,
timestamp
);
...
...
tools/src/main/java/org/apache/rocketmq/tools/admin/MQAdminExt.java
浏览文件 @
3d685903
...
...
@@ -27,6 +27,8 @@ import org.apache.rocketmq.client.exception.MQClientException;
import
org.apache.rocketmq.common.AclConfig
;
import
org.apache.rocketmq.common.PlainAccessConfig
;
import
org.apache.rocketmq.common.TopicConfig
;
import
org.apache.rocketmq.common.rpc.ClientMetadata
;
import
org.apache.rocketmq.common.statictopic.TopicConfigAndQueueMapping
;
import
org.apache.rocketmq.common.statictopic.TopicQueueMappingDetail
;
import
org.apache.rocketmq.common.admin.ConsumeStats
;
import
org.apache.rocketmq.common.admin.RollbackStats
;
...
...
@@ -340,6 +342,9 @@ public interface MQAdminExt extends MQAdmin {
void
createStaticTopic
(
final
String
addr
,
final
String
defaultTopic
,
final
TopicConfig
topicConfig
,
final
TopicQueueMappingDetail
mappingDetail
,
final
boolean
force
)
throws
RemotingException
,
MQBrokerException
,
InterruptedException
,
MQClientException
;
Map
<
String
,
TopicConfigAndQueueMapping
>
getTopicConfigMap
(
ClientMetadata
clientMetadata
,
String
topic
)
throws
RemotingException
,
MQBrokerException
,
InterruptedException
,
MQClientException
;
void
migrateTopicLogicalQueueNotify
(
String
brokerAddr
,
LogicalQueueRouteData
fromQueueRouteData
,
LogicalQueueRouteData
toQueueRouteData
)
throws
InterruptedException
,
RemotingConnectException
,
RemotingTimeoutException
,
RemotingSendRequestException
,
MQBrokerException
;
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录