Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
weixin_44739061
DolphinScheduler
提交
8fea8f91
DolphinScheduler
项目概览
weixin_44739061
/
DolphinScheduler
与 Fork 源项目一致
Fork自
apache / DolphinScheduler
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
8fea8f91
编写于
8月 25, 2020
作者:
Y
Yichao Yang
提交者:
GitHub
8月 25, 2020
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Test][server] Add the master server test case (#3574)
上级
f7c4b0f3
变更
3
显示空白变更内容
内联
并排
Showing
3 changed file
with
38 addition
and
34 deletion
+38
-34
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/registry/MasterRegistry.java
...lphinscheduler/server/master/registry/MasterRegistry.java
+30
-29
dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/registry/MasterRegistryTest.java
...nscheduler/server/master/registry/MasterRegistryTest.java
+7
-4
pom.xml
pom.xml
+1
-1
未找到文件。
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/master/registry/MasterRegistry.java
浏览文件 @
8fea8f91
...
@@ -14,8 +14,20 @@
...
@@ -14,8 +14,20 @@
* See the License for the specific language governing permissions and
* See the License for the specific language governing permissions and
* limitations under the License.
* limitations under the License.
*/
*/
package
org.apache.dolphinscheduler.server.master.registry
;
package
org.apache.dolphinscheduler.server.master.registry
;
import
org.apache.dolphinscheduler.common.utils.DateUtils
;
import
org.apache.dolphinscheduler.common.utils.NetUtils
;
import
org.apache.dolphinscheduler.remote.utils.NamedThreadFactory
;
import
org.apache.dolphinscheduler.server.master.config.MasterConfig
;
import
org.apache.dolphinscheduler.server.registry.HeartBeatTask
;
import
org.apache.dolphinscheduler.server.registry.ZookeeperRegistryCenter
;
import
org.apache.curator.framework.CuratorFramework
;
import
org.apache.curator.framework.state.ConnectionState
;
import
org.apache.curator.framework.state.ConnectionStateListener
;
import
java.util.Date
;
import
java.util.Date
;
import
java.util.concurrent.Executors
;
import
java.util.concurrent.Executors
;
import
java.util.concurrent.ScheduledExecutorService
;
import
java.util.concurrent.ScheduledExecutorService
;
...
@@ -23,15 +35,6 @@ import java.util.concurrent.TimeUnit;
...
@@ -23,15 +35,6 @@ import java.util.concurrent.TimeUnit;
import
javax.annotation.PostConstruct
;
import
javax.annotation.PostConstruct
;
import
org.apache.curator.framework.CuratorFramework
;
import
org.apache.curator.framework.state.ConnectionState
;
import
org.apache.curator.framework.state.ConnectionStateListener
;
import
org.apache.dolphinscheduler.common.utils.DateUtils
;
import
org.apache.dolphinscheduler.common.utils.NetUtils
;
import
org.apache.dolphinscheduler.remote.utils.NamedThreadFactory
;
import
org.apache.dolphinscheduler.server.master.config.MasterConfig
;
import
org.apache.dolphinscheduler.server.registry.HeartBeatTask
;
import
org.apache.dolphinscheduler.server.registry.ZookeeperRegistryCenter
;
import
org.slf4j.Logger
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
org.slf4j.LoggerFactory
;
import
org.springframework.beans.factory.annotation.Autowired
;
import
org.springframework.beans.factory.annotation.Autowired
;
...
@@ -65,13 +68,12 @@ public class MasterRegistry {
...
@@ -65,13 +68,12 @@ public class MasterRegistry {
private
ScheduledExecutorService
heartBeatExecutor
;
private
ScheduledExecutorService
heartBeatExecutor
;
/**
/**
*
work
er start time
*
mast
er start time
*/
*/
private
String
startTime
;
private
String
startTime
;
@PostConstruct
@PostConstruct
public
void
init
(){
public
void
init
()
{
this
.
startTime
=
DateUtils
.
dateToString
(
new
Date
());
this
.
startTime
=
DateUtils
.
dateToString
(
new
Date
());
this
.
heartBeatExecutor
=
Executors
.
newSingleThreadScheduledExecutor
(
new
NamedThreadFactory
(
"HeartBeatExecutor"
));
this
.
heartBeatExecutor
=
Executors
.
newSingleThreadScheduledExecutor
(
new
NamedThreadFactory
(
"HeartBeatExecutor"
));
}
}
...
@@ -86,12 +88,12 @@ public class MasterRegistry {
...
@@ -86,12 +88,12 @@ public class MasterRegistry {
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
getZkClient
().
getConnectionStateListenable
().
addListener
(
new
ConnectionStateListener
()
{
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
getZkClient
().
getConnectionStateListenable
().
addListener
(
new
ConnectionStateListener
()
{
@Override
@Override
public
void
stateChanged
(
CuratorFramework
client
,
ConnectionState
newState
)
{
public
void
stateChanged
(
CuratorFramework
client
,
ConnectionState
newState
)
{
if
(
newState
==
ConnectionState
.
LOST
)
{
if
(
newState
==
ConnectionState
.
LOST
)
{
logger
.
error
(
"master : {} connection lost from zookeeper"
,
address
);
logger
.
error
(
"master : {} connection lost from zookeeper"
,
address
);
}
else
if
(
newState
==
ConnectionState
.
RECONNECTED
)
{
}
else
if
(
newState
==
ConnectionState
.
RECONNECTED
)
{
logger
.
info
(
"master : {} reconnected to zookeeper"
,
address
);
logger
.
info
(
"master : {} reconnected to zookeeper"
,
address
);
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
persistEphemeral
(
localNodePath
,
""
);
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
persistEphemeral
(
localNodePath
,
""
);
}
else
if
(
newState
==
ConnectionState
.
SUSPENDED
)
{
}
else
if
(
newState
==
ConnectionState
.
SUSPENDED
)
{
logger
.
warn
(
"master : {} connection SUSPENDED "
,
address
);
logger
.
warn
(
"master : {} connection SUSPENDED "
,
address
);
}
}
}
}
...
@@ -103,8 +105,9 @@ public class MasterRegistry {
...
@@ -103,8 +105,9 @@ public class MasterRegistry {
Sets
.
newHashSet
(
getMasterPath
()),
Sets
.
newHashSet
(
getMasterPath
()),
zookeeperRegistryCenter
);
zookeeperRegistryCenter
);
this
.
heartBeatExecutor
.
scheduleAtFixedRate
(
heartBeatTask
,
masterHeartbeatInterval
,
masterHeartbeatInterval
,
TimeUnit
.
SECONDS
);
this
.
heartBeatExecutor
.
scheduleAtFixedRate
(
heartBeatTask
,
0
,
masterHeartbeatInterval
,
TimeUnit
.
SECONDS
);
logger
.
info
(
"master node : {} registry to ZK successfully with heartBeatInterval : {}s"
,
address
,
masterHeartbeatInterval
);
logger
.
info
(
"master node : {} registry to ZK path {} successfully with heartBeatInterval : {}s"
,
address
,
localNodePath
,
masterHeartbeatInterval
);
}
}
/**
/**
...
@@ -115,24 +118,22 @@ public class MasterRegistry {
...
@@ -115,24 +118,22 @@ public class MasterRegistry {
String
localNodePath
=
getMasterPath
();
String
localNodePath
=
getMasterPath
();
heartBeatExecutor
.
shutdownNow
();
heartBeatExecutor
.
shutdownNow
();
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
remove
(
localNodePath
);
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
remove
(
localNodePath
);
logger
.
info
(
"master node : {} unRegistry to ZK."
,
address
);
logger
.
info
(
"master node : {} unRegistry from ZK path {}."
,
address
,
localNodePath
);
}
}
/**
/**
* get master path
* get master path
* @return
*/
*/
private
String
getMasterPath
()
{
private
String
getMasterPath
()
{
String
address
=
getLocalAddress
();
String
address
=
getLocalAddress
();
String
localNodePath
=
this
.
zookeeperRegistryCenter
.
getMasterPath
()
+
"/"
+
address
;
return
this
.
zookeeperRegistryCenter
.
getMasterPath
()
+
"/"
+
address
;
return
localNodePath
;
}
}
/**
/**
* get local address
* get local address
* @return
*/
*/
private
String
getLocalAddress
(){
private
String
getLocalAddress
()
{
return
NetUtils
.
getHost
()
+
":"
+
masterConfig
.
getListenPort
();
return
NetUtils
.
getHost
()
+
":"
+
masterConfig
.
getListenPort
();
...
...
dolphinscheduler-server/src/test/java/org/apache/dolphinscheduler/server/master/registry/MasterRegistryTest.java
浏览文件 @
8fea8f91
...
@@ -17,6 +17,8 @@
...
@@ -17,6 +17,8 @@
package
org.apache.dolphinscheduler.server.master.registry
;
package
org.apache.dolphinscheduler.server.master.registry
;
import
static
org
.
apache
.
dolphinscheduler
.
common
.
Constants
.
HEARTBEAT_FOR_ZOOKEEPER_INFO_LENGTH
;
import
org.apache.dolphinscheduler.remote.utils.Constants
;
import
org.apache.dolphinscheduler.remote.utils.Constants
;
import
org.apache.dolphinscheduler.server.master.config.MasterConfig
;
import
org.apache.dolphinscheduler.server.master.config.MasterConfig
;
import
org.apache.dolphinscheduler.server.registry.ZookeeperRegistryCenter
;
import
org.apache.dolphinscheduler.server.registry.ZookeeperRegistryCenter
;
...
@@ -24,6 +26,10 @@ import org.apache.dolphinscheduler.server.zk.SpringZKServer;
...
@@ -24,6 +26,10 @@ import org.apache.dolphinscheduler.server.zk.SpringZKServer;
import
org.apache.dolphinscheduler.service.zk.CuratorZookeeperClient
;
import
org.apache.dolphinscheduler.service.zk.CuratorZookeeperClient
;
import
org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator
;
import
org.apache.dolphinscheduler.service.zk.ZookeeperCachedOperator
;
import
org.apache.dolphinscheduler.service.zk.ZookeeperConfig
;
import
org.apache.dolphinscheduler.service.zk.ZookeeperConfig
;
import
java.util.List
;
import
java.util.concurrent.TimeUnit
;
import
org.junit.Assert
;
import
org.junit.Assert
;
import
org.junit.Test
;
import
org.junit.Test
;
import
org.junit.runner.RunWith
;
import
org.junit.runner.RunWith
;
...
@@ -31,10 +37,6 @@ import org.springframework.beans.factory.annotation.Autowired;
...
@@ -31,10 +37,6 @@ import org.springframework.beans.factory.annotation.Autowired;
import
org.springframework.test.context.ContextConfiguration
;
import
org.springframework.test.context.ContextConfiguration
;
import
org.springframework.test.context.junit4.SpringRunner
;
import
org.springframework.test.context.junit4.SpringRunner
;
import
java.util.List
;
import
java.util.concurrent.TimeUnit
;
import
static
org
.
apache
.
dolphinscheduler
.
common
.
Constants
.
HEARTBEAT_FOR_ZOOKEEPER_INFO_LENGTH
;
/**
/**
* master registry test
* master registry test
*/
*/
...
@@ -65,6 +67,7 @@ public class MasterRegistryTest {
...
@@ -65,6 +67,7 @@ public class MasterRegistryTest {
@Test
@Test
public
void
testUnRegistry
()
throws
InterruptedException
{
public
void
testUnRegistry
()
throws
InterruptedException
{
masterRegistry
.
init
();
masterRegistry
.
registry
();
masterRegistry
.
registry
();
TimeUnit
.
SECONDS
.
sleep
(
masterConfig
.
getMasterHeartbeatInterval
()
+
2
);
//wait heartbeat info write into zk node
TimeUnit
.
SECONDS
.
sleep
(
masterConfig
.
getMasterHeartbeatInterval
()
+
2
);
//wait heartbeat info write into zk node
masterRegistry
.
unRegistry
();
masterRegistry
.
unRegistry
();
...
...
pom.xml
浏览文件 @
8fea8f91
...
@@ -820,7 +820,7 @@
...
@@ -820,7 +820,7 @@
<include>
**/server/master/dispatch/host/assign/LowerWeightRoundRobinTest.java
</include>
<include>
**/server/master/dispatch/host/assign/LowerWeightRoundRobinTest.java
</include>
<include>
**/server/master/dispatch/host/assign/RandomSelectorTest.java
</include>
<include>
**/server/master/dispatch/host/assign/RandomSelectorTest.java
</include>
<include>
**/server/master/dispatch/host/assign/RoundRobinSelectorTest.java
</include>
<include>
**/server/master/dispatch/host/assign/RoundRobinSelectorTest.java
</include>
<
!--<include>**/server/master/register/MasterRegistryTest.java</include>--
>
<
include>
**/server/master/register/MasterRegistryTest.java
</include
>
<include>
**/server/master/AlertManagerTest.java
</include>
<include>
**/server/master/AlertManagerTest.java
</include>
<include>
**/server/master/MasterCommandTest.java
</include>
<include>
**/server/master/MasterCommandTest.java
</include>
<include>
**/server/master/DependentTaskTest.java
</include>
<include>
**/server/master/DependentTaskTest.java
</include>
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录