Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
liyuanzhong001
DolphinScheduler
提交
65d603b7
DolphinScheduler
项目概览
liyuanzhong001
/
DolphinScheduler
与 Fork 源项目一致
Fork自
apache / DolphinScheduler
通知
11
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
65d603b7
编写于
2月 26, 2020
作者:
T
Technoboy-
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
refactor heartbeat logic
上级
72670683
变更
2
隐藏空白更改
内联
并排
Showing
2 changed file
with
58 addition
and
46 deletion
+58
-46
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java
...g/apache/dolphinscheduler/server/worker/WorkerServer.java
+2
-44
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/registry/WorkerRegistry.java
...lphinscheduler/server/worker/registry/WorkerRegistry.java
+56
-2
未找到文件。
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/WorkerServer.java
浏览文件 @
65d603b7
...
...
@@ -90,11 +90,6 @@ public class WorkerServer implements IStoppable {
@Autowired
private
AlertDao
alertDao
;
/**
* heartbeat thread pool
*/
private
ScheduledExecutorService
heartbeatWorkerService
;
/**
* task queue impl
*/
...
...
@@ -155,6 +150,7 @@ public class WorkerServer implements IStoppable {
*/
public
static
void
main
(
String
[]
args
)
{
Thread
.
currentThread
().
setName
(
Constants
.
THREAD_NAME_WORKER_SERVER
);
System
.
setProperty
(
"spring.profiles.active"
,
"worker"
);
new
SpringApplicationBuilder
(
WorkerServer
.
class
).
web
(
WebApplicationType
.
NONE
).
run
(
args
);
}
...
...
@@ -173,7 +169,7 @@ public class WorkerServer implements IStoppable {
this
.
nettyRemotingServer
.
registerProcessor
(
CommandType
.
KILL_TASK_REQUEST
,
new
TaskKillProcessor
());
this
.
nettyRemotingServer
.
start
();
this
.
workerRegistry
=
new
WorkerRegistry
(
zookeeperRegistryCenter
,
serverConfig
.
getListenPort
());
this
.
workerRegistry
=
new
WorkerRegistry
(
zookeeperRegistryCenter
,
serverConfig
.
getListenPort
()
,
workerConfig
.
getWorkerHeartbeatInterval
()
);
this
.
workerRegistry
.
registry
();
this
.
zkWorkerClient
.
init
();
...
...
@@ -184,17 +180,8 @@ public class WorkerServer implements IStoppable {
this
.
fetchTaskExecutorService
=
ThreadUtils
.
newDaemonSingleThreadExecutor
(
"Worker-Fetch-Thread-Executor"
);
heartbeatWorkerService
=
ThreadUtils
.
newDaemonThreadScheduledExecutor
(
"Worker-Heartbeat-Thread-Executor"
,
Constants
.
DEFAUL_WORKER_HEARTBEAT_THREAD_NUM
);
// heartbeat thread implement
Runnable
heartBeatThread
=
heartBeatThread
();
zkWorkerClient
.
setStoppable
(
this
);
// regular heartbeat
// delay 5 seconds, send heartbeat every 30 seconds
heartbeatWorkerService
.
scheduleAtFixedRate
(
heartBeatThread
,
5
,
workerConfig
.
getWorkerHeartbeatInterval
(),
TimeUnit
.
SECONDS
);
// kill process thread implement
Runnable
killProcessThread
=
getKillProcessThread
();
...
...
@@ -255,13 +242,6 @@ public class WorkerServer implements IStoppable {
this
.
nettyRemotingServer
.
close
();
this
.
workerRegistry
.
unRegistry
();
try
{
heartbeatWorkerService
.
shutdownNow
();
}
catch
(
Exception
e
){
logger
.
warn
(
"heartbeat service stopped exception"
);
}
logger
.
info
(
"heartbeat service stopped"
);
try
{
ThreadPoolExecutors
.
getInstance
().
shutdown
();
}
catch
(
Exception
e
){
...
...
@@ -298,28 +278,6 @@ public class WorkerServer implements IStoppable {
}
}
/**
* heartbeat thread implement
*
* @return
*/
private
Runnable
heartBeatThread
(){
logger
.
info
(
"start worker heart beat thread..."
);
Runnable
heartBeatThread
=
new
Runnable
()
{
@Override
public
void
run
()
{
// send heartbeat to zk
if
(
StringUtils
.
isEmpty
(
zkWorkerClient
.
getWorkerZNode
())){
logger
.
error
(
"worker send heartbeat to zk failed"
);
}
zkWorkerClient
.
heartBeatForZk
(
zkWorkerClient
.
getWorkerZNode
()
,
Constants
.
WORKER_PREFIX
);
}
};
return
heartBeatThread
;
}
/**
* kill process thread implement
*
...
...
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/registry/WorkerRegistry.java
浏览文件 @
65d603b7
...
...
@@ -19,11 +19,22 @@ package org.apache.dolphinscheduler.server.worker.registry;
import
org.apache.curator.framework.CuratorFramework
;
import
org.apache.curator.framework.state.ConnectionState
;
import
org.apache.curator.framework.state.ConnectionStateListener
;
import
org.apache.dolphinscheduler.common.utils.DateUtils
;
import
org.apache.dolphinscheduler.common.utils.OSUtils
;
import
org.apache.dolphinscheduler.remote.utils.Constants
;
import
org.apache.dolphinscheduler.remote.utils.NamedThreadFactory
;
import
org.apache.dolphinscheduler.server.registry.ZookeeperRegistryCenter
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
java.util.Date
;
import
java.util.concurrent.Executors
;
import
java.util.concurrent.ScheduledExecutorService
;
import
java.util.concurrent.TimeUnit
;
import
static
org
.
apache
.
dolphinscheduler
.
remote
.
utils
.
Constants
.
COMMA
;
/**
* worker registry
*/
...
...
@@ -41,14 +52,32 @@ public class WorkerRegistry {
*/
private
final
int
port
;
/**
* heartbeat interval
*/
private
final
long
heartBeatInterval
;
/**
* heartbeat executor
*/
private
final
ScheduledExecutorService
heartBeatExecutor
;
/**
* worker start time
*/
private
final
String
startTime
;
/**
* construct
* @param zookeeperRegistryCenter zookeeperRegistryCenter
* @param port port
*/
public
WorkerRegistry
(
ZookeeperRegistryCenter
zookeeperRegistryCenter
,
int
port
){
public
WorkerRegistry
(
ZookeeperRegistryCenter
zookeeperRegistryCenter
,
int
port
,
long
heartBeatInterval
){
this
.
zookeeperRegistryCenter
=
zookeeperRegistryCenter
;
this
.
port
=
port
;
this
.
heartBeatInterval
=
heartBeatInterval
;
this
.
startTime
=
DateUtils
.
dateToString
(
new
Date
());
this
.
heartBeatExecutor
=
Executors
.
newSingleThreadScheduledExecutor
(
new
NamedThreadFactory
(
"HeartBeatExecutor"
));
}
/**
...
...
@@ -71,7 +100,9 @@ public class WorkerRegistry {
}
}
});
logger
.
info
(
"worker node : {} registry to ZK successfully."
,
address
);
this
.
heartBeatExecutor
.
scheduleAtFixedRate
(
new
HeartBeatTask
(),
heartBeatInterval
,
heartBeatInterval
,
TimeUnit
.
SECONDS
);
logger
.
info
(
"worker node : {} registry to ZK successfully with heartBeatInterval : {}s"
,
address
,
heartBeatInterval
);
}
/**
...
...
@@ -81,6 +112,7 @@ public class WorkerRegistry {
String
address
=
getLocalAddress
();
String
localNodePath
=
getWorkerPath
();
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
remove
(
localNodePath
);
this
.
heartBeatExecutor
.
shutdownNow
();
logger
.
info
(
"worker node : {} unRegistry to ZK."
,
address
);
}
...
...
@@ -101,4 +133,26 @@ public class WorkerRegistry {
private
String
getLocalAddress
(){
return
Constants
.
LOCAL_ADDRESS
+
":"
+
port
;
}
/**
* hear beat task
*/
class
HeartBeatTask
implements
Runnable
{
@Override
public
void
run
()
{
try
{
StringBuilder
builder
=
new
StringBuilder
(
100
);
builder
.
append
(
OSUtils
.
cpuUsage
()).
append
(
COMMA
);
builder
.
append
(
OSUtils
.
memoryUsage
()).
append
(
COMMA
);
builder
.
append
(
OSUtils
.
loadAverage
()).
append
(
COMMA
);
builder
.
append
(
startTime
).
append
(
COMMA
);
builder
.
append
(
DateUtils
.
dateToString
(
new
Date
()));
String
workerPath
=
getWorkerPath
();
zookeeperRegistryCenter
.
getZookeeperCachedOperator
().
persist
(
workerPath
,
builder
.
toString
());
}
catch
(
Throwable
ex
){
logger
.
error
(
"error write worker heartbeat info"
,
ex
);
}
}
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录