Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
pentaLiker
DolphinScheduler
提交
4bda3cd9
DolphinScheduler
项目概览
pentaLiker
/
DolphinScheduler
与 Fork 源项目一致
Fork自
apache / DolphinScheduler
通知
1
Star
1
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
4bda3cd9
编写于
4月 01, 2020
作者:
Q
qiaozhanwei
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
conflict reslove
上级
ca096e36
变更
3
隐藏空白更改
内联
并排
Showing
3 changed file
with
13 addition
and
9 deletion
+13
-9
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
...phinscheduler/server/worker/runner/TaskExecuteThread.java
+7
-3
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/zk/ZKMasterClient.java
...org/apache/dolphinscheduler/server/zk/ZKMasterClient.java
+5
-5
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/AbstractZKClient.java
.../apache/dolphinscheduler/service/zk/AbstractZKClient.java
+1
-1
未找到文件。
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/worker/runner/TaskExecuteThread.java
浏览文件 @
4bda3cd9
...
...
@@ -21,6 +21,7 @@ import com.alibaba.fastjson.JSONObject;
import
org.apache.dolphinscheduler.common.enums.ExecutionStatus
;
import
org.apache.dolphinscheduler.common.model.TaskNode
;
import
org.apache.dolphinscheduler.common.process.Property
;
import
org.apache.dolphinscheduler.common.process.ResourceInfo
;
import
org.apache.dolphinscheduler.common.task.AbstractParameters
;
import
org.apache.dolphinscheduler.common.task.TaskTimeoutParameter
;
import
org.apache.dolphinscheduler.common.utils.*
;
...
...
@@ -35,6 +36,7 @@ import org.slf4j.LoggerFactory;
import
java.io.File
;
import
java.util.*
;
import
java.util.stream.Collectors
;
import
java.util.stream.Stream
;
/**
...
...
@@ -211,9 +213,11 @@ public class TaskExecuteThread implements Runnable {
AbstractParameters
baseParam
=
TaskParametersUtils
.
getParameters
(
taskNode
.
getType
(),
taskNode
.
getParams
());
if
(
baseParam
!=
null
)
{
List
<
String
>
projectResourceFiles
=
baseParam
.
getResourceFilesList
();
List
<
ResourceInfo
>
projectResourceFiles
=
baseParam
.
getResourceFilesList
();
if
(
projectResourceFiles
!=
null
)
{
projectFiles
.
addAll
(
projectResourceFiles
);
Stream
<
String
>
resourceInfotream
=
projectResourceFiles
.
stream
().
map
(
resourceInfo
->
resourceInfo
.
getRes
());
projectFiles
.
addAll
(
resourceInfotream
.
collect
(
Collectors
.
toList
()));
}
}
...
...
@@ -236,7 +240,7 @@ public class TaskExecuteThread implements Runnable {
if
(!
resFile
.
exists
())
{
try
{
// query the tenant code of the resource according to the name of the resource
String
resHdfsPath
=
HadoopUtils
.
getHdfs
Filen
ame
(
tenantCode
,
resource
);
String
resHdfsPath
=
HadoopUtils
.
getHdfs
ResourceFileN
ame
(
tenantCode
,
resource
);
logger
.
info
(
"get resource file from hdfs :{}"
,
resHdfsPath
);
HadoopUtils
.
getInstance
().
copyHdfsToLocal
(
resHdfsPath
,
execLocalPath
+
File
.
separator
+
resource
,
false
,
true
);
...
...
dolphinscheduler-server/src/main/java/org/apache/dolphinscheduler/server/zk/ZKMasterClient.java
浏览文件 @
4bda3cd9
...
...
@@ -78,7 +78,7 @@ public class ZKMasterClient extends AbstractZKClient {
}
}
catch
(
Exception
e
){
logger
.
error
(
"master start up
exception"
,
e
);
logger
.
error
(
"master start up exception"
,
e
);
}
finally
{
releaseMutex
(
mutex
);
}
...
...
@@ -97,13 +97,13 @@ public class ZKMasterClient extends AbstractZKClient {
*/
@Override
protected
void
dataChanged
(
CuratorFramework
client
,
TreeCacheEvent
event
,
String
path
)
{
if
(
path
.
startsWith
(
getZNodeParentPath
(
ZKNodeType
.
MASTER
)+
Constants
.
SINGLE_SLASH
)){
//monitor master
//monitor master
if
(
path
.
startsWith
(
getZNodeParentPath
(
ZKNodeType
.
MASTER
)+
Constants
.
SINGLE_SLASH
)){
handleMasterEvent
(
event
,
path
);
}
else
if
(
path
.
startsWith
(
getZNodeParentPath
(
ZKNodeType
.
WORKER
)+
Constants
.
SINGLE_SLASH
)){
//monitor worker
}
else
if
(
path
.
startsWith
(
getZNodeParentPath
(
ZKNodeType
.
WORKER
)+
Constants
.
SINGLE_SLASH
)){
//monitor worker
handleWorkerEvent
(
event
,
path
);
}
//other path event, ignore
}
/**
...
...
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/zk/AbstractZKClient.java
浏览文件 @
4bda3cd9
...
...
@@ -126,7 +126,7 @@ public abstract class AbstractZKClient extends ZookeeperCachedOperator {
int
i
=
0
;
for
(
Map
.
Entry
<
String
,
String
>
entry
:
masterMap
.
entrySet
())
{
Server
masterServer
=
ResInfo
.
parseHeartbeatForZKInfo
(
entry
.
getValue
());
masterServer
.
setZkDirectory
(
parentPath
+
"/"
+
entry
.
getKey
());
masterServer
.
setZkDirectory
(
parentPath
+
"/"
+
entry
.
getKey
());
masterServer
.
setId
(
i
);
i
++;
masterServers
.
add
(
masterServer
);
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录