Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
xiphi1978
DolphinScheduler
提交
8f4bed90
DolphinScheduler
项目概览
xiphi1978
/
DolphinScheduler
与 Fork 源项目一致
Fork自
apache / DolphinScheduler
通知
2
Star
0
Fork
1
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
8f4bed90
编写于
6月 24, 2019
作者:
leon-baoliang
浏览文件
操作
浏览文件
下载
差异文件
merge from upstream
上级
015c7815
4840ba33
变更
8
隐藏空白更改
内联
并排
Showing
8 changed file
with
90 addition
and
22 deletion
+90
-22
docs/zh_CN/后端部署文档.md
docs/zh_CN/后端部署文档.md
+2
-2
escheduler-alert/src/main/java/cn/escheduler/alert/utils/MailUtils.java
...rt/src/main/java/cn/escheduler/alert/utils/MailUtils.java
+1
-0
escheduler-api/src/main/java/cn/escheduler/api/controller/ResourcesController.java
...ava/cn/escheduler/api/controller/ResourcesController.java
+2
-2
escheduler-api/src/main/java/cn/escheduler/api/enums/Status.java
...ler-api/src/main/java/cn/escheduler/api/enums/Status.java
+1
-0
escheduler-api/src/main/java/cn/escheduler/api/service/ResourcesService.java
...main/java/cn/escheduler/api/service/ResourcesService.java
+52
-0
escheduler-api/src/test/java/cn/escheduler/api/controller/ResourcesControllerTest.java
...cn/escheduler/api/controller/ResourcesControllerTest.java
+23
-1
escheduler-server/src/main/java/cn/escheduler/server/worker/task/sql/SqlTask.java
...in/java/cn/escheduler/server/worker/task/sql/SqlTask.java
+8
-16
sql/soft_version
sql/soft_version
+1
-1
未找到文件。
docs/zh_CN/后端部署文档.md
浏览文件 @
8f4bed90
...
...
@@ -4,7 +4,7 @@
## 1、准备工作
目前最新安装包版本是1.0.
3,下载地址:
[
码云下载
](
https://gitee.com/easyscheduler/EasyScheduler/attach_files/
)
,下载escheduler-backend-1.0.3.tar.gz(后端简称escheduler-backend),escheduler-ui-1.0.3
.tar.gz(前端简称escheduler-ui)
目前最新安装包版本是1.0.
4,下载地址:
[
码云下载
](
https://gitee.com/easyscheduler/EasyScheduler/attach_files/
)
,下载escheduler-backend-1.0.4.tar.gz(后端简称escheduler-backend),escheduler-ui-1.0.4
.tar.gz(前端简称escheduler-ui)
#### 准备一: 基础软件安装(必装项请自行安装)
...
...
@@ -149,7 +149,7 @@ install.sh : 一键部署脚本
### 2.2 编译源码来部署
将源码包release版本1.0.
3
下载后,解压进入根目录
将源码包release版本1.0.
4
下载后,解压进入根目录
*
执行编译命令:
...
...
escheduler-alert/src/main/java/cn/escheduler/alert/utils/MailUtils.java
浏览文件 @
8f4bed90
...
...
@@ -160,6 +160,7 @@ public class MailUtils {
return
retMap
;
}
catch
(
Exception
e
){
handleException
(
receivers
,
retMap
,
e
);
return
retMap
;
}
}
return
retMap
;
...
...
escheduler-api/src/main/java/cn/escheduler/api/controller/ResourcesController.java
浏览文件 @
8f4bed90
...
...
@@ -195,9 +195,9 @@ public class ResourcesController extends BaseController{
)
{
try
{
logger
.
info
(
"login user {}, verfiy resource alias: {},resource type: {}"
,
loginUser
.
getUserName
(),
alias
);
loginUser
.
getUserName
(),
alias
,
type
);
return
resourceService
.
verifyResourceName
(
alias
,
type
);
return
resourceService
.
verifyResourceName
(
alias
,
type
,
loginUser
);
}
catch
(
Exception
e
)
{
logger
.
error
(
VERIFY_RESOURCE_BY_NAME_AND_TYPE_ERROR
.
getMsg
(),
e
);
return
error
(
Status
.
VERIFY_RESOURCE_BY_NAME_AND_TYPE_ERROR
.
getCode
(),
Status
.
VERIFY_RESOURCE_BY_NAME_AND_TYPE_ERROR
.
getMsg
());
...
...
escheduler-api/src/main/java/cn/escheduler/api/enums/Status.java
浏览文件 @
8f4bed90
...
...
@@ -174,6 +174,7 @@ public enum Status {
RESOURCE_SUFFIX_FORBID_CHANGE
(
20008
,
"resource suffix not allowed to be modified"
),
UDF_RESOURCE_SUFFIX_NOT_JAR
(
20009
,
"UDF resource suffix name must be jar"
),
HDFS_COPY_FAIL
(
20009
,
"hdfs copy {0} -> {1} fail"
),
RESOURCE_FILE_EXIST
(
20010
,
"resource file {0} already exists in hdfs,please delete it or change name!"
),
...
...
escheduler-api/src/main/java/cn/escheduler/api/service/ResourcesService.java
浏览文件 @
8f4bed90
...
...
@@ -420,6 +420,41 @@ public class ResourcesService extends BaseService {
return
result
;
}
/**
* verify resource by name and type
* @param name
* @param type
* @param loginUser
* @return
*/
public
Result
verifyResourceName
(
String
name
,
ResourceType
type
,
User
loginUser
)
{
Result
result
=
new
Result
();
putMsg
(
result
,
Status
.
SUCCESS
);
Resource
resource
=
resourcesMapper
.
queryResourceByNameAndType
(
name
,
type
.
ordinal
());
if
(
resource
!=
null
)
{
logger
.
error
(
"resource type:{} name:{} has exist, can't create again."
,
type
,
name
);
putMsg
(
result
,
Status
.
RESOURCE_EXIST
);
}
else
{
// query tenant
String
tenantCode
=
tenantMapper
.
queryById
(
loginUser
.
getTenantId
()).
getTenantCode
();
try
{
String
hdfsFilename
=
getHdfsFileName
(
type
,
tenantCode
,
name
);
if
(
HadoopUtils
.
getInstance
().
exists
(
hdfsFilename
)){
logger
.
error
(
"resource type:{} name:{} has exist in hdfs {}, can't create again."
,
type
,
name
,
hdfsFilename
);
putMsg
(
result
,
Status
.
RESOURCE_FILE_EXIST
,
hdfsFilename
);
}
}
catch
(
Exception
e
)
{
logger
.
error
(
e
.
getMessage
(),
e
);
putMsg
(
result
,
Status
.
HDFS_OPERATION_ERROR
);
}
}
return
result
;
}
/**
* verify resource by name and type
*
...
...
@@ -815,6 +850,23 @@ public class ResourcesService extends BaseService {
return
hdfsFileName
;
}
/**
* get hdfs file name
*
* @param resourceType
* @param tenantCode
* @param hdfsFileName
* @return
*/
private
String
getHdfsFileName
(
ResourceType
resourceType
,
String
tenantCode
,
String
hdfsFileName
)
{
if
(
resourceType
.
equals
(
ResourceType
.
FILE
))
{
hdfsFileName
=
HadoopUtils
.
getHdfsFilename
(
tenantCode
,
hdfsFileName
);
}
else
if
(
resourceType
.
equals
(
ResourceType
.
UDF
))
{
hdfsFileName
=
HadoopUtils
.
getHdfsUdfFilename
(
tenantCode
,
hdfsFileName
);
}
return
hdfsFileName
;
}
/**
* get authorized resource list
*
...
...
escheduler-api/src/test/java/cn/escheduler/api/controller/ResourcesControllerTest.java
浏览文件 @
8f4bed90
...
...
@@ -34,6 +34,8 @@ import org.springframework.test.context.junit4.SpringRunner;
import
org.springframework.test.web.servlet.MockMvc
;
import
org.springframework.test.web.servlet.MvcResult
;
import
org.springframework.test.web.servlet.setup.MockMvcBuilders
;
import
org.springframework.util.LinkedMultiValueMap
;
import
org.springframework.util.MultiValueMap
;
import
org.springframework.web.context.WebApplicationContext
;
import
static
org
.
springframework
.
test
.
web
.
servlet
.
request
.
MockMvcRequestBuilders
.
get
;
...
...
@@ -43,7 +45,7 @@ import static org.springframework.test.web.servlet.result.MockMvcResultMatchers.
@RunWith
(
SpringRunner
.
class
)
@SpringBootTest
public
class
ResourcesControllerTest
{
private
static
Logger
logger
=
LoggerFactory
.
getLogger
(
Queue
ControllerTest
.
class
);
private
static
Logger
logger
=
LoggerFactory
.
getLogger
(
Resources
ControllerTest
.
class
);
private
MockMvc
mockMvc
;
...
...
@@ -71,4 +73,24 @@ public class ResourcesControllerTest {
Assert
.
assertEquals
(
Status
.
SUCCESS
.
getCode
(),
result
.
getCode
().
intValue
());
logger
.
info
(
mvcResult
.
getResponse
().
getContentAsString
());
}
@Test
public
void
verifyResourceName
()
throws
Exception
{
MultiValueMap
<
String
,
String
>
paramsMap
=
new
LinkedMultiValueMap
<>();
paramsMap
.
add
(
"name"
,
"list_resources_1.sh"
);
paramsMap
.
add
(
"type"
,
"FILE"
);
MvcResult
mvcResult
=
mockMvc
.
perform
(
get
(
"/resources/verify-name"
)
.
header
(
"sessionId"
,
"c24ed9d9-1c20-48a0-bd9c-5cfca14a4dcb"
)
.
params
(
paramsMap
))
.
andExpect
(
status
().
isOk
())
.
andExpect
(
content
().
contentType
(
MediaType
.
APPLICATION_JSON_UTF8
))
.
andReturn
();
Result
result
=
JSONUtils
.
parseObject
(
mvcResult
.
getResponse
().
getContentAsString
(),
Result
.
class
);
Assert
.
assertEquals
(
Status
.
SUCCESS
.
getCode
(),
result
.
getCode
().
intValue
());
logger
.
info
(
mvcResult
.
getResponse
().
getContentAsString
());
}
}
\ No newline at end of file
escheduler-server/src/main/java/cn/escheduler/server/worker/task/sql/SqlTask.java
浏览文件 @
8f4bed90
...
...
@@ -196,7 +196,7 @@ public class SqlTask extends AbstractTask {
}
// special characters need to be escaped, ${} needs to be escaped
String
rgex
=
"
'?\\$\\{(.*?)\\}'?
"
;
String
rgex
=
"
['\"]*\\$\\{(.*?)\\}['\"]*
"
;
setSqlParamsMap
(
sql
,
rgex
,
sqlParamsMap
,
paramsMap
);
// replace the ${} of the SQL statement with the Placeholder
...
...
@@ -310,6 +310,7 @@ public class SqlTask extends AbstractTask {
}
}
catch
(
Exception
e
)
{
logger
.
error
(
e
.
getMessage
(),
e
);
throw
new
RuntimeException
(
e
.
getMessage
());
}
return
connection
;
}
...
...
@@ -326,6 +327,7 @@ public class SqlTask extends AbstractTask {
ParameterUtils
.
setInParameter
(
key
,
stmt
,
prop
.
getType
(),
prop
.
getValue
());
}
}
logger
.
info
(
"prepare statement replace sql:{}"
,
stmt
.
toString
());
return
stmt
;
}
...
...
@@ -371,9 +373,13 @@ public class SqlTask extends AbstractTask {
String
showTypeName
=
sqlParameters
.
getShowType
().
replace
(
Constants
.
COMMA
,
""
).
trim
();
if
(
EnumUtils
.
isValidEnum
(
ShowType
.
class
,
showTypeName
)){
MailUtils
.
sendMails
(
receviersList
,
receviersCcList
,
title
,
content
,
ShowType
.
valueOf
(
showTypeName
));
Map
<
String
,
Object
>
mailResult
=
MailUtils
.
sendMails
(
receviersList
,
receviersCcList
,
title
,
content
,
ShowType
.
valueOf
(
showTypeName
));
if
(!(
Boolean
)
mailResult
.
get
(
cn
.
escheduler
.
api
.
utils
.
Constants
.
STATUS
)){
throw
new
RuntimeException
(
"send mail failed!"
);
}
}
else
{
logger
.
error
(
"showType: {} is not valid "
,
showTypeName
);
throw
new
RuntimeException
(
String
.
format
(
"showType: %s is not valid "
,
showTypeName
));
}
}
...
...
@@ -411,19 +417,5 @@ public class SqlTask extends AbstractTask {
logPrint
.
append
(
sqlParamsMap
.
get
(
i
).
getValue
()+
"("
+
sqlParamsMap
.
get
(
i
).
getType
()+
")"
);
}
logger
.
info
(
logPrint
.
toString
());
//direct print style
Pattern
pattern
=
Pattern
.
compile
(
rgex
);
Matcher
m
=
pattern
.
matcher
(
content
);
int
index
=
1
;
StringBuffer
sb
=
new
StringBuffer
(
"replaced sql , direct:"
);
while
(
m
.
find
())
{
m
.
appendReplacement
(
sb
,
sqlParamsMap
.
get
(
index
).
getValue
());
index
++;
}
m
.
appendTail
(
sb
);
logger
.
info
(
sb
.
toString
());
}
}
sql/soft_version
浏览文件 @
8f4bed90
1.0.2
\ No newline at end of file
1.0.4
\ No newline at end of file
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录