Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
DolphinScheduler
提交
f3dfaacc
DolphinScheduler
项目概览
apache
/
DolphinScheduler
上一次同步 接近 2 年
通知
707
Star
9572
Fork
3514
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
DolphinScheduler
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
f3dfaacc
编写于
4月 15, 2021
作者:
W
wenjun
提交者:
GitHub
4月 15, 2021
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
[Improvement][Service] Optimize query log (#5205)
* [Improvement][Service] Optimize query log * remove IPUtils
上级
b6453da2
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
12 addition
and
274 deletion
+12
-274
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/IpUtils.java
...ava/org/apache/dolphinscheduler/common/utils/IpUtils.java
+0
-63
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/IpUtilsTest.java
...org/apache/dolphinscheduler/common/utils/IpUtilsTest.java
+0
-55
dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/Constants.java
...a/org/apache/dolphinscheduler/remote/utils/Constants.java
+3
-1
dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/IPUtils.java
...ava/org/apache/dolphinscheduler/remote/utils/IPUtils.java
+0
-145
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/log/LogClientService.java
...apache/dolphinscheduler/service/log/LogClientService.java
+2
-2
dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/log/LogClientServiceTest.java
...he/dolphinscheduler/service/log/LogClientServiceTest.java
+7
-7
pom.xml
pom.xml
+0
-1
未找到文件。
dolphinscheduler-common/src/main/java/org/apache/dolphinscheduler/common/utils/IpUtils.java
已删除
100644 → 0
浏览文件 @
b6453da2
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.dolphinscheduler.common.utils
;
/**
* http utils
*/
public
class
IpUtils
{
private
IpUtils
()
{
throw
new
UnsupportedOperationException
(
"Construct IpUtils"
);
}
public
static
final
String
DOT
=
"."
;
/**
* ip str to long <p>
*
* @param ipStr ip string
* @return ip to long
*/
public
static
Long
ipToLong
(
String
ipStr
)
{
String
[]
ipSet
=
ipStr
.
split
(
"\\"
+
DOT
);
return
Long
.
parseLong
(
ipSet
[
0
])
<<
24
|
Long
.
parseLong
(
ipSet
[
1
])
<<
16
|
Long
.
parseLong
(
ipSet
[
2
])
<<
8
|
Long
.
parseLong
(
ipSet
[
3
]);
}
/**
* long to ip
*
* @param ipLong the long number converted from IP
* @return String
*/
public
static
String
longToIp
(
long
ipLong
)
{
long
[]
ipNumbers
=
new
long
[
4
];
long
tmp
=
0xFF
;
ipNumbers
[
0
]
=
ipLong
>>
24
&
tmp
;
ipNumbers
[
1
]
=
ipLong
>>
16
&
tmp
;
ipNumbers
[
2
]
=
ipLong
>>
8
&
tmp
;
ipNumbers
[
3
]
=
ipLong
&
tmp
;
return
ipNumbers
[
0
]
+
DOT
+
ipNumbers
[
1
]
+
DOT
+
ipNumbers
[
2
]
+
DOT
+
ipNumbers
[
3
];
}
}
dolphinscheduler-common/src/test/java/org/apache/dolphinscheduler/common/utils/IpUtilsTest.java
已删除
100644 → 0
浏览文件 @
b6453da2
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.dolphinscheduler.common.utils
;
import
org.junit.Assert
;
import
org.junit.Test
;
public
class
IpUtilsTest
{
@Test
public
void
ipToLong
()
{
String
ip
=
"192.168.110.1"
;
String
ip2
=
"0.0.0.0"
;
long
longNumber
=
IpUtils
.
ipToLong
(
ip
);
long
longNumber2
=
IpUtils
.
ipToLong
(
ip2
);
System
.
out
.
println
(
longNumber
);
Assert
.
assertEquals
(
3232263681L
,
longNumber
);
Assert
.
assertEquals
(
0L
,
longNumber2
);
String
ip3
=
"255.255.255.255"
;
long
longNumber3
=
IpUtils
.
ipToLong
(
ip3
);
System
.
out
.
println
(
longNumber3
);
Assert
.
assertEquals
(
4294967295L
,
longNumber3
);
}
@Test
public
void
longToIp
()
{
String
ip
=
"192.168.110.1"
;
String
ip2
=
"0.0.0.0"
;
long
longNum
=
3232263681L
;
String
i1
=
IpUtils
.
longToIp
(
longNum
);
String
i2
=
IpUtils
.
longToIp
(
0
);
Assert
.
assertEquals
(
ip
,
i1
);
Assert
.
assertEquals
(
ip2
,
i2
);
}
}
\ No newline at end of file
dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/Constants.java
浏览文件 @
f3dfaacc
...
...
@@ -17,6 +17,8 @@
package
org.apache.dolphinscheduler.remote.utils
;
import
org.apache.dolphinscheduler.common.utils.NetUtils
;
import
java.nio.charset.Charset
;
import
java.nio.charset.StandardCharsets
;
...
...
@@ -48,7 +50,7 @@ public class Constants {
public
static
final
int
CPUS
=
Runtime
.
getRuntime
().
availableProcessors
();
public
static
final
String
LOCAL_ADDRESS
=
IPUtils
.
getFirstNoLoopbackIP4Address
();
public
static
final
String
LOCAL_ADDRESS
=
NetUtils
.
getHost
();
/**
* netty epoll enable switch
...
...
dolphinscheduler-remote/src/main/java/org/apache/dolphinscheduler/remote/utils/IPUtils.java
已删除
100644 → 0
浏览文件 @
b6453da2
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.dolphinscheduler.remote.utils
;
import
org.apache.dolphinscheduler.remote.exceptions.RemoteException
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
java.net.*
;
import
java.util.ArrayList
;
import
java.util.Collection
;
import
java.util.Enumeration
;
import
java.util.regex.Matcher
;
import
java.util.regex.Pattern
;
public
class
IPUtils
{
private
IPUtils
()
{
throw
new
IllegalStateException
(
IPUtils
.
class
.
getName
());
}
private
static
final
Logger
logger
=
LoggerFactory
.
getLogger
(
IPUtils
.
class
);
private
static
String
localHost
=
"unknown"
;
static
{
String
host
=
System
.
getenv
(
"HOSTNAME"
);
if
(
isNotEmpty
(
host
))
{
localHost
=
host
;
}
else
{
try
{
String
hostName
=
InetAddress
.
getLocalHost
().
getHostName
();
if
(
isNotEmpty
(
hostName
))
{
localHost
=
hostName
;
}
}
catch
(
UnknownHostException
e
)
{
logger
.
error
(
"get hostName error!"
,
e
);
}
}
}
public
static
String
getLocalHost
()
{
return
localHost
;
}
public
static
String
getFirstNoLoopbackIP4Address
()
{
Collection
<
String
>
allNoLoopbackIP4Addresses
=
getNoLoopbackIP4Addresses
();
if
(
allNoLoopbackIP4Addresses
.
isEmpty
())
{
return
null
;
}
return
allNoLoopbackIP4Addresses
.
iterator
().
next
();
}
public
static
Collection
<
String
>
getNoLoopbackIP4Addresses
()
{
Collection
<
String
>
noLoopbackIP4Addresses
=
new
ArrayList
<>();
Collection
<
InetAddress
>
allInetAddresses
=
getAllHostAddress
();
for
(
InetAddress
address
:
allInetAddresses
)
{
if
(!
address
.
isLoopbackAddress
()
&&
!
address
.
isSiteLocalAddress
()
&&
!
Inet6Address
.
class
.
isInstance
(
address
))
{
noLoopbackIP4Addresses
.
add
(
address
.
getHostAddress
());
}
}
if
(
noLoopbackIP4Addresses
.
isEmpty
())
{
for
(
InetAddress
address
:
allInetAddresses
)
{
if
(!
address
.
isLoopbackAddress
()
&&
!
Inet6Address
.
class
.
isInstance
(
address
))
{
noLoopbackIP4Addresses
.
add
(
address
.
getHostAddress
());
}
}
}
return
noLoopbackIP4Addresses
;
}
public
static
Collection
<
InetAddress
>
getAllHostAddress
()
{
try
{
Enumeration
<
NetworkInterface
>
networkInterfaces
=
NetworkInterface
.
getNetworkInterfaces
();
Collection
<
InetAddress
>
addresses
=
new
ArrayList
<>();
while
(
networkInterfaces
.
hasMoreElements
())
{
NetworkInterface
networkInterface
=
networkInterfaces
.
nextElement
();
Enumeration
<
InetAddress
>
inetAddresses
=
networkInterface
.
getInetAddresses
();
while
(
inetAddresses
.
hasMoreElements
())
{
InetAddress
inetAddress
=
inetAddresses
.
nextElement
();
addresses
.
add
(
inetAddress
);
}
}
return
addresses
;
}
catch
(
SocketException
e
)
{
throw
new
RemoteException
(
e
.
getMessage
(),
e
);
}
}
public
static
String
getIpByHostName
(
String
host
)
{
InetAddress
address
=
null
;
try
{
address
=
InetAddress
.
getByName
(
host
);
}
catch
(
UnknownHostException
e
)
{
logger
.
error
(
"get IP error"
,
e
);
}
if
(
address
==
null
)
{
return
""
;
}
return
address
.
getHostAddress
();
}
private
static
boolean
isEmpty
(
final
CharSequence
cs
)
{
return
cs
==
null
||
cs
.
length
()
==
0
;
}
private
static
boolean
isNotEmpty
(
final
CharSequence
cs
)
{
return
!
isEmpty
(
cs
);
}
public
static
boolean
isIp
(
String
addr
)
{
if
(
addr
.
length
()
<
7
||
addr
.
length
()
>
15
||
""
.
equals
(
addr
))
{
return
false
;
}
String
ipRegex
=
"([1-9]|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])(\\.(\\d|[1-9]\\d|1\\d{2}|2[0-4]\\d|25[0-5])){3}"
;
Pattern
pat
=
Pattern
.
compile
(
ipRegex
);
Matcher
mat
=
pat
.
matcher
(
addr
);
return
mat
.
find
();
}
}
dolphinscheduler-service/src/main/java/org/apache/dolphinscheduler/service/log/LogClientService.java
浏览文件 @
f3dfaacc
...
...
@@ -19,6 +19,7 @@ package org.apache.dolphinscheduler.service.log;
import
org.apache.dolphinscheduler.common.utils.JSONUtils
;
import
org.apache.dolphinscheduler.common.utils.LoggerUtils
;
import
org.apache.dolphinscheduler.common.utils.NetUtils
;
import
org.apache.dolphinscheduler.remote.NettyRemotingClient
;
import
org.apache.dolphinscheduler.remote.command.Command
;
import
org.apache.dolphinscheduler.remote.command.log.GetLogBytesRequestCommand
;
...
...
@@ -31,7 +32,6 @@ import org.apache.dolphinscheduler.remote.command.log.ViewLogRequestCommand;
import
org.apache.dolphinscheduler.remote.command.log.ViewLogResponseCommand
;
import
org.apache.dolphinscheduler.remote.config.NettyClientConfig
;
import
org.apache.dolphinscheduler.remote.utils.Host
;
import
org.apache.dolphinscheduler.remote.utils.IPUtils
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
...
...
@@ -118,7 +118,7 @@ public class LogClientService {
String
result
=
""
;
final
Host
address
=
new
Host
(
host
,
port
);
try
{
if
(
IPUtils
.
getLocal
Host
().
equals
(
host
))
{
if
(
NetUtils
.
get
Host
().
equals
(
host
))
{
result
=
LoggerUtils
.
readWholeFileContent
(
request
.
getPath
());
}
else
{
Command
command
=
request
.
convert2Command
();
...
...
dolphinscheduler-service/src/test/java/org/apache/dolphinscheduler/service/log/LogClientServiceTest.java
浏览文件 @
f3dfaacc
...
...
@@ -19,6 +19,7 @@ package org.apache.dolphinscheduler.service.log;
import
org.apache.dolphinscheduler.common.utils.JSONUtils
;
import
org.apache.dolphinscheduler.common.utils.LoggerUtils
;
import
org.apache.dolphinscheduler.common.utils.NetUtils
;
import
org.apache.dolphinscheduler.remote.NettyRemotingClient
;
import
org.apache.dolphinscheduler.remote.command.Command
;
import
org.apache.dolphinscheduler.remote.command.log.GetLogBytesResponseCommand
;
...
...
@@ -26,7 +27,6 @@ import org.apache.dolphinscheduler.remote.command.log.RemoveTaskLogResponseComma
import
org.apache.dolphinscheduler.remote.command.log.RollViewLogResponseCommand
;
import
org.apache.dolphinscheduler.remote.command.log.ViewLogResponseCommand
;
import
org.apache.dolphinscheduler.remote.utils.Host
;
import
org.apache.dolphinscheduler.remote.utils.IPUtils
;
import
java.nio.charset.StandardCharsets
;
...
...
@@ -40,7 +40,7 @@ import org.powermock.core.classloader.annotations.PrepareForTest;
import
org.powermock.modules.junit4.PowerMockRunner
;
@RunWith
(
PowerMockRunner
.
class
)
@PrepareForTest
({
LogClientService
.
class
,
IP
Utils
.
class
,
LoggerUtils
.
class
,
NettyRemotingClient
.
class
})
@PrepareForTest
({
LogClientService
.
class
,
Net
Utils
.
class
,
LoggerUtils
.
class
,
NettyRemotingClient
.
class
})
public
class
LogClientServiceTest
{
@Test
...
...
@@ -49,8 +49,8 @@ public class LogClientServiceTest {
int
port
=
1234
;
String
path
=
"/tmp/log"
;
PowerMockito
.
mockStatic
(
IP
Utils
.
class
);
PowerMockito
.
when
(
IPUtils
.
getLocal
Host
()).
thenReturn
(
localMachine
);
PowerMockito
.
mockStatic
(
Net
Utils
.
class
);
PowerMockito
.
when
(
NetUtils
.
get
Host
()).
thenReturn
(
localMachine
);
PowerMockito
.
mockStatic
(
LoggerUtils
.
class
);
PowerMockito
.
when
(
LoggerUtils
.
readWholeFileContent
(
Mockito
.
anyString
())).
thenReturn
(
"application_xx_11"
);
...
...
@@ -61,12 +61,12 @@ public class LogClientServiceTest {
@Test
public
void
testViewLogFromRemote
()
throws
Exception
{
String
localMachine
=
"
LOCAL_MACHINE
"
;
String
localMachine
=
"
127.0.0.1
"
;
int
port
=
1234
;
String
path
=
"/tmp/log"
;
PowerMockito
.
mockStatic
(
IP
Utils
.
class
);
PowerMockito
.
when
(
IPUtils
.
getLocal
Host
()).
thenReturn
(
localMachine
+
"1"
);
PowerMockito
.
mockStatic
(
Net
Utils
.
class
);
PowerMockito
.
when
(
NetUtils
.
get
Host
()).
thenReturn
(
localMachine
+
"1"
);
NettyRemotingClient
remotingClient
=
PowerMockito
.
mock
(
NettyRemotingClient
.
class
);
PowerMockito
.
whenNew
(
NettyRemotingClient
.
class
).
withAnyArguments
().
thenReturn
(
remotingClient
);
...
...
pom.xml
浏览文件 @
f3dfaacc
...
...
@@ -864,7 +864,6 @@
<include>
**/common/utils/DependentUtilsTest.java
</include>
<include>
**/common/utils/EncryptionUtilsTest.java
</include>
<include>
**/common/utils/FileUtilsTest.java
</include>
<include>
**/common/utils/IpUtilsTest.java
</include>
<include>
**/common/utils/JSONUtilsTest.java
</include>
<include>
**/common/utils/LoggerUtilsTest.java
</include>
<include>
**/common/utils/NetUtilsTest.java
</include>
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录