Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
Iotdb
提交
3360cb00
I
Iotdb
项目概览
apache
/
Iotdb
大约 1 年 前同步成功
通知
25
Star
3344
Fork
916
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
DevOps
流水线
流水线任务
计划
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
I
Iotdb
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
DevOps
DevOps
流水线
流水线任务
计划
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
流水线任务
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
3360cb00
编写于
1月 26, 2019
作者:
L
lta
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
fix sonar of postback module
上级
9c073354
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
67 addition
and
85 deletion
+67
-85
iotdb/src/main/java/org/apache/iotdb/db/postback/conf/PostBackSenderDescriptor.java
...ache/iotdb/db/postback/conf/PostBackSenderDescriptor.java
+2
-3
iotdb/src/main/java/org/apache/iotdb/db/postback/receiver/ServerManager.java
.../org/apache/iotdb/db/postback/receiver/ServerManager.java
+1
-2
iotdb/src/main/java/org/apache/iotdb/db/postback/receiver/ServerServiceImpl.java
.../apache/iotdb/db/postback/receiver/ServerServiceImpl.java
+44
-51
iotdb/src/main/java/org/apache/iotdb/db/postback/sender/FileManager.java
...java/org/apache/iotdb/db/postback/sender/FileManager.java
+4
-6
iotdb/src/main/java/org/apache/iotdb/db/postback/sender/FileSenderImpl.java
...a/org/apache/iotdb/db/postback/sender/FileSenderImpl.java
+16
-23
未找到文件。
iotdb/src/main/java/org/apache/iotdb/db/postback/conf/PostBackSenderDescriptor.java
浏览文件 @
3360cb00
...
...
@@ -107,10 +107,9 @@ public class PostBackSenderDescriptor {
conf
.
setIotdbBufferwriteDirectory
(
iotdbBufferwriteDirectory
);
conf
.
setSnapshotPaths
(
snapshots
);
}
catch
(
IOException
e
)
{
LOGGER
.
warn
(
"Cannot load config file because {}, use default configuration"
,
e
.
getMessage
()
);
LOGGER
.
warn
(
"Cannot load config file because {}, use default configuration"
,
e
);
}
catch
(
Exception
e
)
{
LOGGER
.
warn
(
"Error format in config file because {}, use default configuration"
,
e
.
getMessage
());
LOGGER
.
warn
(
"Error format in config file because {}, use default configuration"
,
e
);
}
}
...
...
iotdb/src/main/java/org/apache/iotdb/db/postback/receiver/ServerManager.java
浏览文件 @
3360cb00
...
...
@@ -79,8 +79,7 @@ public class ServerManager {
Thread
thread
=
new
Thread
(
runnable
);
thread
.
start
();
}
catch
(
TTransportException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot start postback server because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: cannot start postback server because {}"
,
e
);
}
}
...
...
iotdb/src/main/java/org/apache/iotdb/db/postback/receiver/ServerServiceImpl.java
浏览文件 @
3360cb00
...
...
@@ -91,8 +91,9 @@ public class ServerServiceImpl implements ServerService.Iface {
*/
@Override
public
void
init
(
String
storageGroup
)
{
LOGGER
.
info
(
"IoTDB post back receiver: postback process starts to receive data of storage "
+
"group {}."
,
storageGroup
);
LOGGER
.
info
(
String
.
format
(
"IoTDB post back receiver: postback process starts to receive data of storage group {}."
),
storageGroup
);
fileNum
.
set
(
0
);
fileNodeMap
.
set
(
new
HashMap
<>());
fileNodeStartTime
.
set
(
new
HashMap
<>());
...
...
@@ -141,35 +142,35 @@ public class ServerServiceImpl implements ServerService.Iface {
public
String
startReceiving
(
String
md5OfSender
,
List
<
String
>
filePathSplit
,
ByteBuffer
dataToReceive
,
int
status
)
throws
TException
{
String
md5OfReceiver
=
""
;
String
filePath
=
""
;
String
Builder
filePathBuilder
=
new
StringBuilder
()
;
FileChannel
channel
=
null
;
for
(
int
i
=
0
;
i
<
filePathSplit
.
size
();
i
++)
{
if
(
i
==
filePathSplit
.
size
()
-
1
)
{
filePath
=
new
StringBuilder
().
append
(
filePath
).
append
(
filePathSplit
.
get
(
i
)).
toString
(
);
filePath
Builder
=
filePathBuilder
.
append
(
filePathSplit
.
get
(
i
)
);
}
else
{
filePath
=
new
StringBuilder
().
append
(
filePath
).
append
(
filePathSplit
.
get
(
i
))
.
append
(
File
.
separator
).
toString
();
filePathBuilder
=
filePathBuilder
.
append
(
filePathSplit
.
get
(
i
)).
append
(
File
.
separator
);
}
}
String
filePath
=
filePathBuilder
.
toString
();
filePath
=
postbackPath
+
uuid
.
get
()
+
File
.
separator
+
filePath
;
if
(
status
==
1
)
{
// there are still data stream to add
File
file
=
new
File
(
filePath
);
if
(!
file
.
getParentFile
().
exists
())
{
try
{
file
.
getParentFile
().
mkdirs
();
if
(
file
.
createNewFile
())
{
if
(
!
file
.
createNewFile
())
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot create file {}"
,
file
.
getAbsoluteFile
());
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot make file
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back receiver: cannot make file
"
,
e
);
}
}
try
(
FileOutputStream
fos
=
new
FileOutputStream
(
file
,
true
))
{
// append new data
channel
=
fos
.
getChannel
();
channel
.
write
(
dataToReceive
);
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot write data to file
because {}"
,
e
.
getMessage
());
}
catch
(
IO
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot write data to file
"
,
e
);
}
}
else
{
// all data in the same file has received successfully
try
(
FileInputStream
fis
=
new
FileInputStream
(
filePath
))
{
...
...
@@ -183,14 +184,14 @@ public class ServerServiceImpl implements ServerService.Iface {
md5OfReceiver
=
(
new
BigInteger
(
1
,
md
.
digest
())).
toString
(
16
);
if
(
md5OfSender
.
equals
(
md5OfReceiver
))
{
fileNum
.
set
(
fileNum
.
get
()
+
1
);
LOGGER
.
info
(
"IoTDB post back receiver : Receiver has received "
+
fileNum
.
get
()
+
" "
+
"files from sender!"
);
LOGGER
.
info
(
String
.
format
(
"IoTDB post back receiver : Receiver has received %d files from sender!"
,
fileNum
.
get
())
);
}
else
{
Files
.
deleteIfExists
((
java
.
nio
.
file
.
Path
)
new
Path
(
filePath
)
);
new
File
(
filePath
).
delete
(
);
}
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot generate md5
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back receiver: cannot generate md5
"
,
e
);
}
}
return
md5OfReceiver
;
...
...
@@ -237,26 +238,23 @@ public class ServerServiceImpl implements ServerService.Iface {
}
}
}
catch
(
FileNotFoundException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot read the file {}
because {}
"
,
schemaFromSenderPath
.
get
(),
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back receiver: cannot read the file {}
.
"
,
schemaFromSenderPath
.
get
(),
e
);
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot insert schema to IoTDB "
+
"because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: cannot insert schema to IoTDB."
,
e
);
}
statement
.
executeBatch
();
statement
.
clearBatch
();
}
catch
(
SQLException
|
ClassNotFoundException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: jdbc can not connect to IoTDB because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: jdbc can not connect to IoTDB."
,
e
);
}
finally
{
try
{
if
(
statement
!=
null
)
{
statement
.
close
();
}
}
catch
(
SQLException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver : can not close JDBC connection "
+
"because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver : can not close JDBC connection."
,
e
);
}
}
}
else
{
...
...
@@ -265,12 +263,11 @@ public class ServerServiceImpl implements ServerService.Iface {
try
{
file
.
getParentFile
().
mkdirs
();
if
(!
file
.
createNewFile
())
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot create file
because
{}"
,
LOGGER
.
error
(
"IoTDB post back receiver: cannot create file {}"
,
file
.
getAbsoluteFile
());
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot make schema file because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: cannot make schema file."
,
e
);
}
}
try
{
...
...
@@ -280,8 +277,8 @@ public class ServerServiceImpl implements ServerService.Iface {
channel
.
close
();
fos
.
close
();
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: cannot write data to file
because {}
"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back receiver: cannot write data to file
.
"
,
e
);
}
}
...
...
@@ -353,16 +350,16 @@ public class ServerServiceImpl implements ServerService.Iface {
endTimeMap
.
put
(
key
,
device
.
getEndTime
());
}
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: unable to read tsfile {}
because {}
"
,
fileTF
.
getAbsolutePath
(),
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back receiver: unable to read tsfile {}"
,
fileTF
.
getAbsolutePath
(),
e
);
}
finally
{
try
{
if
(
reader
!=
null
)
{
reader
.
close
();
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {}
because {}
"
,
fileTF
.
getAbsolutePath
(),
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {}"
,
fileTF
.
getAbsolutePath
(),
e
);
}
}
fileNodeStartTime
.
get
().
put
(
fileTF
.
getAbsolutePath
(),
startTimeMap
);
...
...
@@ -437,7 +434,7 @@ public class ServerServiceImpl implements ServerService.Iface {
}
}
}
String
sql
=
sqlFront
+
sqlRear
+
")"
;
String
sql
=
String
.
format
(
"%s%s)"
,
sqlFront
,
sqlRear
)
;
statement
.
addBatch
(
sql
);
count
++;
...
...
@@ -451,23 +448,23 @@ public class ServerServiceImpl implements ServerService.Iface {
statement
.
executeBatch
();
statement
.
clearBatch
();
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB receiver can not parse tsfile into SQL
because{}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB receiver can not parse tsfile into SQL
"
,
e
);
}
catch
(
SQLException
|
ClassNotFoundException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: jdbc cannot connect to IoTDB because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: jdbc cannot connect to IoTDB"
,
e
);
}
finally
{
try
{
reader
.
close
();
if
(
reader
!=
null
)
{
reader
.
close
();
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {} because {}"
,
filePath
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {}"
,
filePath
,
e
);
}
try
{
if
(
statement
!=
null
)
{
statement
.
close
();
}
}
catch
(
SQLException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Can not close JDBC connection
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB receiver : Can not close JDBC connection
"
,
e
);
}
}
}
...
...
@@ -601,26 +598,23 @@ public class ServerServiceImpl implements ServerService.Iface {
statement
.
executeBatch
();
statement
.
clearBatch
();
}
catch
(
SQLException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: sql cannot execute successfully in IoTDB because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: sql cannot execute successfully in IoTDB"
,
e
);
}
catch
(
ClassNotFoundException
e
)
{
LOGGER
.
error
(
"IoTDB post back receiver: jdbc cannot connect to IoTDB because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back receiver: jdbc cannot connect to IoTDB"
,
e
);
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB receiver can not parse tsfile into SQL
because{}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB receiver can not parse tsfile into SQL
"
,
e
);
}
finally
{
try
{
reader
.
close
();
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {} because {}"
,
filePath
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB receiver : Cannot close file stream {}"
,
filePath
,
e
);
}
try
{
if
(
statement
!=
null
)
{
statement
.
close
();
}
}
catch
(
SQLException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Can not close JDBC connection
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB receiver : Can not close JDBC connection
"
,
e
);
}
}
}
...
...
@@ -687,8 +681,7 @@ public class ServerServiceImpl implements ServerService.Iface {
}
}
}
catch
(
FileNodeManagerException
e
)
{
LOGGER
.
error
(
"IoTDB receiver : Can not load external file because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB receiver : Can not load external file "
,
e
);
}
num
++;
...
...
iotdb/src/main/java/org/apache/iotdb/db/postback/sender/FileManager.java
浏览文件 @
3360cb00
...
...
@@ -104,8 +104,7 @@ public class FileManager {
LOGGER
.
error
(
"IoTDB post back sender: cannot create file {}"
,
file
.
getAbsoluteFile
());
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot get last local file list because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender: cannot get last local file list"
,
e
);
}
}
else
{
try
(
BufferedReader
bf
=
new
BufferedReader
(
new
FileReader
(
file
)))
{
...
...
@@ -115,9 +114,8 @@ public class FileManager {
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot get last local file list when reading file {} "
+
"because {}."
,
postbackConfig
.
getLastFileInfo
(),
e
.
getMessage
());
"IoTDB post back sender: cannot get last local file list when reading file {}."
,
postbackConfig
.
getLastFileInfo
(),
e
);
}
}
...
...
@@ -168,7 +166,7 @@ public class FileManager {
}
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot back up now local file info
because {}
"
,
e
);
LOGGER
.
error
(
"IoTDB post back sender: cannot back up now local file info"
,
e
);
}
}
...
...
iotdb/src/main/java/org/apache/iotdb/db/postback/sender/FileSenderImpl.java
浏览文件 @
3360cb00
...
...
@@ -128,8 +128,7 @@ public class FileSenderImpl implements FileSender {
try
{
transport
.
open
();
}
catch
(
TTransportException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot connect to server because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender: cannot connect to server"
,
e
);
connectionOrElse
=
false
;
}
}
...
...
@@ -152,16 +151,14 @@ public class FileSenderImpl implements FileSender {
uuid
=
"PB"
+
UUID
.
randomUUID
().
toString
().
replaceAll
(
"-"
,
""
);
out
.
write
(
uuid
.
getBytes
());
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot write UUID to file because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender: cannot write UUID to file"
,
e
);
connectionOrElse
=
false
;
}
}
else
{
try
(
BufferedReader
bf
=
new
BufferedReader
((
new
FileReader
(
uuidPath
))))
{
uuid
=
bf
.
readLine
();
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot read UUID from file because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender: cannot read UUID from file"
,
e
);
connectionOrElse
=
false
;
}
}
...
...
@@ -170,11 +167,10 @@ public class FileSenderImpl implements FileSender {
legalConnectionOrNot
=
clientOfServer
.
getUUID
(
uuid
,
InetAddress
.
getLocalHost
().
getHostAddress
());
}
catch
(
TException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot send UUID to receiver because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender: cannot send UUID to receiver"
,
e
);
connectionOrElse
=
false
;
}
catch
(
UnknownHostException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: unable to get local host
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back sender: unable to get local host
"
,
e
);
legalConnectionOrNot
=
false
;
}
return
legalConnectionOrNot
;
...
...
@@ -199,7 +195,7 @@ public class FileSenderImpl implements FileSender {
Files
.
createLink
(
link
,
target
);
}
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: can not make fileSnapshot
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back sender: can not make fileSnapshot
"
,
e
);
}
return
sendingSnapshotFileList
;
}
...
...
@@ -273,7 +269,7 @@ public class FileSenderImpl implements FileSender {
LOGGER
.
error
(
"IoTDB post back sender: cannot sending data because receiver has broken down."
);
connectionOrElse
=
false
;
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back sender: cannot sending data
because {}"
,
e
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back sender: cannot sending data
"
,
e
);
connectionOrElse
=
false
;
}
}
...
...
@@ -301,8 +297,7 @@ public class FileSenderImpl implements FileSender {
// 0 represents the schema file has been transferred completely.
clientOfServer
.
getSchema
(
null
,
0
);
}
catch
(
Exception
e
)
{
LOGGER
.
error
(
"IoTDB post back sender : cannot send schema from mlog.txt because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender : cannot send schema from mlog.txt "
,
e
);
connectionOrElse
=
false
;
}
}
...
...
@@ -342,16 +337,16 @@ public class FileSenderImpl implements FileSender {
while
(
true
)
{
try
{
listenerSocket
.
accept
();
}
catch
(
IOException
e
1
2
)
{
LOGGER
.
error
(
"IoTDB post back sender: unable to listen to port{}
, because {}
"
,
config
.
getClientPort
(),
e
12
.
getMessage
()
);
}
catch
(
IOException
e2
)
{
LOGGER
.
error
(
"IoTDB post back sender: unable to listen to port{}"
,
config
.
getClientPort
(),
e
2
);
}
}
});
listener
.
start
();
}
catch
(
IOException
e1
)
{
LOGGER
.
error
(
"IoTDB post back sender: unable to listen to port{}
, because {}
"
,
config
.
getClientPort
(),
e1
.
getMessage
()
);
LOGGER
.
error
(
"IoTDB post back sender: unable to listen to port{}"
,
config
.
getClientPort
(),
e1
);
}
}
}
...
...
@@ -407,7 +402,7 @@ public class FileSenderImpl implements FileSender {
try
{
PostbackUtils
.
deleteFile
(
new
File
(
snapshotPath
));
}
catch
(
IOException
e
)
{
LOGGER
.
error
(
"can not delete file {}
, {}"
,
snapshotPath
,
e
.
getMessage
()
);
LOGGER
.
error
(
"can not delete file {}
"
,
snapshotPath
,
e
);
}
}
}
...
...
@@ -457,8 +452,7 @@ public class FileSenderImpl implements FileSender {
clientOfServer
.
init
(
entry
.
getKey
());
}
catch
(
TException
e
)
{
connectionOrElse
=
false
;
LOGGER
.
error
(
"IoTDB post back sender : unable to connect to receiver because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender : unable to connect to receiver"
,
e
);
}
if
(!
connectionOrElse
)
{
transport
.
close
();
...
...
@@ -496,8 +490,7 @@ public class FileSenderImpl implements FileSender {
clientOfServer
.
afterReceiving
();
}
catch
(
TException
e
)
{
connectionOrElse
=
false
;
LOGGER
.
error
(
"IoTDB post back sender : unable to connect to receiver because {}"
,
e
.
getMessage
());
LOGGER
.
error
(
"IoTDB post back sender : unable to connect to receiver "
,
e
);
}
if
(!
connectionOrElse
)
{
transport
.
close
();
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录