Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
梦中观雨
cat
提交
059bce26
C
cat
项目概览
梦中观雨
/
cat
与 Fork 源项目一致
从无法访问的项目Fork
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
C
cat
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
059bce26
编写于
12月 28, 2014
作者:
Y
youyong205
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
refactor the cleaner report job task
上级
78253256
变更
2
显示空白变更内容
内联
并排
Showing
2 changed file
with
3 addition
and
50 deletion
+3
-50
cat-consumer/src/main/resources/META-INF/plexus/components.xml
...onsumer/src/main/resources/META-INF/plexus/components.xml
+3
-0
cat-hadoop/src/main/java/com/dianping/cat/hadoop/hdfs/UploaderAndCleaner.java
...java/com/dianping/cat/hadoop/hdfs/UploaderAndCleaner.java
+0
-50
未找到文件。
cat-consumer/src/main/resources/META-INF/plexus/components.xml
浏览文件 @
059bce26
...
@@ -379,6 +379,9 @@
...
@@ -379,6 +379,9 @@
<requirement>
<requirement>
<role>
com.dianping.cat.task.TaskManager
</role>
<role>
com.dianping.cat.task.TaskManager
</role>
</requirement>
</requirement>
<requirement>
<role>
com.dianping.cat.storage.report.ReportBucketManager
</role>
</requirement>
</requirements>
</requirements>
</component>
</component>
<component>
<component>
...
...
cat-hadoop/src/main/java/com/dianping/cat/hadoop/hdfs/UploaderAndCleaner.java
浏览文件 @
059bce26
...
@@ -3,11 +3,9 @@ package com.dianping.cat.hadoop.hdfs;
...
@@ -3,11 +3,9 @@ package com.dianping.cat.hadoop.hdfs;
import
java.io.File
;
import
java.io.File
;
import
java.io.FileInputStream
;
import
java.io.FileInputStream
;
import
java.io.IOException
;
import
java.io.IOException
;
import
java.text.SimpleDateFormat
;
import
java.util.ArrayList
;
import
java.util.ArrayList
;
import
java.util.Calendar
;
import
java.util.Calendar
;
import
java.util.Collections
;
import
java.util.Collections
;
import
java.util.Date
;
import
java.util.List
;
import
java.util.List
;
import
org.apache.hadoop.fs.FSDataOutputStream
;
import
org.apache.hadoop.fs.FSDataOutputStream
;
...
@@ -31,7 +29,6 @@ import org.unidal.lookup.annotation.Inject;
...
@@ -31,7 +29,6 @@ import org.unidal.lookup.annotation.Inject;
import
com.dianping.cat.Cat
;
import
com.dianping.cat.Cat
;
import
com.dianping.cat.configuration.NetworkInterfaceManager
;
import
com.dianping.cat.configuration.NetworkInterfaceManager
;
import
com.dianping.cat.configuration.ServerConfigManager
;
import
com.dianping.cat.configuration.ServerConfigManager
;
import
com.dianping.cat.message.Event
;
import
com.dianping.cat.message.Message
;
import
com.dianping.cat.message.Message
;
import
com.dianping.cat.message.Transaction
;
import
com.dianping.cat.message.Transaction
;
...
@@ -44,57 +41,12 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
...
@@ -44,57 +41,12 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
private
String
m_dumpBaseDir
;
private
String
m_dumpBaseDir
;
private
String
m_reportBaseDir
;
private
Logger
m_logger
;
private
Logger
m_logger
;
private
long
m_sleepPeriod
=
1000L
*
60
;
private
long
m_sleepPeriod
=
1000L
*
60
;
private
volatile
boolean
m_active
=
true
;
private
volatile
boolean
m_active
=
true
;
public
void
deleteOldReports
()
{
Transaction
t
=
Cat
.
newTransaction
(
"System"
,
"DeleteReport"
);
try
{
File
reportDir
=
new
File
(
m_reportBaseDir
);
final
List
<
String
>
toRemovePaths
=
new
ArrayList
<
String
>();
Date
date
=
new
Date
();
SimpleDateFormat
sdf
=
new
SimpleDateFormat
(
"yyyyMMdd"
);
final
String
today
=
sdf
.
format
(
date
);
final
String
yesterday
=
sdf
.
format
(
new
Date
(
date
.
getTime
()
-
24
*
60
*
60
*
1000L
));
Scanners
.
forDir
().
scan
(
reportDir
,
new
FileMatcher
()
{
@Override
public
Direction
matches
(
File
base
,
String
path
)
{
File
file
=
new
File
(
base
,
path
);
if
(
file
.
isFile
()
&&
shouldDeleteReport
(
path
))
{
toRemovePaths
.
add
(
path
);
}
return
Direction
.
DOWN
;
}
private
boolean
shouldDeleteReport
(
String
path
)
{
if
(
path
.
indexOf
(
today
)
>
-
1
||
path
.
indexOf
(
yesterday
)
>
-
1
)
{
return
false
;
}
else
{
return
true
;
}
}
});
for
(
String
path
:
toRemovePaths
)
{
File
file
=
new
File
(
m_reportBaseDir
,
path
);
file
.
delete
();
Cat
.
logEvent
(
"System"
,
"DeleteReport"
,
Event
.
SUCCESS
,
file
.
getAbsolutePath
());
}
removeEmptyDir
(
reportDir
);
t
.
setStatus
(
Transaction
.
SUCCESS
);
}
catch
(
Exception
e
)
{
Cat
.
logError
(
e
);
t
.
setStatus
(
e
);
}
finally
{
t
.
complete
();
}
}
@Override
@Override
public
void
enableLogging
(
Logger
logger
)
{
public
void
enableLogging
(
Logger
logger
)
{
...
@@ -109,7 +61,6 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
...
@@ -109,7 +61,6 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
@Override
@Override
public
void
initialize
()
throws
InitializationException
{
public
void
initialize
()
throws
InitializationException
{
m_dumpBaseDir
=
m_configManager
.
getHdfsLocalBaseDir
(
"dump"
);
m_dumpBaseDir
=
m_configManager
.
getHdfsLocalBaseDir
(
"dump"
);
m_reportBaseDir
=
m_configManager
.
getHdfsLocalBaseDir
(
"report"
);
}
}
private
boolean
isActive
()
{
private
boolean
isActive
()
{
...
@@ -173,7 +124,6 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
...
@@ -173,7 +124,6 @@ public class UploaderAndCleaner implements Initializable, Task, LogEnabled {
if
(
cal
.
get
(
Calendar
.
MINUTE
)
>=
10
)
{
if
(
cal
.
get
(
Calendar
.
MINUTE
)
>=
10
)
{
uploadLogviewFiles
();
uploadLogviewFiles
();
deleteOldReports
();
}
}
}
}
}
catch
(
Exception
e
)
{
}
catch
(
Exception
e
)
{
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录