Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
梦中观雨
cat
提交
6b72855e
C
cat
项目概览
梦中观雨
/
cat
与 Fork 源项目一致
从无法访问的项目Fork
通知
1
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
C
cat
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
6b72855e
编写于
3月 22, 2012
作者:
F
Frankie Wu
浏览文件
操作
浏览文件
下载
差异文件
Merge branch 'master' of
ssh://192.168.8.22:58422/cat
上级
cf1ddce0
d1377018
变更
4
隐藏空白更改
内联
并排
Showing
4 changed file
with
24 addition
and
50 deletion
+24
-50
cat-job/src/main/java/com/dianping/cat/job/hdfs/DefaultInputChannel.java
...n/java/com/dianping/cat/job/hdfs/DefaultInputChannel.java
+11
-0
cat-job/src/main/java/com/dianping/cat/job/hdfs/DefaultInputChannelManager.java
...com/dianping/cat/job/hdfs/DefaultInputChannelManager.java
+7
-41
cat-job/src/main/java/com/dianping/cat/job/hdfs/InputChannel.java
...src/main/java/com/dianping/cat/job/hdfs/InputChannel.java
+5
-0
cat-job/src/main/resources/META-INF/plexus/components.xml
cat-job/src/main/resources/META-INF/plexus/components.xml
+1
-9
未找到文件。
cat-job/src/main/java/com/dianping/cat/job/hdfs/DefaultInputChannel.java
浏览文件 @
6b72855e
...
...
@@ -16,6 +16,8 @@ public class DefaultInputChannel implements InputChannel {
private
MessageCodec
m_codec
;
private
FSDataInputStream
m_in
;
private
String
path
;
@Override
public
void
close
()
{
...
...
@@ -51,4 +53,13 @@ public class DefaultInputChannel implements InputChannel {
// TODO Auto-generated method stub
return
false
;
}
@Override
public
String
getPath
()
{
return
path
;
}
public
void
setPath
(
String
path
)
{
this
.
path
=
path
;
}
}
cat-job/src/main/java/com/dianping/cat/job/hdfs/DefaultInputChannelManager.java
浏览文件 @
6b72855e
...
...
@@ -2,10 +2,6 @@ package com.dianping.cat.job.hdfs;
import
java.io.IOException
;
import
java.net.URI
;
import
java.util.ArrayList
;
import
java.util.HashMap
;
import
java.util.List
;
import
java.util.Map
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.hadoop.fs.FSDataInputStream
;
...
...
@@ -19,8 +15,7 @@ import org.codehaus.plexus.personality.plexus.lifecycle.phase.InitializationExce
import
com.site.lookup.ContainerHolder
;
import
com.site.lookup.annotation.Inject
;
public
class
DefaultInputChannelManager
extends
ContainerHolder
implements
InputChannelManager
,
Initializable
,
LogEnabled
{
public
class
DefaultInputChannelManager
extends
ContainerHolder
implements
InputChannelManager
,
Initializable
,
LogEnabled
{
@Inject
private
URI
m_serverUri
;
...
...
@@ -29,38 +24,16 @@ public class DefaultInputChannelManager extends ContainerHolder implements Input
private
FileSystem
m_fs
;
private
Map
<
String
,
DefaultInputChannel
>
m_channels
=
new
HashMap
<
String
,
DefaultInputChannel
>();
private
Logger
m_logger
;
private
Path
m_basePath
;
@Override
public
void
cleanupChannels
()
{
try
{
List
<
String
>
expired
=
new
ArrayList
<
String
>();
for
(
Map
.
Entry
<
String
,
DefaultInputChannel
>
e
:
m_channels
.
entrySet
())
{
if
(
e
.
getValue
().
isExpired
())
{
expired
.
add
(
e
.
getKey
());
}
}
for
(
String
path
:
expired
)
{
InputChannel
channel
=
m_channels
.
remove
(
path
);
closeChannel
(
channel
);
}
}
catch
(
Exception
e
)
{
m_logger
.
warn
(
"Error when doing cleanup!"
,
e
);
}
}
@Override
public
void
closeAllChannels
()
{
for
(
DefaultInputChannel
channel
:
m_channels
.
values
())
{
closeChannel
(
channel
);
}
}
@Override
...
...
@@ -81,7 +54,7 @@ public class DefaultInputChannelManager extends ContainerHolder implements Input
FileSystem
fs
;
config
.
setInt
(
"io.file.buffer.size"
,
8192
);
if
(
m_serverUri
==
null
)
{
fs
=
FileSystem
.
getLocal
(
config
);
m_basePath
=
new
Path
(
fs
.
getWorkingDirectory
(),
m_baseDir
);
...
...
@@ -98,18 +71,11 @@ public class DefaultInputChannelManager extends ContainerHolder implements Input
@Override
public
InputChannel
openChannel
(
String
path
)
throws
IOException
{
DefaultInputChannel
channel
=
m_channels
.
get
(
path
);
if
(
channel
==
null
)
{
Path
file
=
new
Path
(
m_basePath
,
path
);
FSDataInputStream
in
=
m_fs
.
open
(
file
);
channel
=
(
DefaultInputChannel
)
lookup
(
InputChannel
.
class
);
channel
.
initialize
(
in
);
m_channels
.
put
(
path
,
channel
);
}
Path
file
=
new
Path
(
m_basePath
,
path
);
FSDataInputStream
in
=
m_fs
.
open
(
file
);
DefaultInputChannel
channel
=
(
DefaultInputChannel
)
lookup
(
InputChannel
.
class
);
channel
.
setPath
(
path
);
channel
.
initialize
(
in
);
return
channel
;
}
...
...
cat-job/src/main/java/com/dianping/cat/job/hdfs/InputChannel.java
浏览文件 @
6b72855e
...
...
@@ -26,4 +26,9 @@ public interface InputChannel {
* @throws IOException
*/
public
MessageTree
read
(
long
offset
,
int
length
)
throws
IOException
;
/**
* @return
*/
String
getPath
();
}
cat-job/src/main/resources/META-INF/plexus/components.xml
浏览文件 @
6b72855e
...
...
@@ -5,7 +5,7 @@
<implementation>
com.dianping.cat.job.hdfs.DefaultOutputChannel
</implementation>
<instantiation-strategy>
per-lookup
</instantiation-strategy>
<configuration>
<maxSize>
134217728
</maxSize>
<maxSize>
2097152
</maxSize>
</configuration>
<requirements>
<requirement>
...
...
@@ -17,10 +17,6 @@
<component>
<role>
com.dianping.cat.job.hdfs.OutputChannelManager
</role>
<implementation>
com.dianping.cat.job.hdfs.DefaultOutputChannelManager
</implementation>
<configuration>
<baseDir>
data
</baseDir>
<serverUri>
hdfs://192.168.7.43:9000/user/cat/
</serverUri>
</configuration>
<requirements>
<requirement>
<role>
com.dianping.cat.message.spi.MessagePathBuilder
</role>
...
...
@@ -41,10 +37,6 @@
<component>
<role>
com.dianping.cat.job.hdfs.InputChannelManager
</role>
<implementation>
com.dianping.cat.job.hdfs.DefaultInputChannelManager
</implementation>
<configuration>
<baseDir>
data
</baseDir>
<serverUri>
hdfs://192.168.7.43:9000/user/cat/
</serverUri>
</configuration>
</component>
<component>
<role>
com.dianping.cat.message.spi.MessageStorage
</role>
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录