Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
山不在高_有仙则灵
SkyWalking
提交
3ba2f221
S
SkyWalking
项目概览
山不在高_有仙则灵
/
SkyWalking
与 Fork 源项目一致
Fork自
apache / SkyWalking
通知
12
Star
0
Fork
2
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
SkyWalking
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
3ba2f221
编写于
1月 18, 2016
作者:
Z
zhangxin10
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
1.完成Map部分
上级
b68c0411
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
38 addition
and
155 deletion
+38
-155
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/mapper/CallChainMapper.java
.../ai/cloud/skywalking/analysis/mapper/CallChainMapper.java
+33
-28
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/model/ChainNode.java
...ava/com/ai/cloud/skywalking/analysis/model/ChainNode.java
+0
-9
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
...java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
+3
-36
skywalking-analysis/src/test/java/com/ai/cloud/skywalking/analysis/mapper/CallChainMapperTest.java
...cloud/skywalking/analysis/mapper/CallChainMapperTest.java
+1
-81
skywalking-protocol/src/main/java/com/ai/cloud/skywalking/protocol/Span.java
.../src/main/java/com/ai/cloud/skywalking/protocol/Span.java
+1
-1
未找到文件。
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/mapper/CallChainMapper.java
浏览文件 @
3ba2f221
package
com.ai.cloud.skywalking.analysis.mapper
;
import
java.io.IOException
;
import
java.util.ArrayList
;
import
java.util.HashMap
;
import
java.util.List
;
import
java.util.Map
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessChain
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessFilter
;
import
com.ai.cloud.skywalking.analysis.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.model.ChainNode
;
import
com.ai.cloud.skywalking.analysis.model.CostMap
;
import
com.ai.cloud.skywalking.analysis.model.SpanEntry
;
import
com.ai.cloud.skywalking.analysis.util.HBaseUtil
;
import
com.ai.cloud.skywalking.protocol.Span
;
import
org.apache.hadoop.hbase.Cell
;
import
org.apache.hadoop.hbase.client.Result
;
import
org.apache.hadoop.hbase.io.ImmutableBytesWritable
;
...
...
@@ -15,14 +17,8 @@ import org.apache.hadoop.io.Text;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessChain
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessFilter
;
import
com.ai.cloud.skywalking.analysis.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.model.ChainNode
;
import
com.ai.cloud.skywalking.analysis.model.CostMap
;
import
com.ai.cloud.skywalking.analysis.model.SpanEntry
;
import
com.ai.cloud.skywalking.analysis.util.HBaseUtil
;
import
com.ai.cloud.skywalking.protocol.Span
;
import
java.io.IOException
;
import
java.util.*
;
public
class
CallChainMapper
extends
TableMapper
<
Text
,
ChainInfo
>
{
private
Logger
logger
=
LoggerFactory
.
getLogger
(
CallChainMapper
.
class
.
getName
());
...
...
@@ -30,35 +26,44 @@ public class CallChainMapper extends TableMapper<Text, ChainInfo> {
@Override
protected
void
map
(
ImmutableBytesWritable
key
,
Result
value
,
Context
context
)
throws
IOException
,
InterruptedException
{
ChainInfo
chainInfo
=
new
ChainInfo
();
List
<
Span
>
spanList
=
new
ArrayList
<
Span
>();
CostMap
costMap
=
new
CostMap
();
for
(
Cell
cell
:
value
.
rawCells
())
{
Span
span
=
new
Span
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
()));
spanList
.
add
(
span
);
}
ChainInfo
chainInfo
=
spanToChainInfo
(
key
.
toString
(),
spanList
);
context
.
write
(
new
Text
(
chainInfo
.
getUserId
()
+
":"
+
chainInfo
.
getEntranceNodeToken
()),
chainInfo
);
}
//TODO: sort spanList
Map
<
String
,
SpanEntry
>
spanEntryMap
=
mergeSpanDataset
(
spanList
);
public
static
ChainInfo
spanToChainInfo
(
String
key
,
List
<
Span
>
spanList
)
{
CostMap
costMap
=
new
CostMap
();
ChainInfo
chainInfo
=
new
ChainInfo
();
Collections
.
sort
(
spanList
,
new
Comparator
<
Span
>()
{
@Override
public
int
compare
(
Span
span1
,
Span
span2
)
{
String
span1TraceLevel
=
span1
.
getParentLevel
()
+
"."
+
span1
.
getLevelId
();
String
span2TraceLevel
=
span2
.
getParentLevel
()
+
"."
+
span2
.
getLevelId
();
return
span1TraceLevel
.
compareTo
(
span2TraceLevel
);
}
});
Map
<
String
,
SpanEntry
>
spanEntryMap
=
mergeSpanDataSet
(
spanList
);
for
(
Map
.
Entry
<
String
,
SpanEntry
>
entry
:
spanEntryMap
.
entrySet
())
{
ChainNode
chainNode
=
new
ChainNode
();
SpanNodeProcessFilter
filter
=
SpanNodeProcessChain
.
getProcessChainByCallType
(
entry
.
getValue
().
getSpanType
());
filter
.
doFilter
(
entry
.
getValue
(),
chainNode
,
costMap
);
chainInfo
.
addNodes
(
chainNode
);
}
chainInfo
.
generateChainToken
();
HBaseUtil
.
saveData
(
key
.
toString
(),
chainInfo
);
context
.
write
(
new
Text
(
chainInfo
.
getUserId
()
+
":"
+
chainInfo
.
getEntranceNodeToken
()),
chainInfo
);
chainInfo
.
generateChainToken
();
HBaseUtil
.
saveData
(
key
,
chainInfo
);
return
chainInfo
;
}
private
Map
<
String
,
SpanEntry
>
mergeSpanDataset
(
List
<
Span
>
spanList
)
{
Map
<
String
,
SpanEntry
>
spanEntryMap
=
new
HashMap
<
String
,
SpanEntry
>();
for
(
Span
span
:
spanList
)
{
private
static
Map
<
String
,
SpanEntry
>
mergeSpanDataSet
(
List
<
Span
>
spanList
)
{
Map
<
String
,
SpanEntry
>
spanEntryMap
=
new
LinkedHashMap
<
String
,
SpanEntry
>();
for
(
int
i
=
spanList
.
size
()
-
1
;
i
>=
0
;
i
--)
{
Span
span
=
spanList
.
get
(
i
);
SpanEntry
spanEntry
=
spanEntryMap
.
get
(
span
.
getParentLevel
()
+
"."
+
span
.
getLevelId
());
if
(
spanEntry
==
null
)
{
spanEntry
=
new
SpanEntry
();
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/model/ChainNode.java
浏览文件 @
3ba2f221
...
...
@@ -67,10 +67,6 @@ public class ChainNode {
this
.
callType
=
callType
;
}
public
String
getCallType
()
{
return
callType
;
}
public
void
setUserId
(
String
userId
)
{
this
.
userId
=
userId
;
}
...
...
@@ -79,10 +75,6 @@ public class ChainNode {
return
userId
;
}
public
String
getBusinessKey
()
{
return
businessKey
;
}
public
void
setBusinessKey
(
String
businessKey
)
{
this
.
businessKey
=
businessKey
;
}
...
...
@@ -110,5 +102,4 @@ public class ChainNode {
}
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
浏览文件 @
3ba2f221
...
...
@@ -2,7 +2,6 @@ package com.ai.cloud.skywalking.analysis.util;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.model.ChainNode
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.hadoop.hbase.HBaseConfiguration
;
import
org.apache.hadoop.hbase.HColumnDescriptor
;
...
...
@@ -32,41 +31,9 @@ public class HBaseUtil {
Put
put
=
new
Put
(
Bytes
.
toBytes
(
traceId
));
for
(
ChainNode
chainNode
:
chainInfo
.
getNodes
())
{
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
USER_ID
),
Bytes
.
toBytes
(
chainNode
.
getUserId
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
STATUS
),
Bytes
.
toBytes
(
chainNode
.
getStatus
().
getValue
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
TRACE_INFO_COLUMN_CID
),
Bytes
.
toBytes
(
chainInfo
.
getChainToken
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
TRACE_INFO_COLUMN_CID
),
Bytes
.
toBytes
(
chainInfo
.
getChainToken
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
COST
),
Bytes
.
toBytes
(
chainNode
.
getCost
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
PARENT_LEVEL_ID
),
Bytes
.
toBytes
(
chainNode
.
getParentLevelId
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
LEVEL_ID
),
Bytes
.
toBytes
(
chainNode
.
getLevelId
()));
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
BUSINESS_KEY
),
Bytes
.
toBytes
(
chainNode
.
getBusinessKey
()));
}
put
.
addColumn
(
Bytes
.
toBytes
(
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Config
.
TraceInfo
.
TRACE_INFO_COLUMN_CID
),
Bytes
.
toBytes
(
chainInfo
.
getChainToken
()));
try
{
table
.
put
(
put
);
if
(
logger
.
isDebugEnabled
())
{
...
...
skywalking-analysis/src/test/java/com/ai/cloud/skywalking/analysis/mapper/CallChainMapperTest.java
浏览文件 @
3ba2f221
...
...
@@ -3,14 +3,7 @@ package com.ai.cloud.skywalking.analysis.mapper;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.ConfigInitializer
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessChain
;
import
com.ai.cloud.skywalking.analysis.filter.SpanNodeProcessFilter
;
import
com.ai.cloud.skywalking.analysis.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.model.ChainNode
;
import
com.ai.cloud.skywalking.analysis.model.CostMap
;
import
com.ai.cloud.skywalking.analysis.model.SpanEntry
;
import
com.ai.cloud.skywalking.analysis.util.HBaseUtil
;
import
com.ai.cloud.skywalking.analysis.util.TokenGenerator
;
import
com.ai.cloud.skywalking.protocol.Span
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.hadoop.hbase.Cell
;
...
...
@@ -23,9 +16,7 @@ import org.junit.Test;
import
java.io.IOException
;
import
java.util.ArrayList
;
import
java.util.HashMap
;
import
java.util.List
;
import
java.util.Map
;
/**
* Created by astraea on 2016/1/15.
...
...
@@ -43,78 +34,7 @@ public class CallChainMapperTest {
public
void
testMap
()
throws
Exception
{
ConfigInitializer
.
initialize
();
List
<
Span
>
spanList
=
selectByTraceId
(
chain_Id
);
ChainInfo
chainInfo
=
new
ChainInfo
();
CostMap
costMap
=
new
CostMap
();
Map
<
String
,
SpanEntry
>
spanEntryMap
=
mergeSpanDataset
(
spanList
);
for
(
Map
.
Entry
<
String
,
SpanEntry
>
entry
:
spanEntryMap
.
entrySet
())
{
ChainNode
chainNode
=
new
ChainNode
();
SpanNodeProcessFilter
filter
=
SpanNodeProcessChain
.
getProcessChainByCallType
(
entry
.
getValue
().
getSpanType
());
filter
.
doFilter
(
entry
.
getValue
(),
chainNode
,
costMap
);
chainInfo
.
getNodes
().
add
(
chainNode
);
}
//
String
firstNodeToken
=
null
;
boolean
status
=
true
;
StringBuilder
stringBuilder
=
new
StringBuilder
();
for
(
ChainNode
node
:
chainInfo
.
getNodes
())
{
if
((
node
.
getParentLevelId
()
==
null
||
node
.
getParentLevelId
().
length
()
==
0
)
&&
node
.
getLevelId
()
==
0
)
{
firstNodeToken
=
node
.
getNodeToken
();
chainInfo
.
setUserId
(
node
.
getUserId
());
}
// 状态轮询
if
(
node
.
getStatus
()
==
ChainNode
.
NodeStatus
.
ABNORMAL
)
{
status
=
false
;
}
// 设置时间
computeChainNodeCost
(
costMap
,
node
);
stringBuilder
.
append
(
node
.
getParentLevelId
()
+
"."
+
node
.
getLevelId
()
+
"-"
+
node
.
getNodeToken
()
+
";"
);
}
// 设置状态
if
(
status
)
{
chainInfo
.
setChainStatus
(
ChainInfo
.
ChainStatus
.
NORMAL
);
}
else
{
chainInfo
.
setChainStatus
(
ChainInfo
.
ChainStatus
.
ABNORMAL
);
}
// 设置Token
chainInfo
.
setChainToken
(
TokenGenerator
.
generate
(
stringBuilder
.
toString
()));
//SaveToHbase
HBaseUtil
.
saveData
(
chain_Id
,
chainInfo
);
System
.
out
.
println
(
chainInfo
);
}
private
void
computeChainNodeCost
(
CostMap
costMap
,
ChainNode
node
)
{
String
levelId
=
node
.
getParentLevelId
();
if
(
levelId
!=
null
&&
levelId
.
length
()
>
0
)
{
levelId
+=
"."
;
}
levelId
+=
node
.
getLevelId
()
+
""
;
if
(
costMap
.
get
(
levelId
)
!=
null
)
{
node
.
setCost
(
node
.
getCost
()
-
costMap
.
get
(
levelId
));
}
}
private
Map
<
String
,
SpanEntry
>
mergeSpanDataset
(
List
<
Span
>
spanList
)
{
Map
<
String
,
SpanEntry
>
spanEntryMap
=
new
HashMap
<
String
,
SpanEntry
>();
for
(
Span
span
:
spanList
)
{
SpanEntry
spanEntry
=
spanEntryMap
.
get
(
span
.
getParentLevel
()
+
"."
+
span
.
getLevelId
());
if
(
spanEntry
==
null
)
{
spanEntry
=
new
SpanEntry
();
spanEntryMap
.
put
(
span
.
getParentLevel
()
+
"."
+
span
.
getLevelId
(),
spanEntry
);
}
spanEntry
.
setSpan
(
span
);
}
return
spanEntryMap
;
ChainInfo
chainInfo
=
CallChainMapper
.
spanToChainInfo
(
chain_Id
,
spanList
);
}
public
static
List
<
Span
>
selectByTraceId
(
String
traceId
)
throws
IOException
{
...
...
skywalking-protocol/src/main/java/com/ai/cloud/skywalking/protocol/Span.java
浏览文件 @
3ba2f221
...
...
@@ -48,7 +48,7 @@ public class Span extends SpanData {
processNo
=
fieldValues
[
12
].
trim
();
applicationId
=
fieldValues
[
13
].
trim
();
userId
=
fieldValues
[
14
].
trim
();
if
(
fieldValues
.
length
>
1
4
)
{
if
(
fieldValues
.
length
>
1
5
)
{
callType
=
fieldValues
[
15
].
trim
();
}
this
.
originData
=
originData
;
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录