Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
山不在高_有仙则灵
SkyWalking
提交
22f12c20
S
SkyWalking
项目概览
山不在高_有仙则灵
/
SkyWalking
与 Fork 源项目一致
Fork自
apache / SkyWalking
通知
12
Star
0
Fork
2
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
SkyWalking
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
22f12c20
编写于
1月 29, 2016
作者:
A
ascrutae
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
修改变量名,添加注释
上级
89d01ee9
变更
7
隐藏空白更改
内联
并排
Showing
7 changed file
with
49 addition
and
53 deletion
+49
-53
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
...om/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
+2
-1
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainDetail.java
...oud/skywalking/analysis/categorize2chain/ChainDetail.java
+2
-1
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainRelationship.java
...ywalking/analysis/categorize2chain/ChainRelationship.java
+2
-2
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainSpecificTimeWindowSummary.java
...ysis/categorize2chain/ChainSpecificTimeWindowSummary.java
+2
-1
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/Config.java
.../java/com/ai/cloud/skywalking/analysis/config/Config.java
+2
-26
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/Constants.java
...va/com/ai/cloud/skywalking/analysis/config/Constants.java
+23
-1
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
...java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
+16
-21
未找到文件。
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
浏览文件 @
22f12c20
...
...
@@ -5,6 +5,7 @@ import java.text.ParseException;
import
java.text.SimpleDateFormat
;
import
java.util.Date
;
import
com.ai.cloud.skywalking.analysis.config.Constants
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.hadoop.conf.Configured
;
import
org.apache.hadoop.hbase.client.Scan
;
...
...
@@ -50,7 +51,7 @@ public class AnalysisServerDriver extends Configured implements Tool {
job
.
setJarByClass
(
AnalysisServerDriver
.
class
);
Scan
scan
=
buildHBaseScan
(
args
);
TableMapReduceUtil
.
initTableMapperJob
(
Con
fig
.
HBase
.
TABLE_CALL_CHAIN
,
scan
,
Categorize2ChainMapper
.
class
,
TableMapReduceUtil
.
initTableMapperJob
(
Con
stants
.
TABLE_CALL_CHAIN
,
scan
,
Categorize2ChainMapper
.
class
,
Text
.
class
,
ChainInfo
.
class
,
job
);
job
.
setReducerClass
(
Categorize2ChainReducer
.
class
);
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainDetail.java
浏览文件 @
22f12c20
...
...
@@ -3,6 +3,7 @@ package com.ai.cloud.skywalking.analysis.categorize2chain;
import
com.ai.cloud.skywalking.analysis.categorize2chain.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.model.ChainNode
;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.Constants
;
import
com.google.gson.Gson
;
import
org.apache.hadoop.hbase.client.Put
;
...
...
@@ -35,7 +36,7 @@ public class ChainDetail {
public
void
save
(
Put
put
)
throws
SQLException
{
for
(
Map
.
Entry
<
String
,
ChainNode
>
entry
:
chainNodeMap
.
entrySet
()){
put
.
addColumn
(
Con
fig
.
HBase
.
TRACE_DETAIL_FAMILY_COLUMN
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
put
.
addColumn
(
Con
stants
.
COLUMN_FAMILY_NAME_TRACE_DETAIL
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
if
(
isNormal
)
{
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainRelationship.java
浏览文件 @
22f12c20
...
...
@@ -107,11 +107,11 @@ public class ChainRelationship {
private
void
saveChainRelationship
()
throws
IOException
{
Put
put
=
new
Put
(
getKey
().
getBytes
());
put
.
addColumn
(
Con
fig
.
HBase
.
CHAIN_RELATIONSHIP_COLUMN_FAMILY
.
getBytes
(),
Constants
.
UNCATEGORIZED_QUALIFIER_NAME
.
getBytes
()
put
.
addColumn
(
Con
stants
.
COLUMN_FAMILY_CHAIN_RELATIONSHIP
.
getBytes
(),
Constants
.
UNCATEGORIZE_COLUMN_FAMILY
.
getBytes
()
,
new
GsonBuilder
().
excludeFieldsWithoutExposeAnnotation
().
create
().
toJson
(
getUncategorizeChainInfoList
()).
getBytes
());
for
(
Map
.
Entry
<
String
,
CategorizedChainInfo
>
entry
:
getCategorizedChainInfoMap
().
entrySet
())
{
put
.
addColumn
(
Con
fig
.
HBase
.
CHAIN_RELATIONSHIP_COLUMN_FAMILY
.
getBytes
(),
entry
.
getKey
().
getBytes
()
put
.
addColumn
(
Con
stants
.
COLUMN_FAMILY_CHAIN_RELATIONSHIP
.
getBytes
(),
entry
.
getKey
().
getBytes
()
,
entry
.
getValue
().
toString
().
getBytes
());
}
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/categorize2chain/ChainSpecificTimeWindowSummary.java
浏览文件 @
22f12c20
...
...
@@ -2,6 +2,7 @@ package com.ai.cloud.skywalking.analysis.categorize2chain;
import
com.ai.cloud.skywalking.analysis.categorize2chain.model.ChainNode
;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.Constants
;
import
com.ai.cloud.skywalking.analysis.util.HBaseUtil
;
import
org.apache.hadoop.hbase.client.Put
;
...
...
@@ -51,7 +52,7 @@ public class ChainSpecificTimeWindowSummary {
public
void
save
(
Put
put
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummary
>
entry
:
chainNodeSummaryResultMap
.
entrySet
())
{
put
.
addColumn
(
Con
fig
.
HBase
.
CHAIN_SUMMARY_COLUMN_FAMIL
Y
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
put
.
addColumn
(
Con
stants
.
COLUMN_FAMILY_NAME_CHAIN_SUMMAR
Y
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/Config.java
浏览文件 @
22f12c20
package
com.ai.cloud.skywalking.analysis.config
;
public
class
Config
{
public
static
class
Reducer
{
public
static
class
Reducer
{
public
static
int
REDUCER_NUMBER
=
1
;
}
public
static
class
HBase
{
public
static
String
TRACE_DETAIL_FAMILY_COLUMN
=
"chain_detail"
;
public
static
String
CHAIN_SUMMARY_COLUMN_FAMILY
=
"chain_summary"
;
public
static
String
TRACE_INFO_COLUMN_FAMILY
=
"trace_info"
;
public
static
String
ZK_QUORUM
;
public
static
String
ZK_CLIENT_PORT
;
public
static
String
TRACE_INFO_TABLE_NAME
=
"trace-info"
;
public
static
String
TABLE_CALL_CHAIN_RELATIONSHIP
=
"sw-chain-relationship"
;
public
static
String
CHAIN_RELATIONSHIP_COLUMN_FAMILY
=
"chain-relationship"
;
public
static
String
TABLE_CHAIN_INFO
=
"sw-chain-info"
;
public
static
String
TABLE_CHAIN_SUMMARY
=
"sw-chain-summary"
;
public
static
String
TABLE_CHAIN_DETAIL
=
"sw-chain-detail"
;
public
static
String
TABLE_CALL_CHAIN
=
"sw-call-chain"
;
}
public
static
class
TraceInfo
{
public
static
String
TRACE_INFO_COLUMN_CID
=
"cid"
;
}
public
static
class
MySql
{
...
...
@@ -51,6 +27,6 @@ public class Config {
}
public
static
class
ChainNodeSummary
{
public
static
long
INTERVAL
=
5
L
;
public
static
long
INTERVAL
=
1
L
;
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/Constants.java
浏览文件 @
22f12c20
package
com.ai.cloud.skywalking.analysis.config
;
public
class
Constants
{
public
static
final
String
UNCATEGORIZED_QUALIFIER_NAME
=
"UNCATEGORIZED_CALL_CHAIN"
;
public
static
final
String
UNCATEGORIZE_COLUMN_FAMILY
=
"UNCATEGORIZED_CALL_CHAIN"
;
// HBase 表:用于存放CID,TID的映射关系表
public
static
String
TABLE_CID_TID_MAPPING
=
"sw-cid-tid-mapping"
;
//HBase 表:用于存放调用链一分钟的汇总,汇总结果不包含关系汇总
public
static
String
TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP
=
"sw-chain-1min-summary-ex-rela"
;
public
static
String
TABLE_CHAIN_DETAIL
=
"sw-chain-detail"
;
public
static
String
TABLE_CALL_CHAIN
=
"sw-call-chain"
;
public
static
String
TABLE_CALL_CHAIN_RELATIONSHIP
=
"sw-chain-relationship"
;
public
static
String
COLUMN_FAMILY_CHAIN_RELATIONSHIP
=
"chain-relationship"
;
public
static
String
COLUMN_FAMILY_NAME_TRACE_DETAIL
=
"chain_detail"
;
public
static
String
COLUMN_FAMILY_NAME_CHAIN_SUMMARY
=
"chain_summary"
;
public
static
String
COLUMN_FAMILY_NAME_TRACE_INFO
=
"trace_info"
;
public
static
String
COLUMN_FAMILY_NAME_CID
=
"cid"
;
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/util/HBaseUtil.java
浏览文件 @
22f12c20
package
com.ai.cloud.skywalking.analysis.util
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.CategorizedChainInfo
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.ChainRelationship
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.ChainSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.UncategorizeChainInfo
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.*
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.model.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.Constants
;
import
com.google.gson.Gson
;
import
com.google.gson.reflect.TypeToken
;
import
org.apache.hadoop.conf.Configuration
;
import
org.apache.hadoop.hbase.*
;
import
org.apache.hadoop.hbase.client.*
;
...
...
@@ -31,15 +26,15 @@ public class HBaseUtil {
Table
table
=
null
;
try
{
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CHAIN_INFO
));
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CID_TID_MAPPING
));
}
catch
(
IOException
e
)
{
logger
.
error
(
"Cannot found table["
+
Con
fig
.
HBase
.
TRACE_INFO_TABLE_NAME
+
"]"
,
e
);
logger
.
error
(
"Cannot found table["
+
Con
stants
.
TABLE_CID_TID_MAPPING
+
"]"
,
e
);
}
Put
put
=
new
Put
(
Bytes
.
toBytes
(
traceId
));
put
.
addColumn
(
Bytes
.
toBytes
(
Con
fig
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
),
Bytes
.
toBytes
(
Con
fig
.
TraceInfo
.
TRACE_INFO_COLUMN
_CID
),
put
.
addColumn
(
Bytes
.
toBytes
(
Con
stants
.
COLUMN_FAMILY_NAME_TRACE_INFO
),
Bytes
.
toBytes
(
Con
stants
.
COLUMN_FAMILY_NAME
_CID
),
Bytes
.
toBytes
(
chainInfo
.
getCID
()));
try
{
table
.
put
(
put
);
...
...
@@ -58,16 +53,16 @@ public class HBaseUtil {
try
{
initHBaseClient
();
//
createTableIfNeed
(
Con
fig
.
HBase
.
TABLE_CHAIN_INFO
,
Config
.
HBase
.
TRACE_INFO_COLUMN_FAMILY
);
createTableIfNeed
(
Con
stants
.
TABLE_CID_TID_MAPPING
,
Constants
.
COLUMN_FAMILY_NAME_TRACE_INFO
);
//
createTableIfNeed
(
Con
fig
.
HBase
.
TABLE_CALL_CHAIN_RELATIONSHIP
,
Config
.
HBase
.
CHAIN_RELATIONSHIP_COLUMN_FAMILY
);
createTableIfNeed
(
Con
stants
.
TABLE_CALL_CHAIN_RELATIONSHIP
,
Constants
.
COLUMN_FAMILY_CHAIN_RELATIONSHIP
);
createTableIfNeed
(
Con
fig
.
HBase
.
TABLE_CHAIN_SUMMARY
,
Config
.
HBase
.
CHAIN_SUMMARY_COLUMN_FAMIL
Y
);
createTableIfNeed
(
Con
stants
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP
,
Constants
.
COLUMN_FAMILY_NAME_CHAIN_SUMMAR
Y
);
createTableIfNeed
(
Con
fig
.
HBase
.
TABLE_CHAIN_DETAIL
,
Config
.
HBase
.
TRACE_DETAIL_FAMILY_COLUMN
);
createTableIfNeed
(
Con
stants
.
TABLE_CHAIN_DETAIL
,
Constants
.
COLUMN_FAMILY_NAME_TRACE_DETAIL
);
}
catch
(
IOException
e
)
{
logger
.
error
(
"Create table
[{}] failed"
,
Config
.
HBase
.
TRACE_INFO_TABLE_NAME
,
e
);
logger
.
error
(
"Create table
s failed"
,
e
);
}
}
...
...
@@ -97,7 +92,7 @@ public class HBaseUtil {
public
static
ChainRelationship
selectCallChainRelationship
(
String
key
)
throws
IOException
{
ChainRelationship
chainRelate
=
new
ChainRelationship
(
key
);
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CALL_CHAIN_RELATIONSHIP
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CALL_CHAIN_RELATIONSHIP
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
for
(
Cell
cell
:
r
.
rawCells
())
{
...
...
@@ -105,7 +100,7 @@ public class HBaseUtil {
String
qualifierName
=
Bytes
.
toString
(
cell
.
getQualifierArray
(),
cell
.
getQualifierOffset
(),
cell
.
getQualifierLength
());
if
(
Constants
.
UNCATEGORIZE
D_QUALIFIER_NAME
.
equals
(
qualifierName
))
{
if
(
Constants
.
UNCATEGORIZE
_COLUMN_FAMILY
.
equals
(
qualifierName
))
{
List
<
UncategorizeChainInfo
>
uncategorizeChainInfoList
=
new
Gson
().
fromJson
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
()),
new
TypeToken
<
List
<
UncategorizeChainInfo
>>()
{
...
...
@@ -123,7 +118,7 @@ public class HBaseUtil {
public
static
ChainSpecificTimeWindowSummary
selectChainSummaryResult
(
String
key
)
throws
IOException
{
ChainSpecificTimeWindowSummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CHAIN_SUMMARY
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
...
...
@@ -141,7 +136,7 @@ public class HBaseUtil {
}
public
static
void
saveChainRelationship
(
Put
put
)
throws
IOException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CALL_CHAIN_RELATIONSHIP
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CALL_CHAIN_RELATIONSHIP
));
table
.
put
(
put
);
if
(
logger
.
isDebugEnabled
())
{
...
...
@@ -150,7 +145,7 @@ public class HBaseUtil {
}
public
static
void
batchSaveChainSpecificTimeWindowSummary
(
List
<
Put
>
puts
)
throws
IOException
,
InterruptedException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CHAIN_SUMMARY
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP
));
Object
[]
resultArrays
=
new
Object
[
puts
.
size
()];
table
.
batch
(
puts
,
resultArrays
);
for
(
Object
result
:
resultArrays
)
{
...
...
@@ -161,7 +156,7 @@ public class HBaseUtil {
}
public
static
void
saveChainDetails
(
List
<
Put
>
puts
)
throws
IOException
,
InterruptedException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
fig
.
HBase
.
TABLE_CHAIN_DETAIL
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
Con
stants
.
TABLE_CHAIN_DETAIL
));
if
(
puts
!=
null
&&
puts
.
size
()
>
0
)
{
Object
[]
resultArrays
=
new
Object
[
puts
.
size
()];
table
.
batch
(
puts
,
resultArrays
);
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录