Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
月轩居士
SkyWalking
提交
13421da7
S
SkyWalking
项目概览
月轩居士
/
SkyWalking
与 Fork 源项目一致
Fork自
apache / SkyWalking
通知
4
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
SkyWalking
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
13421da7
编写于
3月 06, 2016
作者:
A
ascrutae
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
1.完成统计部分
2. 删除无用代码
上级
e024a329
变更
26
隐藏空白更改
内联
并排
Showing
26 changed file
with
334 addition
and
1194 deletion
+334
-1194
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
...om/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
+5
-5
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Chain2SummaryMapper.java
...kywalking/analysis/chain2summary/Chain2SummaryMapper.java
+0
-43
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Chain2SummaryReducer.java
...ywalking/analysis/chain2summary/Chain2SummaryReducer.java
+0
-49
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/ChainRelationship4Search.java
...king/analysis/chain2summary/ChainRelationship4Search.java
+0
-32
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Summary.java
...m/ai/cloud/skywalking/analysis/chain2summary/Summary.java
+0
-36
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificDaySummary.java
...sis/chain2summary/entity/ChainNodeSpecificDaySummary.java
+0
-57
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificHourSummary.java
...is/chain2summary/entity/ChainNodeSpecificHourSummary.java
+0
-56
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificMonthSummary.java
...s/chain2summary/entity/ChainNodeSpecificMonthSummary.java
+0
-57
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificDaySummary.java
...nalysis/chain2summary/entity/ChainSpecificDaySummary.java
+0
-41
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificHourSummary.java
...alysis/chain2summary/entity/ChainSpecificHourSummary.java
+0
-42
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificMinSummary.java
...nalysis/chain2summary/entity/ChainSpecificMinSummary.java
+0
-43
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificMonthSummary.java
...lysis/chain2summary/entity/ChainSpecificMonthSummary.java
+0
-38
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSummaryWithRelationship.java
...is/chain2summary/entity/ChainSummaryWithRelationship.java
+0
-101
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/po/ChainSpecificTimeSummary.java
...g/analysis/chain2summary/po/ChainSpecificTimeSummary.java
+0
-115
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/ChainBuildMapper.java
...loud/skywalking/analysis/chainbuild/ChainBuildMapper.java
+1
-1
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/ChainBuildReducer.java
...oud/skywalking/analysis/chainbuild/ChainBuildReducer.java
+1
-0
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/CallChainTree.java
.../skywalking/analysis/chainbuild/entity/CallChainTree.java
+13
-7
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/CallChainTreeNode.java
...walking/analysis/chainbuild/entity/CallChainTreeNode.java
+81
-0
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/ChainNodeForSummary.java
...lking/analysis/chainbuild/entity/ChainNodeForSummary.java
+0
-18
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/ChainNodeSpecificMinSummary.java
...alysis/chainbuild/entity/ChainNodeSpecificMinSummary.java
+42
-0
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/ChainNodeSpecificTimeWindowSummaryValue.java
...build/entity/ChainNodeSpecificTimeWindowSummaryValue.java
+30
-0
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/CallChainTreeNode.java
.../skywalking/analysis/chainbuild/po/CallChainTreeNode.java
+0
-51
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/ChainInfo.java
...ai/cloud/skywalking/analysis/chainbuild/po/ChainInfo.java
+127
-126
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/ChainNode.java
...ai/cloud/skywalking/analysis/chainbuild/po/ChainNode.java
+2
-0
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/util/HBaseUtil.java
.../cloud/skywalking/analysis/chainbuild/util/HBaseUtil.java
+31
-207
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/HBaseTableMetaData.java
.../cloud/skywalking/analysis/config/HBaseTableMetaData.java
+1
-69
未找到文件。
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/AnalysisServerDriver.java
浏览文件 @
13421da7
...
...
@@ -5,6 +5,9 @@ import java.text.ParseException;
import
java.text.SimpleDateFormat
;
import
java.util.Date
;
import
com.ai.cloud.skywalking.analysis.chainbuild.ChainBuildMapper
;
import
com.ai.cloud.skywalking.analysis.chainbuild.ChainBuildReducer
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
org.apache.hadoop.conf.Configuration
;
...
...
@@ -20,9 +23,6 @@ import org.apache.hadoop.util.ToolRunner;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.Categorize2ChainMapper
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.Categorize2ChainReducer
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.po.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.ConfigInitializer
;
...
...
@@ -52,10 +52,10 @@ public class AnalysisServerDriver extends Configured implements Tool {
job
.
setJarByClass
(
AnalysisServerDriver
.
class
);
Scan
scan
=
buildHBaseScan
(
args
);
TableMapReduceUtil
.
initTableMapperJob
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN
.
TABLE_NAME
,
scan
,
C
ategorize2Chain
Mapper
.
class
,
TableMapReduceUtil
.
initTableMapperJob
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN
.
TABLE_NAME
,
scan
,
C
hainBuild
Mapper
.
class
,
Text
.
class
,
ChainInfo
.
class
,
job
);
job
.
setReducerClass
(
C
ategorize2Chain
Reducer
.
class
);
job
.
setReducerClass
(
C
hainBuild
Reducer
.
class
);
job
.
setNumReduceTasks
(
Config
.
Reducer
.
REDUCER_NUMBER
);
job
.
setOutputFormatClass
(
NullOutputFormat
.
class
);
return
job
.
waitForCompletion
(
true
)
?
0
:
1
;
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Chain2SummaryMapper.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.ConfigInitializer
;
import
org.apache.hadoop.hbase.Cell
;
import
org.apache.hadoop.hbase.client.Result
;
import
org.apache.hadoop.hbase.io.ImmutableBytesWritable
;
import
org.apache.hadoop.hbase.mapreduce.TableMapper
;
import
org.apache.hadoop.hbase.util.Bytes
;
import
org.apache.hadoop.io.Text
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
java.io.IOException
;
public
class
Chain2SummaryMapper
extends
TableMapper
<
Text
,
ChainSpecificTimeSummary
>
{
private
Logger
logger
=
LoggerFactory
.
getLogger
(
Chain2SummaryMapper
.
class
);
@Override
protected
void
setup
(
Context
context
)
throws
IOException
,
InterruptedException
{
ConfigInitializer
.
initialize
();
}
@Override
protected
void
map
(
ImmutableBytesWritable
key
,
Result
value
,
Context
context
)
throws
IOException
,
InterruptedException
{
try
{
ChainSpecificTimeSummary
summary
=
new
ChainSpecificTimeSummary
(
Bytes
.
toString
(
key
.
get
()));
for
(
Cell
cell
:
value
.
rawCells
())
{
summary
.
addChainNodeSummaryResult
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
()));
}
context
.
write
(
new
Text
(
summary
.
buildMapperKey
()),
summary
);
}
catch
(
Exception
e
)
{
logger
.
error
(
"Failed to mapper call chain["
+
key
.
toString
()
+
"]"
,
e
);
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Chain2SummaryReducer.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.ConfigInitializer
;
import
org.apache.hadoop.hbase.util.Bytes
;
import
org.apache.hadoop.io.IntWritable
;
import
org.apache.hadoop.io.Text
;
import
org.apache.hadoop.mapreduce.Reducer
;
import
org.slf4j.Logger
;
import
org.slf4j.LoggerFactory
;
import
java.io.IOException
;
import
java.util.Iterator
;
public
class
Chain2SummaryReducer
extends
Reducer
<
Text
,
ChainSpecificTimeSummary
,
Text
,
IntWritable
>
{
private
Logger
logger
=
LoggerFactory
.
getLogger
(
Chain2SummaryReducer
.
class
);
@Override
protected
void
setup
(
Context
context
)
throws
IOException
,
InterruptedException
{
ConfigInitializer
.
initialize
();
}
@Override
protected
void
reduce
(
Text
key
,
Iterable
<
ChainSpecificTimeSummary
>
values
,
Context
context
)
throws
IOException
,
InterruptedException
{
doReduceAction
(
Bytes
.
toString
(
key
.
getBytes
()),
values
.
iterator
());
}
public
void
doReduceAction
(
String
key
,
Iterator
<
ChainSpecificTimeSummary
>
summaryIterator
)
{
try
{
ChainRelationship4Search
chainRelationship
=
ChainRelationship4Search
.
load
(
Bytes
.
toString
(
key
.
getBytes
()));
Summary
summary
=
new
Summary
();
while
(
summaryIterator
.
hasNext
())
{
try
{
ChainSpecificTimeSummary
timeSummary
=
summaryIterator
.
next
();
summary
.
summary
(
timeSummary
,
chainRelationship
);
}
catch
(
Exception
e
)
{
logger
.
error
(
"Failed to reduce"
,
e
);
}
}
summary
.
saveToHBase
();
}
catch
(
Exception
e
)
{
e
.
printStackTrace
();
logger
.
error
(
"Failed to reduce key="
+
Bytes
.
toString
(
key
.
getBytes
()),
e
);
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/ChainRelationship4Search.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.util.HBaseUtil
;
import
java.io.IOException
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainRelationship4Search
{
// key: 正常链路ID value: 正常链路ID
// key: 异常链路ID value: 正常链路ID
// key: 未分类链路ID value: 未分类链路ID
private
Map
<
String
,
String
>
chainRelationshipMap
=
new
HashMap
<
String
,
String
>();
public
static
ChainRelationship4Search
load
(
String
rowkey
)
throws
IOException
{
ChainRelationship4Search
chainRelationship4Search
=
HBaseUtil
.
queryChainRelationship
(
rowkey
);
return
chainRelationship4Search
;
}
public
void
addRelationship
(
String
cid
)
{
chainRelationshipMap
.
put
(
cid
,
cid
);
}
public
void
addRelationship
(
String
normalCID
,
String
abnormalCID
)
{
chainRelationshipMap
.
put
(
normalCID
,
abnormalCID
);
}
public
String
searchRelationship
(
String
cid
)
{
return
chainRelationshipMap
.
get
(
cid
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/Summary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary
;
import
java.io.IOException
;
import
java.util.Map
;
import
java.util.concurrent.ConcurrentHashMap
;
import
com.ai.cloud.skywalking.analysis.chain2summary.entity.ChainSummaryWithRelationship
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
public
class
Summary
{
private
Map
<
String
,
ChainSummaryWithRelationship
>
summaryWithRelationshipMap
;
public
Summary
(){
summaryWithRelationshipMap
=
new
ConcurrentHashMap
<
String
,
ChainSummaryWithRelationship
>();
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
,
ChainRelationship4Search
chainRelationship
)
throws
IOException
{
String
cid
=
chainRelationship
.
searchRelationship
(
timeSummary
.
getcId
());
if
(
cid
==
null
||
cid
.
length
()
==
0
)
{
cid
=
timeSummary
.
getcId
();
}
if
(!
summaryWithRelationshipMap
.
containsKey
(
cid
))
{
summaryWithRelationshipMap
.
put
(
cid
,
new
ChainSummaryWithRelationship
(
cid
));
}
ChainSummaryWithRelationship
chainSummaryWithRelationship
=
summaryWithRelationshipMap
.
get
(
cid
);
chainSummaryWithRelationship
.
summary
(
timeSummary
);
}
public
void
saveToHBase
()
throws
IOException
,
InterruptedException
{
for
(
Map
.
Entry
<
String
,
ChainSummaryWithRelationship
>
entry
:
summaryWithRelationshipMap
.
entrySet
())
{
entry
.
getValue
().
saveToHBase
();
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificDaySummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummaryValue
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
java.util.Calendar
;
import
java.util.Date
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainNodeSpecificDaySummary
{
private
String
traceLevelId
;
// key: 天
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
summaryValueMap
;
private
String
key
;
public
ChainNodeSpecificDaySummary
()
{
summaryValueMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>();
}
public
ChainNodeSpecificDaySummary
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
summaryValueMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summaryValueMap"
).
toString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>>()
{
}.
getType
());
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
}
public
void
summary
(
long
summaryTimestamp
,
ChainNodeSpecificTimeWindowSummary
value
)
{
key
=
generateSummaryValueMapKey
(
summaryTimestamp
);
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
entry
:
value
.
getSummerValueMap
().
entrySet
())
{
if
(
summaryValueMap
.
get
(
key
)
==
null
)
{
summaryValueMap
.
put
(
key
,
new
ChainNodeSpecificTimeWindowSummaryValue
());
}
summaryValueMap
.
get
(
key
).
accumulate
(
entry
.
getValue
());
}
}
private
String
generateSummaryValueMapKey
(
long
timeStamp
)
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
timeStamp
));
return
String
.
valueOf
(
calendar
.
get
(
Calendar
.
DAY_OF_MONTH
));
}
@Override
public
String
toString
()
{
return
new
Gson
().
toJson
(
this
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificHourSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummaryValue
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
java.util.Calendar
;
import
java.util.Date
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainNodeSpecificHourSummary
{
private
String
traceLevelId
;
// key : 小时
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
summerValueMap
;
public
ChainNodeSpecificHourSummary
()
{
summerValueMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>();
}
public
ChainNodeSpecificHourSummary
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
summerValueMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summerValueMap"
).
toString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>>()
{
}.
getType
());
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
}
public
void
summary
(
long
summaryTimestamp
,
ChainNodeSpecificTimeWindowSummary
value
)
{
String
key
=
generateSummaryValueMapKey
(
summaryTimestamp
);
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
entry
:
value
.
getSummerValueMap
().
entrySet
())
{
if
(
summerValueMap
.
get
(
key
)
==
null
)
{
summerValueMap
.
put
(
key
,
new
ChainNodeSpecificTimeWindowSummaryValue
());
}
summerValueMap
.
get
(
key
).
accumulate
(
entry
.
getValue
());
}
}
private
String
generateSummaryValueMapKey
(
long
timeStamp
)
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
timeStamp
));
return
String
.
valueOf
(
calendar
.
get
(
Calendar
.
HOUR
));
}
@Override
public
String
toString
()
{
return
new
Gson
().
toJson
(
this
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainNodeSpecificMonthSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummaryValue
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
java.util.Calendar
;
import
java.util.Date
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainNodeSpecificMonthSummary
{
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
summaryValueMap
;
private
String
traceLevelId
;
private
String
key
;
public
ChainNodeSpecificMonthSummary
()
{
summaryValueMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>();
}
public
ChainNodeSpecificMonthSummary
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
summaryValueMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summaryValueMap"
).
toString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>>()
{
}.
getType
());
}
public
void
summary
(
long
summaryTimestamp
,
ChainNodeSpecificTimeWindowSummary
value
)
{
key
=
generateSummaryValueMapKey
(
summaryTimestamp
);
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
entry
:
value
.
getSummerValueMap
().
entrySet
())
{
if
(
summaryValueMap
.
get
(
key
)
==
null
)
{
summaryValueMap
.
put
(
key
,
new
ChainNodeSpecificTimeWindowSummaryValue
());
}
summaryValueMap
.
get
(
key
).
accumulate
(
entry
.
getValue
());
}
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
}
private
String
generateSummaryValueMapKey
(
long
timeStamp
)
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
timeStamp
));
return
String
.
valueOf
(
calendar
.
get
(
Calendar
.
MONTH
));
}
@Override
public
String
toString
()
{
return
new
Gson
().
toJson
(
this
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificDaySummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainSpecificDaySummary
{
private
Map
<
String
,
ChainNodeSpecificDaySummary
>
chainNodeSpecificHourSummaryMap
;
public
ChainSpecificDaySummary
()
{
chainNodeSpecificHourSummaryMap
=
new
HashMap
<
String
,
ChainNodeSpecificDaySummary
>();
}
public
void
addNodeSummaryResult
(
ChainNodeSpecificDaySummary
chainNodeSpecificHourSummary
)
{
chainNodeSpecificHourSummaryMap
.
put
(
chainNodeSpecificHourSummary
.
getTraceLevelId
(),
chainNodeSpecificHourSummary
);
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
)
{
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
chainNodeSpecificTimeWindowSummaryMap
=
timeSummary
.
getSummaryMap
();
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummary
>
entry
:
chainNodeSpecificTimeWindowSummaryMap
.
entrySet
())
{
if
(
chainNodeSpecificHourSummaryMap
.
get
(
entry
.
getKey
())
==
null
){
chainNodeSpecificHourSummaryMap
.
put
(
entry
.
getKey
(),
new
ChainNodeSpecificDaySummary
());
}
chainNodeSpecificHourSummaryMap
.
get
(
entry
.
getKey
()).
summary
(
timeSummary
.
getSummaryTimestamp
(),
entry
.
getValue
());
}
}
public
void
save
(
Put
put
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificDaySummary
>
entry
:
chainNodeSpecificHourSummaryMap
.
entrySet
())
{
put
.
addColumn
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP
.
COLUMN_FAMILY_NAME
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificHourSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainSpecificHourSummary
{
// key : TraceLevelId
private
Map
<
String
,
ChainNodeSpecificHourSummary
>
chainNodeSpecificHourSummaryMap
;
public
ChainSpecificHourSummary
()
{
chainNodeSpecificHourSummaryMap
=
new
HashMap
<
String
,
ChainNodeSpecificHourSummary
>();
}
public
void
addNodeSummaryResult
(
ChainNodeSpecificHourSummary
chainNodeSpecificHourSummary
)
{
chainNodeSpecificHourSummaryMap
.
put
(
chainNodeSpecificHourSummary
.
getTraceLevelId
(),
chainNodeSpecificHourSummary
);
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
)
{
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
chainNodeSpecificTimeWindowSummaryMap
=
timeSummary
.
getSummaryMap
();
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummary
>
entry
:
chainNodeSpecificTimeWindowSummaryMap
.
entrySet
()){
if
(
chainNodeSpecificHourSummaryMap
.
get
(
entry
.
getKey
())
==
null
){
chainNodeSpecificHourSummaryMap
.
put
(
entry
.
getKey
(),
new
ChainNodeSpecificHourSummary
());
}
chainNodeSpecificHourSummaryMap
.
get
(
entry
.
getKey
()).
summary
(
timeSummary
.
getSummaryTimestamp
(),
entry
.
getValue
());
}
}
public
void
save
(
Put
put
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificHourSummary
>
entry
:
chainNodeSpecificHourSummaryMap
.
entrySet
())
{
put
.
addColumn
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP
.
COLUMN_FAMILY_NAME
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificMinSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainSpecificMinSummary
{
// Key: TraceLevelId
private
Map
<
String
,
ChainNodeSpecificMinSummary
>
chainNodeSpecificMinSummaryMap
;
public
ChainSpecificMinSummary
()
{
this
.
chainNodeSpecificMinSummaryMap
=
new
HashMap
<
String
,
ChainNodeSpecificMinSummary
>();
}
public
void
addNodeSummaryResult
(
ChainNodeSpecificMinSummary
chainNodeSpecificMinSummary
)
{
chainNodeSpecificMinSummaryMap
.
put
(
chainNodeSpecificMinSummary
.
getTraceLevelId
(),
chainNodeSpecificMinSummary
);
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
)
{
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
chainNodeSpecificTimeWindowSummaryMap
=
timeSummary
.
getSummaryMap
();
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummary
>
entry
:
chainNodeSpecificTimeWindowSummaryMap
.
entrySet
())
{
if
(
chainNodeSpecificMinSummaryMap
.
get
(
entry
.
getKey
())
==
null
){
chainNodeSpecificMinSummaryMap
.
put
(
entry
.
getKey
(),
new
ChainNodeSpecificMinSummary
());
}
chainNodeSpecificMinSummaryMap
.
get
(
entry
.
getKey
()).
summary
(
entry
.
getValue
());
}
}
public
void
save
(
Put
put
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificMinSummary
>
entry
:
chainNodeSpecificMinSummaryMap
.
entrySet
())
{
put
.
addColumn
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_INCLUDE_RELATIONSHIP
.
COLUMN_FAMILY_NAME
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSpecificMonthSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.util.Map
;
public
class
ChainSpecificMonthSummary
{
// Key: TraceLevelId
private
Map
<
String
,
ChainNodeSpecificMonthSummary
>
chainNodeSpecificMinSummaryMap
;
public
void
addNodeSummaryResult
(
ChainNodeSpecificMonthSummary
chainNodeSpecificDaySummary
)
{
chainNodeSpecificMinSummaryMap
.
put
(
chainNodeSpecificDaySummary
.
getTraceLevelId
(),
chainNodeSpecificDaySummary
);
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
)
{
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
chainNodeSpecificTimeWindowSummaryMap
=
timeSummary
.
getSummaryMap
();
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummary
>
entry
:
chainNodeSpecificTimeWindowSummaryMap
.
entrySet
())
{
if
(
chainNodeSpecificMinSummaryMap
.
get
(
entry
.
getKey
())
==
null
){
chainNodeSpecificMinSummaryMap
.
put
(
entry
.
getKey
(),
new
ChainNodeSpecificMonthSummary
());
}
chainNodeSpecificMinSummaryMap
.
get
(
entry
.
getKey
()).
summary
(
timeSummary
.
getSummaryTimestamp
(),
entry
.
getValue
());
}
}
public
void
save
(
Put
put
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificMonthSummary
>
entry
:
chainNodeSpecificMinSummaryMap
.
entrySet
())
{
put
.
addColumn
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP
.
COLUMN_FAMILY_NAME
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/entity/ChainSummaryWithRelationship.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.util.HBaseUtil
;
import
com.ai.cloud.skywalking.analysis.chain2summary.po.ChainSpecificTimeSummary
;
import
java.io.IOException
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainSummaryWithRelationship
{
private
String
cid
;
// key : cid + userId + 小时
private
Map
<
String
,
ChainSpecificMinSummary
>
minSummary
;
// key : cid + userId + 天
private
Map
<
String
,
ChainSpecificHourSummary
>
hourSummary
;
// key : cid + userId + 月
private
Map
<
String
,
ChainSpecificDaySummary
>
daySummary
;
// key : cid + userId + 年
private
Map
<
String
,
ChainSpecificMonthSummary
>
monthSummary
;
public
ChainSummaryWithRelationship
(
String
cid
)
{
this
.
cid
=
cid
;
minSummary
=
new
HashMap
<
String
,
ChainSpecificMinSummary
>();
hourSummary
=
new
HashMap
<
String
,
ChainSpecificHourSummary
>();
daySummary
=
new
HashMap
<
String
,
ChainSpecificDaySummary
>();
monthSummary
=
new
HashMap
<
String
,
ChainSpecificMonthSummary
>();
}
public
void
saveToHBase
()
throws
IOException
,
InterruptedException
{
HBaseUtil
.
batchSaveSpecificMinSummary
(
minSummary
);
HBaseUtil
.
batchSaveSpecificHourSummary
(
hourSummary
);
HBaseUtil
.
batchSaveSpecificDaySummary
(
daySummary
);
HBaseUtil
.
batchSaveSpecificMonthSummary
(
monthSummary
);
}
public
void
summary
(
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
loadSummaryIfNecessary
(
cid
,
timeSummary
);
//
minSummary
.
get
(
buildMinSummaryRowKey
(
cid
,
timeSummary
)).
summary
(
timeSummary
);
hourSummary
.
get
(
buildHourSummaryRowKey
(
cid
,
timeSummary
)).
summary
(
timeSummary
);
daySummary
.
get
(
buildDaySummaryRowKey
(
cid
,
timeSummary
)).
summary
(
timeSummary
);
monthSummary
.
get
(
buildMonthSummaryRowKey
(
cid
,
timeSummary
)).
summary
(
timeSummary
);
}
private
void
loadSummaryIfNecessary
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
loadMinSummaryIfNecessary
(
cid
,
timeSummary
);
loadHourSummaryIfNecessary
(
cid
,
timeSummary
);
loadDaySummaryIfNecessary
(
cid
,
timeSummary
);
loadMonthSummaryIfNecessary
(
cid
,
timeSummary
);
}
private
void
loadMonthSummaryIfNecessary
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
String
month_RowKey
=
buildMonthSummaryRowKey
(
cid
,
timeSummary
);
if
(!
monthSummary
.
containsKey
(
month_RowKey
))
{
monthSummary
.
put
(
month_RowKey
,
HBaseUtil
.
loadSpecificMonthSummary
(
month_RowKey
));
}
}
private
void
loadDaySummaryIfNecessary
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
String
day_RowKey
=
buildDaySummaryRowKey
(
cid
,
timeSummary
);
if
(!
daySummary
.
containsKey
(
day_RowKey
))
{
daySummary
.
put
(
day_RowKey
,
HBaseUtil
.
loadSpecificDaySummary
(
day_RowKey
));
}
}
private
void
loadHourSummaryIfNecessary
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
String
hour_RowKey
=
buildHourSummaryRowKey
(
cid
,
timeSummary
);
if
(!
hourSummary
.
containsKey
(
hour_RowKey
))
{
hourSummary
.
put
(
hour_RowKey
,
HBaseUtil
.
loadSpecificHourSummary
(
hour_RowKey
));
}
}
private
void
loadMinSummaryIfNecessary
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
throws
IOException
{
String
min_RowKey
=
buildMinSummaryRowKey
(
cid
,
timeSummary
);
if
(!
minSummary
.
containsKey
(
min_RowKey
))
{
minSummary
.
put
(
min_RowKey
,
HBaseUtil
.
loadSpecificMinSummary
(
min_RowKey
));
}
}
// 月统计是以年作为RowKey的
private
static
String
buildMonthSummaryRowKey
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
{
return
cid
+
"-"
+
timeSummary
.
getUserId
()
+
"-"
+
timeSummary
.
getYearKey
();
}
// 天统计是以月作为RowKey的
private
static
String
buildDaySummaryRowKey
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
{
return
cid
+
"-"
+
timeSummary
.
getUserId
()
+
"-"
+
timeSummary
.
getMonthKey
();
}
// 小时统计是以天作为RowKey的
private
static
String
buildHourSummaryRowKey
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
{
return
cid
+
"-"
+
timeSummary
.
getUserId
()
+
"-"
+
timeSummary
.
getDayKey
();
}
// 分钟统计是以小时作为RowKey的
private
static
String
buildMinSummaryRowKey
(
String
cid
,
ChainSpecificTimeSummary
timeSummary
)
{
return
cid
+
"-"
+
timeSummary
.
getUserId
()
+
"-"
+
timeSummary
.
getHourKey
();
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain2summary/po/ChainSpecificTimeSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chain2summary.po
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
org.apache.hadoop.io.Writable
;
import
java.io.DataInput
;
import
java.io.DataOutput
;
import
java.io.IOException
;
import
java.text.ParseException
;
import
java.text.SimpleDateFormat
;
import
java.util.Calendar
;
import
java.util.Date
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainSpecificTimeSummary
implements
Writable
{
private
String
cId
;
private
String
userId
;
private
String
entranceNodeToken
;
//key : TraceLevelId
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
summaryMap
;
private
long
summaryTimestamp
;
public
ChainSpecificTimeSummary
()
{
}
public
ChainSpecificTimeSummary
(
String
rowKey
)
throws
ParseException
{
String
[]
splitValue
=
rowKey
.
split
(
"-"
);
this
.
cId
=
splitValue
[
0
];
this
.
userId
=
splitValue
[
1
];
this
.
summaryTimestamp
=
new
SimpleDateFormat
(
"yyyy/MM/dd HH:mm:ss"
).
parse
(
splitValue
[
2
]).
getTime
();
summaryMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummary
>();
}
@Override
public
void
write
(
DataOutput
out
)
throws
IOException
{
out
.
write
(
new
Gson
().
toJson
(
this
).
getBytes
());
}
@Override
public
void
readFields
(
DataInput
in
)
throws
IOException
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
in
.
readLine
());
cId
=
jsonObject
.
get
(
"cId"
).
getAsString
();
userId
=
jsonObject
.
get
(
"userId"
).
getAsString
();
if
(
jsonObject
.
get
(
"entranceNodeToken"
)
==
null
)
{
throw
new
IOException
(
"No entryNode Token CId["
+
cId
+
"]"
);
}
entranceNodeToken
=
jsonObject
.
get
(
"entranceNodeToken"
).
getAsString
();
summaryMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summaryMap"
).
toString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>>()
{
}.
getType
());
}
public
void
addChainNodeSummaryResult
(
String
summaryResult
)
{
ChainNodeSpecificTimeWindowSummary
chainNodeSpecificTimeWindowSummary
=
new
Gson
().
fromJson
(
summaryResult
,
ChainNodeSpecificTimeWindowSummary
.
class
);
if
(
"0"
.
equals
(
chainNodeSpecificTimeWindowSummary
.
getTraceLevelId
()))
{
this
.
entranceNodeToken
=
chainNodeSpecificTimeWindowSummary
.
getNodeToken
();
}
summaryMap
.
put
(
chainNodeSpecificTimeWindowSummary
.
getTraceLevelId
(),
chainNodeSpecificTimeWindowSummary
);
}
public
String
buildMapperKey
()
{
return
userId
+
":"
+
entranceNodeToken
;
}
public
String
getcId
()
{
return
cId
;
}
public
String
getHourKey
()
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
summaryTimestamp
));
return
calendar
.
get
(
Calendar
.
YEAR
)
+
"/"
+
(
calendar
.
get
(
Calendar
.
MONTH
)
+
1
)
+
"/"
+
calendar
.
get
(
Calendar
.
DAY_OF_MONTH
)
+
" "
+
calendar
.
get
(
Calendar
.
HOUR
)
+
":00:00"
;
}
public
String
getDayKey
()
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
summaryTimestamp
));
return
calendar
.
get
(
Calendar
.
YEAR
)
+
"-"
+
(
calendar
.
get
(
Calendar
.
MONTH
)
+
1
)
+
"-"
+
(
calendar
.
get
(
Calendar
.
DAY_OF_MONTH
)
+
1
);
}
public
String
getMonthKey
()
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
summaryTimestamp
));
return
calendar
.
get
(
Calendar
.
YEAR
)
+
"-"
+
(
calendar
.
get
(
Calendar
.
MONTH
)
+
1
);
}
public
String
getYearKey
()
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
summaryTimestamp
));
return
String
.
valueOf
(
calendar
.
get
(
Calendar
.
YEAR
));
}
public
String
getUserId
()
{
return
userId
;
}
public
Map
<
String
,
ChainNodeSpecificTimeWindowSummary
>
getSummaryMap
()
{
return
summaryMap
;
}
public
long
getSummaryTimestamp
()
{
return
summaryTimestamp
;
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/ChainBuildMapper.java
浏览文件 @
13421da7
...
...
@@ -82,7 +82,7 @@ public class ChainBuildMapper extends TableMapper<Text, ChainInfo> {
filter
.
doFilter
(
entry
.
getValue
(),
chainNode
,
costMap
);
chainInfo
.
addNodes
(
chainNode
);
}
//
chainInfo.generateChainToken();
chainInfo
.
generateChainToken
();
//HBaseUtil.saveCidTidMapping(key, chainInfo);
return
chainInfo
;
}
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/ChainBuildReducer.java
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chainbuild
;
import
com.ai.cloud.skywalking.analysis.chainbuild.entity.CallChainTree
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.config.ConfigInitializer
;
import
org.apache.hadoop.hbase.util.Bytes
;
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/CallChainTree.java
→
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/
entity/
CallChainTree.java
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chainbuild
;
package
com.ai.cloud.skywalking.analysis.chainbuild
.entity
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.CallChainTreeNode
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainNode
;
import
com.ai.cloud.skywalking.analysis.chainbuild.util.HBaseUtil
;
...
...
@@ -17,6 +16,8 @@ public class CallChainTree {
private
String
callEntrance
;
private
String
treeId
;
//存放已经合并过的调用链ID
private
List
<
String
>
hasBeenMergedChainIds
;
...
...
@@ -35,7 +36,7 @@ public class CallChainTree {
}
public
static
CallChainTree
load
(
String
callEntrance
)
throws
IOException
{
CallChainTree
chain
=
HBaseUtil
.
load
MergedCallChain
(
callEntrance
);
CallChainTree
chain
=
HBaseUtil
.
load
CallChainTree
(
callEntrance
);
chain
.
hasBeenMergedChainIds
.
addAll
(
HBaseUtil
.
loadHasBeenMergeChainIds
(
callEntrance
));
if
(
chain
==
null
)
{
chain
=
new
CallChainTree
(
callEntrance
);
...
...
@@ -57,14 +58,14 @@ public class CallChainTree {
}
}
hasBeenMergedChainIds
.
add
(
chainInfo
.
getC
hainToken
());
combineChains
.
put
(
chainInfo
.
getC
hainToken
(),
chainInfo
);
hasBeenMergedChainIds
.
add
(
chainInfo
.
getC
ID
());
combineChains
.
put
(
chainInfo
.
getC
ID
(),
chainInfo
);
}
public
void
summary
(
ChainInfo
chainInfo
)
{
public
void
summary
(
ChainInfo
chainInfo
)
throws
IOException
{
for
(
ChainNode
node
:
chainInfo
.
getNodes
())
{
CallChainTreeNode
callChainTreeNode
=
nodes
.
get
(
node
.
getTraceLevelId
());
callChainTreeNode
.
summary
(
node
);
callChainTreeNode
.
summary
(
treeId
,
node
);
}
}
...
...
@@ -77,6 +78,11 @@ public class CallChainTree {
}
HBaseUtil
.
saveCallChainTree
(
this
);
//save summary value
for
(
CallChainTreeNode
entry
:
nodes
.
values
())
{
entry
.
saveSummaryResultToHBase
();
}
}
public
String
getCallEntrance
()
{
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/CallChainTreeNode.java
0 → 100644
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chainbuild.entity
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainNode
;
import
com.ai.cloud.skywalking.analysis.chainbuild.util.HBaseUtil
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
com.google.gson.GsonBuilder
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.annotations.Expose
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.io.IOException
;
import
java.util.*
;
public
class
CallChainTreeNode
{
@Expose
private
String
traceLevelId
;
@Expose
private
String
viewPointId
;
// key: treeId + 小时
private
Map
<
String
,
ChainNodeSpecificMinSummary
>
chainNodeContainer
;
public
CallChainTreeNode
(
ChainNode
node
)
{
this
.
traceLevelId
=
node
.
getTraceLevelId
();
chainNodeContainer
.
put
(
node
.
getNodeToken
(),
new
ChainNodeSpecificMinSummary
());
}
public
CallChainTreeNode
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
viewPointId
=
jsonObject
.
get
(
"viewPointId"
).
getAsString
();
}
public
void
mergeIfNess
(
ChainNode
node
)
{
if
(!
chainNodeContainer
.
containsKey
(
node
.
getNodeToken
()))
{
chainNodeContainer
.
put
(
node
.
getNodeToken
(),
new
ChainNodeSpecificMinSummary
());
}
}
public
void
summary
(
String
treeId
,
ChainNode
node
)
throws
IOException
{
Calendar
calendar
=
Calendar
.
getInstance
();
calendar
.
setTime
(
new
Date
(
node
.
getStartDate
()));
// summary min
String
keyOfMinSummaryTable
=
generateKeyOfMinSummaryTable
(
treeId
,
calendar
);
ChainNodeSpecificMinSummary
minSummary
=
chainNodeContainer
.
get
(
keyOfMinSummaryTable
);
if
(
minSummary
==
null
)
{
minSummary
=
HBaseUtil
.
loadSpecificMinSummary
(
keyOfMinSummaryTable
,
node
.
getNodeToken
());
chainNodeContainer
.
put
(
keyOfMinSummaryTable
,
minSummary
);
}
minSummary
.
summary
(
String
.
valueOf
(
calendar
.
get
(
Calendar
.
MINUTE
)
+
1
),
node
);
}
private
String
generateKeyOfMinSummaryTable
(
String
treeId
,
Calendar
calendar
)
{
return
treeId
+
"/"
+
calendar
.
get
(
Calendar
.
YEAR
)
+
"-"
+
calendar
.
get
(
Calendar
.
MONTH
)
+
"-"
+
calendar
.
get
(
Calendar
.
DAY_OF_MONTH
)
+
" "
+
calendar
.
get
(
Calendar
.
HOUR
)
+
":00:00"
;
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
}
@Override
public
String
toString
()
{
return
new
GsonBuilder
().
excludeFieldsWithoutExposeAnnotation
().
create
().
toJson
(
this
);
}
public
void
saveSummaryResultToHBase
()
{
List
<
Put
>
puts
=
new
ArrayList
<
Put
>();
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificMinSummary
>
entry
:
chainNodeContainer
.
entrySet
())
{
Put
put
=
new
Put
(
entry
.
getKey
().
getBytes
());
put
.
addColumn
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY
.
COLUMN_FAMILY_NAME
.
getBytes
()
,
traceLevelId
.
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
puts
.
add
(
put
);
}
HBaseUtil
.
batchSaveMinSummaryResult
(
puts
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/ChainNodeForSummary.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chainbuild.entity
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainNode
;
public
class
ChainNodeForSummary
{
private
String
traceLevelId
;
private
String
viewPointId
;
public
ChainNodeForSummary
(
ChainNode
node
)
{
this
.
traceLevelId
=
node
.
getTraceLevelId
();
this
.
viewPointId
=
node
.
getViewPoint
();
}
public
void
summary
(
ChainNode
node
)
{
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain
2summary
/entity/ChainNodeSpecificMinSummary.java
→
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chain
build
/entity/ChainNodeSpecificMinSummary.java
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chain
2summary
.entity
;
package
com.ai.cloud.skywalking.analysis.chain
build
.entity
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummary
;
import
com.ai.cloud.skywalking.analysis.categorize2chain.entity.ChainNodeSpecificTimeWindowSummaryValue
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainNode
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
org.apache.hadoop.hbase.client.Put
;
import
java.util.HashMap
;
import
java.util.Map
;
public
class
ChainNodeSpecificMinSummary
{
private
String
traceLevelId
;
// key: 分钟 value: 统计结果
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
summerValueMap
;
public
ChainNodeSpecificMinSummary
(){
summerValueMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>();
}
// key : 分钟
private
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
summaryValueMap
;
public
ChainNodeSpecificMinSummary
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
summerValueMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summerValueMap"
).
toString
(),
summaryValueMap
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"summaryValueMap"
).
toString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>>()
{
}.
getType
());
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
public
ChainNodeSpecificMinSummary
()
{
summaryValueMap
=
new
HashMap
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>()
;
}
public
void
summary
(
ChainNodeSpecificTimeWindowSummary
value
)
{
for
(
Map
.
Entry
<
String
,
ChainNodeSpecificTimeWindowSummaryValue
>
entry
:
value
.
getSummerValueMap
().
entrySet
()){
if
(
summerValueMap
.
get
(
entry
.
getKey
())
==
null
){
summerValueMap
.
put
(
entry
.
getKey
(),
new
ChainNodeSpecificTimeWindowSummaryValue
());
}
summerValueMap
.
get
(
entry
.
getKey
()).
accumulate
(
entry
.
getValue
());
public
void
summary
(
String
minute
,
ChainNode
node
)
{
ChainNodeSpecificTimeWindowSummaryValue
summarValue
=
summaryValueMap
.
get
(
minute
);
if
(
summarValue
==
null
)
{
summarValue
=
new
ChainNodeSpecificTimeWindowSummaryValue
();
summaryValueMap
.
put
(
minute
,
summarValue
);
}
summarValue
.
summary
(
node
);
}
@Override
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/entity/ChainNodeSpecificTimeWindowSummaryValue.java
0 → 100644
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chainbuild.entity
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainNode
;
public
class
ChainNodeSpecificTimeWindowSummaryValue
{
private
long
totalCall
;
private
long
totalCostTime
;
private
long
correctNumber
;
private
long
humanInterruptionNumber
;
public
ChainNodeSpecificTimeWindowSummaryValue
()
{
totalCall
=
0
;
totalCostTime
=
0
;
correctNumber
=
0
;
humanInterruptionNumber
=
0
;
}
public
void
summary
(
ChainNode
node
)
{
totalCall
++;
if
(
node
.
getStatus
()
==
ChainNode
.
NodeStatus
.
NORMAL
)
{
correctNumber
++;
}
if
(
node
.
getStatus
()
==
ChainNode
.
NodeStatus
.
HUMAN_INTERRUPTION
)
{
humanInterruptionNumber
++;
}
totalCostTime
+=
node
.
getCost
();
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/CallChainTreeNode.java
已删除
100644 → 0
浏览文件 @
e024a329
package
com.ai.cloud.skywalking.analysis.chainbuild.po
;
import
com.ai.cloud.skywalking.analysis.chainbuild.entity.ChainNodeForSummary
;
import
com.google.gson.Gson
;
import
com.google.gson.JsonObject
;
import
com.google.gson.JsonParser
;
import
com.google.gson.reflect.TypeToken
;
import
java.util.Map
;
public
class
CallChainTreeNode
{
private
String
traceLevelId
;
// key: nodeToken
private
Map
<
String
,
ChainNodeForSummary
>
chainNodeContainer
;
public
CallChainTreeNode
(
ChainNode
node
)
{
this
.
traceLevelId
=
node
.
getTraceLevelId
();
chainNodeContainer
.
put
(
node
.
getNodeToken
(),
new
ChainNodeForSummary
(
node
));
}
public
CallChainTreeNode
(
String
originData
)
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
originData
);
traceLevelId
=
jsonObject
.
get
(
"traceLevelId"
).
getAsString
();
chainNodeContainer
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"chainNodeContainer"
).
getAsString
(),
new
TypeToken
<
Map
<
String
,
ChainNodeForSummary
>>()
{
}.
getType
());
}
public
void
mergeIfNess
(
ChainNode
node
)
{
if
(!
chainNodeContainer
.
containsKey
(
node
.
getNodeToken
()))
{
chainNodeContainer
.
put
(
node
.
getNodeToken
(),
new
ChainNodeForSummary
(
node
));
}
}
public
void
summary
(
ChainNode
node
)
{
ChainNodeForSummary
chainNode
=
chainNodeContainer
.
get
(
node
.
getNodeToken
());
chainNode
.
summary
(
node
);
}
public
String
getTraceLevelId
()
{
return
traceLevelId
;
}
@Override
public
String
toString
()
{
return
new
Gson
().
toJson
(
this
);
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/ChainInfo.java
浏览文件 @
13421da7
...
...
@@ -15,130 +15,131 @@ import java.util.ArrayList;
import
java.util.List
;
public
class
ChainInfo
implements
Writable
{
private
String
cid
;
private
ChainStatus
chainStatus
=
ChainStatus
.
NORMAL
;
private
List
<
ChainNode
>
nodes
;
private
String
userId
=
null
;
private
ChainNode
firstChainNode
;
private
long
startDate
;
private
String
chainToken
;
public
ChainInfo
(
String
userId
)
{
super
();
this
.
userId
=
userId
;
}
public
ChainInfo
()
{
this
.
nodes
=
new
ArrayList
<
ChainNode
>();
}
@Override
public
void
write
(
DataOutput
out
)
throws
IOException
{
out
.
write
(
new
Gson
().
toJson
(
this
).
getBytes
());
}
@Override
public
void
readFields
(
DataInput
in
)
throws
IOException
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
in
.
readLine
());
cid
=
jsonObject
.
get
(
"cid"
).
getAsString
();
chainStatus
=
ChainStatus
.
convert
(
jsonObject
.
get
(
"chainStatus"
)
.
getAsCharacter
());
nodes
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"nodes"
),
new
TypeToken
<
List
<
ChainNode
>>()
{
}.
getType
());
userId
=
jsonObject
.
get
(
"userId"
).
getAsString
();
}
public
String
getCID
()
{
return
cid
;
}
public
String
getEntranceNodeToken
()
{
if
(
firstChainNode
==
null
)
{
return
""
;
}
else
{
return
firstChainNode
.
getNodeToken
();
}
}
public
ChainStatus
getChainStatus
()
{
return
chainStatus
;
}
public
void
setChainStatus
(
ChainStatus
chainStatus
)
{
this
.
chainStatus
=
chainStatus
;
}
public
void
addNodes
(
ChainNode
chainNode
)
{
this
.
nodes
.
add
(
0
,
chainNode
);
if
(
chainNode
.
getStatus
()
==
ChainNode
.
NodeStatus
.
ABNORMAL
||
chainNode
.
getStatus
()
==
ChainNode
.
NodeStatus
.
HUMAN_INTERRUPTION
)
{
chainStatus
=
ChainStatus
.
ABNORMAL
;
}
if
(
userId
==
null
)
{
userId
=
chainNode
.
getUserId
();
}
if
((
chainNode
.
getParentLevelId
()
==
null
||
chainNode
.
getParentLevelId
().
length
()
==
0
)
&&
chainNode
.
getLevelId
()
==
0
)
{
firstChainNode
=
chainNode
;
startDate
=
chainNode
.
getStartDate
();
cid
=
firstChainNode
.
getViewPoint
();
}
}
public
List
<
ChainNode
>
getNodes
()
{
return
nodes
;
}
public
String
getUserId
()
{
return
userId
;
}
public
void
setUserId
(
String
userId
)
{
this
.
userId
=
userId
;
}
public
String
getChainToken
()
{
return
chainToken
;
}
public
void
saveToHBase
(
Put
put
)
{
}
public
enum
ChainStatus
{
NORMAL
(
'N'
),
ABNORMAL
(
'A'
);
private
char
value
;
ChainStatus
(
char
value
)
{
this
.
value
=
value
;
}
public
static
ChainStatus
convert
(
char
value
)
{
switch
(
value
)
{
case
'N'
:
return
NORMAL
;
case
'A'
:
return
ABNORMAL
;
default
:
throw
new
IllegalStateException
(
"Failed to convert["
+
value
+
"]"
);
}
}
@Override
public
String
toString
()
{
return
value
+
""
;
}
}
public
void
setCID
(
String
cid
)
{
this
.
cid
=
cid
;
}
public
long
getStartDate
()
{
return
startDate
;
}
private
String
callEntrance
;
private
String
cid
;
private
ChainStatus
chainStatus
=
ChainStatus
.
NORMAL
;
private
List
<
ChainNode
>
nodes
;
private
String
userId
=
null
;
private
ChainNode
firstChainNode
;
private
long
startDate
;
public
ChainInfo
(
String
userId
)
{
super
();
this
.
userId
=
userId
;
}
public
ChainInfo
()
{
this
.
nodes
=
new
ArrayList
<
ChainNode
>();
}
@Override
public
void
write
(
DataOutput
out
)
throws
IOException
{
out
.
write
(
new
Gson
().
toJson
(
this
).
getBytes
());
}
@Override
public
void
readFields
(
DataInput
in
)
throws
IOException
{
JsonObject
jsonObject
=
(
JsonObject
)
new
JsonParser
().
parse
(
in
.
readLine
());
cid
=
jsonObject
.
get
(
"cid"
).
getAsString
();
chainStatus
=
ChainStatus
.
convert
(
jsonObject
.
get
(
"chainStatus"
)
.
getAsCharacter
());
nodes
=
new
Gson
().
fromJson
(
jsonObject
.
get
(
"nodes"
),
new
TypeToken
<
List
<
ChainNode
>>()
{
}.
getType
());
userId
=
jsonObject
.
get
(
"userId"
).
getAsString
();
}
public
String
getCID
()
{
return
cid
;
}
public
String
getEntranceNodeToken
()
{
if
(
firstChainNode
==
null
)
{
return
""
;
}
else
{
return
firstChainNode
.
getNodeToken
();
}
}
public
void
generateChainToken
()
{
StringBuilder
chainTokenDesc
=
new
StringBuilder
();
for
(
ChainNode
node
:
nodes
)
{
chainTokenDesc
.
append
(
node
.
getParentLevelId
()
+
"."
+
node
.
getLevelId
()
+
"-"
+
node
.
getNodeToken
()
+
";"
);
}
this
.
cid
=
TokenGenerator
.
generateCID
(
chainTokenDesc
.
toString
());
}
public
ChainStatus
getChainStatus
()
{
return
chainStatus
;
}
public
void
setChainStatus
(
ChainStatus
chainStatus
)
{
this
.
chainStatus
=
chainStatus
;
}
public
void
addNodes
(
ChainNode
chainNode
)
{
this
.
nodes
.
add
(
0
,
chainNode
);
if
(
chainNode
.
getStatus
()
==
ChainNode
.
NodeStatus
.
ABNORMAL
||
chainNode
.
getStatus
()
==
ChainNode
.
NodeStatus
.
HUMAN_INTERRUPTION
)
{
chainStatus
=
ChainStatus
.
ABNORMAL
;
}
if
(
userId
==
null
)
{
userId
=
chainNode
.
getUserId
();
}
if
((
chainNode
.
getParentLevelId
()
==
null
||
chainNode
.
getParentLevelId
().
length
()
==
0
)
&&
chainNode
.
getLevelId
()
==
0
)
{
firstChainNode
=
chainNode
;
startDate
=
chainNode
.
getStartDate
();
cid
=
firstChainNode
.
getViewPoint
();
}
}
public
List
<
ChainNode
>
getNodes
()
{
return
nodes
;
}
public
String
getUserId
()
{
return
userId
;
}
public
void
saveToHBase
(
Put
put
)
{
}
public
enum
ChainStatus
{
NORMAL
(
'N'
),
ABNORMAL
(
'A'
);
private
char
value
;
ChainStatus
(
char
value
)
{
this
.
value
=
value
;
}
public
static
ChainStatus
convert
(
char
value
)
{
switch
(
value
)
{
case
'N'
:
return
NORMAL
;
case
'A'
:
return
ABNORMAL
;
default
:
throw
new
IllegalStateException
(
"Failed to convert["
+
value
+
"]"
);
}
}
@Override
public
String
toString
()
{
return
value
+
""
;
}
}
public
void
setCID
(
String
cid
)
{
this
.
cid
=
cid
;
}
public
long
getStartDate
()
{
return
startDate
;
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/po/ChainNode.java
浏览文件 @
13421da7
...
...
@@ -11,6 +11,7 @@ public class ChainNode {
@Expose
private
String
businessKey
;
@Expose
private
long
cost
;
private
NodeStatus
status
;
@Expose
...
...
@@ -19,6 +20,7 @@ public class ChainNode {
private
int
levelId
;
@Expose
private
String
callType
;
@Expose
private
long
startDate
;
@Expose
private
String
userId
;
...
...
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/chainbuild/util/HBaseUtil.java
浏览文件 @
13421da7
package
com.ai.cloud.skywalking.analysis.chainbuild.util
;
import
com.ai.cloud.skywalking.analysis.chain2summary.entity.*
;
import
com.ai.cloud.skywalking.analysis.chainbuild.CallChainTree
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.CallChainTreeNode
;
import
com.ai.cloud.skywalking.analysis.chainbuild.po.ChainInfo
;
import
com.ai.cloud.skywalking.analysis.chainbuild.entity.CallChainTree
;
import
com.ai.cloud.skywalking.analysis.chainbuild.entity.CallChainTreeNode
;
import
com.ai.cloud.skywalking.analysis.chainbuild.entity.ChainNodeSpecificMinSummary
;
import
com.ai.cloud.skywalking.analysis.config.Config
;
import
com.ai.cloud.skywalking.analysis.config.HBaseTableMetaData
;
import
com.google.gson.Gson
;
...
...
@@ -30,32 +29,8 @@ public class HBaseUtil {
try
{
initHBaseClient
();
// createTableIfNeed(HBaseTableMetaData.TABLE_CID_TID_MAPPING.TABLE_NAME,
// HBaseTableMetaData.TABLE_CID_TID_MAPPING.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CALL_CHAIN_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CALL_CHAIN_RELATIONSHIP.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_DETAIL.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_DETAIL.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_ONE_MINUTE_SUMMARY_INCLUDE_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_ONE_MINUTE_SUMMARY_INCLUDE_RELATIONSHIP.COLUMN_FAMILY_NAME);
//
// createTableIfNeed(HBaseTableMetaData.TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP.TABLE_NAME,
// HBaseTableMetaData.TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP.COLUMN_FAMILY_NAME);
createTableIfNeed
(
HBaseTableMetaData
.
TABLE_MERGED_CHAIN_DETAIL
.
TABLE_NAME
,
HBaseTableMetaData
.
TABLE_MERGED_CHAIN_DETAIL
.
COLUMN_FAMILY_NAME
);
createTableIfNeed
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN_TREE_DETAIL
.
TABLE_NAME
,
HBaseTableMetaData
.
TABLE_CALL_CHAIN_TREE_DETAIL
.
COLUMN_FAMILY_NAME
);
createTableIfNeed
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN_TREE_ID_AND_CID_MAPPING
.
TABLE_NAME
,
HBaseTableMetaData
.
TABLE_CALL_CHAIN_TREE_ID_AND_CID_MAPPING
.
COLUMN_FAMILY_NAME
);
...
...
@@ -87,196 +62,32 @@ public class HBaseUtil {
}
}
public
static
boolean
saveCidTidMapping
(
String
traceId
,
ChainInfo
chainInfo
)
{
Table
table
=
null
;
try
{
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CID_TID_MAPPING
.
TABLE_NAME
));
}
catch
(
IOException
e
)
{
logger
.
error
(
"Cannot found table["
+
HBaseTableMetaData
.
TABLE_CID_TID_MAPPING
.
TABLE_NAME
+
"]"
,
e
);
}
Put
put
=
new
Put
(
Bytes
.
toBytes
(
traceId
));
put
.
addColumn
(
Bytes
.
toBytes
(
HBaseTableMetaData
.
TABLE_CID_TID_MAPPING
.
COLUMN_FAMILY_NAME
),
Bytes
.
toBytes
(
HBaseTableMetaData
.
TABLE_CID_TID_MAPPING
.
CID_COLUMN_NAME
),
Bytes
.
toBytes
(
chainInfo
.
getCID
()));
try
{
table
.
put
(
put
);
if
(
logger
.
isDebugEnabled
())
{
logger
.
debug
(
"Insert data[RowKey:{}] success."
,
put
.
getId
());
}
}
catch
(
IOException
e
)
{
logger
.
error
(
"Insert data [Rowkey:{}] failed."
,
put
.
getId
(),
e
);
return
false
;
}
return
true
;
}
public
static
void
saveChainRelationship
(
Put
put
)
throws
IOException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN_RELATIONSHIP
.
TABLE_NAME
));
table
.
put
(
put
);
if
(
logger
.
isDebugEnabled
())
{
logger
.
debug
(
"Insert data[RowKey:{}] success."
,
put
.
getId
());
}
}
public
static
void
batchSaveChainSpecificTimeWindowSummary
(
List
<
Put
>
puts
)
throws
IOException
,
InterruptedException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_EXCLUDE_RELATIONSHIP
.
TABLE_NAME
));
Object
[]
resultArrays
=
new
Object
[
puts
.
size
()];
table
.
batch
(
puts
,
resultArrays
);
for
(
Object
result
:
resultArrays
)
{
if
(
result
==
null
)
{
logger
.
error
(
"Failed to save chain specificTimeWindows Summary."
);
}
}
}
public
static
void
saveChainDetails
(
List
<
Put
>
puts
)
throws
IOException
,
InterruptedException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_DETAIL
.
TABLE_NAME
));
if
(
puts
!=
null
&&
puts
.
size
()
>
0
)
{
Object
[]
resultArrays
=
new
Object
[
puts
.
size
()];
table
.
batch
(
puts
,
resultArrays
);
for
(
Object
result
:
resultArrays
)
{
if
(
result
==
null
)
{
logger
.
error
(
"Failed to save chain specificTimeWindows Summary."
);
}
}
}
}
public
static
ChainSpecificMinSummary
loadSpecificMinSummary
(
String
key
)
throws
IOException
{
ChainSpecificMinSummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
));
public
static
ChainNodeSpecificMinSummary
loadSpecificMinSummary
(
String
key
,
String
qualifier
)
throws
IOException
{
ChainNodeSpecificMinSummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY
.
TABLE_NAME
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
if
(
r
.
rawCells
().
length
==
0
)
{
return
null
;
}
result
=
new
ChainSpecificMinSummary
();
for
(
Cell
cell
:
r
.
rawCells
())
{
if
(
cell
.
getValueArray
().
length
>
0
)
result
.
addNodeSummaryResult
(
new
ChainNodeSpecificMinSummary
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
())));
}
return
result
;
}
public
static
ChainSpecificHourSummary
loadSpecificHourSummary
(
String
key
)
throws
IOException
{
ChainSpecificHourSummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
if
(
r
.
rawCells
().
length
==
0
)
{
return
null
;
}
result
=
new
ChainSpecificHourSummary
();
for
(
Cell
cell
:
r
.
rawCells
())
{
if
(
cell
.
getValueArray
().
length
>
0
)
result
.
addNodeSummaryResult
(
new
ChainNodeSpecificHourSummary
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
())));
}
return
result
;
}
public
static
ChainSpecificDaySummary
loadSpecificDaySummary
(
String
key
)
throws
IOException
{
ChainSpecificDaySummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
Cell
cell
=
r
.
getColumnLatestCell
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY
.
COLUMN_FAMILY_NAME
.
getBytes
(),
qualifier
.
getBytes
());
if
(
r
.
rawCells
().
length
==
0
)
{
return
null
;
}
result
=
new
ChainSpecificDaySummary
();
for
(
Cell
cell
:
r
.
rawCells
())
{
if
(
cell
.
getValueArray
().
length
>
0
)
result
.
addNodeSummaryResult
(
new
ChainNodeSpecificDaySummary
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
())));
if
(
cell
.
getValueArray
().
length
>
0
)
{
result
=
new
ChainNodeSpecificMinSummary
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
()));
}
else
{
result
=
new
ChainNodeSpecificMinSummary
();
}
return
result
;
}
public
static
ChainSpecificMonthSummary
loadSpecificMonthSummary
(
String
key
)
throws
IOException
{
ChainSpecificMonthSummary
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
key
));
Result
r
=
table
.
get
(
g
);
if
(
r
.
rawCells
().
length
==
0
)
{
return
null
;
}
result
=
new
ChainSpecificMonthSummary
();
for
(
Cell
cell
:
r
.
rawCells
())
{
if
(
cell
.
getValueArray
().
length
>
0
)
result
.
addNodeSummaryResult
(
new
ChainNodeSpecificMonthSummary
(
Bytes
.
toString
(
cell
.
getValueArray
(),
cell
.
getValueOffset
(),
cell
.
getValueLength
())));
}
return
result
;
}
public
static
void
batchSaveSpecificMinSummary
(
Map
<
String
,
ChainSpecificMinSummary
>
minSummary
)
throws
IOException
,
InterruptedException
{
List
<
Put
>
puts
=
new
ArrayList
<
Put
>();
for
(
Map
.
Entry
<
String
,
ChainSpecificMinSummary
>
entry
:
minSummary
.
entrySet
())
{
Put
put
=
new
Put
(
entry
.
getKey
().
getBytes
());
entry
.
getValue
().
save
(
put
);
puts
.
add
(
put
);
}
batchSavePuts
(
puts
,
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
);
}
public
static
void
batchSaveSpecificDaySummary
(
Map
<
String
,
ChainSpecificDaySummary
>
daySummaryMap
)
throws
IOException
,
InterruptedException
{
List
<
Put
>
puts
=
new
ArrayList
<
Put
>();
for
(
Map
.
Entry
<
String
,
ChainSpecificDaySummary
>
entry
:
daySummaryMap
.
entrySet
())
{
Put
put
=
new
Put
(
entry
.
getKey
().
getBytes
());
entry
.
getValue
().
save
(
put
);
puts
.
add
(
put
);
}
batchSavePuts
(
puts
,
HBaseTableMetaData
.
TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
);
}
public
static
void
batchSaveSpecificHourSummary
(
Map
<
String
,
ChainSpecificHourSummary
>
hourSummaryMap
)
throws
IOException
,
InterruptedException
{
List
<
Put
>
puts
=
new
ArrayList
<
Put
>();
for
(
Map
.
Entry
<
String
,
ChainSpecificHourSummary
>
entry
:
hourSummaryMap
.
entrySet
())
{
Put
put
=
new
Put
(
entry
.
getKey
().
getBytes
());
entry
.
getValue
().
save
(
put
);
puts
.
add
(
put
);
}
batchSavePuts
(
puts
,
HBaseTableMetaData
.
TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
);
}
public
static
void
batchSaveSpecificMonthSummary
(
Map
<
String
,
ChainSpecificMonthSummary
>
monthSummaryMap
)
throws
IOException
,
InterruptedException
{
List
<
Put
>
puts
=
new
ArrayList
<
Put
>();
for
(
Map
.
Entry
<
String
,
ChainSpecificMonthSummary
>
entry
:
monthSummaryMap
.
entrySet
())
{
Put
put
=
new
Put
(
entry
.
getKey
().
getBytes
());
entry
.
getValue
().
save
(
put
);
puts
.
add
(
put
);
}
batchSavePuts
(
puts
,
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP
.
TABLE_NAME
);
}
private
static
void
batchSavePuts
(
List
<
Put
>
puts
,
String
tableName
)
throws
IOException
,
InterruptedException
{
Object
[]
resultArrays
=
new
Object
[
puts
.
size
()];
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
tableName
));
table
.
batch
(
puts
,
resultArrays
);
for
(
Object
result
:
resultArrays
)
{
if
(
result
==
null
)
{
logger
.
error
(
"Failed to save chain specific Summary"
);
}
}
}
public
static
CallChainTree
loadMergedCallChain
(
String
callEntrance
)
throws
IOException
{
public
static
CallChainTree
loadCallChainTree
(
String
callEntrance
)
throws
IOException
{
CallChainTree
result
=
null
;
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_
MERGED_CHAIN
_DETAIL
.
TABLE_NAME
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_
CALL_CHAIN_TREE
_DETAIL
.
TABLE_NAME
));
Get
g
=
new
Get
(
Bytes
.
toBytes
(
callEntrance
));
Result
r
=
table
.
get
(
g
);
if
(
r
.
rawCells
().
length
==
0
)
{
...
...
@@ -298,7 +109,7 @@ public class HBaseUtil {
callChainTreePut
.
addColumn
(
HBaseTableMetaData
.
TABLE_CALL_CHAIN_TREE_DETAIL
.
COLUMN_FAMILY_NAME
.
getBytes
(),
entry
.
getKey
().
getBytes
(),
entry
.
getValue
().
toString
().
getBytes
());
}
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_
MERGED_CHAIN
_DETAIL
.
TABLE_NAME
));
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_
CALL_CHAIN_TREE
_DETAIL
.
TABLE_NAME
));
table
.
put
(
callChainTreePut
);
// save relationship
Put
treeIdCidMappingPut
=
new
Put
(
callChainTree
.
getCallEntrance
().
getBytes
());
...
...
@@ -327,4 +138,17 @@ public class HBaseUtil {
}
return
result
;
}
public
static
void
batchSaveMinSummaryResult
(
List
<
Put
>
puts
)
throws
IOException
,
InterruptedException
{
Table
table
=
connection
.
getTable
(
TableName
.
valueOf
(
HBaseTableMetaData
.
TABLE_CHAIN_ONE_MINUTE_SUMMARY
.
TABLE_NAME
));
Object
[]
resultArray
=
new
Object
[
puts
.
size
()];
table
.
batch
(
puts
,
resultArray
);
int
index
=
0
;
for
(
Object
result
:
resultArray
)
{
if
(
result
==
null
)
{
logger
.
error
(
"Failed to insert the put the Value["
+
puts
.
get
(
index
).
getId
()
+
"]"
);
}
index
++;
}
}
}
skywalking-analysis/src/main/java/com/ai/cloud/skywalking/analysis/config/HBaseTableMetaData.java
浏览文件 @
13421da7
...
...
@@ -10,18 +10,6 @@ public class HBaseTableMetaData {
public
static
final
String
TABLE_NAME
=
"sw-call-chain"
;
}
/**
* HBase 表:用于存放CID,TID的映射关系表
*
* @author wusheng
*/
public
final
static
class
TABLE_CID_TID_MAPPING
{
public
static
final
String
TABLE_NAME
=
"sw-cid-tid-mapping"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"trace_info"
;
public
static
final
String
CID_COLUMN_NAME
=
"cid"
;
}
/**
* CID明细信息表
...
...
@@ -33,18 +21,6 @@ public class HBaseTableMetaData {
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_detail"
;
}
/**
* CID间关系表,记录异常CID和正常CID间的归属关系
*
* @author wusheng
*/
public
final
static
class
TABLE_CALL_CHAIN_RELATIONSHIP
{
public
static
final
String
TABLE_NAME
=
"sw-chain-relationship"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain-relationship"
;
}
/**
* 用于存放每个CID在一分钟内的汇总,汇总结果不包含关系汇总
*
...
...
@@ -61,56 +37,12 @@ public class HBaseTableMetaData {
*
* @author wusheng
*/
public
final
static
class
TABLE_CHAIN_ONE_MINUTE_SUMMARY
_INCLUDE_RELATIONSHIP
{
public
final
static
class
TABLE_CHAIN_ONE_MINUTE_SUMMARY
{
public
static
final
String
TABLE_NAME
=
"sw-chain-1min-summary-ic-rela"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_summary"
;
}
/**
* 用于存放每个CID在一小时内的汇总,汇总结果不包含关系汇总
*
* @author wusheng
*/
public
final
static
class
TABLE_CHAIN_ONE_HOUR_SUMMARY_INCLUDE_RELATIONSHIP
{
public
static
final
String
TABLE_NAME
=
"sw-chain-1hour-summary-ic-rela"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_summary"
;
}
/**
* 用于存放每个CID在一天内的汇总,汇总结果不包含关系汇总
*
* @author wusheng
*/
public
final
static
class
TABLE_CHAIN_ONE_DAY_SUMMARY_INCLUDE_RELATIONSHIP
{
public
static
final
String
TABLE_NAME
=
"sw-chain-1day-summary-ic-rela"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_summary"
;
}
/**
* 用于存放每个CID在一月内的汇总,汇总结果不包含关系汇总
*
* @author wusheng
*/
public
final
static
class
TABLE_CHAIN_ONE_MONTH_SUMMARY_INCLUDE_RELATIONSHIP
{
public
static
final
String
TABLE_NAME
=
"sw-chain-1mon-summary-ic-rela"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_summary"
;
}
/**
* 用于存放已经合并的调用链的信息
*
* @author zhangxin
*/
public
final
static
class
TABLE_MERGED_CHAIN_DETAIL
{
public
static
final
String
TABLE_NAME
=
"sw-merged-chain-detail"
;
public
static
final
String
COLUMN_FAMILY_NAME
=
"chain_detail"
;
}
/**
* 用于存放调用树和CID的映射关系
*
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录