Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
润土的好友猹
SkyWalking
提交
21978205
S
SkyWalking
项目概览
润土的好友猹
/
SkyWalking
与 Fork 源项目一致
Fork自
apache / SkyWalking
通知
9
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
SkyWalking
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
未验证
提交
21978205
编写于
6月 09, 2019
作者:
wu-sheng
提交者:
GitHub
6月 09, 2019
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Support exporter runs in increment and total modes (#2840)
* Make exporter interface better. * Fix review.
上级
3831c5ee
变更
6
隐藏空白更改
内联
并排
Showing
6 changed file
with
95 addition
and
19 deletion
+95
-19
oap-server/exporter/src/main/java/org/apache/skywalking/oap/server/exporter/provider/grpc/GRPCExporter.java
...lking/oap/server/exporter/provider/grpc/GRPCExporter.java
+10
-4
oap-server/exporter/src/test/java/org/apache/skywalking/oap/server/exporter/provider/grpc/GRPCExporterTest.java
...g/oap/server/exporter/provider/grpc/GRPCExporterTest.java
+10
-2
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/worker/ExportWorker.java
...walking/oap/server/core/analysis/worker/ExportWorker.java
+6
-8
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/worker/MetricsPersistentWorker.java
.../server/core/analysis/worker/MetricsPersistentWorker.java
+10
-3
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/exporter/ExportEvent.java
...ache/skywalking/oap/server/core/exporter/ExportEvent.java
+53
-0
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/exporter/MetricValuesExportService.java
...g/oap/server/core/exporter/MetricValuesExportService.java
+6
-2
未找到文件。
oap-server/exporter/src/main/java/org/apache/skywalking/oap/server/exporter/provider/grpc/GRPCExporter.java
浏览文件 @
21978205
...
@@ -26,7 +26,7 @@ import lombok.*;
...
@@ -26,7 +26,7 @@ import lombok.*;
import
org.apache.skywalking.apm.commons.datacarrier.DataCarrier
;
import
org.apache.skywalking.apm.commons.datacarrier.DataCarrier
;
import
org.apache.skywalking.apm.commons.datacarrier.consumer.IConsumer
;
import
org.apache.skywalking.apm.commons.datacarrier.consumer.IConsumer
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.*
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.*
;
import
org.apache.skywalking.oap.server.core.exporter.
MetricValuesExportService
;
import
org.apache.skywalking.oap.server.core.exporter.
*
;
import
org.apache.skywalking.oap.server.exporter.grpc.*
;
import
org.apache.skywalking.oap.server.exporter.grpc.*
;
import
org.apache.skywalking.oap.server.exporter.provider.MetricFormatter
;
import
org.apache.skywalking.oap.server.exporter.provider.MetricFormatter
;
import
org.apache.skywalking.oap.server.library.client.grpc.GRPCClient
;
import
org.apache.skywalking.oap.server.library.client.grpc.GRPCClient
;
...
@@ -56,9 +56,15 @@ public class GRPCExporter extends MetricFormatter implements MetricValuesExportS
...
@@ -56,9 +56,15 @@ public class GRPCExporter extends MetricFormatter implements MetricValuesExportS
subscriptionSet
=
new
HashSet
<>();
subscriptionSet
=
new
HashSet
<>();
}
}
@Override
public
void
export
(
MetricsMetaInfo
meta
,
Metrics
metrics
)
{
@Override
public
void
export
(
ExportEvent
event
)
{
if
(
subscriptionSet
.
size
()
==
0
||
subscriptionSet
.
contains
(
meta
.
getMetricsName
()))
{
if
(
ExportEvent
.
EventType
.
TOTAL
==
event
.
getType
())
{
exportBuffer
.
produce
(
new
ExportData
(
meta
,
metrics
));
Metrics
metrics
=
event
.
getMetrics
();
if
(
metrics
instanceof
WithMetadata
)
{
MetricsMetaInfo
meta
=
((
WithMetadata
)
metrics
).
getMeta
();
if
(
subscriptionSet
.
size
()
==
0
||
subscriptionSet
.
contains
(
meta
.
getMetricsName
()))
{
exportBuffer
.
produce
(
new
ExportData
(
meta
,
metrics
));
}
}
}
}
}
}
...
...
oap-server/exporter/src/test/java/org/apache/skywalking/oap/server/exporter/provider/grpc/GRPCExporterTest.java
浏览文件 @
21978205
...
@@ -19,7 +19,8 @@
...
@@ -19,7 +19,8 @@
package
org.apache.skywalking.oap.server.exporter.provider.grpc
;
package
org.apache.skywalking.oap.server.exporter.provider.grpc
;
import
io.grpc.testing.GrpcServerRule
;
import
io.grpc.testing.GrpcServerRule
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.MetricsMetaInfo
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.*
;
import
org.apache.skywalking.oap.server.core.exporter.ExportEvent
;
import
org.apache.skywalking.oap.server.core.source.DefaultScopeDefine
;
import
org.apache.skywalking.oap.server.core.source.DefaultScopeDefine
;
import
org.apache.skywalking.oap.server.exporter.grpc.MetricExportServiceGrpc
;
import
org.apache.skywalking.oap.server.exporter.grpc.MetricExportServiceGrpc
;
import
org.junit.Before
;
import
org.junit.Before
;
...
@@ -58,7 +59,14 @@ public class GRPCExporterTest {
...
@@ -58,7 +59,14 @@ public class GRPCExporterTest {
@Test
@Test
public
void
export
()
{
public
void
export
()
{
exporter
.
export
(
metaInfo
,
new
MockMetrics
());
ExportEvent
event
=
new
ExportEvent
(
new
MockExporterMetrics
(),
ExportEvent
.
EventType
.
TOTAL
);
exporter
.
export
(
event
);
}
public
static
class
MockExporterMetrics
extends
MockMetrics
implements
WithMetadata
{
@Override
public
MetricsMetaInfo
getMeta
()
{
return
new
MetricsMetaInfo
(
"mock-metrics"
,
DefaultScopeDefine
.
ALL
);
}
}
}
@Test
@Test
...
...
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/worker/ExportWorker.java
浏览文件 @
21978205
...
@@ -18,7 +18,6 @@
...
@@ -18,7 +18,6 @@
package
org.apache.skywalking.oap.server.core.analysis.worker
;
package
org.apache.skywalking.oap.server.core.analysis.worker
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.*
;
import
org.apache.skywalking.oap.server.core.exporter.*
;
import
org.apache.skywalking.oap.server.core.exporter.*
;
import
org.apache.skywalking.oap.server.core.worker.AbstractWorker
;
import
org.apache.skywalking.oap.server.core.worker.AbstractWorker
;
import
org.apache.skywalking.oap.server.library.module.ModuleDefineHolder
;
import
org.apache.skywalking.oap.server.library.module.ModuleDefineHolder
;
...
@@ -26,21 +25,20 @@ import org.apache.skywalking.oap.server.library.module.ModuleDefineHolder;
...
@@ -26,21 +25,20 @@ import org.apache.skywalking.oap.server.library.module.ModuleDefineHolder;
/**
/**
* @author wusheng
* @author wusheng
*/
*/
public
class
ExportWorker
extends
AbstractWorker
<
Metrics
>
{
public
class
ExportWorker
extends
AbstractWorker
<
ExportEvent
>
{
private
MetricValuesExportService
exportService
;
private
MetricValuesExportService
exportService
;
public
ExportWorker
(
ModuleDefineHolder
moduleDefineHolder
)
{
public
ExportWorker
(
ModuleDefineHolder
moduleDefineHolder
)
{
super
(
moduleDefineHolder
);
super
(
moduleDefineHolder
);
}
}
@Override
public
void
in
(
Metrics
metrics
)
{
@Override
public
void
in
(
ExportEvent
event
)
{
if
(
exportService
!=
null
||
getModuleDefineHolder
().
has
(
ExporterModule
.
NAME
))
{
if
(
exportService
!=
null
||
getModuleDefineHolder
().
has
(
ExporterModule
.
NAME
))
{
if
(
metrics
instanceof
WithMetadata
)
{
if
(
exportService
==
null
)
{
if
(
exportService
==
null
)
{
exportService
=
getModuleDefineHolder
().
find
(
ExporterModule
.
NAME
).
provider
().
getService
(
MetricValuesExportService
.
class
);
exportService
=
getModuleDefineHolder
().
find
(
ExporterModule
.
NAME
).
provider
().
getService
(
MetricValuesExportService
.
class
);
}
exportService
.
export
(((
WithMetadata
)
metrics
).
getMeta
(),
metrics
);
}
}
exportService
.
export
(
event
);
}
}
}
}
}
}
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/worker/MetricsPersistentWorker.java
浏览文件 @
21978205
...
@@ -24,6 +24,7 @@ import org.apache.skywalking.apm.commons.datacarrier.consumer.*;
...
@@ -24,6 +24,7 @@ import org.apache.skywalking.apm.commons.datacarrier.consumer.*;
import
org.apache.skywalking.oap.server.core.UnexpectedException
;
import
org.apache.skywalking.oap.server.core.UnexpectedException
;
import
org.apache.skywalking.oap.server.core.analysis.data.*
;
import
org.apache.skywalking.oap.server.core.analysis.data.*
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.Metrics
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.Metrics
;
import
org.apache.skywalking.oap.server.core.exporter.ExportEvent
;
import
org.apache.skywalking.oap.server.core.storage.IMetricsDAO
;
import
org.apache.skywalking.oap.server.core.storage.IMetricsDAO
;
import
org.apache.skywalking.oap.server.core.storage.model.Model
;
import
org.apache.skywalking.oap.server.core.storage.model.Model
;
import
org.apache.skywalking.oap.server.core.worker.AbstractWorker
;
import
org.apache.skywalking.oap.server.core.worker.AbstractWorker
;
...
@@ -43,12 +44,12 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
...
@@ -43,12 +44,12 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
private
final
MergeDataCache
<
Metrics
>
mergeDataCache
;
private
final
MergeDataCache
<
Metrics
>
mergeDataCache
;
private
final
IMetricsDAO
metricsDAO
;
private
final
IMetricsDAO
metricsDAO
;
private
final
AbstractWorker
<
Metrics
>
nextAlarmWorker
;
private
final
AbstractWorker
<
Metrics
>
nextAlarmWorker
;
private
final
AbstractWorker
<
Metrics
>
nextExportWorker
;
private
final
AbstractWorker
<
ExportEvent
>
nextExportWorker
;
private
final
DataCarrier
<
Metrics
>
dataCarrier
;
private
final
DataCarrier
<
Metrics
>
dataCarrier
;
MetricsPersistentWorker
(
ModuleDefineHolder
moduleDefineHolder
,
Model
model
,
int
batchSize
,
MetricsPersistentWorker
(
ModuleDefineHolder
moduleDefineHolder
,
Model
model
,
int
batchSize
,
IMetricsDAO
metricsDAO
,
AbstractWorker
<
Metrics
>
nextAlarmWorker
,
IMetricsDAO
metricsDAO
,
AbstractWorker
<
Metrics
>
nextAlarmWorker
,
AbstractWorker
<
Metrics
>
nextExportWorker
)
{
AbstractWorker
<
ExportEvent
>
nextExportWorker
)
{
super
(
moduleDefineHolder
,
batchSize
);
super
(
moduleDefineHolder
,
batchSize
);
this
.
model
=
model
;
this
.
model
=
model
;
this
.
mergeDataCache
=
new
MergeDataCache
<>();
this
.
mergeDataCache
=
new
MergeDataCache
<>();
...
@@ -100,6 +101,11 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
...
@@ -100,6 +101,11 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
@Override
public
List
<
Object
>
prepareBatch
(
MergeDataCache
<
Metrics
>
cache
)
{
@Override
public
List
<
Object
>
prepareBatch
(
MergeDataCache
<
Metrics
>
cache
)
{
List
<
Object
>
batchCollection
=
new
LinkedList
<>();
List
<
Object
>
batchCollection
=
new
LinkedList
<>();
cache
.
getLast
().
collection
().
forEach
(
data
->
{
cache
.
getLast
().
collection
().
forEach
(
data
->
{
if
(
Objects
.
nonNull
(
nextExportWorker
))
{
ExportEvent
event
=
new
ExportEvent
(
data
,
ExportEvent
.
EventType
.
INCREMENT
);
nextExportWorker
.
in
(
event
);
}
Metrics
dbData
=
null
;
Metrics
dbData
=
null
;
try
{
try
{
dbData
=
metricsDAO
.
get
(
model
,
data
);
dbData
=
metricsDAO
.
get
(
model
,
data
);
...
@@ -120,7 +126,8 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
...
@@ -120,7 +126,8 @@ public class MetricsPersistentWorker extends PersistenceWorker<Metrics, MergeDat
nextAlarmWorker
.
in
(
data
);
nextAlarmWorker
.
in
(
data
);
}
}
if
(
Objects
.
nonNull
(
nextExportWorker
))
{
if
(
Objects
.
nonNull
(
nextExportWorker
))
{
nextExportWorker
.
in
(
data
);
ExportEvent
event
=
new
ExportEvent
(
data
,
ExportEvent
.
EventType
.
TOTAL
);
nextExportWorker
.
in
(
event
);
}
}
}
catch
(
Throwable
t
)
{
}
catch
(
Throwable
t
)
{
logger
.
error
(
t
.
getMessage
(),
t
);
logger
.
error
(
t
.
getMessage
(),
t
);
...
...
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/exporter/ExportEvent.java
0 → 100644
浏览文件 @
21978205
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package
org.apache.skywalking.oap.server.core.exporter
;
import
lombok.Getter
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.Metrics
;
/**
* The event for exporter {@link MetricValuesExportService} implementation processes.
* {@link #metrics} should not be changed in any case.
*
* @author wusheng
*/
@Getter
public
class
ExportEvent
{
/**
* Fields of this should not be changed in any case.
*/
private
Metrics
metrics
;
private
EventType
type
;
public
ExportEvent
(
Metrics
metrics
,
EventType
type
)
{
this
.
metrics
=
metrics
;
this
.
type
=
type
;
}
public
enum
EventType
{
/**
* The metrics aggregated in this bulk, not include the existing persistent data.
*/
INCREMENT
,
/**
* Final result of the metrics at this moment.
*/
TOTAL
}
}
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/exporter/MetricValuesExportService.java
浏览文件 @
21978205
...
@@ -18,7 +18,6 @@
...
@@ -18,7 +18,6 @@
package
org.apache.skywalking.oap.server.core.exporter
;
package
org.apache.skywalking.oap.server.core.exporter
;
import
org.apache.skywalking.oap.server.core.analysis.metrics.*
;
import
org.apache.skywalking.oap.server.library.module.Service
;
import
org.apache.skywalking.oap.server.library.module.Service
;
/**
/**
...
@@ -27,5 +26,10 @@ import org.apache.skywalking.oap.server.library.module.Service;
...
@@ -27,5 +26,10 @@ import org.apache.skywalking.oap.server.library.module.Service;
* @author wusheng
* @author wusheng
*/
*/
public
interface
MetricValuesExportService
extends
Service
{
public
interface
MetricValuesExportService
extends
Service
{
void
export
(
MetricsMetaInfo
meta
,
Metrics
metrics
);
/**
* This method is sync-mode export, the performance effects the persistence result. Queue mode is highly recommended.
*
* @param event value is only accurate when the method invokes. Don't cache it.
*/
void
export
(
ExportEvent
event
);
}
}
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录