Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
apache
SkyWalking
提交
12a18e69
S
SkyWalking
项目概览
apache
/
SkyWalking
上一次同步 1 年多
通知
302
Star
21345
Fork
6091
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
S
SkyWalking
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
前往新版Gitcode,体验更适合开发者的 AI 搜索 >>
未验证
提交
12a18e69
编写于
9月 12, 2022
作者:
Y
yswdqz
提交者:
GitHub
9月 12, 2022
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
Add layer property to LAL script (#9593)
上级
b3c7658a
变更
21
隐藏空白更改
内联
并排
Showing
21 changed file
with
179 addition
and
57 deletion
+179
-57
docs/en/changes/changes.md
docs/en/changes/changes.md
+1
-0
docs/en/concepts-and-designs/lal.md
docs/en/concepts-and-designs/lal.md
+3
-0
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/dsl/spec/filter/FilterSpec.java
...ywalking/oap/log/analyzer/dsl/spec/filter/FilterSpec.java
+14
-14
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/LALConfig.java
...pache/skywalking/oap/log/analyzer/provider/LALConfig.java
+2
-0
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/LogAnalyzerModuleConfig.java
...ng/oap/log/analyzer/provider/LogAnalyzerModuleConfig.java
+1
-1
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/ILogAnalysisListenerManager.java
...og/analyzer/provider/log/ILogAnalysisListenerManager.java
+5
-0
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/LogAnalyzer.java
...skywalking/oap/log/analyzer/provider/log/LogAnalyzer.java
+26
-7
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/LogAnalyzerServiceImpl.java
...oap/log/analyzer/provider/log/LogAnalyzerServiceImpl.java
+15
-3
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogAnalysisListener.java
...g/analyzer/provider/log/listener/LogAnalysisListener.java
+1
-2
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogAnalysisListenerFactory.java
...zer/provider/log/listener/LogAnalysisListenerFactory.java
+3
-1
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogFilterListener.java
...log/analyzer/provider/log/listener/LogFilterListener.java
+29
-16
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogSinkListener.java
...p/log/analyzer/provider/log/listener/LogSinkListener.java
+35
-0
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogSinkListenerFactory.java
...nalyzer/provider/log/listener/LogSinkListenerFactory.java
+26
-0
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/RecordSinkListener.java
...og/analyzer/provider/log/listener/RecordSinkListener.java
+7
-6
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/TrafficSinkListener.java
...g/analyzer/provider/log/listener/TrafficSinkListener.java
+5
-5
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/dsl/DSLSecurityTest.java
...ache/skywalking/oap/log/analyzer/dsl/DSLSecurityTest.java
+1
-1
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/dsl/DSLTest.java
...a/org/apache/skywalking/oap/log/analyzer/dsl/DSLTest.java
+1
-1
oap-server/server-starter/src/main/resources/lal/default.yaml
...server/server-starter/src/main/resources/lal/default.yaml
+1
-0
oap-server/server-starter/src/main/resources/lal/envoy-als.yaml
...rver/server-starter/src/main/resources/lal/envoy-als.yaml
+1
-0
test/e2e-v2/cases/log/lal.yaml
test/e2e-v2/cases/log/lal.yaml
+1
-0
test/e2e-v2/cases/satellite/native-protocols/lal.yaml
test/e2e-v2/cases/satellite/native-protocols/lal.yaml
+1
-0
未找到文件。
docs/en/changes/changes.md
浏览文件 @
12a18e69
...
...
@@ -9,6 +9,7 @@
*
Use prepareStatement in H2SQLExecutor#getByIDs.(No function change).
*
Bump up snakeyaml to 1.31 for fixing CVE-2022-25857
*
Fix
`DurationUtils.convertToTimeBucket`
missed verify date format.
*
[
**Breaking Change**
] Change the LAL script format(Add layer property).
#### UI
...
...
docs/en/concepts-and-designs/lal.md
浏览文件 @
12a18e69
...
...
@@ -8,6 +8,9 @@ The LAL config files are in YAML format, and are located under directory `lal`.
set
`log-analyzer/default/lalFiles`
in the
`application.yml`
file or set environment variable
`SW_LOG_LAL_FILES`
to
activate specific LAL config files.
## Layer
Layer should be declared in the LAL script to represent the analysis scope of the logs.
## Filter
A filter is a group of
[
parser
](
#parser
)
,
[
extractor
](
#extractor
)
and
[
sink
](
#sink
)
. Users can use one or more filters
...
...
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/dsl/spec/filter/FilterSpec.java
浏览文件 @
12a18e69
...
...
@@ -37,9 +37,9 @@ import org.apache.skywalking.oap.log.analyzer.dsl.spec.parser.TextParserSpec;
import
org.apache.skywalking.oap.log.analyzer.dsl.spec.parser.YamlParserSpec
;
import
org.apache.skywalking.oap.log.analyzer.dsl.spec.sink.SinkSpec
;
import
org.apache.skywalking.oap.log.analyzer.provider.LogAnalyzerModuleConfig
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Log
Analysis
ListenerFactory
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Record
Analysis
Listener
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Traffic
Analysis
Listener
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Log
Sink
ListenerFactory
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Record
Sink
Listener
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.Traffic
Sink
Listener
;
import
org.apache.skywalking.oap.server.core.source.Log
;
import
org.apache.skywalking.oap.server.library.module.ModuleManager
;
import
org.apache.skywalking.oap.server.library.module.ModuleStartException
;
...
...
@@ -49,7 +49,7 @@ import org.slf4j.LoggerFactory;
public
class
FilterSpec
extends
AbstractSpec
{
private
static
final
Logger
LOGGER
=
LoggerFactory
.
getLogger
(
FilterSpec
.
class
);
private
final
List
<
Log
AnalysisListenerFactory
>
f
actories
;
private
final
List
<
Log
SinkListenerFactory
>
sinkListenerF
actories
;
private
final
TextParserSpec
textParser
;
...
...
@@ -70,9 +70,9 @@ public class FilterSpec extends AbstractSpec {
parsedType
=
new
TypeReference
<
Map
<
String
,
Object
>>()
{
};
f
actories
=
Arrays
.
asList
(
new
Record
Analysis
Listener
.
Factory
(
moduleManager
(),
moduleConfig
()),
new
Traffic
Analysis
Listener
.
Factory
(
moduleManager
(),
moduleConfig
())
sinkListenerF
actories
=
Arrays
.
asList
(
new
Record
Sink
Listener
.
Factory
(
moduleManager
(),
moduleConfig
()),
new
Traffic
Sink
Listener
.
Factory
(
moduleManager
(),
moduleConfig
())
);
textParser
=
new
TextParserSpec
(
moduleManager
(),
moduleConfig
());
...
...
@@ -168,17 +168,17 @@ public class FilterSpec extends AbstractSpec {
final
Optional
<
AtomicReference
<
Log
>>
container
=
BINDING
.
get
().
logContainer
();
if
(
container
.
isPresent
())
{
f
actories
.
stream
()
.
map
(
Log
Analysis
ListenerFactory:
:
create
)
.
filter
(
it
->
it
instanceof
Record
Analysis
Listener
)
sinkListenerF
actories
.
stream
()
.
map
(
Log
Sink
ListenerFactory:
:
create
)
.
filter
(
it
->
it
instanceof
Record
Sink
Listener
)
.
map
(
it
->
it
.
parse
(
logData
,
extraLog
))
.
map
(
it
->
(
Record
Analysis
Listener
)
it
)
.
map
(
Record
Analysis
Listener:
:
getLog
)
.
map
(
it
->
(
Record
Sink
Listener
)
it
)
.
map
(
Record
Sink
Listener:
:
getLog
)
.
findFirst
()
.
ifPresent
(
log
->
container
.
get
().
set
(
log
));
}
else
{
f
actories
.
stream
()
.
map
(
Log
Analysis
ListenerFactory:
:
create
)
sinkListenerF
actories
.
stream
()
.
map
(
Log
Sink
ListenerFactory:
:
create
)
.
forEach
(
it
->
it
.
parse
(
logData
,
extraLog
).
build
());
}
}
...
...
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/LALConfig.java
浏览文件 @
12a18e69
...
...
@@ -25,4 +25,6 @@ public class LALConfig {
private
String
name
;
private
String
dsl
;
private
String
layer
;
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/LogAnalyzerModuleConfig.java
浏览文件 @
12a18e69
...
...
@@ -51,7 +51,7 @@ public class LogAnalyzerModuleConfig extends ModuleConfig {
private
List
<
Rule
>
meterConfigs
;
public
List
<
String
>
lalFiles
()
{
return
Splitter
.
on
(
","
).
omitEmptyStrings
().
splitToList
(
Strings
.
nullToEmpty
(
getLalFiles
()));
return
Splitter
.
on
(
","
).
omitEmptyStrings
().
trimResults
().
splitToList
(
Strings
.
nullToEmpty
(
getLalFiles
()));
}
public
List
<
Rule
>
malConfigs
()
throws
ModuleStartException
{
...
...
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/ILogAnalysisListenerManager.java
浏览文件 @
12a18e69
...
...
@@ -19,10 +19,15 @@ package org.apache.skywalking.oap.log.analyzer.provider.log;
import
java.util.List
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.LogAnalysisListenerFactory
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.LogSinkListenerFactory
;
public
interface
ILogAnalysisListenerManager
{
void
addListenerFactory
(
LogAnalysisListenerFactory
factory
);
List
<
LogAnalysisListenerFactory
>
getLogAnalysisListenerFactories
();
void
addSinkListenerFactory
(
LogSinkListenerFactory
factory
);
List
<
LogSinkListenerFactory
>
getSinkListenerFactory
();
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/LogAnalyzer.java
浏览文件 @
12a18e69
...
...
@@ -20,9 +20,13 @@ package org.apache.skywalking.oap.log.analyzer.provider.log;
import
com.google.protobuf.Message
;
import
java.util.ArrayList
;
import
java.util.List
;
import
java.util.Objects
;
import
lombok.RequiredArgsConstructor
;
import
lombok.extern.slf4j.Slf4j
;
import
org.apache.skywalking.apm.network.logging.v3.LogData
;
import
org.apache.skywalking.oap.server.core.UnexpectedException
;
import
org.apache.skywalking.oap.server.core.analysis.Layer
;
import
org.apache.skywalking.oap.server.library.util.StringUtil
;
import
org.apache.skywalking.oap.log.analyzer.provider.LogAnalyzerModuleConfig
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.LogAnalysisListener
;
...
...
@@ -46,26 +50,41 @@ public class LogAnalyzer {
log
.
debug
(
"The log is ignored because the Service name is empty"
);
return
;
}
createListeners
();
Layer
layer
;
if
(
""
.
equals
(
builder
.
getLayer
()))
{
layer
=
Layer
.
GENERAL
;
}
else
{
try
{
layer
=
Layer
.
nameOf
(
builder
.
getLayer
());
}
catch
(
UnexpectedException
e
)
{
log
.
warn
(
"The Layer {} is not found, abandon the log."
,
builder
.
getLayer
());
return
;
}
}
createAnalysisListeners
(
layer
);
if
(
builder
.
getTimestamp
()
==
0
)
{
// If no timestamp, OAP server would use the received timestamp as log's timestamp
builder
.
setTimestamp
(
System
.
currentTimeMillis
());
}
notifyListener
(
builder
,
extraLog
);
notifyListenerToBuild
();
notify
Analysis
Listener
(
builder
,
extraLog
);
notify
Analysis
ListenerToBuild
();
}
private
void
notifyListener
(
LogData
.
Builder
builder
,
final
Message
extraLog
)
{
private
void
notify
Analysis
Listener
(
LogData
.
Builder
builder
,
final
Message
extraLog
)
{
listeners
.
forEach
(
listener
->
listener
.
parse
(
builder
,
extraLog
));
}
private
void
notifyListenerToBuild
()
{
private
void
notify
Analysis
ListenerToBuild
()
{
listeners
.
forEach
(
LogAnalysisListener:
:
build
);
}
private
void
create
Listeners
(
)
{
private
void
create
AnalysisListeners
(
Layer
layer
)
{
factoryManager
.
getLogAnalysisListenerFactories
()
.
forEach
(
factory
->
listeners
.
add
(
factory
.
create
()));
.
stream
()
.
map
(
factory
->
factory
.
create
(
layer
))
.
filter
(
Objects:
:
nonNull
)
.
forEach
(
listeners:
:
add
);
}
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/LogAnalyzerServiceImpl.java
浏览文件 @
12a18e69
...
...
@@ -24,13 +24,15 @@ import lombok.RequiredArgsConstructor;
import
org.apache.skywalking.apm.network.logging.v3.LogData
;
import
org.apache.skywalking.oap.log.analyzer.provider.LogAnalyzerModuleConfig
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.LogAnalysisListenerFactory
;
import
org.apache.skywalking.oap.log.analyzer.provider.log.listener.LogSinkListenerFactory
;
import
org.apache.skywalking.oap.server.library.module.ModuleManager
;
@RequiredArgsConstructor
public
class
LogAnalyzerServiceImpl
implements
ILogAnalyzerService
,
ILogAnalysisListenerManager
{
private
final
ModuleManager
moduleManager
;
private
final
LogAnalyzerModuleConfig
moduleConfig
;
private
final
List
<
LogAnalysisListenerFactory
>
factories
=
new
ArrayList
<>();
private
final
List
<
LogAnalysisListenerFactory
>
analysisListenerFactories
=
new
ArrayList
<>();
private
final
List
<
LogSinkListenerFactory
>
sinkListenerFactories
=
new
ArrayList
<>();
@Override
public
void
doAnalysis
(
final
LogData
.
Builder
log
,
Message
extraLog
)
{
...
...
@@ -40,11 +42,21 @@ public class LogAnalyzerServiceImpl implements ILogAnalyzerService, ILogAnalysis
@Override
public
void
addListenerFactory
(
final
LogAnalysisListenerFactory
factory
)
{
f
actories
.
add
(
factory
);
analysisListenerF
actories
.
add
(
factory
);
}
@Override
public
List
<
LogAnalysisListenerFactory
>
getLogAnalysisListenerFactories
()
{
return
factories
;
return
analysisListenerFactories
;
}
@Override
public
void
addSinkListenerFactory
(
LogSinkListenerFactory
factory
)
{
sinkListenerFactories
.
add
(
factory
);
}
@Override
public
List
<
LogSinkListenerFactory
>
getSinkListenerFactory
()
{
return
sinkListenerFactories
;
}
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogAnalysisListener.java
浏览文件 @
12a18e69
...
...
@@ -25,8 +25,7 @@ import org.apache.skywalking.apm.network.logging.v3.LogData;
*/
public
interface
LogAnalysisListener
{
/**
* The last step of the analysis process. Typically, the implementations forward the analysis results to the source
* receiver.
* The last step of the analysis process. Typically, the implementations execute corresponding DSL.
*/
void
build
();
...
...
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogAnalysisListenerFactory.java
浏览文件 @
12a18e69
...
...
@@ -17,11 +17,13 @@
package
org.apache.skywalking.oap.log.analyzer.provider.log.listener
;
import
org.apache.skywalking.oap.server.core.analysis.Layer
;
/**
* LogAnalysisListenerFactory implementation creates the listener instance when required.
* Every LogAnalysisListener could have its own creation factory.
*/
public
interface
LogAnalysisListenerFactory
{
LogAnalysisListener
create
();
LogAnalysisListener
create
(
Layer
layer
);
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogFilterListener.java
浏览文件 @
12a18e69
...
...
@@ -19,61 +19,74 @@
package
org.apache.skywalking.oap.log.analyzer.provider.log.listener
;
import
com.google.protobuf.Message
;
import
java.util.ArrayList
;
import
java.util.List
;
import
java.util.Map
;
import
java.util.stream.Collectors
;
import
lombok.RequiredArgsConstructor
;
import
lombok.extern.slf4j.Slf4j
;
import
java.util.HashMap
;
import
org.apache.skywalking.apm.network.logging.v3.LogData
;
import
org.apache.skywalking.oap.log.analyzer.dsl.Binding
;
import
org.apache.skywalking.oap.log.analyzer.dsl.DSL
;
import
org.apache.skywalking.oap.log.analyzer.provider.LALConfig
;
import
org.apache.skywalking.oap.log.analyzer.provider.LALConfigs
;
import
org.apache.skywalking.oap.log.analyzer.provider.LogAnalyzerModuleConfig
;
import
org.apache.skywalking.oap.server.core.analysis.Layer
;
import
org.apache.skywalking.oap.server.library.module.ModuleManager
;
import
org.apache.skywalking.oap.server.library.module.ModuleStartException
;
@Slf4j
@RequiredArgsConstructor
public
class
LogFilterListener
implements
LogAnalysisListener
{
private
final
List
<
DSL
>
dsls
;
@lombok
.
NonNull
private
final
DSL
dsl
;
@Override
public
void
build
()
{
dsls
.
forEach
(
dsl
->
{
try
{
dsl
.
evaluate
();
}
catch
(
final
Exception
e
)
{
log
.
warn
(
"Failed to evaluate dsl: {}"
,
dsl
,
e
);
}
});
try
{
dsl
.
evaluate
();
}
catch
(
final
Exception
e
)
{
log
.
warn
(
"Failed to evaluate dsl: {}"
,
dsl
,
e
);
}
}
@Override
public
LogAnalysisListener
parse
(
final
LogData
.
Builder
logData
,
final
Message
extraLog
)
{
dsls
.
forEach
(
dsl
->
dsl
.
bind
(
new
Binding
().
log
(
logData
.
build
())
.
extraLog
(
extraLog
)));
dsl
.
bind
(
new
Binding
().
log
(
logData
.
build
()).
extraLog
(
extraLog
));
return
this
;
}
public
static
class
Factory
implements
LogAnalysisListenerFactory
{
private
final
List
<
DSL
>
dsls
;
private
final
Map
<
Layer
,
DSL
>
dsls
;
public
Factory
(
final
ModuleManager
moduleManager
,
final
LogAnalyzerModuleConfig
config
)
throws
Exception
{
dsls
=
new
ArrayList
<>();
dsls
=
new
HashMap
<>();
final
List
<
LALConfig
>
configList
=
LALConfigs
.
load
(
config
.
getLalPath
(),
config
.
lalFiles
())
.
stream
()
.
flatMap
(
it
->
it
.
getRules
().
stream
())
.
collect
(
Collectors
.
toList
());
for
(
final
LALConfig
c
:
configList
)
{
dsls
.
add
(
DSL
.
of
(
moduleManager
,
config
,
c
.
getDsl
()));
Layer
layer
=
Layer
.
nameOf
(
c
.
getLayer
());
if
(
dsls
.
put
(
layer
,
DSL
.
of
(
moduleManager
,
config
,
c
.
getDsl
()))
!=
null
)
{
throw
new
ModuleStartException
(
"Layer "
+
layer
.
name
()
+
" has already set a rule."
);
}
}
}
@Override
public
LogAnalysisListener
create
()
{
return
new
LogFilterListener
(
dsls
);
public
LogAnalysisListener
create
(
Layer
layer
)
{
if
(
layer
==
null
)
{
return
null
;
}
final
DSL
dsl
=
dsls
.
get
(
layer
);
if
(
dsl
==
null
)
{
return
null
;
}
return
new
LogFilterListener
(
dsl
);
}
}
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogSinkListener.java
0 → 100644
浏览文件 @
12a18e69
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.skywalking.oap.log.analyzer.provider.log.listener
;
import
com.google.protobuf.Message
;
import
org.apache.skywalking.apm.network.logging.v3.LogData
;
public
interface
LogSinkListener
{
/**
* The last step of the sink process. Typically, the implementations forward the results to the source
* receiver.
*/
void
build
();
/**
* Parse the raw data from the probe.
* @return {@code this} for chaining.
*/
LogSinkListener
parse
(
LogData
.
Builder
logData
,
final
Message
extraLog
);
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/LogSinkListenerFactory.java
0 → 100644
浏览文件 @
12a18e69
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package
org.apache.skywalking.oap.log.analyzer.provider.log.listener
;
/**
* LogSinkListenerFactory implementation creates the listener instance when required.
* Every LogSinkListener could have its own creation factory.
*/
public
interface
LogSinkListenerFactory
{
LogSinkListener
create
();
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/Record
Analysis
Listener.java
→
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/Record
Sink
Listener.java
浏览文件 @
12a18e69
...
...
@@ -29,6 +29,7 @@ import lombok.SneakyThrows;
import
org.apache.skywalking.apm.network.logging.v3.LogData
;
import
org.apache.skywalking.apm.network.logging.v3.LogDataBody
;
import
org.apache.skywalking.apm.network.logging.v3.TraceContext
;
import
org.apache.skywalking.oap.server.core.analysis.manual.searchtag.TagType
;
import
org.apache.skywalking.oap.server.core.source.TagAutocomplete
;
import
org.apache.skywalking.oap.server.library.util.StringUtil
;
...
...
@@ -48,10 +49,10 @@ import org.apache.skywalking.oap.server.library.module.ModuleManager;
import
static
org
.
apache
.
skywalking
.
oap
.
server
.
library
.
util
.
ProtoBufJsonUtils
.
toJSON
;
/**
* Record
Analysis
Listener forwards the log data to the persistence layer with the query required conditions.
* Record
Sink
Listener forwards the log data to the persistence layer with the query required conditions.
*/
@RequiredArgsConstructor
public
class
Record
AnalysisListener
implements
LogAnalysis
Listener
{
public
class
Record
SinkListener
implements
LogSink
Listener
{
private
final
SourceReceiver
sourceReceiver
;
private
final
NamingControl
namingControl
;
private
final
List
<
String
>
searchableTagKeys
;
...
...
@@ -66,7 +67,7 @@ public class RecordAnalysisListener implements LogAnalysisListener {
@Override
@SneakyThrows
public
Log
Analysis
Listener
parse
(
final
LogData
.
Builder
logData
,
public
Log
Sink
Listener
parse
(
final
LogData
.
Builder
logData
,
final
Message
extraLog
)
{
LogDataBody
body
=
logData
.
getBody
();
log
.
setUniqueId
(
UUID
.
randomUUID
().
toString
().
replace
(
"-"
,
""
));
...
...
@@ -142,7 +143,7 @@ public class RecordAnalysisListener implements LogAnalysisListener {
});
}
public
static
class
Factory
implements
Log
Analysis
ListenerFactory
{
public
static
class
Factory
implements
Log
Sink
ListenerFactory
{
private
final
SourceReceiver
sourceReceiver
;
private
final
NamingControl
namingControl
;
private
final
List
<
String
>
searchableTagKeys
;
...
...
@@ -161,8 +162,8 @@ public class RecordAnalysisListener implements LogAnalysisListener {
}
@Override
public
LogAnalysis
Listener
create
()
{
return
new
Record
Analysis
Listener
(
sourceReceiver
,
namingControl
,
searchableTagKeys
);
public
RecordSink
Listener
create
()
{
return
new
Record
Sink
Listener
(
sourceReceiver
,
namingControl
,
searchableTagKeys
);
}
}
}
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/Traffic
Analysis
Listener.java
→
oap-server/analyzer/log-analyzer/src/main/java/org/apache/skywalking/oap/log/analyzer/provider/log/listener/Traffic
Sink
Listener.java
浏览文件 @
12a18e69
...
...
@@ -40,7 +40,7 @@ import static java.util.Objects.nonNull;
* Generate service, service instance and endpoint traffic by log data.
*/
@RequiredArgsConstructor
public
class
Traffic
AnalysisListener
implements
LogAnalysis
Listener
{
public
class
Traffic
SinkListener
implements
LogSink
Listener
{
private
final
SourceReceiver
sourceReceiver
;
private
final
NamingControl
namingControl
;
...
...
@@ -62,7 +62,7 @@ public class TrafficAnalysisListener implements LogAnalysisListener {
}
@Override
public
Log
Analysis
Listener
parse
(
final
LogData
.
Builder
logData
,
public
Log
Sink
Listener
parse
(
final
LogData
.
Builder
logData
,
final
Message
extraLog
)
{
Layer
layer
;
if
(
StringUtil
.
isNotEmpty
(
logData
.
getLayer
()))
{
...
...
@@ -97,7 +97,7 @@ public class TrafficAnalysisListener implements LogAnalysisListener {
return
this
;
}
public
static
class
Factory
implements
Log
Analysis
ListenerFactory
{
public
static
class
Factory
implements
Log
Sink
ListenerFactory
{
private
final
SourceReceiver
sourceReceiver
;
private
final
NamingControl
namingControl
;
...
...
@@ -111,8 +111,8 @@ public class TrafficAnalysisListener implements LogAnalysisListener {
}
@Override
public
Log
Analysis
Listener
create
()
{
return
new
Traffic
Analysis
Listener
(
sourceReceiver
,
namingControl
);
public
Log
Sink
Listener
create
()
{
return
new
Traffic
Sink
Listener
(
sourceReceiver
,
namingControl
);
}
}
}
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/dsl/DSLSecurityTest.java
浏览文件 @
12a18e69
...
...
@@ -131,7 +131,7 @@ public class DSLSecurityTest {
public
void
testSecurity
()
throws
ModuleStartException
{
final
DSL
dsl
=
DSL
.
of
(
manager
,
new
LogAnalyzerModuleConfig
(),
script
);
Whitebox
.
setInternalState
(
Whitebox
.
getInternalState
(
dsl
,
"filterSpec"
),
"
f
actories"
,
Collections
.
emptyList
()
Whitebox
.
getInternalState
(
dsl
,
"filterSpec"
),
"
sinkListenerF
actories"
,
Collections
.
emptyList
()
);
dsl
.
bind
(
new
Binding
().
log
(
LogData
.
newBuilder
()));
...
...
oap-server/analyzer/log-analyzer/src/test/java/org/apache/skywalking/oap/log/analyzer/dsl/DSLTest.java
浏览文件 @
12a18e69
...
...
@@ -191,7 +191,7 @@ public class DSLTest {
public
void
testDslStaticCompile
()
throws
ModuleStartException
{
final
DSL
dsl
=
DSL
.
of
(
manager
,
new
LogAnalyzerModuleConfig
(),
script
);
Whitebox
.
setInternalState
(
Whitebox
.
getInternalState
(
dsl
,
"filterSpec"
),
"
f
actories"
,
Collections
.
emptyList
()
Whitebox
.
getInternalState
(
dsl
,
"filterSpec"
),
"
sinkListenerF
actories"
,
Collections
.
emptyList
()
);
dsl
.
bind
(
new
Binding
().
log
(
LogData
.
newBuilder
().
build
()));
...
...
oap-server/server-starter/src/main/resources/lal/default.yaml
浏览文件 @
12a18e69
...
...
@@ -16,6 +16,7 @@
# The default LAL script to save all logs, behaving like the versions before 8.5.0.
rules
:
-
name
:
default
layer
:
GENERAL
dsl
:
|
filter {
sink {
...
...
oap-server/server-starter/src/main/resources/lal/envoy-als.yaml
浏览文件 @
12a18e69
...
...
@@ -15,6 +15,7 @@
rules
:
-
name
:
envoy-als
layer
:
MESH
dsl
:
|
filter {
// only collect abnormal logs (http status code >= 300, or commonProperties?.responseFlags is not empty)
...
...
test/e2e-v2/cases/log/lal.yaml
浏览文件 @
12a18e69
...
...
@@ -15,6 +15,7 @@
rules
:
-
name
:
example
layer
:
GENERAL
dsl
:
|
filter {
text {
...
...
test/e2e-v2/cases/satellite/native-protocols/lal.yaml
浏览文件 @
12a18e69
...
...
@@ -15,6 +15,7 @@
rules
:
-
name
:
example
layer
:
GENERAL
dsl
:
|
filter {
text {
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录