backend-receivers.md 8.8 KB
Newer Older
1

W
Wing 已提交
2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
# Choosing a receiver
Receiver is a defined concept in SkyWalking's backend. All modules which are responsible for receiving telemetry
or tracing data from other systems being monitored are all called **receivers**. If you are looking for the pull mode,
take a look at the [fetcher document](backend-fetcher.md).

We have the following receivers, and `default` implementors are provided in our Apache distribution.
1. **receiver-trace**. gRPC and HTTPRestful services that accept SkyWalking format traces.
1. **receiver-register**. gRPC and HTTPRestful services that provide service, service instance and endpoint register.
1. **service-mesh**. gRPC services that accept data from inbound mesh probes.
1. **receiver-jvm**. gRPC services that accept JVM metrics data.
1. **envoy-metric**. Envoy `metrics_service` and `ALS(access log service)` are supported by this receiver. The OAL script supports all GAUGE type metrics.
1. **receiver-profile**. gRPC services that accept profile task status and snapshot reporter.
1. **receiver-otel**. See [details](#opentelemetry-receiver). A receiver for analyzing metrics data from OpenTelemetry.
1. **receiver-meter**. See [details](backend-meter.md). A receiver for analyzing metrics in SkyWalking native meter format.
1. **receiver-browser**. gRPC services that accept browser performance data and error log.
1. **receiver-log**. A receiver for native log format. See [Log Analyzer](log-analyzer.md) for advanced features. 
1. **configuration-discovery**. gRPC services that handle configurationDiscovery.
1. **receiver-event**. gRPC services that handle events data.
静夜思朝颜's avatar
静夜思朝颜 已提交
20
1. **receiver-zabbix**. See [details](backend-zabbix.md).
21 22
1. Experimental receivers. 
    1. **receiver_zipkin**. See [details](#zipkin-receiver).
23

W
Wing 已提交
24
The sample settings of these receivers are by default included in `application.yml`, and also listed here:
25
```yaml
26
receiver-register:
27
  selector: ${SW_RECEIVER_REGISTER:default}
28
  default:
29

30
receiver-trace:
31
  selector: ${SW_RECEIVER_TRACE:default}
32
  default:
33

34
receiver-jvm:
35
  selector: ${SW_RECEIVER_JVM:default}
36
  default:
37

38
service-mesh:
39
  selector: ${SW_SERVICE_MESH:default}
40
  default:
41

42
envoy-metric:
43
  selector: ${SW_ENVOY_METRIC:default}
44
  default:
45 46 47
    acceptMetricsService: ${SW_ENVOY_METRIC_SERVICE:true}
    alsHTTPAnalysis: ${SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS:""}

48
receiver_zipkin:
49
  selector: ${SW_RECEIVER_ZIPKIN:-}
50
  default:
51 52 53 54 55 56 57 58 59
    host: ${SW_RECEIVER_ZIPKIN_HOST:0.0.0.0}
    port: ${SW_RECEIVER_ZIPKIN_PORT:9411}
    contextPath: ${SW_RECEIVER_ZIPKIN_CONTEXT_PATH:/}
    jettyMinThreads: ${SW_RECEIVER_ZIPKIN_JETTY_MIN_THREADS:1}
    jettyMaxThreads: ${SW_RECEIVER_ZIPKIN_JETTY_MAX_THREADS:200}
    jettyIdleTimeOut: ${SW_RECEIVER_ZIPKIN_JETTY_IDLE_TIMEOUT:30000}
    jettyAcceptorPriorityDelta: ${SW_RECEIVER_ZIPKIN_JETTY_DELTA:0}
    jettyAcceptQueueSize: ${SW_RECEIVER_ZIPKIN_QUEUE_SIZE:0}

60
receiver-profile:
61
  selector: ${SW_RECEIVER_PROFILE:default}
62
  default:
63 64 65 66

receiver-browser:
  selector: ${SW_RECEIVER_BROWSER:default}
  default:
67
    sampleRate: ${SW_RECEIVER_BROWSER_SAMPLE_RATE:10000}
68

69 70 71 72 73
log-analyzer:
   selector: ${SW_LOG_ANALYZER:default}
   default:
      lalFiles: ${SW_LOG_LAL_FILES:default}
      malFiles: ${SW_LOG_MAL_FILES:""}
74 75 76 77
  
configuration-discovery:
  selector: ${SW_CONFIGURATION_DISCOVERY:default}
  default:
78 79 80 81 82

receiver-event:
   selector: ${SW_RECEIVER_EVENT:default}
   default:

83 84 85
```

## gRPC/HTTP server for receiver
W
Wing 已提交
86 87
By default, all gRPC/HTTP services should be served at `core/gRPC` and `core/rest`.
But the `receiver-sharing-server` module allows all receivers to be served at
88 89 90
different ip:port, if you set them explicitly. 
```yaml
receiver-sharing-server:
91
  selector: ${SW_RECEIVER_SHARING_SERVER:default}
92
  default:
93 94 95 96 97 98 99 100
    host: ${SW_RECEIVER_JETTY_HOST:0.0.0.0}
    contextPath: ${SW_RECEIVER_JETTY_CONTEXT_PATH:/}
    authentication: ${SW_AUTHENTICATION:""}
    jettyMinThreads: ${SW_RECEIVER_SHARING_JETTY_MIN_THREADS:1}
    jettyMaxThreads: ${SW_RECEIVER_SHARING_JETTY_MAX_THREADS:200}
    jettyIdleTimeOut: ${SW_RECEIVER_SHARING_JETTY_IDLE_TIMEOUT:30000}
    jettyAcceptorPriorityDelta: ${SW_RECEIVER_SHARING_JETTY_DELTA:0}
    jettyAcceptQueueSize: ${SW_RECEIVER_SHARING_JETTY_QUEUE_SIZE:0}
101 102
```

W
Wing 已提交
103
Note: If you add these settings, make sure that they are not the same as the core module. This is because gRPC/HTTP servers of the core are still used for UI and OAP internal communications.
104

wu-sheng's avatar
wu-sheng 已提交
105 106
## OpenTelemetry receiver

W
Wing 已提交
107 108 109
The OpenTelemetry receiver supports ingesting agent metrics by meter-system. The OAP can load the configuration at bootstrap. 
If the new configuration is not well-formed, the OAP may fail to start up. The files are located at `$CLASSPATH/otel-<handler>-rules`.
E.g. The `oc` handler loads rules from `$CLASSPATH/otel-oc-rules`.
wu-sheng's avatar
wu-sheng 已提交
110 111 112 113 114

Supported handlers:
    * `oc`: [OpenCensus](https://github.com/open-telemetry/opentelemetry-collector/blob/master/exporter/opencensusexporter/README.md) gRPC service handler.

The rule file should be in YAML format, defined by the scheme described in [prometheus-fetcher](./backend-fetcher.md).
W
Wing 已提交
115
Note: `receiver-otel` only supports the `group`, `defaultMetricLevel`, and `metricsRules` nodes of the scheme due to its push mode.
wu-sheng's avatar
wu-sheng 已提交
116

W
Wing 已提交
117
To activate the `oc` handler and relevant rules of `istio`:
wu-sheng's avatar
wu-sheng 已提交
118 119 120 121 122 123 124
```yaml
receiver-otel:
  selector: ${SW_OTEL_RECEIVER:default}
  default:
    enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"oc"}
    enabledOcRules: ${SW_OTEL_RECEIVER_ENABLED_OC_RULES:"istio-controlplane"}
```
W
Wing 已提交
125 126 127
The receiver adds labels with `key = node_identifier_host_name` and `key = node_identifier_pid` to the collected data samples,
and values from `Node.identifier.host_name` and `Node.identifier.pid` defined in OpenCensus Agent Proto,
for identification of the metric data.
wu-sheng's avatar
wu-sheng 已提交
128 129 130 131 132

| Rule Name | Description | Configuration File | Data Source |
|----|----|-----|----|
|istio-controlplane| Metrics of Istio control panel | otel-oc-rules/istio-controlplane.yaml | Istio Control Panel -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
|oap| Metrics of SkyWalking OAP server itself | otel-oc-rules/oap.yaml | SkyWalking OAP Server(SelfObservability) -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
133
|vm| Metrics of VMs | otel-oc-rules/vm.yaml | Prometheus node-exporter(VMs) -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
W
wankai123 已提交
134 135 136
|k8s-cluster| Metrics of K8s cluster | otel-oc-rules/k8s-cluster.yaml | K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
|k8s-node| Metrics of K8s cluster | otel-oc-rules/k8s-node.yaml | cAdvisor & K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
|k8s-service| Metrics of K8s cluster | otel-oc-rules/k8s-service.yaml | cAdvisor & K8s kube-state-metrics -> OpenTelemetry Collector --OC format--> SkyWalking OAP Server |
wu-sheng's avatar
wu-sheng 已提交
137 138 139

## Meter receiver

W
Wing 已提交
140
The meter receiver supports accepting the metrics into the meter-system. The OAP can load the configuration at bootstrap. 
wu-sheng's avatar
wu-sheng 已提交
141 142 143

The file is written in YAML format, defined by the scheme described in [backend-meter](./backend-meter.md).

W
Wing 已提交
144
To activate the `default` implementation:
wu-sheng's avatar
wu-sheng 已提交
145 146 147 148 149 150
```yaml
receiver-meter:
  selector: ${SW_RECEIVER_METER:default}
  default:
```

151 152 153 154 155 156 157 158 159 160 161 162
To activate the meter rule files:
```yaml
agent-analyzer:
  selector: ${SW_AGENT_ANALYZER:default}
  default:
    meterAnalyzerActiveFiles: ${SW_METER_ANALYZER_ACTIVE_FILES:} # Which files could be meter analyzed, files split by ","
```

The receiver adds labels with `key = service` and `key = instance` to the collected data samples,
and values from service and service instance name defined in SkyWalking Agent,
for identification of the metric data.

163
## Zipkin receiver
W
Wing 已提交
164
The Zipkin receiver makes the OAP server work as an alternative Zipkin server implementation. It supports Zipkin v1/v2 formats through HTTP service.
165
Make sure you use this with `SW_STORAGE=zipkin-elasticsearch7` option to activate Zipkin storage implementation.
W
Wing 已提交
166
Once this receiver and storage are activated, SkyWalking's native traces would be ignored, and SkyWalking wouldn't analyze topology, metrics, and endpoint
167
dependency from Zipkin's trace. 
wu-sheng's avatar
wu-sheng 已提交
168

W
Wing 已提交
169
Use the following config to activate it.
wu-sheng's avatar
wu-sheng 已提交
170 171
```yaml
receiver_zipkin:
172
  selector: ${SW_RECEIVER_ZIPKIN:-}
wu-sheng's avatar
wu-sheng 已提交
173 174 175 176
  default:
    host: ${SW_RECEIVER_ZIPKIN_HOST:0.0.0.0}
    port: ${SW_RECEIVER_ZIPKIN_PORT:9411}
    contextPath: ${SW_RECEIVER_ZIPKIN_CONTEXT_PATH:/}
177 178 179 180 181
    jettyMinThreads: ${SW_RECEIVER_ZIPKIN_JETTY_MIN_THREADS:1}
    jettyMaxThreads: ${SW_RECEIVER_ZIPKIN_JETTY_MAX_THREADS:200}
    jettyIdleTimeOut: ${SW_RECEIVER_ZIPKIN_JETTY_IDLE_TIMEOUT:30000}
    jettyAcceptorPriorityDelta: ${SW_RECEIVER_ZIPKIN_JETTY_DELTA:0}
    jettyAcceptQueueSize: ${SW_RECEIVER_ZIPKIN_QUEUE_SIZE:0}
wu-sheng's avatar
wu-sheng 已提交
182 183
```

W
Wing 已提交
184 185 186
NOTE: Zipkin receiver is only provided in `apache-skywalking-apm-es7-x.y.z.tar.gz` tar.
This requires `zipkin-elasticsearch7` storage implementation to be activated.
Read [this](backend-storage.md#elasticsearch-7-with-zipkin-trace-extension) doc to learn about Zipkin as a storage option.