Skip to content

S
SkyWalking

apache / SkyWalking
上一次同步 1 年多

通知 302
Star 21345
Fork 6091
S
SkyWalking

前往新版Gitcode，体验更适合开发者的 AI 搜索 >>

未验证 提交 8de8c7c1 编写于 作者: P pg.yang 提交者： GitHub
浏览文件
操作

Support monitoring AWS Cloud EKS (#10199)

上级 bfb90f6c
展开全部 隐藏空白更改
内联 并排
Showing with 4352 addition and 2 deletion
.github/workflows/skywalking.yaml
浏览文件 @ 8de8c7c1
......@@ -645,6 +645,8 @@ jobs:
config: test/e2e-v2/cases/exporter/kafka/e2e.yaml
- name: Virtual MQ
config: test/e2e-v2/cases/virtual-mq/e2e.yaml
- name: AWS Cloud EKS
config: test/e2e-v2/cases/aws/eks/e2e.yaml
steps:
- uses: actions/checkout@v3
with:
......
docs/en/changes/changes.md
浏览文件 @ 8de8c7c1
......@@ -68,12 +68,16 @@
* Fix `time_bucket` of `ServiceTraffic` not set correctly in `slowSql` of MAL.
* Correct the TopN record query DAO of BanyanDB.
* Tweak interval settings of BanyanDB.
* Support monitoring AWS Cloud EKS.
#### UI
* Add Zipkin Lens UI to webapp, and proxy it to context path `/zipkin`.
* Migrate the build tool from vue cli to Vite4.
* Fix Instance Relation and Endpoint Relation dashboards show up.
* Add Micrometer icon
* Update MySQL UI to support MariaDB
* Add AWS menu for supporting AWS monitoring
#### Documentation
......
docs/en/setup/backend/backend-aws-eks-monitoring.md 0 → 100644
浏览文件 @ 8de8c7c1
# AWS Cloud EKS monitoring
SkyWalking leverages OpenTelemetry Collector with [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) to transfer the metrics to
[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
### Data flow
1. OpenTelemetry Collector fetches metrics from EKS via [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) and pushes metrics to SkyWalking OAP Server via the OpenCensus gRPC Exporter or OpenTelemetry gRPC exporter.
2. The SkyWalking OAP Server parses the expression with [MAL](../../concepts-and-designs/mal.md) to filter/calculate/aggregate and store the results.
### Set up
1. Deploy [amazon/aws-otel-collector](https://hub.docker.com/r/amazon/aws-otel-collector) with [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) to EKS
2. Config SkyWalking [OpenTelemetry receiver](opentelemetry-receiver.md).
### EKS Monitoring
[AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) provides multiple dimensions metrics for EKS cluster, node, service, etc.
Accordingly, SkyWalking observes the status, and payload of the EKS cluster, which is cataloged as a `LAYER: AWS_EKS` `Service` in the OAP. Meanwhile, the k8s nodes would be recognized as `LAYER: AWS_EKS` `instance`s. The k8s service would be recognized as `endpoint`s.
#### Specify Job Name
SkyWalking distinguishes AWS Cloud EKS metrics by attributes `job_name`, which value is `aws-cloud-eks-monitoring`.
You could leverage OTEL Collector processor to add the attribute as follows:
```yaml
processors:
resource/job-name:
attributes:
- key: job_name
value: aws-cloud-eks-monitoring
action: insert
```
Notice, if you don't specify `job_name` attribute, SkyWalking OAP will ignore the metrics
#### Supported Metrics
| Monitoring Panel | Unit | Metric Name | Catalog | Description | Data Source |
|---------------------------------------|---------|--------------------------------------------|------------|--------------------------------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Node Count | | eks_cluster_node_count | Service | The node count of the EKS cluster | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Failed Node Count | | eks_cluster_failed_node_count | Service | The failed node count of the EKS cluster | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Pod Count (namespace dimension) | | eks_cluster_namespace_count | Service | The count of pod in the EKS cluster(namespace dimension) | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Pod Count (service dimension) | | eks_cluster_service_count | Service | The count of pod in the EKS cluster(service dimension) | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX Dropped Count (per second) | count/s | eks_cluster_net_rx_dropped | Service | Network RX dropped count | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX Error Count (per second) | count/s | eks_cluster_net_rx_error | Service | Network RX error count | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX Dropped Count (per second) | count/s | eks_cluster_net_rx_dropped | Service | Network TX dropped count | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX Error Count (per second) | count/s | eks_cluster_net_rx_error | Service | Network TX error count | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Pod Count | | eks_cluster_node_pod_number | Instance | The count of pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| CPU Utilization | percent | eks_cluster_node_cpu_utilization | Instance | The CPU Utilization of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Memory Utilization | percent | eks_cluster_node_memory_utilization | Instance | The Memory Utilization of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX | bytes/s | eks_cluster_node_net_rx_bytes | Instance | Network RX bytes of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX Error Count | count/s | eks_cluster_node_net_rx_bytes | Instance | Network RX error count of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX | bytes/s | eks_cluster_node_net_rx_bytes | Instance | Network TX bytes of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX Error Count | count/s | eks_cluster_node_net_rx_bytes | Instance | Network TX error count of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Disk IO Write | bytes/s | eks_cluster_node_net_rx_bytes | Instance | The IO write bytes of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Disk IO Read | bytes/s | eks_cluster_node_net_rx_bytes | Instance | The IO read bytes of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| FS Utilization | percent | eks_cluster_node_net_rx_bytes | Instance | The filesystem utilization of the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| CPU Utilization | percent | eks_cluster_node_pod_cpu_utilization | Instance | The CPU Utilization of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Memory Utilization | percent | eks_cluster_node_pod_memory_utilization | Instance | The Memory Utilization of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX | bytes/s | eks_cluster_node_pod_net_rx_bytes | Instance | Network RX bytes of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX Error Count | count/s | eks_cluster_node_pod_net_rx_error | Instance | Network RX error count of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX | bytes/s | eks_cluster_node_pod_net_tx_bytes | Instance | Network RX bytes of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX Error Count | count/s | eks_cluster_node_pod_net_tx_error | Instance | Network RX error count of the pod running on the node | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| CPU Utilization | percent | eks_cluster_service_pod_cpu_utilization | Endpoint | The CPU Utilization of pod that belong to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Memory Utilization | percent | eks_cluster_service_pod_memory_utilization | Endpoint | The Memory Utilization of pod that belong to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX | bytes/s | eks_cluster_service_pod_net_rx_bytes | Endpoint | Network RX bytes of the pod that belong to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network RX Error Count | count/s | eks_cluster_service_pod_net_rx_error | Endpoint | Network TX error count of the pod that belongs to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX | bytes/s | eks_cluster_service_pod_net_tx_bytes | Endpoint | Network TX bytes of the pod that belong to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
| Network TX Error Count | count/s | eks_cluster_node_pod_net_tx_error | Endpoint | Network TX error count of the pod that belongs to the service | [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) |
### Customizations
You can customize your own metrics/expression/dashboard panel.
The metrics definition and expression rules are found in `/config/otel-rules/aws-eks/`.
The AWS Cloud EKS dashboard panel configurations are found in `/config/ui-initialized-templates/aws_eks`.
### OTEL Configuration Sample With AWS Container Insights Receiver
```yaml
extensions:
health_check:
receivers:
awscontainerinsightreceiver:
processors:
resource/job-name:
attributes:
- key: job_name
value: aws-cloud-eks-monitoring
action: insert
exporters:
otlp:
endpoint: oap-service:11800
tls:
insecure: true
logging:
loglevel: debug
service:
pipelines:
metrics:
receivers: [awscontainerinsightreceiver]
processors: [resource/job-name]
exporters: [otlp,logging]
extensions: [health_check]
```
Refer to [AWS Container Insights Receiver](https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/awscontainerinsightreceiver/README.md) for more information
docs/en/setup/backend/opentelemetry-receiver.md
浏览文件 @ 8de8c7c1
......@@ -40,5 +40,8 @@ for identification of the metric data.
| Metrics of MYSQL| otel-rules/mysql.yaml | prometheus/mysqld_exporter -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
| Metrics of PostgreSQL| otel-rules/postgresql.yaml | postgres_exporter -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
| Metrics of Apache APISIX| otel-rules/apisix.yaml | apisix prometheus plugin -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
| Metrics of AWS Cloud EKS| otel-rules/aws-eks/eks-cluster.yaml |AWS Container Insights Receiver -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
| Metrics of AWS Cloud EKS| otel-rules/aws-eks/eks-service.yaml |AWS Container Insights Receiver -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
| Metrics of AWS Cloud EKS| otel-rules/aws-eks/eks-node.yaml |AWS Container Insights Receiver -> OpenTelemetry Collector -- OC/OTLP exporter --> SkyWalking OAP Server |
**Note**: You can also use OpenTelemetry exporter to transport the metrics to SkyWalking OAP directly. See [OpenTelemetry Exporter](./backend-meter.md#opentelemetry-exporter).
docs/menu.yml
浏览文件 @ 8de8c7c1
......@@ -179,6 +179,10 @@ catalog:
catalog:
- name: "Linux Monitoring"
path: "/en/setup/backend/backend-vm-monitoring"
- name: "AWS Cloud Monitoring"
catalog:
- name: "EKS Monitoring"
path: "/en/setup/backend/backend-aws-eks-monitoring"
- name: "Browser Monitoring"
path: "/en/setup/service-agent/browser-agent"
- name: "Gateway Monitoring"
......
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java
浏览文件 @ 8de8c7c1
......@@ -136,7 +136,12 @@ public enum Layer {
/**
* Apache APISIX is an open source, dynamic, scalable, and high-performance cloud native API gateway.
*/
APISIX(21, true);
APISIX(21, true),
/**
* EKS (Amazon Elastic Kubernetes Service) is k8s service provided by AWS Cloud
*/
AWS_EKS(22, true);
private final int value;
/**
......
oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java
浏览文件 @ 8de8c7c1
......@@ -60,6 +60,7 @@ public class UITemplateInitializer {
Layer.FAAS.name(),
Layer.APISIX.name(),
Layer.VIRTUAL_MQ.name(),
Layer.AWS_EKS.name(),
"custom"
};
private final UITemplateManagementService uiTemplateManagementService;
......
oap-server/server-starter/src/main/resources/application.yml
浏览文件 @ 8de8c7c1
......@@ -353,7 +353,7 @@ receiver-otel:
selector: ${SW_OTEL_RECEIVER:default}
default:
enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"oc,otlp"}
enabledOtelRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_RULES:"apisix,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap"}
enabledOtelRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_RULES:"apisix,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*"}
receiver-zipkin:
selector: ${SW_RECEIVER_ZIPKIN:-}
......
oap-server/server-starter/src/main/resources/otel-rules/aws-eks/eks-cluster.yaml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'aws-cloud-eks-monitoring' }" # The OpenTelemetry job name
expPrefix: tag({tags -> tags.cluster = 'aws-eks-cluster::' + tags.ClusterName})
expSuffix: service(['cluster'], Layer.AWS_EKS)
metricPrefix: eks_cluster
metricsRules:
- name: node_count
exp: cluster_node_count.downsampling(LATEST)
- name: failed_node_count
exp: cluster_failed_node_count.downsampling(LATEST)
- name: namespace_count
exp: namespace_number_of_running_pods.sum(['Namespace','cluster'])
- name: service_count
exp: service_number_of_running_pods.sum(['Service','cluster'])
- name: net_rx_dropped
exp: node_network_rx_dropped
- name: net_rx_error
exp: node_network_rx_errors
- name: net_tx_dropped
exp: node_network_tx_dropped
- name: net_tx_error
exp: node_network_tx_errors
oap-server/server-starter/src/main/resources/otel-rules/aws-eks/eks-node.yaml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'aws-cloud-eks-monitoring' }" # The OpenTelemetry job name
expPrefix: tag({tags -> tags.cluster = 'aws-eks-cluster::' + tags.ClusterName})
expSuffix: instance(['cluster'],['NodeName'], Layer.AWS_EKS)
metricPrefix: eks_cluster_node
metricsRules:
- name: pod_number
exp: pod_number_of_containers.downsampling(SUM)
- name: cpu_utilization
exp: node_cpu_utilization
- name: memory_utilization
exp: node_memory_utilization
- name: net_rx_bytes
exp: node_network_rx_bytes
- name: net_rx_error
exp: node_network_rx_errors
- name: net_tx_bytes
exp: node_network_tx_bytes
- name: net_tx_error
exp: node_network_tx_errors
- name: disk_io_write
exp: node_diskio_io_service_bytes_write.sum(['device','cluster','NodeName'])
- name: disk_io_read
exp: node_diskio_io_service_bytes_read.sum(['device','cluster','NodeName'])
- name: fs_utilization
exp: node_filesystem_utilization.sum(['device','cluster','NodeName'])
# Pod
- name: pod_cpu_utilization
exp: pod_cpu_utilization.sum(['PodName','cluster','NodeName'])
- name: pod_memory_utilization
exp: pod_memory_utilization.sum(['PodName','cluster','NodeName'])
- name: pod_net_rx_bytes
exp: pod_network_rx_bytes.sum(['PodName','cluster','NodeName'])
- name: pod_net_rx_error
exp: pod_network_rx_errors.sum(['PodName','cluster','NodeName'])
- name: pod_net_tx_bytes
exp: pod_network_tx_bytes.sum(['PodName','cluster','NodeName'])
- name: pod_net_tx_error
exp: pod_network_tx_errors.sum(['PodName','cluster','NodeName'])
oap-server/server-starter/src/main/resources/otel-rules/aws-eks/eks-service.yaml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This will parse a textual representation of a duration. The formats
# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
# with days considered to be exactly 24 hours.
# <p>
# Examples:
# <pre>
# "PT20.345S" -- parses as "20.345 seconds"
# "PT15M" -- parses as "15 minutes" (where a minute is 60 seconds)
# "PT10H" -- parses as "10 hours" (where an hour is 3600 seconds)
# "P2D" -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
# "P2DT3H4M" -- parses as "2 days, 3 hours and 4 minutes"
# "P-6H3M" -- parses as "-6 hours and +3 minutes"
# "-P6H3M" -- parses as "-6 hours and -3 minutes"
# "-P-6H+3M" -- parses as "+6 hours and -3 minutes"
# </pre>
filter: "{ tags -> tags.job_name == 'aws-cloud-eks-monitoring' && tags.Service?.trim() }" # The OpenTelemetry job name
expPrefix: tag({tags -> tags.cluster = 'aws-eks-cluster::' + tags.ClusterName})
expSuffix: endpoint(['cluster'],['Service'], Layer.AWS_EKS)
metricPrefix: eks_cluster_service
metricsRules:
- name: pod_cpu_utilization
exp: pod_cpu_utilization
- name: pod_memory_utilization
exp: pod_memory_utilization
- name: pod_net_rx_bytes
exp: pod_network_rx_bytes
- name: pod_net_rx_error
exp: pod_network_rx_errors
- name: pod_net_tx_bytes
exp: pod_network_tx_bytes
- name: pod_net_tx_error
exp: pod_network_tx_errors
oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_eks/aws_eks-cluster.json 0 → 100644
浏览文件 @ 8de8c7c1
/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
[
{
"id": "AWS_EKS-cluster",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 6,
"h": 13,
"i": "1",
"type": "Widget",
"widget": {
"title": "Node Count"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_node_count"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 13,
"i": "2",
"type": "Widget",
"widget": {
"title": "Failed Node Count"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_failed_node_count"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 12,
"y": 0,
"w": 6,
"h": 13,
"i": "3",
"type": "Widget",
"widget": {
"title": "Pod Count (namespace dimension)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_namespace_count"
],
"metricTypes": [
"readLabeledMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 13,
"i": "4",
"type": "Widget",
"widget": {
"title": "Pod Count (service dimension)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_count"
],
"metricTypes": [
"readLabeledMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 0,
"y": 13,
"w": 6,
"h": 13,
"i": "5",
"type": "Widget",
"widget": {
"title": "Network RX Dropped Count (per second, per node)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_net_rx_dropped"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 6,
"y": 13,
"w": 6,
"h": 13,
"i": "6",
"type": "Widget",
"widget": {
"title": "Network RX Error Count (per second, per node)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_net_rx_error"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 12,
"y": 13,
"w": 6,
"h": 13,
"i": "7",
"type": "Widget",
"widget": {
"title": "Network TX Dropped Count (per second, per node)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_net_tx_dropped"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 18,
"y": 13,
"w": 6,
"h": 13,
"i": "8",
"type": "Widget",
"widget": {
"title": "Network TX Error Count (per second, per node)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_net_tx_error"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
},
{
"widgetId": "7"
},
{
"widgetId": "8"
}
]
},
{
"x": 0,
"y": 26,
"w": 12,
"h": 26,
"i": "12",
"type": "Widget",
"widget": {
"title": "Node List"
},
"graph": {
"type": "InstanceList",
"dashboardName": "AWS_EKS-node",
"fontSize": 12
},
"metrics": [
"eks_cluster_node_cpu_utilization",
"eks_cluster_node_memory_utilization",
"eks_cluster_node_network_tx_errors",
"eks_cluster_node_network_rx_errors"
],
"metricTypes": [
"readMetricsValues",
"readMetricsValues",
"readLabeledMetricsValues",
"readLabeledMetricsValues"
],
"metricConfig": [
{
"label": "CPU Usage",
"unit": "%",
"calculation": "average"
},
{
"label": "Memory Usage",
"unit": "%",
"calculation": "average"
},
{
"label": "Network TX Errors",
"unit": "count/s",
"calculation": "average"
},
{
"label": "Network RX Errors",
"unit": "count/s",
"calculation": "average"
}
],
"moved": false
},
{
"x": 12,
"y": 52,
"w": 12,
"h": 26,
"i": "13",
"type": "Widget",
"widget": {
"title": "Service List"
},
"graph": {
"type": "EndpointList",
"dashboardName": "AWS_EKS-service",
"fontSize": 12,
"showXAxis": false,
"showYAxis": false,
"showGroup": true
},
"metrics": [
"eks_cluster_service_pod_cpu_utilization",
"eks_cluster_service_pod_memory_utilization",
"eks_cluster_service_pod_net_tx_error",
"eks_cluster_service_pod_net_rx_error"
],
"metricTypes": [
"readMetricsValues",
"readMetricsValues",
"readMetricsValues",
"readMetricsValues"
],
"metricConfig": [
{
"label": "CPU Usage",
"unit": "%",
"calculation": "average"
},
{
"label": "Memory Usage",
"unit": "%",
"calculation": "average"
},
{
"label": "Network TX Errors",
"unit": "count/s",
"calculation": "average"
},
{
"label": "Network RX Errors",
"unit": "count/s",
"calculation": "average"
}
]
}
],
"layer": "AWS_EKS",
"entity": "Service",
"name": "AWS_EKS-cluster",
"id": "AWS_EKS-cluster",
"isRoot": false
}
}
]
oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_eks/aws_eks-root.json 0 → 100644
浏览文件 @ 8de8c7c1
/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
[
{
"id": "AWS_EKS-Root",
"configuration": {
"children": [
{
"x": 0,
"y": 2,
"w": 24,
"h": 51,
"i": "0",
"type": "Widget",
"widget": {
"title": ""
},
"graph": {
"type": "ServiceList",
"dashboardName": "AWS_EKS-cluster",
"fontSize": 12,
"showXAxis": false,
"showYAxis": false,
"showGroup": true
},
"moved": false
},
{
"x": 0,
"y": 0,
"w": 24,
"h": 2,
"i": "100",
"type": "Text",
"graph": {
"fontColor":"blue",
"backgroundColor":"white",
"content":"Provide AWS Could monitoring through AWS Container Insights Receiver",
"fontSize":14,
"textAlign":"left",
"url":"https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-aws-eks-monitoring/"
},
"moved": false
}
],
"id": "AWS_EKS-Root",
"layer": "AWS_EKS",
"entity": "All",
"name": "AWS_EKS-Root",
"isRoot": true
}
}
]
oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_eks/aws_eks-service.json 0 → 100644
浏览文件 @ 8de8c7c1
/**
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
[
{
"id": "AWS_EKS-service",
"configuration": {
"children": [
{
"x": 0,
"y": 0,
"w": 24,
"h": 52,
"i": "0",
"type": "Tab",
"children": [
{
"name": "Pod",
"children": [
{
"x": 0,
"y": 0,
"w": 6,
"h": 13,
"i": "1",
"type": "Widget",
"widget": {
"title": "CPU Utilization (%)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_cpu_utilization"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
},
{
"x": 6,
"y": 0,
"w": 6,
"h": 13,
"i": "2",
"type": "Widget",
"widget": {
"title": "Memory Utilization (%)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_memory_utilization"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
},
{
"x": 12,
"y": 0,
"w": 6,
"h": 13,
"i": "3",
"type": "Widget",
"widget": {
"title": "Network RX (KB per second)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_net_rx_bytes"
],
"metricTypes": [
"readMetricsValues"
],
"metricConfig": [
{
"calculation": "byteToKB"
}
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
},
{
"x": 18,
"y": 0,
"w": 6,
"h": 13,
"i": "4",
"type": "Widget",
"widget": {
"title": "Network RX Error Count (per second)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_net_rx_error"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
},
{
"x": 0,
"y": 13,
"w": 6,
"h": 13,
"i": "5",
"type": "Widget",
"widget": {
"title": "Network TX (KB per second)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_net_tx_bytes"
],
"metricTypes": [
"readMetricsValues"
],
"metricConfig": [
{
"calculation": "byteToKB"
}
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
},
{
"x": 6,
"y": 13,
"w": 6,
"h": 13,
"i": "6",
"type": "Widget",
"widget": {
"title": "Network TX Error Count (per second)"
},
"graph": {
"type": "Line",
"step": false,
"smooth": false,
"showSymbol": false,
"showXAxis": true,
"showYAxis": true
},
"metrics": [
"eks_cluster_service_pod_net_tx_error"
],
"metricTypes": [
"readMetricsValues"
],
"associate": [
{
"widgetId": "1"
},
{
"widgetId": "2"
},
{
"widgetId": "3"
},
{
"widgetId": "4"
},
{
"widgetId": "5"
},
{
"widgetId": "6"
}
]
}
]
}
]
}
],
"layer": "AWS_EKS",
"entity": "Endpoint",
"name": "AWS_EKS-service",
"id": "AWS_EKS-service",
"isRoot": false
}
}
]
test/e2e-v2/cases/aws/eks/docker-compose.yml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
version: '2.1'
services:
oap:
extends:
file: ../../../script/docker-compose/base-compose.yml
service: oap
ports:
- 12800
networks:
e2e:
sender:
image: "eclipse-temurin:8-jre"
volumes:
- ./../../../java-test-service/e2e-mock-sender/target/e2e-mock-sender-2.0.0.jar:/e2e-mock-sender-2.0.0.jar
- ./mock-data:/data/otel-metrics
command: [ "java", "-jar", "/e2e-mock-sender-2.0.0.jar" ]
environment:
OAP_HOST: oap
OAP_GRPC_PORT: 11800
OTEL_METRICS_DATA_PATH: /data/otel-metrics
networks:
- e2e
ports:
- 9093:9093
healthcheck:
test: ["CMD", "sh", "-c", "nc -nz 127.0.0.1 9093"]
interval: 5s
timeout: 60s
retries: 120
depends_on:
oap:
condition: service_healthy
networks:
e2e:
test/e2e-v2/cases/aws/eks/e2e.yaml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
setup:
env: compose
file: docker-compose.yml
timeout: 20m
init-system-environment: ../../../script/env
steps:
- name: set PATH
command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
- name: install yq
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
- name: install swctl
command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
trigger:
action: http
interval: 3s
times: 10
url: http://localhost:9093/otel-metrics/send
method: GET
verify:
retry:
count: 10
interval: 3s
cases:
- includes:
- ./eks-cases.yaml
test/e2e-v2/cases/aws/eks/eks-cases.yaml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# This file is used to show how to write configuration files and can be used to test.
cases:
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
expected: expected/service.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql instance ls --service-name=aws-eks-cluster::SkyWalking
expected: expected/instance.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql endpoint ls --service-name=aws-eks-cluster::SkyWalking
expected: expected/endpoint.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=eks_cluster_node_memory_utilization --service-name=aws-eks-cluster::SkyWalking --instance-name=ip-172-31-23-33.ap-northeast-1.compute.internal | yq e 'to_entries' -
expected: expected/metrics-has-value.yml
- query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear --name=eks_cluster_service_pod_net_rx_bytes --service-name=aws-eks-cluster::SkyWalking --endpoint-name=kube-dns | yq e 'to_entries' -
expected: expected/metrics-has-value.yml
test/e2e-v2/cases/aws/eks/expected/endpoint.yml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains .}}
- id: {{ b64enc "aws-eks-cluster::SkyWalking" }}.1_{{ b64enc "kube-dns" }}
name: kube-dns
{{- end}}
test/e2e-v2/cases/aws/eks/expected/instance.yml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ notEmpty .id }}
name: ip-172-31-10-158.ap-northeast-1.compute.internal
instanceuuid: {{ notEmpty .instanceuuid }}
attributes: []
language: UNKNOWN
{{- end }}
test/e2e-v2/cases/aws/eks/expected/metrics-has-value.yml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- key: {{ notEmpty .key }}
value: {{ ge .value 0 }}
{{- end }}
test/e2e-v2/cases/aws/eks/expected/service.yml 0 → 100644
浏览文件 @ 8de8c7c1
# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
{{- contains . }}
- id: {{ b64enc "aws-eks-cluster::SkyWalking" }}.1
name: aws-eks-cluster::SkyWalking
group: aws-eks-cluster
shortname: SkyWalking
normal: true
layers:
- AWS_EKS
{{- end }}
test/e2e-v2/cases/aws/eks/mock-data/otel-mock-metrics.json 0 → 100644
浏览文件 @ 8de8c7c1
{
"resourceMetrics": [
{
"resource": {
"attributes": [
{
"key": "AutoScalingGroupName",
"value": {
"stringValue": "eks-node-2-46c25931-6b80-2e04-5023-cca9e97c589d"
}
},
{
"key": "ClusterName",
"value": {
"stringValue": "SkyWalking"
}
},
{
"key": "Type",
"value": {
"stringValue": "Node"
}
},
{
"key": "NodeName",
"value": {
"stringValue": "ip-172-31-10-158.ap-northeast-1.compute.internal"
}
},
{
"key": "InstanceId",
"value": {
"stringValue": "i-06c02c575d866b59b"
}
},
{
"key": "InstanceType",
"value": {
"stringValue": "t3.medium"
}
},
{
"key": "kubernetes",
"value": {
"stringValue": "{\"host\":\"ip-172-31-10-158.ap-northeast-1.compute.internal\"}"
}
},
{
"key": "Sources",
"value": {
"stringValue": "[\"cadvisor\",\"/proc\",\"pod\",\"calculated\"]"
}
},
{
"key": "Timestamp",
"value": {
"stringValue": "1669442689514"
}
},
{
"key": "Version",
"value": {
"stringValue": "0"
}
},
{
"key": "job_name",
"value": {
"stringValue": "aws-cloud-eks-monitoring"
}
}
]
},
"scopeMetrics": [
{
"scope": {},
"metrics": [
{
"name": "node_memory_utilization",
"unit": "Percent",
"gauge": {
"dataPoints": [
{
"timeUnixNano": "1669442689514625306",
"asDouble": 16.00333280879926
}
]
}
}
]
}
]
},
{
"resource": {
"attributes": [
{
"key": "NodeName",
"value": {
"stringValue": "ip-172-31-10-158.ap-northeast-1.compute.internal"
}
},
{
"key": "Sources",
"value": {
"stringValue": "[\"apiserver\"]"
}
},
{
"key": "Type",
"value": {
"stringValue": "Cluster"
}
},
{
"key": "Version",
"value": {
"stringValue": "0"
}
},
{
"key": "ClusterName",
"value": {
"stringValue": "SkyWalking"
}
},
{
"key": "Timestamp",
"value": {
"stringValue": "1669442693795"
}
},
{
"key": "job_name",
"value": {
"stringValue": "aws-cloud-eks-monitoring"
}
}
]
},
"scopeMetrics": [
{
"scope": {},
"metrics": [
{
"name": "cluster_node_count",
"unit": "Count",
"gauge": {
"dataPoints": [
{
"timeUnixNano": "1669442693795487301",
"asInt": "1"
}
]
}
}
]
}
]
},
{
"resource": {
"attributes": [
{
"key": "Version",
"value": {
"stringValue": "0"
}
},
{
"key": "InstanceId",
"value": {
"stringValue": "i-06c02c575d866b59b"
}
},
{
"key": "InstanceType",
"value": {
"stringValue": "t3.medium"
}
},
{
"key": "AutoScalingGroupName",
"value": {
"stringValue": "eks-node-2-46c25931-6b80-2e04-5023-cca9e97c589d"
}
},
{
"key": "PodName",
"value": {
"stringValue": "eks-sample-linux-deployment"
}
},
{
"key": "Type",
"value": {
"stringValue": "Pod"
}
},
{
"key": "NodeName",
"value": {
"stringValue": "ip-172-31-10-158.ap-northeast-1.compute.internal"
}
},
{
"key": "ClusterName",
"value": {
"stringValue": "SkyWalking"
}
},
{
"key": "pod_status",
"value": {
"stringValue": "Running"
}
},
{
"key": "kubernetes",
"value": {
"stringValue": "{\"host\":\"ip-172-31-10-158.ap-northeast-1.compute.internal\",\"labels\":{\"app\":\"eks-sample-linux-app\",\"pod-template-hash\":\"764959fd66\"},\"namespace_name\":\"default\",\"pod_id\":\"52c5847b-f68c-4d4b-ae9e-e85f95f972bf\",\"pod_name\":\"eks-sample-linux-deployment-764959fd66-fql4v\",\"pod_owners\":[{\"owner_kind\":\"Deployment\",\"owner_name\":\"eks-sample-linux-deployment\"}]}"
}
},
{
"key": "Sources",
"value": {
"stringValue": "[\"cadvisor\",\"pod\",\"calculated\"]"
}
},
{
"key": "Namespace",
"value": {
"stringValue": "default"
}
},
{
"key": "Timestamp",
"value": {
"stringValue": "1669442684150"
}
},
{
"key": "Service",
"value": {
"stringValue": "kube-dns"
}
},
{
"key": "job_name",
"value": {
"stringValue": "aws-cloud-eks-monitoring"
}
}
]
},
"scopeMetrics": [
{
"scope": {},
"metrics": [
{
"name": "pod_network_rx_bytes",
"unit": "Bytes/Second",
"gauge": {
"dataPoints": [
{
"timeUnixNano": "1669442684150753695",
"asDouble": 0
}
]
}
}
]
}
]
}
]
}
test/e2e-v2/java-test-service/e2e-mock-sender/pom.xml
浏览文件 @ 8de8c7c1
......@@ -48,6 +48,21 @@
<artifactId>zipkin</artifactId>
<version>2.23.16</version>
</dependency>
<dependency>
<groupId>org.apache.skywalking</groupId>
<artifactId>opentelemetry-proto</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>com.google.protobuf</groupId>
<artifactId>protobuf-java-util</artifactId>
<version>3.21.12</version>
</dependency>
<dependency>
<groupId>com.jayway.jsonpath</groupId>
<artifactId>json-path</artifactId>
<version>2.7.0</version>
</dependency>
</dependencies>
<build>
<plugins>
......
test/e2e-v2/java-test-service/e2e-mock-sender/src/main/java/org/apache/skywalking/e2e/E2EConfiguration.java
浏览文件 @ 8de8c7c1
......@@ -31,4 +31,5 @@ public class E2EConfiguration {
private String zipkinKafkaBootstrapServers;
private String zipkinKafkaGroupId;
private String zipkinKafkaTopic;
private String otelMetricsDataPath;
}
test/e2e-v2/java-test-service/e2e-mock-sender/src/main/java/org/apache/skywalking/e2e/controller/OtelMetricsSender.java 0 → 100644
浏览文件 @ 8de8c7c1
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
*/
package org.apache.skywalking.e2e.controller;
import com.google.protobuf.util.JsonFormat;
import com.jayway.jsonpath.Configuration;
import com.jayway.jsonpath.DocumentContext;
import com.jayway.jsonpath.EvaluationListener;
import com.jayway.jsonpath.JsonPath;
import com.jayway.jsonpath.Option;
import io.grpc.ManagedChannel;
import io.grpc.internal.DnsNameResolverProvider;
import io.grpc.netty.NettyChannelBuilder;
import io.grpc.stub.StreamObserver;
import io.opentelemetry.proto.collector.metrics.v1.ExportMetricsServiceRequest;
import io.opentelemetry.proto.collector.metrics.v1.ExportMetricsServiceResponse;
import io.opentelemetry.proto.collector.metrics.v1.MetricsServiceGrpc;
import java.io.File;
import java.io.IOException;
import lombok.extern.slf4j.Slf4j;
import org.apache.skywalking.e2e.E2EConfiguration;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
@Slf4j
@RestController
@RequestMapping("/otel-metrics")
public class OtelMetricsSender {
private static final int MAX_INBOUND_MESSAGE_SIZE = 1024 * 1024 * 50;
private final MetricsServiceGrpc.MetricsServiceStub metricsServiceStub;
private final String otelMetricsDataPath;
public OtelMetricsSender(final E2EConfiguration configuration) {
final ManagedChannel channel = NettyChannelBuilder.forAddress(
configuration.getOapHost(), Integer.parseInt(configuration.getOapGrpcPort()))
.nameResolverFactory(new DnsNameResolverProvider())
.maxInboundMessageSize(MAX_INBOUND_MESSAGE_SIZE)
.usePlaintext()
.build();
this.metricsServiceStub = MetricsServiceGrpc.newStub(channel);
this.otelMetricsDataPath = configuration.getOtelMetricsDataPath();
}
/**
* Send otel metrics data that base on JSON formatted file to OAP
*
* This method will detect files whose name end with '.json' in the directory specified by
* `org.apache.skywalking.e2e.E2EConfiguration#otelMetricsDataPath`, and send the data in the file to OAP. Also
* rewrite field which path is '$..startTimeUnixNano', '$..timeUnixNano' to current time in nano
*/
@GetMapping("send")
public String sendMetricsByTemplate() throws IOException {
File otelData = new File(this.otelMetricsDataPath);
if (!otelData.exists() || !otelData.isDirectory()) {
String msg = "The path must be a folder : " + this.otelMetricsDataPath;
log.error(msg);
return msg;
}
final File[] files = findJSONFiles(otelData);
if (files.length == 0) {
String msg = "The folder doesn't contain any json file : " + this.otelMetricsDataPath;
log.error(msg);
return msg;
}
for (File file : files) {
final ExportMetricsServiceRequest.Builder builder = ExportMetricsServiceRequest.newBuilder();
String jsonData = rewriteTimeField(file);
JsonFormat.parser().merge(jsonData, builder);
sendReq(builder);
}
return "ok";
}
private String rewriteTimeField(File file) throws IOException {
final long nanoTime = System.nanoTime();
final Configuration configuration = Configuration.builder()
.options(Option.SUPPRESS_EXCEPTIONS)
.evaluationListener(found -> {
log.info("rewrite json field: {}, {}->{}", found.path(),
found.result(), nanoTime
);
return EvaluationListener.EvaluationContinuation.CONTINUE;
}).build();
final DocumentContext documentContext = JsonPath.using(configuration).parse(file);
documentContext.set("$..timeUnixNano", nanoTime);
documentContext.set("$..startTimeUnixNano", nanoTime);
return documentContext.jsonString();
}
private void sendReq(final ExportMetricsServiceRequest.Builder builder) {
this.metricsServiceStub.export(builder.build(), new StreamObserver<ExportMetricsServiceResponse>() {
@Override
public void onNext(final ExportMetricsServiceResponse exportMetricsServiceResponse) {
}
@Override
public void onError(final Throwable throwable) {
log.error("sendOtelMetrics by template error ", throwable);
}
@Override
public void onCompleted() {
}
});
}
private File[] findJSONFiles(final File otelData) {
return otelData.listFiles((dir, name) -> name.endsWith(".json"));
}
}
test/e2e-v2/java-test-service/e2e-mock-sender/src/main/resources/application.yml
浏览文件 @ 8de8c7c1
......@@ -26,3 +26,4 @@ e2e:
zipkin-kafka-bootstrapServers: ${ZIPKIN_KAFKA_BOOTSTRAP_SERVERS:127.0.0.1:9092}
zipkin-kafka-groupId: ${ZIPKIN_KAFKA_GROUPID:zipkin}
zipkin-kafka-topic: ${ZIPKIN_KAFKA_TOPIC:zipkin}
otel-metrics-data-path: ${OTEL_METRICS_DATA_PATH:/data/otel-metrics/}
test/e2e-v2/java-test-service/opentelemetry-proto/pom.xml 0 → 100644
浏览文件 @ 8de8c7c1
<?xml version="1.0" encoding="UTF-8"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one or more
~ contributor license agreements. See the NOTICE file distributed with
~ this work for additional information regarding copyright ownership.
~ The ASF licenses this file to You under the Apache License, Version 2.0
~ (the "License"); you may not use this file except in compliance with
~ the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing, software
~ distributed under the License is distributed on an "AS IS" BASIS,
~ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
~ See the License for the specific language governing permissions and
~ limitations under the License.
~
-->
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
<artifactId>apache-skywalking-e2e</artifactId>
<groupId>org.apache.skywalking</groupId>
<version>2.0.0</version>
</parent>
<modelVersion>4.0.0</modelVersion>
<artifactId>opentelemetry-proto</artifactId>
<properties>
<grpc.version>1.43.2</grpc.version>
<os-maven-plugin.version>1.5.0.Final</os-maven-plugin.version>
<protobuf-maven-plugin.version>0.6.1</protobuf-maven-plugin.version>
</properties>
<dependencies>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-protobuf</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-stub</artifactId>
<version>${grpc.version}</version>
</dependency>
<dependency>
<groupId>io.grpc</groupId>
<artifactId>grpc-netty</artifactId>
<version>${grpc.version}</version>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>kr.motd.maven</groupId>
<artifactId>os-maven-plugin</artifactId>
<version>${os-maven-plugin.version}</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>detect</goal>
</goals>
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.xolstice.maven.plugins</groupId>
<artifactId>protobuf-maven-plugin</artifactId>
<version>${protobuf-maven-plugin.version}</version>
<configuration>
<!--
The version of protoc must match protobuf-java. If you don't depend on
protobuf-java directly, you will be transitively depending on the
protobuf-java version that grpc depends on.
-->
<protocArtifact>com.google.protobuf:protoc:3.19.2:exe:${os.detected.classifier}
</protocArtifact>
<pluginId>grpc-java</pluginId>
<pluginArtifact>io.grpc:protoc-gen-grpc-java:1.42.1:exe:${os.detected.classifier}
</pluginArtifact>
</configuration>
<executions>
<execution>
<goals>
<goal>compile</goal>
<goal>compile-custom</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project>
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/logs/v1/logs_service.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2020, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.collector.logs.v1;
import "opentelemetry/proto/logs/v1/logs.proto";
option csharp_namespace = "OpenTelemetry.Proto.Collector.Logs.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.collector.logs.v1";
option java_outer_classname = "LogsServiceProto";
option go_package = "go.opentelemetry.io/proto/otlp/collector/logs/v1";
// Service that can be used to push logs between one Application instrumented with
// OpenTelemetry and an collector, or between an collector and a central collector (in this
// case logs are sent/received to/from multiple Applications).
service LogsService {
// For performance reasons, it is recommended to keep this RPC
// alive for the entire life of the application.
rpc Export(ExportLogsServiceRequest) returns (ExportLogsServiceResponse) {}
}
message ExportLogsServiceRequest {
// An array of ResourceLogs.
// For data coming from a single resource this array will typically contain one
// element. Intermediary nodes (such as OpenTelemetry Collector) that receive
// data from multiple origins typically batch the data before forwarding further and
// in that case this array will contain multiple elements.
repeated opentelemetry.proto.logs.v1.ResourceLogs resource_logs = 1;
}
message ExportLogsServiceResponse {
// The details of a partially successful export request.
//
// If the request is only partially accepted
// (i.e. when the server accepts only parts of the data and rejects the rest)
// the server MUST initialize the `partial_success` field and MUST
// set the `rejected_<signal>` with the number of items it rejected.
//
// Servers MAY also make use of the `partial_success` field to convey
// warnings/suggestions to senders even when the request was fully accepted.
// In such cases, the `rejected_<signal>` MUST have a value of `0` and
// the `error_message` MUST be non-empty.
//
// A `partial_success` message with an empty value (rejected_<signal> = 0 and
// `error_message` = "") is equivalent to it not being set/present. Senders
// SHOULD interpret it the same way as in the full success case.
ExportLogsPartialSuccess partial_success = 1;
}
message ExportLogsPartialSuccess {
// The number of rejected log records.
//
// A `rejected_<signal>` field holding a `0` value indicates that the
// request was fully accepted.
int64 rejected_log_records = 1;
// A developer-facing human-readable message in English. It should be used
// either to explain why the server rejected parts of the data during a partial
// success or to convey warnings/suggestions during a full success. The message
// should offer guidance on how users can address such issues.
//
// error_message is an optional field. An error_message with an empty value
// is equivalent to it not being set.
string error_message = 2;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/metrics/v1/metrics_service.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2019, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.collector.metrics.v1;
import "opentelemetry/proto/metrics/v1/metrics.proto";
option csharp_namespace = "OpenTelemetry.Proto.Collector.Metrics.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.collector.metrics.v1";
option java_outer_classname = "MetricsServiceProto";
option go_package = "go.opentelemetry.io/proto/otlp/collector/metrics/v1";
// Service that can be used to push metrics between one Application
// instrumented with OpenTelemetry and a collector, or between a collector and a
// central collector.
service MetricsService {
// For performance reasons, it is recommended to keep this RPC
// alive for the entire life of the application.
rpc Export(ExportMetricsServiceRequest) returns (ExportMetricsServiceResponse) {}
}
message ExportMetricsServiceRequest {
// An array of ResourceMetrics.
// For data coming from a single resource this array will typically contain one
// element. Intermediary nodes (such as OpenTelemetry Collector) that receive
// data from multiple origins typically batch the data before forwarding further and
// in that case this array will contain multiple elements.
repeated opentelemetry.proto.metrics.v1.ResourceMetrics resource_metrics = 1;
}
message ExportMetricsServiceResponse {
// The details of a partially successful export request.
//
// If the request is only partially accepted
// (i.e. when the server accepts only parts of the data and rejects the rest)
// the server MUST initialize the `partial_success` field and MUST
// set the `rejected_<signal>` with the number of items it rejected.
//
// Servers MAY also make use of the `partial_success` field to convey
// warnings/suggestions to senders even when the request was fully accepted.
// In such cases, the `rejected_<signal>` MUST have a value of `0` and
// the `error_message` MUST be non-empty.
//
// A `partial_success` message with an empty value (rejected_<signal> = 0 and
// `error_message` = "") is equivalent to it not being set/present. Senders
// SHOULD interpret it the same way as in the full success case.
ExportMetricsPartialSuccess partial_success = 1;
}
message ExportMetricsPartialSuccess {
// The number of rejected data points.
//
// A `rejected_<signal>` field holding a `0` value indicates that the
// request was fully accepted.
int64 rejected_data_points = 1;
// A developer-facing human-readable message in English. It should be used
// either to explain why the server rejected parts of the data during a partial
// success or to convey warnings/suggestions during a full success. The message
// should offer guidance on how users can address such issues.
//
// error_message is an optional field. An error_message with an empty value
// is equivalent to it not being set.
string error_message = 2;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/collector/trace/v1/trace_service.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2019, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.collector.trace.v1;
import "opentelemetry/proto/trace/v1/trace.proto";
option csharp_namespace = "OpenTelemetry.Proto.Collector.Trace.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.collector.trace.v1";
option java_outer_classname = "TraceServiceProto";
option go_package = "go.opentelemetry.io/proto/otlp/collector/trace/v1";
// Service that can be used to push spans between one Application instrumented with
// OpenTelemetry and a collector, or between a collector and a central collector (in this
// case spans are sent/received to/from multiple Applications).
service TraceService {
// For performance reasons, it is recommended to keep this RPC
// alive for the entire life of the application.
rpc Export(ExportTraceServiceRequest) returns (ExportTraceServiceResponse) {}
}
message ExportTraceServiceRequest {
// An array of ResourceSpans.
// For data coming from a single resource this array will typically contain one
// element. Intermediary nodes (such as OpenTelemetry Collector) that receive
// data from multiple origins typically batch the data before forwarding further and
// in that case this array will contain multiple elements.
repeated opentelemetry.proto.trace.v1.ResourceSpans resource_spans = 1;
}
message ExportTraceServiceResponse {
// The details of a partially successful export request.
//
// If the request is only partially accepted
// (i.e. when the server accepts only parts of the data and rejects the rest)
// the server MUST initialize the `partial_success` field and MUST
// set the `rejected_<signal>` with the number of items it rejected.
//
// Servers MAY also make use of the `partial_success` field to convey
// warnings/suggestions to senders even when the request was fully accepted.
// In such cases, the `rejected_<signal>` MUST have a value of `0` and
// the `error_message` MUST be non-empty.
//
// A `partial_success` message with an empty value (rejected_<signal> = 0 and
// `error_message` = "") is equivalent to it not being set/present. Senders
// SHOULD interpret it the same way as in the full success case.
ExportTracePartialSuccess partial_success = 1;
}
message ExportTracePartialSuccess {
// The number of rejected spans.
//
// A `rejected_<signal>` field holding a `0` value indicates that the
// request was fully accepted.
int64 rejected_spans = 1;
// A developer-facing human-readable message in English. It should be used
// either to explain why the server rejected parts of the data during a partial
// success or to convey warnings/suggestions during a full success. The message
// should offer guidance on how users can address such issues.
//
// error_message is an optional field. An error_message with an empty value
// is equivalent to it not being set.
string error_message = 2;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/common/v1/common.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2019, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.common.v1;
option csharp_namespace = "OpenTelemetry.Proto.Common.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.common.v1";
option java_outer_classname = "CommonProto";
option go_package = "go.opentelemetry.io/proto/otlp/common/v1";
// AnyValue is used to represent any type of attribute value. AnyValue may contain a
// primitive value such as a string or integer or it may contain an arbitrary nested
// object containing arrays, key-value lists and primitives.
message AnyValue {
// The value is one of the listed fields. It is valid for all values to be unspecified
// in which case this AnyValue is considered to be "empty".
oneof value {
string string_value = 1;
bool bool_value = 2;
int64 int_value = 3;
double double_value = 4;
ArrayValue array_value = 5;
KeyValueList kvlist_value = 6;
bytes bytes_value = 7;
}
}
// ArrayValue is a list of AnyValue messages. We need ArrayValue as a message
// since oneof in AnyValue does not allow repeated fields.
message ArrayValue {
// Array of values. The array may be empty (contain 0 elements).
repeated AnyValue values = 1;
}
// KeyValueList is a list of KeyValue messages. We need KeyValueList as a message
// since `oneof` in AnyValue does not allow repeated fields. Everywhere else where we need
// a list of KeyValue messages (e.g. in Span) we use `repeated KeyValue` directly to
// avoid unnecessary extra wrapping (which slows down the protocol). The 2 approaches
// are semantically equivalent.
message KeyValueList {
// A collection of key/value pairs of key-value pairs. The list may be empty (may
// contain 0 elements).
// The keys MUST be unique (it is not allowed to have more than one
// value with the same key).
repeated KeyValue values = 1;
}
// KeyValue is a key-value pair that is used to store Span attributes, Link
// attributes, etc.
message KeyValue {
string key = 1;
AnyValue value = 2;
}
// InstrumentationScope is a message representing the instrumentation scope information
// such as the fully qualified name and version.
message InstrumentationScope {
// An empty instrumentation scope name means the name is unknown.
string name = 1;
string version = 2;
repeated KeyValue attributes = 3;
uint32 dropped_attributes_count = 4;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/logs/v1/logs.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2020, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.logs.v1;
import "opentelemetry/proto/common/v1/common.proto";
import "opentelemetry/proto/resource/v1/resource.proto";
option csharp_namespace = "OpenTelemetry.Proto.Logs.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.logs.v1";
option java_outer_classname = "LogsProto";
option go_package = "go.opentelemetry.io/proto/otlp/logs/v1";
// LogsData represents the logs data that can be stored in a persistent storage,
// OR can be embedded by other protocols that transfer OTLP logs data but do not
// implement the OTLP protocol.
//
// The main difference between this message and collector protocol is that
// in this message there will not be any "control" or "metadata" specific to
// OTLP protocol.
//
// When new fields are added into this message, the OTLP request MUST be updated
// as well.
message LogsData {
// An array of ResourceLogs.
// For data coming from a single resource this array will typically contain
// one element. Intermediary nodes that receive data from multiple origins
// typically batch the data before forwarding further and in that case this
// array will contain multiple elements.
repeated ResourceLogs resource_logs = 1;
}
// A collection of ScopeLogs from a Resource.
message ResourceLogs {
reserved 1000;
// The resource for the logs in this message.
// If this field is not set then resource info is unknown.
opentelemetry.proto.resource.v1.Resource resource = 1;
// A list of ScopeLogs that originate from a resource.
repeated ScopeLogs scope_logs = 2;
// This schema_url applies to the data in the "resource" field. It does not apply
// to the data in the "scope_logs" field which have their own schema_url field.
string schema_url = 3;
}
// A collection of Logs produced by a Scope.
message ScopeLogs {
// The instrumentation scope information for the logs in this message.
// Semantically when InstrumentationScope isn't set, it is equivalent with
// an empty instrumentation scope name (unknown).
opentelemetry.proto.common.v1.InstrumentationScope scope = 1;
// A list of log records.
repeated LogRecord log_records = 2;
// This schema_url applies to all logs in the "logs" field.
string schema_url = 3;
}
// Possible values for LogRecord.SeverityNumber.
enum SeverityNumber {
// UNSPECIFIED is the default SeverityNumber, it MUST NOT be used.
SEVERITY_NUMBER_UNSPECIFIED = 0;
SEVERITY_NUMBER_TRACE = 1;
SEVERITY_NUMBER_TRACE2 = 2;
SEVERITY_NUMBER_TRACE3 = 3;
SEVERITY_NUMBER_TRACE4 = 4;
SEVERITY_NUMBER_DEBUG = 5;
SEVERITY_NUMBER_DEBUG2 = 6;
SEVERITY_NUMBER_DEBUG3 = 7;
SEVERITY_NUMBER_DEBUG4 = 8;
SEVERITY_NUMBER_INFO = 9;
SEVERITY_NUMBER_INFO2 = 10;
SEVERITY_NUMBER_INFO3 = 11;
SEVERITY_NUMBER_INFO4 = 12;
SEVERITY_NUMBER_WARN = 13;
SEVERITY_NUMBER_WARN2 = 14;
SEVERITY_NUMBER_WARN3 = 15;
SEVERITY_NUMBER_WARN4 = 16;
SEVERITY_NUMBER_ERROR = 17;
SEVERITY_NUMBER_ERROR2 = 18;
SEVERITY_NUMBER_ERROR3 = 19;
SEVERITY_NUMBER_ERROR4 = 20;
SEVERITY_NUMBER_FATAL = 21;
SEVERITY_NUMBER_FATAL2 = 22;
SEVERITY_NUMBER_FATAL3 = 23;
SEVERITY_NUMBER_FATAL4 = 24;
}
// Masks for LogRecord.flags field.
enum LogRecordFlags {
LOG_RECORD_FLAG_UNSPECIFIED = 0;
LOG_RECORD_FLAG_TRACE_FLAGS_MASK = 0x000000FF;
}
// A log record according to OpenTelemetry Log Data Model:
// https://github.com/open-telemetry/oteps/blob/main/text/logs/0097-log-data-model.md
message LogRecord {
reserved 4;
// time_unix_nano is the time when the event occurred.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
// Value of 0 indicates unknown or missing timestamp.
fixed64 time_unix_nano = 1;
// Time when the event was observed by the collection system.
// For events that originate in OpenTelemetry (e.g. using OpenTelemetry Logging SDK)
// this timestamp is typically set at the generation time and is equal to Timestamp.
// For events originating externally and collected by OpenTelemetry (e.g. using
// Collector) this is the time when OpenTelemetry's code observed the event measured
// by the clock of the OpenTelemetry code. This field MUST be set once the event is
// observed by OpenTelemetry.
//
// For converting OpenTelemetry log data to formats that support only one timestamp or
// when receiving OpenTelemetry log data by recipients that support only one timestamp
// internally the following logic is recommended:
// - Use time_unix_nano if it is present, otherwise use observed_time_unix_nano.
//
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
// Value of 0 indicates unknown or missing timestamp.
fixed64 observed_time_unix_nano = 11;
// Numerical value of the severity, normalized to values described in Log Data Model.
// [Optional].
SeverityNumber severity_number = 2;
// The severity text (also known as log level). The original string representation as
// it is known at the source. [Optional].
string severity_text = 3;
// A value containing the body of the log record. Can be for example a human-readable
// string message (including multi-line) describing the event in a free form or it can
// be a structured data composed of arrays and maps of other values. [Optional].
opentelemetry.proto.common.v1.AnyValue body = 5;
// Additional attributes that describe the specific event occurrence. [Optional].
// Attribute keys MUST be unique (it is not allowed to have more than one
// attribute with the same key).
repeated opentelemetry.proto.common.v1.KeyValue attributes = 6;
uint32 dropped_attributes_count = 7;
// Flags, a bit field. 8 least significant bits are the trace flags as
// defined in W3C Trace Context specification. 24 most significant bits are reserved
// and must be set to 0. Readers must not assume that 24 most significant bits
// will be zero and must correctly mask the bits when reading 8-bit trace flag (use
// flags & TRACE_FLAGS_MASK). [Optional].
fixed32 flags = 8;
// A unique identifier for a trace. All logs from the same trace share
// the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes
// is considered invalid. Can be set for logs that are part of request processing
// and have an assigned trace id. [Optional].
bytes trace_id = 9;
// A unique identifier for a span within a trace, assigned when the span
// is created. The ID is an 8-byte array. An ID with all zeroes is considered
// invalid. Can be set for logs that are part of a particular processing span.
// If span_id is present trace_id SHOULD be also present. [Optional].
bytes span_id = 10;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/resource/v1/resource.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2019, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.resource.v1;
import "opentelemetry/proto/common/v1/common.proto";
option csharp_namespace = "OpenTelemetry.Proto.Resource.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.resource.v1";
option java_outer_classname = "ResourceProto";
option go_package = "go.opentelemetry.io/proto/otlp/resource/v1";
// Resource information.
message Resource {
// Set of attributes that describe the resource.
// Attribute keys MUST be unique (it is not allowed to have more than one
// attribute with the same key).
repeated opentelemetry.proto.common.v1.KeyValue attributes = 1;
// dropped_attributes_count is the number of dropped attributes. If the value is 0, then
// no attributes were dropped.
uint32 dropped_attributes_count = 2;
}
test/e2e-v2/java-test-service/opentelemetry-proto/src/main/proto/opentelemetry/proto/trace/v1/trace.proto 0 → 100644
浏览文件 @ 8de8c7c1
// Copyright 2019, OpenTelemetry Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
syntax = "proto3";
package opentelemetry.proto.trace.v1;
import "opentelemetry/proto/common/v1/common.proto";
import "opentelemetry/proto/resource/v1/resource.proto";
option csharp_namespace = "OpenTelemetry.Proto.Trace.V1";
option java_multiple_files = true;
option java_package = "io.opentelemetry.proto.trace.v1";
option java_outer_classname = "TraceProto";
option go_package = "go.opentelemetry.io/proto/otlp/trace/v1";
// TracesData represents the traces data that can be stored in a persistent storage,
// OR can be embedded by other protocols that transfer OTLP traces data but do
// not implement the OTLP protocol.
//
// The main difference between this message and collector protocol is that
// in this message there will not be any "control" or "metadata" specific to
// OTLP protocol.
//
// When new fields are added into this message, the OTLP request MUST be updated
// as well.
message TracesData {
// An array of ResourceSpans.
// For data coming from a single resource this array will typically contain
// one element. Intermediary nodes that receive data from multiple origins
// typically batch the data before forwarding further and in that case this
// array will contain multiple elements.
repeated ResourceSpans resource_spans = 1;
}
// A collection of ScopeSpans from a Resource.
message ResourceSpans {
reserved 1000;
// The resource for the spans in this message.
// If this field is not set then no resource info is known.
opentelemetry.proto.resource.v1.Resource resource = 1;
// A list of ScopeSpans that originate from a resource.
repeated ScopeSpans scope_spans = 2;
// This schema_url applies to the data in the "resource" field. It does not apply
// to the data in the "scope_spans" field which have their own schema_url field.
string schema_url = 3;
}
// A collection of Spans produced by an InstrumentationScope.
message ScopeSpans {
// The instrumentation scope information for the spans in this message.
// Semantically when InstrumentationScope isn't set, it is equivalent with
// an empty instrumentation scope name (unknown).
opentelemetry.proto.common.v1.InstrumentationScope scope = 1;
// A list of Spans that originate from an instrumentation scope.
repeated Span spans = 2;
// This schema_url applies to all spans and span events in the "spans" field.
string schema_url = 3;
}
// A Span represents a single operation performed by a single component of the system.
//
// The next available field id is 17.
message Span {
// A unique identifier for a trace. All spans from the same trace share
// the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes
// is considered invalid.
//
// This field is semantically required. Receiver should generate new
// random trace_id if empty or invalid trace_id was received.
//
// This field is required.
bytes trace_id = 1;
// A unique identifier for a span within a trace, assigned when the span
// is created. The ID is an 8-byte array. An ID with all zeroes is considered
// invalid.
//
// This field is semantically required. Receiver should generate new
// random span_id if empty or invalid span_id was received.
//
// This field is required.
bytes span_id = 2;
// trace_state conveys information about request position in multiple distributed tracing graphs.
// It is a trace_state in w3c-trace-context format: https://www.w3.org/TR/trace-context/#tracestate-header
// See also https://github.com/w3c/distributed-tracing for more details about this field.
string trace_state = 3;
// The `span_id` of this span's parent span. If this is a root span, then this
// field must be empty. The ID is an 8-byte array.
bytes parent_span_id = 4;
// A description of the span's operation.
//
// For example, the name can be a qualified method name or a file name
// and a line number where the operation is called. A best practice is to use
// the same display name at the same call point in an application.
// This makes it easier to correlate spans in different traces.
//
// This field is semantically required to be set to non-empty string.
// Empty value is equivalent to an unknown span name.
//
// This field is required.
string name = 5;
// SpanKind is the type of span. Can be used to specify additional relationships between spans
// in addition to a parent/child relationship.
enum SpanKind {
// Unspecified. Do NOT use as default.
// Implementations MAY assume SpanKind to be INTERNAL when receiving UNSPECIFIED.
SPAN_KIND_UNSPECIFIED = 0;
// Indicates that the span represents an internal operation within an application,
// as opposed to an operation happening at the boundaries. Default value.
SPAN_KIND_INTERNAL = 1;
// Indicates that the span covers server-side handling of an RPC or other
// remote network request.
SPAN_KIND_SERVER = 2;
// Indicates that the span describes a request to some remote service.
SPAN_KIND_CLIENT = 3;
// Indicates that the span describes a producer sending a message to a broker.
// Unlike CLIENT and SERVER, there is often no direct critical path latency relationship
// between producer and consumer spans. A PRODUCER span ends when the message was accepted
// by the broker while the logical processing of the message might span a much longer time.
SPAN_KIND_PRODUCER = 4;
// Indicates that the span describes consumer receiving a message from a broker.
// Like the PRODUCER kind, there is often no direct critical path latency relationship
// between producer and consumer spans.
SPAN_KIND_CONSUMER = 5;
}
// Distinguishes between spans generated in a particular context. For example,
// two spans with the same name may be distinguished using `CLIENT` (caller)
// and `SERVER` (callee) to identify queueing latency associated with the span.
SpanKind kind = 6;
// start_time_unix_nano is the start time of the span. On the client side, this is the time
// kept by the local machine where the span execution starts. On the server side, this
// is the time when the server's application handler starts running.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// This field is semantically required and it is expected that end_time >= start_time.
fixed64 start_time_unix_nano = 7;
// end_time_unix_nano is the end time of the span. On the client side, this is the time
// kept by the local machine where the span execution ends. On the server side, this
// is the time when the server application handler stops running.
// Value is UNIX Epoch time in nanoseconds since 00:00:00 UTC on 1 January 1970.
//
// This field is semantically required and it is expected that end_time >= start_time.
fixed64 end_time_unix_nano = 8;
// attributes is a collection of key/value pairs. Note, global attributes
// like server name can be set using the resource API. Examples of attributes:
//
// "/http/user_agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"
// "/http/server_latency": 300
// "abc.com/myattribute": true
// "abc.com/score": 10.239
//
// The OpenTelemetry API specification further restricts the allowed value types:
// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/common/README.md#attribute
// Attribute keys MUST be unique (it is not allowed to have more than one
// attribute with the same key).
repeated opentelemetry.proto.common.v1.KeyValue attributes = 9;
// dropped_attributes_count is the number of attributes that were discarded. Attributes
// can be discarded because their keys are too long or because there are too many
// attributes. If this value is 0, then no attributes were dropped.
uint32 dropped_attributes_count = 10;
// Event is a time-stamped annotation of the span, consisting of user-supplied
// text description and key-value pairs.
message Event {
// time_unix_nano is the time the event occurred.
fixed64 time_unix_nano = 1;
// name of the event.
// This field is semantically required to be set to non-empty string.
string name = 2;
// attributes is a collection of attribute key/value pairs on the event.
// Attribute keys MUST be unique (it is not allowed to have more than one
// attribute with the same key).
repeated opentelemetry.proto.common.v1.KeyValue attributes = 3;
// dropped_attributes_count is the number of dropped attributes. If the value is 0,
// then no attributes were dropped.
uint32 dropped_attributes_count = 4;
}
// events is a collection of Event items.
repeated Event events = 11;
// dropped_events_count is the number of dropped events. If the value is 0, then no
// events were dropped.
uint32 dropped_events_count = 12;
// A pointer from the current span to another span in the same trace or in a
// different trace. For example, this can be used in batching operations,
// where a single batch handler processes multiple requests from different
// traces or when the handler receives a request from a different project.
message Link {
// A unique identifier of a trace that this linked span is part of. The ID is a
// 16-byte array.
bytes trace_id = 1;
// A unique identifier for the linked span. The ID is an 8-byte array.
bytes span_id = 2;
// The trace_state associated with the link.
string trace_state = 3;
// attributes is a collection of attribute key/value pairs on the link.
// Attribute keys MUST be unique (it is not allowed to have more than one
// attribute with the same key).
repeated opentelemetry.proto.common.v1.KeyValue attributes = 4;
// dropped_attributes_count is the number of dropped attributes. If the value is 0,
// then no attributes were dropped.
uint32 dropped_attributes_count = 5;
}
// links is a collection of Links, which are references from this span to a span
// in the same or different trace.
repeated Link links = 13;
// dropped_links_count is the number of dropped links after the maximum size was
// enforced. If this value is 0, then no links were dropped.
uint32 dropped_links_count = 14;
// An optional final status for this span. Semantically when Status isn't set, it means
// span's status code is unset, i.e. assume STATUS_CODE_UNSET (code = 0).
Status status = 15;
}
// The Status type defines a logical error model that is suitable for different
// programming environments, including REST APIs and RPC APIs.
message Status {
reserved 1;
// A developer-facing human readable error message.
string message = 2;
// For the semantics of status codes see
// https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/trace/api.md#set-status
enum StatusCode {
// The default status.
STATUS_CODE_UNSET = 0;
// The Span has been validated by an Application developer or Operator to
// have completed successfully.
STATUS_CODE_OK = 1;
// The Span contains an error.
STATUS_CODE_ERROR = 2;
};
// The status code.
StatusCode code = 3;
}
test/e2e-v2/java-test-service/pom.xml
浏览文件 @ 8de8c7c1
......@@ -32,6 +32,7 @@
<module>e2e-service-consumer</module>
<module>e2e-mock-sender</module>
<module>e2e-protocol</module>
<module>opentelemetry-proto</module>
</modules>
<properties>
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或