.github/workflows/skywalking.yaml

@@ -670,6 +670,8 @@ jobs:
             config: test/e2e-v2/cases/aws/eks/e2e.yaml
           - name: Windows
             config: test/e2e-v2/cases/win/e2e.yaml
+          - name: AWS Cloud S3
+            config: test/e2e-v2/cases/aws/s3/e2e.yaml
     steps:
       - uses: actions/checkout@v3
         with:

docs/en/changes/changes.md

@@ -87,7 +87,9 @@
 * [Optional] Optimize single trace query performance by customizing routing in ElasticSearch. SkyWalking trace segments and Zipkin spans are using trace ID for routing. This is OFF by default, controlled by `storage/elasticsearch/enableCustomRouting`.
 * Enhance OAP HTTP server to support HTTPS
 * Remove handler scan in otel receiver, manual initialization instead
-* Add aws-firehose-receiver to support collecting AWS CloudWatch metric(OpenTelemetry format)
+* Add aws-firehose-receiver to support collecting AWS CloudWatch metric(OpenTelemetry format). Notice, no HTTPS/TLS setup
+  support. By following AWS Firehose request, it uses [proxy request](https://en.wikipedia.org/wiki/Proxy_server#Web_proxy_servers)
+  (`https://...` instead of `/aws/firehose/metrics`), there must be a proxy(Nginx, Envoy, etc.).
 * Avoid Antlr dependencies' versions might be different in compile time and runtime.
 * Now `PrometheusMetricConverter#escapedName` also support converting `/` to `_`.
 * Add missing TCP throughput metrics.
@@ -95,6 +97,7 @@
 * Add Python HTTPX module component ID(7019).
 * Migrate tests from junit 4 to junit 5.
 * Refactor http-based alarm plugins and extract common logic to `HttpAlarmCallback`.
+* Support Amazon Simple Storage Service (Amazon S3) metrics monitoring
 
 #### UI
 

docs/en/setup/backend/aws-firehose-receiver.md

@@ -14,6 +14,12 @@ Usually, the [AWS CloudWatch metrics](https://docs.aws.amazon.com/AmazonCloudWat
 CloudWatch metrics with S3 -->  CloudWatch Metric Stream (OpenTelemetry formart) --> Kinesis Data Firehose Delivery Stream --> AWS Firehose receiver(OAP) --> OpenTelemetry receiver(OAP)
 ```
 
+## Supported metrics
+
+| Description                             | Configuration File                  | Data Source                                                                                                                                       |
+|-----------------------------------------|-------------------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------|
+| Metrics of AWS Cloud S3                 | otel-rules/aws-s3/s3-service.yaml   | AWS CloudWatcher Metrics Stream -> AWS Firehose delivery stream -> SkyWalking OAP Server with [AWS Firehose receiver](./aws-firehose-receiver.md) |
+
 ## Notice
 
 1. Only OpenTelemetry format is supported (refer to [Metric streams output formats](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-metric-streams-formats.html))

docs/en/setup/backend/backend-aws-s3-monitoring.md

+# AWS Cloud S3 monitoring
+Amazon Simple Storage Service (Amazon S3) is an object storage service. SkyWalking leverages [AWS Kinesis Data Firehose receiver](./aws-firehose-receiver.md) to transfer the CloudWatch metrics of s3 to
+[OpenTelemetry receiver](opentelemetry-receiver.md) and into the [Meter System](./../../concepts-and-designs/meter.md).
+
+### Data flow
+1. AWS CloudWatch collect metrics for S3, refer to [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html)
+2. [CloudWatch metric streams](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html) stream CloudWatch metrics of S3 to AWS Kinesis Data Firehose
+3. AWS Kinesis Data Firehose delivery metrics to [AWS Kinesis Data Firehose receiver](./aws-firehose-receiver.md) through the HTTP endpoint
+
+### Set up
+1. Create CloudWatch metrics configuration for S3, refer to [S3 metrics configuration](https://docs.aws.amazon.com/AmazonS3/latest/userguide/metrics-configurations.html)
+2. Create an Amazon Kinesis Data Firehose Delivery Stream, and set [AWS Kinesis Data Firehose receiver](./aws-firehose-receiver.md)'s address as HTTP(s) Destination, refer to [Create Delivery Stream](https://docs.aws.amazon.com/firehose/latest/dev/basic-create.html)
+3. Create CloudWatch metric stream, and select the Firehose Delivery Stream which has been created above, set `Select namespaces` to `AWS/S3`, `Select output format` to `OpenTelemetry 0.7`. refer to [CloudWatch Metric Streams](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Metric-Streams.html)
+
+### S3 Monitoring
+
+SkyWalking observes CloudWatch metrics of the S3 bucket, which is cataloged as a `LAYER: AWS_S3` `Service` in the OAP.
+
+#### Supported Metrics
+
+| Monitoring Panel           | Unit  | Metric Name                 | Catalog        | Description                                                                                                                                | Data Source                                                                                                       |
+|----------------------------|-------|-----------------------------|----------------|--------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------|
+| 4xx Errors                 | count | aws_s3_4xx                  | Service        | The number of HTTP 4xx client error status code requests made to the S3 bucket                                                             | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| 5xx Errors                 | count | aws_s3_5xx                  | Service        | The number of HTTP 5xx client error status code requests made to the S3 bucket                                                             | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Downloaded                 | bytes | aws_s3_downloaded_bytes     | Service        | The number of bytes downloaded for requests made to an Amazon S3 bucket                                                                    | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Uploaded                   | bytes | aws_s3_uploaded_bytes       | Service        | The number of bytes uploaded for requests made to an Amazon S3 bucket                                                                      | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Request Average Latency    | bytes | aws_s3_request_latency      | Service        | The average of elapsed per-request time from the first byte received to the last byte sent to an Amazon S3 bucket                          | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| First Byte Average Latency | bytes | aws_s3_request_latency      | Service        | The average of per-request time from the complete request being received by an Amazon S3 bucket to when the response starts to be returned | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| All Requests               | bytes | aws_s3_delete_requests      | Service        | The number of HTTP All requests made for objects in an Amazon S3 bucket                                                                    | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Get Requests               | bytes | aws_s3_delete_requests      | Service        | The number of HTTP Get requests made for objects in an Amazon S3 bucket                                                                    | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Put Requests               | bytes | aws_s3_delete_requests      | Service        | The number of HTTP PUT requests made for objects in an Amazon S3 bucket                                                                    | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+| Delete Requests            | bytes | aws_s3_delete_requests      | Service        | The number of HTTP Delete requests made for objects in an Amazon S3 bucket                                                                 | [S3 monitoring with CloudWatch](https://docs.aws.amazon.com/AmazonS3/latest/userguide/cloudwatch-monitoring.html) |
+
+### Customizations
+You can customize your own metrics/expression/dashboard panel.
+The metrics definition and expression rules are found in `/config/otel-rules/aws-s3/`.
+The AWS Cloud EKS dashboard panel configurations are found in `/config/ui-initialized-templates/aws_s3`.

docs/en/setup/backend/configuration-vocabulary.md

@@ -312,9 +312,6 @@ The Configuration Vocabulary lists all available configurations provided by `app
 | -                       | -             | idleTimeOut                                                                                                                                                              | Idle timeout of a connection for keep-alive.                                                                                                                                                                                                                                                                                                                                                                                                                    | SW_RECEIVER_AWS_FIREHOSE_HTTP_IDLE_TIME_OUT            | 30000                                                                                               |
 | -                       | -             | acceptQueueSize                                                                                                                                                          | Maximum allowed number of open connections                                                                                                                                                                                                                                                                                                                                                                                                                      | SW_RECEIVER_AWS_FIREHOSE_HTTP_ACCEPT_QUEUE_SIZE        | 0                                                                                          |
 | -                       | -             | maxRequestHeaderSize                                                                                                                                                     | Maximum length of all headers in an HTTP/1 response                                                                                                                                                                                                                                                                                                                                                                                                             | SW_RECEIVER_AWS_FIREHOSE_HTTP_MAX_REQUEST_HEADER_SIZE  | 8192                                                                                             |
-| -                       | -             | enableTLS                                                                                                                                                                | Indicate if enable HTTPS for the server                                                                                                                                                                                                                                                                                                                                                                                                                         | SW_RECEIVER_AWS_FIREHOSE_HTTP_ENABLE_TLS               | false                                                                                             |
-| -                       | -             | tlsKeyPath                                                                                                                                                               | TLS key path                                                                                                                                                                                                                                                                                                                                                                                                                                                    | SW_RECEIVER_AWS_FIREHOSE_HTTP_TLS_KEY_PATH             |                                                                                              |
-| -                       | -             | tlsCertChainPath                                                                                                                                                         | TLS certificate chain path                                                                                                                                                                                                                                                                                                                                                                                                                                      | SW_RECEIVER_AWS_FIREHOSE_HTTP_TLS_CERT_CHAIN_PATH      |                                                                                              |
 
 ## Note
 

docs/menu.yml

@@ -189,6 +189,8 @@ catalog:
             catalog:
               - name: "EKS Monitoring"
                 path: "/en/setup/backend/backend-aws-eks-monitoring"
+              - name: "S3 Monitoring"
+                path: "/en/setup/backend/backend-aws-s3-monitoring"
           - name: "Browser Monitoring"
             path: "/en/setup/service-agent/browser-agent"
           - name: "Gateway Monitoring"

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/analysis/Layer.java

@@ -146,8 +146,12 @@ public enum Layer {
     /**
      * Windows Machine
      */
-    OS_WINDOWS(23, true);
+    OS_WINDOWS(23, true),
 
+    /**
+     * Amazon Simple Storage Service (Amazon S3) is an object storage service provided by AWS Cloud
+     */
+    AWS_S3(24, true);
 
     private final int value;
     /**

oap-server/server-core/src/main/java/org/apache/skywalking/oap/server/core/management/ui/template/UITemplateInitializer.java

@@ -62,6 +62,7 @@ public class UITemplateInitializer {
         Layer.VIRTUAL_MQ.name(),
         Layer.AWS_EKS.name(),
         Layer.OS_WINDOWS.name(),
+        Layer.AWS_S3.name(),
         "custom"
     };
     private final UITemplateManagementService uiTemplateManagementService;

oap-server/server-receiver-plugin/aws-firehose-receiver/src/main/java/org/apache/skywalking/oap/server/receiver/aws/firehose/AWSFirehoseReceiverModuleConfig.java

@@ -29,7 +29,4 @@ public class AWSFirehoseReceiverModuleConfig extends ModuleConfig {
     private long idleTimeOut = 30000;
     private int acceptQueueSize = 0;
     private int maxRequestHeaderSize = 8192;
-    private boolean enableTLS = false;
-    private String tlsKeyPath;
-    private String tlsCertChainPath;
 }

oap-server/server-receiver-plugin/aws-firehose-receiver/src/main/java/org/apache/skywalking/oap/server/receiver/aws/firehose/AWSFirehoseReceiverModuleProvider.java

@@ -70,9 +70,6 @@ public class AWSFirehoseReceiverModuleProvider extends ModuleProvider {
                                                                   .acceptQueueSize(moduleConfig.getAcceptQueueSize())
                                                                   .maxRequestHeaderSize(
                                                                       moduleConfig.getMaxRequestHeaderSize())
-                                                                  .enableTLS(moduleConfig.isEnableTLS())
-                                                                  .tlsKeyPath(moduleConfig.getTlsKeyPath())
-                                                                  .tlsCertChainPath(moduleConfig.getTlsCertChainPath())
                                                                   .build();
         httpServer = new HTTPServer(httpServerConfig);
         httpServer.initialize();

oap-server/server-starter/src/main/resources/application.yml

@@ -356,7 +356,7 @@ receiver-otel:
   selector: ${SW_OTEL_RECEIVER:default}
   default:
     enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"oc,otlp"}
-    enabledOtelRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_RULES:"apisix,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows"}
+    enabledOtelRules: ${SW_OTEL_RECEIVER_ENABLED_OTEL_RULES:"apisix,k8s/*,istio-controlplane,vm,mysql/*,postgresql/*,oap,aws-eks/*,windows,aws-s3/*"}
 
 receiver-zipkin:
   selector: ${SW_RECEIVER_ZIPKIN:-}
@@ -555,6 +555,3 @@ aws-firehose:
     idleTimeOut: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_IDLE_TIME_OUT:30000}
     acceptQueueSize: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_ACCEPT_QUEUE_SIZE:0}
     maxRequestHeaderSize: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_MAX_REQUEST_HEADER_SIZE:8192}
-    enableTLS: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_ENABLE_TLS:false}
-    tlsKeyPath: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_TLS_KEY_PATH:}
-    tlsCertChainPath: ${SW_RECEIVER_AWS_FIREHOSE_HTTP_TLS_CERT_CHAIN_PATH:}

oap-server/server-starter/src/main/resources/otel-rules/aws-s3/s3-service.yaml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This will parse a textual representation of a duration. The formats
+# accepted are based on the ISO-8601 duration format {@code PnDTnHnMn.nS}
+# with days considered to be exactly 24 hours.
+# <p>
+# Examples:
+# <pre>
+#    "PT20.345S" -- parses as "20.345 seconds"
+#    "PT15M"     -- parses as "15 minutes" (where a minute is 60 seconds)
+#    "PT10H"     -- parses as "10 hours" (where an hour is 3600 seconds)
+#    "P2D"       -- parses as "2 days" (where a day is 24 hours or 86400 seconds)
+#    "P2DT3H4M"  -- parses as "2 days, 3 hours and 4 minutes"
+#    "P-6H3M"    -- parses as "-6 hours and +3 minutes"
+#    "-P6H3M"    -- parses as "-6 hours and -3 minutes"
+#    "-P-6H+3M"  -- parses as "+6 hours and -3 minutes"
+# </pre>
+
+filter: "{ tags -> {tags.cloud_provider == 'aws' && tags.Namespace == 'AWS/S3' } }"
+expSuffix: tag({tags -> tags.bucket = 'aws-s3::' + tags.BucketName}).service(['bucket'], Layer.AWS_S3)
+metricPrefix: aws_s3
+metricsRules:
+  - name: 4xx
+    exp: amazonaws_com_AWS_S3_4xxErrors_sum.downsampling(SUM)
+  - name: 5xx
+    exp: amazonaws_com_AWS_S3_5xxErrors_sum.downsampling(SUM)
+  - name: downloaded_bytes
+    exp: amazonaws_com_AWS_S3_BytesDownloaded_sum.downsampling(SUM)
+  - name: uploaded_bytes
+    exp: amazonaws_com_AWS_S3_BytesUploaded_sum.downsampling(SUM)
+  - name: request_latency
+    exp: amazonaws_com_AWS_S3_TotalRequestLatency_sum.div(amazonaws_com_AWS_S3_TotalRequestLatency_count)
+  - name: first_latency_bytes
+    exp: amazonaws_com_AWS_S3_FirstByteLatency_sum.div(amazonaws_com_AWS_S3_FirstByteLatency_count)
+  - name: all_requests
+    exp: amazonaws_com_AWS_S3_AllRequests_sum.downsampling(SUM)
+  - name: put_requests
+    exp: amazonaws_com_AWS_S3_PutRequests_sum.downsampling(SUM)
+  - name: get_requests
+    exp: amazonaws_com_AWS_S3_GetRequests_sum.downsampling(SUM)
+  - name: delete_requests
+    exp: amazonaws_com_AWS_S3_DeleteRequests_sum.downsampling(SUM)

oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_eks/aws_eks-root.json

@@ -50,7 +50,7 @@
           "graph": {
             "fontColor":"blue",
             "backgroundColor":"white",
-            "content":"Provide AWS Could monitoring through AWS Container Insights Receiver",
+            "content":"Provide AWS Cloud EKS monitoring through AWS Container Insights Receiver",
             "fontSize":14,
             "textAlign":"left",
             "url":"https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-aws-eks-monitoring/"

oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_s3/aws_s3-root.json

+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+[
+  {
+    "id": "AWS_S3-Root",
+    "configuration": {
+      "children": [
+        {
+          "x": 0,
+          "y": 2,
+          "w": 24,
+          "h": 51,
+          "i": "0",
+          "type": "Widget",
+          "widget": {
+            "title": ""
+          },
+          "graph": {
+            "type": "ServiceList",
+            "dashboardName": "AWS_S3-service",
+            "fontSize": 12,
+            "showXAxis": false,
+            "showYAxis": false,
+            "showGroup": true
+          },
+          "moved": false
+        },
+        {
+          "x": 0,
+          "y": 0,
+          "w": 24,
+          "h": 2,
+          "i": "100",
+          "type": "Text",
+          "graph": {
+            "fontColor":"blue",
+            "backgroundColor":"white",
+            "content":"Provide AWS Cloud S3 monitoring through AWS FireHose Receiver",
+            "fontSize":14,
+            "textAlign":"left",
+            "url":"https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-aws-s3-monitoring/"
+          },
+          "moved": false
+        }
+      ],
+      "id": "AWS_S3-Root",
+      "layer": "AWS_S3",
+      "entity": "All",
+      "name": "AWS_S3-Root",
+      "isRoot": true
+    }
+  }
+]

oap-server/server-starter/src/main/resources/ui-initialized-templates/aws_s3/aws_s3-service.json

+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *      http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+[
+  {
+    "id": "AWS_S3-service",
+    "configuration": {
+      "children": [
+        {
+          "x": 0,
+          "y": 0,
+          "w": 6,
+          "h": 5,
+          "i": "1",
+          "type": "Widget",
+          "widget": {
+            "title": "4xx Count"
+          },
+          "graph": {
+            "type": "Card",
+            "fontSize": 14,
+            "textAlign": "center",
+            "showUnit": true
+          },
+          "metrics": [
+            "aws_s3_4xx"
+          ],
+          "metricTypes": [
+            "readMetricsValue"
+          ]
+        },
+        {
+          "x": 6,
+          "y": 0,
+          "w": 6,
+          "h": 5,
+          "i": "2",
+          "type": "Widget",
+          "widget": {
+            "title": "5xx Count"
+          },
+          "graph": {
+            "type": "Card",
+            "fontSize": 14,
+            "textAlign": "center",
+            "showUnit": true
+          },
+          "metrics": [
+            "aws_s3_4xx"
+          ],
+          "metricTypes": [
+            "readMetricsValue"
+          ]
+        },
+        {
+          "x": 12,
+          "y": 0,
+          "w": 6,
+          "h": 5,
+          "i": "3",
+          "type": "Widget",
+          "widget": {
+            "title": "All Request Count"
+          },
+          "graph": {
+            "type": "Card",
+            "fontSize": 14,
+            "textAlign": "center",
+            "showUnit": true
+          },
+          "metrics": [
+            "aws_s3_all_requests"
+          ],
+          "metricTypes": [
+            "readMetricsValue"
+          ]
+        },
+        {
+          "x": 18,
+          "y": 0,
+          "w": 6,
+          "h": 5,
+          "i": "4",
+          "type": "Widget",
+          "widget": {
+            "title": "Request Average Latency (ms)"
+          },
+          "graph": {
+            "type": "Card",
+            "fontSize": 14,
+            "textAlign": "center",
+            "showUnit": true
+          },
+          "metrics": [
+            "aws_s3_request_latency"
+          ],
+          "metricTypes": [
+            "readMetricsValue"
+          ]
+        },
+        {
+          "x": 0,
+          "y": 5,
+          "w": 6,
+          "h": 13,
+          "i": "5",
+          "type": "Widget",
+          "widget": {
+            "title": "4xx Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_4xx"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 6,
+          "y": 5,
+          "w": 6,
+          "h": 13,
+          "i": "6",
+          "type": "Widget",
+          "widget": {
+            "title": "5xx Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_4xx"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 12,
+          "y": 5,
+          "w": 6,
+          "h": 13,
+          "i": "7",
+          "type": "Widget",
+          "widget": {
+            "title": "Request Average Latency (ms)"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_request_latency"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 18,
+          "y": 5,
+          "w": 6,
+          "h": 13,
+          "i": "8",
+          "type": "Widget",
+          "widget": {
+            "title": "First Byte Average Latency (ms)"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_first_latency_bytes"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 0,
+          "y": 18,
+          "w": 6,
+          "h": 13,
+          "i": "9",
+          "type": "Widget",
+          "widget": {
+            "title": "All Request Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_all_requests"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 6,
+          "y": 18,
+          "w": 6,
+          "h": 13,
+          "i": "10",
+          "type": "Widget",
+          "widget": {
+            "title": "Get Request Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_get_requests"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 12,
+          "y": 18,
+          "w": 6,
+          "h": 13,
+          "i": "11",
+          "type": "Widget",
+          "widget": {
+            "title": "Put Request Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_put_requests"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 18,
+          "y": 18,
+          "w": 6,
+          "h": 13,
+          "i": "12",
+          "type": "Widget",
+          "widget": {
+            "title": "Delete Request Count"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true
+          },
+          "metrics": [
+            "aws_s3_delete_requests"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ]
+        },
+        {
+          "x": 0,
+          "y": 41,
+          "w": 12,
+          "h": 13,
+          "i": "13",
+          "type": "Widget",
+          "widget": {
+            "title": "Downloaded (KB)"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true,
+            "legend": {
+              "asTable": true,
+              "toTheRight": true,
+              "min": true,
+              "max": true,
+              "mean": true,
+              "total": true
+            }
+          },
+          "metrics": [
+            "aws_s3_downloaded_bytes"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ],
+          "metricConfig": [
+            {
+              "calculation": "byteToKB"
+            }
+          ]
+        },
+        {
+          "x": 12,
+          "y": 41,
+          "w": 12,
+          "h": 13,
+          "i": "14",
+          "type": "Widget",
+          "widget": {
+            "title": "Uploaded (KB)"
+          },
+          "graph": {
+            "type": "Line",
+            "step": false,
+            "smooth": false,
+            "showSymbol": false,
+            "showXAxis": true,
+            "showYAxis": true,
+            "legend": {
+              "asTable": true,
+              "toTheRight": true,
+              "min": true,
+              "max": true,
+              "mean": true,
+              "total": true
+            }
+          },
+          "metrics": [
+            "aws_s3_uploaded_bytes"
+          ],
+          "metricTypes": [
+            "readMetricsValues"
+          ],
+          "metricConfig": [
+            {
+              "calculation": "byteToKB"
+            }
+          ]
+        }
+      ],
+      "layer": "AWS_S3",
+      "entity": "Service",
+      "name": "AWS_S3-service",
+      "id": "AWS_S3-service",
+      "isRoot": false
+    }
+  }
+]

test/e2e-v2/cases/aws/s3/docker-compose.yml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: '2.1'
+
+services:
+  oap:
+    extends:
+      file: ../../../script/docker-compose/base-compose.yml
+      service: oap
+    ports:
+      - 12800
+    networks:
+      e2e:
+
+  sender:
+    image: "eclipse-temurin:8-jre"
+    volumes:
+      - ./../../../java-test-service/e2e-mock-sender/target/e2e-mock-sender-2.0.0.jar:/e2e-mock-sender-2.0.0.jar
+      - ./mock-data:/data/otel-metrics
+    command: [ "java", "-jar", "/e2e-mock-sender-2.0.0.jar" ]
+    environment:
+      OAP_HOST: oap
+      OAP_GRPC_PORT: 11800
+      OTEL_METRICS_DATA_PATH: /data/otel-metrics
+    networks:
+      - e2e
+    ports:
+      - 9093:9093
+    healthcheck:
+      test: ["CMD", "sh", "-c", "nc -nz 127.0.0.1 9093"]
+      interval: 5s
+      timeout: 60s
+      retries: 120
+    depends_on:
+      oap:
+        condition: service_healthy
+networks:
+  e2e:

test/e2e-v2/cases/aws/s3/e2e.yaml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+setup:
+  env: compose
+  file: docker-compose.yml
+  timeout: 20m
+  init-system-environment: ../../../script/env
+  steps:
+    - name: set PATH
+      command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+    - name: install yq
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+    - name: install swctl
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+trigger:
+  action: http
+  interval: 3s
+  times: 10
+  url: http://localhost:9093/otel-metrics/send
+  method: GET
+
+verify:
+  retry:
+    count: 10
+    interval: 3s
+  cases:
+    - includes:
+        - ./s3-cases.yaml

test/e2e-v2/cases/aws/s3/expected/metrics-has-value.yml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- key: {{ notEmpty .key }}
+  value: {{ ge .value 0 }}
+{{- end }}

test/e2e-v2/cases/aws/s3/expected/service.yml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+  {{- contains . }}
+- id: {{ b64enc "aws-s3::skywalking" }}.1
+  name: aws-s3::skywalking
+  group: aws-s3
+  shortname: skywalking
+  normal: true
+  layers:
+    - AWS_S3
+  {{- end }}

test/e2e-v2/cases/aws/s3/mock-data/data-1.json

+{
+  "resourceMetrics": [
+    {
+      "resource": {
+        "attributes": [
+          {
+            "key": "cloud.provider",
+            "value": {
+              "stringValue": "aws"
+            }
+          },
+          {
+            "key": "cloud.account.id",
+            "value": {
+              "stringValue": "xxxxxxxx"
+            }
+          },
+          {
+            "key": "cloud.region",
+            "value": {
+              "stringValue": "ap-northeast-1"
+            }
+          },
+          {
+            "key": "aws.exporter.arn",
+            "value": {
+              "stringValue": "arn:aws:cloudwatch:ap-northeast-1:xxxxxxxx:metric-stream/CustomFull-CmHV2P"
+            }
+          }
+        ]
+      },
+      "scopeMetrics": [
+        {
+          "metrics": [
+            {
+              "name": "amazonaws.com/AWS/S3/4xxErrors",
+              "unit": "{Count}",
+              "summary": {
+                "dataPoints": [
+                  {
+                    "startTimeUnixNano": "1674547500000000000",
+                    "timeUnixNano": "1674547560000000000",
+                    "count": "1",
+                    "quantileValues": [
+                      {
+                      },
+                      {
+                        "quantile": 1.0
+                      }
+                    ],
+                    "attributes": [
+                      {
+                        "key": "Namespace",
+                        "value": {
+                          "stringValue": "AWS/S3"
+                        }
+                      },
+                      {
+                        "key": "MetricName",
+                        "value": {
+                          "stringValue": "4xxErrors"
+                        }
+                      },
+                      {
+                        "key": "BucketName",
+                        "value": {
+                          "stringValue": "skywalking"
+                        }
+                      },
+                      {
+                        "key": "FilterId",
+                        "value": {
+                          "stringValue": "test"
+                        }
+                      }
+                    ]
+                  }
+                ]
+              }
+            },
+            {
+              "name": "amazonaws.com/AWS/S3/5xxErrors",
+              "unit": "{Count}",
+              "summary": {
+                "dataPoints": [
+                  {
+                    "startTimeUnixNano": "1674547500000000000",
+                    "timeUnixNano": "1674547560000000000",
+                    "count": "1",
+                    "quantileValues": [
+                      {
+                      },
+                      {
+                        "quantile": 1.0
+                      }
+                    ],
+                    "attributes": [
+                      {
+                        "key": "Namespace",
+                        "value": {
+                          "stringValue": "AWS/S3"
+                        }
+                      },
+                      {
+                        "key": "MetricName",
+                        "value": {
+                          "stringValue": "5xxErrors"
+                        }
+                      },
+                      {
+                        "key": "BucketName",
+                        "value": {
+                          "stringValue": "skywalking"
+                        }
+                      },
+                      {
+                        "key": "FilterId",
+                        "value": {
+                          "stringValue": "test"
+                        }
+                      }
+                    ]
+                  }
+                ]
+              }
+            }
+          ]
+        }
+      ]
+    }
+  ]
+}

test/e2e-v2/cases/aws/s3/s3-cases.yaml

+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+  cases:
+    - query: swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql service ls
+      expected: expected/service.yml
+    - query: |
+        swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear \
+          --name=aws_s3_4xx --service-name aws-s3::skywalking |yq e 'to_entries' -
+      expected: expected/metrics-has-value.yml
+    - query: |
+        swctl --display yaml --base-url=http://${oap_host}:${oap_12800}/graphql metrics linear \
+          --name=aws_s3_5xx --service-name aws-s3::skywalking |yq e 'to_entries' -
+      expected: expected/metrics-has-value.yml