未验证 提交 8d49b882 编写于 作者: wu-sheng's avatar wu-sheng 提交者: GitHub

Update docs for new data-collecting and query protocols (#9809)

上级 b5cd3fad
......@@ -98,5 +98,7 @@
* Add re-post for blog `Scaling with Apache SkyWalking` in the academy list.
* Add re-post for blog `Diagnose Service Mesh Network Performance with eBPF` in the academy list.
* Add **Security Notice** doc.
* Add new docs for `Report Span Attached Events` data collecting protocol.
* Add new docs for `Record` query protocol
All issues and pull requests are [here](https://github.com/apache/skywalking/milestone/149?closed=1)
# Manual instrument SDK
Our incredible community has contributed to the manual instrument SDK.
- [Rust](https://github.com/apache/skywalking-rust). Rust SDK follows the SkyWalking format.
- [Go2Sky](https://github.com/SkyAPM/go2sky). Go SDK follows the SkyWalking format.
- [C++](https://github.com/SkyAPM/cpp2sky). C++ SDK follows the SkyWalking format.
......
......@@ -13,7 +13,7 @@ Please read SkyWalking language agents documentation to see whether it is suppor
- **Trace Data Protocol** is an out-of-wire data format. Agent/SDK uses this to send traces to SkyWalking OAP server.
[SkyWalking Trace Data Protocol v3](Trace-Data-Protocol-v3.md) defines the communication method and format between the agent and backend.
[SkyWalking Trace Data Protocol v3.1](Trace-Data-Protocol-v3.md) defines the communication method and format between the agent and backend.
### Logging
- **Log Data Protocol** is an out-of-wire data format. Agent/SDK and collector use this to send logs into SkyWalking OAP server.
......
# Trace Data Protocol v3
# Trace Data Protocol v3.1
Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend.
## Overview
Trace data protocol is defined and provided in [gRPC format](https://github.com/apache/skywalking-data-collect-protocol), and implemented in [HTTP 1.1](HTTP-API-Protocol.md).
### Report service instance status
## Report service instance status
1. Service Instance Properties
Service instance contains more information than just a name. In order for the agent to report service instance status, use `ManagementService#reportInstanceProperties` service to provide a string-key/string-value pair list as the parameter. The `language` of target instance must be provided as the minimum requirement.
2. Service Ping
Service instance should keep alive with the backend. The agent should set a scheduler using `ManagementService#keepAlive` service every minute.
### Send trace and metrics
## Send trace and JVM metrics
After you have the service ID and service instance ID ready, you could send traces and metrics. Now we
have
1. `TraceSegmentReportService#collect` for the SkyWalking native trace format
......@@ -40,7 +39,7 @@ See [Cross Process Propagation Headers Protocol v3](Skywalking-Cross-Process-Pro
4. `Span#skipAnalysis` may be TRUE, if this span doesn't require backend analysis.
### Protocol Definition
## Trace Report Protocol
```protobuf
// The segment is a collection of spans. It includes all collected spans in a simple one request context, such as a HTTP request process.
//
......@@ -220,3 +219,70 @@ message SegmentCollection {
repeated SegmentObject segments = 1;
}
```
## Report Span Attached Events
Besides in-process agents, there are other out-of-process agent, such as ebpf agent, could report additional information
as attached events for the relative spans.
`SpanAttachedEventReportService#collect` for attached event reporting.
```protobuf
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// ebpf agent(SkyWalking Rover) collects extra information from the OS(Linux Only) level to attach on the traced span.
// Since v3.1
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
service SpanAttachedEventReportService {
// Collect SpanAttachedEvent to the OAP server in the streaming mode.
rpc collect (stream SpanAttachedEvent) returns (Commands) {
}
}
// SpanAttachedEvent represents an attached event for a traced RPC.
//
// When an RPC is being traced by the in-process language agent, a span would be reported by the client-side agent.
// And the rover would be aware of this RPC due to the existing tracing header.
// Then, the rover agent collects extra information from the OS level to provide assistance information to diagnose network performance.
message SpanAttachedEvent {
// The nanosecond timestamp of the event's start time.
// Notice, most unit of timestamp in SkyWalking is milliseconds, but NANO-SECOND is required here.
// Because the attached event happens in the OS syscall level, most of them are executed rapidly.
Instant startTime = 1;
// The official event name.
// For example, the event name is a method signature from syscall stack.
string event = 2;
// [Optional] The nanosecond timestamp of the event's end time.
Instant endTime = 3;
// The tags for this event includes some extra OS level information,
// such as
// 1. net_device used for this exit span.
// 2. network L7 protocol
repeated KeyStringValuePair tags = 4;
// The summary of statistics during this event.
// Each statistic provides a name(metric name) to represent the name, and an int64/long as the value.
repeated KeyIntValuePair summary = 5;
// Refer to a trace context decoded from `sw8` header through network, such as HTTP header, MQ metadata
// https://skywalking.apache.org/docs/main/next/en/protocols/skywalking-cross-process-propagation-headers-protocol-v3/#standard-header-item
SpanReference traceContext = 6;
message SpanReference {
SpanReferenceType type = 1;
// [Optional] A string id represents the whole trace.
string traceId = 2;
// A unique id represents this segment. Other segments could use this id to reference as a child segment.
// [Optional] when this span reference
string traceSegmentId = 3;
// If type == SKYWALKING
// The number id of the span. Should be unique in the whole segment.
// Starting at 0
//
// If type == ZIPKIN
// The type of span ID is string.
string spanId = 4;
}
enum SpanReferenceType {
SKYWALKING = 0;
ZIPKIN = 1;
}
}
```
\ No newline at end of file
......@@ -80,6 +80,7 @@ extend type Query {
readLabeledMetricsValues(condition: MetricsCondition!, labels: [String!]!, duration: Duration!): [MetricsValues!]!
# Heatmap is bucket based value statistic result.
readHeatMap(condition: MetricsCondition!, duration: Duration!): HeatMap
# Deprecated since 9.3.0, replaced by readRecords defined in record.graphqls
# Read the sampled records
# TopNCondition#scope is not required.
readSampledRecords(condition: TopNCondition!, duration: Duration!): [SelectedRecord!]!
......@@ -127,6 +128,18 @@ extend type Query {
}
```
### Record
Record is a general and abstract type for collected raw data.
In the observability, traces and logs have specific and well-defined meanings, meanwhile, the general records represent other
collected records. Such as sampled slow SQL statement, HTTP request raw data(request/response header/body)
```graphql
extend type Query {
# Query collected records with given metric name and parent entity conditions, and return in the requested order.
readRecords(condition: RecordCondition!, duration: Duration!): [Record!]!
}
```
### Logs
```graphql
extend type Query {
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册