From da112c0fec78fc962128bcbaa300d63e6a752ef5 Mon Sep 17 00:00:00 2001 From: Wu Sheng Date: Thu, 16 Aug 2018 13:12:13 +0800 Subject: [PATCH] Change protocol documents --- docs/en/protocols/JVM-Protocol.md | 5 + docs/en/protocols/README.md | 12 +- docs/en/protocols/Trace-Data-Protocol.md | 231 +++++++++++------------ 3 files changed, 122 insertions(+), 126 deletions(-) create mode 100644 docs/en/protocols/JVM-Protocol.md diff --git a/docs/en/protocols/JVM-Protocol.md b/docs/en/protocols/JVM-Protocol.md new file mode 100644 index 000000000..3e0bc23a4 --- /dev/null +++ b/docs/en/protocols/JVM-Protocol.md @@ -0,0 +1,5 @@ +# JVM Metrics Service +## Abstract +Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every second. + +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/JVMMetricsService.proto) \ No newline at end of file diff --git a/docs/en/protocols/README.md b/docs/en/protocols/README.md index 69413b0c0..376630c25 100644 --- a/docs/en/protocols/README.md +++ b/docs/en/protocols/README.md @@ -5,9 +5,19 @@ There are two types of protocols list here. - [**Query Protocol**](#query-protocol). The backend provide query capability to SkyWalking own UI and others. These queries are based on GraphQL. + ## Probe Protocols They also related to the probe group, for understand that, look [Concepts and Designs](../concepts-and-designs/README.md) document. -These groups are **Language based native agent protocol**, **Service Mesh protocol** and **3rd-party instrument protocol**. +These groups are **Language based native agent protocol**, **Service Mesh protocol** and **3rd-party instrument protocol**. + +## Register Protocol +Include service, service instance, network address and endpoint meta data register. +Purposes of register are +1. For service, network address and endpoint, register returns the unique ID of register object, usually an integer. Probe +can use that to represent the literal String for data compression. Further, some protocols accept IDs only. +1. For service instance, register returns a new unique ID for every new instance. Every service instance register must contain the +service ID. + ### Language based native agent protocol diff --git a/docs/en/protocols/Trace-Data-Protocol.md b/docs/en/protocols/Trace-Data-Protocol.md index 67426c2ee..2c25e9325 100644 --- a/docs/en/protocols/Trace-Data-Protocol.md +++ b/docs/en/protocols/Trace-Data-Protocol.md @@ -5,78 +5,17 @@ Trace Data Protocol describes the data format between SkyWalking agent/sniffer a This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to uplink data to the SkyWalking backend. -- Discovery service provided by HTTP only - Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both. ### Version -v1.1 +v2.0 #### gRPC proto files -[gRPC proto files](https://github.com/apache/incubator-skywalking-data-collect-protocol/tree/v1.1.1) +[gRPC proto files](https://github.com/apache/incubator-skywalking-data-collect-protocol/tree/v2.0) -## Collector discovery service -### Abstract -**Collector discovery service should be the first service after agent started** -Through this service, get the gRPC service list. The agent can choose any one of them for uplink data. Recommend to -acquire the list periodically. - -### HTTP GET -- Input -GET：http://collectorIp:port/agent/gRPC - -- Output -JSON Array. Each element in the array ia a valid gRPC service address. -```json -["ip address1:port1","ip address2:port2","ip address3:port3"] -``` - -## Application Register Service -### Abstract -Register Application Code to the backend, and receive an integer represents the application. - -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/ApplicationRegisterService.proto) - -- applicationCode is the config in your `agent.config`. -- The return id is **ApplicationId** as the value in `KeyWithIntegerValue`, which will be used in further data uplink. - -## Discovery Service - -### Register Instance Service -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L29) - -- agentUUID generated by agent, should be unique. Stay same before reboot, at least. -- **ApplicationInstanceId** will be used in further data uplink. - -### Heart beat service -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L32) - -- Recommend to report heart beat every 20-60 seconds. -- Java agent don't use this, because JVM metrics upstream replace the capabilities of this. - -## Service Name Discovery Service -### Abstract -Replace the literal String service(operation) name by an id(integer) - -[gRPC service define](.https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/DiscoveryService.proto#L70) - -- Optional service, reduce the network cost but use more memory as a buffer mapping. - -## Network Address Register Service -### Abstract -Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc. - -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/NetworkAddressRegisterService.proto) - -- Optional service, reduce the network cost but use more memory as a buffer mapping. - -## JVM Metrics Service -### Abstract -Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every second. - -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/JVMMetricsService.proto) ## Trace Segment Service -[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v1.1.1/TraceSegmentService.proto) +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/TraceSegmentService.proto) - UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) ) - Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md) @@ -88,68 +27,8 @@ Uplink the JVM metrics, including PermSize, HeapSize, CPU, Memory, etc., every s - peerId/peer - componentIds are defined in backend, [here](../../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java) -# HTTP JSON Services -All HTTP Services match the gRPC services, just adjust use short keys. - -## Instance Register Service -- http://ip:port/instance/register(default: localhost:12800) - -Input: -``` -{ - ai: x, #applicationId - au: "", #agentUUID - rt: x, #registerTime - oi: "", #osinfo -} -``` - -Output: -``` -{ - ai: x, #applicationId - ii: x, #applicationInstanceId -} -``` - -## Heart beat service -- http://ip:port/instance/heartbeat(default: localhost:12800) - -Input: -``` -{ - "ii": x, #applicationInstanceId - "ht": x #heartbeatTime, java timestamp format -} -``` - -Output: empty - -## Service Name Discovery -- http://ip:port/servicename/discovery(default: localhost:12800) +HTTP format: -Input： -``` -{ - ai: x, #applicationId - sn: "", #serviceName - st: x, #srcSpanType -} -``` - -Output： -``` -{ - si: x, #osinfo - el: { #element - ai: x, #applicationId - sn: "", #serviceName - st: x, #srcSpanType - } -} -``` - -## Trace Segment Service Input： ``` [ @@ -229,3 +108,105 @@ Input： } ] ``` + + +## Deprecated services +**Deprecated service**(s) are the gRPC service name SkyWalking used before. In SkyWalking v6, in order to match the common +concepts in CloudNative world. There services are still supported at this moment, but it will be removed at Feb. 2019. + +## ~~Application Register Service~~ +**Deprecated service** + +### Abstract +Register Application Code to the backend, and receive an integer represents the application. + +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/ApplicationRegisterService.proto) + +- applicationCode is the config in your `agent.config`. +- The return id is **ApplicationId** as the value in `KeyWithIntegerValue`, which will be used in further data uplink. + +## ~~Discovery Services~~ +**Deprecated services** + +### ~~Register Instance Service~~ +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L29) + +- agentUUID generated by agent, should be unique. Stay same before reboot, at least. +- **ApplicationInstanceId** will be used in further data uplink. + +HTTP format http://ip:port/instance/register(default: localhost:12800) + +Input: +``` +{ + ai: x, #applicationId + au: "", #agentUUID + rt: x, #registerTime + oi: "", #osinfo +} +``` + +Output: +``` +{ + ai: x, #applicationId + ii: x, #applicationInstanceId +} +``` + +### ~~Heart beat service~~ +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L32) + +- Recommend to report heart beat every 20-60 seconds. +- Java agent don't use this, because JVM metrics upstream replace the capabilities of this. + +HTTP format http://ip:port/instance/heartbeat(default: localhost:12800) + +Input: +``` +{ + "ii": x, #applicationInstanceId + "ht": x #heartbeatTime, java timestamp format +} +``` + +## ~~Service Name Discovery Service~~ +**Deprecated services** +### Abstract +Replace the literal String service(operation) name by an id(integer) + +[gRPC service define](.https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L70) + +- Optional service, reduce the network cost but use more memory as a buffer mapping. + +HTTP format http://ip:port/servicename/discovery(default: localhost:12800) + +Input： +``` +{ + ai: x, #applicationId + sn: "", #serviceName + st: x, #srcSpanType +} +``` + +Output： +``` +{ + si: x, #osinfo + el: { #element + ai: x, #applicationId + sn: "", #serviceName + st: x, #srcSpanType + } +} +``` + +## ~~Network Address Register Service~~ +### Abstract +Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc. + +[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/NetworkAddressRegisterService.proto) + +- Optional service, reduce the network cost but use more memory as a buffer mapping. + -- GitLab