Trace-Data-Protocol.md

# Trace Data Protocol 
Trace Data Protocol describes the data format between SkyWalking agent/sniffer and backend. 

## Abstract
This protocol includes the downstream and upstream data format. Other languages agents/SDKs can use this protocol to 
uplink data to the SkyWalking backend.

- Other services, includes Register, Trace, etc., provided by HTTP/JSON and gRPC both. 

### Version
v2.0

#### gRPC proto files
[gRPC proto files](https://github.com/apache/incubator-skywalking-data-collect-protocol/tree/v2.0)


## Trace Segment Service
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/TraceSegmentService.proto)

- UniqueId represents segmentId and globalTraceId. It have 3 parts(Longs), 1) applicationInstanceId, 2) ThreadId, 3) Timestamp + 10000 + seq(seq is in [0, 100000) )
- Span data please refs to [Plugin Development Guide](../setup/service-agent/java-agent/Plugin-Development-Guide.md)
- Id and name both exist, please use id if possible.
  - operationNameId/operationName 
  - networkAddress/networkAddressId
  - entryServiceName/entryServiceId
  - parentServiceName/parentServiceId
  - peerId/peer
- componentIds are defined in backend, [here](../../../apm-protocol/apm-network/src/main/java/org/apache/skywalking/apm/network/trace/component/ComponentsDefine.java)

HTTP format:

Input：
```
[
  {
    "gt": [[230150, 185809, 24040000]], 
    "sg": { //TraceSegmentObject 
      "ts": [137150, 185809, 48780000], 
      "ai": 2, //applicationId
      "ii": 3, //applicationInstanceId
      "ss": [ //SpanObject
        {
          "si": 0, //spanId
          "tv": 0, //SpanType
          "lv": 2, //SpanLayer
          "ps": -1, //parentSpanId
          "st": 1501858094726, //startTime
          "et": 1501858096804, //endTime
          "ci": 3, //componentId
          "cn": "", //component
          "oi": 0, //operationNameId
          "on": "org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()", //operationName
          "pi": 0, //peerId
          "pn": "", //peer
          "ie": false, //isError
          "rs": [ //TraceSegmentReference
            {
              "pts": [230150, 185809, 24040000], //parentTraceSegmentId
              "pii": 2, //parentApplicationInstanceId
              "psp": 1, //parentSpanId
              "psi": 0, //parentServiceId
              "psn": "/dubbox-case/case/dubbox-rest", //parentServiceName
              "ni": 0,  //networkAddressId
              "nn": "172.25.0.4:20880", //networkAddress
              "eii": 2, //entryApplicationInstanceId
              "esi": 0, //entryServiceId
              "esn": "/dubbox-case/case/dubbox-rest", //entryServiceName
              "rv": 0 //RefTypeValue
            }
          ],
          "to": [ //KeyWithStringValue
            {
              "k": "url", //key
              "v": "rest://172.25.0.4:20880/org.skywaking.apm.testcase.dubbo.services.GreetService.doBusiness()" //value
            },
            {
              "k": "http.method",
              "v": "GET"
            }
          ],
          "lo": [{
                "ti": 1501858094726,
                "ld": [{ 
                        "k": "NullPointException",
                        "v": "Error Stack"
                    }]
           }]
        },
        {
          "si": 1,
          "tv": 1,
          "lv": 1,
          "ps": 0,
          "st": 1501858094726,
          "et": 1501858095804,
          "ci": 9,
          "cn": "",
          "oi": 0,
          "on": "mongodb://[username:password@]host1[:port1][,host2[:port2],...[,hostN[:portN]]][/[database][?options]]",
          "pi": 0,
          "pn": "localhost:27017",
          "ie": false,
          "to": [],
          "lo": []
        }
      ]
    }
  }
]
```


## Deprecated services
**Deprecated service**(s) are the gRPC services SkyWalking used before. In SkyWalking v6, in order to match the common 
concepts in CloudNative world, these services are deprecated. 
Although there services are still supported at this moment, but it will be removed after 6 months later(Feb. 2019).

## ~~Application Register Service~~
**Deprecated service** 

### Abstract
Register Application Code to the backend, and receive an integer represents the application.

[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/ApplicationRegisterService.proto)

- applicationCode is the config in your `agent.config`.
- The return id is **ApplicationId** as the value in `KeyWithIntegerValue`, which will be used in further data uplink.

## ~~Discovery Services~~
**Deprecated services** 

### ~~Register Instance Service~~
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L29)

- agentUUID generated by agent, should be unique. Stay same before reboot, at least.
- **ApplicationInstanceId** will be used in further data uplink.

HTTP format http://ip:port/instance/register(default: localhost:12800) 

Input:
```
{
    ai: x, #applicationId
    au: "", #agentUUID
    rt: x, #registerTime
    oi: "", #osinfo
}
```

Output:
```
{
    ai: x, #applicationId
    ii: x, #applicationInstanceId
}
```

### ~~Heart beat service~~
[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L32)

- Recommend to report heart beat every 20-60 seconds.
- Java agent don't use this, because JVM metrics upstream replace the capabilities of this. 

HTTP format http://ip:port/instance/heartbeat(default: localhost:12800) 

Input:
```
{
    "ii": x, #applicationInstanceId
    "ht": x #heartbeatTime, java timestamp format
}
```

## ~~Service Name Discovery Service~~
**Deprecated services**
### Abstract
Replace the literal String service(operation) name by an id(integer)

[gRPC service define](.https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/DiscoveryService.proto#L70)

- Optional service, reduce the network cost but use more memory as a buffer mapping.

HTTP format http://ip:port/servicename/discovery(default: localhost:12800) 

Input：
```
{
    ai: x, #applicationId
    sn: "", #serviceName
    st: x, #srcSpanType
}
```

Output：
```
{
    si: x, #osinfo
    el: { #element
        ai: x, #applicationId
        sn: "", #serviceName
        st: x, #srcSpanType
    }
}
```

## ~~Network Address Register Service~~
### Abstract
Network Address includes all remove service address, includes ip, port, hostname, etc., which used in RPC framework, MQ, DB, etc.

[gRPC service define](https://github.com/apache/incubator-skywalking-data-collect-protocol/blob/v2.0/NetworkAddressRegisterService.proto)

- Optional service, reduce the network cost but use more memory as a buffer mapping.