未验证 提交 6038b728 编写于 作者: sangshuduo's avatar sangshuduo 提交者: GitHub

Docs/sangshuduo/td 12050 en doc sync (#9689)

* [TD-12050]<doc>: sync data insertion telegraf part.

* remove one line description regarding MQTT

* update notice format
上级 538ad2d1
...@@ -20,7 +20,7 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6, ...@@ -20,7 +20,7 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6,
详细的SQL INSERT语法规则请见 [TAOS SQL 的数据写入](https://www.taosdata.com/cn/documentation/taos-sql#insert) 章节。 详细的SQL INSERT语法规则请见 [TAOS SQL 的数据写入](https://www.taosdata.com/cn/documentation/taos-sql#insert) 章节。
**Tips:** **Tips:**
- 要提高写入效率,需要批量写入。一批写入的记录条数越多,插入效率就越高。但一条记录不能超过16K,一条SQL语句总长度不能超过1M 。 - 要提高写入效率,需要批量写入。一批写入的记录条数越多,插入效率就越高。但一条记录不能超过16K,一条SQL语句总长度不能超过1M 。
- TDengine支持多线程同时写入,要进一步提高写入速度,一个客户端需要打开20个以上的线程同时写。但线程数达到一定数量后,无法再提高,甚至还会下降,因为线程频繁切换,带来额外开销。 - TDengine支持多线程同时写入,要进一步提高写入速度,一个客户端需要打开20个以上的线程同时写。但线程数达到一定数量后,无法再提高,甚至还会下降,因为线程频繁切换,带来额外开销。
...@@ -56,7 +56,7 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型,并不需要 ...@@ -56,7 +56,7 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型,并不需要
* 对空格、等号(=)、逗号(,)、双引号("),前面需要使用反斜杠(\)进行转义。(都指的是英文半角符号) * 对空格、等号(=)、逗号(,)、双引号("),前面需要使用反斜杠(\)进行转义。(都指的是英文半角符号)
* 数值类型将通过后缀来区分数据类型: * 数值类型将通过后缀来区分数据类型:
| **序号** | **后缀** | **映射类型** | **大小(字节)** | | **序号** | **后缀** | **映射类型** | **大小(字节)** |
| -- | ------- | ---------| ------ | | -- | ------- | ---------| ------ |
| 1 | 无或f64 | double | 8 | | 1 | 无或f64 | double | 8 |
| 2 | f32 | float | 4 | | 2 | f32 | float | 4 |
...@@ -231,16 +231,16 @@ prometheus产生的数据格式如下: ...@@ -231,16 +231,16 @@ prometheus产生的数据格式如下:
```json ```json
{ {
Timestamp: 1576466279341, Timestamp: 1576466279341,
Value: 37.000000, Value: 37.000000,
apiserver_request_latencies_bucket { apiserver_request_latencies_bucket {
component="apiserver", component="apiserver",
instance="192.168.99.116:8443", instance="192.168.99.116:8443",
job="kubernetes-apiservers", job="kubernetes-apiservers",
le="125000", le="125000",
resource="persistentvolumes", resource="persistentvolumes",
scope="cluster", scope="cluster",
verb="LIST", verb="LIST",
version="v1" version="v1"
} }
} }
``` ```
...@@ -251,6 +251,7 @@ select * from apiserver_request_latencies_bucket; ...@@ -251,6 +251,7 @@ select * from apiserver_request_latencies_bucket;
``` ```
## <a class="anchor" id="telegraf"></a> Telegraf 直接写入(通过 taosAdapter) ## <a class="anchor" id="telegraf"></a> Telegraf 直接写入(通过 taosAdapter)
安装 Telegraf 请参考[官方文档](https://portal.influxdata.com/downloads/) 安装 Telegraf 请参考[官方文档](https://portal.influxdata.com/downloads/)
TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 Telegraf 的多种应用的数据写入。 TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 Telegraf 的多种应用的数据写入。
...@@ -276,6 +277,7 @@ sudo systemctl start telegraf ...@@ -276,6 +277,7 @@ sudo systemctl start telegraf
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。 taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter) ## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter)
安装 collectd,请参考[官方文档](https://collectd.org/download.shtml) 安装 collectd,请参考[官方文档](https://collectd.org/download.shtml)
TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 collectd 的多种应用的数据写入。 TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 collectd 的多种应用的数据写入。
...@@ -294,6 +296,7 @@ sudo systemctl start collectd ...@@ -294,6 +296,7 @@ sudo systemctl start collectd
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。 taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter) ## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter)
安装 StatsD 安装 StatsD
请参考[官方文档](https://github.com/statsd/statsd) 请参考[官方文档](https://github.com/statsd/statsd)
...@@ -316,6 +319,30 @@ port: 8125 ...@@ -316,6 +319,30 @@ port: 8125
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。 taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
icinga2 可以收集监控和性能数据并写入 OpenTSDB,taosAdapter 可以支持接收 icinga2 的数据并写入到 TDengine 中。
## <a class="anchor" id="icinga2"></a> icinga2 直接写入(通过 taosAdapter)
* 参考链接 https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer 使能 opentsdb-writer
* 使能 taosAdapter 配置项 opentsdb_telnet.enable
* 修改配置文件 /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="tcollector"></a> TCollector 直接写入(通过 taosAdapter)
TCollector 是一个在客户侧收集本地收集器并发送数据到 OpenTSDB 的进程,taosAdaapter 可以支持接收 TCollector 的数据并写入到 TDengine 中。
使能 taosAdapter 配置项 opentsdb_telnet.enable
修改 TCollector 配置文件,修改 OpenTSDB 宿主机地址为 taosAdapter 被部署的地址,并修改端口号为 taosAdapter 使用的端口(默认6049)。
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="taosadapter2-telegraf"></a> 使用 Bailongma 2.0 接入 Telegraf 数据写入 ## <a class="anchor" id="taosadapter2-telegraf"></a> 使用 Bailongma 2.0 接入 Telegraf 数据写入
......
...@@ -2,7 +2,7 @@ ...@@ -2,7 +2,7 @@
TDengine supports multiple ways to write data, including SQL, Prometheus, Telegraf, collectd, StatsD, EMQ MQTT Broker, HiveMQ Broker, CSV file, etc. Kafka, OPC and other interfaces will be provided in the future. Data can be inserted in one single record or in batches, data from one or multiple data collection points can be inserted at the same time. TDengine supports multi-thread insertion, out-of-order data insertion, and also historical data insertion. TDengine supports multiple ways to write data, including SQL, Prometheus, Telegraf, collectd, StatsD, EMQ MQTT Broker, HiveMQ Broker, CSV file, etc. Kafka, OPC and other interfaces will be provided in the future. Data can be inserted in one single record or in batches, data from one or multiple data collection points can be inserted at the same time. TDengine supports multi-thread insertion, out-of-order data insertion, and also historical data insertion.
## <a class="anchor" id="sql"></a> Data Writing via SQL ## <a class="anchor" id="sql"></a> Data Writing via SQL
Applications insert data by executing SQL insert statements through C/C++, Java, Go, C#, Python, Node.js Connectors, and users can manually enter SQL insert statements to insert data through TAOS Shell. For example, the following insert writes a record to table d1001: Applications insert data by executing SQL insert statements through C/C++, Java, Go, C#, Python, Node.js Connectors, and users can manually enter SQL insert statements to insert data through TAOS Shell. For example, the following insert writes a record to table d1001:
...@@ -119,16 +119,16 @@ The format of generated data by Prometheus is as follows: ...@@ -119,16 +119,16 @@ The format of generated data by Prometheus is as follows:
```json ```json
{ {
Timestamp: 1576466279341, Timestamp: 1576466279341,
Value: 37.000000, Value: 37.000000,
apiserver_request_latencies_bucket { apiserver_request_latencies_bucket {
component="apiserver", component="apiserver",
instance="192.168.99.116:8443", instance="192.168.99.116:8443",
job="kubernetes-apiservers", job="kubernetes-apiservers",
le="125000", le="125000",
resource="persistentvolumes", s resource="persistentvolumes", s
cope="cluster", cope="cluster",
verb="LIST", verb="LIST",
version=“v1" version=“v1"
} }
} }
``` ```
...@@ -167,13 +167,13 @@ Now you can query the metrics data of Telegraf from TDengine. ...@@ -167,13 +167,13 @@ Now you can query the metrics data of Telegraf from TDengine.
Please find taosAdapter configuration and usage from `taosadapter --help` output. Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter) ## <a class="anchor" id="collectd"></a> Data Writing via collectd and taosAdapter
Please refer to [official document](https://collectd.org/download.shtml) for collectd installation. Please refer to [official document](https://collectd.org/download.shtml) for collectd installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from collectd. TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from collectd.
Configuration: Configuration:
Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using. Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using.
``` ```
LoadPlugin network LoadPlugin network
<Plugin network> <Plugin network>
...@@ -186,12 +186,12 @@ sudo systemctl start collectd ...@@ -186,12 +186,12 @@ sudo systemctl start collectd
``` ```
Please find taosAdapter configuration and usage from `taosadapter --help` output. Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter) ## <a class="anchor" id="statsd"></a> Data Writting via StatsD and taosAdapter
Please refer to [official document](https://github.com/statsd/statsd) for StatsD installation. Please refer to [official document](https://github.com/statsd/statsd) for StatsD installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from StatsD. TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from StatsD.
Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using. Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using.
``` ```
add "./backends/repeater" to backends section. add "./backends/repeater" to backends section.
add { host:'<TDengine server/cluster host>', port: <port for StatsD>} to repeater section. add { host:'<TDengine server/cluster host>', port: <port for StatsD>} to repeater section.
...@@ -206,8 +206,30 @@ port: 8125 ...@@ -206,8 +206,30 @@ port: 8125
} }
``` ```
## <a class="anchor" id="cinga2"></a> Data Writting via icinga2 and taosAdapter
Use icinga2 to collect check result metrics and performance data
* Follow the doc to enable opentsdb-writer https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the configuration file /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
Please find taosAdapter configuration and usage from `taosadapter --help` output. Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="tcollector"></a> Data Writting via TCollector and taosAdapter
TCollector is a client-side process that gathers data from local collectors and pushes the data to OpenTSDB. You run it on all your hosts, and it does the work of sending each host’s data to the TSD (OpenTSDB backend process).
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the TCollector configuration file, modify the OpenTSDB host to the host where taosAdapter is deployed, and modify the port to 6049
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="taosadapter2-telegraf"></a> Insert data via Bailongma 2.0 and Telegraf ## <a class="anchor" id="taosadapter2-telegraf"></a> Insert data via Bailongma 2.0 and Telegraf
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册