From afa744842032b9863f8839d1ce6e473ab6f7f8e5 Mon Sep 17 00:00:00 2001 From: wangmm0220 Date: Wed, 16 Aug 2023 11:07:43 +0800 Subject: [PATCH] docs:modify format & add logic for dot in schemaless --- .../07-develop/03-insert-data/30-influxdb-line.mdx | 10 ++++++---- .../03-insert-data/40-opentsdb-telnet.mdx | 10 ++++++---- .../07-develop/03-insert-data/50-opentsdb-json.mdx | 10 ++++++---- docs/en/14-reference/03-connector/03-cpp.mdx | 7 ++++--- .../en/14-reference/13-schemaless/13-schemaless.md | 4 ++-- .../07-develop/03-insert-data/30-influxdb-line.mdx | 9 +++++---- .../03-insert-data/40-opentsdb-telnet.mdx | 11 ++++++----- .../07-develop/03-insert-data/50-opentsdb-json.mdx | 9 +++++---- docs/zh/08-connector/10-cpp.mdx | 14 +++++++------- .../zh/14-reference/13-schemaless/13-schemaless.md | 4 ++-- 10 files changed, 49 insertions(+), 39 deletions(-) diff --git a/docs/en/07-develop/03-insert-data/30-influxdb-line.mdx b/docs/en/07-develop/03-insert-data/30-influxdb-line.mdx index 4c850532df..e2e31ba07f 100644 --- a/docs/en/07-develop/03-insert-data/30-influxdb-line.mdx +++ b/docs/en/07-develop/03-insert-data/30-influxdb-line.mdx @@ -35,11 +35,13 @@ meters,location=California.LosAngeles,groupid=2 current=13.4,voltage=223,phase=0 :::note -- All the data in `tag_set` will be converted to NCHAR type automatically . -- Each data in `field_set` must be self-descriptive for its data type. For example 1.2f32 means a value 1.2 of float type. Without the "f" type suffix, it will be treated as type double. -- Multiple kinds of precision can be used for the `timestamp` field. Time precision can be from nanosecond (ns) to hour (h). -- The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be created automatically. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. +- All the data in `tag_set` will be converted to NCHAR type automatically +- Each data in `field_set` must be self-descriptive for its data type. For example 1.2f32 means a value 1.2 of float type. Without the "f" type suffix, it will be treated as type double +- Multiple kinds of precision can be used for the `timestamp` field. Time precision can be from nanosecond (ns) to hour (h) +- The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be created automatically. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored - It is assumed that the order of field_set in a supertable is consistent, meaning that the first record contains all fields and subsequent records store fields in the same order. If the order is not consistent, set smlDataFormat in taos.cfg to false. Otherwise, data will be written out of order and a database error will occur.(smlDataFormat in taos.cfg default to false after version of 3.0.1.3, smlDataFormat is discarded since 3.0.3.0) +- Due to the fact that SQL table names do not support point(.), schemaless has also processed point(.) and automatically replaced them with underscores(_) + ::: For more details please refer to [InfluxDB Line Protocol](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) and [TDengine Schemaless](/reference/schemaless/#Schemaless-Line-Protocol) diff --git a/docs/en/07-develop/03-insert-data/40-opentsdb-telnet.mdx b/docs/en/07-develop/03-insert-data/40-opentsdb-telnet.mdx index 78ced058bf..fe35df2f43 100644 --- a/docs/en/07-develop/03-insert-data/40-opentsdb-telnet.mdx +++ b/docs/en/07-develop/03-insert-data/40-opentsdb-telnet.mdx @@ -34,6 +34,8 @@ meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3 ``` - The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. +- Due to the fact that SQL table names do not support point(.), schemaless has also processed point(.) and automatically replaced them with underscores(_) + Please refer to [OpenTSDB Telnet API](http://opentsdb.net/docs/build/html/api_telnet/put.html) for more details. ## Examples @@ -68,11 +70,11 @@ Database changed. taos> show stables; name | ================================= - meters.current | - meters.voltage | + meters_current | + meters_voltage | Query OK, 2 row(s) in set (0.002544s) -taos> select tbname, * from `meters.current`; +taos> select tbname, * from `meters_current`; tbname | _ts | _value | groupid | location | ================================================================================================================================== t_0e7bcfa21a02331c06764f275... | 2022-03-28 09:56:51.249 | 10.800000000 | 3 | California.LosAngeles | @@ -87,5 +89,5 @@ Query OK, 4 row(s) in set (0.005399s) If you want query the data of `location=California.LosAngeles groupid=3`, here is the query SQL: ```sql -SELECT * FROM `meters.current` WHERE location = "California.LosAngeles" AND groupid = 3; +SELECT * FROM `meters_current` WHERE location = "California.LosAngeles" AND groupid = 3; ``` diff --git a/docs/en/07-develop/03-insert-data/50-opentsdb-json.mdx b/docs/en/07-develop/03-insert-data/50-opentsdb-json.mdx index b63bf0d93f..8ed1133960 100644 --- a/docs/en/07-develop/03-insert-data/50-opentsdb-json.mdx +++ b/docs/en/07-develop/03-insert-data/50-opentsdb-json.mdx @@ -49,6 +49,8 @@ Please refer to [OpenTSDB HTTP API](http://opentsdb.net/docs/build/html/api_http - In JSON protocol, strings will be converted to NCHAR type and numeric values will be converted to double type. - The child table name is created automatically in a rule to guarantee its uniqueness. But you can configure `smlChildTableName` in taos.cfg to specify a tag value as the table names if the tag value is unique globally. For example, if a tag is called `tname` and you set `smlChildTableName=tname` in taos.cfg, when you insert `st,tname=cpu1,t1=4 c1=3 1626006833639000000`, the child table `cpu1` will be automatically created. Note that if multiple rows have the same tname but different tag_set values, the tag_set of the first row is used to create the table and the others are ignored. +- Due to the fact that SQL table names do not support point(.), schemaless has also processed point(.) and automatically replaced them with underscores(_) + ::: ## Examples @@ -83,11 +85,11 @@ Database changed. taos> show stables; name | ================================= - meters.current | - meters.voltage | + meters_current | + meters_voltage | Query OK, 2 row(s) in set (0.001954s) -taos> select * from `meters.current`; +taos> select * from `meters_current`; _ts | _value | groupid | location | =================================================================================================================== 2022-03-28 09:56:51.249 | 10.300000000 | 2.000000000 | California.SanFrancisco | @@ -100,5 +102,5 @@ Query OK, 2 row(s) in set (0.004076s) If you want query the data of "tags": {"location": "California.LosAngeles", "groupid": 1}, here is the query SQL: ```sql -SELECT * FROM `meters.current` WHERE location = "California.LosAngeles" AND groupid = 3; +SELECT * FROM `meters_current` WHERE location = "California.LosAngeles" AND groupid = 3; ``` diff --git a/docs/en/14-reference/03-connector/03-cpp.mdx b/docs/en/14-reference/03-connector/03-cpp.mdx index 0009902425..8874398c76 100644 --- a/docs/en/14-reference/03-connector/03-cpp.mdx +++ b/docs/en/14-reference/03-connector/03-cpp.mdx @@ -454,6 +454,7 @@ In addition to writing data using the SQL method or the parameter binding API, w - zero success,none zero failed, wrong message can be obtained through `char *tmq_err2str(int32_t code)` - `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)` + **Function description** - get the committed offset @@ -467,9 +468,9 @@ In addition to writing data using the SQL method or the parameter binding API, w **Function description** - The commit interface is divided into two types, each with synchronous and asynchronous interfaces: - - The first type: based on message submission, submit the progress in the message. If the message passes NULL, submit the current progress of all vgroups consumed by the current consumer: tmq_commit_sync/tmq_commit_async - - The second type: submit based on the offset of a Vgroup in a topic: tmq_commit_offset_sync/tmq_commit_offset_async + - The commit interface is divided into two types, each with synchronous and asynchronous interfaces: + - The first type: based on message submission, submit the progress in the message. If the message passes NULL, submit the current progress of all vgroups consumed by the current consumer: tmq_commit_sync/tmq_commit_async + - The second type: submit based on the offset of a Vgroup in a topic: tmq_commit_offset_sync/tmq_commit_offset_async **Parameter description** - msg：Message consumed, If the message passes NULL, submit the current progress of all vgroups consumed by the current consumer diff --git a/docs/en/14-reference/13-schemaless/13-schemaless.md b/docs/en/14-reference/13-schemaless/13-schemaless.md index 54be18eea3..e17bc5f31e 100644 --- a/docs/en/14-reference/13-schemaless/13-schemaless.md +++ b/docs/en/14-reference/13-schemaless/13-schemaless.md @@ -32,8 +32,8 @@ All data in tag_set is automatically converted to the NCHAR data type and does n In the schemaless writing data line protocol, each data item in the field_set needs to be described with its data type. Let's explain in detail: -- If there are English double quotes on both sides, it indicates the BINARY(32) type. For example, `"abc"`. -- If there are double quotes on both sides and an L prefix, it means NCHAR(32) type. For example, `L"error message"`. +- If there are English double quotes on both sides, it indicates the VARCHAR(N) type. For example, `"abc"`. +- If there are double quotes on both sides and an L prefix, it means NCHAR(N) type. For example, `L"error message"`. - Spaces, equals sign (=), comma (,), double quote ("), and backslash (\\) need to be escaped with a backslash (\\) in front. (All refer to the ASCII character). The rules are as follows: | **Serial number** | **Element** | **Escape characters** | diff --git a/docs/zh/07-develop/03-insert-data/30-influxdb-line.mdx b/docs/zh/07-develop/03-insert-data/30-influxdb-line.mdx index 258adaaeed..df69974e4b 100644 --- a/docs/zh/07-develop/03-insert-data/30-influxdb-line.mdx +++ b/docs/zh/07-develop/03-insert-data/30-influxdb-line.mdx @@ -34,12 +34,13 @@ meters,location=California.LosAngeles,groupid=2 current=13.4,voltage=223,phase=0 :::note -- tag_set 中的所有的数据自动转化为 NCHAR 数据类型; -- field_set 中的每个数据项都需要对自身的数据类型进行描述, 比如 1.2f32 代表 FLOAT 类型的数值 1.2, 如果不带类型后缀会被当作 DOUBLE 处理; -- timestamp 支持多种时间精度。写入数据的时候需要用参数指定时间精度，支持从小时到纳秒的 6 种时间精度。 +- tag_set 中的所有的数据自动转化为 NCHAR 数据类型 +- field_set 中的每个数据项都需要对自身的数据类型进行描述, 比如 1.2f32 代表 FLOAT 类型的数值 1.2, 如果不带类型后缀会被当作 DOUBLE 处理 +- timestamp 支持多种时间精度。写入数据的时候需要用参数指定时间精度，支持从小时到纳秒的 6 种时间精度 - 为了提高写入的效率，默认假设同一个超级表中 field_set 的顺序是一样的（第一条数据包含所有的 field，后面的数据按照这个顺序），如果顺序不一样，需要配置参数 smlDataFormat 为 false，否则，数据写入按照相同顺序写入，库中数据会异常。（3.0.1.3 之后的版本 smlDataFormat 默认为 false，从3.0.3.0开始，该配置废弃） [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) - 默认产生的子表名是根据规则生成的唯一 ID 值。用户也可以通过在client端的 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 st,tname=cpu1,t1=4 c1=3 1626006833639000000 则创建的子表名为 cpu1。注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。[TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) -::: +- 由于sql建表表名不支持点号（.），所以schemaless也对点号（.）做了处理，自动替换为下划线（_） +:: 要了解更多可参考：[InfluxDB Line 协议官方文档](https://docs.influxdata.com/influxdb/v2.0/reference/syntax/line-protocol/) 和 [TDengine 无模式写入参考指南](/reference/schemaless/#无模式写入行协议) diff --git a/docs/zh/07-develop/03-insert-data/40-opentsdb-telnet.mdx b/docs/zh/07-develop/03-insert-data/40-opentsdb-telnet.mdx index 9da767a691..2219c70a2a 100644 --- a/docs/zh/07-develop/03-insert-data/40-opentsdb-telnet.mdx +++ b/docs/zh/07-develop/03-insert-data/40-opentsdb-telnet.mdx @@ -33,6 +33,7 @@ meters.current 1648432611250 11.3 location=California.LosAngeles groupid=3 ``` - 默认生产的子表名是根据规则生成的唯一 ID 值。用户也可以通过在client端的 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 meters.current 1648432611250 11.3 tname=cpu1 location=California.LosAngeles groupid=3 则创建的表名为 cpu1，注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。 +- 由于sql建表表名不支持点号（.），所以schemaless也对点号（.）做了处理，自动替换为下划线（_）。 参考 [OpenTSDB Telnet API 文档](http://opentsdb.net/docs/build/html/api_telnet/put.html)。 ## 示例代码 @@ -67,11 +68,11 @@ Database changed. taos> SHOW STABLES; name | ================================= - meters.current | - meters.voltage | + meters_current | + meters_voltage | Query OK, 2 row(s) in set (0.002544s) -taos> SELECT TBNAME, * FROM `meters.current`; +taos> SELECT TBNAME, * FROM `meters_current`; tbname | _ts | _value | groupid | location | ================================================================================================================================== t_0e7bcfa21a02331c06764f275... | 2022-03-28 09:56:51.249 | 10.800000000 | 3 | California.LosAngeles | @@ -83,10 +84,10 @@ Query OK, 4 row(s) in set (0.005399s) ## SQL 查询示例 -`meters.current` 是插入数据的超级表名。 +`meters_current` 是插入数据的超级表名。 可以通过超级表的 TAG 来过滤数据，比如查询 `location=California.LosAngeles groupid=3` 可以通过如下 SQL： ```sql -SELECT * FROM `meters.current` WHERE location = "California.LosAngeles" AND groupid = 3; +SELECT * FROM `meters_current` WHERE location = "California.LosAngeles" AND groupid = 3; ``` diff --git a/docs/zh/07-develop/03-insert-data/50-opentsdb-json.mdx b/docs/zh/07-develop/03-insert-data/50-opentsdb-json.mdx index f2cfbc857b..6f978d9267 100644 --- a/docs/zh/07-develop/03-insert-data/50-opentsdb-json.mdx +++ b/docs/zh/07-develop/03-insert-data/50-opentsdb-json.mdx @@ -48,6 +48,7 @@ OpenTSDB JSON 格式协议采用一个 JSON 字符串表示一行或多行数据 - 对于 JSON 格式协议，TDengine 并不会自动把所有标签转成 NCHAR 类型， 字符串将将转为 NCHAR 类型， 数值将同样转换为 DOUBLE 类型。 - 默认生成的子表名是根据规则生成的唯一 ID 值。用户也可以通过在client端的 taos.cfg 里配置 smlChildTableName 参数来指定某个标签值作为子表名。该标签值应该具有全局唯一性。举例如下：假设有个标签名为tname, 配置 smlChildTableName=tname, 插入数据为 `"tags": { "host": "web02","dc": "lga","tname":"cpu1"}` 则创建的子表名为 cpu1。注意如果多行数据 tname 相同，但是后面的 tag_set 不同，则使用第一行自动建表时指定的 tag_set，其他的行会忽略）。 +- 由于sql建表表名不支持点号（.），所以schemaless也对点号（.）做了处理，自动替换为下划线（_）。 ::: ## 示例代码 @@ -82,8 +83,8 @@ Database changed. taos> SHOW STABLES; name | ================================= - meters.current | - meters.voltage | + meters_current | + meters_voltage | Query OK, 2 row(s) in set (0.001954s) taos> SELECT * FROM `meters.current`; @@ -96,10 +97,10 @@ Query OK, 2 row(s) in set (0.004076s) ## SQL 查询示例 -`meters.voltage` 是插入数据的超级表名。 +`meters_voltage` 是插入数据的超级表名。 可以通过超级表的 TAG 来过滤数据，比如查询 `location=California.LosAngeles groupid=1` 可以通过如下 SQL： ```sql -SELECT * FROM `meters.current` WHERE location = "California.LosAngeles" AND groupid = 3; +SELECT * FROM `meters_current` WHERE location = "California.LosAngeles" AND groupid = 3; ``` diff --git a/docs/zh/08-connector/10-cpp.mdx b/docs/zh/08-connector/10-cpp.mdx index c0723cd85c..e2cd9005bd 100644 --- a/docs/zh/08-connector/10-cpp.mdx +++ b/docs/zh/08-connector/10-cpp.mdx @@ -542,6 +542,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多 - 错误码，0成功，非0失败，可通过 `char *tmq_err2str(int32_t code)` 函数获取错误信息。 - `int64_t tmq_committed(tmq_t *tmq, const char *pTopicName, int32_t vgId)` + **功能说明** - 获取当前 consumer 在某个 topic 和 vgroup上的 commit 位置。 @@ -555,9 +556,9 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多 **功能说明** - commit接口分为两种类型，每种类型有同步和异步接口： - - 第一种类型：根据消息提交，提交消息里的进度，如果消息传NULL，提交当前consumer所有消费的vgroup的当前进度 : tmq_commit_sync/tmq_commit_async - - 第二种类型：根据某个topic的某个vgroup的offset提交 : tmq_commit_offset_sync/tmq_commit_offset_async + - commit接口分为两种类型，每种类型有同步和异步接口： + - 第一种类型：根据消息提交，提交消息里的进度，如果消息传NULL，提交当前consumer所有消费的vgroup的当前进度 : tmq_commit_sync/tmq_commit_async + - 第二种类型：根据某个topic的某个vgroup的offset提交 : tmq_commit_offset_sync/tmq_commit_offset_async **参数说明** - msg：消费到的消息结构，如果msg传NULL，提交当前consumer所有消费的vgroup的当前进度 @@ -584,8 +585,7 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多 - `int32_t int64_t tmq_get_vgroup_offset(TAOS_RES* res)` **功能说明** - - 获取 poll 消费到的数据的起始offset + - 获取 poll 消费到的数据的起始offset **参数说明** - msg：消费到的消息结构 @@ -596,8 +596,8 @@ TDengine 的异步 API 均采用非阻塞调用模式。应用程序可以用多 - `int32_t int32_t tmq_subscription(tmq_t *tmq, tmq_list_t **topics)` **功能说明** - - 获取消费者订阅的 topic 列表 + - 获取消费者订阅的 topic 列表 + **参数说明** - topics: 获取的 topic 列表存储在这个结构中，接口内分配内存，需调用tmq_list_destroy释放 diff --git a/docs/zh/14-reference/13-schemaless/13-schemaless.md b/docs/zh/14-reference/13-schemaless/13-schemaless.md index 9f5bae081c..7f9d24170e 100644 --- a/docs/zh/14-reference/13-schemaless/13-schemaless.md +++ b/docs/zh/14-reference/13-schemaless/13-schemaless.md @@ -33,8 +33,8 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型，并不需要 在无模式写入数据行协议中，field_set 中的每个数据项都需要对自身的数据类型进行描述。具体来说： -- 如果两边有英文双引号，表示 BINARY(32) 类型。例如 `"abc"`。 -- 如果两边有英文双引号而且带有 L 前缀，表示 NCHAR(32) 类型。例如 `L"报错信息"`。 +- 如果两边有英文双引号，表示 VARCHAR(N) 类型。例如 `"abc"`。 +- 如果两边有英文双引号而且带有 L 前缀，表示 NCHAR(N) 类型。例如 `L"报错信息"`。 - 对空格、等号（=）、逗号（,）、双引号（"）、反斜杠（\），前面需要使用反斜杠（\）进行转义。（都指的是英文半角符号）。具体转义规则如下： | **序号** | **域** | **需转义字符** | -- GitLab