提交 48923827 编写于 作者: W wenzhouwww

Merge branch 'develop' of github.com:taosdata/TDengine into develop

...@@ -53,33 +53,37 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES( ...@@ -53,33 +53,37 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
## <a class="anchor" id="version"></a>TAOS-JDBCDriver 版本以及支持的 TDengine 版本和 JDK 版本 ## <a class="anchor" id="version"></a>TAOS-JDBCDriver 版本以及支持的 TDengine 版本和 JDK 版本
| taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 | | taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 |
| -------------------- | ----------------- | -------- | |--------------------|--------------------| -------- |
| 2.0.33 - 2.0.34 | 2.0.3.0 及以上 | 1.8.x | | 2.0.36 | 2.4.0 及以上 | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 及以上 | 1.8.x | | 2.0.35 | 2.3.0 及以上 | 1.8.x |
| 2.0.33 - 2.0.34 | 2.0.3.0 及以上 | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 及以上 | 1.8.x |
| 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x | | 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x | | 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x | | 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.3 | 1.6.1.x 及以上 | 1.8.x | | 1.0.3 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.2 | 1.6.1.x 及以上 | 1.8.x | | 1.0.2 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.1 | 1.6.1.x 及以上 | 1.8.x | | 1.0.1 | 1.6.1.x 及以上 | 1.8.x |
## TDengine DataType 和 Java DataType ## TDengine DataType 和 Java DataType
TDengine 目前支持时间戳、数字、字符、布尔类型,与 Java 对应类型转换如下: TDengine 目前支持时间戳、数字、字符、布尔类型,与 Java 对应类型转换如下:
| TDengine DataType | JDBCType (driver 版本 < 2.0.24) | JDBCType driver 版本 >= 2.0.24) | | TDengine DataType | JDBCType (driver 版本 < 2.0.24) | JDBCType driver 版本 >= 2.0.24) |
| ----------------- | ------------------ | ------------------ | |-------------------|-------------------------------| ------------------ |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp | | TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer | | INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long | | BIGINT | java.lang.Long | java.lang.Long |
| FLOAT | java.lang.Float | java.lang.Float | | FLOAT | java.lang.Float | java.lang.Float |
| DOUBLE | java.lang.Double | java.lang.Double | | DOUBLE | java.lang.Double | java.lang.Double |
| SMALLINT | java.lang.Short | java.lang.Short | | SMALLINT | java.lang.Short | java.lang.Short |
| TINYINT | java.lang.Byte | java.lang.Byte | | TINYINT | java.lang.Byte | java.lang.Byte |
| BOOL | java.lang.Boolean | java.lang.Boolean | | BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array | | BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String | | NCHAR | java.lang.String | java.lang.String |
| JSON | - | java.lang.String |
注意:JSON类型仅在tag中支持。
## 安装Java Connector ## 安装Java Connector
......
...@@ -749,6 +749,49 @@ conn.execute("drop database pytest") ...@@ -749,6 +749,49 @@ conn.execute("drop database pytest")
conn.close() conn.close()
``` ```
#### JSON 类型
`taospy` `v2.2.0` 开始,Python连接器开始支持 JSON 数据类型的标签(TDengine版本要求 Beta 版 2.3.5+, 稳定版 2.4.0+)。
创建一个使用JSON类型标签的超级表及其子表:
```python
# encoding:UTF-8
import taos
conn = taos.connect()
conn.execute("create database if not exists py_test_json_type")
conn.execute("use py_test_json_type")
conn.execute("create stable s1 (ts timestamp, v1 int) tags (info json)")
conn.execute("create table s1_1 using s1 tags ('{\"k1\": \"v1\"}')")
```
查询子表标签及表名:
```python
tags = conn.query("select info, tbname from s1").fetch_all_into_dict()
tags
```
`tags` 内容为:
```python
[{'info': '{"k1":"v1"}', 'tbname': 's1_1'}]
```
获取 JSON 中某值:
```python
k1 = conn.query("select info->'k1' as k1 from s1").fetch_all_into_dict()
"""
>>> k1
[{'k1': '"v1"'}]
"""
```
更多JSON类型的操作方式请参考 [JSON 类型使用说明](https://www.taosdata.com/cn/documentation/taos-sql)
#### 关于纳秒 (nanosecond) 在 Python 连接器中的说明 #### 关于纳秒 (nanosecond) 在 Python 连接器中的说明
由于目前 Python 对 nanosecond 支持的不完善(参见链接 1. 2. ),目前的实现方式是在 nanosecond 精度时返回整数,而不是 ms 和 us 返回的 datetime 类型,应用开发者需要自行处理,建议使用 pandas 的 to_datetime()。未来如果 Python 正式完整支持了纳秒,涛思数据可能会修改相关接口。 由于目前 Python 对 nanosecond 支持的不完善(参见链接 1. 2. ),目前的实现方式是在 nanosecond 精度时返回整数,而不是 ms 和 us 返回的 datetime 类型,应用开发者需要自行处理,建议使用 pandas 的 to_datetime()。未来如果 Python 正式完整支持了纳秒,涛思数据可能会修改相关接口。
......
...@@ -54,23 +54,25 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES( ...@@ -54,23 +54,25 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
## JDBC driver version and supported TDengine and JDK versions ## JDBC driver version and supported TDengine and JDK versions
| taos-jdbcdriver | TDengine | JDK | | taos-jdbcdriver | TDengine | JDK |
| --------------- | ------------------ | ----- | | --------------- |--------------------|--------|
| 2.0.33 - 2.0.34 | 2.0.3.0 and above | 1.8.x | | 2.0.36 | 2.4.0 and above | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 and above | 1.8.x | | 2.0.35 | 2.3.0 and above | 1.8.x |
| 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x | | 2.0.33 - 2.0.34 | 2.0.3.0 and above | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x | | 2.0.31 - 2.0.32 | 2.1.3.0 and above | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x | | 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x |
| 1.0.3 | 1.6.1.x and above | 1.8.x | | 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 1.0.2 | 1.6.1.x and above | 1.8.x | | 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.1 | 1.6.1.x and above | 1.8.x | | 1.0.3 | 1.6.1.x and above | 1.8.x |
| 1.0.2 | 1.6.1.x and above | 1.8.x |
| 1.0.1 | 1.6.1.x and above | 1.8.x |
## DataType in TDengine and Java connector ## DataType in TDengine and Java connector
The TDengine supports the following data types and Java data types: The TDengine supports the following data types and Java data types:
| TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version >= 2.0.24) | | TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version >= 2.0.24) |
| ----------------- | ---------------------------------- | ----------------------------------- | |-------------------|------------------------------------| ----------------------------------- |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp | | TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer | | INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long | | BIGINT | java.lang.Long | java.lang.Long |
...@@ -81,7 +83,8 @@ The TDengine supports the following data types and Java data types: ...@@ -81,7 +83,8 @@ The TDengine supports the following data types and Java data types:
| BOOL | java.lang.Boolean | java.lang.Boolean | | BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array | | BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String | | NCHAR | java.lang.String | java.lang.String |
| JSON | - | java.lang.String |
**Note**: JSON type can only be used in tag.
## Install Java connector ## Install Java connector
### Runtime Requirements ### Runtime Requirements
......
...@@ -575,6 +575,49 @@ Close connection. ...@@ -575,6 +575,49 @@ Close connection.
conn.close() conn.close()
``` ```
#### JSON Type Support
Python connector `taospy` starts supporting JSON type as tags since `v2.2.0` (requires TDengine beta v2.3.5+, or stable v2.4.0+).
Create stable and table with JSON tag.
```python
# encoding:UTF-8
import taos
conn = taos.connect()
conn.execute("create database if not exists py_test_json_type")
conn.execute("use py_test_json_type")
conn.execute("create stable s1 (ts timestamp, v1 int) tags (info json)")
conn.execute("create table s1_1 using s1 tags ('{\"k1\": \"v1\"}')")
```
Query JSON tag and table name from a stable.
```python
tags = conn.query("select info, tbname from s1").fetch_all_into_dict()
tags
```
The `tags` value is:
```python
[{'info': '{"k1":"v1"}', 'tbname': 's1_1'}]
```
To get value from JSON tag by key:
```python
k1 = conn.query("select info->'k1' as k1 from s1").fetch_all_into_dict()
"""
>>> k1
[{'k1': '"v1"'}]
"""
```
Refer to [JSON type instructions](https://www.taosdata.com/en/documentation/taos-sql) for more usage of JSON type.
#### Using nanosecond in Python connector #### Using nanosecond in Python connector
So far Python still does not completely support nanosecond type. Please refer to the link 1 and 2. The implementation of the python connector is to return an integer number for nanosecond value rather than datatime type as what ms and us do. The developer needs to handle it themselves. We recommend using pandas to_datetime() function. If Python officially support nanosecond in the future, TAOS Data might be possible to change the interface accordingly, which mean the application need change too. So far Python still does not completely support nanosecond type. Please refer to the link 1 and 2. The implementation of the python connector is to return an integer number for nanosecond value rather than datatime type as what ms and us do. The developer needs to handle it themselves. We recommend using pandas to_datetime() function. If Python officially support nanosecond in the future, TAOS Data might be possible to change the interface accordingly, which mean the application need change too.
......
...@@ -360,7 +360,7 @@ int32_t tsParseOneColumn(SSchema *pSchema, SStrToken *pToken, char *payload, cha ...@@ -360,7 +360,7 @@ int32_t tsParseOneColumn(SSchema *pSchema, SStrToken *pToken, char *payload, cha
return tscInvalidOperationMsg(msg, "json tag length too long", pToken->z); return tscInvalidOperationMsg(msg, "json tag length too long", pToken->z);
} }
if (pToken->type == TK_NULL) { if (pToken->type == TK_NULL) {
*(int8_t *)payload = TSDB_DATA_TINYINT_NULL; *(int8_t *)payload = TSDB_DATA_JSON_PLACEHOLDER;
} else if (pToken->type != TK_STRING){ } else if (pToken->type != TK_STRING){
tscInvalidOperationMsg(msg, "invalid json data", pToken->z); tscInvalidOperationMsg(msg, "invalid json data", pToken->z);
} else{ } else{
...@@ -1063,7 +1063,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC ...@@ -1063,7 +1063,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
sToken.n -= 2; sToken.n -= 2;
} }
char tagVal[TSDB_MAX_TAGS_LEN]; char tagVal[TSDB_MAX_TAGS_LEN] = {0};
code = tsParseOneColumn(pSchema, &sToken, tagVal, pInsertParam->msg, &sql, false, tinfo.precision); code = tsParseOneColumn(pSchema, &sToken, tagVal, pInsertParam->msg, &sql, false, tinfo.precision);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
......
...@@ -6871,7 +6871,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -6871,7 +6871,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
} }
SKVRowBuilder kvRowBuilder = {0}; SKVRowBuilder kvRowBuilder = {0};
if (pTagsSchema->type == TSDB_DATA_TYPE_JSON) { if (pTagsSchema->type == TSDB_DATA_TYPE_JSON) {
if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) { if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY && pItem->pVar.nType != TSDB_DATA_TYPE_NULL) {
tscError("json type error, should be string"); tscError("json type error, should be string");
return invalidOperationMsg(pMsg, msg25); return invalidOperationMsg(pMsg, msg25);
} }
...@@ -8714,7 +8714,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { ...@@ -8714,7 +8714,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
} }
tVariantListItem* pItem = taosArrayGet(pValList, 0); tVariantListItem* pItem = taosArrayGet(pValList, 0);
if(pItem->pVar.nType != TSDB_DATA_TYPE_BINARY){ if(pItem->pVar.nType != TSDB_DATA_TYPE_BINARY && pItem->pVar.nType != TSDB_DATA_TYPE_NULL){
tscError("json type error, should be string"); tscError("json type error, should be string");
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
...@@ -8724,6 +8724,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) { ...@@ -8724,6 +8724,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
ret = parseJsontoTagData(pItem->pVar.pz, &kvRowBuilder, tscGetErrorMsgPayload(pCmd), pTagSchema[0].colId); ret = parseJsontoTagData(pItem->pVar.pz, &kvRowBuilder, tscGetErrorMsgPayload(pCmd), pTagSchema[0].colId);
if (ret != TSDB_CODE_SUCCESS) { if (ret != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder); tdDestroyKVRowBuilder(&kvRowBuilder);
......
...@@ -5459,7 +5459,7 @@ int parseJsontoTagData(char* json, SKVRowBuilder* kvRowBuilder, char* errMsg, in ...@@ -5459,7 +5459,7 @@ int parseJsontoTagData(char* json, SKVRowBuilder* kvRowBuilder, char* errMsg, in
varDataSetLen(nullTypeVal + CHAR_BYTES, INT_BYTES); varDataSetLen(nullTypeVal + CHAR_BYTES, INT_BYTES);
*(uint32_t*)(varDataVal(nullTypeKey)) = jsonNULL; *(uint32_t*)(varDataVal(nullTypeKey)) = jsonNULL;
tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeKey, false); // add json null type tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeKey, false); // add json null type
if (strtrim(json) == 0 || strcasecmp(json, "null") == 0){ if (!json || strtrim(json) == 0 || strcasecmp(json, "null") == 0){
*(uint32_t*)(varDataVal(nullTypeVal + CHAR_BYTES)) = jsonNULL; *(uint32_t*)(varDataVal(nullTypeVal + CHAR_BYTES)) = jsonNULL;
tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeVal, true); // add json null value tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeVal, true); // add json null value
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
......
...@@ -953,7 +953,7 @@ int32_t tVariantDumpEx(tVariant *pVariant, char *payload, int16_t type, bool inc ...@@ -953,7 +953,7 @@ int32_t tVariantDumpEx(tVariant *pVariant, char *payload, int16_t type, bool inc
break; break;
} }
case TSDB_DATA_TYPE_JSON:{ case TSDB_DATA_TYPE_JSON:{
if (pVariant->nType == TSDB_DATA_TYPE_BINARY){ if (pVariant->nType == TSDB_DATA_TYPE_BINARY || pVariant->nType == TSDB_DATA_TYPE_NULL){
*((int8_t *)payload) = TSDB_DATA_JSON_PLACEHOLDER; *((int8_t *)payload) = TSDB_DATA_JSON_PLACEHOLDER;
} else if (pVariant->nType == TSDB_DATA_TYPE_JSON){ // select * from stable, set tag type to json,from setTagValue/tag_project_function } else if (pVariant->nType == TSDB_DATA_TYPE_JSON){ // select * from stable, set tag type to json,from setTagValue/tag_project_function
memcpy(payload, pVariant->pz, pVariant->nLen); memcpy(payload, pVariant->pz, pVariant->nLen);
......
...@@ -22,9 +22,9 @@ import json ...@@ -22,9 +22,9 @@ import json
class TDTestCase: class TDTestCase:
def caseDescription(self): def caseDescription(self):
''' '''
Json tag test case, include create table with json tag, Json tag test case, include create table with json tag, select json tag and query with json tag in where condition, besides, include json tag in group by/order by/join/subquery.
select json tag and query with json tag in where condition, case1: [TD-12452] fix error if json tag is NULL
besides, include json tag in group by/order by/join/subquery. case2: [TD-12389] describe child table, tag length error if the tag is json tag
''' '''
return return
...@@ -515,6 +515,23 @@ class TDTestCase: ...@@ -515,6 +515,23 @@ class TDTestCase:
tdSql.query("select jtag->'tag3' from jsons1_16") tdSql.query("select jtag->'tag3' from jsons1_16")
tdSql.checkData(0, 0, '-2.111000000') tdSql.checkData(0, 0, '-2.111000000')
# test TD-12452
tdSql.execute("ALTER TABLE jsons1_1 SET TAG jtag=NULL")
tdSql.query("select jtag from jsons1_1")
tdSql.checkData(0, 0, None)
tdSql.execute("CREATE TABLE if not exists jsons1_20 using jsons1 tags(NULL)")
tdSql.query("select jtag from jsons1_20")
tdSql.checkData(0, 0, None)
tdSql.execute("insert into jsons1_21 using jsons1 tags(NULL) values(1591061628000, 11, false, '你就会','')")
tdSql.query("select jtag from jsons1_21")
tdSql.checkData(0, 0, None)
#test TD-12389
tdSql.query("describe jsons1")
tdSql.checkData(5, 2, 4096)
tdSql.query("describe jsons1_1")
tdSql.checkData(5, 2, 4096)
def stop(self): def stop(self):
tdSql.close() tdSql.close()
tdLog.success("%s successfully executed" % __file__) tdLog.success("%s successfully executed" % __file__)
......
python3 test.py -f 1-insert/TD-11970.py
python3 ./test.py -f 1-insert/batchInsert.py \ No newline at end of file
python3 test.py -f 1-insert/TD-11970.py
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册