提交 6af73688 编写于 作者: W wenzhouwww

Merge branch 'develop' into case_TD_12191

......@@ -53,33 +53,37 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
## <a class="anchor" id="version"></a>TAOS-JDBCDriver 版本以及支持的 TDengine 版本和 JDK 版本
| taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 |
| -------------------- | ----------------- | -------- |
| 2.0.33 - 2.0.34 | 2.0.3.0 及以上 | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 及以上 | 1.8.x |
| taos-jdbcdriver 版本 | TDengine 版本 | JDK 版本 |
|--------------------|--------------------| -------- |
| 2.0.36 | 2.4.0 及以上 | 1.8.x |
| 2.0.35 | 2.3.0 及以上 | 1.8.x |
| 2.0.33 - 2.0.34 | 2.0.3.0 及以上 | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 及以上 | 1.8.x |
| 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.3 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.2 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.1 | 1.6.1.x 及以上 | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.3 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.2 | 1.6.1.x 及以上 | 1.8.x |
| 1.0.1 | 1.6.1.x 及以上 | 1.8.x |
## TDengine DataType 和 Java DataType
TDengine 目前支持时间戳、数字、字符、布尔类型,与 Java 对应类型转换如下:
| TDengine DataType | JDBCType (driver 版本 < 2.0.24) | JDBCType driver 版本 >= 2.0.24) |
| ----------------- | ------------------ | ------------------ |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long |
| FLOAT | java.lang.Float | java.lang.Float |
| DOUBLE | java.lang.Double | java.lang.Double |
| SMALLINT | java.lang.Short | java.lang.Short |
| TINYINT | java.lang.Byte | java.lang.Byte |
| BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String |
|-------------------|-------------------------------| ------------------ |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long |
| FLOAT | java.lang.Float | java.lang.Float |
| DOUBLE | java.lang.Double | java.lang.Double |
| SMALLINT | java.lang.Short | java.lang.Short |
| TINYINT | java.lang.Byte | java.lang.Byte |
| BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String |
| JSON | - | java.lang.String |
注意:JSON类型仅在tag中支持。
## 安装Java Connector
......
......@@ -749,6 +749,49 @@ conn.execute("drop database pytest")
conn.close()
```
#### JSON 类型
`taospy` `v2.2.0` 开始,Python连接器开始支持 JSON 数据类型的标签(TDengine版本要求 Beta 版 2.3.5+, 稳定版 2.4.0+)。
创建一个使用JSON类型标签的超级表及其子表:
```python
# encoding:UTF-8
import taos
conn = taos.connect()
conn.execute("create database if not exists py_test_json_type")
conn.execute("use py_test_json_type")
conn.execute("create stable s1 (ts timestamp, v1 int) tags (info json)")
conn.execute("create table s1_1 using s1 tags ('{\"k1\": \"v1\"}')")
```
查询子表标签及表名:
```python
tags = conn.query("select info, tbname from s1").fetch_all_into_dict()
tags
```
`tags` 内容为:
```python
[{'info': '{"k1":"v1"}', 'tbname': 's1_1'}]
```
获取 JSON 中某值:
```python
k1 = conn.query("select info->'k1' as k1 from s1").fetch_all_into_dict()
"""
>>> k1
[{'k1': '"v1"'}]
"""
```
更多JSON类型的操作方式请参考 [JSON 类型使用说明](https://www.taosdata.com/cn/documentation/taos-sql)
#### 关于纳秒 (nanosecond) 在 Python 连接器中的说明
由于目前 Python 对 nanosecond 支持的不完善(参见链接 1. 2. ),目前的实现方式是在 nanosecond 精度时返回整数,而不是 ms 和 us 返回的 datetime 类型,应用开发者需要自行处理,建议使用 pandas 的 to_datetime()。未来如果 Python 正式完整支持了纳秒,涛思数据可能会修改相关接口。
......
......@@ -54,23 +54,25 @@ INSERT INTO test.t1 USING test.weather (ts, temperature) TAGS('beijing') VALUES(
## JDBC driver version and supported TDengine and JDK versions
| taos-jdbcdriver | TDengine | JDK |
| --------------- | ------------------ | ----- |
| 2.0.33 - 2.0.34 | 2.0.3.0 and above | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 and above | 1.8.x |
| 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.3 | 1.6.1.x and above | 1.8.x |
| 1.0.2 | 1.6.1.x and above | 1.8.x |
| 1.0.1 | 1.6.1.x and above | 1.8.x |
| taos-jdbcdriver | TDengine | JDK |
| --------------- |--------------------|--------|
| 2.0.36 | 2.4.0 and above | 1.8.x |
| 2.0.35 | 2.3.0 and above | 1.8.x |
| 2.0.33 - 2.0.34 | 2.0.3.0 and above | 1.8.x |
| 2.0.31 - 2.0.32 | 2.1.3.0 and above | 1.8.x |
| 2.0.22 - 2.0.30 | 2.0.18.0 - 2.1.2.x | 1.8.x |
| 2.0.12 - 2.0.21 | 2.0.8.0 - 2.0.17.x | 1.8.x |
| 2.0.4 - 2.0.11 | 2.0.0.0 - 2.0.7.x | 1.8.x |
| 1.0.3 | 1.6.1.x and above | 1.8.x |
| 1.0.2 | 1.6.1.x and above | 1.8.x |
| 1.0.1 | 1.6.1.x and above | 1.8.x |
## DataType in TDengine and Java connector
The TDengine supports the following data types and Java data types:
| TDengine DataType | JDBCType (driver version < 2.0.24) | JDBCType (driver version >= 2.0.24) |
| ----------------- | ---------------------------------- | ----------------------------------- |
|-------------------|------------------------------------| ----------------------------------- |
| TIMESTAMP | java.lang.Long | java.sql.Timestamp |
| INT | java.lang.Integer | java.lang.Integer |
| BIGINT | java.lang.Long | java.lang.Long |
......@@ -81,7 +83,8 @@ The TDengine supports the following data types and Java data types:
| BOOL | java.lang.Boolean | java.lang.Boolean |
| BINARY | java.lang.String | byte array |
| NCHAR | java.lang.String | java.lang.String |
| JSON | - | java.lang.String |
**Note**: JSON type can only be used in tag.
## Install Java connector
### Runtime Requirements
......
......@@ -575,6 +575,49 @@ Close connection.
conn.close()
```
#### JSON Type Support
Python connector `taospy` starts supporting JSON type as tags since `v2.2.0` (requires TDengine beta v2.3.5+, or stable v2.4.0+).
Create stable and table with JSON tag.
```python
# encoding:UTF-8
import taos
conn = taos.connect()
conn.execute("create database if not exists py_test_json_type")
conn.execute("use py_test_json_type")
conn.execute("create stable s1 (ts timestamp, v1 int) tags (info json)")
conn.execute("create table s1_1 using s1 tags ('{\"k1\": \"v1\"}')")
```
Query JSON tag and table name from a stable.
```python
tags = conn.query("select info, tbname from s1").fetch_all_into_dict()
tags
```
The `tags` value is:
```python
[{'info': '{"k1":"v1"}', 'tbname': 's1_1'}]
```
To get value from JSON tag by key:
```python
k1 = conn.query("select info->'k1' as k1 from s1").fetch_all_into_dict()
"""
>>> k1
[{'k1': '"v1"'}]
"""
```
Refer to [JSON type instructions](https://www.taosdata.com/en/documentation/taos-sql) for more usage of JSON type.
#### Using nanosecond in Python connector
So far Python still does not completely support nanosecond type. Please refer to the link 1 and 2. The implementation of the python connector is to return an integer number for nanosecond value rather than datatime type as what ms and us do. The developer needs to handle it themselves. We recommend using pandas to_datetime() function. If Python officially support nanosecond in the future, TAOS Data might be possible to change the interface accordingly, which mean the application need change too.
......
......@@ -360,7 +360,7 @@ int32_t tsParseOneColumn(SSchema *pSchema, SStrToken *pToken, char *payload, cha
return tscInvalidOperationMsg(msg, "json tag length too long", pToken->z);
}
if (pToken->type == TK_NULL) {
*(int8_t *)payload = TSDB_DATA_TINYINT_NULL;
*(int8_t *)payload = TSDB_DATA_JSON_PLACEHOLDER;
} else if (pToken->type != TK_STRING){
tscInvalidOperationMsg(msg, "invalid json data", pToken->z);
} else{
......@@ -1063,7 +1063,7 @@ static int32_t tscCheckIfCreateTable(char **sqlstr, SSqlObj *pSql, char** boundC
sToken.n -= 2;
}
char tagVal[TSDB_MAX_TAGS_LEN];
char tagVal[TSDB_MAX_TAGS_LEN] = {0};
code = tsParseOneColumn(pSchema, &sToken, tagVal, pInsertParam->msg, &sql, false, tinfo.precision);
if (code != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder);
......
......@@ -6871,7 +6871,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
}
SKVRowBuilder kvRowBuilder = {0};
if (pTagsSchema->type == TSDB_DATA_TYPE_JSON) {
if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY) {
if (pItem->pVar.nType != TSDB_DATA_TYPE_BINARY && pItem->pVar.nType != TSDB_DATA_TYPE_NULL) {
tscError("json type error, should be string");
return invalidOperationMsg(pMsg, msg25);
}
......@@ -8714,7 +8714,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
}
tVariantListItem* pItem = taosArrayGet(pValList, 0);
if(pItem->pVar.nType != TSDB_DATA_TYPE_BINARY){
if(pItem->pVar.nType != TSDB_DATA_TYPE_BINARY && pItem->pVar.nType != TSDB_DATA_TYPE_NULL){
tscError("json type error, should be string");
tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg7);
......@@ -8724,6 +8724,7 @@ int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo) {
tdDestroyKVRowBuilder(&kvRowBuilder);
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
ret = parseJsontoTagData(pItem->pVar.pz, &kvRowBuilder, tscGetErrorMsgPayload(pCmd), pTagSchema[0].colId);
if (ret != TSDB_CODE_SUCCESS) {
tdDestroyKVRowBuilder(&kvRowBuilder);
......
......@@ -5459,7 +5459,7 @@ int parseJsontoTagData(char* json, SKVRowBuilder* kvRowBuilder, char* errMsg, in
varDataSetLen(nullTypeVal + CHAR_BYTES, INT_BYTES);
*(uint32_t*)(varDataVal(nullTypeKey)) = jsonNULL;
tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeKey, false); // add json null type
if (strtrim(json) == 0 || strcasecmp(json, "null") == 0){
if (!json || strtrim(json) == 0 || strcasecmp(json, "null") == 0){
*(uint32_t*)(varDataVal(nullTypeVal + CHAR_BYTES)) = jsonNULL;
tdAddColToKVRow(kvRowBuilder, jsonIndex++, TSDB_DATA_TYPE_NCHAR, nullTypeVal, true); // add json null value
return TSDB_CODE_SUCCESS;
......
......@@ -953,7 +953,7 @@ int32_t tVariantDumpEx(tVariant *pVariant, char *payload, int16_t type, bool inc
break;
}
case TSDB_DATA_TYPE_JSON:{
if (pVariant->nType == TSDB_DATA_TYPE_BINARY){
if (pVariant->nType == TSDB_DATA_TYPE_BINARY || pVariant->nType == TSDB_DATA_TYPE_NULL){
*((int8_t *)payload) = TSDB_DATA_JSON_PLACEHOLDER;
} else if (pVariant->nType == TSDB_DATA_TYPE_JSON){ // select * from stable, set tag type to json,from setTagValue/tag_project_function
memcpy(payload, pVariant->pz, pVariant->nLen);
......
......@@ -22,9 +22,9 @@ import json
class TDTestCase:
def caseDescription(self):
'''
Json tag test case, include create table with json tag,
select json tag and query with json tag in where condition,
besides, include json tag in group by/order by/join/subquery.
Json tag test case, include create table with json tag, select json tag and query with json tag in where condition, besides, include json tag in group by/order by/join/subquery.
case1: [TD-12452] fix error if json tag is NULL
case2: [TD-12389] describe child table, tag length error if the tag is json tag
'''
return
......@@ -515,6 +515,23 @@ class TDTestCase:
tdSql.query("select jtag->'tag3' from jsons1_16")
tdSql.checkData(0, 0, '-2.111000000')
# test TD-12452
tdSql.execute("ALTER TABLE jsons1_1 SET TAG jtag=NULL")
tdSql.query("select jtag from jsons1_1")
tdSql.checkData(0, 0, None)
tdSql.execute("CREATE TABLE if not exists jsons1_20 using jsons1 tags(NULL)")
tdSql.query("select jtag from jsons1_20")
tdSql.checkData(0, 0, None)
tdSql.execute("insert into jsons1_21 using jsons1 tags(NULL) values(1591061628000, 11, false, '你就会','')")
tdSql.query("select jtag from jsons1_21")
tdSql.checkData(0, 0, None)
#test TD-12389
tdSql.query("describe jsons1")
tdSql.checkData(5, 2, 4096)
tdSql.query("describe jsons1_1")
tdSql.checkData(5, 2, 4096)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
python3 ./test.py -f 1-insert/batchInsert.py
python3 test.py -f 1-insert/TD-11970.py
python3 test.py -f 1-insert/TD-11970.py
\ No newline at end of file
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册