Skip to content

T
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1193
Star 22018
Fork 4786
T
TDengine
提交 b5dac67b 编写于 作者: wmmhello's avatar wmmhello
浏览文件
操作

Merge branch 'develop' into fix/TD-5578

上级 67201b16 e45b047e
展开全部 隐藏空白更改
内联 并排
Showing with 8704 addition and 1769 deletion
Jenkinsfile
浏览文件 @ b5dac67b
...@@ -41,6 +41,7 @@ def pre_test(){ ...@@ -41,6 +41,7 @@ def pre_test(){
sh ''' sh '''
killall -9 taosd ||echo "no taosd running" killall -9 taosd ||echo "no taosd running"
killall -9 gdb || echo "no gdb running" killall -9 gdb || echo "no gdb running"
killall -9 python3.8 || echo "no python program running"
cd ${WKC} cd ${WKC}
git reset --hard HEAD~10 >/dev/null git reset --hard HEAD~10 >/dev/null
''' '''
...@@ -223,24 +224,27 @@ pipeline { ...@@ -223,24 +224,27 @@ pipeline {
steps { steps {
pre_test() pre_test()
catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') { catchError(buildResult: 'SUCCESS', stageResult: 'FAILURE') {
sh ''' timeout(time: 60, unit: 'MINUTES'){
cd ${WKC}/tests/pytest sh '''
./crash_gen.sh -a -p -t 4 -s 2000 cd ${WKC}/tests/pytest
''' ./crash_gen.sh -a -p -t 4 -s 2000
'''
}
}
timeout(time: 60, unit: 'MINUTES'){
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
rm -rf /var/log/taos/*
./handle_crash_gen_val_log.sh
'''
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
rm -rf /var/log/taos/*
./handle_taosd_val_log.sh
'''
} }
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
rm -rf /var/log/taos/*
./handle_crash_gen_val_log.sh
'''
sh '''
cd ${WKC}/tests/pytest
rm -rf /var/lib/taos/*
rm -rf /var/log/taos/*
./handle_taosd_val_log.sh
'''
timeout(time: 45, unit: 'MINUTES'){ timeout(time: 45, unit: 'MINUTES'){
sh ''' sh '''
date date
......
documentation20/cn/11.administrator/docs.md
浏览文件 @ b5dac67b
...@@ -143,7 +143,7 @@ taosd -C ...@@ -143,7 +143,7 @@ taosd -C
TDengine集群中加入一个新的dnode时，涉及集群相关的一些参数必须与已有集群的配置相同，否则不能成功加入到集群中。会进行校验的参数如下： TDengine集群中加入一个新的dnode时，涉及集群相关的一些参数必须与已有集群的配置相同，否则不能成功加入到集群中。会进行校验的参数如下：
- numOfMnodes：系统中管理节点个数。默认值：3。 - numOfMnodes：系统中管理节点个数。默认值：3。（2.0 版本从 2.0.20.11 开始、2.1 及以上版本从 2.1.6.0 开始，numOfMnodes 默认值改为 1。）
- mnodeEqualVnodeNum: 一个mnode等同于vnode消耗的个数。默认值：4。 - mnodeEqualVnodeNum: 一个mnode等同于vnode消耗的个数。默认值：4。
- offlineThreshold: dnode离线阈值，超过该时间将导致该dnode从集群中删除。单位为秒，默认值：86400*10（即10天）。 - offlineThreshold: dnode离线阈值，超过该时间将导致该dnode从集群中删除。单位为秒，默认值：86400*10（即10天）。
- statusInterval: dnode向mnode报告状态时长。单位为秒，默认值：1。 - statusInterval: dnode向mnode报告状态时长。单位为秒，默认值：1。
......
documentation20/cn/12.taos-sql/docs.md
浏览文件 @ b5dac67b
...@@ -34,7 +34,7 @@ taos> DESCRIBE meters; ...@@ -34,7 +34,7 @@ taos> DESCRIBE meters;
- 时间格式为 ```YYYY-MM-DD HH:mm:ss.MS```，默认时间分辨率为毫秒。比如：```2017-08-12 18:25:58.128``` - 时间格式为 ```YYYY-MM-DD HH:mm:ss.MS```，默认时间分辨率为毫秒。比如：```2017-08-12 18:25:58.128```
- 内部函数 now 是客户端的当前时间 - 内部函数 now 是客户端的当前时间
- 插入记录时，如果时间戳为 now，插入数据时使用提交这条记录的客户端的当前时间 - 插入记录时，如果时间戳为 now，插入数据时使用提交这条记录的客户端的当前时间
- Epoch Time：时间戳也可以是一个长整数，表示从 1970-01-01 08:00:00.000 开始的毫秒数 - Epoch Time：时间戳也可以是一个长整数，表示从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的毫秒数（相应地，如果所在 Database 的时间精度设置为“微秒”，则长整型格式的时间戳含义也就对应于从格林威治时间 1970-01-01 00:00:00.000 (UTC/GMT) 开始的微秒数）
- 时间可以加减，比如 now-2h，表明查询时刻向前推 2 个小时（最近 2 小时）。数字后面的时间单位可以是 u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。 比如 `select * from t1 where ts > now-2w and ts <= now-1w`，表示查询两周前整整一周的数据。在指定降频操作（down sampling）的时间窗口（interval）时，时间单位还可以使用 n(自然月) 和 y(自然年)。 - 时间可以加减，比如 now-2h，表明查询时刻向前推 2 个小时（最近 2 小时）。数字后面的时间单位可以是 u(微秒)、a(毫秒)、s(秒)、m(分)、h(小时)、d(天)、w(周)。 比如 `select * from t1 where ts > now-2w and ts <= now-1w`，表示查询两周前整整一周的数据。在指定降频操作（down sampling）的时间窗口（interval）时，时间单位还可以使用 n(自然月) 和 y(自然年)。
TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传递的 PRECISION 参数就可以支持微秒。 TDengine 缺省的时间戳是毫秒精度，但通过在 CREATE DATABASE 时传递的 PRECISION 参数就可以支持微秒。
......
documentation20/en/11.administrator/docs.md
浏览文件 @ b5dac67b
...@@ -132,7 +132,7 @@ The SQL creates a database demo, each data file stores 10 days of data, the memo ...@@ -132,7 +132,7 @@ The SQL creates a database demo, each data file stores 10 days of data, the memo
When adding a new dnode to the TDengine cluster, some parameters related to the cluster must be the same as the configuration of the existing cluster, otherwise it cannot be successfully added to the cluster. The parameters that will be verified are as follows: When adding a new dnode to the TDengine cluster, some parameters related to the cluster must be the same as the configuration of the existing cluster, otherwise it cannot be successfully added to the cluster. The parameters that will be verified are as follows:
- numOfMnodes: the number of management nodes in the system. Default: 3. - numOfMnodes: the number of management nodes in the system. Default: 3. (Since version 2.0.20.11 and version 2.1.6.0, the default value of "numOfMnodes" has been changed to 1.)
- balance: whether to enable load balancing. 0: No, 1: Yes. Default: 1. - balance: whether to enable load balancing. 0: No, 1: Yes. Default: 1.
- mnodeEqualVnodeNum: an mnode is equal to the number of vnodes consumed. Default: 4. - mnodeEqualVnodeNum: an mnode is equal to the number of vnodes consumed. Default: 4.
- offlineThreshold: the threshold for a dnode to be offline, exceed which the dnode will be removed from the cluster. The unit is seconds, and the default value is 86400*10 (that is, 10 days). - offlineThreshold: the threshold for a dnode to be offline, exceed which the dnode will be removed from the cluster. The unit is seconds, and the default value is 86400*10 (that is, 10 days).
......
src/client/inc/tscUtil.h
浏览文件 @ b5dac67b
...@@ -340,7 +340,7 @@ STableMeta* createSuperTableMeta(STableMetaMsg* pChild); ...@@ -340,7 +340,7 @@ STableMeta* createSuperTableMeta(STableMetaMsg* pChild);
uint32_t tscGetTableMetaSize(STableMeta* pTableMeta); uint32_t tscGetTableMetaSize(STableMeta* pTableMeta);
CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta); CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta);
uint32_t tscGetTableMetaMaxSize(); uint32_t tscGetTableMetaMaxSize();
int32_t tscCreateTableMetaFromSTableMeta(STableMeta* pChild, const char* name, void* buf); int32_t tscCreateTableMetaFromSTableMeta(STableMeta** pChild, const char* name, size_t *tableMetaCapacity);
STableMeta* tscTableMetaDup(STableMeta* pTableMeta); STableMeta* tscTableMetaDup(STableMeta* pTableMeta);
SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo); SVgroupsInfo* tscVgroupsInfoDup(SVgroupsInfo* pVgroupsInfo);
......
src/client/src/tscParseInsert.c
浏览文件 @ b5dac67b
...@@ -84,8 +84,6 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1 ...@@ -84,8 +84,6 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1
int64_t useconds = 0; int64_t useconds = 0;
char * pTokenEnd = *next; char * pTokenEnd = *next;
index = 0;
if (pToken->type == TK_NOW) { if (pToken->type == TK_NOW) {
useconds = taosGetTimestamp(timePrec); useconds = taosGetTimestamp(timePrec);
} else if (strncmp(pToken->z, "0", 1) == 0 && pToken->n == 1) { } else if (strncmp(pToken->z, "0", 1) == 0 && pToken->n == 1) {
...@@ -130,7 +128,8 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1 ...@@ -130,7 +128,8 @@ int tsParseTime(SStrToken *pToken, int64_t *time, char **next, char *error, int1
return tscInvalidOperationMsg(error, "value expected in timestamp", sToken.z); return tscInvalidOperationMsg(error, "value expected in timestamp", sToken.z);
} }
if (parseAbsoluteDuration(valueToken.z, valueToken.n, &interval, timePrec) != TSDB_CODE_SUCCESS) { char unit = 0;
if (parseAbsoluteDuration(valueToken.z, valueToken.n, &interval, &unit, timePrec) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
......
src/client/src/tscParseLineProtocol.c
浏览文件 @ b5dac67b
...@@ -458,9 +458,9 @@ int32_t loadTableMeta(TAOS* taos, char* tableName, SSmlSTableSchema* schema, SSm ...@@ -458,9 +458,9 @@ int32_t loadTableMeta(TAOS* taos, char* tableName, SSmlSTableSchema* schema, SSm
schema->tagHash = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false); schema->tagHash = taosHashInit(8, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
schema->fieldHash = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false); schema->fieldHash = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
uint32_t size = tscGetTableMetaMaxSize(); size_t size = 0;
STableMeta* tableMeta = calloc(1, size); STableMeta* tableMeta = NULL;
taosHashGetClone(tscTableMetaMap, fullTableName, strlen(fullTableName), NULL, tableMeta); taosHashGetCloneExt(tscTableMetaMap, fullTableName, strlen(fullTableName), NULL, (void **)&tableMeta, &size);
tstrncpy(schema->sTableName, tableName, strlen(tableName)+1); tstrncpy(schema->sTableName, tableName, strlen(tableName)+1);
schema->precision = tableMeta->tableInfo.precision; schema->precision = tableMeta->tableInfo.precision;
...@@ -1417,7 +1417,7 @@ static bool isTimeStamp(char *pVal, uint16_t len, SMLTimeStampType *tsType) { ...@@ -1417,7 +1417,7 @@ static bool isTimeStamp(char *pVal, uint16_t len, SMLTimeStampType *tsType) {
return false; return false;
} }
static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str) { static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str, SSmlLinesInfo* info) {
errno = 0; errno = 0;
uint8_t type = pVal->type; uint8_t type = pVal->type;
int16_t length = pVal->length; int16_t length = pVal->length;
...@@ -1436,7 +1436,7 @@ static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str) { ...@@ -1436,7 +1436,7 @@ static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str) {
} }
if (errno == ERANGE) { if (errno == ERANGE) {
tscError("Converted number out of range"); tscError("SML:0x%"PRIx64" Convert number(%s) out of range", info->id, str);
return false; return false;
} }
...@@ -1518,7 +1518,7 @@ static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str) { ...@@ -1518,7 +1518,7 @@ static bool convertStrToNumber(TAOS_SML_KV *pVal, char*str) {
} }
//len does not include '\0' from value. //len does not include '\0' from value.
static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
uint16_t len) { uint16_t len, SSmlLinesInfo* info) {
if (len <= 0) { if (len <= 0) {
return false; return false;
} }
...@@ -1528,7 +1528,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1528,7 +1528,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_TINYINT; pVal->type = TSDB_DATA_TYPE_TINYINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 2] = '\0'; value[len - 2] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1537,7 +1537,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1537,7 +1537,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_UTINYINT; pVal->type = TSDB_DATA_TYPE_UTINYINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 2] = '\0'; value[len - 2] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1546,7 +1546,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1546,7 +1546,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_SMALLINT; pVal->type = TSDB_DATA_TYPE_SMALLINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1555,7 +1555,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1555,7 +1555,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_USMALLINT; pVal->type = TSDB_DATA_TYPE_USMALLINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1564,7 +1564,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1564,7 +1564,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_INT; pVal->type = TSDB_DATA_TYPE_INT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1573,7 +1573,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1573,7 +1573,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_UINT; pVal->type = TSDB_DATA_TYPE_UINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1582,7 +1582,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1582,7 +1582,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_BIGINT; pVal->type = TSDB_DATA_TYPE_BIGINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1591,7 +1591,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1591,7 +1591,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_UBIGINT; pVal->type = TSDB_DATA_TYPE_UBIGINT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidInteger(value) || !convertStrToNumber(pVal, value)) { if (!isValidInteger(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1601,7 +1601,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1601,7 +1601,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_FLOAT; pVal->type = TSDB_DATA_TYPE_FLOAT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidFloat(value) || !convertStrToNumber(pVal, value)) { if (!isValidFloat(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1610,7 +1610,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1610,7 +1610,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
pVal->type = TSDB_DATA_TYPE_DOUBLE; pVal->type = TSDB_DATA_TYPE_DOUBLE;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
value[len - 3] = '\0'; value[len - 3] = '\0';
if (!isValidFloat(value) || !convertStrToNumber(pVal, value)) { if (!isValidFloat(value) || !convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1646,7 +1646,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value, ...@@ -1646,7 +1646,7 @@ static bool convertSmlValueType(TAOS_SML_KV *pVal, char *value,
if (isValidInteger(value) || isValidFloat(value)) { if (isValidInteger(value) || isValidFloat(value)) {
pVal->type = TSDB_DATA_TYPE_FLOAT; pVal->type = TSDB_DATA_TYPE_FLOAT;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
if (!convertStrToNumber(pVal, value)) { if (!convertStrToNumber(pVal, value, info)) {
return false; return false;
} }
return true; return true;
...@@ -1702,7 +1702,7 @@ static int32_t getTimeStampValue(char *value, uint16_t len, ...@@ -1702,7 +1702,7 @@ static int32_t getTimeStampValue(char *value, uint16_t len,
} }
static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value, static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
uint16_t len) { uint16_t len, SSmlLinesInfo* info) {
int32_t ret; int32_t ret;
SMLTimeStampType type; SMLTimeStampType type;
int64_t tsVal; int64_t tsVal;
...@@ -1715,7 +1715,7 @@ static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value, ...@@ -1715,7 +1715,7 @@ static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
if (ret) { if (ret) {
return ret; return ret;
} }
tscDebug("Timestamp after conversion:%"PRId64, tsVal); tscDebug("SML:0x%"PRIx64"Timestamp after conversion:%"PRId64, info->id, tsVal);
pVal->type = TSDB_DATA_TYPE_TIMESTAMP; pVal->type = TSDB_DATA_TYPE_TIMESTAMP;
pVal->length = (int16_t)tDataTypes[pVal->type].bytes; pVal->length = (int16_t)tDataTypes[pVal->type].bytes;
...@@ -1724,7 +1724,7 @@ static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value, ...@@ -1724,7 +1724,7 @@ static int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index) { static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index, SSmlLinesInfo* info) {
const char *start, *cur; const char *start, *cur;
int32_t ret = TSDB_CODE_SUCCESS; int32_t ret = TSDB_CODE_SUCCESS;
int len = 0; int len = 0;
...@@ -1744,7 +1744,7 @@ static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index) { ...@@ -1744,7 +1744,7 @@ static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index) {
memcpy(value, start, len); memcpy(value, start, len);
} }
ret = convertSmlTimeStamp(*pTS, value, len); ret = convertSmlTimeStamp(*pTS, value, len, info);
if (ret) { if (ret) {
free(value); free(value);
free(*pTS); free(*pTS);
...@@ -1757,7 +1757,7 @@ static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index) { ...@@ -1757,7 +1757,7 @@ static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **index) {
return ret; return ret;
} }
static bool checkDuplicateKey(char *key, SHashObj *pHash) { static bool checkDuplicateKey(char *key, SHashObj *pHash, SSmlLinesInfo* info) {
char *val = NULL; char *val = NULL;
char *cur = key; char *cur = key;
char keyLower[TSDB_COL_NAME_LEN]; char keyLower[TSDB_COL_NAME_LEN];
...@@ -1771,7 +1771,7 @@ static bool checkDuplicateKey(char *key, SHashObj *pHash) { ...@@ -1771,7 +1771,7 @@ static bool checkDuplicateKey(char *key, SHashObj *pHash) {
val = taosHashGet(pHash, keyLower, keyLen); val = taosHashGet(pHash, keyLower, keyLen);
if (val) { if (val) {
tscError("Duplicate key:%s", keyLower); tscError("SML:0x%"PRIx64" Duplicate key detected:%s", info->id, keyLower);
return true; return true;
} }
...@@ -1781,19 +1781,19 @@ static bool checkDuplicateKey(char *key, SHashObj *pHash) { ...@@ -1781,19 +1781,19 @@ static bool checkDuplicateKey(char *key, SHashObj *pHash) {
return false; return false;
} }
static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **index, SHashObj *pHash) { static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **index, SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *index; const char *cur = *index;
char key[TSDB_COL_NAME_LEN + 1]; // +1 to avoid key[len] over write char key[TSDB_COL_NAME_LEN + 1]; // +1 to avoid key[len] over write
uint16_t len = 0; uint16_t len = 0;
//key field cannot start with digit //key field cannot start with digit
if (isdigit(*cur)) { if (isdigit(*cur)) {
tscError("Tag key cannnot start with digit\n"); tscError("SML:0x%"PRIx64" Tag key cannnot start with digit", info->id);
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR; return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
} }
while (*cur != '\0') { while (*cur != '\0') {
if (len > TSDB_COL_NAME_LEN) { if (len > TSDB_COL_NAME_LEN) {
tscDebug("Key field cannot exceeds 65 characters"); tscError("SML:0x%"PRIx64" Key field cannot exceeds 65 characters", info->id);
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR; return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
} }
//unescaped '=' identifies a tag key //unescaped '=' identifies a tag key
...@@ -1810,20 +1810,20 @@ static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **index, SHashObj *pHash ...@@ -1810,20 +1810,20 @@ static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **index, SHashObj *pHash
} }
key[len] = '\0'; key[len] = '\0';
if (checkDuplicateKey(key, pHash)) { if (checkDuplicateKey(key, pHash, info)) {
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR; return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
} }
pKV->key = calloc(len + 1, 1); pKV->key = calloc(len + 1, 1);
memcpy(pKV->key, key, len + 1); memcpy(pKV->key, key, len + 1);
//tscDebug("Key:%s|len:%d", pKV->key, len); //tscDebug("SML:0x%"PRIx64" Key:%s|len:%d", info->id, pKV->key, len);
*index = cur + 1; *index = cur + 1;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index, static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index,
bool *is_last_kv) { bool *is_last_kv, SSmlLinesInfo* info) {
const char *start, *cur; const char *start, *cur;
char *value = NULL; char *value = NULL;
uint16_t len = 0; uint16_t len = 0;
...@@ -1847,7 +1847,9 @@ static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index, ...@@ -1847,7 +1847,9 @@ static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index,
value = calloc(len + 1, 1); value = calloc(len + 1, 1);
memcpy(value, start, len); memcpy(value, start, len);
value[len] = '\0'; value[len] = '\0';
if (!convertSmlValueType(pKV, value, len)) { if (!convertSmlValueType(pKV, value, len, info)) {
tscError("SML:0x%"PRIx64" Failed to convert sml value string(%s) to any type",
info->id, value);
//free previous alocated key field //free previous alocated key field
free(pKV->key); free(pKV->key);
pKV->key = NULL; pKV->key = NULL;
...@@ -1861,7 +1863,7 @@ static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index, ...@@ -1861,7 +1863,7 @@ static bool parseSmlValue(TAOS_SML_KV *pKV, const char **index,
} }
static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index, static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index,
uint8_t *has_tags) { uint8_t *has_tags, SSmlLinesInfo* info) {
const char *cur = *index; const char *cur = *index;
uint16_t len = 0; uint16_t len = 0;
...@@ -1870,7 +1872,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index ...@@ -1870,7 +1872,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index
return TSDB_CODE_TSC_OUT_OF_MEMORY; return TSDB_CODE_TSC_OUT_OF_MEMORY;
} }
if (isdigit(*cur)) { if (isdigit(*cur)) {
tscError("Measurement field cannnot start with digit"); tscError("SML:0x%"PRIx64" Measurement field cannnot start with digit", info->id);
free(pSml->stableName); free(pSml->stableName);
pSml->stableName = NULL; pSml->stableName = NULL;
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR; return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
...@@ -1878,7 +1880,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index ...@@ -1878,7 +1880,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index
while (*cur != '\0') { while (*cur != '\0') {
if (len > TSDB_TABLE_NAME_LEN) { if (len > TSDB_TABLE_NAME_LEN) {
tscError("Measurement field cannot exceeds 193 characters"); tscError("SML:0x%"PRIx64" Measurement field cannot exceeds 193 characters", info->id);
free(pSml->stableName); free(pSml->stableName);
pSml->stableName = NULL; pSml->stableName = NULL;
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR; return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
...@@ -1902,7 +1904,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index ...@@ -1902,7 +1904,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **index
} }
pSml->stableName[len] = '\0'; pSml->stableName[len] = '\0';
*index = cur + 1; *index = cur + 1;
tscDebug("Stable name in measurement:%s|len:%d", pSml->stableName, len); tscDebug("SML:0x%"PRIx64" Stable name in measurement:%s|len:%d", info->id, pSml->stableName, len);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
...@@ -1921,7 +1923,8 @@ static int32_t isValidChildTableName(const char *pTbName, int16_t len) { ...@@ -1921,7 +1923,8 @@ static int32_t isValidChildTableName(const char *pTbName, int16_t len) {
static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs, static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
const char **index, bool isField, const char **index, bool isField,
TAOS_SML_DATA_POINT* smlData, SHashObj *pHash) { TAOS_SML_DATA_POINT* smlData, SHashObj *pHash,
SSmlLinesInfo* info) {
const char *cur = *index; const char *cur = *index;
int32_t ret = TSDB_CODE_SUCCESS; int32_t ret = TSDB_CODE_SUCCESS;
TAOS_SML_KV *pkv; TAOS_SML_KV *pkv;
...@@ -1941,14 +1944,14 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs, ...@@ -1941,14 +1944,14 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
} }
while (*cur != '\0') { while (*cur != '\0') {
ret = parseSmlKey(pkv, &cur, pHash); ret = parseSmlKey(pkv, &cur, pHash, info);
if (ret) { if (ret) {
tscError("Unable to parse key field"); tscError("SML:0x%"PRIx64" Unable to parse key", info->id);
goto error; goto error;
} }
ret = parseSmlValue(pkv, &cur, &is_last_kv); ret = parseSmlValue(pkv, &cur, &is_last_kv, info);
if (ret) { if (ret) {
tscError("Unable to parse value field"); tscError("SML:0x%"PRIx64" Unable to parse value", info->id);
goto error; goto error;
} }
if (!isField && if (!isField &&
...@@ -1966,7 +1969,6 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs, ...@@ -1966,7 +1969,6 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
*num_kvs += 1; *num_kvs += 1;
} }
if (is_last_kv) { if (is_last_kv) {
//tscDebug("last key-value field detected");
goto done; goto done;
} }
...@@ -2024,50 +2026,50 @@ static void moveTimeStampToFirstKv(TAOS_SML_DATA_POINT** smlData, TAOS_SML_KV *t ...@@ -2024,50 +2026,50 @@ static void moveTimeStampToFirstKv(TAOS_SML_DATA_POINT** smlData, TAOS_SML_KV *t
free(ts); free(ts);
} }
int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData) { int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
const char* index = sql; const char* index = sql;
int32_t ret = TSDB_CODE_SUCCESS; int32_t ret = TSDB_CODE_SUCCESS;
uint8_t has_tags = 0; uint8_t has_tags = 0;
TAOS_SML_KV *timestamp = NULL; TAOS_SML_KV *timestamp = NULL;
SHashObj *keyHashTable = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false); SHashObj *keyHashTable = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
ret = parseSmlMeasurement(smlData, &index, &has_tags); ret = parseSmlMeasurement(smlData, &index, &has_tags, info);
if (ret) { if (ret) {
tscError("Unable to parse measurement"); tscError("SML:0x%"PRIx64" Unable to parse measurement", info->id);
taosHashCleanup(keyHashTable); taosHashCleanup(keyHashTable);
return ret; return ret;
} }
tscDebug("Parse measurement finished, has_tags:%d", has_tags); tscDebug("SML:0x%"PRIx64" Parse measurement finished, has_tags:%d", info->id, has_tags);
//Parse Tags //Parse Tags
if (has_tags) { if (has_tags) {
ret = parseSmlKvPairs(&smlData->tags, &smlData->tagNum, &index, false, smlData, keyHashTable); ret = parseSmlKvPairs(&smlData->tags, &smlData->tagNum, &index, false, smlData, keyHashTable, info);
if (ret) { if (ret) {
tscError("Unable to parse tag"); tscError("SML:0x%"PRIx64" Unable to parse tag", info->id);
taosHashCleanup(keyHashTable); taosHashCleanup(keyHashTable);
return ret; return ret;
} }
} }
tscDebug("Parse tags finished, num of tags:%d", smlData->tagNum); tscDebug("SML:0x%"PRIx64" Parse tags finished, num of tags:%d", info->id, smlData->tagNum);
//Parse fields //Parse fields
ret = parseSmlKvPairs(&smlData->fields, &smlData->fieldNum, &index, true, smlData, keyHashTable); ret = parseSmlKvPairs(&smlData->fields, &smlData->fieldNum, &index, true, smlData, keyHashTable, info);
if (ret) { if (ret) {
tscError("Unable to parse field"); tscError("SML:0x%"PRIx64" Unable to parse field", info->id);
taosHashCleanup(keyHashTable); taosHashCleanup(keyHashTable);
return ret; return ret;
} }
tscDebug("Parse fields finished, num of fields:%d", smlData->fieldNum); tscDebug("SML:0x%"PRIx64" Parse fields finished, num of fields:%d", info->id, smlData->fieldNum);
taosHashCleanup(keyHashTable); taosHashCleanup(keyHashTable);
//Parse timestamp //Parse timestamp
ret = parseSmlTimeStamp(&timestamp, &index); ret = parseSmlTimeStamp(&timestamp, &index, info);
if (ret) { if (ret) {
tscError("Unable to parse timestamp"); tscError("SML:0x%"PRIx64" Unable to parse timestamp", info->id);
return ret; return ret;
} }
moveTimeStampToFirstKv(&smlData, timestamp); moveTimeStampToFirstKv(&smlData, timestamp);
tscDebug("Parse timestamp finished"); tscDebug("SML:0x%"PRIx64" Parse timestamp finished", info->id);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
...@@ -2104,7 +2106,7 @@ void destroySmlDataPoint(TAOS_SML_DATA_POINT* point) { ...@@ -2104,7 +2106,7 @@ void destroySmlDataPoint(TAOS_SML_DATA_POINT* point) {
int32_t tscParseLines(char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) { int32_t tscParseLines(char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
for (int32_t i = 0; i < numLines; ++i) { for (int32_t i = 0; i < numLines; ++i) {
TAOS_SML_DATA_POINT point = {0}; TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseLine(lines[i], &point); int32_t code = tscParseLine(lines[i], &point, info);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
tscError("SML:0x%"PRIx64" data point line parse failed. line %d : %s", info->id, i, lines[i]); tscError("SML:0x%"PRIx64" data point line parse failed. line %d : %s", info->id, i, lines[i]);
destroySmlDataPoint(&point); destroySmlDataPoint(&point);
......
src/client/src/tscSQLParser.c
浏览文件 @ b5dac67b
...@@ -900,6 +900,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -900,6 +900,7 @@ int32_t tscValidateSqlInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SSqlNode* pSqlNode = taosArrayGetP(pInfo->list, i); SSqlNode* pSqlNode = taosArrayGetP(pInfo->list, i);
tscTrace("0x%"PRIx64" start to parse the %dth subclause, total:%"PRIzu, pSql->self, i, size); tscTrace("0x%"PRIx64" start to parse the %dth subclause, total:%"PRIzu, pSql->self, i, size);
// normalizeSqlNode(pSqlNode); // normalize the column name in each function // normalizeSqlNode(pSqlNode); // normalize the column name in each function
if ((code = validateSqlNode(pSql, pSqlNode, pQueryInfo)) != TSDB_CODE_SUCCESS) { if ((code = validateSqlNode(pSql, pSqlNode, pQueryInfo)) != TSDB_CODE_SUCCESS) {
return code; return code;
...@@ -1288,35 +1289,31 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* pSl ...@@ -1288,35 +1289,31 @@ int32_t parseSlidingClause(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SStrToken* pSl
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
SInterval* pInterval = &pQueryInfo->interval;
if (pSliding->n == 0) { if (pSliding->n == 0) {
pQueryInfo->interval.slidingUnit = pQueryInfo->interval.intervalUnit; pInterval->slidingUnit = pInterval->intervalUnit;
pQueryInfo->interval.sliding = pQueryInfo->interval.interval; pInterval->sliding = pInterval->interval;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
if (pQueryInfo->interval.intervalUnit == 'n' || pQueryInfo->interval.intervalUnit == 'y') { if (pInterval->intervalUnit == 'n' || pInterval->intervalUnit == 'y') {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
} }
parseAbsoluteDuration(pSliding->z, pSliding->n, &pQueryInfo->interval.sliding, tinfo.precision); parseAbsoluteDuration(pSliding->z, pSliding->n, &pInterval->sliding, &pInterval->slidingUnit, tinfo.precision);
if (pQueryInfo->interval.sliding < if (pInterval->sliding < convertTimePrecision(tsMinSlidingTime, TSDB_TIME_PRECISION_MILLI, tinfo.precision)) {
convertTimePrecision(tsMinSlidingTime, TSDB_TIME_PRECISION_MILLI, tinfo.precision)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg0);
} }
if (pQueryInfo->interval.sliding > pQueryInfo->interval.interval) { if (pInterval->sliding > pInterval->interval) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg1);
} }
if ((pQueryInfo->interval.interval != 0) && (pQueryInfo->interval.interval/pQueryInfo->interval.sliding > INTERVAL_SLIDING_FACTOR)) { if ((pInterval->interval != 0) && (pInterval->interval/pInterval->sliding > INTERVAL_SLIDING_FACTOR)) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2); return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
} }
// if (pQueryInfo->interval.sliding != pQueryInfo->interval.interval && pSql->pStream == NULL) {
// return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg4);
// }
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
...@@ -2028,7 +2025,6 @@ static SUdfInfo* isValidUdf(SArray* pUdfInfo, const char* name, int32_t len) { ...@@ -2028,7 +2025,6 @@ static SUdfInfo* isValidUdf(SArray* pUdfInfo, const char* name, int32_t len) {
tscError("udfinfo is null"); tscError("udfinfo is null");
return NULL; return NULL;
} }
size_t t = taosArrayGetSize(pUdfInfo); size_t t = taosArrayGetSize(pUdfInfo);
for(int32_t i = 0; i < t; ++i) { for(int32_t i = 0; i < t; ++i) {
SUdfInfo* pUdf = taosArrayGet(pUdfInfo, i); SUdfInfo* pUdf = taosArrayGet(pUdfInfo, i);
...@@ -6080,10 +6076,12 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -6080,10 +6076,12 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SSchema* pSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta); SSchema* pSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta);
int16_t numOfTags = tscGetNumOfTags(pTableMetaInfo->pTableMeta); int16_t numOfTags = tscGetNumOfTags(pTableMetaInfo->pTableMeta);
int16_t i; int32_t numOfCols = tscGetNumOfColumns(pTableMetaInfo->pTableMeta);
int32_t tagIndex = columnIndex.columnIndex - numOfCols;
assert(tagIndex>=0);
uint32_t nLen = 0; uint32_t nLen = 0;
for (i = 0; i < numOfTags; ++i) { for (int i = 0; i < numOfTags; ++i) {
nLen += (i != columnIndex.columnIndex) ? pSchema[i].bytes : pItem->bytes; nLen += (i != tagIndex) ? pSchema[i].bytes : pItem->bytes;
} }
if (nLen >= TSDB_MAX_TAGS_LEN) { if (nLen >= TSDB_MAX_TAGS_LEN) {
return invalidOperationMsg(pMsg, msg24); return invalidOperationMsg(pMsg, msg24);
...@@ -8093,6 +8091,7 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -8093,6 +8091,7 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SArray* pVgroupList = NULL; SArray* pVgroupList = NULL;
SArray* plist = NULL; SArray* plist = NULL;
STableMeta* pTableMeta = NULL; STableMeta* pTableMeta = NULL;
size_t tableMetaCapacity = 0;
SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd); SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd);
pCmd->pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK); pCmd->pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_NO_LOCK);
...@@ -8119,18 +8118,14 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -8119,18 +8118,14 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) {
} }
} }
uint32_t maxSize = tscGetTableMetaMaxSize();
char name[TSDB_TABLE_FNAME_LEN] = {0}; char name[TSDB_TABLE_FNAME_LEN] = {0};
assert(maxSize < 80 * TSDB_MAX_COLUMNS); //if (!pSql->pBuf) {
if (!pSql->pBuf) { // if (NULL == (pSql->pBuf = tcalloc(1, 80 * TSDB_MAX_COLUMNS))) {
if (NULL == (pSql->pBuf = tcalloc(1, 80 * TSDB_MAX_COLUMNS))) { // code = TSDB_CODE_TSC_OUT_OF_MEMORY;
code = TSDB_CODE_TSC_OUT_OF_MEMORY; // goto _end;
goto _end; // }
} //}
}
pTableMeta = calloc(1, maxSize);
plist = taosArrayInit(4, POINTER_BYTES); plist = taosArrayInit(4, POINTER_BYTES);
pVgroupList = taosArrayInit(4, POINTER_BYTES); pVgroupList = taosArrayInit(4, POINTER_BYTES);
...@@ -8144,16 +8139,16 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) { ...@@ -8144,16 +8139,16 @@ int32_t loadAllTableMeta(SSqlObj* pSql, struct SSqlInfo* pInfo) {
tNameExtractFullName(pname, name); tNameExtractFullName(pname, name);
size_t len = strlen(name); size_t len = strlen(name);
memset(pTableMeta, 0, maxSize);
taosHashGetClone(tscTableMetaMap, name, len, NULL, pTableMeta); taosHashGetCloneExt(tscTableMetaMap, name, len, NULL, (void **)&pTableMeta, &tableMetaCapacity);
if (pTableMeta->id.uid > 0) { if (pTableMeta && pTableMeta->id.uid > 0) {
tscDebug("0x%"PRIx64" retrieve table meta %s from local buf", pSql->self, name); tscDebug("0x%"PRIx64" retrieve table meta %s from local buf", pSql->self, name);
// avoid mem leak, may should update pTableMeta // avoid mem leak, may should update pTableMeta
void* pVgroupIdList = NULL; void* pVgroupIdList = NULL;
if (pTableMeta->tableType == TSDB_CHILD_TABLE) { if (pTableMeta->tableType == TSDB_CHILD_TABLE) {
code = tscCreateTableMetaFromSTableMeta(pTableMeta, name, pSql->pBuf); code = tscCreateTableMetaFromSTableMeta((STableMeta **)(&pTableMeta), name, &tableMetaCapacity);
// create the child table meta from super table failed, try load it from mnode // create the child table meta from super table failed, try load it from mnode
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
...@@ -8308,7 +8303,8 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod ...@@ -8308,7 +8303,8 @@ static int32_t doLoadAllTableMeta(SSqlObj* pSql, SQueryInfo* pQueryInfo, SSqlNod
tNameExtractFullName(&pTableMetaInfo->name, fname); tNameExtractFullName(&pTableMetaInfo->name, fname);
STableMetaVgroupInfo* p = taosHashGet(pCmd->pTableMetaMap, fname, strnlen(fname, TSDB_TABLE_FNAME_LEN)); STableMetaVgroupInfo* p = taosHashGet(pCmd->pTableMetaMap, fname, strnlen(fname, TSDB_TABLE_FNAME_LEN));
pTableMetaInfo->pTableMeta = tscTableMetaDup(p->pTableMeta); pTableMetaInfo->pTableMeta = tscTableMetaDup(p->pTableMeta);
pTableMetaInfo->tableMetaCapacity = tscGetTableMetaSize(pTableMetaInfo->pTableMeta);
assert(pTableMetaInfo->pTableMeta != NULL); assert(pTableMetaInfo->pTableMeta != NULL);
if (p->vgroupIdList != NULL) { if (p->vgroupIdList != NULL) {
...@@ -8408,7 +8404,8 @@ static int32_t doValidateSubquery(SSqlNode* pSqlNode, int32_t index, SSqlObj* pS ...@@ -8408,7 +8404,8 @@ static int32_t doValidateSubquery(SSqlNode* pSqlNode, int32_t index, SSqlObj* pS
if (pTableMetaInfo1 == NULL) { if (pTableMetaInfo1 == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY; return TSDB_CODE_TSC_OUT_OF_MEMORY;
} }
pTableMetaInfo1->pTableMeta = extractTempTableMetaFromSubquery(pSub); pTableMetaInfo1->pTableMeta = extractTempTableMetaFromSubquery(pSub);
pTableMetaInfo1->tableMetaCapacity = tscGetTableMetaSize(pTableMetaInfo1->pTableMeta);
if (subInfo->aliasName.n > 0) { if (subInfo->aliasName.n > 0) {
if (subInfo->aliasName.n >= TSDB_TABLE_FNAME_LEN) { if (subInfo->aliasName.n >= TSDB_TABLE_FNAME_LEN) {
...@@ -8457,6 +8454,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -8457,6 +8454,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
const char* msg6 = "not support stddev/percentile/interp in the outer query yet"; const char* msg6 = "not support stddev/percentile/interp in the outer query yet";
const char* msg7 = "derivative/twa/irate requires timestamp column exists in subquery"; const char* msg7 = "derivative/twa/irate requires timestamp column exists in subquery";
const char* msg8 = "condition missing for join query"; const char* msg8 = "condition missing for join query";
const char* msg9 = "not support 3 level select";
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
...@@ -8487,6 +8485,13 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -8487,6 +8485,13 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
// parse the subquery in the first place // parse the subquery in the first place
int32_t numOfSub = (int32_t)taosArrayGetSize(pSqlNode->from->list); int32_t numOfSub = (int32_t)taosArrayGetSize(pSqlNode->from->list);
for (int32_t i = 0; i < numOfSub; ++i) { for (int32_t i = 0; i < numOfSub; ++i) {
// check if there is 3 level select
SRelElementPair* subInfo = taosArrayGet(pSqlNode->from->list, i);
SSqlNode* p = taosArrayGetP(subInfo->pSubquery, 0);
if (p->from->type == SQL_NODE_FROM_SUBQUERY){
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg9);
}
code = doValidateSubquery(pSqlNode, i, pSql, pQueryInfo, tscGetErrorMsgPayload(pCmd)); code = doValidateSubquery(pSqlNode, i, pSql, pQueryInfo, tscGetErrorMsgPayload(pCmd));
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return code; return code;
...@@ -8767,8 +8772,8 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf ...@@ -8767,8 +8772,8 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
#if 0 #if 0
SQueryNode* p = qCreateQueryPlan(pQueryInfo); SQueryNode* p = qCreateQueryPlan(pQueryInfo);
char* s = queryPlanToString(p); char* s = queryPlanToString(p);
printf("%s\n", s);
tfree(s); tfree(s);
qDestroyQueryPlan(p); qDestroyQueryPlan(p);
#endif #endif
......
src/client/src/tscServer.c
浏览文件 @ b5dac67b
...@@ -401,10 +401,8 @@ static void doProcessMsgFromServer(SSchedMsg* pSchedMsg) { ...@@ -401,10 +401,8 @@ static void doProcessMsgFromServer(SSchedMsg* pSchedMsg) {
// single table query error need to be handled here. // single table query error need to be handled here.
if ((cmd == TSDB_SQL_SELECT || cmd == TSDB_SQL_UPDATE_TAGS_VAL) && if ((cmd == TSDB_SQL_SELECT || cmd == TSDB_SQL_UPDATE_TAGS_VAL) &&
(((rpcMsg->code == TSDB_CODE_TDB_INVALID_TABLE_ID || (((rpcMsg->code == TSDB_CODE_TDB_INVALID_TABLE_ID || rpcMsg->code == TSDB_CODE_VND_INVALID_VGROUP_ID)) ||
rpcMsg->code == TSDB_CODE_VND_INVALID_VGROUP_ID)) || rpcMsg->code == TSDB_CODE_RPC_NETWORK_UNAVAIL || rpcMsg->code == TSDB_CODE_APP_NOT_READY)) {
rpcMsg->code == TSDB_CODE_RPC_NETWORK_UNAVAIL ||
rpcMsg->code == TSDB_CODE_APP_NOT_READY)) {
// 1. super table subquery // 1. super table subquery
// 2. nest queries are all not updated the tablemeta and retry parse the sql after cleanup local tablemeta/vgroup id buffer // 2. nest queries are all not updated the tablemeta and retry parse the sql after cleanup local tablemeta/vgroup id buffer
...@@ -2840,43 +2838,22 @@ int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVg ...@@ -2840,43 +2838,22 @@ int32_t getMultiTableMetaFromMnode(SSqlObj *pSql, SArray* pNameList, SArray* pVg
int32_t tscGetTableMetaImpl(SSqlObj* pSql, STableMetaInfo *pTableMetaInfo, bool autocreate, bool onlyLocal) { int32_t tscGetTableMetaImpl(SSqlObj* pSql, STableMetaInfo *pTableMetaInfo, bool autocreate, bool onlyLocal) {
assert(tIsValidName(&pTableMetaInfo->name)); assert(tIsValidName(&pTableMetaInfo->name));
uint32_t size = tscGetTableMetaMaxSize();
if (pTableMetaInfo->pTableMeta == NULL) {
pTableMetaInfo->pTableMeta = calloc(1, size);
pTableMetaInfo->tableMetaSize = size;
} else if (pTableMetaInfo->tableMetaSize < size) {
char *tmp = realloc(pTableMetaInfo->pTableMeta, size);
if (tmp == NULL) {
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
pTableMetaInfo->pTableMeta = (STableMeta *)tmp;
}
memset(pTableMetaInfo->pTableMeta, 0, size);
pTableMetaInfo->tableMetaSize = size;
pTableMetaInfo->pTableMeta->tableType = -1;
pTableMetaInfo->pTableMeta->tableInfo.numOfColumns = -1;
char name[TSDB_TABLE_FNAME_LEN] = {0}; char name[TSDB_TABLE_FNAME_LEN] = {0};
tNameExtractFullName(&pTableMetaInfo->name, name); tNameExtractFullName(&pTableMetaInfo->name, name);
size_t len = strlen(name); size_t len = strlen(name);
taosHashGetClone(tscTableMetaMap, name, len, NULL, pTableMetaInfo->pTableMeta); if (pTableMetaInfo->tableMetaCapacity != 0) {
if (pTableMetaInfo->pTableMeta != NULL) {
// TODO resize the tableMeta memset(pTableMetaInfo->pTableMeta, 0, pTableMetaInfo->tableMetaCapacity);
assert(size < 80 * TSDB_MAX_COLUMNS); }
if (!pSql->pBuf) { }
if (NULL == (pSql->pBuf = tcalloc(1, 80 * TSDB_MAX_COLUMNS))) { taosHashGetCloneExt(tscTableMetaMap, name, len, NULL, (void **)&(pTableMetaInfo->pTableMeta), &pTableMetaInfo->tableMetaCapacity);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
}
STableMeta* pMeta = pTableMetaInfo->pTableMeta; STableMeta* pMeta = pTableMetaInfo->pTableMeta;
if (pMeta->id.uid > 0) { if (pMeta && pMeta->id.uid > 0) {
// in case of child table, here only get the // in case of child table, here only get the
if (pMeta->tableType == TSDB_CHILD_TABLE) { if (pMeta->tableType == TSDB_CHILD_TABLE) {
int32_t code = tscCreateTableMetaFromSTableMeta(pTableMetaInfo->pTableMeta, name, pSql->pBuf); int32_t code = tscCreateTableMetaFromSTableMeta(&pTableMetaInfo->pTableMeta, name, &pTableMetaInfo->tableMetaCapacity);
if (code != TSDB_CODE_SUCCESS) { if (code != TSDB_CODE_SUCCESS) {
return getTableMetaFromMnode(pSql, pTableMetaInfo, autocreate); return getTableMetaFromMnode(pSql, pTableMetaInfo, autocreate);
} }
......
src/client/src/tscSubquery.c
浏览文件 @ b5dac67b
...@@ -1635,6 +1635,8 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) { ...@@ -1635,6 +1635,8 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
continue; continue;
} }
SSqlRes* pRes1 = &pSql1->res; SSqlRes* pRes1 = &pSql1->res;
if (pRes1->row >= pRes1->numOfRows) { if (pRes1->row >= pRes1->numOfRows) {
subquerySetState(pSql1, &pSql->subState, i, 0); subquerySetState(pSql1, &pSql->subState, i, 0);
......
src/client/src/tscUtil.c
浏览文件 @ b5dac67b
...@@ -1276,10 +1276,10 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) { ...@@ -1276,10 +1276,10 @@ static void tscDestroyResPointerInfo(SSqlRes* pRes) {
for (int i = 0; i < pRes->numOfCols; i++) { for (int i = 0; i < pRes->numOfCols; i++) {
tfree(pRes->buffer[i]); tfree(pRes->buffer[i]);
} }
pRes->numOfCols = 0; pRes->numOfCols = 0;
} }
tfree(pRes->pRsp); tfree(pRes->pRsp);
tfree(pRes->tsrow); tfree(pRes->tsrow);
...@@ -1657,6 +1657,7 @@ int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) { ...@@ -1657,6 +1657,7 @@ int32_t tscCopyDataBlockToPayload(SSqlObj* pSql, STableDataBlocks* pDataBlock) {
pTableMetaInfo->pTableMeta = tscTableMetaDup(pDataBlock->pTableMeta); pTableMetaInfo->pTableMeta = tscTableMetaDup(pDataBlock->pTableMeta);
pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pDataBlock->pTableMeta); pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pDataBlock->pTableMeta);
pTableMetaInfo->tableMetaCapacity = (size_t)(pTableMetaInfo->tableMetaSize);
} }
/* /*
...@@ -1719,7 +1720,7 @@ int32_t tscCreateDataBlock(size_t defaultSize, int32_t rowSize, int32_t startOff ...@@ -1719,7 +1720,7 @@ int32_t tscCreateDataBlock(size_t defaultSize, int32_t rowSize, int32_t startOff
if (dataBuf->nAllocSize <= dataBuf->headerSize) { if (dataBuf->nAllocSize <= dataBuf->headerSize) {
dataBuf->nAllocSize = dataBuf->headerSize * 2; dataBuf->nAllocSize = dataBuf->headerSize * 2;
} }
//dataBuf->pData = calloc(1, dataBuf->nAllocSize); //dataBuf->pData = calloc(1, dataBuf->nAllocSize);
dataBuf->pData = malloc(dataBuf->nAllocSize); dataBuf->pData = malloc(dataBuf->nAllocSize);
if (dataBuf->pData == NULL) { if (dataBuf->pData == NULL) {
...@@ -1835,7 +1836,7 @@ static SMemRow tdGenMemRowFromBuilder(SMemRowBuilder* pBuilder) { ...@@ -1835,7 +1836,7 @@ static SMemRow tdGenMemRowFromBuilder(SMemRowBuilder* pBuilder) {
toffset += TYPE_BYTES[pSchema[j].type]; toffset += TYPE_BYTES[pSchema[j].type];
++j; ++j;
} }
#if 0 // no need anymore #if 0 // no need anymore
while (i < nColsBound) { while (i < nColsBound) {
p = payloadNextCol(p); p = payloadNextCol(p);
...@@ -2005,7 +2006,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl ...@@ -2005,7 +2006,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl
// the maximum expanded size in byte when a row-wise data is converted to SDataRow format // the maximum expanded size in byte when a row-wise data is converted to SDataRow format
int32_t expandSize = getRowExpandSize(pOneTableBlock->pTableMeta); int32_t expandSize = getRowExpandSize(pOneTableBlock->pTableMeta);
STableDataBlocks* dataBuf = NULL; STableDataBlocks* dataBuf = NULL;
int32_t ret = tscGetDataBlockFromList(pVnodeDataBlockHashList, pOneTableBlock->vgId, TSDB_PAYLOAD_SIZE, int32_t ret = tscGetDataBlockFromList(pVnodeDataBlockHashList, pOneTableBlock->vgId, TSDB_PAYLOAD_SIZE,
INSERT_HEAD_SIZE, 0, &pOneTableBlock->tableName, pOneTableBlock->pTableMeta, &dataBuf, pVnodeDataBlockList); INSERT_HEAD_SIZE, 0, &pOneTableBlock->tableName, pOneTableBlock->pTableMeta, &dataBuf, pVnodeDataBlockList);
if (ret != TSDB_CODE_SUCCESS) { if (ret != TSDB_CODE_SUCCESS) {
...@@ -2059,7 +2060,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl ...@@ -2059,7 +2060,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl
pInsertParam->objectId, tNameGetTableName(&pOneTableBlock->tableName), pBlocks->tid, pInsertParam->objectId, tNameGetTableName(&pOneTableBlock->tableName), pBlocks->tid,
pBlocks->numOfRows, pBlocks->sversion, blkKeyInfo.pKeyTuple->skey, pLastKeyTuple->skey); pBlocks->numOfRows, pBlocks->sversion, blkKeyInfo.pKeyTuple->skey, pLastKeyTuple->skey);
} }
int32_t len = pBlocks->numOfRows * (pOneTableBlock->rowSize + expandSize) + sizeof(STColumn) * tscGetNumOfColumns(pOneTableBlock->pTableMeta); int32_t len = pBlocks->numOfRows * (pOneTableBlock->rowSize + expandSize) + sizeof(STColumn) * tscGetNumOfColumns(pOneTableBlock->pTableMeta);
pBlocks->tid = htonl(pBlocks->tid); pBlocks->tid = htonl(pBlocks->tid);
...@@ -2083,7 +2084,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl ...@@ -2083,7 +2084,7 @@ int32_t tscMergeTableDataBlocks(SInsertStatementParam *pInsertParam, bool freeBl
}else { }else {
tscDebug("0x%"PRIx64" table %s data block is empty", pInsertParam->objectId, pOneTableBlock->tableName.tname); tscDebug("0x%"PRIx64" table %s data block is empty", pInsertParam->objectId, pOneTableBlock->tableName.tname);
} }
p = taosHashIterate(pInsertParam->pTableBlockHashList, p); p = taosHashIterate(pInsertParam->pTableBlockHashList, p);
if (p == NULL) { if (p == NULL) {
break; break;
...@@ -2147,7 +2148,7 @@ int tscAllocPayload(SSqlCmd* pCmd, int size) { ...@@ -2147,7 +2148,7 @@ int tscAllocPayload(SSqlCmd* pCmd, int size) {
pCmd->payload = b; pCmd->payload = b;
pCmd->allocSize = size; pCmd->allocSize = size;
} }
memset(pCmd->payload, 0, pCmd->allocSize); memset(pCmd->payload, 0, pCmd->allocSize);
} }
...@@ -2164,7 +2165,7 @@ TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes) { ...@@ -2164,7 +2165,7 @@ TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes) {
SInternalField* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField) { SInternalField* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField) {
assert(pFieldInfo != NULL); assert(pFieldInfo != NULL);
pFieldInfo->numOfOutput++; pFieldInfo->numOfOutput++;
struct SInternalField info = { .pExpr = NULL, .visible = true }; struct SInternalField info = { .pExpr = NULL, .visible = true };
info.field = *pField; info.field = *pField;
...@@ -2256,13 +2257,13 @@ int32_t tscGetResRowLength(SArray* pExprList) { ...@@ -2256,13 +2257,13 @@ int32_t tscGetResRowLength(SArray* pExprList) {
if (num == 0) { if (num == 0) {
return 0; return 0;
} }
int32_t size = 0; int32_t size = 0;
for(int32_t i = 0; i < num; ++i) { for(int32_t i = 0; i < num; ++i) {
SExprInfo* pExpr = taosArrayGetP(pExprList, i); SExprInfo* pExpr = taosArrayGetP(pExprList, i);
size += pExpr->base.resBytes; size += pExpr->base.resBytes;
} }
return size; return size;
} }
...@@ -2402,7 +2403,7 @@ SExprInfo* tscExprCreate(STableMetaInfo* pTableMetaInfo, int16_t functionId, SCo ...@@ -2402,7 +2403,7 @@ SExprInfo* tscExprCreate(STableMetaInfo* pTableMetaInfo, int16_t functionId, SCo
snprintf(p->colInfo.name, len, "%s.%s", pTableMetaInfo->aliasName, pSchema->name); snprintf(p->colInfo.name, len, "%s.%s", pTableMetaInfo->aliasName, pSchema->name);
} }
} }
p->colInfo.flag = colType; p->colInfo.flag = colType;
p->colInfo.colIndex = pColIndex->columnIndex; p->colInfo.colIndex = pColIndex->columnIndex;
...@@ -2414,7 +2415,7 @@ SExprInfo* tscExprCreate(STableMetaInfo* pTableMetaInfo, int16_t functionId, SCo ...@@ -2414,7 +2415,7 @@ SExprInfo* tscExprCreate(STableMetaInfo* pTableMetaInfo, int16_t functionId, SCo
if (pTableMetaInfo->pTableMeta) { if (pTableMetaInfo->pTableMeta) {
p->uid = pTableMetaInfo->pTableMeta->id.uid; p->uid = pTableMetaInfo->pTableMeta->id.uid;
} }
return pExpr; return pExpr;
} }
...@@ -2500,18 +2501,18 @@ SExprInfo* tscExprGet(SQueryInfo* pQueryInfo, int32_t index) { ...@@ -2500,18 +2501,18 @@ SExprInfo* tscExprGet(SQueryInfo* pQueryInfo, int32_t index) {
*/ */
void tscExprDestroy(SArray* pExprInfo) { void tscExprDestroy(SArray* pExprInfo) {
size_t size = taosArrayGetSize(pExprInfo); size_t size = taosArrayGetSize(pExprInfo);
for(int32_t i = 0; i < size; ++i) { for(int32_t i = 0; i < size; ++i) {
SExprInfo* pExpr = taosArrayGetP(pExprInfo, i); SExprInfo* pExpr = taosArrayGetP(pExprInfo, i);
sqlExprDestroy(pExpr); sqlExprDestroy(pExpr);
} }
taosArrayDestroy(pExprInfo); taosArrayDestroy(pExprInfo);
} }
int32_t tscExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy) { int32_t tscExprCopy(SArray* dst, const SArray* src, uint64_t uid, bool deepcopy) {
assert(src != NULL && dst != NULL); assert(src != NULL && dst != NULL);
size_t size = taosArrayGetSize(src); size_t size = taosArrayGetSize(src);
for (int32_t i = 0; i < size; ++i) { for (int32_t i = 0; i < size; ++i) {
SExprInfo* pExpr = taosArrayGetP(src, i); SExprInfo* pExpr = taosArrayGetP(src, i);
...@@ -2596,7 +2597,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t ...@@ -2596,7 +2597,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t
if (columnIndex < 0) { if (columnIndex < 0) {
return NULL; return NULL;
} }
size_t numOfCols = taosArrayGetSize(pColumnList); size_t numOfCols = taosArrayGetSize(pColumnList);
int32_t i = 0; int32_t i = 0;
...@@ -2626,7 +2627,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t ...@@ -2626,7 +2627,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t
taosArrayInsert(pColumnList, i, &b); taosArrayInsert(pColumnList, i, &b);
} else { } else {
SColumn* pCol = taosArrayGetP(pColumnList, i); SColumn* pCol = taosArrayGetP(pColumnList, i);
if (i < numOfCols && (pCol->columnIndex > columnIndex || pCol->tableUid != uid)) { if (i < numOfCols && (pCol->columnIndex > columnIndex || pCol->tableUid != uid)) {
SColumn* b = calloc(1, sizeof(SColumn)); SColumn* b = calloc(1, sizeof(SColumn));
if (b == NULL) { if (b == NULL) {
...@@ -2650,7 +2651,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t ...@@ -2650,7 +2651,7 @@ SColumn* tscColumnListInsert(SArray* pColumnList, int32_t columnIndex, uint64_t
SColumn* tscColumnClone(const SColumn* src) { SColumn* tscColumnClone(const SColumn* src) {
assert(src != NULL); assert(src != NULL);
SColumn* dst = calloc(1, sizeof(SColumn)); SColumn* dst = calloc(1, sizeof(SColumn));
if (dst == NULL) { if (dst == NULL) {
return NULL; return NULL;
...@@ -2679,7 +2680,7 @@ void tscColumnCopy(SColumn* pDest, const SColumn* pSrc) { ...@@ -2679,7 +2680,7 @@ void tscColumnCopy(SColumn* pDest, const SColumn* pSrc) {
void tscColumnListCopy(SArray* dst, const SArray* src, uint64_t tableUid) { void tscColumnListCopy(SArray* dst, const SArray* src, uint64_t tableUid) {
assert(src != NULL && dst != NULL); assert(src != NULL && dst != NULL);
size_t num = taosArrayGetSize(src); size_t num = taosArrayGetSize(src);
for (int32_t i = 0; i < num; ++i) { for (int32_t i = 0; i < num; ++i) {
SColumn* pCol = taosArrayGetP(src, i); SColumn* pCol = taosArrayGetP(src, i);
...@@ -2790,18 +2791,19 @@ void tscDequoteAndTrimToken(SStrToken* pToken) { ...@@ -2790,18 +2791,19 @@ void tscDequoteAndTrimToken(SStrToken* pToken) {
} }
int32_t tscValidateName(SStrToken* pToken) { int32_t tscValidateName(SStrToken* pToken) {
if (pToken->type != TK_STRING && pToken->type != TK_ID) { if (pToken == NULL || pToken->z == NULL ||
(pToken->type != TK_STRING && pToken->type != TK_ID)) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
char* sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true); char* sep = strnchr(pToken->z, TS_PATH_DELIMITER[0], pToken->n, true);
if (sep == NULL) { // single part if (sep == NULL) { // single part
if (pToken->type == TK_STRING) { if (pToken->type == TK_STRING) {
tscDequoteAndTrimToken(pToken); tscDequoteAndTrimToken(pToken);
tscStrToLower(pToken->z, pToken->n); tscStrToLower(pToken->z, pToken->n);
//pToken->n = (uint32_t)strtrim(pToken->z); //pToken->n = (uint32_t)strtrim(pToken->z);
int len = tGetToken(pToken->z, &pToken->type); int len = tGetToken(pToken->z, &pToken->type);
// single token, validate it // single token, validate it
...@@ -2853,7 +2855,7 @@ int32_t tscValidateName(SStrToken* pToken) { ...@@ -2853,7 +2855,7 @@ int32_t tscValidateName(SStrToken* pToken) {
if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) { if (pToken->type == TK_STRING && validateQuoteToken(pToken) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION; return TSDB_CODE_TSC_INVALID_OPERATION;
} }
// re-build the whole name string // re-build the whole name string
if (pStr[firstPartLen] == TS_PATH_DELIMITER[0]) { if (pStr[firstPartLen] == TS_PATH_DELIMITER[0]) {
// first part do not have quote do nothing // first part do not have quote do nothing
...@@ -2892,7 +2894,7 @@ bool tscValidateColumnId(STableMetaInfo* pTableMetaInfo, int32_t colId, int32_t ...@@ -2892,7 +2894,7 @@ bool tscValidateColumnId(STableMetaInfo* pTableMetaInfo, int32_t colId, int32_t
SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta); SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta); STableComInfo tinfo = tscGetTableInfo(pTableMetaInfo->pTableMeta);
int32_t numOfTotal = tinfo.numOfTags + tinfo.numOfColumns; int32_t numOfTotal = tinfo.numOfTags + tinfo.numOfColumns;
for (int32_t i = 0; i < numOfTotal; ++i) { for (int32_t i = 0; i < numOfTotal; ++i) {
...@@ -2937,21 +2939,21 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) { ...@@ -2937,21 +2939,21 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
dest->relType = src->relType; dest->relType = src->relType;
if (src->pCond == NULL) { if (src->pCond == NULL) {
return 0; return 0;
} }
size_t s = taosArrayGetSize(src->pCond); size_t s = taosArrayGetSize(src->pCond);
dest->pCond = taosArrayInit(s, sizeof(SCond)); dest->pCond = taosArrayInit(s, sizeof(SCond));
for (int32_t i = 0; i < s; ++i) { for (int32_t i = 0; i < s; ++i) {
SCond* pCond = taosArrayGet(src->pCond, i); SCond* pCond = taosArrayGet(src->pCond, i);
SCond c = {0}; SCond c = {0};
c.len = pCond->len; c.len = pCond->len;
c.uid = pCond->uid; c.uid = pCond->uid;
if (pCond->len > 0) { if (pCond->len > 0) {
assert(pCond->cond != NULL); assert(pCond->cond != NULL);
c.cond = malloc(c.len); c.cond = malloc(c.len);
...@@ -2961,7 +2963,7 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) { ...@@ -2961,7 +2963,7 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
memcpy(c.cond, pCond->cond, c.len); memcpy(c.cond, pCond->cond, c.len);
} }
taosArrayPush(dest->pCond, &c); taosArrayPush(dest->pCond, &c);
} }
...@@ -2970,14 +2972,14 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) { ...@@ -2970,14 +2972,14 @@ int32_t tscTagCondCopy(STagCond* dest, const STagCond* src) {
void tscTagCondRelease(STagCond* pTagCond) { void tscTagCondRelease(STagCond* pTagCond) {
free(pTagCond->tbnameCond.cond); free(pTagCond->tbnameCond.cond);
if (pTagCond->pCond != NULL) { if (pTagCond->pCond != NULL) {
size_t s = taosArrayGetSize(pTagCond->pCond); size_t s = taosArrayGetSize(pTagCond->pCond);
for (int32_t i = 0; i < s; ++i) { for (int32_t i = 0; i < s; ++i) {
SCond* p = taosArrayGet(pTagCond->pCond, i); SCond* p = taosArrayGet(pTagCond->pCond, i);
tfree(p->cond); tfree(p->cond);
} }
taosArrayDestroy(pTagCond->pCond); taosArrayDestroy(pTagCond->pCond);
} }
...@@ -3004,7 +3006,7 @@ void tscTagCondRelease(STagCond* pTagCond) { ...@@ -3004,7 +3006,7 @@ void tscTagCondRelease(STagCond* pTagCond) {
void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo) { void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta); SSchema* pSchema = tscGetTableSchema(pTableMetaInfo->pTableMeta);
size_t numOfExprs = tscNumOfExprs(pQueryInfo); size_t numOfExprs = tscNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < numOfExprs; ++i) { for (int32_t i = 0; i < numOfExprs; ++i) {
SExprInfo* pExpr = tscExprGet(pQueryInfo, i); SExprInfo* pExpr = tscExprGet(pQueryInfo, i);
...@@ -3012,7 +3014,7 @@ void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo) { ...@@ -3012,7 +3014,7 @@ void tscGetSrcColumnInfo(SSrcColumnInfo* pColInfo, SQueryInfo* pQueryInfo) {
if (TSDB_COL_IS_TAG(pExpr->base.colInfo.flag)) { if (TSDB_COL_IS_TAG(pExpr->base.colInfo.flag)) {
SSchema* pTagSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta); SSchema* pTagSchema = tscGetTableTagSchema(pTableMetaInfo->pTableMeta);
int16_t index = pExpr->base.colInfo.colIndex; int16_t index = pExpr->base.colInfo.colIndex;
pColInfo[i].type = (index != -1) ? pTagSchema[index].type : TSDB_DATA_TYPE_BINARY; pColInfo[i].type = (index != -1) ? pTagSchema[index].type : TSDB_DATA_TYPE_BINARY;
} else { } else {
...@@ -3037,7 +3039,7 @@ bool tscShouldBeFreed(SSqlObj* pSql) { ...@@ -3037,7 +3039,7 @@ bool tscShouldBeFreed(SSqlObj* pSql) {
if (pSql == NULL || pSql->signature != pSql) { if (pSql == NULL || pSql->signature != pSql) {
return false; return false;
} }
STscObj* pTscObj = pSql->pTscObj; STscObj* pTscObj = pSql->pTscObj;
if (pSql->pStream != NULL || pTscObj->hbrid == pSql->self || pSql->pSubscription != NULL) { if (pSql->pStream != NULL || pTscObj->hbrid == pSql->self || pSql->pSubscription != NULL) {
return false; return false;
...@@ -3116,7 +3118,7 @@ STableMetaInfo* tscGetTableMetaInfoByUid(SQueryInfo* pQueryInfo, uint64_t uid, i ...@@ -3116,7 +3118,7 @@ STableMetaInfo* tscGetTableMetaInfoByUid(SQueryInfo* pQueryInfo, uint64_t uid, i
void tscInitQueryInfo(SQueryInfo* pQueryInfo) { void tscInitQueryInfo(SQueryInfo* pQueryInfo) {
assert(pQueryInfo->fieldsInfo.internalField == NULL); assert(pQueryInfo->fieldsInfo.internalField == NULL);
pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField)); pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField));
assert(pQueryInfo->exprList == NULL); assert(pQueryInfo->exprList == NULL);
pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES);
...@@ -3178,7 +3180,7 @@ static void freeQueryInfoImpl(SQueryInfo* pQueryInfo) { ...@@ -3178,7 +3180,7 @@ static void freeQueryInfoImpl(SQueryInfo* pQueryInfo) {
pQueryInfo->groupbyExpr.columnInfo = NULL; pQueryInfo->groupbyExpr.columnInfo = NULL;
pQueryInfo->groupbyExpr.numOfGroupCols = 0; pQueryInfo->groupbyExpr.numOfGroupCols = 0;
} }
pQueryInfo->tsBuf = tsBufDestroy(pQueryInfo->tsBuf); pQueryInfo->tsBuf = tsBufDestroy(pQueryInfo->tsBuf);
pQueryInfo->fillType = 0; pQueryInfo->fillType = 0;
...@@ -3375,7 +3377,7 @@ void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta) { ...@@ -3375,7 +3377,7 @@ void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, bool removeMeta) {
tNameExtractFullName(&pTableMetaInfo->name, name); tNameExtractFullName(&pTableMetaInfo->name, name);
taosHashRemove(tscTableMetaMap, name, strnlen(name, TSDB_TABLE_FNAME_LEN)); taosHashRemove(tscTableMetaMap, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
} }
tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables); tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables);
tscClearTableMetaInfo(pTableMetaInfo); tscClearTableMetaInfo(pTableMetaInfo);
...@@ -3409,11 +3411,13 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM ...@@ -3409,11 +3411,13 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM
pTableMetaInfo->pTableMeta = pTableMeta; pTableMetaInfo->pTableMeta = pTableMeta;
if (pTableMetaInfo->pTableMeta == NULL) { if (pTableMetaInfo->pTableMeta == NULL) {
pTableMetaInfo->tableMetaSize = 0; pTableMetaInfo->tableMetaSize = 0;
} else { } else {
pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pTableMeta); pTableMetaInfo->tableMetaSize = tscGetTableMetaSize(pTableMeta);
} }
pTableMetaInfo->tableMetaCapacity = (size_t)(pTableMetaInfo->tableMetaSize);
if (vgroupList != NULL) { if (vgroupList != NULL) {
pTableMetaInfo->vgroupList = tscVgroupInfoClone(vgroupList); pTableMetaInfo->vgroupList = tscVgroupInfoClone(vgroupList);
} }
...@@ -3429,7 +3433,7 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM ...@@ -3429,7 +3433,7 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, SName* name, STableM
} }
pTableMetaInfo->pVgroupTables = tscVgroupTableInfoDup(pVgroupTables); pTableMetaInfo->pVgroupTables = tscVgroupTableInfoDup(pVgroupTables);
pQueryInfo->numOfTables += 1; pQueryInfo->numOfTables += 1;
return pTableMetaInfo; return pTableMetaInfo;
} }
...@@ -3637,7 +3641,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3637,7 +3641,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
goto _error; goto _error;
} }
} }
if (tscTagCondCopy(&pNewQueryInfo->tagCond, &pQueryInfo->tagCond) != 0) { if (tscTagCondCopy(&pNewQueryInfo->tagCond, &pQueryInfo->tagCond) != 0) {
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
goto _error; goto _error;
...@@ -3645,7 +3649,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3645,7 +3649,7 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
if (pQueryInfo->fillType != TSDB_FILL_NONE) { if (pQueryInfo->fillType != TSDB_FILL_NONE) {
//just make memory memory sanitizer happy //just make memory memory sanitizer happy
//refator later //refactor later
pNewQueryInfo->fillVal = calloc(1, pQueryInfo->fieldsInfo.numOfOutput * sizeof(int64_t)); pNewQueryInfo->fillVal = calloc(1, pQueryInfo->fieldsInfo.numOfOutput * sizeof(int64_t));
if (pNewQueryInfo->fillVal == NULL) { if (pNewQueryInfo->fillVal == NULL) {
terrno = TSDB_CODE_TSC_OUT_OF_MEMORY; terrno = TSDB_CODE_TSC_OUT_OF_MEMORY;
...@@ -3693,14 +3697,14 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3693,14 +3697,14 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pTableMeta, pTableMetaInfo->vgroupList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pTableMeta, pTableMetaInfo->vgroupList,
pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables); pTableMetaInfo->tagColList, pTableMetaInfo->pVgroupTables);
} else { // transfer the ownership of pTableMeta to the newly create sql object. } else { // transfer the ownership of pTableMeta to the newly create sql object.
STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, 0); STableMetaInfo* pPrevInfo = tscGetTableMetaInfoFromCmd(&pPrevSql->cmd, 0);
if (pPrevInfo->pTableMeta && pPrevInfo->pTableMeta->tableType < 0) { if (pPrevInfo->pTableMeta && pPrevInfo->pTableMeta->tableType < 0) {
terrno = TSDB_CODE_TSC_APP_ERROR; terrno = TSDB_CODE_TSC_APP_ERROR;
goto _error; goto _error;
} }
STableMeta* pPrevTableMeta = tscTableMetaDup(pPrevInfo->pTableMeta); STableMeta* pPrevTableMeta = tscTableMetaDup(pPrevInfo->pTableMeta);
SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList; SVgroupsInfo* pVgroupsInfo = pPrevInfo->vgroupList;
pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pPrevTableMeta, pVgroupsInfo, pTableMetaInfo->tagColList, pFinalInfo = tscAddTableMetaInfo(pNewQueryInfo, &pTableMetaInfo->name, pPrevTableMeta, pVgroupsInfo, pTableMetaInfo->tagColList,
...@@ -3720,9 +3724,9 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3720,9 +3724,9 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
goto _error; goto _error;
} }
assert(pNewQueryInfo->numOfTables == 1); assert(pNewQueryInfo->numOfTables == 1);
if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) { if (UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
assert(pFinalInfo->vgroupList != NULL); assert(pFinalInfo->vgroupList != NULL);
} }
...@@ -3731,13 +3735,13 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t ...@@ -3731,13 +3735,13 @@ SSqlObj* createSubqueryObj(SSqlObj* pSql, int16_t tableIndex, __async_cb_func_t
if (cmd == TSDB_SQL_SELECT) { if (cmd == TSDB_SQL_SELECT) {
size_t size = taosArrayGetSize(pNewQueryInfo->colList); size_t size = taosArrayGetSize(pNewQueryInfo->colList);
tscDebug("0x%"PRIx64" new subquery:0x%"PRIx64", tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu "," tscDebug("0x%"PRIx64" new subquery:0x%"PRIx64", tableIndex:%d, vgroupIndex:%d, type:%d, exprInfo:%" PRIzu ", colList:%" PRIzu ","
"fieldInfo:%d, name:%s, qrang:%" PRId64 " - %" PRId64 " order:%d, limit:%" PRId64, "fieldInfo:%d, name:%s, qrang:%" PRId64 " - %" PRId64 " order:%d, limit:%" PRId64,
pSql->self, pNew->self, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscNumOfExprs(pNewQueryInfo), pSql->self, pNew->self, tableIndex, pTableMetaInfo->vgroupIndex, pNewQueryInfo->type, tscNumOfExprs(pNewQueryInfo),
size, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pFinalInfo->name), pNewQueryInfo->window.skey, size, pNewQueryInfo->fieldsInfo.numOfOutput, tNameGetTableName(&pFinalInfo->name), pNewQueryInfo->window.skey,
pNewQueryInfo->window.ekey, pNewQueryInfo->order.order, pNewQueryInfo->limit.limit); pNewQueryInfo->window.ekey, pNewQueryInfo->order.order, pNewQueryInfo->limit.limit);
tscPrintSelNodeList(pNew, 0); tscPrintSelNodeList(pNew, 0);
} else { } else {
tscDebug("0x%"PRIx64" new sub insertion: %p, vnodeIdx:%d", pSql->self, pNew, pTableMetaInfo->vgroupIndex); tscDebug("0x%"PRIx64" new sub insertion: %p, vnodeIdx:%d", pSql->self, pNew, pTableMetaInfo->vgroupIndex);
...@@ -4045,7 +4049,7 @@ int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* s ...@@ -4045,7 +4049,7 @@ int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* s
const char* msgFormat2 = "syntax error near \'%s\' (%s)"; const char* msgFormat2 = "syntax error near \'%s\' (%s)";
const char* msgFormat3 = "%s"; const char* msgFormat3 = "%s";
const char* prefix = "syntax error"; const char* prefix = "syntax error";
const int32_t BACKWARD_CHAR_STEP = 0; const int32_t BACKWARD_CHAR_STEP = 0;
if (sql == NULL) { if (sql == NULL) {
...@@ -4060,7 +4064,7 @@ int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* s ...@@ -4060,7 +4064,7 @@ int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* s
if (additionalInfo != NULL) { if (additionalInfo != NULL) {
sprintf(msg, msgFormat2, buf, additionalInfo); sprintf(msg, msgFormat2, buf, additionalInfo);
} else { } else {
const char* msgFormat = (0 == strncmp(sql, prefix, strlen(prefix))) ? msgFormat3 : msgFormat1; const char* msgFormat = (0 == strncmp(sql, prefix, strlen(prefix))) ? msgFormat3 : msgFormat1;
sprintf(msg, msgFormat, buf); sprintf(msg, msgFormat, buf);
} }
...@@ -4118,7 +4122,7 @@ bool hasMoreVnodesToTry(SSqlObj* pSql) { ...@@ -4118,7 +4122,7 @@ bool hasMoreVnodesToTry(SSqlObj* pSql) {
if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo) || (pTableMetaInfo->vgroupList == NULL)) { if (!UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo) || (pTableMetaInfo->vgroupList == NULL)) {
return false; return false;
} }
int32_t numOfVgroups = pTableMetaInfo->vgroupList->numOfVgroups; int32_t numOfVgroups = pTableMetaInfo->vgroupList->numOfVgroups;
if (pTableMetaInfo->pVgroupTables != NULL) { if (pTableMetaInfo->pVgroupTables != NULL) {
numOfVgroups = (int32_t)taosArrayGetSize(pTableMetaInfo->pVgroupTables); numOfVgroups = (int32_t)taosArrayGetSize(pTableMetaInfo->pVgroupTables);
...@@ -4145,7 +4149,7 @@ void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) { ...@@ -4145,7 +4149,7 @@ void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) {
*/ */
assert(pRes->numOfRows == 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0) && !tscHasReachLimitation(pQueryInfo, pRes)); assert(pRes->numOfRows == 0 && tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0) && !tscHasReachLimitation(pQueryInfo, pRes));
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0); STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
int32_t totalVgroups = pTableMetaInfo->vgroupList->numOfVgroups; int32_t totalVgroups = pTableMetaInfo->vgroupList->numOfVgroups;
if (++pTableMetaInfo->vgroupIndex < totalVgroups) { if (++pTableMetaInfo->vgroupIndex < totalVgroups) {
tscDebug("0x%"PRIx64" results from vgroup index:%d completed, try next:%d. total vgroups:%d. current numOfRes:%" PRId64, pSql->self, tscDebug("0x%"PRIx64" results from vgroup index:%d completed, try next:%d. total vgroups:%d. current numOfRes:%" PRId64, pSql->self,
...@@ -4166,7 +4170,7 @@ void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) { ...@@ -4166,7 +4170,7 @@ void tscTryQueryNextVnode(SSqlObj* pSql, __async_cb_func_t fp) {
pQueryInfo->limit.offset = pRes->offset; pQueryInfo->limit.offset = pRes->offset;
assert((pRes->offset >= 0 && pRes->numOfRows == 0) || (pRes->offset == 0 && pRes->numOfRows >= 0)); assert((pRes->offset >= 0 && pRes->numOfRows == 0) || (pRes->offset == 0 && pRes->numOfRows >= 0));
tscDebug("0x%"PRIx64" new query to next vgroup, index:%d, limit:%" PRId64 ", offset:%" PRId64 ", glimit:%" PRId64, tscDebug("0x%"PRIx64" new query to next vgroup, index:%d, limit:%" PRId64 ", offset:%" PRId64 ", glimit:%" PRId64,
pSql->self, pTableMetaInfo->vgroupIndex, pQueryInfo->limit.limit, pQueryInfo->limit.offset, pQueryInfo->clauseLimit); pSql->self, pTableMetaInfo->vgroupIndex, pQueryInfo->limit.limit, pQueryInfo->limit.offset, pQueryInfo->clauseLimit);
...@@ -4251,7 +4255,7 @@ char* strdup_throw(const char* str) { ...@@ -4251,7 +4255,7 @@ char* strdup_throw(const char* str) {
int tscSetMgmtEpSetFromCfg(const char *first, const char *second, SRpcCorEpSet *corMgmtEpSet) { int tscSetMgmtEpSetFromCfg(const char *first, const char *second, SRpcCorEpSet *corMgmtEpSet) {
corMgmtEpSet->version = 0; corMgmtEpSet->version = 0;
// init mgmt ip set // init mgmt ip set
SRpcEpSet *mgmtEpSet = &(corMgmtEpSet->epSet); SRpcEpSet *mgmtEpSet = &(corMgmtEpSet->epSet);
mgmtEpSet->numOfEps = 0; mgmtEpSet->numOfEps = 0;
mgmtEpSet->inUse = 0; mgmtEpSet->inUse = 0;
...@@ -4426,7 +4430,7 @@ uint32_t tscGetTableMetaSize(STableMeta* pTableMeta) { ...@@ -4426,7 +4430,7 @@ uint32_t tscGetTableMetaSize(STableMeta* pTableMeta) {
if (pTableMeta->tableInfo.numOfColumns >= 0) { if (pTableMeta->tableInfo.numOfColumns >= 0) {
totalCols = pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags; totalCols = pTableMeta->tableInfo.numOfColumns + pTableMeta->tableInfo.numOfTags;
} }
return sizeof(STableMeta) + totalCols * sizeof(SSchema); return sizeof(STableMeta) + totalCols * sizeof(SSchema);
} }
...@@ -4444,24 +4448,36 @@ CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta) { ...@@ -4444,24 +4448,36 @@ CChildTableMeta* tscCreateChildMeta(STableMeta* pTableMeta) {
return cMeta; return cMeta;
} }
int32_t tscCreateTableMetaFromSTableMeta(STableMeta* pChild, const char* name, void* buf) { int32_t tscCreateTableMetaFromSTableMeta(STableMeta** ppChild, const char* name, size_t *tableMetaCapacity) {
assert(pChild != NULL && buf != NULL); assert(*ppChild != NULL);
STableMeta* p = buf; STableMeta* p = NULL;
taosHashGetClone(tscTableMetaMap, pChild->sTableName, strnlen(pChild->sTableName, TSDB_TABLE_FNAME_LEN), NULL, p); size_t sz = 0;
STableMeta* pChild = *ppChild;
taosHashGetCloneExt(tscTableMetaMap, pChild->sTableName, strnlen(pChild->sTableName, TSDB_TABLE_FNAME_LEN), NULL, (void **)&p, &sz);
// tableMeta exists, build child table meta according to the super table meta // tableMeta exists, build child table meta according to the super table meta
// the uid need to be checked in addition to the general name of the super table. // the uid need to be checked in addition to the general name of the super table.
if (p->id.uid > 0 && pChild->suid == p->id.uid) { if (p && p->id.uid > 0 && pChild->suid == p->id.uid) {
int32_t totalBytes = (p->tableInfo.numOfColumns + p->tableInfo.numOfTags) * sizeof(SSchema);
int32_t tableMetaSize = sizeof(STableMeta) + totalBytes;
if (*tableMetaCapacity < tableMetaSize) {
pChild = realloc(pChild, tableMetaSize);
*tableMetaCapacity = (size_t)tableMetaSize;
}
pChild->sversion = p->sversion; pChild->sversion = p->sversion;
pChild->tversion = p->tversion; pChild->tversion = p->tversion;
memcpy(&pChild->tableInfo, &p->tableInfo, sizeof(STableComInfo)); memcpy(&pChild->tableInfo, &p->tableInfo, sizeof(STableComInfo));
int32_t total = pChild->tableInfo.numOfColumns + pChild->tableInfo.numOfTags; memcpy(pChild->schema, p->schema, totalBytes);
memcpy(pChild->schema, p->schema, sizeof(SSchema) *total); *ppChild = pChild;
tfree(p);
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} else { // super table has been removed, current tableMeta is also expired. remove it here } else { // super table has been removed, current tableMeta is also expired. remove it here
tfree(p);
taosHashRemove(tscTableMetaMap, name, strnlen(name, TSDB_TABLE_FNAME_LEN)); taosHashRemove(tscTableMetaMap, name, strnlen(name, TSDB_TABLE_FNAME_LEN));
return -1; return -1;
} }
...@@ -4976,4 +4992,4 @@ void tscRemoveTableMetaBuf(STableMetaInfo* pTableMetaInfo, uint64_t id) { ...@@ -4976,4 +4992,4 @@ void tscRemoveTableMetaBuf(STableMetaInfo* pTableMetaInfo, uint64_t id) {
taosHashRemove(tscTableMetaMap, fname, len); taosHashRemove(tscTableMetaMap, fname, len);
tscDebug("0x%"PRIx64" remove table meta %s, numOfRemain:%d", id, fname, (int32_t) taosHashGetSize(tscTableMetaMap)); tscDebug("0x%"PRIx64" remove table meta %s, numOfRemain:%d", id, fname, (int32_t) taosHashGetSize(tscTableMetaMap));
} }
\ No newline at end of file
src/common/inc/tdataformat.h
浏览文件 @ b5dac67b
...@@ -325,7 +325,9 @@ typedef struct SDataCol { ...@@ -325,7 +325,9 @@ typedef struct SDataCol {
#define isAllRowsNull(pCol) ((pCol)->len == 0) #define isAllRowsNull(pCol) ((pCol)->len == 0)
static FORCE_INLINE void dataColReset(SDataCol *pDataCol) { pDataCol->len = 0; } static FORCE_INLINE void dataColReset(SDataCol *pDataCol) { pDataCol->len = 0; }
void dataColInit(SDataCol *pDataCol, STColumn *pCol, void **pBuf, int maxPoints); int tdAllocMemForCol(SDataCol *pCol, int maxPoints);
void dataColInit(SDataCol *pDataCol, STColumn *pCol, int maxPoints);
void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxPoints); void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxPoints);
void dataColSetOffset(SDataCol *pCol, int nEle); void dataColSetOffset(SDataCol *pCol, int nEle);
...@@ -358,12 +360,10 @@ typedef struct { ...@@ -358,12 +360,10 @@ typedef struct {
int maxRowSize; int maxRowSize;
int maxCols; // max number of columns int maxCols; // max number of columns
int maxPoints; // max number of points int maxPoints; // max number of points
int bufSize;
int numOfRows; int numOfRows;
int numOfCols; // Total number of cols int numOfCols; // Total number of cols
int sversion; // TODO: set sversion int sversion; // TODO: set sversion
void * buf;
SDataCol *cols; SDataCol *cols;
} SDataCols; } SDataCols;
......
src/common/src/tdataformat.c
浏览文件 @ b5dac67b
...@@ -22,6 +22,29 @@ ...@@ -22,6 +22,29 @@
static void tdMergeTwoDataCols(SDataCols *target, SDataCols *src1, int *iter1, int limit1, SDataCols *src2, int *iter2, static void tdMergeTwoDataCols(SDataCols *target, SDataCols *src1, int *iter1, int limit1, SDataCols *src2, int *iter2,
int limit2, int tRows, bool forceSetNull); int limit2, int tRows, bool forceSetNull);
//TODO: change caller to use return val
int tdAllocMemForCol(SDataCol *pCol, int maxPoints) {
int spaceNeeded = pCol->bytes * maxPoints;
if(IS_VAR_DATA_TYPE(pCol->type)) {
spaceNeeded += sizeof(VarDataOffsetT) * maxPoints;
}
if(pCol->spaceSize < spaceNeeded) {
void* ptr = realloc(pCol->pData, spaceNeeded);
if(ptr == NULL) {
uDebug("malloc failure, size:%" PRId64 " failed, reason:%s", (int64_t)pCol->spaceSize,
strerror(errno));
return -1;
} else {
pCol->pData = ptr;
pCol->spaceSize = spaceNeeded;
}
}
if(IS_VAR_DATA_TYPE(pCol->type)) {
pCol->dataOff = POINTER_SHIFT(pCol->pData, pCol->bytes * maxPoints);
}
return 0;
}
/** /**
* Duplicate the schema and return a new object * Duplicate the schema and return a new object
*/ */
...@@ -207,24 +230,13 @@ SMemRow tdMemRowDup(SMemRow row) { ...@@ -207,24 +230,13 @@ SMemRow tdMemRowDup(SMemRow row) {
return trow; return trow;
} }
void dataColInit(SDataCol *pDataCol, STColumn *pCol, void **pBuf, int maxPoints) { void dataColInit(SDataCol *pDataCol, STColumn *pCol, int maxPoints) {
pDataCol->type = colType(pCol); pDataCol->type = colType(pCol);
pDataCol->colId = colColId(pCol); pDataCol->colId = colColId(pCol);
pDataCol->bytes = colBytes(pCol); pDataCol->bytes = colBytes(pCol);
pDataCol->offset = colOffset(pCol) + TD_DATA_ROW_HEAD_SIZE; pDataCol->offset = colOffset(pCol) + TD_DATA_ROW_HEAD_SIZE;
pDataCol->len = 0; pDataCol->len = 0;
if (IS_VAR_DATA_TYPE(pDataCol->type)) {
pDataCol->dataOff = (VarDataOffsetT *)(*pBuf);
pDataCol->pData = POINTER_SHIFT(*pBuf, sizeof(VarDataOffsetT) * maxPoints);
pDataCol->spaceSize = pDataCol->bytes * maxPoints;
*pBuf = POINTER_SHIFT(*pBuf, pDataCol->spaceSize + sizeof(VarDataOffsetT) * maxPoints);
} else {
pDataCol->spaceSize = pDataCol->bytes * maxPoints;
pDataCol->dataOff = NULL;
pDataCol->pData = *pBuf;
*pBuf = POINTER_SHIFT(*pBuf, pDataCol->spaceSize);
}
} }
// value from timestamp should be TKEY here instead of TSKEY // value from timestamp should be TKEY here instead of TSKEY
void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxPoints) { void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxPoints) {
...@@ -239,6 +251,8 @@ void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxP ...@@ -239,6 +251,8 @@ void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxP
if (numOfRows > 0) { if (numOfRows > 0) {
// Find the first not null value, fill all previouse values as NULL // Find the first not null value, fill all previouse values as NULL
dataColSetNEleNull(pCol, numOfRows, maxPoints); dataColSetNEleNull(pCol, numOfRows, maxPoints);
} else {
tdAllocMemForCol(pCol, maxPoints);
} }
} }
...@@ -257,13 +271,14 @@ void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxP ...@@ -257,13 +271,14 @@ void dataColAppendVal(SDataCol *pCol, const void *value, int numOfRows, int maxP
} }
bool isNEleNull(SDataCol *pCol, int nEle) { bool isNEleNull(SDataCol *pCol, int nEle) {
if(isAllRowsNull(pCol)) return true;
for (int i = 0; i < nEle; i++) { for (int i = 0; i < nEle; i++) {
if (!isNull(tdGetColDataOfRow(pCol, i), pCol->type)) return false; if (!isNull(tdGetColDataOfRow(pCol, i), pCol->type)) return false;
} }
return true; return true;
} }
FORCE_INLINE void dataColSetNullAt(SDataCol *pCol, int index) { static FORCE_INLINE void dataColSetNullAt(SDataCol *pCol, int index) {
if (IS_VAR_DATA_TYPE(pCol->type)) { if (IS_VAR_DATA_TYPE(pCol->type)) {
pCol->dataOff[index] = pCol->len; pCol->dataOff[index] = pCol->len;
char *ptr = POINTER_SHIFT(pCol->pData, pCol->len); char *ptr = POINTER_SHIFT(pCol->pData, pCol->len);
...@@ -276,6 +291,7 @@ FORCE_INLINE void dataColSetNullAt(SDataCol *pCol, int index) { ...@@ -276,6 +291,7 @@ FORCE_INLINE void dataColSetNullAt(SDataCol *pCol, int index) {
} }
void dataColSetNEleNull(SDataCol *pCol, int nEle, int maxPoints) { void dataColSetNEleNull(SDataCol *pCol, int nEle, int maxPoints) {
tdAllocMemForCol(pCol, maxPoints);
if (IS_VAR_DATA_TYPE(pCol->type)) { if (IS_VAR_DATA_TYPE(pCol->type)) {
pCol->len = 0; pCol->len = 0;
...@@ -319,56 +335,63 @@ SDataCols *tdNewDataCols(int maxRowSize, int maxCols, int maxRows) { ...@@ -319,56 +335,63 @@ SDataCols *tdNewDataCols(int maxRowSize, int maxCols, int maxRows) {
tdFreeDataCols(pCols); tdFreeDataCols(pCols);
return NULL; return NULL;
} }
int i;
for(i = 0; i < maxCols; i++) {
pCols->cols[i].spaceSize = 0;
pCols->cols[i].len = 0;
pCols->cols[i].pData = NULL;
pCols->cols[i].dataOff = NULL;
}
pCols->maxCols = maxCols; pCols->maxCols = maxCols;
} }
pCols->maxRowSize = maxRowSize; pCols->maxRowSize = maxRowSize;
pCols->bufSize = maxRowSize * maxRows;
if (pCols->bufSize > 0) {
pCols->buf = malloc(pCols->bufSize);
if (pCols->buf == NULL) {
uDebug("malloc failure, size:%" PRId64 " failed, reason:%s", (int64_t)sizeof(SDataCol) * maxCols,
strerror(errno));
tdFreeDataCols(pCols);
return NULL;
}
}
return pCols; return pCols;
} }
int tdInitDataCols(SDataCols *pCols, STSchema *pSchema) { int tdInitDataCols(SDataCols *pCols, STSchema *pSchema) {
if (schemaNCols(pSchema) > pCols->maxCols) { int i;
int oldMaxCols = pCols->maxCols;
if (schemaNCols(pSchema) > oldMaxCols) {
pCols->maxCols = schemaNCols(pSchema); pCols->maxCols = schemaNCols(pSchema);
pCols->cols = (SDataCol *)realloc(pCols->cols, sizeof(SDataCol) * pCols->maxCols); pCols->cols = (SDataCol *)realloc(pCols->cols, sizeof(SDataCol) * pCols->maxCols);
if (pCols->cols == NULL) return -1; if (pCols->cols == NULL) return -1;
for(i = oldMaxCols; i < pCols->maxCols; i++) {
pCols->cols[i].pData = NULL;
pCols->cols[i].dataOff = NULL;
pCols->cols[i].spaceSize = 0;
}
} }
if (schemaTLen(pSchema) > pCols->maxRowSize) { if (schemaTLen(pSchema) > pCols->maxRowSize) {
pCols->maxRowSize = schemaTLen(pSchema); pCols->maxRowSize = schemaTLen(pSchema);
pCols->bufSize = schemaTLen(pSchema) * pCols->maxPoints;
pCols->buf = realloc(pCols->buf, pCols->bufSize);
if (pCols->buf == NULL) return -1;
} }
tdResetDataCols(pCols); tdResetDataCols(pCols);
pCols->numOfCols = schemaNCols(pSchema); pCols->numOfCols = schemaNCols(pSchema);
void *ptr = pCols->buf; for (i = 0; i < schemaNCols(pSchema); i++) {
for (int i = 0; i < schemaNCols(pSchema); i++) { dataColInit(pCols->cols + i, schemaColAt(pSchema, i), pCols->maxPoints);
dataColInit(pCols->cols + i, schemaColAt(pSchema, i), &ptr, pCols->maxPoints);
ASSERT((char *)ptr - (char *)(pCols->buf) <= pCols->bufSize);
} }
return 0; return 0;
} }
SDataCols *tdFreeDataCols(SDataCols *pCols) { SDataCols *tdFreeDataCols(SDataCols *pCols) {
int i;
if (pCols) { if (pCols) {
tfree(pCols->buf); if(pCols->cols) {
tfree(pCols->cols); int maxCols = pCols->maxCols;
for(i = 0; i < maxCols; i++) {
SDataCol *pCol = &pCols->cols[i];
tfree(pCol->pData);
}
free(pCols->cols);
pCols->cols = NULL;
}
free(pCols); free(pCols);
} }
return NULL; return NULL;
...@@ -388,21 +411,14 @@ SDataCols *tdDupDataCols(SDataCols *pDataCols, bool keepData) { ...@@ -388,21 +411,14 @@ SDataCols *tdDupDataCols(SDataCols *pDataCols, bool keepData) {
pRet->cols[i].bytes = pDataCols->cols[i].bytes; pRet->cols[i].bytes = pDataCols->cols[i].bytes;
pRet->cols[i].offset = pDataCols->cols[i].offset; pRet->cols[i].offset = pDataCols->cols[i].offset;
pRet->cols[i].spaceSize = pDataCols->cols[i].spaceSize;
pRet->cols[i].pData = (void *)((char *)pRet->buf + ((char *)(pDataCols->cols[i].pData) - (char *)(pDataCols->buf)));
if (IS_VAR_DATA_TYPE(pRet->cols[i].type)) {
ASSERT(pDataCols->cols[i].dataOff != NULL);
pRet->cols[i].dataOff =
(int32_t *)((char *)pRet->buf + ((char *)(pDataCols->cols[i].dataOff) - (char *)(pDataCols->buf)));
}
if (keepData) { if (keepData) {
pRet->cols[i].len = pDataCols->cols[i].len;
if (pDataCols->cols[i].len > 0) { if (pDataCols->cols[i].len > 0) {
tdAllocMemForCol(&pRet->cols[i], pRet->maxPoints);
pRet->cols[i].len = pDataCols->cols[i].len;
memcpy(pRet->cols[i].pData, pDataCols->cols[i].pData, pDataCols->cols[i].len); memcpy(pRet->cols[i].pData, pDataCols->cols[i].pData, pDataCols->cols[i].len);
if (IS_VAR_DATA_TYPE(pRet->cols[i].type)) { if (IS_VAR_DATA_TYPE(pRet->cols[i].type)) {
memcpy(pRet->cols[i].dataOff, pDataCols->cols[i].dataOff, sizeof(VarDataOffsetT) * pDataCols->maxPoints); int dataOffSize = sizeof(VarDataOffsetT) * pDataCols->maxPoints;
memcpy(pRet->cols[i].dataOff, pDataCols->cols[i].dataOff, dataOffSize);
} }
} }
} }
...@@ -426,40 +442,27 @@ static void tdAppendDataRowToDataCol(SDataRow row, STSchema *pSchema, SDataCols ...@@ -426,40 +442,27 @@ static void tdAppendDataRowToDataCol(SDataRow row, STSchema *pSchema, SDataCols
int rcol = 0; int rcol = 0;
int dcol = 0; int dcol = 0;
if (dataRowDeleted(row)) { while (dcol < pCols->numOfCols) {
for (; dcol < pCols->numOfCols; dcol++) { SDataCol *pDataCol = &(pCols->cols[dcol]);
SDataCol *pDataCol = &(pCols->cols[dcol]); if (rcol >= schemaNCols(pSchema)) {
if (dcol == 0) { dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
dataColAppendVal(pDataCol, dataRowTuple(row), pCols->numOfRows, pCols->maxPoints); dcol++;
} else { continue;
dataColSetNullAt(pDataCol, pCols->numOfRows);
}
} }
} else {
while (dcol < pCols->numOfCols) {
SDataCol *pDataCol = &(pCols->cols[dcol]);
if (rcol >= schemaNCols(pSchema)) {
// dataColSetNullAt(pDataCol, pCols->numOfRows);
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
dcol++;
continue;
}
STColumn *pRowCol = schemaColAt(pSchema, rcol); STColumn *pRowCol = schemaColAt(pSchema, rcol);
if (pRowCol->colId == pDataCol->colId) { if (pRowCol->colId == pDataCol->colId) {
void *value = tdGetRowDataOfCol(row, pRowCol->type, pRowCol->offset + TD_DATA_ROW_HEAD_SIZE); void *value = tdGetRowDataOfCol(row, pRowCol->type, pRowCol->offset + TD_DATA_ROW_HEAD_SIZE);
dataColAppendVal(pDataCol, value, pCols->numOfRows, pCols->maxPoints); dataColAppendVal(pDataCol, value, pCols->numOfRows, pCols->maxPoints);
dcol++; dcol++;
rcol++; rcol++;
} else if (pRowCol->colId < pDataCol->colId) { } else if (pRowCol->colId < pDataCol->colId) {
rcol++; rcol++;
} else { } else {
if(forceSetNull) { if(forceSetNull) {
//dataColSetNullAt(pDataCol, pCols->numOfRows); dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
}
dcol++;
} }
dcol++;
} }
} }
pCols->numOfRows++; pCols->numOfRows++;
...@@ -471,43 +474,30 @@ static void tdAppendKvRowToDataCol(SKVRow row, STSchema *pSchema, SDataCols *pCo ...@@ -471,43 +474,30 @@ static void tdAppendKvRowToDataCol(SKVRow row, STSchema *pSchema, SDataCols *pCo
int rcol = 0; int rcol = 0;
int dcol = 0; int dcol = 0;
if (kvRowDeleted(row)) { int nRowCols = kvRowNCols(row);
for (; dcol < pCols->numOfCols; dcol++) {
SDataCol *pDataCol = &(pCols->cols[dcol]); while (dcol < pCols->numOfCols) {
if (dcol == 0) { SDataCol *pDataCol = &(pCols->cols[dcol]);
dataColAppendVal(pDataCol, kvRowValues(row), pCols->numOfRows, pCols->maxPoints); if (rcol >= nRowCols || rcol >= schemaNCols(pSchema)) {
} else { dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
dataColSetNullAt(pDataCol, pCols->numOfRows); ++dcol;
} continue;
} }
} else {
int nRowCols = kvRowNCols(row);
while (dcol < pCols->numOfCols) { SColIdx *colIdx = kvRowColIdxAt(row, rcol);
SDataCol *pDataCol = &(pCols->cols[dcol]);
if (rcol >= nRowCols || rcol >= schemaNCols(pSchema)) {
// dataColSetNullAt(pDataCol, pCols->numOfRows);
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
++dcol;
continue;
}
SColIdx *colIdx = kvRowColIdxAt(row, rcol); if (colIdx->colId == pDataCol->colId) {
void *value = tdGetKvRowDataOfCol(row, colIdx->offset);
if (colIdx->colId == pDataCol->colId) { dataColAppendVal(pDataCol, value, pCols->numOfRows, pCols->maxPoints);
void *value = tdGetKvRowDataOfCol(row, colIdx->offset); ++dcol;
dataColAppendVal(pDataCol, value, pCols->numOfRows, pCols->maxPoints); ++rcol;
++dcol; } else if (colIdx->colId < pDataCol->colId) {
++rcol; ++rcol;
} else if (colIdx->colId < pDataCol->colId) { } else {
++rcol; if (forceSetNull) {
} else { dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
if (forceSetNull) {
// dataColSetNullAt(pDataCol, pCols->numOfRows);
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints);
}
++dcol;
} }
++dcol;
} }
} }
pCols->numOfRows++; pCols->numOfRows++;
......
src/connector/jdbc/pom.xml
浏览文件 @ b5dac67b
...@@ -122,6 +122,7 @@ ...@@ -122,6 +122,7 @@
<exclude>**/TSDBJNIConnectorTest.java</exclude> <exclude>**/TSDBJNIConnectorTest.java</exclude>
<exclude>**/TaosInfoMonitorTest.java</exclude> <exclude>**/TaosInfoMonitorTest.java</exclude>
<exclude>**/UnsignedNumberJniTest.java</exclude> <exclude>**/UnsignedNumberJniTest.java</exclude>
<exclude>**/TimeZoneTest.java</exclude>
</excludes> </excludes>
<testFailureIgnore>true</testFailureIgnore> <testFailureIgnore>true</testFailureIgnore>
</configuration> </configuration>
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java
浏览文件 @ b5dac67b
...@@ -80,7 +80,8 @@ public class TSDBJNIConnector { ...@@ -80,7 +80,8 @@ public class TSDBJNIConnector {
this.taos = this.connectImp(host, port, dbName, user, password); this.taos = this.connectImp(host, port, dbName, user, password);
if (this.taos == TSDBConstants.JNI_NULL_POINTER) { if (this.taos == TSDBConstants.JNI_NULL_POINTER) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL); String errMsg = this.getErrMsg(0);
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL, errMsg);
} }
// invoke connectImp only here // invoke connectImp only here
taosInfo.conn_open_increment(); taosInfo.conn_open_increment();
......
src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/QueryDataTest.java
浏览文件 @ b5dac67b
...@@ -34,9 +34,8 @@ public class QueryDataTest { ...@@ -34,9 +34,8 @@ public class QueryDataTest {
String createTableSql = "create table " + stbName + "(ts timestamp, name binary(64))"; String createTableSql = "create table " + stbName + "(ts timestamp, name binary(64))";
statement.executeUpdate(createTableSql); statement.executeUpdate(createTableSql);
} catch (SQLException e) { } catch (SQLException e) {
return; e.printStackTrace();
} }
} }
......
src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/TimeZoneTest.java 0 → 100644
浏览文件 @ b5dac67b
package com.taosdata.jdbc.cases;
import com.taosdata.jdbc.TSDBDriver;
import org.junit.Test;
import java.sql.*;
import java.time.Instant;
import java.time.LocalDateTime;
import java.time.ZoneId;
import java.util.Properties;
public class TimeZoneTest {
private String url = "jdbc:TAOS://127.0.0.1:6030/?user=root&password=taosdata";
@Test
public void javaTimeZone() {
LocalDateTime localDateTime = LocalDateTime.of(1970, 1, 1, 0, 0, 0);
Instant instant = localDateTime.atZone(ZoneId.of("UTC-8")).toInstant();
System.out.println("UTC-8: " + instant.getEpochSecond() + "," + instant);
instant = localDateTime.atZone(ZoneId.of("UT")).toInstant();
System.out.println("UTC: " + instant.getEpochSecond() + "," + instant);
instant = localDateTime.atZone(ZoneId.of("UTC+8")).toInstant();
System.out.println("UTC+8: " + instant.getEpochSecond() + "," + instant);
}
@Test
public void taosTimeZone() {
// given
Properties props = new Properties();
props.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
// when and then
try (Connection connection = DriverManager.getConnection(url, props)) {
Statement stmt = connection.createStatement();
stmt.execute("drop database if exists timezone_test");
stmt.execute("create database if not exists timezone_test keep 365000");
stmt.execute("use timezone_test");
stmt.execute("create table weather(ts timestamp, temperature float)");
stmt.execute("insert into timezone_test.weather(ts, temperature) values('1970-01-01 00:00:00', 1.0)");
ResultSet rs = stmt.executeQuery("select * from timezone_test.weather");
while (rs.next()) {
Timestamp ts = rs.getTimestamp("ts");
System.out.println("ts: " + ts.getTime() + "," + ts);
}
stmt.execute("insert into timezone_test.weather(ts, temperature, humidity) values('1970-01-02 00:00:00', 1.0, 2.0)");
rs = stmt.executeQuery("select * from timezone_test.weather");
while (rs.next()) {
Timestamp ts = rs.getTimestamp("ts");
System.out.println("ts: " + ts.getTime() + "," + ts);
}
stmt.execute("drop database if exists timezone_test");
stmt.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
\ No newline at end of file
src/inc/tsdb.h
浏览文件 @ b5dac67b
...@@ -94,7 +94,7 @@ STsdbRepo *tsdbOpenRepo(STsdbCfg *pCfg, STsdbAppH *pAppH); ...@@ -94,7 +94,7 @@ STsdbRepo *tsdbOpenRepo(STsdbCfg *pCfg, STsdbAppH *pAppH);
int tsdbCloseRepo(STsdbRepo *repo, int toCommit); int tsdbCloseRepo(STsdbRepo *repo, int toCommit);
int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg); int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg);
int tsdbGetState(STsdbRepo *repo); int tsdbGetState(STsdbRepo *repo);
bool tsdbInCompact(STsdbRepo *repo); int8_t tsdbGetCompactState(STsdbRepo *repo);
// --------- TSDB TABLE DEFINITION // --------- TSDB TABLE DEFINITION
typedef struct { typedef struct {
uint64_t uid; // the unique table ID uint64_t uid; // the unique table ID
......
src/kit/taosdemo/taosdemo.c
浏览文件 @ b5dac67b
...@@ -5093,7 +5093,9 @@ static int getRowDataFromSample( ...@@ -5093,7 +5093,9 @@ static int getRowDataFromSample(
static int64_t generateStbRowData( static int64_t generateStbRowData(
SSuperTable* stbInfo, SSuperTable* stbInfo,
char* recBuf, int64_t timestamp) char* recBuf,
int64_t remainderBufLen,
int64_t timestamp)
{ {
int64_t dataLen = 0; int64_t dataLen = 0;
char *pstr = recBuf; char *pstr = recBuf;
...@@ -5121,6 +5123,7 @@ static int64_t generateStbRowData( ...@@ -5121,6 +5123,7 @@ static int64_t generateStbRowData(
rand_string(buf, stbInfo->columns[i].dataLen); rand_string(buf, stbInfo->columns[i].dataLen);
dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "\'%s\',", buf); dataLen += snprintf(pstr + dataLen, maxLen - dataLen, "\'%s\',", buf);
tmfree(buf); tmfree(buf);
} else { } else {
char *tmp; char *tmp;
...@@ -5177,6 +5180,9 @@ static int64_t generateStbRowData( ...@@ -5177,6 +5180,9 @@ static int64_t generateStbRowData(
tstrncpy(pstr + dataLen, ",", 2); tstrncpy(pstr + dataLen, ",", 2);
dataLen += 1; dataLen += 1;
} }
if (dataLen > remainderBufLen)
return 0;
} }
dataLen -= 1; dataLen -= 1;
...@@ -5383,7 +5389,7 @@ static int32_t generateDataTailWithoutStb( ...@@ -5383,7 +5389,7 @@ static int32_t generateDataTailWithoutStb(
int32_t k = 0; int32_t k = 0;
for (k = 0; k < batch;) { for (k = 0; k < batch;) {
char data[MAX_DATA_SIZE]; char *data = pstr;
memset(data, 0, MAX_DATA_SIZE); memset(data, 0, MAX_DATA_SIZE);
int64_t retLen = 0; int64_t retLen = 0;
...@@ -5407,7 +5413,7 @@ static int32_t generateDataTailWithoutStb( ...@@ -5407,7 +5413,7 @@ static int32_t generateDataTailWithoutStb(
if (len > remainderBufLen) if (len > remainderBufLen)
break; break;
pstr += sprintf(pstr, "%s", data); pstr += retLen;
k++; k++;
len += retLen; len += retLen;
remainderBufLen -= retLen; remainderBufLen -= retLen;
...@@ -5463,14 +5469,14 @@ static int32_t generateStbDataTail( ...@@ -5463,14 +5469,14 @@ static int32_t generateStbDataTail(
int32_t k; int32_t k;
for (k = 0; k < batch;) { for (k = 0; k < batch;) {
char data[MAX_DATA_SIZE]; char *data = pstr;
memset(data, 0, MAX_DATA_SIZE);
int64_t lenOfRow = 0; int64_t lenOfRow = 0;
if (tsRand) { if (tsRand) {
if (superTblInfo->disorderRatio > 0) { if (superTblInfo->disorderRatio > 0) {
lenOfRow = generateStbRowData(superTblInfo, data, lenOfRow = generateStbRowData(superTblInfo, data,
remainderBufLen,
startTime + getTSRandTail( startTime + getTSRandTail(
superTblInfo->timeStampStep, k, superTblInfo->timeStampStep, k,
superTblInfo->disorderRatio, superTblInfo->disorderRatio,
...@@ -5478,6 +5484,7 @@ static int32_t generateStbDataTail( ...@@ -5478,6 +5484,7 @@ static int32_t generateStbDataTail(
); );
} else { } else {
lenOfRow = generateStbRowData(superTblInfo, data, lenOfRow = generateStbRowData(superTblInfo, data,
remainderBufLen,
startTime + superTblInfo->timeStampStep * k startTime + superTblInfo->timeStampStep * k
); );
} }
...@@ -5490,11 +5497,15 @@ static int32_t generateStbDataTail( ...@@ -5490,11 +5497,15 @@ static int32_t generateStbDataTail(
pSamplePos); pSamplePos);
} }
if (lenOfRow == 0) {
data[0] = '\0';
break;
}
if ((lenOfRow + 1) > remainderBufLen) { if ((lenOfRow + 1) > remainderBufLen) {
break; break;
} }
pstr += snprintf(pstr , lenOfRow + 1, "%s", data); pstr += lenOfRow;
k++; k++;
len += lenOfRow; len += lenOfRow;
remainderBufLen -= lenOfRow; remainderBufLen -= lenOfRow;
...@@ -6246,7 +6257,7 @@ static int32_t generateStbProgressiveData( ...@@ -6246,7 +6257,7 @@ static int32_t generateStbProgressiveData(
assert(buffer != NULL); assert(buffer != NULL);
char *pstr = buffer; char *pstr = buffer;
memset(buffer, 0, *pRemainderBufLen); memset(pstr, 0, *pRemainderBufLen);
int64_t headLen = generateStbSQLHead( int64_t headLen = generateStbSQLHead(
superTblInfo, superTblInfo,
...@@ -6640,7 +6651,7 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) { ...@@ -6640,7 +6651,7 @@ static void* syncWriteProgressive(threadInfo *pThreadInfo) {
return NULL; return NULL;
} }
int64_t remainderBufLen = maxSqlLen; int64_t remainderBufLen = maxSqlLen - 2000;
char *pstr = pThreadInfo->buffer; char *pstr = pThreadInfo->buffer;
int len = snprintf(pstr, int len = snprintf(pstr,
...@@ -6822,10 +6833,14 @@ static void callBack(void *param, TAOS_RES *res, int code) { ...@@ -6822,10 +6833,14 @@ static void callBack(void *param, TAOS_RES *res, int code) {
&& rand_num < pThreadInfo->superTblInfo->disorderRatio) { && rand_num < pThreadInfo->superTblInfo->disorderRatio) {
int64_t d = pThreadInfo->lastTs int64_t d = pThreadInfo->lastTs
- (taosRandom() % pThreadInfo->superTblInfo->disorderRange + 1); - (taosRandom() % pThreadInfo->superTblInfo->disorderRange + 1);
generateStbRowData(pThreadInfo->superTblInfo, data, d); generateStbRowData(pThreadInfo->superTblInfo, data,
MAX_DATA_SIZE,
d);
} else { } else {
generateStbRowData(pThreadInfo->superTblInfo, generateStbRowData(pThreadInfo->superTblInfo,
data, pThreadInfo->lastTs += 1000); data,
MAX_DATA_SIZE,
pThreadInfo->lastTs += 1000);
} }
pstr += sprintf(pstr, "%s", data); pstr += sprintf(pstr, "%s", data);
pThreadInfo->counter++; pThreadInfo->counter++;
...@@ -7050,6 +7065,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, ...@@ -7050,6 +7065,7 @@ static void startMultiThreadInsertData(int threads, char* db_name,
for (int i = 0; i < threads; i++) { for (int i = 0; i < threads; i++) {
threadInfo *pThreadInfo = infos + i; threadInfo *pThreadInfo = infos + i;
pThreadInfo->threadID = i; pThreadInfo->threadID = i;
tstrncpy(pThreadInfo->db_name, db_name, TSDB_DB_NAME_LEN); tstrncpy(pThreadInfo->db_name, db_name, TSDB_DB_NAME_LEN);
pThreadInfo->time_precision = timePrec; pThreadInfo->time_precision = timePrec;
pThreadInfo->superTblInfo = superTblInfo; pThreadInfo->superTblInfo = superTblInfo;
......
src/kit/taosdump/taosdump.c
浏览文件 @ b5dac67b
...@@ -307,7 +307,7 @@ static void taosDumpCreateTableClause(STableDef *tableDes, int numOfCols, ...@@ -307,7 +307,7 @@ static void taosDumpCreateTableClause(STableDef *tableDes, int numOfCols,
FILE *fp, char* dbName); FILE *fp, char* dbName);
static void taosDumpCreateMTableClause(STableDef *tableDes, char *metric, static void taosDumpCreateMTableClause(STableDef *tableDes, char *metric,
int numOfCols, FILE *fp, char* dbName); int numOfCols, FILE *fp, char* dbName);
static int32_t taosDumpTable(char *table, char *metric, static int32_t taosDumpTable(char *tbName, char *metric,
FILE *fp, TAOS* taosCon, char* dbName); FILE *fp, TAOS* taosCon, char* dbName);
static int taosDumpTableData(FILE *fp, char *tbName, static int taosDumpTableData(FILE *fp, char *tbName,
TAOS* taosCon, char* dbName, TAOS* taosCon, char* dbName,
...@@ -340,7 +340,7 @@ struct arguments g_args = { ...@@ -340,7 +340,7 @@ struct arguments g_args = {
false, // schemeonly false, // schemeonly
true, // with_property true, // with_property
false, // avro format false, // avro format
-INT64_MAX, // start_time -INT64_MAX, // start_time
INT64_MAX, // end_time INT64_MAX, // end_time
"ms", // precision "ms", // precision
1, // data_batch 1, // data_batch
...@@ -798,11 +798,11 @@ static int taosGetTableRecordInfo( ...@@ -798,11 +798,11 @@ static int taosGetTableRecordInfo(
tstrncpy(pTableRecordInfo->tableRecord.name, tstrncpy(pTableRecordInfo->tableRecord.name,
(char *)row[TSDB_SHOW_TABLES_NAME_INDEX], (char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes) + 1); fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes + 1));
tstrncpy(pTableRecordInfo->tableRecord.metric, tstrncpy(pTableRecordInfo->tableRecord.metric,
(char *)row[TSDB_SHOW_TABLES_METRIC_INDEX], (char *)row[TSDB_SHOW_TABLES_METRIC_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes) + 1); fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes + 1));
break; break;
} }
...@@ -945,7 +945,7 @@ static int32_t taosSaveTableOfMetricToTempFile( ...@@ -945,7 +945,7 @@ static int32_t taosSaveTableOfMetricToTempFile(
int32_t numOfThread = *totalNumOfThread; int32_t numOfThread = *totalNumOfThread;
int subFd = -1; int subFd = -1;
for (; numOfThread < maxThreads; numOfThread++) { for (; numOfThread <= maxThreads; numOfThread++) {
memset(tmpBuf, 0, MAX_FILE_NAME_LEN); memset(tmpBuf, 0, MAX_FILE_NAME_LEN);
sprintf(tmpBuf, ".tables.tmp.%d", numOfThread); sprintf(tmpBuf, ".tables.tmp.%d", numOfThread);
subFd = open(tmpBuf, O_RDWR | O_CREAT | O_TRUNC, S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH); subFd = open(tmpBuf, O_RDWR | O_CREAT | O_TRUNC, S_IRWXU | S_IRGRP | S_IXGRP | S_IROTH);
...@@ -1084,7 +1084,7 @@ _dump_db_point: ...@@ -1084,7 +1084,7 @@ _dump_db_point:
} }
tstrncpy(g_dbInfos[count]->name, (char *)row[TSDB_SHOW_DB_NAME_INDEX], tstrncpy(g_dbInfos[count]->name, (char *)row[TSDB_SHOW_DB_NAME_INDEX],
min(TSDB_DB_NAME_LEN, fields[TSDB_SHOW_DB_NAME_INDEX].bytes) + 1); min(TSDB_DB_NAME_LEN, fields[TSDB_SHOW_DB_NAME_INDEX].bytes + 1));
if (g_args.with_property) { if (g_args.with_property) {
g_dbInfos[count]->ntables = *((int32_t *)row[TSDB_SHOW_DB_NTABLES_INDEX]); g_dbInfos[count]->ntables = *((int32_t *)row[TSDB_SHOW_DB_NTABLES_INDEX]);
g_dbInfos[count]->vgroups = *((int32_t *)row[TSDB_SHOW_DB_VGROUPS_INDEX]); g_dbInfos[count]->vgroups = *((int32_t *)row[TSDB_SHOW_DB_VGROUPS_INDEX]);
...@@ -1093,7 +1093,7 @@ _dump_db_point: ...@@ -1093,7 +1093,7 @@ _dump_db_point:
g_dbInfos[count]->days = *((int16_t *)row[TSDB_SHOW_DB_DAYS_INDEX]); g_dbInfos[count]->days = *((int16_t *)row[TSDB_SHOW_DB_DAYS_INDEX]);
tstrncpy(g_dbInfos[count]->keeplist, (char *)row[TSDB_SHOW_DB_KEEP_INDEX], tstrncpy(g_dbInfos[count]->keeplist, (char *)row[TSDB_SHOW_DB_KEEP_INDEX],
min(32, fields[TSDB_SHOW_DB_KEEP_INDEX].bytes) + 1); min(32, fields[TSDB_SHOW_DB_KEEP_INDEX].bytes + 1));
//g_dbInfos[count]->daysToKeep = *((int16_t *)row[TSDB_SHOW_DB_KEEP_INDEX]); //g_dbInfos[count]->daysToKeep = *((int16_t *)row[TSDB_SHOW_DB_KEEP_INDEX]);
//g_dbInfos[count]->daysToKeep1; //g_dbInfos[count]->daysToKeep1;
//g_dbInfos[count]->daysToKeep2; //g_dbInfos[count]->daysToKeep2;
...@@ -1107,7 +1107,7 @@ _dump_db_point: ...@@ -1107,7 +1107,7 @@ _dump_db_point:
g_dbInfos[count]->cachelast = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_CACHELAST_INDEX])); g_dbInfos[count]->cachelast = (int8_t)(*((int8_t *)row[TSDB_SHOW_DB_CACHELAST_INDEX]));
tstrncpy(g_dbInfos[count]->precision, (char *)row[TSDB_SHOW_DB_PRECISION_INDEX], tstrncpy(g_dbInfos[count]->precision, (char *)row[TSDB_SHOW_DB_PRECISION_INDEX],
min(8, fields[TSDB_SHOW_DB_PRECISION_INDEX].bytes) + 1); min(8, fields[TSDB_SHOW_DB_PRECISION_INDEX].bytes + 1));
//g_dbInfos[count]->precision = *((int8_t *)row[TSDB_SHOW_DB_PRECISION_INDEX]); //g_dbInfos[count]->precision = *((int8_t *)row[TSDB_SHOW_DB_PRECISION_INDEX]);
g_dbInfos[count]->update = *((int8_t *)row[TSDB_SHOW_DB_UPDATE_INDEX]); g_dbInfos[count]->update = *((int8_t *)row[TSDB_SHOW_DB_UPDATE_INDEX]);
} }
...@@ -1237,7 +1237,7 @@ _exit_failure: ...@@ -1237,7 +1237,7 @@ _exit_failure:
static int taosGetTableDes( static int taosGetTableDes(
char* dbName, char *table, char* dbName, char *table,
STableDef *tableDes, TAOS* taosCon, bool isSuperTable) { STableDef *stableDes, TAOS* taosCon, bool isSuperTable) {
TAOS_ROW row = NULL; TAOS_ROW row = NULL;
TAOS_RES* res = NULL; TAOS_RES* res = NULL;
int count = 0; int count = 0;
...@@ -1256,18 +1256,18 @@ static int taosGetTableDes( ...@@ -1256,18 +1256,18 @@ static int taosGetTableDes(
TAOS_FIELD *fields = taos_fetch_fields(res); TAOS_FIELD *fields = taos_fetch_fields(res);
tstrncpy(tableDes->name, table, TSDB_TABLE_NAME_LEN); tstrncpy(stableDes->name, table, TSDB_TABLE_NAME_LEN);
while ((row = taos_fetch_row(res)) != NULL) { while ((row = taos_fetch_row(res)) != NULL) {
tstrncpy(tableDes->cols[count].field, tstrncpy(stableDes->cols[count].field,
(char *)row[TSDB_DESCRIBE_METRIC_FIELD_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_FIELD_INDEX],
min(TSDB_COL_NAME_LEN + 1, min(TSDB_COL_NAME_LEN + 1,
fields[TSDB_DESCRIBE_METRIC_FIELD_INDEX].bytes + 1)); fields[TSDB_DESCRIBE_METRIC_FIELD_INDEX].bytes + 1));
tstrncpy(tableDes->cols[count].type, tstrncpy(stableDes->cols[count].type,
(char *)row[TSDB_DESCRIBE_METRIC_TYPE_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_TYPE_INDEX],
min(16, fields[TSDB_DESCRIBE_METRIC_TYPE_INDEX].bytes + 1)); min(16, fields[TSDB_DESCRIBE_METRIC_TYPE_INDEX].bytes + 1));
tableDes->cols[count].length = stableDes->cols[count].length =
*((int *)row[TSDB_DESCRIBE_METRIC_LENGTH_INDEX]); *((int *)row[TSDB_DESCRIBE_METRIC_LENGTH_INDEX]);
tstrncpy(tableDes->cols[count].note, tstrncpy(stableDes->cols[count].note,
(char *)row[TSDB_DESCRIBE_METRIC_NOTE_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_NOTE_INDEX],
min(COL_NOTE_LEN, min(COL_NOTE_LEN,
fields[TSDB_DESCRIBE_METRIC_NOTE_INDEX].bytes + 1)); fields[TSDB_DESCRIBE_METRIC_NOTE_INDEX].bytes + 1));
...@@ -1284,11 +1284,11 @@ static int taosGetTableDes( ...@@ -1284,11 +1284,11 @@ static int taosGetTableDes(
// if chidl-table have tag, using select tagName from table to get tagValue // if chidl-table have tag, using select tagName from table to get tagValue
for (int i = 0 ; i < count; i++) { for (int i = 0 ; i < count; i++) {
if (strcmp(tableDes->cols[i].note, "TAG") != 0) continue; if (strcmp(stableDes->cols[i].note, "TAG") != 0) continue;
sprintf(sqlstr, "select %s from %s.%s", sprintf(sqlstr, "select %s from %s.%s",
tableDes->cols[i].field, dbName, table); stableDes->cols[i].field, dbName, table);
res = taos_query(taosCon, sqlstr); res = taos_query(taosCon, sqlstr);
code = taos_errno(res); code = taos_errno(res);
...@@ -1310,7 +1310,7 @@ static int taosGetTableDes( ...@@ -1310,7 +1310,7 @@ static int taosGetTableDes(
} }
if (row[0] == NULL) { if (row[0] == NULL) {
sprintf(tableDes->cols[i].note, "%s", "NULL"); sprintf(stableDes->cols[i].note, "%s", "NULL");
taos_free_result(res); taos_free_result(res);
res = NULL; res = NULL;
continue; continue;
...@@ -1321,47 +1321,47 @@ static int taosGetTableDes( ...@@ -1321,47 +1321,47 @@ static int taosGetTableDes(
//int32_t* length = taos_fetch_lengths(tmpResult); //int32_t* length = taos_fetch_lengths(tmpResult);
switch (fields[0].type) { switch (fields[0].type) {
case TSDB_DATA_TYPE_BOOL: case TSDB_DATA_TYPE_BOOL:
sprintf(tableDes->cols[i].note, "%d", sprintf(stableDes->cols[i].note, "%d",
((((int32_t)(*((char *)row[0]))) == 1) ? 1 : 0)); ((((int32_t)(*((char *)row[0]))) == 1) ? 1 : 0));
break; break;
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
sprintf(tableDes->cols[i].note, "%d", *((int8_t *)row[0])); sprintf(stableDes->cols[i].note, "%d", *((int8_t *)row[0]));
break; break;
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
sprintf(tableDes->cols[i].note, "%d", *((int16_t *)row[0])); sprintf(stableDes->cols[i].note, "%d", *((int16_t *)row[0]));
break; break;
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
sprintf(tableDes->cols[i].note, "%d", *((int32_t *)row[0])); sprintf(stableDes->cols[i].note, "%d", *((int32_t *)row[0]));
break; break;
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
sprintf(tableDes->cols[i].note, "%" PRId64 "", *((int64_t *)row[0])); sprintf(stableDes->cols[i].note, "%" PRId64 "", *((int64_t *)row[0]));
break; break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
sprintf(tableDes->cols[i].note, "%f", GET_FLOAT_VAL(row[0])); sprintf(stableDes->cols[i].note, "%f", GET_FLOAT_VAL(row[0]));
break; break;
case TSDB_DATA_TYPE_DOUBLE: case TSDB_DATA_TYPE_DOUBLE:
sprintf(tableDes->cols[i].note, "%f", GET_DOUBLE_VAL(row[0])); sprintf(stableDes->cols[i].note, "%f", GET_DOUBLE_VAL(row[0]));
break; break;
case TSDB_DATA_TYPE_BINARY: case TSDB_DATA_TYPE_BINARY:
{ {
memset(tableDes->cols[i].note, 0, sizeof(tableDes->cols[i].note)); memset(stableDes->cols[i].note, 0, sizeof(stableDes->cols[i].note));
tableDes->cols[i].note[0] = '\''; stableDes->cols[i].note[0] = '\'';
char tbuf[COL_NOTE_LEN]; char tbuf[COL_NOTE_LEN];
converStringToReadable((char *)row[0], length[0], tbuf, COL_NOTE_LEN); converStringToReadable((char *)row[0], length[0], tbuf, COL_NOTE_LEN);
char* pstr = stpcpy(&(tableDes->cols[i].note[1]), tbuf); char* pstr = stpcpy(&(stableDes->cols[i].note[1]), tbuf);
*(pstr++) = '\''; *(pstr++) = '\'';
break; break;
} }
case TSDB_DATA_TYPE_NCHAR: case TSDB_DATA_TYPE_NCHAR:
{ {
memset(tableDes->cols[i].note, 0, sizeof(tableDes->cols[i].note)); memset(stableDes->cols[i].note, 0, sizeof(stableDes->cols[i].note));
char tbuf[COL_NOTE_LEN-2]; // need reserve 2 bytes for ' ' char tbuf[COL_NOTE_LEN-2]; // need reserve 2 bytes for ' '
convertNCharToReadable((char *)row[0], length[0], tbuf, COL_NOTE_LEN); convertNCharToReadable((char *)row[0], length[0], tbuf, COL_NOTE_LEN);
sprintf(tableDes->cols[i].note, "\'%s\'", tbuf); sprintf(stableDes->cols[i].note, "\'%s\'", tbuf);
break; break;
} }
case TSDB_DATA_TYPE_TIMESTAMP: case TSDB_DATA_TYPE_TIMESTAMP:
sprintf(tableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[0]); sprintf(stableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[0]);
#if 0 #if 0
if (!g_args.mysqlFlag) { if (!g_args.mysqlFlag) {
sprintf(tableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[0]); sprintf(tableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[0]);
...@@ -1386,7 +1386,7 @@ static int taosGetTableDes( ...@@ -1386,7 +1386,7 @@ static int taosGetTableDes(
return count; return count;
} }
static int convertSchemaToAvroSchema(STableDef *tableDes, char **avroSchema) static int convertSchemaToAvroSchema(STableDef *stableDes, char **avroSchema)
{ {
errorPrint("%s() LN%d TODO: covert table schema to avro schema\n", errorPrint("%s() LN%d TODO: covert table schema to avro schema\n",
__func__, __LINE__); __func__, __LINE__);
...@@ -1394,7 +1394,7 @@ static int convertSchemaToAvroSchema(STableDef *tableDes, char **avroSchema) ...@@ -1394,7 +1394,7 @@ static int convertSchemaToAvroSchema(STableDef *tableDes, char **avroSchema)
} }
static int32_t taosDumpTable( static int32_t taosDumpTable(
char *table, char *metric, char *tbName, char *metric,
FILE *fp, TAOS* taosCon, char* dbName) { FILE *fp, TAOS* taosCon, char* dbName) {
int count = 0; int count = 0;
...@@ -1415,7 +1415,7 @@ static int32_t taosDumpTable( ...@@ -1415,7 +1415,7 @@ static int32_t taosDumpTable(
memset(tableDes, 0, sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS); memset(tableDes, 0, sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS);
*/ */
count = taosGetTableDes(dbName, table, tableDes, taosCon, false); count = taosGetTableDes(dbName, tbName, tableDes, taosCon, false);
if (count < 0) { if (count < 0) {
free(tableDes); free(tableDes);
...@@ -1426,7 +1426,7 @@ static int32_t taosDumpTable( ...@@ -1426,7 +1426,7 @@ static int32_t taosDumpTable(
taosDumpCreateMTableClause(tableDes, metric, count, fp, dbName); taosDumpCreateMTableClause(tableDes, metric, count, fp, dbName);
} else { // dump table definition } else { // dump table definition
count = taosGetTableDes(dbName, table, tableDes, taosCon, false); count = taosGetTableDes(dbName, tbName, tableDes, taosCon, false);
if (count < 0) { if (count < 0) {
free(tableDes); free(tableDes);
...@@ -1446,7 +1446,7 @@ static int32_t taosDumpTable( ...@@ -1446,7 +1446,7 @@ static int32_t taosDumpTable(
int32_t ret = 0; int32_t ret = 0;
if (!g_args.schemaonly) { if (!g_args.schemaonly) {
ret = taosDumpTableData(fp, table, taosCon, dbName, ret = taosDumpTableData(fp, tbName, taosCon, dbName,
jsonAvroSchema); jsonAvroSchema);
} }
...@@ -1648,26 +1648,27 @@ static void taosStartDumpOutWorkThreads(int32_t numOfThread, char *dbName) ...@@ -1648,26 +1648,27 @@ static void taosStartDumpOutWorkThreads(int32_t numOfThread, char *dbName)
static int32_t taosDumpStable(char *table, FILE *fp, static int32_t taosDumpStable(char *table, FILE *fp,
TAOS* taosCon, char* dbName) { TAOS* taosCon, char* dbName) {
uint64_t sizeOfTableDes = (uint64_t)(sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS); uint64_t sizeOfTableDes =
STableDef *tableDes = (STableDef *)calloc(1, sizeOfTableDes); (uint64_t)(sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS);
if (NULL == tableDes) { STableDef *stableDes = (STableDef *)calloc(1, sizeOfTableDes);
if (NULL == stableDes) {
errorPrint("%s() LN%d, failed to allocate %"PRIu64" memory\n", errorPrint("%s() LN%d, failed to allocate %"PRIu64" memory\n",
__func__, __LINE__, sizeOfTableDes); __func__, __LINE__, sizeOfTableDes);
exit(-1); exit(-1);
} }
int count = taosGetTableDes(dbName, table, tableDes, taosCon, true); int count = taosGetTableDes(dbName, table, stableDes, taosCon, true);
if (count < 0) { if (count < 0) {
free(tableDes); free(stableDes);
errorPrint("%s() LN%d, failed to get stable[%s] schema\n", errorPrint("%s() LN%d, failed to get stable[%s] schema\n",
__func__, __LINE__, table); __func__, __LINE__, table);
exit(-1); exit(-1);
} }
taosDumpCreateTableClause(tableDes, count, fp, dbName); taosDumpCreateTableClause(stableDes, count, fp, dbName);
free(tableDes); free(stableDes);
return 0; return 0;
} }
...@@ -1707,7 +1708,7 @@ static int32_t taosDumpCreateSuperTableClause(TAOS* taosCon, char* dbName, FILE ...@@ -1707,7 +1708,7 @@ static int32_t taosDumpCreateSuperTableClause(TAOS* taosCon, char* dbName, FILE
memset(&tableRecord, 0, sizeof(STableRecord)); memset(&tableRecord, 0, sizeof(STableRecord));
tstrncpy(tableRecord.name, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX], tstrncpy(tableRecord.name, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes) + 1); fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes + 1));
taosWrite(fd, &tableRecord, sizeof(STableRecord)); taosWrite(fd, &tableRecord, sizeof(STableRecord));
} }
...@@ -1782,10 +1783,10 @@ static int taosDumpDb(SDbInfo *dbInfo, FILE *fp, TAOS *taosCon) { ...@@ -1782,10 +1783,10 @@ static int taosDumpDb(SDbInfo *dbInfo, FILE *fp, TAOS *taosCon) {
memset(&tableRecord, 0, sizeof(STableRecord)); memset(&tableRecord, 0, sizeof(STableRecord));
tstrncpy(tableRecord.name, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX], tstrncpy(tableRecord.name, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes) + 1); fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes + 1));
tstrncpy(tableRecord.metric, (char *)row[TSDB_SHOW_TABLES_METRIC_INDEX], tstrncpy(tableRecord.metric, (char *)row[TSDB_SHOW_TABLES_METRIC_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes) + 1); fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes + 1));
taosWrite(fd, &tableRecord, sizeof(STableRecord)); taosWrite(fd, &tableRecord, sizeof(STableRecord));
...@@ -1865,52 +1866,52 @@ static int taosDumpDb(SDbInfo *dbInfo, FILE *fp, TAOS *taosCon) { ...@@ -1865,52 +1866,52 @@ static int taosDumpDb(SDbInfo *dbInfo, FILE *fp, TAOS *taosCon) {
static void taosDumpCreateTableClause(STableDef *tableDes, int numOfCols, static void taosDumpCreateTableClause(STableDef *tableDes, int numOfCols,
FILE *fp, char* dbName) { FILE *fp, char* dbName) {
int counter = 0; int counter = 0;
int count_temp = 0; int count_temp = 0;
char sqlstr[COMMAND_SIZE]; char sqlstr[COMMAND_SIZE];
char* pstr = sqlstr; char* pstr = sqlstr;
pstr += sprintf(sqlstr, "CREATE TABLE IF NOT EXISTS %s.%s", pstr += sprintf(sqlstr, "CREATE TABLE IF NOT EXISTS %s.%s",
dbName, tableDes->name); dbName, tableDes->name);
for (; counter < numOfCols; counter++) { for (; counter < numOfCols; counter++) {
if (tableDes->cols[counter].note[0] != '\0') break; if (tableDes->cols[counter].note[0] != '\0') break;
if (counter == 0) { if (counter == 0) {
pstr += sprintf(pstr, " (%s %s", pstr += sprintf(pstr, " (%s %s",
tableDes->cols[counter].field, tableDes->cols[counter].type); tableDes->cols[counter].field, tableDes->cols[counter].type);
} else { } else {
pstr += sprintf(pstr, ", %s %s", pstr += sprintf(pstr, ", %s %s",
tableDes->cols[counter].field, tableDes->cols[counter].type); tableDes->cols[counter].field, tableDes->cols[counter].type);
} }
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 || if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) { strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length); pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length);
}
} }
}
count_temp = counter; count_temp = counter;
for (; counter < numOfCols; counter++) { for (; counter < numOfCols; counter++) {
if (counter == count_temp) { if (counter == count_temp) {
pstr += sprintf(pstr, ") TAGS (%s %s", pstr += sprintf(pstr, ") TAGS (%s %s",
tableDes->cols[counter].field, tableDes->cols[counter].type); tableDes->cols[counter].field, tableDes->cols[counter].type);
} else { } else {
pstr += sprintf(pstr, ", %s %s", pstr += sprintf(pstr, ", %s %s",
tableDes->cols[counter].field, tableDes->cols[counter].type); tableDes->cols[counter].field, tableDes->cols[counter].type);
} }
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 || if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) { strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length); pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length);
}
} }
}
pstr += sprintf(pstr, ");"); pstr += sprintf(pstr, ");");
fprintf(fp, "%s\n\n", sqlstr); fprintf(fp, "%s\n\n", sqlstr);
} }
static void taosDumpCreateMTableClause(STableDef *tableDes, char *metric, static void taosDumpCreateMTableClause(STableDef *tableDes, char *metric,
......
src/os/inc/osTime.h
浏览文件 @ b5dac67b
...@@ -97,7 +97,7 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision); ...@@ -97,7 +97,7 @@ int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision);
int64_t taosTimeTruncate(int64_t t, const SInterval* pInterval, int32_t precision); int64_t taosTimeTruncate(int64_t t, const SInterval* pInterval, int32_t precision);
int32_t taosTimeCountInterval(int64_t skey, int64_t ekey, int64_t interval, char unit, int32_t precision); int32_t taosTimeCountInterval(int64_t skey, int64_t ekey, int64_t interval, char unit, int32_t precision);
int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* ts, int32_t timePrecision); int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* ts, char* unit, int32_t timePrecision);
int32_t parseNatualDuration(const char* token, int32_t tokenLen, int64_t* duration, char* unit, int32_t timePrecision); int32_t parseNatualDuration(const char* token, int32_t tokenLen, int64_t* duration, char* unit, int32_t timePrecision);
int32_t taosParseTime(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t dayligth); int32_t taosParseTime(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t dayligth);
......
src/os/src/detail/osTime.c
浏览文件 @ b5dac67b
...@@ -397,7 +397,7 @@ static int32_t getDuration(int64_t val, char unit, int64_t* result, int32_t time ...@@ -397,7 +397,7 @@ static int32_t getDuration(int64_t val, char unit, int64_t* result, int32_t time
* n - Months (30 days) * n - Months (30 days)
* y - Years (365 days) * y - Years (365 days)
*/ */
int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* duration, int32_t timePrecision) { int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* duration, char* unit, int32_t timePrecision) {
errno = 0; errno = 0;
char* endPtr = NULL; char* endPtr = NULL;
...@@ -408,12 +408,12 @@ int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* duration, ...@@ -408,12 +408,12 @@ int32_t parseAbsoluteDuration(char* token, int32_t tokenlen, int64_t* duration,
} }
/* natual month/year are not allowed in absolute duration */ /* natual month/year are not allowed in absolute duration */
char unit = token[tokenlen - 1]; *unit = token[tokenlen - 1];
if (unit == 'n' || unit == 'y') { if (*unit == 'n' || *unit == 'y') {
return -1; return -1;
} }
return getDuration(timestamp, unit, duration, timePrecision); return getDuration(timestamp, *unit, duration, timePrecision);
} }
int32_t parseNatualDuration(const char* token, int32_t tokenLen, int64_t* duration, char* unit, int32_t timePrecision) { int32_t parseNatualDuration(const char* token, int32_t tokenLen, int64_t* duration, char* unit, int32_t timePrecision) {
......
src/query/inc/qTableMeta.h
浏览文件 @ b5dac67b
...@@ -71,7 +71,8 @@ typedef struct STableMeta { ...@@ -71,7 +71,8 @@ typedef struct STableMeta {
typedef struct STableMetaInfo { typedef struct STableMetaInfo {
STableMeta *pTableMeta; // table meta, cached in client side and acquired by name STableMeta *pTableMeta; // table meta, cached in client side and acquired by name
uint32_t tableMetaSize; uint32_t tableMetaSize;
size_t tableMetaCapacity;
SVgroupsInfo *vgroupList; SVgroupsInfo *vgroupList;
SArray *pVgroupTables; // SArray<SVgroupTableInfo> SArray *pVgroupTables; // SArray<SVgroupTableInfo>
......
src/query/inc/sql.y
浏览文件 @ b5dac67b
...@@ -479,7 +479,7 @@ tagitem(A) ::= PLUS(X) FLOAT(Y). { ...@@ -479,7 +479,7 @@ tagitem(A) ::= PLUS(X) FLOAT(Y). {
//////////////////////// The SELECT statement ///////////////////////////////// //////////////////////// The SELECT statement /////////////////////////////////
%type select {SSqlNode*} %type select {SSqlNode*}
%destructor select {destroySqlNode($$);} %destructor select {destroySqlNode($$);}
select(A) ::= SELECT(T) selcollist(W) from(X) where_opt(Y) interval_opt(K) session_option(H) windowstate_option(D) fill_opt(F) sliding_opt(S) groupby_opt(P) having_opt(N) orderby_opt(Z) slimit_opt(G) limit_opt(L). { select(A) ::= SELECT(T) selcollist(W) from(X) where_opt(Y) interval_opt(K) sliding_opt(S) session_option(H) windowstate_option(D) fill_opt(F)groupby_opt(P) having_opt(N) orderby_opt(Z) slimit_opt(G) limit_opt(L). {
A = tSetQuerySqlNode(&T, W, X, Y, P, Z, &K, &H, &D, &S, F, &L, &G, N); A = tSetQuerySqlNode(&T, W, X, Y, P, Z, &K, &H, &D, &S, F, &L, &G, N);
} }
......
src/query/src/qExecutor.c
浏览文件 @ b5dac67b
...@@ -6661,19 +6661,20 @@ static SSDataBlock* hashDistinct(void* param, bool* newgroup) { ...@@ -6661,19 +6661,20 @@ static SSDataBlock* hashDistinct(void* param, bool* newgroup) {
if (isNull(val, type)) { if (isNull(val, type)) {
continue; continue;
} }
char* p = val;
size_t keyLen = 0; size_t keyLen = 0;
if (IS_VAR_DATA_TYPE(pOperator->pExpr->base.colType)) { if (IS_VAR_DATA_TYPE(pOperator->pExpr->base.colType)) {
tstr* var = (tstr*)(val); tstr* var = (tstr*)(val);
p = var->data;
keyLen = varDataLen(var); keyLen = varDataLen(var);
} else { } else {
keyLen = bytes; keyLen = bytes;
} }
int dummy; int dummy;
void* res = taosHashGet(pInfo->pSet, val, keyLen); void* res = taosHashGet(pInfo->pSet, p, keyLen);
if (res == NULL) { if (res == NULL) {
taosHashPut(pInfo->pSet, val, keyLen, &dummy, sizeof(dummy)); taosHashPut(pInfo->pSet, p, keyLen, &dummy, sizeof(dummy));
char* start = pResultColInfoData->pData + bytes * pInfo->pRes->info.rows; char* start = pResultColInfoData->pData + bytes * pInfo->pRes->info.rows;
memcpy(start, val, bytes); memcpy(start, val, bytes);
pRes->info.rows += 1; pRes->info.rows += 1;
......
src/query/src/qPlan.c
浏览文件 @ b5dac67b
...@@ -32,8 +32,8 @@ typedef struct SJoinCond { ...@@ -32,8 +32,8 @@ typedef struct SJoinCond {
SColumn *colCond[2]; SColumn *colCond[2];
} SJoinCond; } SJoinCond;
static SQueryNode* createQueryNode(int32_t type, const char* name, SQueryNode** prev, static SQueryNode* createQueryNode(int32_t type, const char* name, SQueryNode** prev, int32_t numOfPrev,
int32_t numOfPrev, SExprInfo** pExpr, int32_t numOfOutput, SQueryTableInfo* pTableInfo, SExprInfo** pExpr, int32_t numOfOutput, SQueryTableInfo* pTableInfo,
void* pExtInfo) { void* pExtInfo) {
SQueryNode* pNode = calloc(1, sizeof(SQueryNode)); SQueryNode* pNode = calloc(1, sizeof(SQueryNode));
...@@ -112,8 +112,8 @@ static SQueryNode* doAddTableColumnNode(SQueryInfo* pQueryInfo, STableMetaInfo* ...@@ -112,8 +112,8 @@ static SQueryNode* doAddTableColumnNode(SQueryInfo* pQueryInfo, STableMetaInfo*
} }
STimeWindow* window = &pQueryInfo->window; STimeWindow* window = &pQueryInfo->window;
SQueryNode* pNode = createQueryNode(QNODE_TABLESCAN, "TableScan", NULL, 0, NULL, 0, SQueryNode* pNode = createQueryNode(QNODE_TABLESCAN, "TableScan", NULL, 0, NULL, 0, info, window);
info, window);
if (pQueryInfo->projectionQuery) { if (pQueryInfo->projectionQuery) {
int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs);
pNode = createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pExprs->pData, numOfOutput, info, NULL); pNode = createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pExprs->pData, numOfOutput, info, NULL);
...@@ -146,39 +146,41 @@ static SQueryNode* doAddTableColumnNode(SQueryInfo* pQueryInfo, STableMetaInfo* ...@@ -146,39 +146,41 @@ static SQueryNode* doAddTableColumnNode(SQueryInfo* pQueryInfo, STableMetaInfo*
} }
static SQueryNode* doCreateQueryPlanForOneTableImpl(SQueryInfo* pQueryInfo, SQueryNode* pNode, SQueryTableInfo* info, static SQueryNode* doCreateQueryPlanForOneTableImpl(SQueryInfo* pQueryInfo, SQueryNode* pNode, SQueryTableInfo* info,
SArray* pExprs) { SArray* pExprs) {
// check for aggregation // check for aggregation
if (pQueryInfo->interval.interval > 0) { if (pQueryInfo->interval.interval > 0) {
int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); int32_t numOfOutput = (int32_t)taosArrayGetSize(pExprs);
pNode = createQueryNode(QNODE_TIMEWINDOW, "TimeWindowAgg", &pNode, 1, pExprs->pData, numOfOutput, info, pNode = createQueryNode(QNODE_TIMEWINDOW, "TimeWindowAgg", &pNode, 1, pExprs->pData, numOfOutput, info,
&pQueryInfo->interval); &pQueryInfo->interval);
} else if (pQueryInfo->groupbyColumn) { if (pQueryInfo->groupbyExpr.numOfGroupCols != 0) {
int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); pNode = createQueryNode(QNODE_GROUPBY, "Groupby", &pNode, 1, pExprs->pData, numOfOutput, info, &pQueryInfo->groupbyExpr);
pNode = createQueryNode(QNODE_GROUPBY, "Groupby", &pNode, 1, pExprs->pData, numOfOutput, info, }
&pQueryInfo->groupbyExpr); } else if (pQueryInfo->groupbyColumn) {
} else if (pQueryInfo->sessionWindow.gap > 0) { int32_t numOfOutput = (int32_t)taosArrayGetSize(pExprs);
pNode = createQueryNode(QNODE_SESSIONWINDOW, "SessionWindowAgg", &pNode, 1, NULL, 0, info, NULL); pNode = createQueryNode(QNODE_GROUPBY, "Groupby", &pNode, 1, pExprs->pData, numOfOutput, info,
} else if (pQueryInfo->simpleAgg) { &pQueryInfo->groupbyExpr);
int32_t numOfOutput = (int32_t) taosArrayGetSize(pExprs); } else if (pQueryInfo->sessionWindow.gap > 0) {
pNode = createQueryNode(QNODE_AGGREGATE, "Aggregate", &pNode, 1, pExprs->pData, numOfOutput, info, NULL); pNode = createQueryNode(QNODE_SESSIONWINDOW, "SessionWindowAgg", &pNode, 1, NULL, 0, info, NULL);
} } else if (pQueryInfo->simpleAgg) {
int32_t numOfOutput = (int32_t)taosArrayGetSize(pExprs);
if (pQueryInfo->havingFieldNum > 0 || pQueryInfo->arithmeticOnAgg) { pNode = createQueryNode(QNODE_AGGREGATE, "Aggregate", &pNode, 1, pExprs->pData, numOfOutput, info, NULL);
int32_t numOfExpr = (int32_t) taosArrayGetSize(pQueryInfo->exprList1); }
pNode =
createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pQueryInfo->exprList1->pData, numOfExpr, info, NULL);
}
if (pQueryInfo->fillType != TSDB_FILL_NONE) { if (pQueryInfo->havingFieldNum > 0 || pQueryInfo->arithmeticOnAgg) {
SFillEssInfo* pInfo = calloc(1, sizeof(SFillEssInfo)); int32_t numOfExpr = (int32_t)taosArrayGetSize(pQueryInfo->exprList1);
pInfo->fillType = pQueryInfo->fillType; pNode =
pInfo->val = calloc(pNode->numOfOutput, sizeof(int64_t)); createQueryNode(QNODE_PROJECT, "Projection", &pNode, 1, pQueryInfo->exprList1->pData, numOfExpr, info, NULL);
memcpy(pInfo->val, pQueryInfo->fillVal, pNode->numOfOutput); }
pNode = createQueryNode(QNODE_FILL, "Fill", &pNode, 1, NULL, 0, info, pInfo); if (pQueryInfo->fillType != TSDB_FILL_NONE) {
} SFillEssInfo* pInfo = calloc(1, sizeof(SFillEssInfo));
pInfo->fillType = pQueryInfo->fillType;
pInfo->val = calloc(pNode->numOfOutput, sizeof(int64_t));
memcpy(pInfo->val, pQueryInfo->fillVal, pNode->numOfOutput);
pNode = createQueryNode(QNODE_FILL, "Fill", &pNode, 1, NULL, 0, info, pInfo);
}
if (pQueryInfo->limit.limit != -1 || pQueryInfo->limit.offset != 0) { if (pQueryInfo->limit.limit != -1 || pQueryInfo->limit.offset != 0) {
pNode = createQueryNode(QNODE_LIMIT, "Limit", &pNode, 1, NULL, 0, info, &pQueryInfo->limit); pNode = createQueryNode(QNODE_LIMIT, "Limit", &pNode, 1, NULL, 0, info, &pQueryInfo->limit);
...@@ -330,7 +332,7 @@ static int32_t doPrintPlan(char* buf, SQueryNode* pQueryNode, int32_t level, int ...@@ -330,7 +332,7 @@ static int32_t doPrintPlan(char* buf, SQueryNode* pQueryNode, int32_t level, int
switch(pQueryNode->info.type) { switch(pQueryNode->info.type) {
case QNODE_TABLESCAN: { case QNODE_TABLESCAN: {
STimeWindow* win = (STimeWindow*)pQueryNode->pExtInfo; STimeWindow* win = (STimeWindow*)pQueryNode->pExtInfo;
len1 = sprintf(buf + len, "%s #0x%" PRIx64 ") time_range: %" PRId64 " - %" PRId64 "\n", len1 = sprintf(buf + len, "%s #%" PRIu64 ") time_range: %" PRId64 " - %" PRId64 "\n",
pQueryNode->tableInfo.tableName, pQueryNode->tableInfo.id.uid, win->skey, win->ekey); pQueryNode->tableInfo.tableName, pQueryNode->tableInfo.id.uid, win->skey, win->ekey);
len += len1; len += len1;
break; break;
...@@ -401,8 +403,8 @@ static int32_t doPrintPlan(char* buf, SQueryNode* pQueryNode, int32_t level, int ...@@ -401,8 +403,8 @@ static int32_t doPrintPlan(char* buf, SQueryNode* pQueryNode, int32_t level, int
len += len1; len += len1;
SInterval* pInterval = pQueryNode->pExtInfo; SInterval* pInterval = pQueryNode->pExtInfo;
len1 = sprintf(buf + len, "interval:%" PRId64 "(%c), sliding:%" PRId64 "(%c), offset:%" PRId64 "\n", len1 = sprintf(buf + len, "interval:%" PRId64 "(%s), sliding:%" PRId64 "(%s), offset:%" PRId64 "\n",
pInterval->interval, pInterval->intervalUnit, pInterval->sliding, pInterval->slidingUnit, pInterval->interval, TSDB_TIME_PRECISION_MILLI_STR, pInterval->sliding, TSDB_TIME_PRECISION_MILLI_STR,
pInterval->offset); pInterval->offset);
len += len1; len += len1;
......
src/query/src/qSqlParser.c
浏览文件 @ b5dac67b
...@@ -166,7 +166,8 @@ tSqlExpr *tSqlExprCreateIdValue(SStrToken *pToken, int32_t optrType) { ...@@ -166,7 +166,8 @@ tSqlExpr *tSqlExprCreateIdValue(SStrToken *pToken, int32_t optrType) {
// use nanosecond by default // use nanosecond by default
// TODO set value after getting database precision // TODO set value after getting database precision
if (pToken) { if (pToken) {
int32_t ret = parseAbsoluteDuration(pToken->z, pToken->n, &pSqlExpr->value.i64, TSDB_TIME_PRECISION_NANO); char unit = 0;
int32_t ret = parseAbsoluteDuration(pToken->z, pToken->n, &pSqlExpr->value.i64, &unit, TSDB_TIME_PRECISION_NANO);
if (ret != TSDB_CODE_SUCCESS) { if (ret != TSDB_CODE_SUCCESS) {
terrno = TSDB_CODE_TSC_SQL_SYNTAX_ERROR; terrno = TSDB_CODE_TSC_SQL_SYNTAX_ERROR;
} }
......
src/query/src/sql.c
浏览文件 @ b5dac67b
...@@ -257,27 +257,27 @@ static const YYACTIONTYPE yy_action[] = { ...@@ -257,27 +257,27 @@ static const YYACTIONTYPE yy_action[] = {
/* 480 */ 208, 211, 205, 212, 213, 217, 218, 219, 216, 202, /* 480 */ 208, 211, 205, 212, 213, 217, 218, 219, 216, 202,
/* 490 */ 1143, 1082, 1135, 236, 267, 1079, 1078, 237, 338, 151, /* 490 */ 1143, 1082, 1135, 236, 267, 1079, 1078, 237, 338, 151,
/* 500 */ 1035, 1046, 47, 1065, 1043, 149, 1064, 1025, 1028, 1044, /* 500 */ 1035, 1046, 47, 1065, 1043, 149, 1064, 1025, 1028, 1044,
/* 510 */ 274, 1048, 153, 170, 157, 1009, 278, 283, 171, 1007, /* 510 */ 274, 1048, 153, 170, 158, 1009, 278, 285, 171, 1007,
/* 520 */ 172, 233, 166, 280, 161, 757, 160, 173, 162, 922, /* 520 */ 172, 233, 166, 280, 161, 757, 160, 173, 162, 922,
/* 530 */ 163, 299, 300, 301, 304, 305, 287, 292, 45, 290, /* 530 */ 163, 299, 300, 301, 304, 305, 282, 292, 45, 290,
/* 540 */ 75, 200, 288, 813, 272, 41, 72, 49, 316, 164, /* 540 */ 75, 200, 288, 813, 272, 41, 72, 49, 316, 164,
/* 550 */ 916, 323, 1142, 110, 1141, 1138, 286, 179, 330, 1134, /* 550 */ 916, 323, 1142, 110, 1141, 1138, 286, 179, 330, 1134,
/* 560 */ 284, 116, 1133, 1130, 180, 282, 942, 42, 39, 46, /* 560 */ 284, 116, 1133, 1130, 180, 281, 942, 42, 39, 46,
/* 570 */ 201, 904, 279, 126, 48, 902, 128, 129, 900, 899, /* 570 */ 201, 904, 279, 126, 303, 902, 128, 129, 900, 899,
/* 580 */ 259, 191, 897, 896, 895, 894, 893, 892, 891, 194, /* 580 */ 259, 191, 897, 896, 895, 894, 893, 892, 891, 194,
/* 590 */ 196, 888, 886, 884, 882, 198, 879, 199, 303, 81, /* 590 */ 196, 888, 886, 884, 882, 198, 879, 199, 48, 81,
/* 600 */ 86, 348, 281, 1066, 121, 340, 341, 342, 343, 344, /* 600 */ 86, 348, 283, 1066, 121, 340, 341, 342, 343, 344,
/* 610 */ 223, 345, 346, 356, 855, 243, 298, 260, 261, 854, /* 610 */ 223, 345, 346, 356, 855, 243, 298, 260, 261, 854,
/* 620 */ 263, 220, 221, 264, 853, 836, 104, 921, 920, 105, /* 620 */ 263, 220, 221, 264, 853, 836, 104, 921, 920, 105,
/* 630 */ 835, 268, 273, 10, 293, 734, 275, 84, 30, 87, /* 630 */ 835, 268, 273, 10, 293, 734, 275, 84, 30, 898,
/* 640 */ 898, 890, 182, 943, 186, 181, 184, 140, 183, 187, /* 640 */ 890, 183, 182, 943, 187, 181, 184, 185, 2, 140,
/* 650 */ 185, 141, 142, 889, 4, 143, 980, 881, 880, 944, /* 650 */ 186, 141, 142, 889, 4, 143, 980, 881, 87, 944,
/* 660 */ 759, 165, 167, 168, 155, 169, 762, 156, 2, 990, /* 660 */ 759, 165, 167, 168, 169, 880, 155, 157, 768, 156,
/* 670 */ 88, 235, 764, 89, 285, 31, 768, 158, 11, 12, /* 670 */ 235, 762, 88, 89, 990, 764, 287, 31, 11, 32,
/* 680 */ 13, 32, 27, 295, 28, 96, 98, 101, 35, 100, /* 680 */ 12, 13, 27, 295, 28, 96, 98, 101, 35, 100,
/* 690 */ 632, 36, 102, 667, 665, 664, 663, 661, 660, 659, /* 690 */ 632, 36, 102, 667, 665, 664, 663, 661, 660, 659,
/* 700 */ 656, 314, 622, 106, 7, 320, 812, 814, 8, 321, /* 700 */ 656, 314, 622, 106, 7, 320, 812, 321, 8, 109,
/* 710 */ 109, 111, 68, 69, 115, 704, 703, 38, 117, 700, /* 710 */ 814, 111, 68, 69, 115, 704, 703, 38, 117, 700,
/* 720 */ 648, 646, 638, 644, 640, 642, 636, 634, 670, 669, /* 720 */ 648, 646, 638, 644, 640, 642, 636, 634, 670, 669,
/* 730 */ 668, 666, 662, 658, 657, 190, 620, 585, 583, 859, /* 730 */ 668, 666, 662, 658, 657, 190, 620, 585, 583, 859,
/* 740 */ 858, 858, 858, 858, 858, 858, 858, 858, 858, 858, /* 740 */ 858, 858, 858, 858, 858, 858, 858, 858, 858, 858,
...@@ -338,24 +338,24 @@ static const YYCODETYPE yy_lookahead[] = { ...@@ -338,24 +338,24 @@ static const YYCODETYPE yy_lookahead[] = {
/* 510 */ 246, 199, 199, 250, 199, 246, 269, 199, 199, 199, /* 510 */ 246, 199, 199, 250, 199, 246, 269, 199, 199, 199,
/* 520 */ 199, 269, 254, 269, 259, 124, 260, 199, 258, 199, /* 520 */ 199, 269, 254, 269, 259, 124, 260, 199, 258, 199,
/* 530 */ 257, 199, 199, 199, 199, 199, 269, 130, 199, 134, /* 530 */ 257, 199, 199, 199, 199, 199, 269, 130, 199, 134,
/* 540 */ 136, 199, 129, 117, 200, 199, 138, 135, 199, 256, /* 540 */ 136, 199, 128, 117, 200, 199, 138, 135, 199, 256,
/* 550 */ 199, 199, 199, 199, 199, 199, 128, 199, 199, 199, /* 550 */ 199, 199, 199, 199, 199, 199, 127, 199, 199, 199,
/* 560 */ 127, 199, 199, 199, 199, 126, 199, 199, 199, 199, /* 560 */ 126, 199, 199, 199, 199, 129, 199, 199, 199, 199,
/* 570 */ 199, 199, 125, 199, 140, 199, 199, 199, 199, 199, /* 570 */ 199, 199, 125, 199, 89, 199, 199, 199, 199, 199,
/* 580 */ 199, 199, 199, 199, 199, 199, 199, 199, 199, 199, /* 580 */ 199, 199, 199, 199, 199, 199, 199, 199, 199, 199,
/* 590 */ 199, 199, 199, 199, 199, 199, 199, 199, 89, 200, /* 590 */ 199, 199, 199, 199, 199, 199, 199, 199, 140, 200,
/* 600 */ 200, 113, 200, 200, 96, 95, 51, 92, 94, 55, /* 600 */ 200, 113, 200, 200, 96, 95, 51, 92, 94, 55,
/* 610 */ 200, 93, 91, 84, 5, 200, 200, 153, 5, 5, /* 610 */ 200, 93, 91, 84, 5, 200, 200, 153, 5, 5,
/* 620 */ 153, 200, 200, 5, 5, 100, 206, 210, 210, 206, /* 620 */ 153, 200, 200, 5, 5, 100, 206, 210, 210, 206,
/* 630 */ 99, 142, 120, 82, 115, 83, 97, 121, 82, 97, /* 630 */ 99, 142, 120, 82, 115, 83, 97, 121, 82, 200,
/* 640 */ 200, 200, 217, 219, 215, 218, 216, 201, 213, 212, /* 640 */ 200, 213, 217, 219, 212, 218, 216, 214, 207, 201,
/* 650 */ 214, 201, 201, 200, 202, 201, 237, 200, 200, 221, /* 650 */ 215, 201, 201, 200, 202, 201, 237, 200, 97, 221,
/* 660 */ 83, 255, 253, 252, 82, 251, 83, 97, 207, 237, /* 660 */ 83, 255, 253, 252, 251, 200, 82, 97, 83, 82,
/* 670 */ 82, 1, 83, 82, 82, 97, 83, 82, 131, 131, /* 670 */ 1, 83, 82, 82, 237, 83, 82, 97, 131, 97,
/* 680 */ 82, 97, 82, 115, 82, 116, 78, 71, 87, 86, /* 680 */ 131, 82, 82, 115, 82, 116, 78, 71, 87, 86,
/* 690 */ 5, 87, 86, 9, 5, 5, 5, 5, 5, 5, /* 690 */ 5, 87, 86, 9, 5, 5, 5, 5, 5, 5,
/* 700 */ 5, 15, 85, 78, 82, 24, 83, 117, 82, 59, /* 700 */ 5, 15, 85, 78, 82, 24, 83, 59, 82, 147,
/* 710 */ 147, 147, 16, 16, 147, 5, 5, 97, 147, 83, /* 710 */ 117, 147, 16, 16, 147, 5, 5, 97, 147, 83,
/* 720 */ 5, 5, 5, 5, 5, 5, 5, 5, 5, 5, /* 720 */ 5, 5, 5, 5, 5, 5, 5, 5, 5, 5,
/* 730 */ 5, 5, 5, 5, 5, 97, 85, 61, 60, 0, /* 730 */ 5, 5, 5, 5, 5, 97, 85, 61, 60, 0,
/* 740 */ 276, 276, 276, 276, 276, 276, 276, 276, 276, 276, /* 740 */ 276, 276, 276, 276, 276, 276, 276, 276, 276, 276,
...@@ -399,9 +399,9 @@ static const unsigned short int yy_shift_ofst[] = { ...@@ -399,9 +399,9 @@ static const unsigned short int yy_shift_ofst[] = {
/* 120 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, /* 120 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143,
/* 130 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, /* 130 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143,
/* 140 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143, /* 140 */ 143, 143, 143, 143, 143, 143, 143, 143, 143, 143,
/* 150 */ 143, 444, 444, 444, 401, 401, 401, 444, 401, 444, /* 150 */ 143, 444, 444, 444, 401, 401, 401, 401, 444, 444,
/* 160 */ 404, 408, 407, 412, 405, 413, 428, 433, 439, 447, /* 160 */ 404, 408, 407, 412, 405, 414, 429, 434, 436, 447,
/* 170 */ 434, 444, 444, 444, 509, 509, 488, 12, 12, 444, /* 170 */ 458, 444, 444, 444, 485, 485, 488, 12, 12, 444,
/* 180 */ 444, 508, 510, 555, 515, 514, 554, 518, 521, 488, /* 180 */ 444, 508, 510, 555, 515, 514, 554, 518, 521, 488,
/* 190 */ 13, 444, 529, 529, 444, 529, 444, 529, 444, 444, /* 190 */ 13, 444, 529, 529, 444, 529, 444, 529, 444, 444,
/* 200 */ 753, 753, 54, 81, 81, 108, 81, 134, 188, 205, /* 200 */ 753, 753, 54, 81, 81, 108, 81, 134, 188, 205,
...@@ -411,12 +411,12 @@ static const unsigned short int yy_shift_ofst[] = { ...@@ -411,12 +411,12 @@ static const unsigned short int yy_shift_ofst[] = {
/* 240 */ 345, 347, 348, 343, 350, 357, 431, 375, 426, 366, /* 240 */ 345, 347, 348, 343, 350, 357, 431, 375, 426, 366,
/* 250 */ 118, 308, 314, 459, 460, 324, 327, 361, 331, 373, /* 250 */ 118, 308, 314, 459, 460, 324, 327, 361, 331, 373,
/* 260 */ 609, 464, 613, 614, 467, 618, 619, 525, 531, 489, /* 260 */ 609, 464, 613, 614, 467, 618, 619, 525, 531, 489,
/* 270 */ 512, 519, 551, 516, 552, 556, 539, 542, 577, 582, /* 270 */ 512, 519, 551, 516, 552, 556, 539, 561, 577, 584,
/* 280 */ 583, 570, 588, 589, 591, 670, 592, 593, 595, 578, /* 280 */ 585, 587, 588, 570, 590, 592, 591, 669, 594, 580,
/* 290 */ 547, 584, 548, 598, 519, 600, 568, 602, 569, 608, /* 290 */ 547, 582, 549, 599, 519, 600, 568, 602, 569, 608,
/* 300 */ 601, 603, 616, 685, 604, 606, 684, 689, 690, 691, /* 300 */ 601, 603, 616, 685, 604, 606, 684, 689, 690, 691,
/* 310 */ 692, 693, 694, 695, 617, 686, 625, 622, 623, 590, /* 310 */ 692, 693, 694, 695, 617, 686, 625, 622, 623, 593,
/* 320 */ 626, 681, 650, 696, 563, 564, 620, 620, 620, 620, /* 320 */ 626, 681, 648, 696, 562, 564, 620, 620, 620, 620,
/* 330 */ 697, 567, 571, 620, 620, 620, 710, 711, 636, 620, /* 330 */ 697, 567, 571, 620, 620, 620, 710, 711, 636, 620,
/* 340 */ 715, 716, 717, 718, 719, 720, 721, 722, 723, 724, /* 340 */ 715, 716, 717, 718, 719, 720, 721, 722, 723, 724,
/* 350 */ 725, 726, 727, 728, 729, 638, 651, 730, 731, 676, /* 350 */ 725, 726, 727, 728, 729, 638, 651, 730, 731, 676,
...@@ -424,7 +424,7 @@ static const unsigned short int yy_shift_ofst[] = { ...@@ -424,7 +424,7 @@ static const unsigned short int yy_shift_ofst[] = {
}; };
#define YY_REDUCE_COUNT (201) #define YY_REDUCE_COUNT (201)
#define YY_REDUCE_MIN (-265) #define YY_REDUCE_MIN (-265)
#define YY_REDUCE_MAX (461) #define YY_REDUCE_MAX (465)
static const short yy_reduce_ofst[] = { static const short yy_reduce_ofst[] = {
/* 0 */ -27, -33, -33, -193, -193, -76, -203, -199, -175, -184, /* 0 */ -27, -33, -33, -193, -193, -76, -203, -199, -175, -184,
/* 10 */ -130, -134, 93, 24, 67, 119, 126, 135, 142, 148, /* 10 */ -130, -134, 93, 24, 67, 119, 126, 135, 142, 148,
...@@ -441,12 +441,12 @@ static const short yy_reduce_ofst[] = { ...@@ -441,12 +441,12 @@ static const short yy_reduce_ofst[] = {
/* 120 */ 365, 367, 368, 369, 370, 371, 372, 374, 376, 377, /* 120 */ 365, 367, 368, 369, 370, 371, 372, 374, 376, 377,
/* 130 */ 378, 379, 380, 381, 382, 383, 384, 385, 386, 387, /* 130 */ 378, 379, 380, 381, 382, 383, 384, 385, 386, 387,
/* 140 */ 388, 389, 390, 391, 392, 393, 394, 395, 396, 397, /* 140 */ 388, 389, 390, 391, 392, 393, 394, 395, 396, 397,
/* 150 */ 398, 344, 399, 400, 247, 252, 254, 402, 267, 403, /* 150 */ 398, 344, 399, 400, 247, 252, 254, 267, 402, 403,
/* 160 */ 246, 266, 265, 270, 273, 293, 406, 268, 409, 411, /* 160 */ 246, 266, 265, 270, 273, 293, 406, 268, 409, 411,
/* 170 */ 414, 410, 415, 416, 417, 418, 419, 420, 423, 421, /* 170 */ 413, 410, 415, 416, 417, 418, 419, 420, 423, 421,
/* 180 */ 422, 424, 427, 425, 435, 430, 436, 429, 437, 432, /* 180 */ 422, 424, 427, 425, 428, 430, 433, 435, 432, 437,
/* 190 */ 438, 440, 446, 450, 441, 451, 453, 454, 457, 458, /* 190 */ 438, 439, 448, 450, 440, 451, 453, 454, 457, 465,
/* 200 */ 461, 452, /* 200 */ 441, 452,
}; };
static const YYACTIONTYPE yy_default[] = { static const YYACTIONTYPE yy_default[] = {
/* 0 */ 856, 979, 918, 989, 905, 915, 1126, 1126, 1126, 856, /* 0 */ 856, 979, 918, 989, 905, 915, 1126, 1126, 1126, 856,
...@@ -464,8 +464,8 @@ static const YYACTIONTYPE yy_default[] = { ...@@ -464,8 +464,8 @@ static const YYACTIONTYPE yy_default[] = {
/* 120 */ 856, 856, 856, 856, 856, 856, 903, 856, 901, 856, /* 120 */ 856, 856, 856, 856, 856, 856, 903, 856, 901, 856,
/* 130 */ 856, 856, 856, 856, 856, 856, 856, 856, 856, 856, /* 130 */ 856, 856, 856, 856, 856, 856, 856, 856, 856, 856,
/* 140 */ 856, 856, 856, 856, 887, 856, 856, 856, 856, 856, /* 140 */ 856, 856, 856, 856, 887, 856, 856, 856, 856, 856,
/* 150 */ 856, 878, 878, 878, 856, 856, 856, 878, 856, 878, /* 150 */ 856, 878, 878, 878, 856, 856, 856, 856, 878, 878,
/* 160 */ 1076, 1080, 1062, 1074, 1070, 1061, 1057, 1055, 1053, 1052, /* 160 */ 1076, 1080, 1062, 1074, 1070, 1057, 1055, 1053, 1061, 1052,
/* 170 */ 1084, 878, 878, 878, 923, 923, 919, 915, 915, 878, /* 170 */ 1084, 878, 878, 878, 923, 923, 919, 915, 915, 878,
/* 180 */ 878, 941, 939, 937, 929, 935, 931, 933, 927, 906, /* 180 */ 878, 941, 939, 937, 929, 935, 931, 933, 927, 906,
/* 190 */ 856, 878, 913, 913, 878, 913, 878, 913, 878, 878, /* 190 */ 856, 878, 913, 913, 878, 913, 878, 913, 878, 878,
...@@ -1039,10 +1039,10 @@ static const char *const yyTokenName[] = { ...@@ -1039,10 +1039,10 @@ static const char *const yyTokenName[] = {
/* 250 */ "from", /* 250 */ "from",
/* 251 */ "where_opt", /* 251 */ "where_opt",
/* 252 */ "interval_opt", /* 252 */ "interval_opt",
/* 253 */ "session_option", /* 253 */ "sliding_opt",
/* 254 */ "windowstate_option", /* 254 */ "session_option",
/* 255 */ "fill_opt", /* 255 */ "windowstate_option",
/* 256 */ "sliding_opt", /* 256 */ "fill_opt",
/* 257 */ "groupby_opt", /* 257 */ "groupby_opt",
/* 258 */ "having_opt", /* 258 */ "having_opt",
/* 259 */ "orderby_opt", /* 259 */ "orderby_opt",
...@@ -1235,7 +1235,7 @@ static const char *const yyRuleName[] = { ...@@ -1235,7 +1235,7 @@ static const char *const yyRuleName[] = {
/* 163 */ "tagitem ::= MINUS FLOAT", /* 163 */ "tagitem ::= MINUS FLOAT",
/* 164 */ "tagitem ::= PLUS INTEGER", /* 164 */ "tagitem ::= PLUS INTEGER",
/* 165 */ "tagitem ::= PLUS FLOAT", /* 165 */ "tagitem ::= PLUS FLOAT",
/* 166 */ "select ::= SELECT selcollist from where_opt interval_opt session_option windowstate_option fill_opt sliding_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt", /* 166 */ "select ::= SELECT selcollist from where_opt interval_opt sliding_opt session_option windowstate_option fill_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt",
/* 167 */ "select ::= LP select RP", /* 167 */ "select ::= LP select RP",
/* 168 */ "union ::= select", /* 168 */ "union ::= select",
/* 169 */ "union ::= union UNION ALL select", /* 169 */ "union ::= union UNION ALL select",
...@@ -1490,7 +1490,7 @@ tSqlExprListDestroy((yypminor->yy525)); ...@@ -1490,7 +1490,7 @@ tSqlExprListDestroy((yypminor->yy525));
case 243: /* columnlist */ case 243: /* columnlist */
case 244: /* tagitemlist */ case 244: /* tagitemlist */
case 245: /* tagNamelist */ case 245: /* tagNamelist */
case 255: /* fill_opt */ case 256: /* fill_opt */
case 257: /* groupby_opt */ case 257: /* groupby_opt */
case 259: /* orderby_opt */ case 259: /* orderby_opt */
case 270: /* sortlist */ case 270: /* sortlist */
...@@ -1991,7 +1991,7 @@ static const struct { ...@@ -1991,7 +1991,7 @@ static const struct {
{ 248, -2 }, /* (163) tagitem ::= MINUS FLOAT */ { 248, -2 }, /* (163) tagitem ::= MINUS FLOAT */
{ 248, -2 }, /* (164) tagitem ::= PLUS INTEGER */ { 248, -2 }, /* (164) tagitem ::= PLUS INTEGER */
{ 248, -2 }, /* (165) tagitem ::= PLUS FLOAT */ { 248, -2 }, /* (165) tagitem ::= PLUS FLOAT */
{ 246, -14 }, /* (166) select ::= SELECT selcollist from where_opt interval_opt session_option windowstate_option fill_opt sliding_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt */ { 246, -14 }, /* (166) select ::= SELECT selcollist from where_opt interval_opt sliding_opt session_option windowstate_option fill_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt */
{ 246, -3 }, /* (167) select ::= LP select RP */ { 246, -3 }, /* (167) select ::= LP select RP */
{ 262, -1 }, /* (168) union ::= select */ { 262, -1 }, /* (168) union ::= select */
{ 262, -4 }, /* (169) union ::= union UNION ALL select */ { 262, -4 }, /* (169) union ::= union UNION ALL select */
...@@ -2019,15 +2019,15 @@ static const struct { ...@@ -2019,15 +2019,15 @@ static const struct {
{ 252, -4 }, /* (191) interval_opt ::= INTERVAL LP tmvar RP */ { 252, -4 }, /* (191) interval_opt ::= INTERVAL LP tmvar RP */
{ 252, -6 }, /* (192) interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */ { 252, -6 }, /* (192) interval_opt ::= INTERVAL LP tmvar COMMA tmvar RP */
{ 252, 0 }, /* (193) interval_opt ::= */ { 252, 0 }, /* (193) interval_opt ::= */
{ 253, 0 }, /* (194) session_option ::= */ { 254, 0 }, /* (194) session_option ::= */
{ 253, -7 }, /* (195) session_option ::= SESSION LP ids cpxName COMMA tmvar RP */ { 254, -7 }, /* (195) session_option ::= SESSION LP ids cpxName COMMA tmvar RP */
{ 254, 0 }, /* (196) windowstate_option ::= */ { 255, 0 }, /* (196) windowstate_option ::= */
{ 254, -4 }, /* (197) windowstate_option ::= STATE_WINDOW LP ids RP */ { 255, -4 }, /* (197) windowstate_option ::= STATE_WINDOW LP ids RP */
{ 255, 0 }, /* (198) fill_opt ::= */ { 256, 0 }, /* (198) fill_opt ::= */
{ 255, -6 }, /* (199) fill_opt ::= FILL LP ID COMMA tagitemlist RP */ { 256, -6 }, /* (199) fill_opt ::= FILL LP ID COMMA tagitemlist RP */
{ 255, -4 }, /* (200) fill_opt ::= FILL LP ID RP */ { 256, -4 }, /* (200) fill_opt ::= FILL LP ID RP */
{ 256, -4 }, /* (201) sliding_opt ::= SLIDING LP tmvar RP */ { 253, -4 }, /* (201) sliding_opt ::= SLIDING LP tmvar RP */
{ 256, 0 }, /* (202) sliding_opt ::= */ { 253, 0 }, /* (202) sliding_opt ::= */
{ 259, 0 }, /* (203) orderby_opt ::= */ { 259, 0 }, /* (203) orderby_opt ::= */
{ 259, -3 }, /* (204) orderby_opt ::= ORDER BY sortlist */ { 259, -3 }, /* (204) orderby_opt ::= ORDER BY sortlist */
{ 270, -4 }, /* (205) sortlist ::= sortlist COMMA item sortorder */ { 270, -4 }, /* (205) sortlist ::= sortlist COMMA item sortorder */
...@@ -2723,9 +2723,9 @@ static void yy_reduce( ...@@ -2723,9 +2723,9 @@ static void yy_reduce(
} }
yymsp[-1].minor.yy506 = yylhsminor.yy506; yymsp[-1].minor.yy506 = yylhsminor.yy506;
break; break;
case 166: /* select ::= SELECT selcollist from where_opt interval_opt session_option windowstate_option fill_opt sliding_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt */ case 166: /* select ::= SELECT selcollist from where_opt interval_opt sliding_opt session_option windowstate_option fill_opt groupby_opt having_opt orderby_opt slimit_opt limit_opt */
{ {
yylhsminor.yy464 = tSetQuerySqlNode(&yymsp[-13].minor.yy0, yymsp[-12].minor.yy525, yymsp[-11].minor.yy412, yymsp[-10].minor.yy370, yymsp[-4].minor.yy525, yymsp[-2].minor.yy525, &yymsp[-9].minor.yy520, &yymsp[-8].minor.yy259, &yymsp[-7].minor.yy144, &yymsp[-5].minor.yy0, yymsp[-6].minor.yy525, &yymsp[0].minor.yy126, &yymsp[-1].minor.yy126, yymsp[-3].minor.yy370); yylhsminor.yy464 = tSetQuerySqlNode(&yymsp[-13].minor.yy0, yymsp[-12].minor.yy525, yymsp[-11].minor.yy412, yymsp[-10].minor.yy370, yymsp[-4].minor.yy525, yymsp[-2].minor.yy525, &yymsp[-9].minor.yy520, &yymsp[-7].minor.yy259, &yymsp[-6].minor.yy144, &yymsp[-8].minor.yy0, yymsp[-5].minor.yy525, &yymsp[0].minor.yy126, &yymsp[-1].minor.yy126, yymsp[-3].minor.yy370);
} }
yymsp[-13].minor.yy464 = yylhsminor.yy464; yymsp[-13].minor.yy464 = yylhsminor.yy464;
break; break;
......
src/tsdb/inc/tsdbRowMergeBuf.h
浏览文件 @ b5dac67b
...@@ -29,7 +29,9 @@ typedef void* SMergeBuf; ...@@ -29,7 +29,9 @@ typedef void* SMergeBuf;
SDataRow tsdbMergeTwoRows(SMergeBuf *pBuf, SMemRow row1, SMemRow row2, STSchema *pSchema1, STSchema *pSchema2); SDataRow tsdbMergeTwoRows(SMergeBuf *pBuf, SMemRow row1, SMemRow row2, STSchema *pSchema1, STSchema *pSchema2);
static FORCE_INLINE int tsdbMergeBufMakeSureRoom(SMergeBuf *pBuf, STSchema* pSchema1, STSchema* pSchema2) { static FORCE_INLINE int tsdbMergeBufMakeSureRoom(SMergeBuf *pBuf, STSchema* pSchema1, STSchema* pSchema2) {
return tsdbMakeRoom(pBuf, MAX(dataRowMaxBytesFromSchema(pSchema1), dataRowMaxBytesFromSchema(pSchema2))); size_t len1 = dataRowMaxBytesFromSchema(pSchema1);
size_t len2 = dataRowMaxBytesFromSchema(pSchema2);
return tsdbMakeRoom(pBuf, MAX(len1, len2));
} }
static FORCE_INLINE void tsdbFreeMergeBuf(SMergeBuf buf) { static FORCE_INLINE void tsdbFreeMergeBuf(SMergeBuf buf) {
......
src/tsdb/inc/tsdbint.h
浏览文件 @ b5dac67b
...@@ -94,8 +94,9 @@ struct STsdbRepo { ...@@ -94,8 +94,9 @@ struct STsdbRepo {
pthread_mutex_t mutex; pthread_mutex_t mutex;
bool repoLocked; bool repoLocked;
int32_t code; // Commit code int32_t code; // Commit code
SMergeBuf mergeBuf; //used when update=2 SMergeBuf mergeBuf; //used when update=2
bool inCompact; // is in compact process? int8_t compactState; // compact state: inCompact/noCompact/waitingCompact?
}; };
#define REPO_ID(r) (r)->config.tsdbId #define REPO_ID(r) (r)->config.tsdbId
......
src/tsdb/src/tsdbCompact.c
浏览文件 @ b5dac67b
...@@ -58,6 +58,7 @@ static int tsdbCompactFSetImpl(SCompactH *pComph); ...@@ -58,6 +58,7 @@ static int tsdbCompactFSetImpl(SCompactH *pComph);
static int tsdbWriteBlockToRightFile(SCompactH *pComph, STable *pTable, SDataCols *pDataCols, void **ppBuf, static int tsdbWriteBlockToRightFile(SCompactH *pComph, STable *pTable, SDataCols *pDataCols, void **ppBuf,
void **ppCBuf); void **ppCBuf);
enum { TSDB_NO_COMPACT, TSDB_IN_COMPACT, TSDB_WAITING_COMPACT};
int tsdbCompact(STsdbRepo *pRepo) { return tsdbAsyncCompact(pRepo); } int tsdbCompact(STsdbRepo *pRepo) { return tsdbAsyncCompact(pRepo); }
void *tsdbCompactImpl(STsdbRepo *pRepo) { void *tsdbCompactImpl(STsdbRepo *pRepo) {
...@@ -89,16 +90,21 @@ _err: ...@@ -89,16 +90,21 @@ _err:
} }
static int tsdbAsyncCompact(STsdbRepo *pRepo) { static int tsdbAsyncCompact(STsdbRepo *pRepo) {
if (pRepo->compactState != TSDB_NO_COMPACT) {
tsdbInfo("vgId:%d not compact tsdb again ", REPO_ID(pRepo));
return 0;
}
pRepo->compactState = TSDB_WAITING_COMPACT;
tsem_wait(&(pRepo->readyToCommit)); tsem_wait(&(pRepo->readyToCommit));
return tsdbScheduleCommit(pRepo, COMPACT_REQ); return tsdbScheduleCommit(pRepo, COMPACT_REQ);
} }
static void tsdbStartCompact(STsdbRepo *pRepo) { static void tsdbStartCompact(STsdbRepo *pRepo) {
ASSERT(!pRepo->inCompact); assert(pRepo->compactState != TSDB_IN_COMPACT);
tsdbInfo("vgId:%d start to compact!", REPO_ID(pRepo)); tsdbInfo("vgId:%d start to compact!", REPO_ID(pRepo));
tsdbStartFSTxn(pRepo, 0, 0); tsdbStartFSTxn(pRepo, 0, 0);
pRepo->code = TSDB_CODE_SUCCESS; pRepo->code = TSDB_CODE_SUCCESS;
pRepo->inCompact = true; pRepo->compactState = TSDB_IN_COMPACT;
} }
static void tsdbEndCompact(STsdbRepo *pRepo, int eno) { static void tsdbEndCompact(STsdbRepo *pRepo, int eno) {
...@@ -107,7 +113,7 @@ static void tsdbEndCompact(STsdbRepo *pRepo, int eno) { ...@@ -107,7 +113,7 @@ static void tsdbEndCompact(STsdbRepo *pRepo, int eno) {
} else { } else {
tsdbEndFSTxn(pRepo); tsdbEndFSTxn(pRepo);
} }
pRepo->inCompact = false; pRepo->compactState = TSDB_NO_COMPACT;
tsdbInfo("vgId:%d compact over, %s", REPO_ID(pRepo), (eno == TSDB_CODE_SUCCESS) ? "succeed" : "failed"); tsdbInfo("vgId:%d compact over, %s", REPO_ID(pRepo), (eno == TSDB_CODE_SUCCESS) ? "succeed" : "failed");
tsem_post(&(pRepo->readyToCommit)); tsem_post(&(pRepo->readyToCommit));
} }
......
src/tsdb/src/tsdbMain.c
浏览文件 @ b5dac67b
...@@ -200,7 +200,7 @@ STsdbRepoInfo *tsdbGetStatus(STsdbRepo *pRepo) { return NULL; } ...@@ -200,7 +200,7 @@ STsdbRepoInfo *tsdbGetStatus(STsdbRepo *pRepo) { return NULL; }
int tsdbGetState(STsdbRepo *repo) { return repo->state; } int tsdbGetState(STsdbRepo *repo) { return repo->state; }
bool tsdbInCompact(STsdbRepo *repo) { return repo->inCompact; } int8_t tsdbGetCompactState(STsdbRepo *repo) { return (int8_t)(repo->compactState); }
void tsdbReportStat(void *repo, int64_t *totalPoints, int64_t *totalStorage, int64_t *compStorage) { void tsdbReportStat(void *repo, int64_t *totalPoints, int64_t *totalStorage, int64_t *compStorage) {
ASSERT(repo != NULL); ASSERT(repo != NULL);
...@@ -541,7 +541,7 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH) { ...@@ -541,7 +541,7 @@ static STsdbRepo *tsdbNewRepo(STsdbCfg *pCfg, STsdbAppH *pAppH) {
pRepo->state = TSDB_STATE_OK; pRepo->state = TSDB_STATE_OK;
pRepo->code = TSDB_CODE_SUCCESS; pRepo->code = TSDB_CODE_SUCCESS;
pRepo->inCompact = false; pRepo->compactState = 0;
pRepo->config = *pCfg; pRepo->config = *pCfg;
if (pAppH) { if (pAppH) {
pRepo->appH = *pAppH; pRepo->appH = *pAppH;
......
src/tsdb/src/tsdbMeta.c
浏览文件 @ b5dac67b
...@@ -1035,6 +1035,8 @@ static void tsdbRemoveTableFromMeta(STsdbRepo *pRepo, STable *pTable, bool rmFro ...@@ -1035,6 +1035,8 @@ static void tsdbRemoveTableFromMeta(STsdbRepo *pRepo, STable *pTable, bool rmFro
} }
} }
} }
pMeta->maxCols = maxCols;
pMeta->maxRowBytes = maxRowBytes;
if (lock) tsdbUnlockRepoMeta(pRepo); if (lock) tsdbUnlockRepoMeta(pRepo);
tsdbDebug("vgId:%d table %s uid %" PRIu64 " is removed from meta", REPO_ID(pRepo), TABLE_CHAR_NAME(pTable), TABLE_UID(pTable)); tsdbDebug("vgId:%d table %s uid %" PRIu64 " is removed from meta", REPO_ID(pRepo), TABLE_CHAR_NAME(pTable), TABLE_UID(pTable));
......
src/tsdb/src/tsdbRead.c
浏览文件 @ b5dac67b
...@@ -1234,7 +1234,6 @@ static int32_t doCopyRowsFromFileBlock(STsdbQueryHandle* pQueryHandle, int32_t c ...@@ -1234,7 +1234,6 @@ static int32_t doCopyRowsFromFileBlock(STsdbQueryHandle* pQueryHandle, int32_t c
static void moveDataToFront(STsdbQueryHandle* pQueryHandle, int32_t numOfRows, int32_t numOfCols); static void moveDataToFront(STsdbQueryHandle* pQueryHandle, int32_t numOfRows, int32_t numOfCols);
static void doCheckGeneratedBlockRange(STsdbQueryHandle* pQueryHandle); static void doCheckGeneratedBlockRange(STsdbQueryHandle* pQueryHandle);
static void copyAllRemainRowsFromFileBlock(STsdbQueryHandle* pQueryHandle, STableCheckInfo* pCheckInfo, SDataBlockInfo* pBlockInfo, int32_t endPos); static void copyAllRemainRowsFromFileBlock(STsdbQueryHandle* pQueryHandle, STableCheckInfo* pCheckInfo, SDataBlockInfo* pBlockInfo, int32_t endPos);
static TSKEY extractFirstTraverseKey(STableCheckInfo* pCheckInfo, int32_t order, int32_t update);
static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* pBlock, STableCheckInfo* pCheckInfo){ static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* pBlock, STableCheckInfo* pCheckInfo){
SQueryFilePos* cur = &pQueryHandle->cur; SQueryFilePos* cur = &pQueryHandle->cur;
...@@ -1244,7 +1243,6 @@ static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* p ...@@ -1244,7 +1243,6 @@ static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* p
int32_t code = TSDB_CODE_SUCCESS; int32_t code = TSDB_CODE_SUCCESS;
/*bool hasData = */ initTableMemIterator(pQueryHandle, pCheckInfo); /*bool hasData = */ initTableMemIterator(pQueryHandle, pCheckInfo);
assert(cur->pos >= 0 && cur->pos <= binfo.rows); assert(cur->pos >= 0 && cur->pos <= binfo.rows);
key = extractFirstTraverseKey(pCheckInfo, pQueryHandle->order, pCfg->update); key = extractFirstTraverseKey(pCheckInfo, pQueryHandle->order, pCfg->update);
...@@ -1255,7 +1253,6 @@ static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* p ...@@ -1255,7 +1253,6 @@ static int32_t handleDataMergeIfNeeded(STsdbQueryHandle* pQueryHandle, SBlock* p
tsdbDebug("%p no data in mem, 0x%"PRIx64, pQueryHandle, pQueryHandle->qId); tsdbDebug("%p no data in mem, 0x%"PRIx64, pQueryHandle, pQueryHandle->qId);
} }
if ((ASCENDING_TRAVERSE(pQueryHandle->order) && (key != TSKEY_INITIAL_VAL && key <= binfo.window.ekey)) || if ((ASCENDING_TRAVERSE(pQueryHandle->order) && (key != TSKEY_INITIAL_VAL && key <= binfo.window.ekey)) ||
(!ASCENDING_TRAVERSE(pQueryHandle->order) && (key != TSKEY_INITIAL_VAL && key >= binfo.window.skey))) { (!ASCENDING_TRAVERSE(pQueryHandle->order) && (key != TSKEY_INITIAL_VAL && key >= binfo.window.skey))) {
...@@ -2708,6 +2705,7 @@ static bool loadBlockOfActiveTable(STsdbQueryHandle* pQueryHandle) { ...@@ -2708,6 +2705,7 @@ static bool loadBlockOfActiveTable(STsdbQueryHandle* pQueryHandle) {
} }
if (exists) { if (exists) {
tsdbRetrieveDataBlock((TsdbQueryHandleT*) pQueryHandle, NULL);
if (pQueryHandle->currentLoadExternalRows && pQueryHandle->window.skey == pQueryHandle->window.ekey) { if (pQueryHandle->currentLoadExternalRows && pQueryHandle->window.skey == pQueryHandle->window.ekey) {
SColumnInfoData* pColInfo = taosArrayGet(pQueryHandle->pColumns, 0); SColumnInfoData* pColInfo = taosArrayGet(pQueryHandle->pColumns, 0);
assert(*(int64_t*)pColInfo->pData == pQueryHandle->window.skey); assert(*(int64_t*)pColInfo->pData == pQueryHandle->window.skey);
......
src/tsdb/src/tsdbReadImpl.c
浏览文件 @ b5dac67b
...@@ -518,6 +518,8 @@ static int tsdbCheckAndDecodeColumnData(SDataCol *pDataCol, void *content, int32 ...@@ -518,6 +518,8 @@ static int tsdbCheckAndDecodeColumnData(SDataCol *pDataCol, void *content, int32
return -1; return -1;
} }
tdAllocMemForCol(pDataCol, maxPoints);
// Decode the data // Decode the data
if (comp) { if (comp) {
// Need to decompress // Need to decompress
......
src/util/inc/hash.h
浏览文件 @ b5dac67b
...@@ -127,6 +127,16 @@ void *taosHashGet(SHashObj *pHashObj, const void *key, size_t keyLen); ...@@ -127,6 +127,16 @@ void *taosHashGet(SHashObj *pHashObj, const void *key, size_t keyLen);
*/ */
void* taosHashGetClone(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void* d); void* taosHashGetClone(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void* d);
/**
* @param pHashObj
* @param key
* @param keyLen
* @param fp
* @param d
* @param sz
* @return
*/
void* taosHashGetCloneExt(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void** d, size_t *sz);
/** /**
* remove item with the specified key * remove item with the specified key
* @param pHashObj * @param pHashObj
......
src/util/src/hash.c
浏览文件 @ b5dac67b
...@@ -18,6 +18,8 @@ ...@@ -18,6 +18,8 @@
#include "tulog.h" #include "tulog.h"
#include "taosdef.h" #include "taosdef.h"
#define EXT_SIZE 1024
#define HASH_NEED_RESIZE(_h) ((_h)->size >= (_h)->capacity * HASH_DEFAULT_LOAD_FACTOR) #define HASH_NEED_RESIZE(_h) ((_h)->size >= (_h)->capacity * HASH_DEFAULT_LOAD_FACTOR)
#define DO_FREE_HASH_NODE(_n) \ #define DO_FREE_HASH_NODE(_n) \
...@@ -296,6 +298,68 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, void *da ...@@ -296,6 +298,68 @@ int32_t taosHashPut(SHashObj *pHashObj, const void *key, size_t keyLen, void *da
void *taosHashGet(SHashObj *pHashObj, const void *key, size_t keyLen) { void *taosHashGet(SHashObj *pHashObj, const void *key, size_t keyLen) {
return taosHashGetClone(pHashObj, key, keyLen, NULL, NULL); return taosHashGetClone(pHashObj, key, keyLen, NULL, NULL);
} }
//TODO(yihaoDeng), merge with taosHashGetClone
void* taosHashGetCloneExt(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void** d, size_t *sz) {
if (taosHashTableEmpty(pHashObj) || keyLen == 0 || key == NULL) {
return NULL;
}
uint32_t hashVal = (*pHashObj->hashFp)(key, (uint32_t)keyLen);
// only add the read lock to disable the resize process
__rd_lock(&pHashObj->lock, pHashObj->type);
int32_t slot = HASH_INDEX(hashVal, pHashObj->capacity);
SHashEntry *pe = pHashObj->hashList[slot];
// no data, return directly
if (atomic_load_32(&pe->num) == 0) {
__rd_unlock(&pHashObj->lock, pHashObj->type);
return NULL;
}
char *data = NULL;
// lock entry
if (pHashObj->type == HASH_ENTRY_LOCK) {
taosRLockLatch(&pe->latch);
}
if (pe->num > 0) {
assert(pe->next != NULL);
} else {
assert(pe->next == NULL);
}
SHashNode *pNode = doSearchInEntryList(pHashObj, pe, key, keyLen, hashVal);
if (pNode != NULL) {
if (fp != NULL) {
fp(GET_HASH_NODE_DATA(pNode));
}
if (*d == NULL) {
*sz = pNode->dataLen + EXT_SIZE;
*d = calloc(1, *sz);
} else if (*sz < pNode->dataLen){
*sz = pNode->dataLen + EXT_SIZE;
*d = realloc(*d, *sz);
}
memcpy((char *)(*d), GET_HASH_NODE_DATA(pNode), pNode->dataLen);
// just make runtime happy
if ((*sz) - pNode->dataLen > 0) {
memset((char *)(*d) + pNode->dataLen, 0, (*sz) - pNode->dataLen);
}
data = GET_HASH_NODE_DATA(pNode);
}
if (pHashObj->type == HASH_ENTRY_LOCK) {
taosRUnLockLatch(&pe->latch);
}
__rd_unlock(&pHashObj->lock, pHashObj->type);
return data;
}
void* taosHashGetClone(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void* d) { void* taosHashGetClone(SHashObj *pHashObj, const void *key, size_t keyLen, void (*fp)(void *), void* d) {
if (taosHashTableEmpty(pHashObj) || keyLen == 0 || key == NULL) { if (taosHashTableEmpty(pHashObj) || keyLen == 0 || key == NULL) {
......
src/vnode/inc/vnodeInt.h
浏览文件 @ b5dac67b
...@@ -26,6 +26,7 @@ extern "C" { ...@@ -26,6 +26,7 @@ extern "C" {
#include "vnode.h" #include "vnode.h"
extern int32_t vDebugFlag; extern int32_t vDebugFlag;
extern int32_t vNumOfExistedQHandle; // current initialized and existed query handle in current dnode
#define vFatal(...) { if (vDebugFlag & DEBUG_FATAL) { taosPrintLog("VND FATAL ", 255, __VA_ARGS__); }} #define vFatal(...) { if (vDebugFlag & DEBUG_FATAL) { taosPrintLog("VND FATAL ", 255, __VA_ARGS__); }}
#define vError(...) { if (vDebugFlag & DEBUG_ERROR) { taosPrintLog("VND ERROR ", 255, __VA_ARGS__); }} #define vError(...) { if (vDebugFlag & DEBUG_ERROR) { taosPrintLog("VND ERROR ", 255, __VA_ARGS__); }}
......
src/vnode/src/vnodeMgmt.c
浏览文件 @ b5dac67b
...@@ -162,7 +162,7 @@ static void vnodeBuildVloadMsg(SVnodeObj *pVnode, SStatusMsg *pStatus) { ...@@ -162,7 +162,7 @@ static void vnodeBuildVloadMsg(SVnodeObj *pVnode, SStatusMsg *pStatus) {
pLoad->status = pVnode->status; pLoad->status = pVnode->status;
pLoad->role = pVnode->role; pLoad->role = pVnode->role;
pLoad->replica = pVnode->syncCfg.replica; pLoad->replica = pVnode->syncCfg.replica;
pLoad->compact = (pVnode->tsdb != NULL) && tsdbInCompact(pVnode->tsdb) ? 1 : 0; pLoad->compact = (pVnode->tsdb != NULL) ? tsdbGetCompactState(pVnode->tsdb) : 0;
} }
int32_t vnodeGetVnodeList(int32_t vnodeList[], int32_t *numOfVnodes) { int32_t vnodeGetVnodeList(int32_t vnodeList[], int32_t *numOfVnodes) {
......
src/vnode/src/vnodeRead.c
浏览文件 @ b5dac67b
...@@ -21,6 +21,8 @@ ...@@ -21,6 +21,8 @@
#include "query.h" #include "query.h"
#include "vnodeStatus.h" #include "vnodeStatus.h"
int32_t vNumOfExistedQHandle; // current initialized and existed query handle in current dnode
static int32_t (*vnodeProcessReadMsgFp[TSDB_MSG_TYPE_MAX])(SVnodeObj *pVnode, SVReadMsg *pRead); static int32_t (*vnodeProcessReadMsgFp[TSDB_MSG_TYPE_MAX])(SVnodeObj *pVnode, SVReadMsg *pRead);
static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead); static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead);
static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead); static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead);
...@@ -247,7 +249,8 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -247,7 +249,8 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
if (handle == NULL) { // failed to register qhandle if (handle == NULL) { // failed to register qhandle
pRsp->code = terrno; pRsp->code = terrno;
terrno = 0; terrno = 0;
vError("vgId:%d, QInfo:0x%"PRIx64 "-%p register qhandle failed, return to app, code:%s", pVnode->vgId, qId, (void *)pQInfo,
vError("vgId:%d, QInfo:0x%"PRIx64 "-%p register qhandle failed, return to app, code:%s,", pVnode->vgId, qId, (void *)pQInfo,
tstrerror(pRsp->code)); tstrerror(pRsp->code));
qDestroyQueryInfo(pQInfo); // destroy it directly qDestroyQueryInfo(pQInfo); // destroy it directly
return pRsp->code; return pRsp->code;
...@@ -260,10 +263,12 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -260,10 +263,12 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
vnodeNotifyCurrentQhandle(pRead->rpcHandle, qId, *handle, pVnode->vgId) != TSDB_CODE_SUCCESS) { vnodeNotifyCurrentQhandle(pRead->rpcHandle, qId, *handle, pVnode->vgId) != TSDB_CODE_SUCCESS) {
vError("vgId:%d, QInfo:0x%"PRIx64 "-%p, query discarded since link is broken, %p", pVnode->vgId, qId, *handle, vError("vgId:%d, QInfo:0x%"PRIx64 "-%p, query discarded since link is broken, %p", pVnode->vgId, qId, *handle,
pRead->rpcHandle); pRead->rpcHandle);
pRsp->code = TSDB_CODE_RPC_NETWORK_UNAVAIL; pRsp->code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true);
return pRsp->code; return pRsp->code;
} }
} else { } else {
assert(pQInfo == NULL); assert(pQInfo == NULL);
} }
...@@ -277,6 +282,9 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -277,6 +282,9 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
return pRsp->code; return pRsp->code;
} }
} }
int32_t remain = atomic_add_fetch_32(&vNumOfExistedQHandle, 1);
vTrace("vgId:%d, new qhandle created, total qhandle:%d", pVnode->vgId, remain);
} else { } else {
assert(pCont != NULL); assert(pCont != NULL);
void **qhandle = (void **)pRead->qhandle; void **qhandle = (void **)pRead->qhandle;
...@@ -318,8 +326,14 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -318,8 +326,14 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
// NOTE: if the qhandle is not put into vread queue or query is completed, free the qhandle. // NOTE: if the qhandle is not put into vread queue or query is completed, free the qhandle.
// If the building of result is not required, simply free it. Otherwise, mandatorily free the qhandle // If the building of result is not required, simply free it. Otherwise, mandatorily free the qhandle
if (freehandle || (!buildRes)) { if (freehandle || (!buildRes)) {
if (freehandle) {
int32_t remain = atomic_sub_fetch_32(&vNumOfExistedQHandle, 1);
vTrace("vgId:%d, QInfo:%p, start to free qhandle, remain qhandle:%d", pVnode->vgId, *qhandle, remain);
}
qReleaseQInfo(pVnode->qMgmt, (void **)&qhandle, freehandle); qReleaseQInfo(pVnode->qMgmt, (void **)&qhandle, freehandle);
} }
} }
} }
...@@ -357,7 +371,10 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -357,7 +371,10 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
// kill current query and free corresponding resources. // kill current query and free corresponding resources.
if (pRetrieve->free == 1) { if (pRetrieve->free == 1) {
vWarn("vgId:%d, QInfo:%"PRIx64 "-%p, retrieve msg received to kill query and free qhandle", pVnode->vgId, pRetrieve->qId, *handle); int32_t remain = atomic_sub_fetch_32(&vNumOfExistedQHandle, 1);
vWarn("vgId:%d, QInfo:%"PRIx64 "-%p, retrieve msg received to kill query and free qhandle, remain qhandle:%d", pVnode->vgId, pRetrieve->qId,
*handle, remain);
qKillQuery(*handle); qKillQuery(*handle);
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true);
...@@ -368,7 +385,10 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -368,7 +385,10 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
// register the qhandle to connect to quit query immediate if connection is broken // register the qhandle to connect to quit query immediate if connection is broken
if (vnodeNotifyCurrentQhandle(pRead->rpcHandle, pRetrieve->qId, *handle, pVnode->vgId) != TSDB_CODE_SUCCESS) { if (vnodeNotifyCurrentQhandle(pRead->rpcHandle, pRetrieve->qId, *handle, pVnode->vgId) != TSDB_CODE_SUCCESS) {
vError("vgId:%d, QInfo:%"PRIu64 "-%p, retrieve discarded since link is broken, conn:%p", pVnode->vgId, pRetrieve->qhandle, *handle, pRead->rpcHandle); int32_t remain = atomic_sub_fetch_32(&vNumOfExistedQHandle, 1);
vError("vgId:%d, QInfo:%"PRIu64 "-%p, retrieve discarded since link is broken, conn:%p, remain qhandle:%d", pVnode->vgId, pRetrieve->qhandle,
*handle, pRead->rpcHandle, remain);
code = TSDB_CODE_RPC_NETWORK_UNAVAIL; code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
qKillQuery(*handle); qKillQuery(*handle);
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true);
...@@ -390,7 +410,6 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -390,7 +410,6 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
if (!tsRetrieveBlockingModel) { if (!tsRetrieveBlockingModel) {
if (!buildRes) { if (!buildRes) {
assert(pRead->rpcHandle != NULL); assert(pRead->rpcHandle != NULL);
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, false); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, false);
return TSDB_CODE_QRY_NOT_READY; return TSDB_CODE_QRY_NOT_READY;
} }
...@@ -403,6 +422,8 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) { ...@@ -403,6 +422,8 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SVReadMsg *pRead) {
// If qhandle is not added into vread queue, the query should be completed already or paused with error. // If qhandle is not added into vread queue, the query should be completed already or paused with error.
// Here free qhandle immediately // Here free qhandle immediately
if (freeHandle) { if (freeHandle) {
int32_t remain = atomic_sub_fetch_32(&vNumOfExistedQHandle, 1);
vTrace("vgId:%d, QInfo:%p, start to free qhandle, remain qhandle:%d", pVnode->vgId, *handle, remain);
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true); qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true);
} }
......
tests/pytest/fulltest.sh
浏览文件 @ b5dac67b
...@@ -148,6 +148,9 @@ python3 ./test.py -f import_merge/importTPORestart.py ...@@ -148,6 +148,9 @@ python3 ./test.py -f import_merge/importTPORestart.py
python3 ./test.py -f import_merge/importTRestart.py python3 ./test.py -f import_merge/importTRestart.py
python3 ./test.py -f import_merge/importInsertThenImport.py python3 ./test.py -f import_merge/importInsertThenImport.py
python3 ./test.py -f import_merge/importCSV.py python3 ./test.py -f import_merge/importCSV.py
python3 ./test.py -f import_merge/import_update_0.py
python3 ./test.py -f import_merge/import_update_1.py
python3 ./test.py -f import_merge/import_update_2.py
#======================p1-end=============== #======================p1-end===============
#======================p2-start=============== #======================p2-start===============
# tools # tools
...@@ -376,6 +379,7 @@ python3 test.py -f alter/alter_cacheLastRow.py ...@@ -376,6 +379,7 @@ python3 test.py -f alter/alter_cacheLastRow.py
python3 ./test.py -f query/querySession.py python3 ./test.py -f query/querySession.py
python3 test.py -f alter/alter_create_exception.py python3 test.py -f alter/alter_create_exception.py
python3 ./test.py -f insert/flushwhiledrop.py python3 ./test.py -f insert/flushwhiledrop.py
python3 ./test.py -f insert/schemalessInsert.py
#======================p4-end=============== #======================p4-end===============
python3 test.py -f tools/taosdemoAllTest/pytest.py python3 test.py -f tools/taosdemoAllTest/pytest.py
......
tests/pytest/import_merge/import_update_0.py 0 → 100644
浏览文件 @ b5dac67b
此差异已折叠。
tests/pytest/import_merge/import_update_1.py 0 → 100644
浏览文件 @ b5dac67b
此差异已折叠。
tests/pytest/import_merge/import_update_2.py 0 → 100644
浏览文件 @ b5dac67b
此差异已折叠。
tests/pytest/insert/schemalessInsert.py 0 → 100644
浏览文件 @ b5dac67b
此差异已折叠。
tests/pytest/query/operator.py 已删除 100644 → 0
浏览文件 @ 67201b16
此差异已折叠。
tests/pytest/util/sql.py
浏览文件 @ b5dac67b
...@@ -65,7 +65,7 @@ class TDSql: ...@@ -65,7 +65,7 @@ class TDSql:
self.queryResult = None self.queryResult = None
tdLog.info("sql:%s, expect error occured" % (sql)) tdLog.info("sql:%s, expect error occured" % (sql))
def query(self, sql): def query(self, sql, row_tag=None):
self.sql = sql self.sql = sql
try: try:
self.cursor.execute(sql) self.cursor.execute(sql)
...@@ -77,21 +77,27 @@ class TDSql: ...@@ -77,21 +77,27 @@ class TDSql:
args = (caller.filename, caller.lineno, sql, repr(e)) args = (caller.filename, caller.lineno, sql, repr(e))
tdLog.notice("%s(%d) failed: sql:%s, %s" % args) tdLog.notice("%s(%d) failed: sql:%s, %s" % args)
raise Exception(repr(e)) raise Exception(repr(e))
if row_tag:
return self.queryResult
return self.queryRows return self.queryRows
def getColNameList(self, sql): def getColNameList(self, sql, col_tag=None):
self.sql = sql self.sql = sql
try: try:
col_name_list = [] col_name_list = []
col_type_list = []
self.cursor.execute(sql) self.cursor.execute(sql)
self.queryCols = self.cursor.description self.queryCols = self.cursor.description
for query_col in self.queryCols: for query_col in self.queryCols:
col_name_list.append(query_col[0]) col_name_list.append(query_col[0])
col_type_list.append(query_col[1])
except Exception as e: except Exception as e:
caller = inspect.getframeinfo(inspect.stack()[1][0]) caller = inspect.getframeinfo(inspect.stack()[1][0])
args = (caller.filename, caller.lineno, sql, repr(e)) args = (caller.filename, caller.lineno, sql, repr(e))
tdLog.notice("%s(%d) failed: sql:%s, %s" % args) tdLog.notice("%s(%d) failed: sql:%s, %s" % args)
raise Exception(repr(e)) raise Exception(repr(e))
if col_tag:
return col_name_list, col_type_list
return col_name_list return col_name_list
def waitedQuery(self, sql, expectRows, timeout): def waitedQuery(self, sql, expectRows, timeout):
...@@ -245,6 +251,22 @@ class TDSql: ...@@ -245,6 +251,22 @@ class TDSql:
args = (caller.filename, caller.lineno, self.sql, col_name_list, expect_col_name_list) args = (caller.filename, caller.lineno, self.sql, col_name_list, expect_col_name_list)
tdLog.exit("%s(%d) failed: sql:%s, col_name_list:%s != expect_col_name_list:%s" % args) tdLog.exit("%s(%d) failed: sql:%s, col_name_list:%s != expect_col_name_list:%s" % args)
def checkEqual(self, elm, expect_elm):
if elm == expect_elm:
tdLog.info("sql:%s, elm:%s == expect_elm:%s" % (self.sql, elm, expect_elm))
else:
caller = inspect.getframeinfo(inspect.stack()[1][0])
args = (caller.filename, caller.lineno, self.sql, elm, expect_elm)
tdLog.exit("%s(%d) failed: sql:%s, elm:%s != expect_elm:%s" % args)
def checkNotEqual(self, elm, expect_elm):
if elm != expect_elm:
tdLog.info("sql:%s, elm:%s != expect_elm:%s" % (self.sql, elm, expect_elm))
else:
caller = inspect.getframeinfo(inspect.stack()[1][0])
args = (caller.filename, caller.lineno, self.sql, elm, expect_elm)
tdLog.exit("%s(%d) failed: sql:%s, elm:%s == expect_elm:%s" % args)
def taosdStatus(self, state): def taosdStatus(self, state):
tdLog.sleep(5) tdLog.sleep(5)
pstate = 0 pstate = 0
......
tests/script/general/field/2.sim
浏览文件 @ b5dac67b
...@@ -30,7 +30,7 @@ while $i < 5 ...@@ -30,7 +30,7 @@ while $i < 5
$x = 0 $x = 0
while $x < $rowNum while $x < $rowNum
$ms = $x . m $ms = $x . m
sql insert into $tb values (now + $ms , 0, 0 ) sql insert into $tb values (1626739200000 + $ms , 0, 0 )
$x = $x + 1 $x = $x + 1
endw endw
$i = $i + 1 $i = $i + 1
...@@ -41,7 +41,7 @@ while $i < 10 ...@@ -41,7 +41,7 @@ while $i < 10
$x = 0 $x = 0
while $x < $rowNum while $x < $rowNum
$ms = $x . m $ms = $x . m
sql insert into $tb values (now + $ms , 1, 1 ) sql insert into $tb values (1626739200000 + $ms , 1, 1 )
$x = $x + 1 $x = $x + 1
endw endw
$i = $i + 1 $i = $i + 1
...@@ -116,103 +116,104 @@ if $rows != 100 then ...@@ -116,103 +116,104 @@ if $rows != 100 then
endi endi
print =============== step4 print =============== step4
sql select * from $mt where ts > now + 4m and tbcol = 1 # sql select * from $mt where ts > 1626739440001 and tbcol = 1
sql select * from $mt where ts > 1626739440001 and tbcol = 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol <> 1 sql select * from $mt where ts > 1626739440001 and tbcol <> 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol = 0 sql select * from $mt where ts < 1626739440001 and tbcol = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol <> 0 sql select * from $mt where ts < 1626739440001 and tbcol <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol = 0 sql select * from $mt where ts <= 1626739440001 and tbcol = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol <> 0 sql select * from $mt where ts <= 1626739440001 and tbcol <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and ts < now + 5m and tbcol <> 0 sql select * from $mt where ts >= 1626739440001 and ts < 1626739500001 and tbcol <> 0
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol <> 0 and ts < now + 5m sql select * from $mt where ts >= 1626739440001 and tbcol <> 0 and ts < 1626739500001
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
print =============== step5 print =============== step5
sql select * from $mt where ts > now + 4m and tbcol2 = 1 sql select * from $mt where ts > 1626739440001 and tbcol2 = 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol2 <> 1 sql select * from $mt where ts > 1626739440001 and tbcol2 <> 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol2 = 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol2 <> 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol2 = 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol2 <> 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and ts < now + 5m and tbcol2 <> 0 sql select * from $mt where ts >= 1626739440001 and ts < 1626739500001 and tbcol2 <> 0
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol2 <> 0 and ts < now + 5m sql select * from $mt where ts >= 1626739440001 and tbcol2 <> 0 and ts < 1626739500001
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
print =============== step6 print =============== step6
sql select * from $mt where ts > now + 4m and tbcol2 = 1 and tbcol = 1 sql select * from $mt where ts > 1626739440001 and tbcol2 = 1 and tbcol = 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol2 <> 1 and tbcol <> 1 sql select * from $mt where ts > 1626739440001 and tbcol2 <> 1 and tbcol <> 1
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol2 = 0 and tbcol = 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 = 0 and tbcol = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts < now + 4m and tbcol2 <> 0 and tbcol <> 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 <> 0 and tbcol <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol2 = 0 and tbcol = 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 = 0 and tbcol = 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts <= now + 4m and tbcol2 <> 0 and tbcol <> 0 sql select * from $mt where ts <= 1626739440001 and tbcol2 <> 0 and tbcol <> 0
if $rows != 25 then if $rows != 25 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and ts < now + 5m and tbcol2 <> 0 and tbcol <> 0 sql select * from $mt where ts >= 1626739440001 and ts < 1626739500001 and tbcol2 <> 0 and tbcol <> 0
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol2 <> 0 and ts < now + 5m and ts < now + 5m and tbcol <> 0 sql select * from $mt where ts >= 1626739440001 and tbcol2 <> 0 and ts < 1626739500001 and ts < 1626739500001 and tbcol <> 0
if $rows != 5 then if $rows != 5 then
return -1 return -1
endi endi
...@@ -246,7 +247,7 @@ if $data00 != 100 then ...@@ -246,7 +247,7 @@ if $data00 != 100 then
endi endi
print =============== step9 print =============== step9
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts < now + 4m sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts <= 1626739440001
print $data00 $data01 $data02 $data03 $data04 $data05 $data06 print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data00 != 50 then if $data00 != 50 then
return -1 return -1
...@@ -272,7 +273,7 @@ if $data00 != 100 then ...@@ -272,7 +273,7 @@ if $data00 != 100 then
endi endi
print =============== step11 print =============== step11
sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts < now + 4m and tbcol = 1 group by tgcol sql select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts <= 1626739440001 and tbcol = 1 group by tgcol
print $data00 $data01 $data02 $data03 $data04 $data05 $data06 print $data00 $data01 $data02 $data03 $data04 $data05 $data06
if $data00 != 25 then if $data00 != 25 then
return -1 return -1
......
tests/script/general/field/3.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/field/4.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/field/5.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/field/6.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/field/binary.sim
浏览文件 @ b5dac67b
...@@ -30,7 +30,7 @@ while $i < 5 ...@@ -30,7 +30,7 @@ while $i < 5
$x = 0 $x = 0
while $x < $rowNum while $x < $rowNum
$ms = $x . m $ms = $x . m
sql insert into $tb values (now + $ms , '0' ) sql insert into $tb values (1626739200000 + $ms , '0' )
$x = $x + 1 $x = $x + 1
endw endw
$i = $i + 1 $i = $i + 1
...@@ -41,7 +41,7 @@ while $i < 10 ...@@ -41,7 +41,7 @@ while $i < 10
$x = 0 $x = 0
while $x < $rowNum while $x < $rowNum
$ms = $x . m $ms = $x . m
sql insert into $tb values (now + $ms , '1' ) sql insert into $tb values (1626739200000 + $ms , '1' )
$x = $x + 1 $x = $x + 1
endw endw
$i = $i + 1 $i = $i + 1
...@@ -55,14 +55,14 @@ if $rows != 100 then ...@@ -55,14 +55,14 @@ if $rows != 100 then
return -1 return -1
endi endi
sql select * from $mt where ts > now + 4m and tbcol = '1' sql select * from $mt where ts > 1626739440001 and tbcol = '1'
if $rows != 75 then if $rows != 75 then
return -1 return -1
endi endi
print select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1' print select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1'
sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1' group by tgcol sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1' group by tgcol
sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts < now + 4m and tbcol = '1' group by tgcol sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where ts < 1626739440001 and tbcol = '1' group by tgcol
sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1' interval(1d) group by tgcol sql_error select count(tbcol), avg(tbcol), sum(tbcol), min(tbcol), max(tbcol), first(tbcol), last(tbcol) from $mt where tbcol = '1' interval(1d) group by tgcol
#can't filter binary fields #can't filter binary fields
......
tests/script/general/field/bool.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/field/int.sim
浏览文件 @ b5dac67b
此差异已折叠。
tests/script/general/table/fill.sim 已删除 100644 → 0
浏览文件 @ 67201b16
此差异已折叠。
tests/script/jenkins/basic.txt
浏览文件 @ b5dac67b
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或