Skip to content

T
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1193
Star 22018
Fork 4786
T
TDengine
提交 574bfd96 编写于 作者: Z zhihaop
浏览文件
操作

Merge branch '2.6' into improve/taosc-async-enhancement-for-2.6

上级 8f5b3f48 f88270f0
隐藏空白更改
内联 并排
Showing with 3278 addition and 2792 deletion
docs/en/12-taos-sql/02-database.md
浏览文件 @ 574bfd96
......@@ -20,20 +20,20 @@ CREATE DATABASE [IF NOT EXISTS] db_name [KEEP keep] [DAYS days] [UPDATE 1];
3. The maximum length of database name is 33 bytes.
4. The maximum length of a SQL statement is 65,480 bytes.
5. Below are the parameters that can be used when creating a database
- cache: [Description](/reference/config/#cache)
- blocks: [Description](/reference/config/#blocks)
- days: [Description](/reference/config/#days)
- keep: [Description](/reference/config/#keep)
- minRows: [Description](/reference/config/#minrows)
- maxRows: [Description](/reference/config/#maxrows)
- wal: [Description](/reference/config/#wallevel)
- fsync: [Description](/reference/config/#fsync)
- update: [Description](/reference/config/#update)
- cacheLast: [Description](/reference/config/#cachelast)
- replica: [Description](/reference/config/#replica)
- quorum: [Description](/reference/config/#quorum)
- comp: [Description](/reference/config/#comp)
- precision: [Description](/reference/config/#precision)
- cache: [Description](../../reference/config/#cache)
- blocks: [Description](../../reference/config/#blocks)
- days: [Description](../../reference/config/#days)
- keep: [Description](../../reference/config/#keep)
- minRows: [Description](../../reference/config/#minrows)
- maxRows: [Description](../../reference/config/#maxrows)
- wal: [Description](../../reference/config/#wallevel)
- fsync: [Description](../../reference/config/#fsync)
- update: [Description](../../reference/config/#update)
- cacheLast: [Description](../../reference/config/#cachelast)
- replica: [Description](../../reference/config/#replica)
- quorum: [Description](../../reference/config/#quorum)
- comp: [Description](../../reference/config/#comp)
- precision: [Description](../../reference/config/#precision)
6. Please note that all of the parameters mentioned in this section are configured in configuration file `taos.cfg` on the TDengine server. If not specified in the `create database` statement, the values from taos.cfg are used by default. To override default parameters, they must be specified in the `create database` statement.
:::
......
docs/en/12-taos-sql/10-function.md
浏览文件 @ 574bfd96
......@@ -191,7 +191,7 @@ Query OK, 1 row(s) in set (0.000921s)
SELECT MODE(field_name) FROM tb_name [WHERE clause];
```
**Description**:The value which has the highest frequency of occurrence. NULL is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column or tags.
**Description**:The value which has the highest frequency of occurrence. One random value is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column or tags.
**Return value type**:Same as the data type of the column being operated upon
......
docs/en/14-reference/05-taosbenchmark.md
浏览文件 @ 574bfd96
......@@ -234,7 +234,7 @@ The parameters related to database creation are configured in `dbinfo` in the js
- **name**: specify the name of the database.
- **drop**: indicate whether to delete the database before inserting. The default is true.
- **drop**: indicate whether to delete the database before inserting. The value can be 'yes' or 'no'. No means do not drop. The default is to drop.
- **replica**: specify the number of replicas when creating the database.
......
docs/en/14-reference/_telegraf.mdx
浏览文件 @ 574bfd96
......@@ -22,5 +22,4 @@ An example is as follows.
username = "root"
password = "taosdata"
data_format = "influx"
influx_max_line_bytes = 250
```
docs/en/25-application/01-telegraf.md
浏览文件 @ 574bfd96
......@@ -60,7 +60,6 @@ For the configuration method, add the following text to `/etc/telegraf/telegraf.
username = "<TDengine's username>"
password = "<TDengine's password>"
data_format = "influx"
influx_max_line_bytes = 250
```
Then restart telegraf:
......
docs/zh/12-taos-sql/02-database.md
浏览文件 @ 574bfd96
......@@ -20,20 +20,20 @@ CREATE DATABASE [IF NOT EXISTS] db_name [KEEP keep] [DAYS days] [UPDATE 1];
3. 数据库名最大长度为 33；
4. 一条 SQL 语句的最大长度为 65480 个字符；
5. 创建数据库时可用的参数有：
- cache: [详细说明](/reference/config/#cache)
- blocks: [详细说明](/reference/config/#blocks)
- days: [详细说明](/reference/config/#days)
- keep: [详细说明](/reference/config/#keep)
- minRows: [详细说明](/reference/config/#minrows)
- maxRows: [详细说明](/reference/config/#maxrows)
- wal: [详细说明](/reference/config/#wallevel)
- fsync: [详细说明](/reference/config/#fsync)
- update: [详细说明](/reference/config/#update)
- cacheLast: [详细说明](/reference/config/#cachelast)
- replica: [详细说明](/reference/config/#replica)
- quorum: [详细说明](/reference/config/#quorum)
- comp: [详细说明](/reference/config/#comp)
- precision: [详细说明](/reference/config/#precision)
- cache: [详细说明](../../reference/config/#cache)
- blocks: [详细说明](../../reference/config/#blocks)
- days: [详细说明](../../reference/config/#days)
- keep: [详细说明](../../reference/config/#keep)
- minRows: [详细说明](../../reference/config/#minrows)
- maxRows: [详细说明](../../reference/config/#maxrows)
- wal: [详细说明](../../reference/config/#wallevel)
- fsync: [详细说明](../../reference/config/#fsync)
- update: [详细说明](../../reference/config/#update)
- cacheLast: [详细说明](../../reference/config/#cachelast)
- replica: [详细说明](../../reference/config/#replica)
- quorum: [详细说明](../../reference/config/#quorum)
- comp: [详细说明](../../reference/config/#comp)
- precision: [详细说明](../../reference/config/#precision)
6. 请注意上面列出的所有参数都可以配置在配置文件 `taosd.cfg` 中作为创建数据库时使用的默认配置， `create database` 的参数中明确指定的会覆盖配置文件中的设置。
:::
......@@ -86,7 +86,7 @@ REPLICA 参数是指修改数据库副本数，取值范围 [1, 3]。在集群
ALTER DATABASE db_name KEEP 365;
```
KEEP 参数是指修改数据文件保存的天数，缺省值为 3650，取值范围 [days, 365000]，必须大于或等于 days 参数值。
KEEP 参数是指修改数据文件保存的天数，缺省值为 3650，取值范围 [days, 36500]，必须大于或等于 days 参数值。
```
ALTER DATABASE db_name QUORUM 2;
......
docs/zh/12-taos-sql/10-function.md
浏览文件 @ 574bfd96
......@@ -193,7 +193,7 @@ Query OK, 1 row(s) in set (0.000921s)
SELECT MODE(field_name) FROM tb_name [WHERE clause];
```
**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，输出空。不能匹配标签、时间戳输出。
**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，则随机输出其中某个值。不能匹配标签、时间戳输出。
**返回数据类型**：同应用的字段。
......
docs/zh/14-reference/05-taosbenchmark.md
浏览文件 @ 574bfd96
......@@ -231,7 +231,7 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
- **name** : 数据库名。
- **drop** : 插入前是否删除数据库，默认为 true。
- **drop** : 插入前是否删除数据库，可选项为 "yes" 或者 "no", 为 "no" 时不创建。默认删除。
- **replica** : 创建数据库时指定的副本数。
......
docs/zh/14-reference/_telegraf.mdx
浏览文件 @ 574bfd96
......@@ -22,6 +22,5 @@
username = "root"
password = "taosdata"
data_format = "influx"
influx_max_line_bytes = 250
```
docs/zh/25-application/01-telegraf.md
浏览文件 @ 574bfd96
......@@ -60,7 +60,6 @@ IT 运维监测数据通常都是对时间特性比较敏感的数据，例如
username = "<TDengine's username>"
password = "<TDengine's password>"
data_format = "influx"
influx_max_line_bytes = 250
```
然后重启 Telegraf：
......
packaging/docker/bin/entrypoint.sh
浏览文件 @ 574bfd96
......@@ -35,6 +35,9 @@ FIRST_EP_PORT=${TAOS_FIRST_EP#*:}
SERVER_PORT=$(cat $CFG_FILE | grep -E -v "^#|^$" | grep serverPort | tail -n1 | sed -E 's/.*serverPort\s+//')
SERVER_PORT=${SERVER_PORT:-6030}
# parse dataDir from taos.cfg
DATA_DIR=$(cat $CFG_FILE | grep -E -v "^#|^$" | grep dataDir | tail -n1 | sed -E 's/.*dataDir\s+//')
# for other binaries like interpreters
if echo $1 | grep -E "taosd$" - >/dev/null; then
true # will run taosd
......
packaging/release.sh
浏览文件 @ 574bfd96
......@@ -3,7 +3,7 @@
# Generate the deb package for ubuntu, or rpm package for centos, or tar.gz package for other linux os
set -e
#set -x
# set -x
# release.sh -v [cluster | edge]
# -c [aarch32 | aarch64 | x64 | x86 | mips64 ...]
......@@ -263,7 +263,7 @@ if [ "$osType" != "Darwin" ]; then
if [[ "$pagMode" == "full" ]]; then
if [ -d ${top_dir}/src/kit/taos-tools/packaging/deb ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/deb
taos_tools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}')
taos_tools_ver=$(git tag |grep -v taos | sort | tail -1)
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-deb.sh ${top_dir} \
......@@ -288,7 +288,7 @@ if [ "$osType" != "Darwin" ]; then
if [[ "$pagMode" == "full" ]]; then
if [ -d ${top_dir}/src/kit/taos-tools/packaging/rpm ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/rpm
taos_tools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}' | sed -e 's/-/_/g')
taos_tools_ver=$(git tag |grep -v taos | sort | tail -1)
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-rpm.sh ${top_dir} \
......
packaging/tools/install.sh
浏览文件 @ 574bfd96
......@@ -105,6 +105,10 @@ elif echo $osinfo | grep -qwi "debian"; then
elif echo $osinfo | grep -qwi "Kylin"; then
# echo "This is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "KylinSec"; then
# echo "This is KylinSec system"
os_type=2
csudo=""
elif echo $osinfo | grep -qwi "Red"; then
# echo "This is Red Hat system"
os_type=1
......
packaging/tools/makepkg.sh
浏览文件 @ 574bfd96
......@@ -47,7 +47,7 @@ if [ -d ${top_dir}/src/kit/taos-tools/packaging/deb ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/deb
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
taostools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}')
taostools_ver=$(git tag |grep -v taos | sort | tail -1)
taostools_install_dir="${release_dir}/${clientName}Tools-${taostools_ver}"
cd ${curr_dir}
......
src/client/inc/tscParseLine.h
浏览文件 @ 574bfd96
......@@ -103,11 +103,11 @@ int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
void destroySmlDataPoint(TAOS_SML_DATA_POINT* point);
int taos_insert_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol,
int taos_insert_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol,
SMLTimeStampType tsType, int* affectedRows);
int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol,
int taos_insert_telnet_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol,
SMLTimeStampType tsType, int* affectedRows);
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol,
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, int *numLines,
SMLTimeStampType tsType, int* affectedRows);
......
src/client/src/tscParseLineProtocol.c
浏览文件 @ 574bfd96
......@@ -577,7 +577,9 @@ static int32_t getSuperTableMetaFromLocalCache(TAOS* taos, char* tableName, STab
// Check if the table name available or not
if (tscValidateName(&tableToken, true, &dbIncluded) != TSDB_CODE_SUCCESS) {
code = TSDB_CODE_TSC_INVALID_TABLE_ID_LENGTH;
sprintf(pSql->cmd.payload, "table name is invalid");
if(pSql->cmd.payload){
sprintf(pSql->cmd.payload, "table name is invalid");
}
taosReleaseRef(tscObjRef, pSql->self);
return code;
}
......@@ -755,25 +757,25 @@ static int32_t arrangePointsByChildTableName(TAOS_SML_DATA_POINT* points, int nu
SHashObj* cname2points, SArray* stableSchemas, SSmlLinesInfo* info) {
for (int32_t i = 0; i < numPoints; ++i) {
TAOS_SML_DATA_POINT * point = points + i;
SSmlSTableSchema* stableSchema = taosArrayGet(stableSchemas, point->schemaIdx);
for (int j = 0; j < point->tagNum; ++j) {
TAOS_SML_KV* kv = point->tags + j;
if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t ts = *(int64_t*)(kv->value);
ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, stableSchema->precision);
*(int64_t*)(kv->value) = ts;
}
}
for (int j = 0; j < point->fieldNum; ++j) {
TAOS_SML_KV* kv = point->fields + j;
if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t ts = *(int64_t*)(kv->value);
ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, stableSchema->precision);
*(int64_t*)(kv->value) = ts;
}
}
// SSmlSTableSchema* stableSchema = taosArrayGet(stableSchemas, point->schemaIdx);
// for (int j = 0; j < point->tagNum; ++j) {
// TAOS_SML_KV* kv = point->tags + j;
// if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
// int64_t ts = *(int64_t*)(kv->value);
// ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, stableSchema->precision);
// *(int64_t*)(kv->value) = ts;
// }
// }
//
// for (int j = 0; j < point->fieldNum; ++j) {
// TAOS_SML_KV* kv = point->fields + j;
// if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
// int64_t ts = *(int64_t*)(kv->value);
// ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, stableSchema->precision);
// *(int64_t*)(kv->value) = ts;
// }
// }
SArray* cTablePoints = NULL;
SArray** pCTablePoints = taosHashGet(cname2points, point->childTableName, strlen(point->childTableName));
......@@ -858,7 +860,14 @@ static int32_t addChildTableDataPointsToInsertSql(char* cTableName, char* sTable
TAOS_SML_KV* kv = tagKVs[i];
size_t beforeLen = totalLen;
int32_t len = 0;
converToStr(sql + beforeLen, kv->type, kv->value, kv->length, &len);
if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t ts = *(int64_t*)(kv->value);
ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, sTableSchema->precision);
converToStr(sql + beforeLen, kv->type, &ts, kv->length, &len);
}else{
converToStr(sql + beforeLen, kv->type, kv->value, kv->length, &len);
}
totalLen += len;
ret = smlSnprintf(sql, &totalLen, capacity, ",");
......@@ -912,7 +921,13 @@ static int32_t addChildTableDataPointsToInsertSql(char* cTableName, char* sTable
TAOS_SML_KV* kv = colKVs[i];
size_t beforeLen = totalLen;
int32_t len = 0;
converToStr(sql + beforeLen, kv->type, kv->value, kv->length, &len);
if (kv->type == TSDB_DATA_TYPE_TIMESTAMP) {
int64_t ts = *(int64_t*)(kv->value);
ts = convertTimePrecision(ts, TSDB_TIME_PRECISION_NANO, sTableSchema->precision);
converToStr(sql + beforeLen, kv->type, &ts, kv->length, &len);
}else{
converToStr(sql + beforeLen, kv->type, kv->value, kv->length, &len);
}
totalLen += len;
ret = smlSnprintf(sql, &totalLen, capacity, ",");
if (ret != 0) {
......@@ -963,7 +978,7 @@ static void insertCallback(void *param, TAOS_RES *res, int32_t notUsedCode) {
batch->tryAgain = true;
}
if (code == TSDB_CODE_TDB_INVALID_TABLE_ID || code == TSDB_CODE_VND_INVALID_VGROUP_ID) {
if (code == TSDB_CODE_TDB_INVALID_TABLE_ID || code == TSDB_CODE_VND_INVALID_VGROUP_ID || code == TSDB_CODE_MND_INVALID_TABLE_NAME) {
batch->resetQueryCache = true;
if (batch->tryAgain) {
batch->sleep = true;
......@@ -1103,6 +1118,9 @@ static int32_t applyDataPointsWithSqlInsert(TAOS* taos, TAOS_SML_DATA_POINT* poi
SSmlSqlInsertBatch* b = info->batches + i;
if (b->code != 0) {
code = b->code;
if(code == TSDB_CODE_MND_INVALID_TABLE_NAME){
break;
}
}
}
......@@ -1219,42 +1237,52 @@ int tscSmlInsert(TAOS* taos, TAOS_SML_DATA_POINT* points, int numPoint, SSmlLine
info->affectedRows = 0;
if (numPoint == 1) {
TAOS_SML_DATA_POINT* point = points + 0;
code = doSmlInsertOneDataPoint(taos, point, info);
if (code == TSDB_CODE_SUCCESS) {
return code;
int32_t tableNotExistTryTimes = 3;
while(tableNotExistTryTimes){
if (numPoint == 1) {
TAOS_SML_DATA_POINT* point = points + 0;
code = doSmlInsertOneDataPoint(taos, point, info);
if (code == TSDB_CODE_SUCCESS) {
return code;
}
}
}
tscDebug("SML:0x%"PRIx64" build data point schemas", info->id);
SArray* stableSchemas = taosArrayInit(32, sizeof(SSmlSTableSchema)); // SArray<STableColumnsSchema>
code = buildDataPointSchemas(points, numPoint, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error building data point schemas : %s", info->id, tstrerror(code));
goto clean_up;
}
tscDebug("SML:0x%"PRIx64" build data point schemas", info->id);
SArray* stableSchemas = taosArrayInit(32, sizeof(SSmlSTableSchema)); // SArray<STableColumnsSchema>
code = buildDataPointSchemas(points, numPoint, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error building data point schemas : %s", info->id, tstrerror(code));
goto clean_up;
}
tscDebug("SML:0x%"PRIx64" modify db schemas", info->id);
code = modifyDBSchemas(taos, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error change db schema : %s", info->id, tstrerror(code));
goto clean_up;
}
tscDebug("SML:0x%"PRIx64" modify db schemas", info->id);
code = modifyDBSchemas(taos, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error change db schema : %s", info->id, tstrerror(code));
goto clean_up;
}
tscDebug("SML:0x%"PRIx64" apply data points", info->id);
code = applyDataPointsWithSqlInsert(taos, points, numPoint, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error apply data points : %s", info->id, tstrerror(code));
}
tscDebug("SML:0x%"PRIx64" apply data points", info->id);
code = applyDataPointsWithSqlInsert(taos, points, numPoint, stableSchemas, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" error apply data points : %s", info->id, tstrerror(code));
}
clean_up:
for (int i = 0; i < taosArrayGetSize(stableSchemas); ++i) {
SSmlSTableSchema* schema = taosArrayGet(stableSchemas, i);
taosArrayDestroy(&schema->fields);
taosArrayDestroy(&schema->tags);
clean_up:
for (int i = 0; i < taosArrayGetSize(stableSchemas); ++i) {
SSmlSTableSchema* schema = taosArrayGet(stableSchemas, i);
taosArrayDestroy(&schema->fields);
taosArrayDestroy(&schema->tags);
}
taosArrayDestroy(&stableSchemas);
if(code == TSDB_CODE_MND_INVALID_TABLE_NAME){
tableNotExistTryTimes--;
}else{
tableNotExistTryTimes = 0;
}
}
taosArrayDestroy(&stableSchemas);
return code;
}
......@@ -1966,24 +1994,19 @@ int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
return TSDB_CODE_SUCCESS;
}
static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **idx, SSmlLinesInfo* info) {
const char *start, *cur;
static int32_t parseSmlTimeStamp(TAOS_SML_KV **pTS, const char **idx, int32_t len, SSmlLinesInfo* info) {
const char *start;
int32_t ret = TSDB_CODE_SUCCESS;
int len = 0;
char key[] = "ts";
char *value = NULL;
start = cur = *idx;
start = *idx;
*pTS = calloc(1, sizeof(TAOS_SML_KV));
while(*cur != '\0') {
cur++;
len++;
}
if (len > 0) {
value = calloc(len + 1, 1);
memcpy(value, start, len);
len = strtrim(value);
}
ret = convertSmlTimeStamp(*pTS, value, len, info);
......@@ -2013,12 +2036,12 @@ bool checkDuplicateKey(char *key, SHashObj *pHash, SSmlLinesInfo* info) {
return false;
}
static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *pHash, SSmlLinesInfo* info) {
static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen, SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *idx;
char key[TSDB_COL_NAME_LEN + 1]; // +1 to avoid key[len] over write
int16_t len = 0;
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_COL_NAME_LEN - 1) {
tscError("SML:0x%"PRIx64" Key field cannot exceeds %d characters", info->id, TSDB_COL_NAME_LEN - 1);
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
......@@ -2053,7 +2076,7 @@ static int32_t parseSmlKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *pHash,
}
static int32_t parseSmlValue(TAOS_SML_KV *pKV, const char **idx,
static int32_t parseSmlValue(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen,
bool *is_last_kv, SSmlLinesInfo* info, bool isTag) {
const char *start, *cur;
int32_t ret = TSDB_CODE_SUCCESS;
......@@ -2163,13 +2186,13 @@ static int32_t parseSmlValue(TAOS_SML_KV *pKV, const char **idx,
cur++;
break;
} else if (double_quote == true) {
if (*cur != ' ' && *cur != ',' && *cur != '\0') {
if (*cur != ' ' && *cur != ',' && (cur - *idx < sqlLen)) {
tscError("SML:0x%"PRIx64" tag value: state(%d), incorrect character(%c) behind closing \"", info->id, tag_state, *cur);
ret = TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
goto error;
}
if (*cur == ' ' || *cur == '\0') {
if (*cur == ' ' || (cur - *idx == sqlLen)) {
*is_last_kv = true;
}
......@@ -2308,13 +2331,13 @@ static int32_t parseSmlValue(TAOS_SML_KV *pKV, const char **idx,
cur++;
break;
} else if (double_quote == true) {
if (*cur != ' ' && *cur != ',' && *cur != '\0') {
if (*cur != ' ' && *cur != ',' && (cur - *idx < sqlLen)) {
tscError("SML:0x%"PRIx64" field value: state(%d), incorrect character(%c) behind closing \"", info->id, val_state, *cur);
ret = TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
goto error;
}
if (*cur == ' ' || *cur == '\0') {
if (*cur == ' ' || (cur - *idx == sqlLen)) {
*is_last_kv = true;
}
......@@ -2384,7 +2407,7 @@ static int32_t parseSmlValue(TAOS_SML_KV *pKV, const char **idx,
}
free(value);
*idx = (*cur == '\0') ? cur : cur + 1;
*idx = (cur - *idx == sqlLen) ? cur : cur + 1;
return ret;
error:
......@@ -2395,7 +2418,7 @@ error:
return ret;
}
static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **idx,
static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **idx, int32_t sqlLen,
uint8_t *has_tags, SSmlLinesInfo* info) {
const char *cur = *idx;
int16_t len = 0;
......@@ -2405,7 +2428,7 @@ static int32_t parseSmlMeasurement(TAOS_SML_DATA_POINT *pSml, const char **idx,
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_TABLE_NAME_LEN - 1) {
tscError("SML:0x%"PRIx64" Measurement field cannot exceeds %d characters", info->id, TSDB_TABLE_NAME_LEN - 1);
free(pSml->stableName);
......@@ -2464,7 +2487,7 @@ int32_t isValidChildTableName(const char *pTbName, int16_t len, SSmlLinesInfo* i
static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
const char **idx, bool isField,
const char **idx, int32_t len, bool isField,
TAOS_SML_DATA_POINT* smlData, SHashObj *pHash,
SSmlLinesInfo* info) {
const char *cur = *idx;
......@@ -2495,13 +2518,13 @@ static int32_t parseSmlKvPairs(TAOS_SML_KV **pKVs, int *num_kvs,
addEscapeCharToString(childTableName, (int32_t)(childTableNameLen));
}
while (*cur != '\0') {
ret = parseSmlKey(pkv, &cur, pHash, info);
while (cur - *idx < len) {
ret = parseSmlKey(pkv, &cur, len - (cur - *idx), pHash, info);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse key", info->id);
goto error;
}
ret = parseSmlValue(pkv, &cur, &is_last_kv, info, !isField);
ret = parseSmlValue(pkv, &cur, len - (cur - *idx), &is_last_kv, info, !isField);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse value", info->id);
goto error;
......@@ -2574,14 +2597,14 @@ static void moveTimeStampToFirstKv(TAOS_SML_DATA_POINT** smlData, TAOS_SML_KV *t
free(ts);
}
int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
int32_t tscParseLine(const char* sql, int32_t len, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
const char* idx = sql;
int32_t ret = TSDB_CODE_SUCCESS;
uint8_t has_tags = 0;
TAOS_SML_KV *timestamp = NULL;
SHashObj *keyHashTable = taosHashInit(32, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
ret = parseSmlMeasurement(smlData, &idx, &has_tags, info);
ret = parseSmlMeasurement(smlData, &idx, len, &has_tags, info);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse measurement", info->id);
taosHashCleanup(keyHashTable);
......@@ -2591,7 +2614,7 @@ int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData, SSmlLinesInf
//Parse Tags
if (has_tags) {
ret = parseSmlKvPairs(&smlData->tags, &smlData->tagNum, &idx, false, smlData, keyHashTable, info);
ret = parseSmlKvPairs(&smlData->tags, &smlData->tagNum, &idx, len - (idx - sql), false, smlData, keyHashTable, info);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse tag", info->id);
taosHashCleanup(keyHashTable);
......@@ -2601,7 +2624,7 @@ int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData, SSmlLinesInf
tscDebug("SML:0x%"PRIx64" Parse tags finished, num of tags:%d", info->id, smlData->tagNum);
//Parse fields
ret = parseSmlKvPairs(&smlData->fields, &smlData->fieldNum, &idx, true, smlData, keyHashTable, info);
ret = parseSmlKvPairs(&smlData->fields, &smlData->fieldNum, &idx, len - (idx - sql), true, smlData, keyHashTable, info);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse field", info->id);
taosHashCleanup(keyHashTable);
......@@ -2617,7 +2640,7 @@ int32_t tscParseLine(const char* sql, TAOS_SML_DATA_POINT* smlData, SSmlLinesInf
taosHashCleanup(keyHashTable);
//Parse timestamp
ret = parseSmlTimeStamp(&timestamp, &idx, info);
ret = parseSmlTimeStamp(&timestamp, &idx, len - (idx - sql), info);
if (ret) {
tscError("SML:0x%"PRIx64" Unable to parse timestamp", info->id);
return ret;
......@@ -2652,24 +2675,58 @@ void destroySmlDataPoint(TAOS_SML_DATA_POINT* point) {
free(point->childTableName);
}
int32_t tscParseLines(char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
for (int32_t i = 0; i < numLines; ++i) {
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseLine(lines[i], &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("SML:0x%"PRIx64" data point line parse failed. line %d : %s", info->id, i, lines[i]);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("SML:0x%"PRIx64" data point line parse success. line %d", info->id, i);
}
taosArrayPush(points, &point);
static int32_t tscParseLinesInner(char* line, int32_t len, SArray* points, SSmlLinesInfo* info){
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseLine(line, len, &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("SML:0x%"PRIx64" data point line parse failed.", info->id);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("SML:0x%"PRIx64" data point line parse success.", info->id);
}
taosArrayPush(points, &point);
return TSDB_CODE_SUCCESS;
}
int taos_insert_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int *affectedRows) {
int32_t tscParseLines(char* data, int32_t len, char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
int32_t code = TSDB_CODE_SUCCESS;
if(!data && lines){
for (int32_t i = 0; i < numLines; ++i) {
code = tscParseLinesInner(lines[i], strlen(lines[i]), points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
}else if(data && !lines){
char* tmp = data;
int32_t lenTmp = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\n' || i == len - 1){
if(data[i] != '\n' && i == len - 1){
lenTmp ++;
}
if(lenTmp > 0) {
code = tscParseLinesInner(tmp, lenTmp, points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
if(i < len - 1) {
tmp = data + i + 1;
}
lenTmp = 0;
}else{
lenTmp ++;
}
}
}
return code;
}
int taos_insert_lines(TAOS* taos, char* data, int len, char* lines[], int *numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int *affectedRows) {
int32_t code = 0;
SSmlLinesInfo* info = tcalloc(1, sizeof(SSmlLinesInfo));
......@@ -2677,38 +2734,52 @@ int taos_insert_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType p
info->tsType = tsType;
info->protocol = protocol;
if (numLines <= 0 || numLines > 65536*32) {
tscError("SML:0x%"PRIx64" taos_insert_lines numLines should be between 1 and 65536*32. numLines: %d", info->id, numLines);
if (data){
*numLines = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\0'){
data[i] = '0';
}
if(data[i] == '\n' || i == len - 1){
(*numLines)++;
}
}
}
if (*numLines <= 0 || *numLines > 65536*32) {
tscError("SML:0x%"PRIx64" taos_insert_lines numLines should be between 1 and 65536*32. numLines: %d", info->id, *numLines);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
for (int i = 0; i < numLines; ++i) {
if (lines[i] == NULL) {
tscError("SML:0x%"PRIx64" taos_insert_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
if(lines){
for (int i = 0; i < *numLines; ++i) {
if (lines[i] == NULL) {
tscError("SML:0x%"PRIx64" taos_insert_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
}
}
SArray* lpPoints = taosArrayInit(numLines, sizeof(TAOS_SML_DATA_POINT));
SArray* lpPoints = taosArrayInit(*numLines, sizeof(TAOS_SML_DATA_POINT));
if (lpPoints == NULL) {
tscError("SML:0x%"PRIx64" taos_insert_lines failed to allocate memory", info->id);
tfree(info);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
tscDebug("SML:0x%"PRIx64" taos_insert_lines begin inserting %d lines, first line: %s", info->id, numLines, lines[0]);
code = tscParseLines(lines, numLines, lpPoints, NULL, info);
tscDebug("SML:0x%"PRIx64" taos_insert_lines begin inserting %d lines", info->id, *numLines);
code = tscParseLines(data, len, lines, *numLines, lpPoints, NULL, info);
size_t numPoints = taosArrayGetSize(lpPoints);
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
if (code != 0) {
goto cleanup;
}
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
code = tscSmlInsert(taos, points, (int)numPoints, info);
if (code != 0) {
tscError("SML:0x%"PRIx64" taos_sml_insert error: %s", info->id, tstrerror((code)));
......@@ -2718,9 +2789,7 @@ int taos_insert_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType p
}
cleanup:
tscDebug("SML:0x%"PRIx64" taos_insert_lines finish inserting %d lines. code: %d", info->id, numLines, code);
points = TARRAY_GET_START(lpPoints);
numPoints = taosArrayGetSize(lpPoints);
tscDebug("SML:0x%"PRIx64" taos_insert_lines finish inserting %d lines. code: %d", info->id, *numLines, code);
for (int i=0; i<numPoints; ++i) {
destroySmlDataPoint(points+i);
}
......@@ -2816,13 +2885,51 @@ TAOS_RES* taos_schemaless_insert(TAOS* taos, char* lines[], int numLines, int pr
switch (protocol) {
case TSDB_SML_LINE_PROTOCOL:
code = taos_insert_lines(taos, lines, numLines, protocol, tsType, &affected_rows);
code = taos_insert_lines(taos, NULL, 0, lines, &numLines, protocol, tsType, &affected_rows);
break;
case TSDB_SML_TELNET_PROTOCOL:
code = taos_insert_telnet_lines(taos, NULL, 0, lines, &numLines, protocol, tsType, &affected_rows);
break;
case TSDB_SML_JSON_PROTOCOL:
code = taos_insert_json_payload(taos, *lines, protocol, &numLines, tsType, &affected_rows);
break;
default:
code = TSDB_CODE_TSC_INVALID_PROTOCOL_TYPE;
break;
}
SSqlObj *pSql = createSmlQueryObj(taos, affected_rows, code);
return (TAOS_RES*)pSql;
}
TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *totalRows, int protocol, int precision){
int code = TSDB_CODE_SUCCESS;
int affected_rows = 0;
SMLTimeStampType tsType = SML_TIME_STAMP_NOW;
if (protocol == TSDB_SML_LINE_PROTOCOL) {
code = convertPrecisionType(precision, &tsType);
if (code != TSDB_CODE_SUCCESS) {
return NULL;
}
}
if(!totalRows){
tscError("totalRows is null");
return NULL;
}
switch (protocol) {
case TSDB_SML_LINE_PROTOCOL:
code = taos_insert_lines(taos, lines, len, NULL, totalRows, protocol, tsType, &affected_rows);
break;
case TSDB_SML_TELNET_PROTOCOL:
code = taos_insert_telnet_lines(taos, lines, numLines, protocol, tsType, &affected_rows);
code = taos_insert_telnet_lines(taos, lines, len, NULL, totalRows, protocol, tsType, &affected_rows);
break;
case TSDB_SML_JSON_PROTOCOL:
code = taos_insert_json_payload(taos, *lines, protocol, tsType, &affected_rows);
code = taos_insert_json_payload(taos, lines, protocol, totalRows, tsType, &affected_rows);
break;
default:
code = TSDB_CODE_TSC_INVALID_PROTOCOL_TYPE;
......
src/client/src/tscParseOpenTSDB.c
浏览文件 @ 574bfd96
......@@ -33,7 +33,7 @@ static uint64_t genUID() {
return id;
}
static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
const char *cur = *idx;
uint16_t len = 0;
......@@ -49,7 +49,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SS
}
*/
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_TABLE_NAME_LEN - 1) {
tscError("OTD:0x%"PRIx64" Metric cannot exceeds %d characters", info->id, TSDB_TABLE_NAME_LEN - 1);
tfree(pSml->stableName);
......@@ -82,7 +82,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SS
return TSDB_CODE_SUCCESS;
}
static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
//Timestamp must be the first KV to parse
assert(*num_kvs == 0);
......@@ -96,7 +96,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
//allocate fields for timestamp and value
*pTS = tcalloc(OTD_MAX_FIELDS_NUM, sizeof(TAOS_SML_KV));
while(*cur != '\0') {
while(cur - *idx < sqlLen) {
if (*cur == ' ') {
if (*(cur + 1) != ' ') {
break;
......@@ -109,7 +109,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
len++;
}
if (len > 0 && *cur != '\0') {
if (len > 0 && cur - *idx < sqlLen) {
value = tcalloc(len + 1, 1);
memcpy(value, start, len);
} else {
......@@ -135,7 +135,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
return ret;
}
static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
//skip timestamp
TAOS_SML_KV *pVal = *pKVs + 1;
const char *start, *cur;
......@@ -158,7 +158,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
len += 2;
}
while(*cur != '\0') {
while(cur - *idx < sqlLen) {
if (*cur == ' ') {
if (searchQuote == true) {
if (*(cur - 1) == '"' && len != 1 && len != 2) {
......@@ -181,7 +181,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
len++;
}
if (len > 0 && *cur != '\0') {
if (len > 0 && cur - *idx < sqlLen) {
value = tcalloc(len + 1, 1);
memcpy(value, start, len);
} else {
......@@ -205,7 +205,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
return ret;
}
static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *pHash, SSmlLinesInfo* info) {
static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen, SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *idx;
char key[TSDB_COL_NAME_LEN];
uint16_t len = 0;
......@@ -215,7 +215,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
// tscError("OTD:0x%"PRIx64" Tag key cannot start with digit", info->id);
// return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
//}
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_COL_NAME_LEN - 1) {
tscError("OTD:0x%"PRIx64" Tag key cannot exceeds %d characters", info->id, TSDB_COL_NAME_LEN - 1);
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
......@@ -231,7 +231,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
cur++;
len++;
}
if (len == 0 || *cur == '\0') {
if (len == 0 || cur - *idx == sqlLen) {
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
}
key[len] = '\0';
......@@ -249,7 +249,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
}
static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen,
bool *is_last_kv, SSmlLinesInfo* info) {
const char *start, *cur;
char *value = NULL;
......@@ -258,9 +258,9 @@ static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
while (1) {
// whitespace or '\0' identifies a value
if (*cur == ' ' || *cur == '\0') {
if (*cur == ' ' || cur - *idx == sqlLen) {
// '\0' indicates end of value
*is_last_kv = (*cur == '\0') ? true : false;
*is_last_kv = (cur - *idx == sqlLen) ? true : false;
if (*cur == ' ' && *(cur + 1) == ' ') {
cur++;
continue;
......@@ -290,12 +290,12 @@ static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
}
tfree(value);
*idx = (*cur == '\0') ? cur : cur + 1;
*idx = (cur - *idx == sqlLen) ? cur : cur + 1;
return TSDB_CODE_SUCCESS;
}
static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
const char **idx, char **childTableName,
const char **idx, int32_t sqlLen, char **childTableName,
SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *idx;
int32_t ret = TSDB_CODE_SUCCESS;
......@@ -312,13 +312,13 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
memcpy(childTbName, tsSmlChildTableName, childTableNameLen);
addEscapeCharToString(childTbName, (int32_t)(childTableNameLen));
}
while (*cur != '\0') {
ret = parseTelnetTagKey(pkv, &cur, pHash, info);
while (cur - *idx < sqlLen) {
ret = parseTelnetTagKey(pkv, &cur, sqlLen - (cur - *idx), pHash, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse key", info->id);
return ret;
}
ret = parseTelnetTagValue(pkv, &cur, &is_last_kv, info);
ret = parseTelnetTagValue(pkv, &cur, sqlLen - (cur - *idx), &is_last_kv, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse value", info->id);
return ret;
......@@ -356,12 +356,12 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
return ret;
}
static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
static int32_t tscParseTelnetLine(const char* line, int32_t len, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
const char* idx = line;
int32_t ret = TSDB_CODE_SUCCESS;
//Parse metric
ret = parseTelnetMetric(smlData, &idx, info);
ret = parseTelnetMetric(smlData, &idx, len, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse metric", info->id);
return ret;
......@@ -369,7 +369,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
tscDebug("OTD:0x%"PRIx64" Parse metric finished", info->id);
//Parse timestamp
ret = parseTelnetTimeStamp(&smlData->fields, &smlData->fieldNum, &idx, info);
ret = parseTelnetTimeStamp(&smlData->fields, &smlData->fieldNum, &idx, len - (idx - line), info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse timestamp", info->id);
return ret;
......@@ -377,7 +377,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
tscDebug("OTD:0x%"PRIx64" Parse timestamp finished", info->id);
//Parse value
ret = parseTelnetMetricValue(&smlData->fields, &smlData->fieldNum, &idx, info);
ret = parseTelnetMetricValue(&smlData->fields, &smlData->fieldNum, &idx, len - (idx - line), info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse metric value", info->id);
return ret;
......@@ -386,7 +386,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
//Parse tagKVs
SHashObj *keyHashTable = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
ret = parseTelnetTagKvs(&smlData->tags, &smlData->tagNum, &idx, &smlData->childTableName, keyHashTable, info);
ret = parseTelnetTagKvs(&smlData->tags, &smlData->tagNum, &idx, len - (idx - line), &smlData->childTableName, keyHashTable, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse tags", info->id);
taosHashCleanup(keyHashTable);
......@@ -399,24 +399,58 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
return TSDB_CODE_SUCCESS;
}
static int32_t tscParseTelnetLines(char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
for (int32_t i = 0; i < numLines; ++i) {
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseTelnetLine(lines[i], &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("OTD:0x%"PRIx64" data point line parse failed. line %d : %s", info->id, i, lines[i]);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("OTD:0x%"PRIx64" data point line parse success. line %d", info->id, i);
}
taosArrayPush(points, &point);
static int32_t tscParseTelnetLinesInner(char* data, int32_t len, SArray* points, SSmlLinesInfo* info) {
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseTelnetLine(data, len, &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("OTD:0x%"PRIx64" data point line parse failed.", info->id);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("OTD:0x%"PRIx64" data point line parse success.", info->id);
}
taosArrayPush(points, &point);
return TSDB_CODE_SUCCESS;
}
int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
static int32_t tscParseTelnetLines(char* data, int32_t len, char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
int32_t code = TSDB_CODE_SUCCESS;
if(!data && lines){
for (int32_t i = 0; i < numLines; ++i) {
code = tscParseTelnetLinesInner(lines[i], strlen(lines[i]), points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
}else if(data && !lines){
char* tmp = data;
int32_t lenTmp = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\n' || i == len - 1){
if(data[i] != '\n' && i == len - 1){
lenTmp++;
}
if(lenTmp > 0) {
code = tscParseTelnetLinesInner(tmp, lenTmp, points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
if(i < len - 1) {
tmp = data + i + 1;
}
lenTmp = 0;
}else{
lenTmp ++;
}
}
}
return code;
}
int taos_insert_telnet_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
int32_t code = 0;
SSmlLinesInfo* info = tcalloc(1, sizeof(SSmlLinesInfo));
......@@ -424,38 +458,52 @@ int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtoco
info->tsType = tsType;
info->protocol = protocol;
if (numLines <= 0 || numLines > 65536) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines numLines should be between 1 and 65536. numLines: %d", info->id, numLines);
if (data && !lines){
*numLines = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\0'){
data[i] = '0';
}
if(data[i] == '\n' || i == len - 1){
(*numLines)++;
}
}
}
if (*numLines <= 0 || *numLines > 65536) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines numLines should be between 1 and 65536. numLines: %d", info->id, *numLines);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
for (int i = 0; i < numLines; ++i) {
if (lines[i] == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
if(!data && lines){
for (int i = 0; i < *numLines; ++i) {
if (lines[i] == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
}
}
SArray* lpPoints = taosArrayInit(numLines, sizeof(TAOS_SML_DATA_POINT));
SArray* lpPoints = taosArrayInit(*numLines, sizeof(TAOS_SML_DATA_POINT));
if (lpPoints == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines failed to allocate memory", info->id);
tfree(info);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d lines, first line: %s", info->id, numLines, lines[0]);
code = tscParseTelnetLines(lines, numLines, lpPoints, NULL, info);
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d lines, first line: %s", info->id, *numLines, lines[0]);
code = tscParseTelnetLines(data, len, lines, *numLines, lpPoints, NULL, info);
size_t numPoints = taosArrayGetSize(lpPoints);
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
if (code != 0) {
goto cleanup;
}
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
code = tscSmlInsert(taos, points, (int)numPoints, info);
if (code != 0) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines error: %s", info->id, tstrerror((code)));
......@@ -465,9 +513,7 @@ int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtoco
}
cleanup:
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines finish inserting %d lines. code: %d", info->id, numLines, code);
points = TARRAY_GET_START(lpPoints);
numPoints = taosArrayGetSize(lpPoints);
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines finish inserting %d lines. code: %d", info->id, *numLines, code);
for (int i = 0; i < numPoints; ++i) {
destroySmlDataPoint(points+i);
}
......@@ -1059,8 +1105,9 @@ PARSE_JSON_OVER:
return ret;
}
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, int32_t *totalRows, SMLTimeStampType tsType, int* affectedRows) {
int32_t code = 0;
*totalRows = 0;
SSmlLinesInfo* info = tcalloc(1, sizeof(SSmlLinesInfo));
info->id = genUID();
......@@ -1084,12 +1131,12 @@ int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d points", info->id, 1);
code = tscParseMultiJSONPayload(payload, lpPoints, info);
size_t numPoints = taosArrayGetSize(lpPoints);
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
if (code != 0) {
goto cleanup;
}
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
code = tscSmlInsert(taos, points, (int)numPoints, info);
if (code != 0) {
tscError("OTD:0x%"PRIx64" taos_insert_json_payload error: %s", info->id, tstrerror((code)));
......@@ -1097,11 +1144,10 @@ int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol
if (affectedRows != NULL) {
*affectedRows = info->affectedRows;
}
*totalRows = numPoints;
cleanup:
tscDebug("OTD:0x%"PRIx64" taos_insert_json_payload finish inserting 1 Point. code: %d", info->id, code);
points = TARRAY_GET_START(lpPoints);
numPoints = taosArrayGetSize(lpPoints);
for (int i = 0; i < numPoints; ++i) {
destroySmlDataPoint(points+i);
}
......
src/client/src/tscServer.c
浏览文件 @ 574bfd96
......@@ -3393,7 +3393,7 @@ int tscRenewTableMeta(SSqlObj *pSql) {
SSqlCmd* pCmd2 = &pSql->rootObj->cmd;
SHashObj *pmap = pCmd2->pTableMetaMap;
if (pmap == atomic_val_compare_exchange_ptr(&pCmd2->pTableMetaMap, pmap, NULL)) {
tscCleanupTableMetaMap(pCmd2->pTableMetaMap);
tscCleanupTableMetaMap(pmap);
}
pCmd2->pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_ENTRY_LOCK);
......
src/client/src/tscUtil.c
浏览文件 @ 574bfd96
......@@ -1373,6 +1373,10 @@ void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, b
// set the correct result
SSDataBlock* p = pQueryInfo->pQInfo->runtimeEnv.outputBuf;
pRes->numOfRows = (p != NULL)? p->info.rows: 0;
bool completed = pRes->numOfRows == 0;
if (p && pRes->numOfRows == 0 && tsAggAlways) {
pRes->numOfRows = 1;
}
if (pRes->code == TSDB_CODE_SUCCESS && pRes->numOfRows > 0) {
tscCreateResPointerInfo(pRes, pQueryInfo);
......@@ -1381,7 +1385,7 @@ void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, b
tscDebug("0x%"PRIx64" retrieve result in pRes, numOfRows:%d", objId, pRes->numOfRows);
pRes->row = 0;
pRes->completed = (pRes->numOfRows == 0);
pRes->completed = completed;
}
/*
......@@ -1650,7 +1654,7 @@ void tscResetSqlCmd(SSqlCmd* pCmd, bool clearCachedMeta, uint64_t id) {
tscFreeQueryInfo(pCmd, clearCachedMeta, id);
SHashObj *pmap = pCmd->pTableMetaMap;
if (pmap == atomic_val_compare_exchange_ptr(&pCmd->pTableMetaMap, pmap, NULL)) {
tscCleanupTableMetaMap(pCmd->pTableMetaMap);
tscCleanupTableMetaMap(pmap);
}
taosReleaseRef(tscObjRef, id);
}
......@@ -5243,6 +5247,8 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt
STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[0];
pQueryAttr->precision = pTableMetaInfo->pTableMeta->tableInfo.precision;
if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) {
pQueryAttr->pGroupbyExpr = calloc(1, sizeof(SGroupbyExpr));
*(pQueryAttr->pGroupbyExpr) = pQueryInfo->groupbyExpr;
......
src/common/inc/tglobal.h
浏览文件 @ 574bfd96
......@@ -222,6 +222,7 @@ extern int32_t cqDebugFlag;
extern int32_t debugFlag;
extern int8_t tsClientMerge;
extern int8_t tsAggAlways;
// probe alive connection
extern int32_t tsProbeSeconds;
......
src/common/src/tglobal.c
浏览文件 @ 574bfd96
......@@ -278,6 +278,7 @@ int32_t cqDebugFlag = 131;
int32_t fsDebugFlag = 135;
int8_t tsClientMerge = 0;
int8_t tsAggAlways = 0; // Agg function always return value even if zero row
// probe alive connection
int32_t tsProbeSeconds = 5 * 60; // start probe link alive after tsProbeSeconds from starting query
......@@ -1695,6 +1696,16 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "aggAlways";
cfg.ptr = &tsAggAlways;
cfg.valType = TAOS_CFG_VTYPE_INT8;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 0;
cfg.maxValue = 1;
cfg.ptrLength = 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
// default JSON string type option "binary"/"nchar"
cfg.option = "defaultJSONStrType";
cfg.ptr = tsDefaultJSONStrType;
......
src/inc/taos.h
浏览文件 @ 574bfd96
......@@ -218,6 +218,7 @@ DLL_EXPORT void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress);
DLL_EXPORT int taos_load_table_info(TAOS *taos, const char* tableNameList);
DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS* taos, char* lines[], int numLines, int protocol, int precision);
DLL_EXPORT TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *totalRows, int protocol, int precision);
DLL_EXPORT int32_t taos_parse_time(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t dayligth);
......
src/kit/shell/src/shellAuto.c
浏览文件 @ 574bfd96
......@@ -31,7 +31,7 @@
// extern function
void insertChar(Command *cmd, char *c, int size);
void insertStr(Command *cmd, char *str, int size);
typedef struct SAutoPtr {
......@@ -78,6 +78,7 @@ SWords shellCommands[] = {
{"alter user <user_name> privilege read", 0, 0, NULL},
{"alter user <user_name> privilege write", 0, 0, NULL},
{"create table <anyword> using <stb_name> tags(", 0, 0, NULL},
{"create table <anyword> as select ", 0, 0, NULL},
{"create database ", 0, 0, NULL},
{"create table <anyword> as ", 0, 0, NULL},
{"create dnode ", 0, 0, NULL},
......@@ -126,6 +127,7 @@ SWords shellCommands[] = {
{"show vgroups;", 0, 0, NULL},
{"insert into <tb_name> values(", 0, 0, NULL},
{"insert into <tb_name> using <stb_name> tags( <anyword> ) values(", 0, 0, NULL},
{"insert into <tb_name> using <stb_name> <anyword> values(", 0, 0, NULL},
{"use <db_name>", 0, 0, NULL},
{"quit", 0, 0, NULL}
};
......@@ -251,6 +253,9 @@ char * key_tags[] = {
"tags("
};
char * key_select[] = {
"select "
};
//
// ------- gobal variant define ---------
......@@ -278,7 +283,8 @@ bool waitAutoFill = false;
#define WT_VAR_DATATYPE 10
#define WT_VAR_KEYTAGS 11
#define WT_VAR_ANYWORD 12
#define WT_VAR_CNT 13
#define WT_VAR_KEYSELECT 13
#define WT_VAR_CNT 14
#define WT_FROM_DB_MAX 4 // max get content from db
#define WT_FROM_DB_CNT (WT_FROM_DB_MAX + 1)
......@@ -305,7 +311,8 @@ char varTypes[WT_VAR_CNT][64] = {
"<db_options>",
"<data_types>",
"<key_tags>",
"<anyword>"
"<anyword>",
"<key_select>"
};
char varSqls[WT_FROM_DB_CNT][64] = {
......@@ -330,23 +337,23 @@ int cntDel = 0; // delete byte count after next press tab
// show auto tab introduction
void printfIntroduction() {
printf(" ********************* How to Use TAB in TAOS Shell ******************************\n");
printf(" * Taos shell supports pressing TAB key to complete word. You can try it. *\n");
printf(" * Press TAB key anywhere, You'll get surprise. *\n");
printf(" * KEYBOARD SHORTCUT: *\n");
printf(" * [ TAB ] ...... Complete the word or show help if no input *\n");
printf(" * [ Ctrl + A ] ...... move cursor to [A]head of line *\n");
printf(" * [ Ctrl + E ] ...... move cursor to [E]nd of line *\n");
printf(" * [ Ctrl + W ] ...... move cursor to line of middle *\n");
printf(" * [ Ctrl + L ] ...... clean screen *\n");
printf(" * [ Ctrl + K ] ...... clean after cursor *\n");
printf(" * [ Ctrl + U ] ...... clean before cursor *\n");
printf(" * *\n");
printf(" **********************************************************************************\n\n");
printf(" ****************************** Tab Completion **********************************\n");
printf(" * The TDengine CLI supports tab completion for a variety of items, *\n");
printf(" * including database names, table names, function names and keywords. *\n");
printf(" * The full list of shortcut keys is as follows: *\n");
printf(" * [ TAB ] ...... complete the current word *\n");
printf(" * ...... if used on a blank line, display all valid commands *\n");
printf(" * [ Ctrl + A ] ...... move cursor to the st[A]rt of the line *\n");
printf(" * [ Ctrl + E ] ...... move cursor to the [E]nd of the line *\n");
printf(" * [ Ctrl + W ] ...... move cursor to the middle of the line *\n");
printf(" * [ Ctrl + L ] ...... clear the entire screen *\n");
printf(" * [ Ctrl + K ] ...... clear the screen after the cursor *\n");
printf(" * [ Ctrl + U ] ...... clear the screen before the cursor *\n");
printf(" **********************************************************************************\n\n");
}
void showHelp() {
printf("\nThe following are supported commands for Taos shell:");
printf("\nFThe TDengine CLI supports the following commands:");
printf("\n\
----- A ----- \n\
alter database <db_name> <db_options> \n\
......@@ -526,26 +533,16 @@ void parseCommand(SWords * command, bool pattern) {
}
// free Command
void freeCommand(SWords * command) {
SWord * word = command->head;
if (word == NULL) {
return ;
}
// loop
while (word->next) {
SWord * tmp = word;
word = word->next;
void freeCommand(SWords* command) {
SWord* item = command->head;
// loop
while (item) {
SWord* tmp = item;
item = item->next;
// if malloc need free
if(tmp->free && tmp->word)
free(tmp->word);
if (tmp->free && tmp->word) free(tmp->word);
free(tmp);
}
// if malloc need free
if(word->free && word->word)
free(word->word);
free(word);
}
void GenerateVarType(int type, char** p, int count) {
......@@ -586,6 +583,7 @@ bool shellAutoInit() {
GenerateVarType(WT_VAR_TBACTION, tb_actions, sizeof(tb_actions) /sizeof(char *));
GenerateVarType(WT_VAR_DATATYPE, data_types, sizeof(data_types) /sizeof(char *));
GenerateVarType(WT_VAR_KEYTAGS, key_tags, sizeof(key_tags) /sizeof(char *));
GenerateVarType(WT_VAR_KEYSELECT,key_select, sizeof(key_select) /sizeof(char *));
printfIntroduction();
......@@ -613,7 +611,7 @@ void shellAutoExit() {
pthread_mutex_destroy(&tiresMutex);
// free threads
for (int32_t i = 0; i < WT_VAR_CNT; i++) {
for (int32_t i = 0; i < WT_FROM_DB_CNT; i++) {
if (threads[i]) {
taosDestroyThread(threads[i]);
threads[i] = NULL;
......@@ -1079,7 +1077,7 @@ void printScreen(TAOS * con, Command * cmd, SWords * match) {
}
// insert new
insertChar(cmd, (char *)str, strLen);
insertStr(cmd, (char *)str, strLen);
}
......@@ -1170,11 +1168,11 @@ bool nextMatchCommand(TAOS * con, Command * cmd, SWords * firstMatch) {
printScreen(con, cmd, match);
// free
freeCommand(input);
if (input->source) {
free(input->source);
input->source = NULL;
}
freeCommand(input);
free(input);
return true;
......@@ -1194,7 +1192,7 @@ bool fillWithType(TAOS * con, Command * cmd, char* pre, int type) {
// show
int count = strlen(part);
insertChar(cmd, part, count);
insertStr(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
......@@ -1222,7 +1220,7 @@ bool fillTableName(TAOS * con, Command * cmd, char* pre) {
// show
int count = strlen(part);
insertChar(cmd, part, count);
insertStr(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
......@@ -1242,7 +1240,7 @@ char * lastWord(char * p) {
char * p2 = strrchr(p, ',');
if (p1 && p2) {
return MAX(p1, p2) + 1;
return p1 > p2 ? p1 + 1 : p2 + 1;
} else if (p1) {
return p1 + 1;
} else if(p2) {
......@@ -1325,37 +1323,9 @@ bool needInsertFrom(char * sql, int len) {
return true;
}
bool matchSelectQuery(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 512) {
return false;
}
// select and from
if(strncasecmp(p, "select ", 7) != 0) {
// not select query clause
return false;
}
p += 7;
len -= 7;
char* ps = p = strndup(p, len);
// p is string following select keyword
bool appendAfterSelect(TAOS * con, Command * cmd, char* sql, int32_t len) {
char* p = strndup(sql, len);
// union all
char * p1;
......@@ -1374,8 +1344,8 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
bool fieldEnd = fieldsInputEnd(p);
// cheeck fields input end then insert from keyword
if (fieldEnd && p[len-1] == ' ') {
insertChar(cmd, "from", 4);
free(ps);
insertStr(cmd, "from", 4);
free(p);
return true;
}
......@@ -1387,7 +1357,7 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
ret = fillWithType(con, cmd, last, WT_VAR_FUNC);
}
free(ps);
free(p);
return ret;
}
......@@ -1400,10 +1370,73 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD);
}
free(ps);
free(p);
return ret;
}
int32_t searchAfterSelect(char* p, int32_t len) {
// select * from st;
if(strncasecmp(p, "select ", 7) == 0) {
// check nest query
char *p1 = p + 7;
while(1) {
char *p2 = strstr(p1, "select ");
if(p2 == NULL)
break;
p1 = p2 + 7;
}
return p1 - p;
}
char* as_pos_end = strstr(p, " as select ");
if (as_pos_end == NULL)
return -1;
as_pos_end += 11;
// create table <stream_name> as select
if(strncasecmp(p, "create table ", 13) == 0) {
return as_pos_end - p;;
}
return -1;
}
bool matchSelectQuery(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 512) {
return false;
}
// search
char* sql_cp = strndup(p, len);
int32_t n = searchAfterSelect(sql_cp, len);
free(sql_cp);
if(n == -1 || n > len)
return false;
p += n;
len -= n;
// append
return appendAfterSelect(con, cmd, p, len);
}
// if is input create fields or tags area, return true
bool isCreateFieldsArea(char * p) {
char * left = strrchr(p, '(');
......@@ -1460,6 +1493,12 @@ bool matchCreateTable(TAOS * con, Command * cmd) {
bool ret = false;
char * last = lastWord(ps);
// except create table m1 as select ....
if (strstr(ps, " as select")) {
free(ps);
return false;
}
// check in create fields or tags input area
if (isCreateFieldsArea(ps)) {
ret = fillWithType(con, cmd, last, WT_VAR_DATATYPE);
......@@ -1484,18 +1523,76 @@ bool matchCreateTable(TAOS * con, Command * cmd) {
bool matchOther(TAOS * con, Command * cmd) {
int len = cmd->commandSize;
char* p = cmd->command;
// '\\'
if (p[len - 1] == '\\') {
// append '\G'
char a[] = "G;";
insertChar(cmd, a, 2);
insertStr(cmd, a, 2);
return true;
}
// too small
if(len < 8)
return false;
// like 'from ( '
char* sql = strndup(p, len);
char* last = lastWord(sql);
if (strcmp(last, "from(") == 0) {
fillWithType(con, cmd, "", WT_VAR_KEYSELECT);
free(sql);
return true;
}
if (strncmp(last, "(", 1) == 0) {
last += 1;
}
char* from = strstr(sql, " from");
// find last ' from'
while (from) {
char* p1 = strstr(from + 5, " from");
if (p1 == NULL)
break;
from = p1;
}
if (from) {
// find next is '('
char * p2 = from + 5;
bool found = false; // found 'from ... ( ...' ... is any count of blank
bool found1 = false; // found '('
while (1) {
if ( p2 == last || *p2 == '\0') {
// last word or string end
if (found1) {
found = true;
}
break;
} else if(*p2 == '(') {
found1 = true;
} else if(*p2 == ' ') {
// do nothing
} else {
// have any other char
break;
}
// move next
p2++;
}
if (found) {
fillWithType(con, cmd, last, WT_VAR_KEYSELECT);
free(sql);
return true;
}
}
free(sql);
return false;
}
// main key press tab
void pressTabKey(TAOS * con, Command * cmd) {
// check
......@@ -1761,4 +1858,4 @@ void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb) {
}
return ;
}
}
\ No newline at end of file
src/kit/shell/src/shellCheck.c
浏览文件 @ 574bfd96
......@@ -131,7 +131,7 @@ static void *shellCheckThreadFp(void *arg) {
char *tbname = tbNames[t];
if (tbname == NULL) break;
snprintf(sql, SHELL_SQL_LEN, "select count(*) from %s;", tbname);
snprintf(sql, SHELL_SQL_LEN, "select last_row(_c0) from `%s`;", tbname);
TAOS_RES *pSql = taos_query(pThread->taos, sql);
int32_t code = taos_errno(pSql);
......
src/kit/shell/src/shellCommand.c
浏览文件 @ 574bfd96
......@@ -79,11 +79,22 @@ void insertChar(Command *cmd, char *c, int size) {
/* update the values */
cmd->commandSize += size;
cmd->cursorOffset += size;
for (int i = 0; i < size; i++) {
mbtowc(&wc, c + i, size);
cmd->screenOffset += wcwidth(wc);
cmd->endOffset += wcwidth(wc);
}
cmd->screenOffset += wcwidth(wc);
cmd->endOffset += wcwidth(wc);
showOnScreen(cmd);
}
void insertStr(Command *cmd, char *str, int size) {
clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size);
/* update the buffer */
memmove(cmd->command + cmd->cursorOffset + size, cmd->command + cmd->cursorOffset,
cmd->commandSize - cmd->cursorOffset);
memcpy(cmd->command + cmd->cursorOffset, str, size);
/* update the values */
cmd->commandSize += size;
cmd->cursorOffset += size;
cmd->screenOffset += size;
cmd->endOffset += size;
showOnScreen(cmd);
}
......
src/kit/shell/src/shellDarwin.c
浏览文件 @ 574bfd96
......@@ -35,7 +35,7 @@ void insertChar(Command *cmd, char *c, int size);
void printHelp() {
char indent[10] = " ";
printf("taos shell is used to test the TDengine database\n");
printf("TDengine Command Line is used to test the TDengine database\n");
printf("%s%s\n", indent, "-h");
printf("%s%s%s\n", indent, indent, "TDengine server IP address to connect. The default host is localhost.");
......@@ -68,7 +68,7 @@ void printHelp() {
exit(EXIT_SUCCESS);
}
char DARWINCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char DARWINCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char g_password[SHELL_MAX_PASSWORD_LEN];
......
src/kit/shell/src/shellEngine.c
浏览文件 @ 574bfd96
......@@ -32,7 +32,7 @@
#include <regex.h>
/**************** Global variables ****************/
char CLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char CLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char PROMPT_HEADER[] = "taos> ";
char CONTINUE_PROMPT[] = " -> ";
......@@ -192,9 +192,15 @@ static int32_t shellRunSingleCommand(TAOS *con, char *command) {
if (regex_match(command, "^[ \t]*source[\t ]+[^ ]+[ \t;]*$", REG_EXTENDED | REG_ICASE)) {
/* If source file. */
char *c_ptr = strtok(command, " ;");
assert(c_ptr != NULL);
if (c_ptr == NULL) {
shellRunCommandOnServer(con, command);
return 0;
}
c_ptr = strtok(NULL, " ;");
assert(c_ptr != NULL);
if (c_ptr == NULL) {
shellRunCommandOnServer(con, command);
return 0;
}
source_file(con, c_ptr);
return 0;
}
......
src/kit/shell/src/shellLinux.c
浏览文件 @ 574bfd96
......@@ -194,7 +194,7 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
/* Our argp parser. */
static struct argp argp = {options, parse_opt, args_doc, doc};
char LINUXCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char LINUXCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char g_password[SHELL_MAX_PASSWORD_LEN];
......
src/kit/shell/src/shellWindows.c
浏览文件 @ 574bfd96
......@@ -21,7 +21,7 @@
extern char configDir[];
char WINCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char WINCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
void printVersion() {
......@@ -30,7 +30,7 @@ void printVersion() {
void printHelp() {
char indent[10] = " ";
printf("taos shell is used to test the TDengine database\n");
printf("TDengine Command Line is used to test the TDengine database\n");
printf("%s%s\n", indent, "-h");
printf("%s%s%s\n", indent, indent, "TDengine server FQDN to connect. The default host is localhost.");
......
src/kit/shell/src/tire.c
浏览文件 @ 574bfd96
......@@ -325,30 +325,26 @@ void matchPrefixFromTree(STire* tire, char* prefix, SMatch* match) {
}
SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match) {
if(match == NULL) {
match = (SMatch* )tmalloc(sizeof(SMatch));
memset(match, 0, sizeof(SMatch));
}
switch (tire->type) {
case TIRE_TREE:
matchPrefixFromTree(tire, prefix, match);
case TIRE_LIST:
matchPrefixFromList(tire, prefix, match);
default:
break;
}
// return if need
if (match->count == 0) {
freeMatch(match);
match = NULL;
}
return match;
SMatch* rMatch = match; // define return match
if (rMatch == NULL) {
rMatch = (SMatch*)malloc(sizeof(SMatch));
memset(rMatch, 0, sizeof(SMatch));
}
switch (tire->type) {
case TIRE_TREE:
matchPrefixFromTree(tire, prefix, rMatch);
break;
case TIRE_LIST:
matchPrefixFromList(tire, prefix, rMatch);
break;
default:
break;
}
return rMatch;
}
// get all items from tires tree
void enumFromList(STire* tire, SMatch* match) {
StrName * item = tire->head;
......@@ -395,8 +391,10 @@ SMatch* enumAll(STire* tire) {
switch (tire->type) {
case TIRE_TREE:
enumFromTree(tire, match);
break;
case TIRE_LIST:
enumFromList(tire, match);
break;
default:
break;
}
......
taos-tools @ cc973e0c
比较 f03c09a9...cc973e0c
Subproject commit f03c09a9f0882949b52f34f11607808ee17bfbc5
Subproject commit cc973e0c2999a739fa2d06c25ab5adb2a7537c8f
src/mnode/src/mnodeDb.c
浏览文件 @ 574bfd96
......@@ -1013,9 +1013,9 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) {
newCfg.daysToKeep2 = daysToKeep2;
}
if (minRows > 0 && minRows != pDb->cfg.minRowsPerFileBlock) {
mError("db:%s, can't alter minRows option", pDb->name);
terrno = TSDB_CODE_MND_INVALID_DB_OPTION;
if (minRows >= 0 && minRows != pDb->cfg.minRowsPerFileBlock) {
mDebug("db:%s, minRows:%d change to %d", pDb->name, pDb->cfg.minRowsPerFileBlock, minRows);
newCfg.minRowsPerFileBlock = minRows;
}
if (maxRows > 0 && maxRows != pDb->cfg.maxRowsPerFileBlock) {
......
taosadapter @ 1e0a0209
比较 4c8d9259...1e0a0209
Subproject commit 4c8d92596d3c57ebc25e9ce0e0d8582685e6bcda
Subproject commit 1e0a02097f8433f557125898c756f293c97da88d
src/query/inc/sql.y
浏览文件 @ 574bfd96
......@@ -311,6 +311,7 @@ alter_db_optr(Y) ::= alter_db_optr(Z) blocks(X). { Y = Z; Y.numOfBlocks = s
alter_db_optr(Y) ::= alter_db_optr(Z) comp(X). { Y = Z; Y.compressionLevel = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) update(X). { Y = Z; Y.update = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) cachelast(X). { Y = Z; Y.cachelast = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) minrows(X). { Y = Z; Y.minRowsPerBlock = strtol(X.z, NULL, 10); }
// dynamically update the following two parameters are not allowed.
//alter_db_optr(Y) ::= alter_db_optr(Z) fsync(X). { Y = Z; Y.fsyncPeriod = strtol(X.z, NULL, 10); }
......
src/query/src/qAggMain.c
浏览文件 @ 574bfd96
......@@ -30,6 +30,7 @@
#include "qUdf.h"
#include "tcompare.h"
#include "hashfunc.h"
#include "tglobal.h"
#define GET_INPUT_DATA_LIST(x) ((char *)((x)->pInput))
#define GET_INPUT_DATA(x, y) (GET_INPUT_DATA_LIST(x) + (y) * (x)->inputBytes)
......@@ -1952,7 +1953,7 @@ static void stddev_function(SQLFunctionCtx *pCtx) {
pStd->avg = GET_DOUBLE_VAL(pCtx->pOutput);
assert((isnan(pAvg->sum) && pAvg->num == 0) || (pStd->num == pAvg->num && pStd->avg == pAvg->sum));
}
if (pStd->stage == 0) {
// the first stage is to calculate average value
avg_function(pCtx);
......@@ -1961,7 +1962,7 @@ static void stddev_function(SQLFunctionCtx *pCtx) {
// if pStd->num == 0, there are no numbers in the first round check. No need to do the second round
double *retVal = &pStd->res;
double avg = pStd->avg;
void *pData = GET_INPUT_DATA_LIST(pCtx);
int32_t num = 0;
......@@ -2015,14 +2016,14 @@ static void stddev_function(SQLFunctionCtx *pCtx) {
default:
qError("stddev function not support data type:%d", pCtx->inputType);
}
SET_VAL(pCtx, 1, 1);
}
}
static void stddev_finalizer(SQLFunctionCtx *pCtx) {
SStddevInfo *pStd = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
if (pStd->num <= 0) {
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
} else {
......@@ -2030,7 +2031,7 @@ static void stddev_finalizer(SQLFunctionCtx *pCtx) {
SET_DOUBLE_VAL(retValue, sqrt(pStd->res / pStd->num));
SET_VAL(pCtx, 1, 1);
}
doFinalizer(pCtx);
}
......@@ -2183,11 +2184,11 @@ static bool first_last_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo*
if (!function_setup(pCtx, pResInfo)) {
return false;
}
// used to keep the timestamp for comparison
pCtx->param[1].nType = 0;
pCtx->param[1].i64 = 0;
return true;
}
......@@ -2216,7 +2217,7 @@ static void first_function(SQLFunctionCtx *pCtx) {
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
continue;
}
memcpy(pCtx->pOutput, data, pCtx->inputBytes);
if (pCtx->ptsList != NULL) {
TSKEY k = GET_TS_DATA(pCtx, i);
......@@ -2226,7 +2227,7 @@ static void first_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pInfo = GET_RES_INFO(pCtx);
pInfo->hasResult = DATA_SET_FLAG;
pInfo->complete = true;
notNullElems++;
break;
}
......@@ -2257,14 +2258,14 @@ static void first_function(SQLFunctionCtx *pCtx) {
static void first_data_assign_impl(SQLFunctionCtx *pCtx, char *pData, int32_t idx) {
int64_t *timestamp = GET_TS_LIST(pCtx);
SFirstLastInfo *pInfo = (SFirstLastInfo *)(pCtx->pOutput + pCtx->inputBytes);
if (pInfo->hasResult != DATA_SET_FLAG || timestamp[idx] < pInfo->ts) {
memcpy(pCtx->pOutput, pData, pCtx->inputBytes);
pInfo->hasResult = DATA_SET_FLAG;
pInfo->ts = timestamp[idx];
DO_UPDATE_TAG_COLUMNS(pCtx, pInfo->ts);
}
}
......@@ -2282,7 +2283,7 @@ static void first_dist_function(SQLFunctionCtx *pCtx) {
if (pCtx->order == TSDB_ORDER_DESC/* || pCtx->preAggVals.dataBlockLoaded == false*/) {
return;
}
int32_t notNullElems = 0;
// find the first not null value
......@@ -2291,16 +2292,16 @@ static void first_dist_function(SQLFunctionCtx *pCtx) {
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
continue;
}
first_data_assign_impl(pCtx, data, i);
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
notNullElems++;
break;
}
SET_VAL(pCtx, notNullElems, 1);
}
......@@ -2312,16 +2313,16 @@ static void first_dist_func_merge(SQLFunctionCtx *pCtx) {
if (pInput->hasResult != DATA_SET_FLAG) {
return;
}
// The param[1] is used to keep the initial value of max ts value
if (pCtx->param[1].nType != pCtx->outputType || pCtx->param[1].i64 > pInput->ts) {
memcpy(pCtx->pOutput, pData, pCtx->outputBytes);
pCtx->param[1].i64 = pInput->ts;
pCtx->param[1].nType = pCtx->outputType;
DO_UPDATE_TAG_COLUMNS(pCtx, pInput->ts);
}
SET_VAL(pCtx, 1, 1);
GET_RES_INFO(pCtx)->hasResult = DATA_SET_FLAG;
}
......@@ -2395,18 +2396,18 @@ static void last_function(SQLFunctionCtx *pCtx) {
static void last_data_assign_impl(SQLFunctionCtx *pCtx, char *pData, int32_t idx) {
int64_t *timestamp = GET_TS_LIST(pCtx);
SFirstLastInfo *pInfo = (SFirstLastInfo *)(pCtx->pOutput + pCtx->inputBytes);
if (pInfo->hasResult != DATA_SET_FLAG || pInfo->ts < timestamp[idx]) {
#if defined(_DEBUG_VIEW)
qDebug("assign index:%d, ts:%" PRId64 ", val:%d, ", idx, timestamp[idx], *(int32_t *)pData);
#endif
memcpy(pCtx->pOutput, pData, pCtx->inputBytes);
pInfo->hasResult = DATA_SET_FLAG;
pInfo->ts = timestamp[idx];
DO_UPDATE_TAG_COLUMNS(pCtx, pInfo->ts);
}
}
......@@ -2425,19 +2426,19 @@ static void last_dist_function(SQLFunctionCtx *pCtx) {
char *data = GET_INPUT_DATA(pCtx, i);
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
if (!pCtx->requireNull) {
continue;
continue;
}
}
last_data_assign_impl(pCtx, data, i);
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
notNullElems++;
break;
}
SET_VAL(pCtx, notNullElems, 1);
}
......@@ -2448,12 +2449,12 @@ static void last_dist_function(SQLFunctionCtx *pCtx) {
*/
static void last_dist_func_merge(SQLFunctionCtx *pCtx) {
char *pData = GET_INPUT_DATA_LIST(pCtx);
SFirstLastInfo *pInput = (SFirstLastInfo*) (pData + pCtx->outputBytes);
if (pInput->hasResult != DATA_SET_FLAG) {
return;
}
/*
* param[1] used to keep the corresponding timestamp to decide if current result is
* the true last result
......@@ -2462,10 +2463,10 @@ static void last_dist_func_merge(SQLFunctionCtx *pCtx) {
memcpy(pCtx->pOutput, pData, pCtx->outputBytes);
pCtx->param[1].i64 = pInput->ts;
pCtx->param[1].nType = pCtx->outputType;
DO_UPDATE_TAG_COLUMNS(pCtx, pInput->ts);
}
SET_VAL(pCtx, 1, 1);
GET_RES_INFO(pCtx)->hasResult = DATA_SET_FLAG;
}
......@@ -2480,16 +2481,16 @@ static void last_row_function(SQLFunctionCtx *pCtx) {
// assign the last element in current data block
assignVal(pCtx->pOutput, pData + (pCtx->size - 1) * pCtx->inputBytes, pCtx->inputBytes, pCtx->inputType);
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
// set the result to final result buffer in case of super table query
if (pCtx->stableQuery) {
SLastrowInfo *pInfo1 = (SLastrowInfo *)(pCtx->pOutput + pCtx->inputBytes);
pInfo1->ts = GET_TS_DATA(pCtx, pCtx->size - 1);
pInfo1->hasResult = DATA_SET_FLAG;
DO_UPDATE_TAG_COLUMNS(pCtx, pInfo1->ts);
} else {
TSKEY ts = GET_TS_DATA(pCtx, pCtx->size - 1);
......@@ -2506,7 +2507,7 @@ static void last_row_finalizer(SQLFunctionCtx *pCtx) {
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
return;
}
GET_RES_INFO(pCtx)->numOfRes = 1;
doFinalizer(pCtx);
}
......@@ -2517,7 +2518,7 @@ static void valuePairAssign(tValuePair *dst, int16_t type, const char *val, int6
dst->v.nType = type;
dst->v.i64 = *(int64_t *)val;
dst->timestamp = tsKey;
int32_t size = 0;
if (stage == MERGE_STAGE) {
memcpy(dst->pTags, pTags, (size_t)pTagInfo->tagsLen);
......@@ -2528,7 +2529,7 @@ static void valuePairAssign(tValuePair *dst, int16_t type, const char *val, int6
ctx->tag.nType = TSDB_DATA_TYPE_BIGINT;
ctx->tag.i64 = tsKey;
}
tVariantDump(&ctx->tag, dst->pTags + size, ctx->tag.nType, true);
size += pTagInfo->pTagCtxList[i]->outputBytes;
}
......@@ -2589,7 +2590,7 @@ static void do_top_function_add(STopBotInfo *pInfo, int32_t maxLen, void *pData,
SExtTagsInfo *pTagInfo, char *pTags, int16_t stage) {
tVariant val = {0};
tVariantCreateFromBinary(&val, pData, tDataTypes[type].bytes, type);
tValuePair **pList = pInfo->res;
assert(pList != NULL);
......@@ -2597,7 +2598,7 @@ static void do_top_function_add(STopBotInfo *pInfo, int32_t maxLen, void *pData,
valuePairAssign(pList[pInfo->num], type, (const char *)&val.i64, ts, pTags, pTagInfo, stage);
taosheapsort((void *) pList, sizeof(tValuePair **), pInfo->num + 1, (const void *) &type, topBotComparFn, (const void *) &pTagInfo->tagsLen, topBotSwapFn, 0);
pInfo->num++;
} else {
if ((IS_SIGNED_NUMERIC_TYPE(type) && val.i64 > pList[0]->v.i64) ||
......@@ -2636,7 +2637,7 @@ static void do_bottom_function_add(STopBotInfo *pInfo, int32_t maxLen, void *pDa
static int32_t resAscComparFn(const void *pLeft, const void *pRight) {
tValuePair *pLeftElem = *(tValuePair **)pLeft;
tValuePair *pRightElem = *(tValuePair **)pRight;
if (pLeftElem->timestamp == pRightElem->timestamp) {
return 0;
} else {
......@@ -2649,7 +2650,7 @@ static int32_t resDescComparFn(const void *pLeft, const void *pRight) { return -
static int32_t resDataAscComparFn(const void *pLeft, const void *pRight) {
tValuePair *pLeftElem = *(tValuePair **)pLeft;
tValuePair *pRightElem = *(tValuePair **)pRight;
if (IS_FLOAT_TYPE(pLeftElem->v.nType)) {
if (pLeftElem->v.dKey == pRightElem->v.dKey) {
return 0;
......@@ -2676,9 +2677,9 @@ static int32_t resDataDescComparFn(const void *pLeft, const void *pRight) { retu
static void copyTopBotRes(SQLFunctionCtx *pCtx, int32_t type) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
STopBotInfo *pRes = GET_ROWCELL_INTERBUF(pResInfo);
tValuePair **tvp = pRes->res;
int32_t step = QUERY_ASC_FORWARD_STEP;
int32_t len = (int32_t)(GET_RES_INFO(pCtx)->numOfRes);
......@@ -2734,13 +2735,13 @@ static void copyTopBotRes(SQLFunctionCtx *pCtx, int32_t type) {
return;
}
}
// set the output timestamp of each record.
TSKEY *output = pCtx->ptsOutputBuf;
for (int32_t i = 0; i < len; ++i, output += step) {
*output = tvp[i]->timestamp;
}
// set the corresponding tag data for each record
// todo check malloc failure
if (pCtx->tagInfo.numOfTagCols == 0) {
......@@ -2751,7 +2752,7 @@ static void copyTopBotRes(SQLFunctionCtx *pCtx, int32_t type) {
for (int32_t i = 0; i < pCtx->tagInfo.numOfTagCols; ++i) {
pData[i] = pCtx->tagInfo.pTagCtxList[i]->pOutput;
}
for (int32_t i = 0; i < len; ++i, output += step) {
int16_t offset = 0;
for (int32_t j = 0; j < pCtx->tagInfo.numOfTagCols; ++j) {
......@@ -2760,7 +2761,7 @@ static void copyTopBotRes(SQLFunctionCtx *pCtx, int32_t type) {
pData[j] += pCtx->tagInfo.pTagCtxList[j]->outputBytes;
}
}
tfree(pData);
}
......@@ -2791,18 +2792,18 @@ bool topbot_datablock_filter(SQLFunctionCtx *pCtx, const char *minval, const cha
}
STopBotInfo *pTopBotInfo = getOutputInfo(pCtx);
// required number of results are not reached, continue load data block
if (pTopBotInfo->num < pCtx->param[0].i64) {
return true;
}
if ((void *)pTopBotInfo->res[0] != (void *)((char *)pTopBotInfo + sizeof(STopBotInfo) + POINTER_BYTES * pCtx->param[0].i64)) {
buildTopBotStruct(pTopBotInfo, pCtx);
}
tValuePair **pRes = (tValuePair**) pTopBotInfo->res;
if (pCtx->functionId == TSDB_FUNC_TOP) {
switch (pCtx->inputType) {
case TSDB_DATA_TYPE_TINYINT:
......@@ -2844,7 +2845,7 @@ static bool top_bottom_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo*
if (!function_setup(pCtx, pResInfo)) {
return false;
}
STopBotInfo *pInfo = getOutputInfo(pCtx);
buildTopBotStruct(pInfo, pCtx);
return true;
......@@ -2859,27 +2860,27 @@ static void top_function(SQLFunctionCtx *pCtx) {
if ((void *)pRes->res[0] != (void *)((char *)pRes + sizeof(STopBotInfo) + POINTER_BYTES * pCtx->param[0].i64)) {
buildTopBotStruct(pRes, pCtx);
}
for (int32_t i = 0; i < pCtx->size; ++i) {
char *data = GET_INPUT_DATA(pCtx, i);
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
continue;
}
notNullElems++;
// NOTE: Set the default timestamp if it is missing [todo refactor]
TSKEY ts = (pCtx->ptsList != NULL)? GET_TS_DATA(pCtx, i):0;
do_top_function_add(pRes, (int32_t)pCtx->param[0].i64, data, ts, pCtx->inputType, &pCtx->tagInfo, NULL, 0);
}
if (!pCtx->hasNull) {
assert(pCtx->size == notNullElems);
}
// treat the result as only one result
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
......@@ -2888,21 +2889,21 @@ static void top_function(SQLFunctionCtx *pCtx) {
static void top_func_merge(SQLFunctionCtx *pCtx) {
STopBotInfo *pInput = (STopBotInfo *)GET_INPUT_DATA_LIST(pCtx);
// construct the input data struct from binary data
buildTopBotStruct(pInput, pCtx);
STopBotInfo *pOutput = getOutputInfo(pCtx);
// the intermediate result is binary, we only use the output data type
for (int32_t i = 0; i < pInput->num; ++i) {
int16_t type = (pCtx->outputType == TSDB_DATA_TYPE_FLOAT)? TSDB_DATA_TYPE_DOUBLE:pCtx->outputType;
do_top_function_add(pOutput, (int32_t)pCtx->param[0].i64, &pInput->res[i]->v.i64, pInput->res[i]->timestamp,
type, &pCtx->tagInfo, pInput->res[i]->pTags, pCtx->currentStage);
}
SET_VAL(pCtx, pInput->num, pOutput->num);
if (pOutput->num > 0) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
......@@ -2911,9 +2912,9 @@ static void top_func_merge(SQLFunctionCtx *pCtx) {
static void bottom_function(SQLFunctionCtx *pCtx) {
int32_t notNullElems = 0;
STopBotInfo *pRes = getOutputInfo(pCtx);
if ((void *)pRes->res[0] != (void *)((char *)pRes + sizeof(STopBotInfo) + POINTER_BYTES * pCtx->param[0].i64)) {
buildTopBotStruct(pRes, pCtx);
}
......@@ -2929,14 +2930,14 @@ static void bottom_function(SQLFunctionCtx *pCtx) {
TSKEY ts = (pCtx->ptsList != NULL)? GET_TS_DATA(pCtx, i):0;
do_bottom_function_add(pRes, (int32_t)pCtx->param[0].i64, data, ts, pCtx->inputType, &pCtx->tagInfo, NULL, 0);
}
if (!pCtx->hasNull) {
assert(pCtx->size == notNullElems);
}
// treat the result as only one result
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
......@@ -2945,12 +2946,12 @@ static void bottom_function(SQLFunctionCtx *pCtx) {
static void bottom_func_merge(SQLFunctionCtx *pCtx) {
STopBotInfo *pInput = (STopBotInfo *)GET_INPUT_DATA_LIST(pCtx);
// construct the input data struct from binary data
buildTopBotStruct(pInput, pCtx);
STopBotInfo *pOutput = getOutputInfo(pCtx);
// the intermediate result is binary, we only use the output data type
for (int32_t i = 0; i < pInput->num; ++i) {
int16_t type = (pCtx->outputType == TSDB_DATA_TYPE_FLOAT) ? TSDB_DATA_TYPE_DOUBLE : pCtx->outputType;
......@@ -2959,7 +2960,7 @@ static void bottom_func_merge(SQLFunctionCtx *pCtx) {
}
SET_VAL(pCtx, pInput->num, pOutput->num);
if (pOutput->num > 0) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
......@@ -2968,17 +2969,17 @@ static void bottom_func_merge(SQLFunctionCtx *pCtx) {
static void top_bottom_func_finalizer(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
// data in temporary list is less than the required number of results, not enough qualified number of results
STopBotInfo *pRes = GET_ROWCELL_INTERBUF(pResInfo);
if (pRes->num == 0) { // no result
assert(pResInfo->hasResult != DATA_SET_FLAG);
// TODO:
}
GET_RES_INFO(pCtx)->numOfRes = pRes->num;
tValuePair **tvp = pRes->res;
// user specify the order of output by sort the result according to timestamp
if (pCtx->param[2].i64 == PRIMARYKEY_TIMESTAMP_COL_INDEX) {
__compar_fn_t comparator = (pCtx->param[3].i64 == TSDB_ORDER_ASC) ? resAscComparFn : resDescComparFn;
......@@ -2987,10 +2988,10 @@ static void top_bottom_func_finalizer(SQLFunctionCtx *pCtx) {
__compar_fn_t comparator = (pCtx->param[3].i64 == TSDB_ORDER_ASC) ? resDataAscComparFn : resDataDescComparFn;
qsort(tvp, (size_t)pResInfo->numOfRes, POINTER_BYTES, comparator);
}
GET_TRUE_DATA_TYPE();
copyTopBotRes(pCtx, type);
doFinalizer(pCtx);
}
......@@ -3011,7 +3012,7 @@ static bool percentile_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo*
static void percentile_function(SQLFunctionCtx *pCtx) {
int32_t notNullElems = 0;
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
SPercentileInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
......@@ -3021,7 +3022,7 @@ static void percentile_function(SQLFunctionCtx *pCtx) {
// all data are null, set it completed
if (pInfo->numOfElems == 0) {
pResInfo->complete = true;
return;
} else {
pInfo->pMemBucket = tMemBucketCreate(pCtx->inputBytes, pCtx->inputType, pInfo->minval, pInfo->maxval);
......@@ -3085,18 +3086,18 @@ static void percentile_function(SQLFunctionCtx *pCtx) {
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
continue;
}
notNullElems += 1;
tMemBucketPut(pInfo->pMemBucket, data, 1);
}
SET_VAL(pCtx, notNullElems, 1);
pResInfo->hasResult = DATA_SET_FLAG;
}
static void percentile_finalizer(SQLFunctionCtx *pCtx) {
double v = pCtx->param[0].nType == TSDB_DATA_TYPE_INT ? pCtx->param[0].i64 : pCtx->param[0].dKey;
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
SPercentileInfo* ppInfo = (SPercentileInfo *) GET_ROWCELL_INTERBUF(pResInfo);
......@@ -3108,7 +3109,7 @@ static void percentile_finalizer(SQLFunctionCtx *pCtx) {
} else {
SET_DOUBLE_VAL((double *)pCtx->pOutput, getPercentile(pMemBucket, v));
}
tMemBucketDestroy(pMemBucket);
doFinalizer(pCtx);
}
......@@ -3128,7 +3129,7 @@ static bool tdigest_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo *pResultInfo)
if (!function_setup(pCtx, pResultInfo)) {
return false;
}
// new TDigest
SAPercentileInfo *pInfo = getOutputInfo(pCtx);
char *tmp = (char *)pInfo + sizeof(SAPercentileInfo);
......@@ -3155,7 +3156,7 @@ static void tdigest_do(SQLFunctionCtx *pCtx) {
}
notNullElems += 1;
double v = 0; // value
double v = 0; // value
long long w = 1; // weigth
GET_TYPED_DATA(v, double, pCtx->inputType, data);
tdigestAdd(pAPerc->pTDigest, v, w);
......@@ -3189,7 +3190,7 @@ static void tdigest_merge(SQLFunctionCtx *pCtx) {
} else {
tdigestMerge(pOutput->pTDigest, pInput->pTDigest);
}
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
pResInfo->hasResult = DATA_SET_FLAG;
SET_VAL(pCtx, 1, 1);
......@@ -3242,10 +3243,10 @@ static bool apercentile_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo*
if (!function_setup(pCtx, pResultInfo)) {
return false;
}
SAPercentileInfo *pInfo = getOutputInfo(pCtx);
buildHistogramInfo(pInfo);
char *tmp = (char *)pInfo + sizeof(SAPercentileInfo);
pInfo->pHisto = tHistogramCreateFrom(tmp, MAX_HISTOGRAM_BIN);
return true;
......@@ -3258,32 +3259,32 @@ static void apercentile_function(SQLFunctionCtx *pCtx) {
}
int32_t notNullElems = 0;
SResultRowCellInfo * pResInfo = GET_RES_INFO(pCtx);
SAPercentileInfo *pInfo = getOutputInfo(pCtx);
buildHistogramInfo(pInfo);
assert(pInfo->pHisto->elems != NULL);
for (int32_t i = 0; i < pCtx->size; ++i) {
char *data = GET_INPUT_DATA(pCtx, i);
if (pCtx->hasNull && isNull(data, pCtx->inputType)) {
continue;
}
notNullElems += 1;
double v = 0;
GET_TYPED_DATA(v, double, pCtx->inputType, data);
tHistogramAdd(&pInfo->pHisto, v);
}
if (!pCtx->hasNull) {
assert(pCtx->size == notNullElems);
}
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
pResInfo->hasResult = DATA_SET_FLAG;
}
......@@ -3296,23 +3297,23 @@ static void apercentile_func_merge(SQLFunctionCtx *pCtx) {
}
SAPercentileInfo *pInput = (SAPercentileInfo *)GET_INPUT_DATA_LIST(pCtx);
pInput->pHisto = (SHistogramInfo*) ((char *)pInput + sizeof(SAPercentileInfo));
pInput->pHisto->elems = (SHistBin*) ((char *)pInput->pHisto + sizeof(SHistogramInfo));
if (pInput->pHisto->numOfElems <= 0) {
return;
}
SAPercentileInfo *pOutput = getOutputInfo(pCtx);
buildHistogramInfo(pOutput);
SHistogramInfo *pHisto = pOutput->pHisto;
if (pHisto->numOfElems <= 0) {
memcpy(pHisto, pInput->pHisto, sizeof(SHistogramInfo) + sizeof(SHistBin) * (MAX_HISTOGRAM_BIN + 1));
pHisto->elems = (SHistBin*) ((char *)pHisto + sizeof(SHistogramInfo));
} else {
//TODO(dengyihao): avoid memcpy
//TODO(dengyihao): avoid memcpy
pHisto->elems = (SHistBin*) ((char *)pHisto + sizeof(SHistogramInfo));
SHistogramInfo *pRes = tHistogramMerge(pHisto, pInput->pHisto, MAX_HISTOGRAM_BIN);
memcpy(pHisto, pRes, sizeof(SHistogramInfo) + sizeof(SHistBin) * MAX_HISTOGRAM_BIN);
......@@ -3332,17 +3333,17 @@ static void apercentile_finalizer(SQLFunctionCtx *pCtx) {
}
double v = (pCtx->param[0].nType == TSDB_DATA_TYPE_INT) ? pCtx->param[0].i64 : pCtx->param[0].dKey;
SResultRowCellInfo * pResInfo = GET_RES_INFO(pCtx);
SAPercentileInfo *pOutput = GET_ROWCELL_INTERBUF(pResInfo);
if (pCtx->currentStage == MERGE_STAGE) {
if (pResInfo->hasResult == DATA_SET_FLAG) { // check for null
assert(pOutput->pHisto->numOfElems > 0);
double ratio[] = {v};
double *res = tHistogramUniform(pOutput->pHisto, ratio, 1);
memcpy(pCtx->pOutput, res, sizeof(double));
free(res);
} else {
......@@ -3352,7 +3353,7 @@ static void apercentile_finalizer(SQLFunctionCtx *pCtx) {
} else {
if (pOutput->pHisto->numOfElems > 0) {
double ratio[] = {v};
double *res = tHistogramUniform(pOutput->pHisto, ratio, 1);
memcpy(pCtx->pOutput, res, sizeof(double));
free(res);
......@@ -3361,7 +3362,7 @@ static void apercentile_finalizer(SQLFunctionCtx *pCtx) {
return;
}
}
doFinalizer(pCtx);
}
......@@ -3372,7 +3373,7 @@ static bool leastsquares_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo
}
SLeastsquaresInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
// 2*3 matrix
pInfo->startVal = pCtx->param[0].dKey;
return true;
......@@ -3399,12 +3400,12 @@ static bool leastsquares_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo
static void leastsquares_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo * pResInfo = GET_RES_INFO(pCtx);
SLeastsquaresInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
double(*param)[3] = pInfo->mat;
double x = pInfo->startVal;
void *pData = GET_INPUT_DATA_LIST(pCtx);
int32_t numOfElem = 0;
switch (pCtx->inputType) {
case TSDB_DATA_TYPE_INT: {
......@@ -3414,12 +3415,12 @@ static void leastsquares_function(SQLFunctionCtx *pCtx) {
if (pCtx->hasNull && isNull((const char*) p, pCtx->inputType)) {
continue;
}
param[0][0] += x * x;
param[0][1] += x;
param[0][2] += x * p[i];
param[1][2] += p[i];
x += pCtx->param[1].dKey;
numOfElem++;
}
......@@ -3471,14 +3472,14 @@ static void leastsquares_function(SQLFunctionCtx *pCtx) {
break;
}
}
pInfo->startVal = x;
pInfo->num += numOfElem;
if (pInfo->num > 0) {
pResInfo->hasResult = DATA_SET_FLAG;
}
SET_VAL(pCtx, numOfElem, 1);
}
......@@ -3486,30 +3487,30 @@ static void leastsquares_finalizer(SQLFunctionCtx *pCtx) {
// no data in query
SResultRowCellInfo * pResInfo = GET_RES_INFO(pCtx);
SLeastsquaresInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
if (pInfo->num == 0) {
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
return;
}
double(*param)[3] = pInfo->mat;
param[1][1] = (double)pInfo->num;
param[1][0] = param[0][1];
param[0][0] -= param[1][0] * (param[0][1] / param[1][1]);
param[0][2] -= param[1][2] * (param[0][1] / param[1][1]);
param[0][1] = 0;
param[1][2] -= param[0][2] * (param[1][0] / param[0][0]);
param[1][0] = 0;
param[0][2] /= param[0][0];
param[1][2] /= param[1][1];
int32_t maxOutputSize = TSDB_AVG_FUNCTION_INTER_BUFFER_SIZE - VARSTR_HEADER_SIZE;
size_t n = snprintf(varDataVal(pCtx->pOutput), maxOutputSize, "{slop:%.6lf, intercept:%.6lf}",
param[0][2], param[1][2]);
varDataSetLen(pCtx->pOutput, n);
doFinalizer(pCtx);
}
......@@ -3562,7 +3563,7 @@ static void col_project_function(SQLFunctionCtx *pCtx) {
*/
static void tag_project_function(SQLFunctionCtx *pCtx) {
INC_INIT_VAL(pCtx, pCtx->size);
assert(pCtx->inputBytes == pCtx->outputBytes);
tVariantDump(&pCtx->tag, pCtx->pOutput, pCtx->outputType, true);
......@@ -3597,7 +3598,7 @@ static void tag_function(SQLFunctionCtx *pCtx) {
static void copy_function(SQLFunctionCtx *pCtx) {
SET_VAL(pCtx, pCtx->size, 1);
char *pData = GET_INPUT_DATA_LIST(pCtx);
assignVal(pCtx->pOutput, pData, pCtx->inputBytes, pCtx->inputType);
}
......@@ -3668,7 +3669,7 @@ static void row_copy_function(SQLFunctionCtx *pCtx) {
static void full_copy_function(SQLFunctionCtx *pCtx) {
copy_function(pCtx);
for (int t = 0; t < pCtx->tagInfo.numOfTagCols; ++t) {
SQLFunctionCtx* tagCtx = pCtx->tagInfo.pTagCtxList[t];
if (tagCtx->functionId == TSDB_FUNC_TAG_DUMMY) {
......@@ -3682,7 +3683,7 @@ static bool diff_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pResIn
if (!function_setup(pCtx, pResInfo)) {
return false;
}
SDiffFuncInfo* pDiffInfo = GET_ROWCELL_INTERBUF(pResInfo);
pDiffInfo->valueAssigned = false;
pDiffInfo->i64Prev = 0;
......@@ -3997,10 +3998,10 @@ static void diff_function(SQLFunctionCtx *pCtx) {
continue;
}
if (pDiffInfo->valueAssigned) {
if (pDiffInfo->valueAssigned) {
float diff = (float)(pData[i] - pDiffInfo->d64Prev);
if (diff >= 0 || !pDiffInfo->ignoreNegative) {
*pOutput = diff;
*pOutput = diff;
*pTimestamp = (tsList != NULL)? tsList[i]:0;
pOutput += 1;
pTimestamp += 1;
......@@ -4084,7 +4085,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
char *getScalarExprColumnData(void *param, const char* name, int32_t colId) {
SScalarExprSupport *pSupport = (SScalarExprSupport *)param;
int32_t idx = -1;
for (int32_t i = 0; i < pSupport->numOfCols; ++i) {
if (colId == pSupport->colList[i].colId) {
......@@ -4092,7 +4093,7 @@ char *getScalarExprColumnData(void *param, const char* name, int32_t colId) {
break;
}
}
assert(idx >= 0);
return pSupport->data[idx] + pSupport->offset * pSupport->colList[idx].bytes;
}
......@@ -4127,9 +4128,9 @@ static bool spread_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pRes
if (!function_setup(pCtx, pResInfo)) {
return false;
}
SSpreadInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
// this is the server-side setup function in client-side, the secondary merge do not need this procedure
if (pCtx->currentStage == MERGE_STAGE) {
pCtx->param[0].dKey = DBL_MAX;
......@@ -4138,21 +4139,21 @@ static bool spread_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pRes
pInfo->min = DBL_MAX;
pInfo->max = -DBL_MAX;
}
return true;
}
static void spread_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
SSpreadInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
int32_t numOfElems = 0;
// todo : opt with pre-calculated result
// column missing cause the hasNull to be true
if (pCtx->preAggVals.isSet) {
numOfElems = pCtx->size - pCtx->preAggVals.statis.numOfNull;
// all data are null in current data block, ignore current data block
if (numOfElems == 0) {
goto _spread_over;
......@@ -4171,18 +4172,18 @@ static void spread_function(SQLFunctionCtx *pCtx) {
if (pInfo->min > GET_DOUBLE_VAL((const char *)&(pCtx->preAggVals.statis.min))) {
pInfo->min = GET_DOUBLE_VAL((const char *)&(pCtx->preAggVals.statis.min));
}
if (pInfo->max < GET_DOUBLE_VAL((const char *)&(pCtx->preAggVals.statis.max))) {
pInfo->max = GET_DOUBLE_VAL((const char *)&(pCtx->preAggVals.statis.max));
}
}
goto _spread_over;
}
void *pData = GET_INPUT_DATA_LIST(pCtx);
numOfElems = 0;
if (pCtx->inputType == TSDB_DATA_TYPE_TINYINT) {
LIST_MINMAX_N(pCtx, pInfo->min, pInfo->max, pCtx->size, pData, int8_t, pCtx->inputType, numOfElems);
} else if (pCtx->inputType == TSDB_DATA_TYPE_SMALLINT) {
......@@ -4204,19 +4205,19 @@ static void spread_function(SQLFunctionCtx *pCtx) {
} else if (pCtx->inputType == TSDB_DATA_TYPE_UBIGINT) {
LIST_MINMAX_N(pCtx, pInfo->min, pInfo->max, pCtx->size, pData, uint64_t, pCtx->inputType, numOfElems);
}
if (!pCtx->hasNull) {
assert(pCtx->size == numOfElems);
}
_spread_over:
SET_VAL(pCtx, numOfElems, 1);
if (numOfElems > 0) {
pResInfo->hasResult = DATA_SET_FLAG;
pInfo->hasResult = DATA_SET_FLAG;
}
// keep the data into the final output buffer for super table query since this execution may be the last one
if (pCtx->stableQuery) {
memcpy(pCtx->pOutput, GET_ROWCELL_INTERBUF(pResInfo), sizeof(SSpreadInfo));
......@@ -4232,15 +4233,15 @@ void spread_func_merge(SQLFunctionCtx *pCtx) {
if (pData->hasResult != DATA_SET_FLAG) {
return;
}
if (pCtx->param[0].dKey > pData->min) {
pCtx->param[0].dKey = pData->min;
}
if (pCtx->param[3].dKey < pData->max) {
pCtx->param[3].dKey = pData->max;
}
GET_RES_INFO(pCtx)->hasResult = DATA_SET_FLAG;
}
......@@ -4253,25 +4254,25 @@ void spread_function_finalizer(SQLFunctionCtx *pCtx) {
if (pCtx->currentStage == MERGE_STAGE) {
assert(pCtx->inputType == TSDB_DATA_TYPE_BINARY);
if (pResInfo->hasResult != DATA_SET_FLAG) {
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
return;
}
SET_DOUBLE_VAL((double *)pCtx->pOutput, pCtx->param[3].dKey - pCtx->param[0].dKey);
} else {
assert(IS_NUMERIC_TYPE(pCtx->inputType) || (pCtx->inputType == TSDB_DATA_TYPE_TIMESTAMP));
SSpreadInfo *pInfo = GET_ROWCELL_INTERBUF(GET_RES_INFO(pCtx));
if (pInfo->hasResult != DATA_SET_FLAG) {
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
return;
}
SET_DOUBLE_VAL((double *)pCtx->pOutput, pInfo->max - pInfo->min);
}
GET_RES_INFO(pCtx)->numOfRes = 1; // todo add test case
doFinalizer(pCtx);
}
......@@ -4354,7 +4355,7 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4366,14 +4367,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4385,14 +4386,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4404,14 +4405,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = (double) val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = (double)val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4423,14 +4424,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = (double)val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4442,14 +4443,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4468,7 +4469,7 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4487,7 +4488,7 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4506,7 +4507,7 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
SPoint1 st;
st.key = tsList[i];
st.val = val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4518,14 +4519,14 @@ static int32_t twa_function_impl(SQLFunctionCtx* pCtx, int32_t idx, int32_t size
if (pCtx->hasNull && isNull((const char*) &val[i], pCtx->inputType)) {
continue;
}
#ifndef _TD_NINGSI_60
SPoint1 st = {.key = tsList[i], .val = (double) val[i]};
#else
SPoint1 st;
st.key = tsList[i];
st.val = (double) val[i];
#endif
#endif
pInfo->dOutput += twa_get_area(pInfo->p, st);
pInfo->p = st;
}
......@@ -4549,7 +4550,7 @@ static void twa_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
STwaInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
// skip null value
int32_t step = GET_FORWARD_DIRECTION_FACTOR(pCtx->order);
int32_t i = (pCtx->order == TSDB_ORDER_ASC)? 0:(pCtx->size - 1);
......@@ -4563,11 +4564,11 @@ static void twa_function(SQLFunctionCtx *pCtx) {
}
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
pResInfo->hasResult = DATA_SET_FLAG;
}
if (pCtx->stableQuery) {
memcpy(pCtx->pOutput, pInfo, sizeof(STwaInfo));
}
......@@ -4581,14 +4582,14 @@ static void twa_function(SQLFunctionCtx *pCtx) {
void twa_function_copy(SQLFunctionCtx *pCtx) {
assert(pCtx->inputType == TSDB_DATA_TYPE_BINARY);
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
memcpy(GET_ROWCELL_INTERBUF(pResInfo), pCtx->pInput, (size_t)pCtx->inputBytes);
pResInfo->hasResult = ((STwaInfo *)pCtx->pInput)->hasResult;
}
void twa_function_finalizer(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
STwaInfo *pInfo = (STwaInfo *)GET_ROWCELL_INTERBUF(pResInfo);
if (pInfo->hasResult != DATA_SET_FLAG) {
setNull(pCtx->pOutput, TSDB_DATA_TYPE_DOUBLE, sizeof(double));
......@@ -4601,7 +4602,7 @@ void twa_function_finalizer(SQLFunctionCtx *pCtx) {
} else {
SET_DOUBLE_VAL((double *)pCtx->pOutput , pInfo->dOutput / (pInfo->win.ekey - pInfo->win.skey));
}
GET_RES_INFO(pCtx)->numOfRes = 1;
doFinalizer(pCtx);
}
......@@ -4612,27 +4613,27 @@ static void interp_function(SQLFunctionCtx *pCtx) {
if (pCtx->start.key == pCtx->startTs) {
assert(pCtx->start.key != INT64_MIN);
COPY_TYPED_DATA(pCtx->pOutput, pCtx->inputType, &pCtx->start.val);
goto interp_success_exit;
goto interp_success_exit;
} else if (pCtx->end.key == pCtx->startTs && pCtx->end.key != INT64_MIN && fillType == TSDB_FILL_NEXT) {
COPY_TYPED_DATA(pCtx->pOutput, pCtx->inputType, &pCtx->end.val);
goto interp_success_exit;
goto interp_success_exit;
}
switch (fillType) {
case TSDB_FILL_NULL:
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
break;
case TSDB_FILL_SET_VALUE:
tVariantDump(&pCtx->param[1], pCtx->pOutput, pCtx->inputType, true);
break;
case TSDB_FILL_LINEAR:
if (pCtx->start.key == INT64_MIN || pCtx->start.key > pCtx->startTs
if (pCtx->start.key == INT64_MIN || pCtx->start.key > pCtx->startTs
|| pCtx->end.key == INT64_MIN || pCtx->end.key < pCtx->startTs) {
goto interp_exit;
}
......@@ -4640,7 +4641,7 @@ static void interp_function(SQLFunctionCtx *pCtx) {
double v1 = -1, v2 = -1;
GET_TYPED_DATA(v1, double, pCtx->inputType, &pCtx->start.val);
GET_TYPED_DATA(v2, double, pCtx->inputType, &pCtx->end.val);
SPoint point1 = {.key = pCtx->start.key, .val = &v1};
SPoint point2 = {.key = pCtx->end.key, .val = &v2};
SPoint point = {.key = pCtx->startTs, .val = pCtx->pOutput};
......@@ -4653,7 +4654,7 @@ static void interp_function(SQLFunctionCtx *pCtx) {
taosGetLinearInterpolationVal(&point, pCtx->outputType, &point1, &point2, TSDB_DATA_TYPE_DOUBLE, &exceedMax, &exceedMin);
if (exceedMax || exceedMin) {
__compar_fn_t func = getComparFunc((int32_t)pCtx->inputType, 0);
if (func(&pCtx->start.val, &pCtx->end.val) <= 0) {
if (func(&pCtx->start.val, &pCtx->end.val) <= 0) {
COPY_TYPED_DATA(pCtx->pOutput, pCtx->inputType, exceedMax ? &pCtx->start.val : &pCtx->end.val);
} else {
COPY_TYPED_DATA(pCtx->pOutput, pCtx->inputType, exceedMax ? &pCtx->end.val : &pCtx->start.val);
......@@ -4674,7 +4675,7 @@ static void interp_function(SQLFunctionCtx *pCtx) {
if (pCtx->end.key == INT64_MIN || pCtx->end.key < pCtx->startTs) {
goto interp_exit;
}
COPY_TYPED_DATA(pCtx->pOutput, pCtx->inputType, &pCtx->end.val);
break;
......@@ -4684,7 +4685,7 @@ static void interp_function(SQLFunctionCtx *pCtx) {
}
interp_success_exit:
interp_success_exit:
*(TSKEY*)pCtx->ptsOutputBuf = pCtx->startTs;
......@@ -4713,9 +4714,9 @@ static bool ts_comp_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pRe
static void ts_comp_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
STSBuf * pTSbuf = ((STSCompInfo *)(GET_ROWCELL_INTERBUF(pResInfo)))->pTSBuf;
const char *input = GET_INPUT_DATA_LIST(pCtx);
// primary ts must be existed, so no need to check its existance
if (pCtx->order == TSDB_ORDER_ASC) {
tsBufAppend(pTSbuf, (int32_t)pCtx->param[0].i64, &pCtx->tag, input, pCtx->size * TSDB_KEYSIZE);
......@@ -4725,17 +4726,17 @@ static void ts_comp_function(SQLFunctionCtx *pCtx) {
tsBufAppend(pTSbuf, (int32_t)pCtx->param[0].i64, &pCtx->tag, d, (int32_t)TSDB_KEYSIZE);
}
}
SET_VAL(pCtx, pCtx->size, 1);
pResInfo->hasResult = DATA_SET_FLAG;
}
static void ts_comp_finalize(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
STSCompInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
STSBuf * pTSbuf = pInfo->pTSBuf;
tsBufFlush(pTSbuf);
qDebug("total timestamp :%"PRId64, pTSbuf->numOfTotal);
......@@ -4778,7 +4779,7 @@ static double do_calc_rate(const SRateInfo* pRateInfo, double tickPerSec) {
return 0;
}
}
int64_t duration = pRateInfo->lastKey - pRateInfo->firstKey;
if (duration == 0) {
return 0;
......@@ -4791,7 +4792,7 @@ static bool rate_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pResIn
if (!function_setup(pCtx, pResInfo)) {
return false;
}
SRateInfo *pInfo = GET_ROWCELL_INTERBUF(pResInfo);
pInfo->correctionValue = 0;
pInfo->firstKey = INT64_MIN;
......@@ -4806,51 +4807,51 @@ static bool rate_function_setup(SQLFunctionCtx *pCtx, SResultRowCellInfo* pResIn
static void rate_function(SQLFunctionCtx *pCtx) {
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
int32_t notNullElems = 0;
SRateInfo *pRateInfo = (SRateInfo *)GET_ROWCELL_INTERBUF(pResInfo);
TSKEY *primaryKey = GET_TS_LIST(pCtx);
qDebug("%p rate_function() size:%d, hasNull:%d", pCtx, pCtx->size, pCtx->hasNull);
for (int32_t i = 0; i < pCtx->size; ++i) {
char *pData = GET_INPUT_DATA(pCtx, i);
if (pCtx->hasNull && isNull(pData, pCtx->inputType)) {
qDebug("%p rate_function() index of null data:%d", pCtx, i);
continue;
}
notNullElems++;
double v = 0;
GET_TYPED_DATA(v, double, pCtx->inputType, pData);
if ((INT64_MIN == pRateInfo->firstValue) || (INT64_MIN == pRateInfo->firstKey)) {
pRateInfo->firstValue = v;
pRateInfo->firstKey = primaryKey[i];
}
if (INT64_MIN == pRateInfo->lastValue) {
pRateInfo->lastValue = v;
} else if (v < pRateInfo->lastValue) {
pRateInfo->correctionValue += pRateInfo->lastValue;
}
pRateInfo->lastValue = v;
pRateInfo->lastKey = primaryKey[i];
}
if (!pCtx->hasNull) {
assert(pCtx->size == notNullElems);
}
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
pRateInfo->hasResult = DATA_SET_FLAG;
pResInfo->hasResult = DATA_SET_FLAG;
}
// keep the data into the final output buffer for super table query since this execution may be the last one
if (pCtx->stableQuery) {
memcpy(pCtx->pOutput, GET_ROWCELL_INTERBUF(pResInfo), sizeof(SRateInfo));
......@@ -4859,7 +4860,7 @@ static void rate_function(SQLFunctionCtx *pCtx) {
static void rate_func_copy(SQLFunctionCtx *pCtx) {
assert(pCtx->inputType == TSDB_DATA_TYPE_BINARY);
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
memcpy(GET_ROWCELL_INTERBUF(pResInfo), pCtx->pInput, (size_t)pCtx->inputBytes);
pResInfo->hasResult = ((SRateInfo*)pCtx->pInput)->hasResult;
......@@ -4873,13 +4874,13 @@ static void rate_finalizer(SQLFunctionCtx *pCtx) {
setNull(pCtx->pOutput, TSDB_DATA_TYPE_DOUBLE, sizeof(double));
return;
}
SET_DOUBLE_VAL((double*) pCtx->pOutput, do_calc_rate(pRateInfo, (double) TSDB_TICK_PER_SECOND(pCtx->param[0].i64)));
// cannot set the numOfIteratedElems again since it is set during previous iteration
pResInfo->numOfRes = 1;
pResInfo->hasResult = DATA_SET_FLAG;
doFinalizer(pCtx);
}
......@@ -4895,9 +4896,9 @@ static void irate_function(SQLFunctionCtx *pCtx) {
if (pCtx->hasNull && isNull(pData, pCtx->inputType)) {
continue;
}
notNullElems++;
double v = 0;
GET_TYPED_DATA(v, double, pCtx->inputType, pData);
......@@ -4915,24 +4916,24 @@ static void irate_function(SQLFunctionCtx *pCtx) {
pRateInfo->lastValue = v;
pRateInfo->lastKey = primaryKey[i];
continue;
}
if ((INT64_MIN == pRateInfo->firstKey) || primaryKey[i] > pRateInfo->firstKey) {
pRateInfo->firstValue = v;
pRateInfo->firstKey = primaryKey[i];
break;
}
}
SET_VAL(pCtx, notNullElems, 1);
if (notNullElems > 0) {
pRateInfo->hasResult = DATA_SET_FLAG;
pResInfo->hasResult = DATA_SET_FLAG;
}
// keep the data into the final output buffer for super table query since this execution may be the last one
if (pCtx->stableQuery) {
memcpy(pCtx->pOutput, GET_ROWCELL_INTERBUF(pResInfo), sizeof(SRateInfo));
......@@ -5508,7 +5509,7 @@ static void elapsedFunction(SQLFunctionCtx *pCtx) {
elapsedOver:
SET_VAL(pCtx, pCtx->size, 1);
if (pCtx->size > 0) {
GET_RES_INFO(pCtx)->hasResult = DATA_SET_FLAG;
pInfo->hasResult = DATA_SET_FLAG;
......@@ -5901,16 +5902,21 @@ static void mode_func_finalizer(SQLFunctionCtx *pCtx) {
int32_t type = 0;
if (pCtx->currentStage == MERGE_STAGE) {
bytes = pCtx->outputBytes;
type = pCtx->outputType;
type = pCtx->outputType;
assert(pCtx->inputType == TSDB_DATA_TYPE_BINARY);
} else {
bytes = pCtx->inputBytes;
type = pCtx->inputType;
type = pCtx->inputType;
}
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
SModeFuncInfo *pRes = GET_ROWCELL_INTERBUF(pResInfo);
if (pRes->num == 0) {
setNull(pCtx->pOutput, type, 0);
goto _mode_over;
}
size_t size = sizeof(ModeUnit) + bytes;
char *tvp = pRes->res;
......@@ -5918,20 +5924,18 @@ static void mode_func_finalizer(SQLFunctionCtx *pCtx) {
int64_t maxCount = 0;
for (int32_t i = 0; i < pRes->num; ++i) {
int64_t count = ((ModeUnit*)tvp)->count;
if (count > maxCount){
if (count >= maxCount){
maxCount = count;
result = tvp;
}else if(count == maxCount){
result = NULL;
}
tvp += size;
}
if (result){
memcpy(pCtx->pOutput, result + sizeof(ModeUnit), bytes);
}else{
setNull(pCtx->pOutput, type, 0);
}
memcpy(pCtx->pOutput, result + sizeof(ModeUnit), bytes);
_mode_over:
pResInfo->numOfRes = 1;
doFinalizer(pCtx);
}
......
src/query/src/qExecutor.c
浏览文件 @ 574bfd96
......@@ -3668,7 +3668,7 @@ int32_t binarySearchForKey(char* pValue, int num, TSKEY key, int order) {
int32_t midPos = -1;
int32_t numOfRows;
if (num <= 0) {
if (num <= 0 || pValue == NULL) {
return -1;
}
......@@ -5447,7 +5447,8 @@ int32_t doInitQInfo(SQInfo* pQInfo, STSBuf* pTsBuf, void* tsdb, void* sourceOptr
int16_t order = (pQueryAttr->order.order == pRuntimeEnv->pTsBuf->tsOrder) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC;
tsBufResetPos(pRuntimeEnv->pTsBuf);
tsBufSetTraverseOrder(pRuntimeEnv->pTsBuf, order);
tsBufNextPos(pTsBuf);
bool ret = tsBufNextPos(pTsBuf);
UNUSED(ret);
}
int32_t ps = DEFAULT_PAGE_SIZE;
......@@ -6154,7 +6155,7 @@ SOperatorInfo* createGlobalAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv,
goto _clean;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
setDefaultOutputBuf(pRuntimeEnv, &pInfo->binfo, pInfo->seed, MERGE_STAGE);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
......@@ -6329,6 +6330,7 @@ SOperatorInfo* createOrderOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorI
pDataBlock->pDataBlock = taosArrayInit(numOfOutput, sizeof(SColumnInfoData));
if (pDataBlock->pDataBlock == NULL) {
free(pDataBlock);
goto _clean;
}
......@@ -6623,7 +6625,7 @@ static SSDataBlock* doProjectOperation(void* param, bool* newgroup) {
}
copyTsColoum(pRes, pInfo->pCtx, pOperator->numOfOutput);
clearNumOfRes(pInfo->pCtx, pOperator->numOfOutput);
return (pInfo->pRes->info.rows > 0) ? pInfo->pRes : NULL;
return pInfo->pRes;
}
static SSDataBlock* doLimit(void* param, bool* newgroup) {
......@@ -6872,14 +6874,18 @@ static bool doEveryInterpolation(SOperatorInfo* pOperatorInfo, SSDataBlock* pBlo
break;
}
}
if (pCtx == NULL) {
goto group_finished_exit;
}
TSKEY* tsCols = NULL;
if (pBlock && pBlock->pDataBlock != NULL) {
SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, 0);
tsCols = (int64_t*)pColDataInfo->pData;
assert(tsCols[0] == pBlock->info.window.skey && tsCols[pBlock->info.rows - 1] == pBlock->info.window.ekey);
}
if (pCtx->startTs == INT64_MIN) {
if (pQueryAttr->range.skey == INT64_MIN) {
if (NULL == tsCols) {
......@@ -6977,8 +6983,8 @@ static bool doEveryInterpolation(SOperatorInfo* pOperatorInfo, SSDataBlock* pBlo
return false;
}
int32_t startPos = binarySearchForKey((char*)tsCols, pBlock->info.rows, pCtx->startTs, pQueryAttr->order.order);
int32_t nRows = pBlock ? pBlock->info.rows : 0;
int32_t startPos = binarySearchForKey((char*) tsCols, nRows, pCtx->startTs, pQueryAttr->order.order);
if (ascQuery && pQueryAttr->fillType != TSDB_FILL_NEXT && pCtx->start.key == INT64_MIN) {
if (startPos < 0) {
......@@ -7778,7 +7784,7 @@ SOperatorInfo* createAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOpera
goto _clean;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
setDefaultOutputBuf(pRuntimeEnv, &pInfo->binfo, pInfo->seed, MASTER_SCAN);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
......@@ -7985,7 +7991,7 @@ SOperatorInfo* createProjectOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato
return NULL;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
pInfo->bufCapacity = pRuntimeEnv->resultInfo.capacity;
SOptrBasicInfo* pBInfo = &pInfo->binfo;
......@@ -8171,7 +8177,7 @@ SOperatorInfo* createTimeEveryOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOpera
SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr;
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
pInfo->bufCapacity = pRuntimeEnv->resultInfo.capacity;
pInfo->groupDone = true;
pInfo->lastGroupIdx = -1;
......@@ -8516,15 +8522,18 @@ SOperatorInfo* createSLimitOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperator
SColIndex* idx = taosArrayGet(pInfo->orderColumnList, i);
offset += pExpr[idx->colIndex].base.resBytes;
}
pInfo->pRes = createOutputBuf(pExpr, numOfOutput, pRuntimeEnv->resultInfo.capacity);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
if (pInfo->pRes == NULL || pOperator == NULL) {
if (pOperator == NULL) {
goto _clean;
}
pInfo->pRes = createOutputBuf(pExpr, numOfOutput, pRuntimeEnv->resultInfo.capacity);
if (pInfo->pRes == NULL) {
tfree(pOperator);
goto _clean;
}
pOperator->name = "SLimitOperator";
pOperator->operatorType = OP_SLimit;
pOperator->blockingOptr = false;
......
src/query/src/queryMain.c
浏览文件 @ 574bfd96
......@@ -338,6 +338,45 @@ int32_t qRetrieveQueryResultInfo(qinfo_t qinfo, bool* buildRes, void* pRspContex
return code;
}
bool qFitAlwaysValue(SQInfo * pQInfo) {
// must agg query
if (!pQInfo->query.simpleAgg)
return false;
// must not include ts column
SSDataBlock* pBlock = pQInfo->runtimeEnv.outputBuf;
if (pBlock == NULL || pBlock->pDataBlock == NULL)
return false;
SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, 0);
if (pColInfoData == NULL)
return false;
// must not first column is timestamp
if (pColInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP)
return false;
// fit ok
return true;
}
bool doFillResultWithNull(SQInfo* pQInfo, char * data, size_t numOfRows) {
SQueryRuntimeEnv* pRuntimeEnv = &pQInfo->runtimeEnv;
SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr;
SSDataBlock* pRes = pRuntimeEnv->outputBuf;
int32_t numOfCols = pQueryAttr->numOfOutput;
for (int32_t i = 0; i < numOfCols; ++i) {
int16_t functionId = pQueryAttr->pExpr1[i].base.functionId;
// col
SColumnInfoData* pCol = taosArrayGet(pRes->pDataBlock, i);
if (functionId != TSDB_FUNC_COUNT)
setNull(data, pCol->info.type, pCol->info.bytes);
data += pCol->info.bytes * numOfRows;
}
return true;
}
int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *contLen, bool* continueExec) {
SQInfo *pQInfo = (SQInfo *)qinfo;
int32_t compLen = 0;
......@@ -350,6 +389,14 @@ int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *co
SQueryRuntimeEnv* pRuntimeEnv = &pQInfo->runtimeEnv;
int32_t s = GET_NUM_OF_RESULTS(pRuntimeEnv);
bool fillNull = false;
if (s == 0 && tsAggAlways) {
if (qFitAlwaysValue(pQInfo)) {
s = 1;
fillNull = true;
}
}
size_t size = pQueryAttr->resultRowSize * s;
size += sizeof(int32_t);
size += sizeof(STableIdInfo) * taosHashGetSize(pRuntimeEnv->pTableRetrieveTsMap);
......@@ -378,6 +425,9 @@ int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *co
if (GET_NUM_OF_RESULTS(&(pQInfo->runtimeEnv)) > 0 && pQInfo->code == TSDB_CODE_SUCCESS) {
doDumpQueryResult(pQInfo, (*pRsp)->data, (*pRsp)->compressed, &compLen);
} else {
if (fillNull) {
doFillResultWithNull(pQInfo, (*pRsp)->data, 1);
}
setQueryStatus(pRuntimeEnv, QUERY_OVER);
}
......
src/query/src/sql.c
浏览文件 @ 574bfd96
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
src/rpc/src/rpcMain.c
浏览文件 @ 574bfd96
......@@ -1080,7 +1080,9 @@ static void rpcProcessBrokenLink(SRpcConn *pConn) {
pContext->code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
pContext->pConn = NULL;
pConn->pReqMsg = NULL;
taosTmrStart(rpcProcessConnError, 0, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 0, rid, pRpc->tmrCtrl);
}
if (pConn->inType) rpcReportBrokenLinkToServer(pConn);
......@@ -1293,7 +1295,9 @@ static void rpcProcessIncomingMsg(SRpcConn *pConn, SRpcHead *pHead, SRpcReqConte
rpcFreeCont(rpcMsg.pCont);
} else if (pHead->code == TSDB_CODE_RPC_NOT_READY || pHead->code == TSDB_CODE_APP_NOT_READY || pHead->code == TSDB_CODE_DND_EXITING) {
pContext->code = pHead->code;
rpcProcessConnError(pContext, NULL);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
rpcProcessConnError(rid, NULL);
rpcFreeCont(rpcMsg.pCont);
} else {
rpcNotifyClient(pContext, &rpcMsg);
......@@ -1395,7 +1399,9 @@ static TBOOL rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) {
if (pConn == NULL) {
pContext->code = terrno;
// in rpcProcessConnError if numOfTry over limit, could call rpcNotifyClient to stop query
taosTmrStart(rpcProcessConnError, 1, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
return BOOL_ASYNC;
}
......@@ -1442,7 +1448,9 @@ static TBOOL rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) {
// try next ip again
pContext->code = terrno;
// in rpcProcessConnError if numOfTry over limit, could call rpcNotifyClient to stop query
taosTmrStart(rpcProcessConnError, 1, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
return BOOL_ASYNC;
}
......@@ -1480,11 +1488,23 @@ static bool rpcSendMsgToPeer(SRpcConn *pConn, void *msg, int msgLen) {
}
static void rpcProcessConnError(void *param, void *id) {
SRpcReqContext *pContext = (SRpcReqContext *)param;
if (NULL == param) {
return;
}
int64_t *rid = (int64_t*)param;
SRpcReqContext *pContext = (SRpcReqContext *)taosAcquireRef(tsRpcRefId, *rid);
if (NULL == pContext) {
free(param);
return;
}
SRpcInfo *pRpc = pContext->pRpc;
SRpcMsg rpcMsg;
if (pRpc == NULL) {
taosReleaseRef(tsRpcRefId, *rid);
free(param);
return;
}
......@@ -1504,6 +1524,9 @@ static void rpcProcessConnError(void *param, void *id) {
pContext->epSet.inUse = pContext->epSet.inUse % pContext->epSet.numOfEps;
rpcSendReqToServer(pRpc, pContext);
}
taosReleaseRef(tsRpcRefId, *rid);
free(param);
}
static void rpcProcessRetryTimer(void *param, void *tmrId) {
......@@ -1528,7 +1551,9 @@ static void rpcProcessRetryTimer(void *param, void *tmrId) {
pConn->pContext->code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
pConn->pContext->pConn = NULL;
pConn->pReqMsg = NULL;
taosTmrStart(rpcProcessConnError, 1, pConn->pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pConn->pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
rpcReleaseConn(pConn);
}
}
......@@ -1892,4 +1917,4 @@ bool rpcSaveSendInfo(int64_t rpcRid, void** ppContext) {
taosReleaseRef(tsRpcRefId, rpcRid);
return true;
}
\ No newline at end of file
}
src/tsdb/src/tsdbCommitQueue.c
浏览文件 @ 574bfd96
......@@ -132,13 +132,16 @@ static void tsdbApplyRepoConfig(STsdbRepo *pRepo) {
pRepo->config.keep2 = pRepo->save_config.keep2;
pRepo->config.cacheLastRow = pRepo->save_config.cacheLastRow;
pRepo->config.totalBlocks = pRepo->save_config.totalBlocks;
pRepo->config.minRowsPerFileBlock = pRepo->save_config.minRowsPerFileBlock;
pthread_mutex_unlock(&pRepo->save_mutex);
tsdbInfo("vgId:%d apply new config: compression(%d), keep(%d,%d,%d), totalBlocks(%d), cacheLastRow(%d->%d),totalBlocks(%d->%d)",
REPO_ID(pRepo),
pSaveCfg->compression, pSaveCfg->keep,pSaveCfg->keep1, pSaveCfg->keep2,
pSaveCfg->totalBlocks, oldCfg.cacheLastRow, pSaveCfg->cacheLastRow, oldTotalBlocks, pSaveCfg->totalBlocks);
tsdbInfo(
"vgId:%d apply new config: "
"compression(%d),keep(%d,%d,%d),totalBlocks(%d),cacheLastRow(%d->%d),totalBlocks(%d->%d),minRows(%d->%d)",
REPO_ID(pRepo), pSaveCfg->compression, pSaveCfg->keep, pSaveCfg->keep1, pSaveCfg->keep2, pSaveCfg->totalBlocks,
oldCfg.cacheLastRow, pSaveCfg->cacheLastRow, oldTotalBlocks, pSaveCfg->totalBlocks, oldCfg.minRowsPerFileBlock,
pSaveCfg->minRowsPerFileBlock);
int err = tsdbExpandPool(pRepo, oldTotalBlocks);
if (!TAOS_SUCCEEDED(err)) {
......
src/tsdb/src/tsdbMain.c
浏览文件 @ 574bfd96
......@@ -233,7 +233,7 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
ASSERT(pRCfg->tsdbId == pCfg->tsdbId);
ASSERT(pRCfg->cacheBlockSize == pCfg->cacheBlockSize);
ASSERT(pRCfg->daysPerFile == pCfg->daysPerFile);
ASSERT(pRCfg->minRowsPerFileBlock == pCfg->minRowsPerFileBlock);
// ASSERT(pRCfg->minRowsPerFileBlock == pCfg->minRowsPerFileBlock);
ASSERT(pRCfg->maxRowsPerFileBlock == pCfg->maxRowsPerFileBlock);
ASSERT(pRCfg->precision == pCfg->precision);
......@@ -256,6 +256,9 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
if (pRCfg->totalBlocks != pCfg->totalBlocks) {
configChanged = true;
}
if (pRCfg->minRowsPerFileBlock != pCfg->minRowsPerFileBlock) {
configChanged = true;
}
if (!configChanged) {
tsdbError("vgId:%d no config changed", REPO_ID(repo));
......@@ -277,15 +280,16 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
pSaveCfg->keep2 = pCfg->keep2;
pSaveCfg->cacheLastRow = pCfg->cacheLastRow;
pSaveCfg->totalBlocks = pCfg->totalBlocks;
pSaveCfg->minRowsPerFileBlock = pCfg->minRowsPerFileBlock;
tsdbInfo("vgId:%d old config: compression(%d), keep(%d,%d,%d), cacheLastRow(%d),totalBlocks(%d)",
tsdbInfo("vgId:%d old config: compression(%d),keep(%d,%d,%d),cacheLastRow(%d),totalBlocks(%d),minRows(%d)",
REPO_ID(repo),
pRCfg->compression, pRCfg->keep, pRCfg->keep1,pRCfg->keep2,
pRCfg->cacheLastRow, pRCfg->totalBlocks);
tsdbInfo("vgId:%d new config: compression(%d), keep(%d,%d,%d), cacheLastRow(%d),totalBlocks(%d)",
pRCfg->cacheLastRow, pRCfg->totalBlocks, pRCfg->minRowsPerFileBlock);
tsdbInfo("vgId:%d new config: compression(%d),keep(%d,%d,%d),cacheLastRow(%d),totalBlocks(%d),minRows(%d)",
REPO_ID(repo),
pSaveCfg->compression, pSaveCfg->keep,pSaveCfg->keep1, pSaveCfg->keep2,
pSaveCfg->cacheLastRow,pSaveCfg->totalBlocks);
pSaveCfg->cacheLastRow,pSaveCfg->totalBlocks,pSaveCfg->minRowsPerFileBlock);
repo->config_changed = true;
......@@ -641,6 +645,7 @@ static void tsdbFreeRepo(STsdbRepo *pRepo) {
// tsdbFreeMemTable(pRepo->imem);
tsem_destroy(&(pRepo->readyToCommit));
pthread_mutex_destroy(&pRepo->mutex);
pthread_mutex_destroy(&pRepo->save_mutex);
free(pRepo);
}
}
......
src/tsdb/src/tsdbRead.c
浏览文件 @ 574bfd96
......@@ -836,9 +836,11 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
if (ASCENDING_TRAVERSE(order)) {
assert(pCheckInfo->lastKey <= key);
} else {
assert(pCheckInfo->lastKey >= key);
}
// mem data tskey maybe large than pCheckInfo->lastKey
//} else {
// assert(pCheckInfo->lastKey >= key);
//}
} else {
tsdbDebug("%p uid:%"PRId64", tid:%d no data in mem, 0x%"PRIx64, pHandle, pCheckInfo->tableId.uid, pCheckInfo->tableId.tid,
......@@ -858,9 +860,11 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
if (ASCENDING_TRAVERSE(order)) {
assert(pCheckInfo->lastKey <= key);
} else {
assert(pCheckInfo->lastKey >= key);
}
// imem data tskey maybe large than pCheckInfo->lastKey
//} else {
// assert(pCheckInfo->lastKey >= key);
//}
} else {
tsdbDebug("%p uid:%"PRId64", tid:%d no data in imem, 0x%"PRIx64, pHandle, pCheckInfo->tableId.uid, pCheckInfo->tableId.tid,
pHandle->qId);
......
src/util/inc/tarray.h
浏览文件 @ 574bfd96
......@@ -36,10 +36,10 @@ typedef struct SArray {
} SArray;
/**
*
* @param size
* @param elemSize
* @return
* Initializes a heap-allocated array with `size` elements, the size of element is `elemSize`.
* @param size the number of element.
* @param elemSize the size of element.
* @return the pointer points to the array.
*/
void* taosArrayInit(size_t size, size_t elemSize);
......
src/util/inc/tconfig.h
浏览文件 @ 574bfd96
......@@ -20,7 +20,7 @@
extern "C" {
#endif
#define TSDB_CFG_MAX_NUM 139
#define TSDB_CFG_MAX_NUM 140
#define TSDB_CFG_PRINT_LEN 23
#define TSDB_CFG_OPTION_LEN 24
#define TSDB_CFG_VALUE_LEN 41
......
tests/develop-test/2-query/function_mode.py
浏览文件 @ 574bfd96
......@@ -57,10 +57,10 @@ class TDTestCase:
tdSql.checkData(0, 1, 2)
tdSql.query('select mode(num) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, -32767)
tdSql.query('select mode(dbool) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, False)
tdSql.query('select mode(dtiny) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
......@@ -87,7 +87,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 1)
tdSql.query('select mode(num),mode(voltage) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.checkData(0, 1, 1)
tdSql.query('select mode(dbool) from smode')
tdSql.checkRows(1)
......@@ -97,7 +97,7 @@ class TDTestCase:
tdSql.checkData(0, 0, None)
tdSql.query('select mode(dfloat) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3.3232219219207764)
tdSql.query('select mode(ddouble) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, 4.982392323)
......@@ -112,16 +112,16 @@ class TDTestCase:
#group by column
tdSql.query('select mode(num) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.checkData(1, 0, None)
tdSql.execute('insert into D002 values("2021-11-17 20:31:31", 1, 3276, true, NULL, 3.32322, 4.982392323, "你好吗", "sdf", 333)')
tdSql.query('select mode(num) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, 3276)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.query('select mode(dfloat) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, 3.3232200145721436)
tdSql.query('select mode(dchar) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, "你好吗")
......@@ -138,7 +138,7 @@ class TDTestCase:
#group by tbname
tdSql.query('select mode(dchar) from smode group by tbname')
tdSql.checkRows(2)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, "你好吗")
tdSql.checkData(0, 1, "d001")
tdSql.checkData(1, 0, "你好吗")
tdSql.checkData(1, 1, "d002")
......@@ -148,7 +148,7 @@ class TDTestCase:
tdSql.checkRows(2)
tdSql.checkData(0, 0, 4.982392323)
tdSql.checkData(0, 1, "Beijing.haidian")
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, 4.982392323)
tdSql.checkData(1, 1, "Beijing.Chaoyang")
#group by tag,column
......@@ -169,7 +169,7 @@ class TDTestCase:
tdSql.checkRows(2)
tdSql.checkData(0, 0, "你好吗")
tdSql.checkData(0, 1, "d002")
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, "你好吗")
tdSql.checkData(1, 1, "d001")
#where
......@@ -178,7 +178,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 19)
tdSql.query('select mode(voltage) from smode where num > 9')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 19)
#interval
tdSql.query('select mode(voltage) from smode interval(1s)')
......@@ -188,7 +188,7 @@ class TDTestCase:
tdSql.checkData(0, 0, "2021-01-01 00:00:00")
tdSql.checkData(0, 1, 1)
tdSql.checkData(1, 0, "2022-01-01 00:00:00")
tdSql.checkData(1, 1, None)
tdSql.checkData(1, 1, 19)
tdSql.query('select mode(voltage) from smode interval(1n)')
tdSql.checkRows(4)
tdSql.checkData(0, 0, "2021-10-01 00:00:00")
......@@ -198,7 +198,7 @@ class TDTestCase:
tdSql.checkData(2, 0, "2021-12-01 00:00:00")
tdSql.checkData(2, 1, 2)
tdSql.checkData(3, 0, "2022-01-01 00:00:00")
tdSql.checkData(3, 1, None)
tdSql.checkData(3, 1, 19)
tdSql.query('select mode(voltage) from smode where ts > "2021-09-01 00:00:00" and ts <"2022-02-02 00:00:00" interval(1n) fill(prev)')
tdSql.checkRows(6)
......@@ -207,7 +207,7 @@ class TDTestCase:
tdSql.checkData(3, 0, "2021-12-01 00:00:00")
tdSql.checkData(3, 1, 2)
tdSql.checkData(5, 0, "2022-02-01 00:00:00")
tdSql.checkData(5, 1, 2)
tdSql.checkData(5, 1, 19)
#session
tdSql.query('select mode(voltage) from d002 session(ts,1w)')
......@@ -222,7 +222,7 @@ class TDTestCase:
#state_window
tdSql.query('select mode(dfloat) from d002 state_window(voltage)')
tdSql.checkRows(3)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3.3232200145721436)
tdSql.checkData(1, 0, None)
#slimit/soffset
......@@ -237,9 +237,9 @@ class TDTestCase:
#having
tdSql.query('select mode(ddouble) from smode group by location having mode(ddouble)>3')
tdSql.checkRows(1)
tdSql.checkRows(2)
tdSql.checkData(0, 0, 4.982392323)
tdSql.checkData(0, 1, "Beijing.haidian")
tdSql.checkData(0, 1, "Beijing.Chaoyang")
#subquery
tdSql.query('select mode(ddouble) from (select * from smode where voltage = 1)')
......
tests/develop-test/5-taos-tools/taosbenchmark/default_json.py
浏览文件 @ 574bfd96
......@@ -19,10 +19,9 @@ from util.dnodes import *
class TDTestCase:
def caseDescription(self):
'''
"""
[TD-11510] taosBenchmark test cases
'''
return
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
......@@ -31,19 +30,23 @@ class TDTestCase:
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
else:
projPath = selfPath[:selfPath.find("tests")]
projPath = selfPath[: selfPath.find("tests")]
paths = []
for root, dirs, files in os.walk(projPath):
if ((tool) in files):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
if (len(paths) == 0):
if len(paths) == 0:
tdLog.exit("taosBenchmark not found!")
return
else:
......@@ -52,12 +55,12 @@ class TDTestCase:
def run(self):
binPath = self.getPath()
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/default.json" %binPath
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/default.json" % binPath
tdLog.info("%s" % cmd)
os.system("%s" % cmd)
tdSql.execute("reset query cache")
tdSql.query("select count(tbname) from db.stb")
tdSql.checkData(0, 0, 10)
tdSql.query("show db.tables")
tdSql.checkRows(10)
tdSql.query("select count(*) from db.stb")
tdSql.checkData(0, 0, 100)
......@@ -67,4 +70,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
tests/develop-test/5-taos-tools/taosbenchmark/json/default.json
浏览文件 @ 574bfd96
......@@ -20,8 +20,10 @@
"super_tables": [{
"name": "stb",
"childtable_prefix": "stb_",
"childtable_count": 10,
"insert_rows": 10,
"columns": [{"type": "INT"}],
"tags": [{"type": "INT"}]
}]
}]
}
\ No newline at end of file
}
tests/perftest-scripts/coverage_test.sh
浏览文件 @ 574bfd96
......@@ -64,6 +64,21 @@ function runGeneralCaseOneByOne {
done < $1
}
function runTestNGCaseOneByOne {
while read -r line; do
if [[ $line =~ ^./taostest* ]]; then
case=`echo $line | cut -d' ' -f 3 | cut -d'=' -f 2`
yaml=`echo $line | cut -d' ' -f 2`
if [ -n "$case" ]; then
date +%F\ %T | tee -a $TDENGINE_COVERAGE_REPORT && ./taostest $yaml --case=$case --keep > /dev/null 2>&1 && \
echo -e "${GREEN}$case success${NC}" | tee -a $TDENGINE_COVERAGE_REPORT \
|| echo -e "${RED}$case failed${NC}" | tee -a $TDENGINE_COVERAGE_REPORT
fi
fi
done < $1
}
function runTest {
echo "run Test"
......@@ -87,6 +102,13 @@ function runTest {
else
sed -i '3i\\n' $TDENGINE_COVERAGE_REPORT
fi
# run TestNG cases
stopTaosd
$TDENGINE_DIR/debug/build/bin/taosd -c $TDENGINE_DIR/debug/test/cfg > /dev/null &
sleep 10
cd $TDENGINE_DIR/../TestNG/cases
runTestNGCaseOneByOne ../scripts/cases.txt
cd $TDENGINE_DIR/tests
rm -rf ../sim
......
tests/pytest/query/query1970YearsAf.py
浏览文件 @ 574bfd96
......@@ -25,8 +25,8 @@ from util.cases import *
from util.dnodes import *
from util.dnodes import TDDnode
class TDTestCase:
class TDTestCase:
def __init__(self):
self.path = ""
......@@ -68,7 +68,7 @@ class TDTestCase:
"cachelast": 0,
"quorum": 1,
"fsync": 3000,
"update": 0
"update": 0,
}
# set stable schema
......@@ -99,8 +99,7 @@ class TDTestCase:
{"type": "TINYINT", "count": 2},
{"type": "BOOL", "count": 2},
{"type": "NCHAR", "len": 3, "count": 1},
{"type": "BINARY", "len": 8, "count": 1}
{"type": "BINARY", "len": 8, "count": 1},
],
"tags": [
{"type": "INT", "count": 2},
......@@ -111,17 +110,14 @@ class TDTestCase:
{"type": "TINYINT", "count": 2},
{"type": "BOOL", "count": 2},
{"type": "NCHAR", "len": 3, "count": 1},
{"type": "BINARY", "len": 8, "count": 1}
]
{"type": "BINARY", "len": 8, "count": 1},
],
}
# create different stables like stable1 and add to list super_tables
super_tables = []
super_tables.append(stable1)
database = {
"dbinfo": dbinfo,
"super_tables": super_tables
}
database = {"dbinfo": dbinfo, "super_tables": super_tables}
cfgdir = self.getCfgDir()
create_table = {
......@@ -137,33 +133,60 @@ class TDTestCase:
"confirm_parameter_prompt": "no",
"insert_interval": 0,
"num_of_records_per_req": 100,
"databases": [database]
"databases": [database],
}
return create_table
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
elif "/tests/" in selfPath:
projPath = selfPath[: selfPath.find("/tests/")]
else:
tdLog.exit("path: %s is not supported" % selfPath)
paths = []
for root, dirs, files in os.walk(projPath):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
if len(paths) == 0:
return ""
return paths[0]
def createinsertfile(self):
create_table = self.creatcfg()
date = datetime.datetime.now().strftime("%Y%m%d%H%M")
file_create_table = f"/tmp/insert_{date}.json"
with open(file_create_table, 'w') as f:
with open(file_create_table, "w") as f:
json.dump(create_table, f)
return file_create_table
def inserttable(self, filepath):
create_table_cmd = f"taosdemo -f {filepath} > /dev/null 2>&1"
binPath = self.getPath("taosBenchmark")
if binPath == "":
tdLog.exit("taosBenchmark not found!")
else:
tdLog.info("taosBenchmark found in %s" % binPath)
create_table_cmd = "%s -f %s > /dev/null 2>&1" % (binPath, filepath)
_ = subprocess.check_output(create_table_cmd, shell=True).decode("utf-8")
def sqlsquery(self):
# stable query
tdSql.query(
"select * from stb2 where stb2.ts < '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from stb2 where stb2.ts < '1970-01-01 00:00:00.000' ")
tdSql.checkRows(43200)
tdSql.query(
"select * from stb2 where stb2.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from stb2 where stb2.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(6800)
tdSql.query(
......@@ -172,14 +195,10 @@ class TDTestCase:
tdSql.checkRows(3590)
# child-tables query
tdSql.query(
"select * from t0 where t0.ts < '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from t0 where t0.ts < '1970-01-01 00:00:00.000' ")
tdSql.checkRows(4320)
tdSql.query(
"select * from t1 where t1.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from t1 where t1.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(680)
tdSql.query(
......@@ -192,9 +211,7 @@ class TDTestCase:
)
tdSql.checkRows(680)
tdSql.query(
"select diff(c1) from t0 where t0.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select diff(c1) from t0 where t0.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(679)
tdSql.query(
......@@ -203,9 +220,7 @@ class TDTestCase:
tdSql.checkRows(43200)
# query with timestamp in 'where ...'
tdSql.query(
"select * from stb2 where stb2.ts > -28800000 "
)
tdSql.query("select * from stb2 where stb2.ts > -28800000 ")
tdSql.checkRows(6790)
tdSql.query(
......@@ -219,14 +234,16 @@ class TDTestCase:
tdSql.checkRows(3590)
def run(self):
s = 'reset query cache'
s = "reset query cache"
tdSql.execute(s)
s = 'create database if not exists db'
s = "create database if not exists db"
tdSql.execute(s)
s = 'use db'
s = "use db"
tdSql.execute(s)
tdLog.info("==========step1:create table stable and child table,then insert data automatically")
tdLog.info(
"==========step1:create table stable and child table,then insert data automatically"
)
insertfile = self.createinsertfile()
self.inserttable(insertfile)
......
tests/pytest/query/queryInterval.py
浏览文件 @ 574bfd96
......@@ -197,6 +197,25 @@ class TDTestCase:
else:
tdLog.exit("sql:%s, column : ts is not sorted in accending order as expected" % (tdSql.sql))
# TS-1582
tdSql.execute("create table stb1(ts timestamp, c1 int) tags(t1 int)")
tdSql.execute("insert into tb1 using stb1 tags(1) values(%d, 1)(%d, 5)(%d, 10)" % (self.ts, self.ts + 1000, self.ts + 2000))
tdSql.execute("insert into tb2 using stb1 tags(2) values(%d, 1)(%d, 5)(%d, 10)" % (self.ts, self.ts + 1000, self.ts + 2000))
tdSql.query("select t1, last(c1) from stb1 where ts between 1593548685000 and 1593548688000 interval(2s) fill(NULL) group by tbname")
tdSql.checkRows(6)
tdSql.checkData(2, 2, None)
tdSql.checkData(2, 3, "tb1")
tdSql.checkData(5, 2, None)
tdSql.checkData(5, 3, "tb2")
tdSql.query("select t1, last(c1) - 1 from stb1 where ts between 1593548685000 and 1593548688000 interval(2s) fill(NULL) group by tbname")
tdSql.checkRows(6)
tdSql.checkData(2, 2, None)
tdSql.checkData(2, 3, "tb1")
tdSql.checkData(5, 2, None)
tdSql.checkData(5, 3, "tb2")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
tests/pytest/tools/taosdumpTestBenchmark.py 已删除 100644 → 0
浏览文件 @ 8f5b3f48
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.ts = 1538548685000
self.numberOfTables = 10000
self.numberOfRecords = 100
def checkCommunity(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
return False
else:
return True
def getPath(self, tool="taosdump"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
paths = []
for root, dirs, files in os.walk(projPath):
if ((tool) in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
paths.append(os.path.join(root, tool))
break
if (len(paths) == 0):
return ""
return paths[0]
def insert_data(self, tbname, ts_start, count):
pre_insert = "insert into %s values" % tbname
sql = pre_insert
tdLog.debug("doing insert table %s rows=%d ..." % (tbname, count))
for i in range(count):
sql += " (%d,%d)" % (ts_start + i * 1000, i)
if i > 0 and i % 30000 == 0:
tdSql.execute(sql)
sql = pre_insert
# end sql
if sql != pre_insert:
tdSql.execute(sql)
tdLog.debug("INSERT TABLE DATA ............ [OK]")
return
def run(self):
if not os.path.exists("./taosdumptest"):
os.makedirs("./taosdumptest")
else:
os.system("rm -rf ./taosdumptest")
os.makedirs("./taosdumptest")
for i in range(2):
if not os.path.exists("./taosdumptest/tmp%d" % i):
os.makedirs("./taosdumptest/tmp%d" % i)
else:
os.system("rm -rf ./taosdumptest/tmp%d" % i)
os.makedirs("./taosdumptest/tmp%d" % i)
binPath = self.getPath("taosdump")
if (binPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found: %s" % binPath)
# create db1 , one stables and one table ; create general tables
tdSql.execute("drop database if exists dp1")
tdSql.execute("drop database if exists dp2")
tdSql.execute("create database if not exists dp1")
tdSql.execute("use dp1")
tdSql.execute(
'''create table st0(ts timestamp, c1 tinyint, c2 smallint, c3 int, c4 bigint, c5 float, c6 double,
c7 bool, c8 binary(20), c9 nchar(20), c11 tinyint unsigned, c12 smallint unsigned, c13 int unsigned, c14 bigint unsigned, c15 timestamp )
tags(t1 tinyint, t2 smallint, t3 int, t4 bigint, t5 float, t6 double, t7 bool, t8 binary(20), t9 nchar(20), t11 tinyint unsigned,
t12 smallint unsigned, t13 int unsigned, t14 bigint unsigned, t15 timestamp)''')
tdSql.execute(
'''create table st1(ts timestamp, c1 tinyint, c2 smallint, c3 int, c4 bigint, c5 float, c6 double,
c7 bool, c8 binary(20), c9 nchar(20), c11 tinyint unsigned, c12 smallint unsigned, c13 int unsigned, c14 bigint unsigned, c15 timestamp ) tags(jtag json)''')
intData = []
floatData = []
rowNum = 10
tabNum = 10
ts = 1537146000000
for j in range(tabNum):
tdSql.execute(
"create table st0_%d using st0 tags( %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d);" %
(j,
j +
1,
j +
1,
j +
1,
j +
1,
j +
0.1,
j +
0.1,
j %
2,
j +
1,
j +
1,
j +
1,
j +
1,
j +
1,
j +
1,
ts))
for i in range(rowNum):
tdSql.execute(
"insert into st0_%d values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d)" %
(j, ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i %
2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, ts))
intData.append(i + 1)
floatData.append(i + 0.1)
rowNum = 20
tabNum = 20
for j in range(tabNum):
tdSql.execute(
"create table st1_%d using st1 tags('{\"nv\":null,\"tea\":true,\"\":false,\" \":123%d,\"tea\":false}');" %
(j, j + 1))
for i in range(rowNum):
tdSql.execute(
"insert into st1_%d values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d)" %
(j, self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i %
2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, self.ts))
intData.append(i + 1)
floatData.append(i + 0.1)
# os.system("%staosBenchmark -f tools/taosdump-insert-dp1.json -y " % benchBinPath)
# create db1 , three stables:stb0,include ctables stb0_0 \ stb0_1,stb1 include ctables stb1_0 and stb1_1
# \stb3,include ctables stb3_0 and stb3_1
# create general three tables gt0 gt1 gt2
tdSql.execute("create database if not exists dp2")
tdSql.execute("use dp2")
tdSql.execute(
"create stable st0(ts timestamp, c01 int, c02 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st0_0 using st0 tags(0) st0_1 using st0 tags(1) ")
tdSql.execute(
"insert into st0_0 values(1614218412000,8600,'R')(1614218422000,8600,'E')")
tdSql.execute(
"insert into st0_1 values(1614218413000,8601,'A')(1614218423000,8601,'D')")
tdSql.execute(
"create stable st1(ts timestamp, c11 float, c12 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st1_0 using st1 tags(0) st1_1 using st1 tags(1) ")
tdSql.execute(
"insert into st1_0 values(1614218412000,8610.1,'R')(1614218422000,8610.1,'E')")
tdSql.execute(
"insert into st1_1 values(1614218413000,8611.2,'A')(1614218423000,8611.1,'D')")
tdSql.execute(
"create stable st2(ts timestamp, c21 float, c22 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st20 using st2 tags(0) st21 using st2 tags(1) ")
tdSql.execute(
"insert into st20 values(1614218412000,8620.3,'R')(1614218422000,8620.3,'E')")
tdSql.execute(
"insert into st21 values(1614218413000,8621.4,'A')(1614218423000,8621.4,'D')")
tdSql.execute(
"create table if not exists gt0 (ts timestamp, c00 int, c01 float) ")
tdSql.execute(
"create table if not exists gt1 (ts timestamp, c10 int, c11 double) ")
tdSql.execute(
"create table if not exists gt2 (ts timestamp, c20 int, c21 float) ")
tdSql.execute("insert into gt0 values(1614218412700,8637,78.86155)")
tdSql.execute(
"insert into gt1 values(1614218413800,8638,78.862020199)")
tdSql.execute("insert into gt2 values(1614218413900,8639,78.863)")
# self.insert_data("t", self.ts, 300*10000);
# os.system("%staosBenchmark -f tools/taosdump-insert-dp2.json -y " % benchBinPath)
# # taosdump data
# os.system("%s -o ./taosdumptest/tmp1 taosdump -h -ptaosdata -P 6030 -u root -o taosdumptest \
# -D dp1,dp3 -N -c /home/chr/TDinternal/community/sim/dnode1/cfg/taos.cfg -s -d deflate" % binPath)
os.system(
"%s -o ./taosdumptest/tmp0 -D dp2,dp1 -T 8" %
binPath)
os.system(
"%s -o ./taosdumptest/tmp1 dp2 st0 st1_0 gt0 -T 8" %
binPath)
# check taosdumptest/tmp0
tdSql.execute("drop database dp1")
tdSql.execute("drop database dp2")
os.system("%s -i ./taosdumptest/tmp0 -T 8 " % binPath)
tdSql.execute("reset query cache")
tdSql.execute("use dp1")
tdSql.query("show stables")
tdSql.checkRows(3)
for i in range(3):
for j in range(3):
if j < 2:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, (j + 1) * 10)
else:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, 100002)
tdSql.query("select count(*) from st0")
tdSql.checkData(0, 0, 100)
tdSql.query("select count(*) from st1")
tdSql.checkData(0, 0, 400)
tdSql.query("select count(*) from st2")
tdSql.checkData(0, 0, 1000020)
tdSql.execute("use dp2")
tdSql.query("show stables")
tdSql.checkRows(3)
for i in range(3):
for j in range(3):
if j < 2:
if tdSql.queryResult[i][0] == 'st%d' % j:
# print(i,"stb%d"%j)
tdSql.checkData(i, 4, 2)
else:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, 100002)
tdSql.query("select count(*) from st0")
tdSql.checkData(0, 0, 4)
tdSql.query("select count(*) from st1")
tdSql.checkData(0, 0, 4)
tdSql.query("select count(*) from st2")
tdSql.checkData(0, 0, 1000024)
tdSql.query("select ts from gt0")
tdSql.checkData(0, 0, '2021-02-25 10:00:12.700')
tdSql.query("select c10 from gt1")
tdSql.checkData(0, 0, 8638)
tdSql.query("select c20 from gt2")
tdSql.checkData(0, 0, 8639)
# check taosdumptest/tmp1
tdSql.execute("drop database dp1")
tdSql.execute("drop database dp2")
os.system("%s -i ./taosdumptest/tmp1 -T 8 " % binPath)
tdSql.execute("reset query cache")
tdSql.execute("use dp2")
tdSql.query("show stables")
tdSql.checkRows(2)
tdSql.query("show tables")
tdSql.checkRows(4)
tdSql.query("select count(*) from st1_0")
tdSql.checkData(0, 0, 2)
tdSql.query("select ts from gt0")
tdSql.checkData(0, 0, '2021-02-25 10:00:12.700')
tdSql.error("use dp1")
tdSql.error("select count(*) from st2_0")
tdSql.error("select count(*) from gt2")
# #check taosdumptest/tmp2
# tdSql.execute("drop database dp1")
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp2 -T 8 " % binPath)
# tdSql.execute("use dp1")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(3)
# tdSql.query("select c1 from st0_0 order by ts")
# tdSql.checkData(0,0,8537)
# tdSql.query("select c2 from st0_1 order by ts")
# tdSql.checkData(1,0,"D")
# tdSql.query("select * from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000')
# tdSql.checkData(0,1,637)
# tdSql.error("select count(*) from gt1")
# tdSql.error("use dp2")
# #check taosdumptest/tmp3
# tdSql.execute("drop database dp1")
# os.system("%s -i ./taosdumptest/tmp3 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(2)
# tdSql.query("show tables")
# tdSql.checkRows(4)
# tdSql.query("select count(*) from st1_0")
# tdSql.checkData(0,0,2)
# tdSql.query("select ts from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.700')
# tdSql.error("use dp1")
# tdSql.error("select count(*) from st2_0")
# tdSql.error("select count(*) from gt2")
# #check taosdumptest/tmp4
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp4 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(2)
# tdSql.query("show tables")
# tdSql.checkRows(6)
# tdSql.query("select c20 from gt2")
# tdSql.checkData(0, 0, 8639)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st0_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_0")
# tdSql.checkData(0, 0, 2)
# tdSql.error("use dp1")
# tdSql.error("select count(*) from st1_0")
# tdSql.error("select count(*) from st1_1")
# tdSql.error("select count(*) from gt3")
# #check taosdumptest/tmp5
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp5 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(3)
# tdSql.query("show tables")
# tdSql.checkRows(9)
# tdSql.query("select c20 from gt2")
# tdSql.checkData(0, 0, 8639)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st0_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st1_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st1_0")
# tdSql.checkData(0, 0, 2)
# tdSql.execute("use dp1")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(4)
# tdSql.query("select c1 from st0_0 order by ts")
# tdSql.checkData(0,0,8537)
# tdSql.query("select c2 from st0_1 order by ts")
# tdSql.checkData(1,0,"D")
# tdSql.query("select * from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000')
# tdSql.checkData(0,1,637)
# # check taosdumptest/tmp6
# tdSql.execute("drop database dp1")
# tdSql.execute("drop database dp2")
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp6 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show databases")
# tdSql.checkRows(1)
# tdSql.checkData(0,16,'ns')
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select * from st0 order by ts")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# tdSql.checkData(0,1,8600)
# # check taosdumptest/tmp7
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp7 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show databases")
# tdSql.checkRows(1)
# tdSql.checkData(0,16,'ms')
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkRows(0)
# # tdSql.query("select * from st0 order by ts")
# # tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# # tdSql.checkData(0,1,8600)
# # check taosdumptest/tmp8
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp8 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkRows(0)
# # tdSql.query("select * from st0 order by ts")
# # tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# # tdSql.checkData(0,1,8600)
# os.system("rm -rf ./taosdumptest/tmp1")
# os.system("rm -rf ./taosdumptest/tmp2")
# os.system("rm -rf ./taosdumptest/tmp3")
# os.system("rm -rf ./taosdumptest/tmp4")
# os.system("rm -rf ./taosdumptest/tmp5")
# os.system("rm -rf ./dump_result.txt")
# os.system("rm -rf ./db.csv")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
tests/script/general/db/alter_option.sim
浏览文件 @ 574bfd96
......@@ -9,7 +9,7 @@ sleep 2000
sql connect
print ============= create database
sql create database db cache 2 blocks 4 days 10 keep 20 minRows 300 maxRows 400 ctime 120 precision 'ms' comp 2 wal 1 replica 1
sql create database db cache 2 blocks 4 days 10 keep 20 minRows 300 maxRows 1200 ctime 120 precision 'ms' comp 2 wal 1 replica 1
sql show databases
if $data00 != db then
return -1
......@@ -182,9 +182,13 @@ sql_error alter database db blocks -1
sql_error alter database db blocks 10001
print ============== step minrows
sql_error alter database db minrows 0
sql_error alter database db minrows 1
sql_error alter database db minrows 100
sql_error alter database db minrows 1000
sql_error alter database db minrows -1
sql_error alter database db minrows 9
sql_error alter database db minrows 1001
sql alter database db minrows 100
sql alter database db minrows 1000
print ============== step maxrows
sql_error alter database db maxrows 1
......
tests/script/general/db/topic1.sim
浏览文件 @ 574bfd96
......@@ -471,8 +471,9 @@ sql_error alter topic db blocks 10001
print ============== step minrows
sql_error alter database db minrows 1
sql_error alter database db minrows 100
sql_error alter database db minrows 1000
sql alter database db minrows 100
sql alter database db minrows 399
sql_error alter database db minrows 1001
sql_error alter topic db minrows 1
sql_error alter topic db minrows 100
......
tests/system-test/5-taos-tools/TD-12478.py
浏览文件 @ 574bfd96
......@@ -28,13 +28,13 @@ from itertools import combinations
from faker import Faker
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<xyguo>[TD-12434]:taosdump null nchar/binary length can cause core:taos-tools/src/taosdump.c
case2<xyguo>[TD-12478]:taos_stmt_execute() failed! reason: WAL size exceeds limit
'''
return
"""
case1<xyguo>[TD-12434]:taosdump null nchar/binary length can cause core:taos-tools/src/taosdump.c
case2<xyguo>[TD-12478]:taos_stmt_execute() failed! reason: WAL size exceeds limit
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
......@@ -43,104 +43,184 @@ class TDTestCase:
os.system("rm -rf 5-taos-tools/TD-12478.py.sql")
os.system("rm db*")
os.system("rm dump_result.txt*")
os.system("rm -rf taosdump.*")
def restartDnodes(self):
tdDnodes.stop(1)
tdDnodes.start(1)
def dropandcreateDB_random(self,n):
def dropandcreateDB_random(self, n):
self.ts = 1630000000000
fake = Faker('zh_CN')
fake = Faker("zh_CN")
self.num_random = fake.random_int(min=1000, max=5000, step=1)
print(self.num_random)
for i in range(n):
tdSql.execute('''drop database if exists db ;''')
tdSql.execute('''create database db keep 36500;''')
tdSql.execute('''use db;''')
tdSql.execute("""drop database if exists db ;""")
tdSql.execute("""create database db keep 36500;""")
tdSql.execute("""use db;""")
tdSql.execute('''create stable stable_1 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
tdSql.execute(
"""create stable stable_1 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);''')
tdSql.execute('''create stable stable_2 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);"""
)
tdSql.execute(
"""create stable stable_2 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);''')
tdSql.execute('''create table table_1 using stable_1 tags('table_1', '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')''')
tdSql.execute('''create table table_2 using stable_1 tags('table_2', '2147483647' , '9223372036854775807' , '32767' , '127' , 1 , 'binary2' , 'nchar2' , '2' , '22' , \'1999-09-09 09:09:09.090\')''')
tdSql.execute('''create table table_3 using stable_1 tags('table_3', '-2147483647' , '-9223372036854775807' , '-32767' , '-127' , false , 'binary3' , 'nchar3nchar3' , '-3.3' , '-33.33' , \'2099-09-09 09:09:09.090\')''')
tdSql.execute('''create table table_21 using stable_2 tags('table_21' , '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')''')
#regular table
tdSql.execute('''create table regular_table_1 \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);"""
)
tdSql.execute(
"""create table table_1 using stable_1 tags('table_1', '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')"""
)
tdSql.execute(
"""create table table_2 using stable_1 tags('table_2', '2147483647' , '9223372036854775807' , '32767' , '127' , 1 , 'binary2' , 'nchar2' , '2' , '22' , \'1999-09-09 09:09:09.090\')"""
)
tdSql.execute(
"""create table table_3 using stable_1 tags('table_3', '-2147483647' , '-9223372036854775807' , '-32767' , '-127' , false , 'binary3' , 'nchar3nchar3' , '-3.3' , '-33.33' , \'2099-09-09 09:09:09.090\')"""
)
tdSql.execute(
"""create table table_21 using stable_2 tags('table_21' , '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')"""
)
# regular table
tdSql.execute(
"""create table regular_table_1 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
tdSql.execute('''create table regular_table_2 \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
tdSql.execute(
"""create table regular_table_2 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
tdSql.execute('''create table regular_table_3 \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
tdSql.execute(
"""create table regular_table_3 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
for i in range(self.num_random):
tdSql.execute('''insert into table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double , q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(min=-9223372036854775807, max=9223372036854775807, step=1),
fake.random_int(min=-32767, max=32767, step=1) , fake.random_int(min=-127, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=2147483647, step=1) ,
fake.random_int(min=-9223372036854775807, max=9223372036854775807, step=1) ,
fake.random_int(min=-32767, max=32767, step=1) , fake.random_int(min=-127, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1) , fake.random_int(min=0, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1) , fake.random_int(min=0, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1) , fake.random_int(min=-127, max=0, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1) , fake.random_int(min=-127, max=0, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
for i in range(self.num_random):
tdSql.execute(
"""insert into table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double , q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(
min=-9223372036854775807, max=9223372036854775807, step=1
),
fake.random_int(min=-32767, max=32767, step=1),
fake.random_int(min=-127, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(
min=-9223372036854775807, max=9223372036854775807, step=1
),
fake.random_int(min=-32767, max=32767, step=1),
fake.random_int(min=-127, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1),
fake.random_int(min=0, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1),
fake.random_int(min=0, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1),
fake.random_int(min=-127, max=0, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1),
fake.random_int(min=-127, max=0, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.query("select count(*) from stable_1;")
tdSql.checkData(0,0,3*self.num_random)
tdSql.checkData(0, 0, 3 * self.num_random)
tdSql.query("select count(*) from regular_table_1;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
def run(self):
tdSql.prepare()
dcDB = self.dropandcreateDB_random(1)
assert os.system("taosdump -D db") == 0
assert os.system("taosdump -i . -g") == 0
tdSql.query("select count(*) from stable_1;")
tdSql.checkData(0,0,3*self.num_random)
tdSql.checkData(0, 0, 3 * self.num_random)
tdSql.query("select count(*) from regular_table_1;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
tdSql.query("select count(*) from regular_table_2;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
tdSql.query("select count(*) from regular_table_3;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
def stop(self):
tdSql.close()
......@@ -148,4 +228,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
tests/system-test/5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson.py
浏览文件 @ 574bfd96
......@@ -24,36 +24,44 @@ class TDTestCase:
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def getBuildPath(self):
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
elif "/tests/" in selfPath:
projPath = selfPath[: selfPath.find("/tests/")]
else:
projPath = selfPath[:selfPath.find("tests")]
tdLog.exit("path: %s is not supported" % selfPath)
paths = []
for root, dirs, files in os.walk(projPath):
if ("taosd" in files):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root)-len("/build/bin")]
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
return buildPath
if len(paths) == 0:
return ""
return paths[0]
def run(self):
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosd not found!")
binPath = self.getPath()
if binPath == "":
tdLog.exit("taosBenchmark not found!")
else:
tdLog.info("taosd found in %s" % buildPath)
binPath = buildPath+ "/build/bin/"
tdLog.info("taosBenchmark found in %s" % binPath)
testcaseFilename = os.path.split(__file__)[-1]
os.system("rm -rf ./insert*_res.txt*")
os.system("rm -rf 5-taos-tools/taosbenchmark/%s.sql" % testcaseFilename )
os.system("rm -rf 5-taos-tools/taosbenchmark/%s.sql" % testcaseFilename)
# insert: create one or mutiple tables per sql and insert multiple rows per sql
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-1s1tnt1r.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insert-1s1tnt1r.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 11)
......@@ -69,7 +77,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 2000)
# restful connector insert data
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insertRestful.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insertRestful.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 10)
......@@ -84,19 +92,19 @@ class TDTestCase:
tdSql.query("select count(*) from stb1")
tdSql.checkData(0, 0, 200)
# default values json files
tdSql.execute("drop database if exists db")
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-default.json -y " % binPath)
# default values json files
tdSql.execute("drop database if exists db")
os.system("%s -f 5-taos-tools/taosbenchmark/insert-default.json -y " % binPath)
tdSql.query("show databases;")
for i in range(tdSql.queryRows):
if tdSql.queryResult[i][0] == 'db':
tdSql.checkData(i, 2, 100)
tdSql.checkData(i, 4, 1)
tdSql.checkData(i, 6, 10)
tdSql.checkData(i, 16, 'ms')
if tdSql.queryResult[i][0] == "db":
tdSql.checkData(i, 2, 100)
tdSql.checkData(i, 4, 1)
tdSql.checkData(i, 6, 10)
tdSql.checkData(i, 16, "ms")
# insert: create mutiple tables per sql and insert one rows per sql .
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-1s1tntmr.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insert-1s1tntmr.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 10)
......@@ -113,7 +121,9 @@ class TDTestCase:
# insert: using parament "insert_interval to controls spped of insert.
# but We need to have accurate methods to control the speed, such as getting the speed value, checking the count and so on。
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-interval-speed.json -y" % binPath)
os.system(
"%s -f 5-taos-tools/taosbenchmark/insert-interval-speed.json -y" % binPath
)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkData(0, 4, 10)
......@@ -131,11 +141,6 @@ class TDTestCase:
# rm useless files
os.system("rm -rf ./insert*_res.txt*")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或