Skip to content

TDengine
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1193
Star 22018
Fork 4786
TDengine
TDengine
提交 574bfd96 编写于 作者: Z zhihaop
浏览文件
操作

Merge branch '2.6' into improve/taosc-async-enhancement-for-2.6

上级 8f5b3f48 f88270f0
展开全部 隐藏空白更改
内联 并排
Showing with 3278 addition and 2792 deletion
docs/en/12-taos-sql/02-database.md
浏览文件 @ 574bfd96
......@@ -20,20 +20,20 @@ CREATE DATABASE [IF NOT EXISTS] db_name [KEEP keep] [DAYS days] [UPDATE 1];
3. The maximum length of database name is 33 bytes.
4. The maximum length of a SQL statement is 65,480 bytes.
5. Below are the parameters that can be used when creating a database
- cache: [Description](/reference/config/#cache)
- blocks: [Description](/reference/config/#blocks)
- days: [Description](/reference/config/#days)
- keep: [Description](/reference/config/#keep)
- minRows: [Description](/reference/config/#minrows)
- maxRows: [Description](/reference/config/#maxrows)
- wal: [Description](/reference/config/#wallevel)
- fsync: [Description](/reference/config/#fsync)
- update: [Description](/reference/config/#update)
- cacheLast: [Description](/reference/config/#cachelast)
- replica: [Description](/reference/config/#replica)
- quorum: [Description](/reference/config/#quorum)
- comp: [Description](/reference/config/#comp)
- precision: [Description](/reference/config/#precision)
- cache: [Description](../../reference/config/#cache)
- blocks: [Description](../../reference/config/#blocks)
- days: [Description](../../reference/config/#days)
- keep: [Description](../../reference/config/#keep)
- minRows: [Description](../../reference/config/#minrows)
- maxRows: [Description](../../reference/config/#maxrows)
- wal: [Description](../../reference/config/#wallevel)
- fsync: [Description](../../reference/config/#fsync)
- update: [Description](../../reference/config/#update)
- cacheLast: [Description](../../reference/config/#cachelast)
- replica: [Description](../../reference/config/#replica)
- quorum: [Description](../../reference/config/#quorum)
- comp: [Description](../../reference/config/#comp)
- precision: [Description](../../reference/config/#precision)
6. Please note that all of the parameters mentioned in this section are configured in configuration file `taos.cfg` on the TDengine server. If not specified in the `create database` statement, the values from taos.cfg are used by default. To override default parameters, they must be specified in the `create database` statement.
:::
......
docs/en/12-taos-sql/10-function.md
浏览文件 @ 574bfd96
......@@ -191,7 +191,7 @@ Query OK, 1 row(s) in set (0.000921s)
SELECT MODE(field_name) FROM tb_name [WHERE clause];
```
**Description**:The value which has the highest frequency of occurrence. NULL is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column or tags.
**Description**:The value which has the highest frequency of occurrence. One random value is returned if there are multiple values which have highest frequency of occurrence. It can't be used on timestamp column or tags.
**Return value type**:Same as the data type of the column being operated upon
......
docs/en/14-reference/05-taosbenchmark.md
浏览文件 @ 574bfd96
......@@ -234,7 +234,7 @@ The parameters related to database creation are configured in `dbinfo` in the js
- **name**: specify the name of the database.
- **drop**: indicate whether to delete the database before inserting. The default is true.
- **drop**: indicate whether to delete the database before inserting. The value can be 'yes' or 'no'. No means do not drop. The default is to drop.
- **replica**: specify the number of replicas when creating the database.
......
docs/en/14-reference/_telegraf.mdx
浏览文件 @ 574bfd96
......@@ -22,5 +22,4 @@ An example is as follows.
username = "root"
password = "taosdata"
data_format = "influx"
influx_max_line_bytes = 250
```
docs/en/25-application/01-telegraf.md
浏览文件 @ 574bfd96
......@@ -60,7 +60,6 @@ For the configuration method, add the following text to `/etc/telegraf/telegraf.
username = "<TDengine's username>"
password = "<TDengine's password>"
data_format = "influx"
influx_max_line_bytes = 250
```
Then restart telegraf:
......
docs/zh/12-taos-sql/02-database.md
浏览文件 @ 574bfd96
......@@ -20,20 +20,20 @@ CREATE DATABASE [IF NOT EXISTS] db_name [KEEP keep] [DAYS days] [UPDATE 1];
3. 数据库名最大长度为 33；
4. 一条 SQL 语句的最大长度为 65480 个字符；
5. 创建数据库时可用的参数有：
- cache: [详细说明](/reference/config/#cache)
- blocks: [详细说明](/reference/config/#blocks)
- days: [详细说明](/reference/config/#days)
- keep: [详细说明](/reference/config/#keep)
- minRows: [详细说明](/reference/config/#minrows)
- maxRows: [详细说明](/reference/config/#maxrows)
- wal: [详细说明](/reference/config/#wallevel)
- fsync: [详细说明](/reference/config/#fsync)
- update: [详细说明](/reference/config/#update)
- cacheLast: [详细说明](/reference/config/#cachelast)
- replica: [详细说明](/reference/config/#replica)
- quorum: [详细说明](/reference/config/#quorum)
- comp: [详细说明](/reference/config/#comp)
- precision: [详细说明](/reference/config/#precision)
- cache: [详细说明](../../reference/config/#cache)
- blocks: [详细说明](../../reference/config/#blocks)
- days: [详细说明](../../reference/config/#days)
- keep: [详细说明](../../reference/config/#keep)
- minRows: [详细说明](../../reference/config/#minrows)
- maxRows: [详细说明](../../reference/config/#maxrows)
- wal: [详细说明](../../reference/config/#wallevel)
- fsync: [详细说明](../../reference/config/#fsync)
- update: [详细说明](../../reference/config/#update)
- cacheLast: [详细说明](../../reference/config/#cachelast)
- replica: [详细说明](../../reference/config/#replica)
- quorum: [详细说明](../../reference/config/#quorum)
- comp: [详细说明](../../reference/config/#comp)
- precision: [详细说明](../../reference/config/#precision)
6. 请注意上面列出的所有参数都可以配置在配置文件 `taosd.cfg` 中作为创建数据库时使用的默认配置， `create database` 的参数中明确指定的会覆盖配置文件中的设置。
:::
......@@ -86,7 +86,7 @@ REPLICA 参数是指修改数据库副本数，取值范围 [1, 3]。在集群
ALTER DATABASE db_name KEEP 365;
```
KEEP 参数是指修改数据文件保存的天数，缺省值为 3650，取值范围 [days, 365000]，必须大于或等于 days 参数值。
KEEP 参数是指修改数据文件保存的天数，缺省值为 3650，取值范围 [days, 36500]，必须大于或等于 days 参数值。
```
ALTER DATABASE db_name QUORUM 2;
......
docs/zh/12-taos-sql/10-function.md
浏览文件 @ 574bfd96
......@@ -193,7 +193,7 @@ Query OK, 1 row(s) in set (0.000921s)
SELECT MODE(field_name) FROM tb_name [WHERE clause];
```
**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，输出空。不能匹配标签、时间戳输出。
**功能说明**：返回出现频率最高的值，若存在多个频率相同的最高值，则随机输出其中某个值。不能匹配标签、时间戳输出。
**返回数据类型**：同应用的字段。
......
docs/zh/14-reference/05-taosbenchmark.md
浏览文件 @ 574bfd96
......@@ -231,7 +231,7 @@ taosBenchmark -A INT,DOUBLE,NCHAR,BINARY\(16\)
- **name** : 数据库名。
- **drop** : 插入前是否删除数据库，默认为 true。
- **drop** : 插入前是否删除数据库，可选项为 "yes" 或者 "no", 为 "no" 时不创建。默认删除。
- **replica** : 创建数据库时指定的副本数。
......
docs/zh/14-reference/_telegraf.mdx
浏览文件 @ 574bfd96
......@@ -22,6 +22,5 @@
username = "root"
password = "taosdata"
data_format = "influx"
influx_max_line_bytes = 250
```
docs/zh/25-application/01-telegraf.md
浏览文件 @ 574bfd96
......@@ -60,7 +60,6 @@ IT 运维监测数据通常都是对时间特性比较敏感的数据，例如
username = "<TDengine's username>"
password = "<TDengine's password>"
data_format = "influx"
influx_max_line_bytes = 250
```
然后重启 Telegraf：
......
packaging/docker/bin/entrypoint.sh
浏览文件 @ 574bfd96
......@@ -35,6 +35,9 @@ FIRST_EP_PORT=${TAOS_FIRST_EP#*:}
SERVER_PORT=$(cat $CFG_FILE | grep -E -v "^#|^$" | grep serverPort | tail -n1 | sed -E 's/.*serverPort\s+//')
SERVER_PORT=${SERVER_PORT:-6030}
# parse dataDir from taos.cfg
DATA_DIR=$(cat $CFG_FILE | grep -E -v "^#|^$" | grep dataDir | tail -n1 | sed -E 's/.*dataDir\s+//')
# for other binaries like interpreters
if echo $1 | grep -E "taosd$" - >/dev/null; then
true # will run taosd
......
packaging/release.sh
浏览文件 @ 574bfd96
......@@ -3,7 +3,7 @@
# Generate the deb package for ubuntu, or rpm package for centos, or tar.gz package for other linux os
set -e
#set -x
# set -x
# release.sh -v [cluster | edge]
# -c [aarch32 | aarch64 | x64 | x86 | mips64 ...]
......@@ -263,7 +263,7 @@ if [ "$osType" != "Darwin" ]; then
if [[ "$pagMode" == "full" ]]; then
if [ -d ${top_dir}/src/kit/taos-tools/packaging/deb ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/deb
taos_tools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}')
taos_tools_ver=$(git tag |grep -v taos | sort | tail -1)
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-deb.sh ${top_dir} \
......@@ -288,7 +288,7 @@ if [ "$osType" != "Darwin" ]; then
if [[ "$pagMode" == "full" ]]; then
if [ -d ${top_dir}/src/kit/taos-tools/packaging/rpm ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/rpm
taos_tools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}' | sed -e 's/-/_/g')
taos_tools_ver=$(git tag |grep -v taos | sort | tail -1)
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-rpm.sh ${top_dir} \
......
packaging/tools/install.sh
浏览文件 @ 574bfd96
......@@ -105,6 +105,10 @@ elif echo $osinfo | grep -qwi "debian"; then
elif echo $osinfo | grep -qwi "Kylin"; then
# echo "This is Kylin system"
os_type=1
elif echo $osinfo | grep -qwi "KylinSec"; then
# echo "This is KylinSec system"
os_type=2
csudo=""
elif echo $osinfo | grep -qwi "Red"; then
# echo "This is Red Hat system"
os_type=1
......
packaging/tools/makepkg.sh
浏览文件 @ 574bfd96
......@@ -47,7 +47,7 @@ if [ -d ${top_dir}/src/kit/taos-tools/packaging/deb ]; then
cd ${top_dir}/src/kit/taos-tools/packaging/deb
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
taostools_ver=$(git describe --tags | sed -e 's/ver-//g' | awk -F '-' '{print $1}')
taostools_ver=$(git tag |grep -v taos | sort | tail -1)
taostools_install_dir="${release_dir}/${clientName}Tools-${taostools_ver}"
cd ${curr_dir}
......
src/client/inc/tscParseLine.h
浏览文件 @ 574bfd96
......@@ -103,11 +103,11 @@ int32_t convertSmlTimeStamp(TAOS_SML_KV *pVal, char *value,
void destroySmlDataPoint(TAOS_SML_DATA_POINT* point);
int taos_insert_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol,
int taos_insert_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol,
SMLTimeStampType tsType, int* affectedRows);
int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol,
int taos_insert_telnet_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol,
SMLTimeStampType tsType, int* affectedRows);
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol,
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, int *numLines,
SMLTimeStampType tsType, int* affectedRows);
......
src/client/src/tscParseOpenTSDB.c
浏览文件 @ 574bfd96
......@@ -33,7 +33,7 @@ static uint64_t genUID() {
return id;
}
static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
const char *cur = *idx;
uint16_t len = 0;
......@@ -49,7 +49,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SS
}
*/
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_TABLE_NAME_LEN - 1) {
tscError("OTD:0x%"PRIx64" Metric cannot exceeds %d characters", info->id, TSDB_TABLE_NAME_LEN - 1);
tfree(pSml->stableName);
......@@ -82,7 +82,7 @@ static int32_t parseTelnetMetric(TAOS_SML_DATA_POINT *pSml, const char **idx, SS
return TSDB_CODE_SUCCESS;
}
static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
//Timestamp must be the first KV to parse
assert(*num_kvs == 0);
......@@ -96,7 +96,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
//allocate fields for timestamp and value
*pTS = tcalloc(OTD_MAX_FIELDS_NUM, sizeof(TAOS_SML_KV));
while(*cur != '\0') {
while(cur - *idx < sqlLen) {
if (*cur == ' ') {
if (*(cur + 1) != ' ') {
break;
......@@ -109,7 +109,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
len++;
}
if (len > 0 && *cur != '\0') {
if (len > 0 && cur - *idx < sqlLen) {
value = tcalloc(len + 1, 1);
memcpy(value, start, len);
} else {
......@@ -135,7 +135,7 @@ static int32_t parseTelnetTimeStamp(TAOS_SML_KV **pTS, int *num_kvs, const char
return ret;
}
static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const char **idx, SSmlLinesInfo* info) {
static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const char **idx, int32_t sqlLen, SSmlLinesInfo* info) {
//skip timestamp
TAOS_SML_KV *pVal = *pKVs + 1;
const char *start, *cur;
......@@ -158,7 +158,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
len += 2;
}
while(*cur != '\0') {
while(cur - *idx < sqlLen) {
if (*cur == ' ') {
if (searchQuote == true) {
if (*(cur - 1) == '"' && len != 1 && len != 2) {
......@@ -181,7 +181,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
len++;
}
if (len > 0 && *cur != '\0') {
if (len > 0 && cur - *idx < sqlLen) {
value = tcalloc(len + 1, 1);
memcpy(value, start, len);
} else {
......@@ -205,7 +205,7 @@ static int32_t parseTelnetMetricValue(TAOS_SML_KV **pKVs, int *num_kvs, const ch
return ret;
}
static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *pHash, SSmlLinesInfo* info) {
static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen, SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *idx;
char key[TSDB_COL_NAME_LEN];
uint16_t len = 0;
......@@ -215,7 +215,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
// tscError("OTD:0x%"PRIx64" Tag key cannot start with digit", info->id);
// return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
//}
while (*cur != '\0') {
while (cur - *idx < sqlLen) {
if (len > TSDB_COL_NAME_LEN - 1) {
tscError("OTD:0x%"PRIx64" Tag key cannot exceeds %d characters", info->id, TSDB_COL_NAME_LEN - 1);
return TSDB_CODE_TSC_INVALID_COLUMN_LENGTH;
......@@ -231,7 +231,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
cur++;
len++;
}
if (len == 0 || *cur == '\0') {
if (len == 0 || cur - *idx == sqlLen) {
return TSDB_CODE_TSC_LINE_SYNTAX_ERROR;
}
key[len] = '\0';
......@@ -249,7 +249,7 @@ static int32_t parseTelnetTagKey(TAOS_SML_KV *pKV, const char **idx, SHashObj *p
}
static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx, int32_t sqlLen,
bool *is_last_kv, SSmlLinesInfo* info) {
const char *start, *cur;
char *value = NULL;
......@@ -258,9 +258,9 @@ static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
while (1) {
// whitespace or '\0' identifies a value
if (*cur == ' ' || *cur == '\0') {
if (*cur == ' ' || cur - *idx == sqlLen) {
// '\0' indicates end of value
*is_last_kv = (*cur == '\0') ? true : false;
*is_last_kv = (cur - *idx == sqlLen) ? true : false;
if (*cur == ' ' && *(cur + 1) == ' ') {
cur++;
continue;
......@@ -290,12 +290,12 @@ static int32_t parseTelnetTagValue(TAOS_SML_KV *pKV, const char **idx,
}
tfree(value);
*idx = (*cur == '\0') ? cur : cur + 1;
*idx = (cur - *idx == sqlLen) ? cur : cur + 1;
return TSDB_CODE_SUCCESS;
}
static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
const char **idx, char **childTableName,
const char **idx, int32_t sqlLen, char **childTableName,
SHashObj *pHash, SSmlLinesInfo* info) {
const char *cur = *idx;
int32_t ret = TSDB_CODE_SUCCESS;
......@@ -312,13 +312,13 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
memcpy(childTbName, tsSmlChildTableName, childTableNameLen);
addEscapeCharToString(childTbName, (int32_t)(childTableNameLen));
}
while (*cur != '\0') {
ret = parseTelnetTagKey(pkv, &cur, pHash, info);
while (cur - *idx < sqlLen) {
ret = parseTelnetTagKey(pkv, &cur, sqlLen - (cur - *idx), pHash, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse key", info->id);
return ret;
}
ret = parseTelnetTagValue(pkv, &cur, &is_last_kv, info);
ret = parseTelnetTagValue(pkv, &cur, sqlLen - (cur - *idx), &is_last_kv, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse value", info->id);
return ret;
......@@ -356,12 +356,12 @@ static int32_t parseTelnetTagKvs(TAOS_SML_KV **pKVs, int *num_kvs,
return ret;
}
static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
static int32_t tscParseTelnetLine(const char* line, int32_t len, TAOS_SML_DATA_POINT* smlData, SSmlLinesInfo* info) {
const char* idx = line;
int32_t ret = TSDB_CODE_SUCCESS;
//Parse metric
ret = parseTelnetMetric(smlData, &idx, info);
ret = parseTelnetMetric(smlData, &idx, len, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse metric", info->id);
return ret;
......@@ -369,7 +369,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
tscDebug("OTD:0x%"PRIx64" Parse metric finished", info->id);
//Parse timestamp
ret = parseTelnetTimeStamp(&smlData->fields, &smlData->fieldNum, &idx, info);
ret = parseTelnetTimeStamp(&smlData->fields, &smlData->fieldNum, &idx, len - (idx - line), info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse timestamp", info->id);
return ret;
......@@ -377,7 +377,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
tscDebug("OTD:0x%"PRIx64" Parse timestamp finished", info->id);
//Parse value
ret = parseTelnetMetricValue(&smlData->fields, &smlData->fieldNum, &idx, info);
ret = parseTelnetMetricValue(&smlData->fields, &smlData->fieldNum, &idx, len - (idx - line), info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse metric value", info->id);
return ret;
......@@ -386,7 +386,7 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
//Parse tagKVs
SHashObj *keyHashTable = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), true, false);
ret = parseTelnetTagKvs(&smlData->tags, &smlData->tagNum, &idx, &smlData->childTableName, keyHashTable, info);
ret = parseTelnetTagKvs(&smlData->tags, &smlData->tagNum, &idx, len - (idx - line), &smlData->childTableName, keyHashTable, info);
if (ret) {
tscError("OTD:0x%"PRIx64" Unable to parse tags", info->id);
taosHashCleanup(keyHashTable);
......@@ -399,24 +399,58 @@ static int32_t tscParseTelnetLine(const char* line, TAOS_SML_DATA_POINT* smlData
return TSDB_CODE_SUCCESS;
}
static int32_t tscParseTelnetLines(char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
for (int32_t i = 0; i < numLines; ++i) {
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseTelnetLine(lines[i], &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("OTD:0x%"PRIx64" data point line parse failed. line %d : %s", info->id, i, lines[i]);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("OTD:0x%"PRIx64" data point line parse success. line %d", info->id, i);
}
taosArrayPush(points, &point);
static int32_t tscParseTelnetLinesInner(char* data, int32_t len, SArray* points, SSmlLinesInfo* info) {
TAOS_SML_DATA_POINT point = {0};
int32_t code = tscParseTelnetLine(data, len, &point, info);
if (code != TSDB_CODE_SUCCESS) {
tscError("OTD:0x%"PRIx64" data point line parse failed.", info->id);
destroySmlDataPoint(&point);
return code;
} else {
tscDebug("OTD:0x%"PRIx64" data point line parse success.", info->id);
}
taosArrayPush(points, &point);
return TSDB_CODE_SUCCESS;
}
int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
static int32_t tscParseTelnetLines(char* data, int32_t len, char* lines[], int numLines, SArray* points, SArray* failedLines, SSmlLinesInfo* info) {
int32_t code = TSDB_CODE_SUCCESS;
if(!data && lines){
for (int32_t i = 0; i < numLines; ++i) {
code = tscParseTelnetLinesInner(lines[i], strlen(lines[i]), points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
}else if(data && !lines){
char* tmp = data;
int32_t lenTmp = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\n' || i == len - 1){
if(data[i] != '\n' && i == len - 1){
lenTmp++;
}
if(lenTmp > 0) {
code = tscParseTelnetLinesInner(tmp, lenTmp, points, info);
if(code != TSDB_CODE_SUCCESS){
return code;
}
}
if(i < len - 1) {
tmp = data + i + 1;
}
lenTmp = 0;
}else{
lenTmp ++;
}
}
}
return code;
}
int taos_insert_telnet_lines(TAOS* taos, char* data, int32_t len, char* lines[], int *numLines, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
int32_t code = 0;
SSmlLinesInfo* info = tcalloc(1, sizeof(SSmlLinesInfo));
......@@ -424,38 +458,52 @@ int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtoco
info->tsType = tsType;
info->protocol = protocol;
if (numLines <= 0 || numLines > 65536) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines numLines should be between 1 and 65536. numLines: %d", info->id, numLines);
if (data && !lines){
*numLines = 0;
for(int i = 0; i < len; i++){
if(data[i] == '\0'){
data[i] = '0';
}
if(data[i] == '\n' || i == len - 1){
(*numLines)++;
}
}
}
if (*numLines <= 0 || *numLines > 65536) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines numLines should be between 1 and 65536. numLines: %d", info->id, *numLines);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
for (int i = 0; i < numLines; ++i) {
if (lines[i] == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
if(!data && lines){
for (int i = 0; i < *numLines; ++i) {
if (lines[i] == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines line %d is NULL", info->id, i);
tfree(info);
code = TSDB_CODE_TSC_APP_ERROR;
return code;
}
}
}
SArray* lpPoints = taosArrayInit(numLines, sizeof(TAOS_SML_DATA_POINT));
SArray* lpPoints = taosArrayInit(*numLines, sizeof(TAOS_SML_DATA_POINT));
if (lpPoints == NULL) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines failed to allocate memory", info->id);
tfree(info);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d lines, first line: %s", info->id, numLines, lines[0]);
code = tscParseTelnetLines(lines, numLines, lpPoints, NULL, info);
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d lines, first line: %s", info->id, *numLines, lines[0]);
code = tscParseTelnetLines(data, len, lines, *numLines, lpPoints, NULL, info);
size_t numPoints = taosArrayGetSize(lpPoints);
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
if (code != 0) {
goto cleanup;
}
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
code = tscSmlInsert(taos, points, (int)numPoints, info);
if (code != 0) {
tscError("OTD:0x%"PRIx64" taos_insert_telnet_lines error: %s", info->id, tstrerror((code)));
......@@ -465,9 +513,7 @@ int taos_insert_telnet_lines(TAOS* taos, char* lines[], int numLines, SMLProtoco
}
cleanup:
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines finish inserting %d lines. code: %d", info->id, numLines, code);
points = TARRAY_GET_START(lpPoints);
numPoints = taosArrayGetSize(lpPoints);
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines finish inserting %d lines. code: %d", info->id, *numLines, code);
for (int i = 0; i < numPoints; ++i) {
destroySmlDataPoint(points+i);
}
......@@ -1059,8 +1105,9 @@ PARSE_JSON_OVER:
return ret;
}
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, SMLTimeStampType tsType, int* affectedRows) {
int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol, int32_t *totalRows, SMLTimeStampType tsType, int* affectedRows) {
int32_t code = 0;
*totalRows = 0;
SSmlLinesInfo* info = tcalloc(1, sizeof(SSmlLinesInfo));
info->id = genUID();
......@@ -1084,12 +1131,12 @@ int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol
tscDebug("OTD:0x%"PRIx64" taos_insert_telnet_lines begin inserting %d points", info->id, 1);
code = tscParseMultiJSONPayload(payload, lpPoints, info);
size_t numPoints = taosArrayGetSize(lpPoints);
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
if (code != 0) {
goto cleanup;
}
TAOS_SML_DATA_POINT* points = TARRAY_GET_START(lpPoints);
code = tscSmlInsert(taos, points, (int)numPoints, info);
if (code != 0) {
tscError("OTD:0x%"PRIx64" taos_insert_json_payload error: %s", info->id, tstrerror((code)));
......@@ -1097,11 +1144,10 @@ int taos_insert_json_payload(TAOS* taos, char* payload, SMLProtocolType protocol
if (affectedRows != NULL) {
*affectedRows = info->affectedRows;
}
*totalRows = numPoints;
cleanup:
tscDebug("OTD:0x%"PRIx64" taos_insert_json_payload finish inserting 1 Point. code: %d", info->id, code);
points = TARRAY_GET_START(lpPoints);
numPoints = taosArrayGetSize(lpPoints);
for (int i = 0; i < numPoints; ++i) {
destroySmlDataPoint(points+i);
}
......
src/client/src/tscServer.c
浏览文件 @ 574bfd96
......@@ -3393,7 +3393,7 @@ int tscRenewTableMeta(SSqlObj *pSql) {
SSqlCmd* pCmd2 = &pSql->rootObj->cmd;
SHashObj *pmap = pCmd2->pTableMetaMap;
if (pmap == atomic_val_compare_exchange_ptr(&pCmd2->pTableMetaMap, pmap, NULL)) {
tscCleanupTableMetaMap(pCmd2->pTableMetaMap);
tscCleanupTableMetaMap(pmap);
}
pCmd2->pTableMetaMap = taosHashInit(4, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BINARY), false, HASH_ENTRY_LOCK);
......
src/client/src/tscUtil.c
浏览文件 @ 574bfd96
......@@ -1373,6 +1373,10 @@ void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, b
// set the correct result
SSDataBlock* p = pQueryInfo->pQInfo->runtimeEnv.outputBuf;
pRes->numOfRows = (p != NULL)? p->info.rows: 0;
bool completed = pRes->numOfRows == 0;
if (p && pRes->numOfRows == 0 && tsAggAlways) {
pRes->numOfRows = 1;
}
if (pRes->code == TSDB_CODE_SUCCESS && pRes->numOfRows > 0) {
tscCreateResPointerInfo(pRes, pQueryInfo);
......@@ -1381,7 +1385,7 @@ void convertQueryResult(SSqlRes* pRes, SQueryInfo* pQueryInfo, uint64_t objId, b
tscDebug("0x%"PRIx64" retrieve result in pRes, numOfRows:%d", objId, pRes->numOfRows);
pRes->row = 0;
pRes->completed = (pRes->numOfRows == 0);
pRes->completed = completed;
}
/*
......@@ -1650,7 +1654,7 @@ void tscResetSqlCmd(SSqlCmd* pCmd, bool clearCachedMeta, uint64_t id) {
tscFreeQueryInfo(pCmd, clearCachedMeta, id);
SHashObj *pmap = pCmd->pTableMetaMap;
if (pmap == atomic_val_compare_exchange_ptr(&pCmd->pTableMetaMap, pmap, NULL)) {
tscCleanupTableMetaMap(pCmd->pTableMetaMap);
tscCleanupTableMetaMap(pmap);
}
taosReleaseRef(tscObjRef, id);
}
......@@ -5243,6 +5247,8 @@ int32_t tscCreateQueryFromQueryInfo(SQueryInfo* pQueryInfo, SQueryAttr* pQueryAt
STableMetaInfo* pTableMetaInfo = pQueryInfo->pTableMetaInfo[0];
pQueryAttr->precision = pTableMetaInfo->pTableMeta->tableInfo.precision;
if (pQueryInfo->groupbyExpr.numOfGroupCols > 0) {
pQueryAttr->pGroupbyExpr = calloc(1, sizeof(SGroupbyExpr));
*(pQueryAttr->pGroupbyExpr) = pQueryInfo->groupbyExpr;
......
src/common/inc/tglobal.h
浏览文件 @ 574bfd96
......@@ -222,6 +222,7 @@ extern int32_t cqDebugFlag;
extern int32_t debugFlag;
extern int8_t tsClientMerge;
extern int8_t tsAggAlways;
// probe alive connection
extern int32_t tsProbeSeconds;
......
src/common/src/tglobal.c
浏览文件 @ 574bfd96
......@@ -278,6 +278,7 @@ int32_t cqDebugFlag = 131;
int32_t fsDebugFlag = 135;
int8_t tsClientMerge = 0;
int8_t tsAggAlways = 0; // Agg function always return value even if zero row
// probe alive connection
int32_t tsProbeSeconds = 5 * 60; // start probe link alive after tsProbeSeconds from starting query
......@@ -1695,6 +1696,16 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "aggAlways";
cfg.ptr = &tsAggAlways;
cfg.valType = TAOS_CFG_VTYPE_INT8;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW | TSDB_CFG_CTYPE_B_CLIENT;
cfg.minValue = 0;
cfg.maxValue = 1;
cfg.ptrLength = 1;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
// default JSON string type option "binary"/"nchar"
cfg.option = "defaultJSONStrType";
cfg.ptr = tsDefaultJSONStrType;
......
src/inc/taos.h
浏览文件 @ 574bfd96
......@@ -218,6 +218,7 @@ DLL_EXPORT void taos_unsubscribe(TAOS_SUB *tsub, int keepProgress);
DLL_EXPORT int taos_load_table_info(TAOS *taos, const char* tableNameList);
DLL_EXPORT TAOS_RES *taos_schemaless_insert(TAOS* taos, char* lines[], int numLines, int protocol, int precision);
DLL_EXPORT TAOS_RES *taos_schemaless_insert_raw(TAOS* taos, char* lines, int len, int32_t *totalRows, int protocol, int precision);
DLL_EXPORT int32_t taos_parse_time(char* timestr, int64_t* time, int32_t len, int32_t timePrec, int8_t dayligth);
......
src/kit/shell/src/shellAuto.c
浏览文件 @ 574bfd96
......@@ -31,7 +31,7 @@
// extern function
void insertChar(Command *cmd, char *c, int size);
void insertStr(Command *cmd, char *str, int size);
typedef struct SAutoPtr {
......@@ -78,6 +78,7 @@ SWords shellCommands[] = {
{"alter user <user_name> privilege read", 0, 0, NULL},
{"alter user <user_name> privilege write", 0, 0, NULL},
{"create table <anyword> using <stb_name> tags(", 0, 0, NULL},
{"create table <anyword> as select ", 0, 0, NULL},
{"create database ", 0, 0, NULL},
{"create table <anyword> as ", 0, 0, NULL},
{"create dnode ", 0, 0, NULL},
......@@ -126,6 +127,7 @@ SWords shellCommands[] = {
{"show vgroups;", 0, 0, NULL},
{"insert into <tb_name> values(", 0, 0, NULL},
{"insert into <tb_name> using <stb_name> tags( <anyword> ) values(", 0, 0, NULL},
{"insert into <tb_name> using <stb_name> <anyword> values(", 0, 0, NULL},
{"use <db_name>", 0, 0, NULL},
{"quit", 0, 0, NULL}
};
......@@ -251,6 +253,9 @@ char * key_tags[] = {
"tags("
};
char * key_select[] = {
"select "
};
//
// ------- gobal variant define ---------
......@@ -278,7 +283,8 @@ bool waitAutoFill = false;
#define WT_VAR_DATATYPE 10
#define WT_VAR_KEYTAGS 11
#define WT_VAR_ANYWORD 12
#define WT_VAR_CNT 13
#define WT_VAR_KEYSELECT 13
#define WT_VAR_CNT 14
#define WT_FROM_DB_MAX 4 // max get content from db
#define WT_FROM_DB_CNT (WT_FROM_DB_MAX + 1)
......@@ -305,7 +311,8 @@ char varTypes[WT_VAR_CNT][64] = {
"<db_options>",
"<data_types>",
"<key_tags>",
"<anyword>"
"<anyword>",
"<key_select>"
};
char varSqls[WT_FROM_DB_CNT][64] = {
......@@ -330,23 +337,23 @@ int cntDel = 0; // delete byte count after next press tab
// show auto tab introduction
void printfIntroduction() {
printf(" ********************* How to Use TAB in TAOS Shell ******************************\n");
printf(" * Taos shell supports pressing TAB key to complete word. You can try it. *\n");
printf(" * Press TAB key anywhere, You'll get surprise. *\n");
printf(" * KEYBOARD SHORTCUT: *\n");
printf(" * [ TAB ] ...... Complete the word or show help if no input *\n");
printf(" * [ Ctrl + A ] ...... move cursor to [A]head of line *\n");
printf(" * [ Ctrl + E ] ...... move cursor to [E]nd of line *\n");
printf(" * [ Ctrl + W ] ...... move cursor to line of middle *\n");
printf(" * [ Ctrl + L ] ...... clean screen *\n");
printf(" * [ Ctrl + K ] ...... clean after cursor *\n");
printf(" * [ Ctrl + U ] ...... clean before cursor *\n");
printf(" * *\n");
printf(" **********************************************************************************\n\n");
printf(" ****************************** Tab Completion **********************************\n");
printf(" * The TDengine CLI supports tab completion for a variety of items, *\n");
printf(" * including database names, table names, function names and keywords. *\n");
printf(" * The full list of shortcut keys is as follows: *\n");
printf(" * [ TAB ] ...... complete the current word *\n");
printf(" * ...... if used on a blank line, display all valid commands *\n");
printf(" * [ Ctrl + A ] ...... move cursor to the st[A]rt of the line *\n");
printf(" * [ Ctrl + E ] ...... move cursor to the [E]nd of the line *\n");
printf(" * [ Ctrl + W ] ...... move cursor to the middle of the line *\n");
printf(" * [ Ctrl + L ] ...... clear the entire screen *\n");
printf(" * [ Ctrl + K ] ...... clear the screen after the cursor *\n");
printf(" * [ Ctrl + U ] ...... clear the screen before the cursor *\n");
printf(" **********************************************************************************\n\n");
}
void showHelp() {
printf("\nThe following are supported commands for Taos shell:");
printf("\nFThe TDengine CLI supports the following commands:");
printf("\n\
----- A ----- \n\
alter database <db_name> <db_options> \n\
......@@ -526,26 +533,16 @@ void parseCommand(SWords * command, bool pattern) {
}
// free Command
void freeCommand(SWords * command) {
SWord * word = command->head;
if (word == NULL) {
return ;
}
// loop
while (word->next) {
SWord * tmp = word;
word = word->next;
void freeCommand(SWords* command) {
SWord* item = command->head;
// loop
while (item) {
SWord* tmp = item;
item = item->next;
// if malloc need free
if(tmp->free && tmp->word)
free(tmp->word);
if (tmp->free && tmp->word) free(tmp->word);
free(tmp);
}
// if malloc need free
if(word->free && word->word)
free(word->word);
free(word);
}
void GenerateVarType(int type, char** p, int count) {
......@@ -586,6 +583,7 @@ bool shellAutoInit() {
GenerateVarType(WT_VAR_TBACTION, tb_actions, sizeof(tb_actions) /sizeof(char *));
GenerateVarType(WT_VAR_DATATYPE, data_types, sizeof(data_types) /sizeof(char *));
GenerateVarType(WT_VAR_KEYTAGS, key_tags, sizeof(key_tags) /sizeof(char *));
GenerateVarType(WT_VAR_KEYSELECT,key_select, sizeof(key_select) /sizeof(char *));
printfIntroduction();
......@@ -613,7 +611,7 @@ void shellAutoExit() {
pthread_mutex_destroy(&tiresMutex);
// free threads
for (int32_t i = 0; i < WT_VAR_CNT; i++) {
for (int32_t i = 0; i < WT_FROM_DB_CNT; i++) {
if (threads[i]) {
taosDestroyThread(threads[i]);
threads[i] = NULL;
......@@ -1079,7 +1077,7 @@ void printScreen(TAOS * con, Command * cmd, SWords * match) {
}
// insert new
insertChar(cmd, (char *)str, strLen);
insertStr(cmd, (char *)str, strLen);
}
......@@ -1170,11 +1168,11 @@ bool nextMatchCommand(TAOS * con, Command * cmd, SWords * firstMatch) {
printScreen(con, cmd, match);
// free
freeCommand(input);
if (input->source) {
free(input->source);
input->source = NULL;
}
freeCommand(input);
free(input);
return true;
......@@ -1194,7 +1192,7 @@ bool fillWithType(TAOS * con, Command * cmd, char* pre, int type) {
// show
int count = strlen(part);
insertChar(cmd, part, count);
insertStr(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
......@@ -1222,7 +1220,7 @@ bool fillTableName(TAOS * con, Command * cmd, char* pre) {
// show
int count = strlen(part);
insertChar(cmd, part, count);
insertStr(cmd, part, count);
cntDel = count; // next press tab delete current append count
free(str);
......@@ -1242,7 +1240,7 @@ char * lastWord(char * p) {
char * p2 = strrchr(p, ',');
if (p1 && p2) {
return MAX(p1, p2) + 1;
return p1 > p2 ? p1 + 1 : p2 + 1;
} else if (p1) {
return p1 + 1;
} else if(p2) {
......@@ -1325,37 +1323,9 @@ bool needInsertFrom(char * sql, int len) {
return true;
}
bool matchSelectQuery(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 512) {
return false;
}
// select and from
if(strncasecmp(p, "select ", 7) != 0) {
// not select query clause
return false;
}
p += 7;
len -= 7;
char* ps = p = strndup(p, len);
// p is string following select keyword
bool appendAfterSelect(TAOS * con, Command * cmd, char* sql, int32_t len) {
char* p = strndup(sql, len);
// union all
char * p1;
......@@ -1374,8 +1344,8 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
bool fieldEnd = fieldsInputEnd(p);
// cheeck fields input end then insert from keyword
if (fieldEnd && p[len-1] == ' ') {
insertChar(cmd, "from", 4);
free(ps);
insertStr(cmd, "from", 4);
free(p);
return true;
}
......@@ -1387,7 +1357,7 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
ret = fillWithType(con, cmd, last, WT_VAR_FUNC);
}
free(ps);
free(p);
return ret;
}
......@@ -1400,10 +1370,73 @@ bool matchSelectQuery(TAOS * con, Command * cmd) {
ret = fillWithType(con, cmd, last, WT_VAR_KEYWORD);
}
free(ps);
free(p);
return ret;
}
int32_t searchAfterSelect(char* p, int32_t len) {
// select * from st;
if(strncasecmp(p, "select ", 7) == 0) {
// check nest query
char *p1 = p + 7;
while(1) {
char *p2 = strstr(p1, "select ");
if(p2 == NULL)
break;
p1 = p2 + 7;
}
return p1 - p;
}
char* as_pos_end = strstr(p, " as select ");
if (as_pos_end == NULL)
return -1;
as_pos_end += 11;
// create table <stream_name> as select
if(strncasecmp(p, "create table ", 13) == 0) {
return as_pos_end - p;;
}
return -1;
}
bool matchSelectQuery(TAOS * con, Command * cmd) {
// if continue press Tab , delete bytes by previous autofill
if (cntDel > 0) {
deleteCount(cmd, cntDel);
cntDel = 0;
}
// match select ...
int len = cmd->commandSize;
char * p = cmd->command;
// remove prefix blank
while (p[0] == ' ' && len > 0) {
p++;
len--;
}
// special range
if(len < 7 || len > 512) {
return false;
}
// search
char* sql_cp = strndup(p, len);
int32_t n = searchAfterSelect(sql_cp, len);
free(sql_cp);
if(n == -1 || n > len)
return false;
p += n;
len -= n;
// append
return appendAfterSelect(con, cmd, p, len);
}
// if is input create fields or tags area, return true
bool isCreateFieldsArea(char * p) {
char * left = strrchr(p, '(');
......@@ -1460,6 +1493,12 @@ bool matchCreateTable(TAOS * con, Command * cmd) {
bool ret = false;
char * last = lastWord(ps);
// except create table m1 as select ....
if (strstr(ps, " as select")) {
free(ps);
return false;
}
// check in create fields or tags input area
if (isCreateFieldsArea(ps)) {
ret = fillWithType(con, cmd, last, WT_VAR_DATATYPE);
......@@ -1484,18 +1523,76 @@ bool matchCreateTable(TAOS * con, Command * cmd) {
bool matchOther(TAOS * con, Command * cmd) {
int len = cmd->commandSize;
char* p = cmd->command;
// '\\'
if (p[len - 1] == '\\') {
// append '\G'
char a[] = "G;";
insertChar(cmd, a, 2);
insertStr(cmd, a, 2);
return true;
}
// too small
if(len < 8)
return false;
// like 'from ( '
char* sql = strndup(p, len);
char* last = lastWord(sql);
if (strcmp(last, "from(") == 0) {
fillWithType(con, cmd, "", WT_VAR_KEYSELECT);
free(sql);
return true;
}
if (strncmp(last, "(", 1) == 0) {
last += 1;
}
char* from = strstr(sql, " from");
// find last ' from'
while (from) {
char* p1 = strstr(from + 5, " from");
if (p1 == NULL)
break;
from = p1;
}
if (from) {
// find next is '('
char * p2 = from + 5;
bool found = false; // found 'from ... ( ...' ... is any count of blank
bool found1 = false; // found '('
while (1) {
if ( p2 == last || *p2 == '\0') {
// last word or string end
if (found1) {
found = true;
}
break;
} else if(*p2 == '(') {
found1 = true;
} else if(*p2 == ' ') {
// do nothing
} else {
// have any other char
break;
}
// move next
p2++;
}
if (found) {
fillWithType(con, cmd, last, WT_VAR_KEYSELECT);
free(sql);
return true;
}
}
free(sql);
return false;
}
// main key press tab
void pressTabKey(TAOS * con, Command * cmd) {
// check
......@@ -1761,4 +1858,4 @@ void callbackAutoTab(char* sqlstr, TAOS* pSql, bool usedb) {
}
return ;
}
}
\ No newline at end of file
src/kit/shell/src/shellCheck.c
浏览文件 @ 574bfd96
......@@ -131,7 +131,7 @@ static void *shellCheckThreadFp(void *arg) {
char *tbname = tbNames[t];
if (tbname == NULL) break;
snprintf(sql, SHELL_SQL_LEN, "select count(*) from %s;", tbname);
snprintf(sql, SHELL_SQL_LEN, "select last_row(_c0) from `%s`;", tbname);
TAOS_RES *pSql = taos_query(pThread->taos, sql);
int32_t code = taos_errno(pSql);
......
src/kit/shell/src/shellCommand.c
浏览文件 @ 574bfd96
......@@ -79,11 +79,22 @@ void insertChar(Command *cmd, char *c, int size) {
/* update the values */
cmd->commandSize += size;
cmd->cursorOffset += size;
for (int i = 0; i < size; i++) {
mbtowc(&wc, c + i, size);
cmd->screenOffset += wcwidth(wc);
cmd->endOffset += wcwidth(wc);
}
cmd->screenOffset += wcwidth(wc);
cmd->endOffset += wcwidth(wc);
showOnScreen(cmd);
}
void insertStr(Command *cmd, char *str, int size) {
clearScreen(cmd->endOffset + prompt_size, cmd->screenOffset + prompt_size);
/* update the buffer */
memmove(cmd->command + cmd->cursorOffset + size, cmd->command + cmd->cursorOffset,
cmd->commandSize - cmd->cursorOffset);
memcpy(cmd->command + cmd->cursorOffset, str, size);
/* update the values */
cmd->commandSize += size;
cmd->cursorOffset += size;
cmd->screenOffset += size;
cmd->endOffset += size;
showOnScreen(cmd);
}
......
src/kit/shell/src/shellDarwin.c
浏览文件 @ 574bfd96
......@@ -35,7 +35,7 @@ void insertChar(Command *cmd, char *c, int size);
void printHelp() {
char indent[10] = " ";
printf("taos shell is used to test the TDengine database\n");
printf("TDengine Command Line is used to test the TDengine database\n");
printf("%s%s\n", indent, "-h");
printf("%s%s%s\n", indent, indent, "TDengine server IP address to connect. The default host is localhost.");
......@@ -68,7 +68,7 @@ void printHelp() {
exit(EXIT_SUCCESS);
}
char DARWINCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char DARWINCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char g_password[SHELL_MAX_PASSWORD_LEN];
......
src/kit/shell/src/shellEngine.c
浏览文件 @ 574bfd96
......@@ -32,7 +32,7 @@
#include <regex.h>
/**************** Global variables ****************/
char CLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char CLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char PROMPT_HEADER[] = "taos> ";
char CONTINUE_PROMPT[] = " -> ";
......@@ -192,9 +192,15 @@ static int32_t shellRunSingleCommand(TAOS *con, char *command) {
if (regex_match(command, "^[ \t]*source[\t ]+[^ ]+[ \t;]*$", REG_EXTENDED | REG_ICASE)) {
/* If source file. */
char *c_ptr = strtok(command, " ;");
assert(c_ptr != NULL);
if (c_ptr == NULL) {
shellRunCommandOnServer(con, command);
return 0;
}
c_ptr = strtok(NULL, " ;");
assert(c_ptr != NULL);
if (c_ptr == NULL) {
shellRunCommandOnServer(con, command);
return 0;
}
source_file(con, c_ptr);
return 0;
}
......
src/kit/shell/src/shellLinux.c
浏览文件 @ 574bfd96
......@@ -194,7 +194,7 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
/* Our argp parser. */
static struct argp argp = {options, parse_opt, args_doc, doc};
char LINUXCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char LINUXCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
char g_password[SHELL_MAX_PASSWORD_LEN];
......
src/kit/shell/src/shellWindows.c
浏览文件 @ 574bfd96
......@@ -21,7 +21,7 @@
extern char configDir[];
char WINCLIENT_VERSION[] = "Welcome to the TDengine shell from %s, Client Version:%s\n"
char WINCLIENT_VERSION[] = "Welcome to the TDengine Command Line Interface from %s, Client Version:%s\n"
"Copyright (c) 2022 by TAOS Data, Inc. All rights reserved.\n\n";
void printVersion() {
......@@ -30,7 +30,7 @@ void printVersion() {
void printHelp() {
char indent[10] = " ";
printf("taos shell is used to test the TDengine database\n");
printf("TDengine Command Line is used to test the TDengine database\n");
printf("%s%s\n", indent, "-h");
printf("%s%s%s\n", indent, indent, "TDengine server FQDN to connect. The default host is localhost.");
......
src/kit/shell/src/tire.c
浏览文件 @ 574bfd96
......@@ -325,30 +325,26 @@ void matchPrefixFromTree(STire* tire, char* prefix, SMatch* match) {
}
SMatch* matchPrefix(STire* tire, char* prefix, SMatch* match) {
if(match == NULL) {
match = (SMatch* )tmalloc(sizeof(SMatch));
memset(match, 0, sizeof(SMatch));
}
switch (tire->type) {
case TIRE_TREE:
matchPrefixFromTree(tire, prefix, match);
case TIRE_LIST:
matchPrefixFromList(tire, prefix, match);
default:
break;
}
// return if need
if (match->count == 0) {
freeMatch(match);
match = NULL;
}
return match;
SMatch* rMatch = match; // define return match
if (rMatch == NULL) {
rMatch = (SMatch*)malloc(sizeof(SMatch));
memset(rMatch, 0, sizeof(SMatch));
}
switch (tire->type) {
case TIRE_TREE:
matchPrefixFromTree(tire, prefix, rMatch);
break;
case TIRE_LIST:
matchPrefixFromList(tire, prefix, rMatch);
break;
default:
break;
}
return rMatch;
}
// get all items from tires tree
void enumFromList(STire* tire, SMatch* match) {
StrName * item = tire->head;
......@@ -395,8 +391,10 @@ SMatch* enumAll(STire* tire) {
switch (tire->type) {
case TIRE_TREE:
enumFromTree(tire, match);
break;
case TIRE_LIST:
enumFromList(tire, match);
break;
default:
break;
}
......
taos-tools @ cc973e0c
比较 f03c09a9...cc973e0c
Subproject commit f03c09a9f0882949b52f34f11607808ee17bfbc5
Subproject commit cc973e0c2999a739fa2d06c25ab5adb2a7537c8f
src/mnode/src/mnodeDb.c
浏览文件 @ 574bfd96
......@@ -1013,9 +1013,9 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) {
newCfg.daysToKeep2 = daysToKeep2;
}
if (minRows > 0 && minRows != pDb->cfg.minRowsPerFileBlock) {
mError("db:%s, can't alter minRows option", pDb->name);
terrno = TSDB_CODE_MND_INVALID_DB_OPTION;
if (minRows >= 0 && minRows != pDb->cfg.minRowsPerFileBlock) {
mDebug("db:%s, minRows:%d change to %d", pDb->name, pDb->cfg.minRowsPerFileBlock, minRows);
newCfg.minRowsPerFileBlock = minRows;
}
if (maxRows > 0 && maxRows != pDb->cfg.maxRowsPerFileBlock) {
......
taosadapter @ 1e0a0209
比较 4c8d9259...1e0a0209
Subproject commit 4c8d92596d3c57ebc25e9ce0e0d8582685e6bcda
Subproject commit 1e0a02097f8433f557125898c756f293c97da88d
src/query/inc/sql.y
浏览文件 @ 574bfd96
......@@ -311,6 +311,7 @@ alter_db_optr(Y) ::= alter_db_optr(Z) blocks(X). { Y = Z; Y.numOfBlocks = s
alter_db_optr(Y) ::= alter_db_optr(Z) comp(X). { Y = Z; Y.compressionLevel = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) update(X). { Y = Z; Y.update = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) cachelast(X). { Y = Z; Y.cachelast = strtol(X.z, NULL, 10); }
alter_db_optr(Y) ::= alter_db_optr(Z) minrows(X). { Y = Z; Y.minRowsPerBlock = strtol(X.z, NULL, 10); }
// dynamically update the following two parameters are not allowed.
//alter_db_optr(Y) ::= alter_db_optr(Z) fsync(X). { Y = Z; Y.fsyncPeriod = strtol(X.z, NULL, 10); }
......
src/query/src/qAggMain.c
浏览文件 @ 574bfd96
此差异已折叠。
src/query/src/qExecutor.c
浏览文件 @ 574bfd96
......@@ -3668,7 +3668,7 @@ int32_t binarySearchForKey(char* pValue, int num, TSKEY key, int order) {
int32_t midPos = -1;
int32_t numOfRows;
if (num <= 0) {
if (num <= 0 || pValue == NULL) {
return -1;
}
......@@ -5447,7 +5447,8 @@ int32_t doInitQInfo(SQInfo* pQInfo, STSBuf* pTsBuf, void* tsdb, void* sourceOptr
int16_t order = (pQueryAttr->order.order == pRuntimeEnv->pTsBuf->tsOrder) ? TSDB_ORDER_ASC : TSDB_ORDER_DESC;
tsBufResetPos(pRuntimeEnv->pTsBuf);
tsBufSetTraverseOrder(pRuntimeEnv->pTsBuf, order);
tsBufNextPos(pTsBuf);
bool ret = tsBufNextPos(pTsBuf);
UNUSED(ret);
}
int32_t ps = DEFAULT_PAGE_SIZE;
......@@ -6154,7 +6155,7 @@ SOperatorInfo* createGlobalAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv,
goto _clean;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
setDefaultOutputBuf(pRuntimeEnv, &pInfo->binfo, pInfo->seed, MERGE_STAGE);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
......@@ -6329,6 +6330,7 @@ SOperatorInfo* createOrderOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperatorI
pDataBlock->pDataBlock = taosArrayInit(numOfOutput, sizeof(SColumnInfoData));
if (pDataBlock->pDataBlock == NULL) {
free(pDataBlock);
goto _clean;
}
......@@ -6623,7 +6625,7 @@ static SSDataBlock* doProjectOperation(void* param, bool* newgroup) {
}
copyTsColoum(pRes, pInfo->pCtx, pOperator->numOfOutput);
clearNumOfRes(pInfo->pCtx, pOperator->numOfOutput);
return (pInfo->pRes->info.rows > 0) ? pInfo->pRes : NULL;
return pInfo->pRes;
}
static SSDataBlock* doLimit(void* param, bool* newgroup) {
......@@ -6872,14 +6874,18 @@ static bool doEveryInterpolation(SOperatorInfo* pOperatorInfo, SSDataBlock* pBlo
break;
}
}
if (pCtx == NULL) {
goto group_finished_exit;
}
TSKEY* tsCols = NULL;
if (pBlock && pBlock->pDataBlock != NULL) {
SColumnInfoData* pColDataInfo = taosArrayGet(pBlock->pDataBlock, 0);
tsCols = (int64_t*)pColDataInfo->pData;
assert(tsCols[0] == pBlock->info.window.skey && tsCols[pBlock->info.rows - 1] == pBlock->info.window.ekey);
}
if (pCtx->startTs == INT64_MIN) {
if (pQueryAttr->range.skey == INT64_MIN) {
if (NULL == tsCols) {
......@@ -6977,8 +6983,8 @@ static bool doEveryInterpolation(SOperatorInfo* pOperatorInfo, SSDataBlock* pBlo
return false;
}
int32_t startPos = binarySearchForKey((char*)tsCols, pBlock->info.rows, pCtx->startTs, pQueryAttr->order.order);
int32_t nRows = pBlock ? pBlock->info.rows : 0;
int32_t startPos = binarySearchForKey((char*) tsCols, nRows, pCtx->startTs, pQueryAttr->order.order);
if (ascQuery && pQueryAttr->fillType != TSDB_FILL_NEXT && pCtx->start.key == INT64_MIN) {
if (startPos < 0) {
......@@ -7778,7 +7784,7 @@ SOperatorInfo* createAggregateOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOpera
goto _clean;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
setDefaultOutputBuf(pRuntimeEnv, &pInfo->binfo, pInfo->seed, MASTER_SCAN);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
......@@ -7985,7 +7991,7 @@ SOperatorInfo* createProjectOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperato
return NULL;
}
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
pInfo->bufCapacity = pRuntimeEnv->resultInfo.capacity;
SOptrBasicInfo* pBInfo = &pInfo->binfo;
......@@ -8171,7 +8177,7 @@ SOperatorInfo* createTimeEveryOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOpera
SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr;
pInfo->seed = rand();
pInfo->seed = taosSafeRand();
pInfo->bufCapacity = pRuntimeEnv->resultInfo.capacity;
pInfo->groupDone = true;
pInfo->lastGroupIdx = -1;
......@@ -8516,15 +8522,18 @@ SOperatorInfo* createSLimitOperatorInfo(SQueryRuntimeEnv* pRuntimeEnv, SOperator
SColIndex* idx = taosArrayGet(pInfo->orderColumnList, i);
offset += pExpr[idx->colIndex].base.resBytes;
}
pInfo->pRes = createOutputBuf(pExpr, numOfOutput, pRuntimeEnv->resultInfo.capacity);
SOperatorInfo* pOperator = calloc(1, sizeof(SOperatorInfo));
if (pInfo->pRes == NULL || pOperator == NULL) {
if (pOperator == NULL) {
goto _clean;
}
pInfo->pRes = createOutputBuf(pExpr, numOfOutput, pRuntimeEnv->resultInfo.capacity);
if (pInfo->pRes == NULL) {
tfree(pOperator);
goto _clean;
}
pOperator->name = "SLimitOperator";
pOperator->operatorType = OP_SLimit;
pOperator->blockingOptr = false;
......
src/query/src/queryMain.c
浏览文件 @ 574bfd96
......@@ -338,6 +338,45 @@ int32_t qRetrieveQueryResultInfo(qinfo_t qinfo, bool* buildRes, void* pRspContex
return code;
}
bool qFitAlwaysValue(SQInfo * pQInfo) {
// must agg query
if (!pQInfo->query.simpleAgg)
return false;
// must not include ts column
SSDataBlock* pBlock = pQInfo->runtimeEnv.outputBuf;
if (pBlock == NULL || pBlock->pDataBlock == NULL)
return false;
SColumnInfoData* pColInfoData = taosArrayGet(pBlock->pDataBlock, 0);
if (pColInfoData == NULL)
return false;
// must not first column is timestamp
if (pColInfoData->info.type == TSDB_DATA_TYPE_TIMESTAMP)
return false;
// fit ok
return true;
}
bool doFillResultWithNull(SQInfo* pQInfo, char * data, size_t numOfRows) {
SQueryRuntimeEnv* pRuntimeEnv = &pQInfo->runtimeEnv;
SQueryAttr* pQueryAttr = pRuntimeEnv->pQueryAttr;
SSDataBlock* pRes = pRuntimeEnv->outputBuf;
int32_t numOfCols = pQueryAttr->numOfOutput;
for (int32_t i = 0; i < numOfCols; ++i) {
int16_t functionId = pQueryAttr->pExpr1[i].base.functionId;
// col
SColumnInfoData* pCol = taosArrayGet(pRes->pDataBlock, i);
if (functionId != TSDB_FUNC_COUNT)
setNull(data, pCol->info.type, pCol->info.bytes);
data += pCol->info.bytes * numOfRows;
}
return true;
}
int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *contLen, bool* continueExec) {
SQInfo *pQInfo = (SQInfo *)qinfo;
int32_t compLen = 0;
......@@ -350,6 +389,14 @@ int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *co
SQueryRuntimeEnv* pRuntimeEnv = &pQInfo->runtimeEnv;
int32_t s = GET_NUM_OF_RESULTS(pRuntimeEnv);
bool fillNull = false;
if (s == 0 && tsAggAlways) {
if (qFitAlwaysValue(pQInfo)) {
s = 1;
fillNull = true;
}
}
size_t size = pQueryAttr->resultRowSize * s;
size += sizeof(int32_t);
size += sizeof(STableIdInfo) * taosHashGetSize(pRuntimeEnv->pTableRetrieveTsMap);
......@@ -378,6 +425,9 @@ int32_t qDumpRetrieveResult(qinfo_t qinfo, SRetrieveTableRsp **pRsp, int32_t *co
if (GET_NUM_OF_RESULTS(&(pQInfo->runtimeEnv)) > 0 && pQInfo->code == TSDB_CODE_SUCCESS) {
doDumpQueryResult(pQInfo, (*pRsp)->data, (*pRsp)->compressed, &compLen);
} else {
if (fillNull) {
doFillResultWithNull(pQInfo, (*pRsp)->data, 1);
}
setQueryStatus(pRuntimeEnv, QUERY_OVER);
}
......
src/query/src/sql.c
浏览文件 @ 574bfd96
此差异已折叠。
src/rpc/src/rpcMain.c
浏览文件 @ 574bfd96
......@@ -1080,7 +1080,9 @@ static void rpcProcessBrokenLink(SRpcConn *pConn) {
pContext->code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
pContext->pConn = NULL;
pConn->pReqMsg = NULL;
taosTmrStart(rpcProcessConnError, 0, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 0, rid, pRpc->tmrCtrl);
}
if (pConn->inType) rpcReportBrokenLinkToServer(pConn);
......@@ -1293,7 +1295,9 @@ static void rpcProcessIncomingMsg(SRpcConn *pConn, SRpcHead *pHead, SRpcReqConte
rpcFreeCont(rpcMsg.pCont);
} else if (pHead->code == TSDB_CODE_RPC_NOT_READY || pHead->code == TSDB_CODE_APP_NOT_READY || pHead->code == TSDB_CODE_DND_EXITING) {
pContext->code = pHead->code;
rpcProcessConnError(pContext, NULL);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
rpcProcessConnError(rid, NULL);
rpcFreeCont(rpcMsg.pCont);
} else {
rpcNotifyClient(pContext, &rpcMsg);
......@@ -1395,7 +1399,9 @@ static TBOOL rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) {
if (pConn == NULL) {
pContext->code = terrno;
// in rpcProcessConnError if numOfTry over limit, could call rpcNotifyClient to stop query
taosTmrStart(rpcProcessConnError, 1, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
return BOOL_ASYNC;
}
......@@ -1442,7 +1448,9 @@ static TBOOL rpcSendReqToServer(SRpcInfo *pRpc, SRpcReqContext *pContext) {
// try next ip again
pContext->code = terrno;
// in rpcProcessConnError if numOfTry over limit, could call rpcNotifyClient to stop query
taosTmrStart(rpcProcessConnError, 1, pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
return BOOL_ASYNC;
}
......@@ -1480,11 +1488,23 @@ static bool rpcSendMsgToPeer(SRpcConn *pConn, void *msg, int msgLen) {
}
static void rpcProcessConnError(void *param, void *id) {
SRpcReqContext *pContext = (SRpcReqContext *)param;
if (NULL == param) {
return;
}
int64_t *rid = (int64_t*)param;
SRpcReqContext *pContext = (SRpcReqContext *)taosAcquireRef(tsRpcRefId, *rid);
if (NULL == pContext) {
free(param);
return;
}
SRpcInfo *pRpc = pContext->pRpc;
SRpcMsg rpcMsg;
if (pRpc == NULL) {
taosReleaseRef(tsRpcRefId, *rid);
free(param);
return;
}
......@@ -1504,6 +1524,9 @@ static void rpcProcessConnError(void *param, void *id) {
pContext->epSet.inUse = pContext->epSet.inUse % pContext->epSet.numOfEps;
rpcSendReqToServer(pRpc, pContext);
}
taosReleaseRef(tsRpcRefId, *rid);
free(param);
}
static void rpcProcessRetryTimer(void *param, void *tmrId) {
......@@ -1528,7 +1551,9 @@ static void rpcProcessRetryTimer(void *param, void *tmrId) {
pConn->pContext->code = TSDB_CODE_RPC_NETWORK_UNAVAIL;
pConn->pContext->pConn = NULL;
pConn->pReqMsg = NULL;
taosTmrStart(rpcProcessConnError, 1, pConn->pContext, pRpc->tmrCtrl);
int64_t *rid = malloc(sizeof(int64_t));
*rid = pConn->pContext->rid;
taosTmrStart(rpcProcessConnError, 1, rid, pRpc->tmrCtrl);
rpcReleaseConn(pConn);
}
}
......@@ -1892,4 +1917,4 @@ bool rpcSaveSendInfo(int64_t rpcRid, void** ppContext) {
taosReleaseRef(tsRpcRefId, rpcRid);
return true;
}
\ No newline at end of file
}
src/tsdb/src/tsdbCommitQueue.c
浏览文件 @ 574bfd96
......@@ -132,13 +132,16 @@ static void tsdbApplyRepoConfig(STsdbRepo *pRepo) {
pRepo->config.keep2 = pRepo->save_config.keep2;
pRepo->config.cacheLastRow = pRepo->save_config.cacheLastRow;
pRepo->config.totalBlocks = pRepo->save_config.totalBlocks;
pRepo->config.minRowsPerFileBlock = pRepo->save_config.minRowsPerFileBlock;
pthread_mutex_unlock(&pRepo->save_mutex);
tsdbInfo("vgId:%d apply new config: compression(%d), keep(%d,%d,%d), totalBlocks(%d), cacheLastRow(%d->%d),totalBlocks(%d->%d)",
REPO_ID(pRepo),
pSaveCfg->compression, pSaveCfg->keep,pSaveCfg->keep1, pSaveCfg->keep2,
pSaveCfg->totalBlocks, oldCfg.cacheLastRow, pSaveCfg->cacheLastRow, oldTotalBlocks, pSaveCfg->totalBlocks);
tsdbInfo(
"vgId:%d apply new config: "
"compression(%d),keep(%d,%d,%d),totalBlocks(%d),cacheLastRow(%d->%d),totalBlocks(%d->%d),minRows(%d->%d)",
REPO_ID(pRepo), pSaveCfg->compression, pSaveCfg->keep, pSaveCfg->keep1, pSaveCfg->keep2, pSaveCfg->totalBlocks,
oldCfg.cacheLastRow, pSaveCfg->cacheLastRow, oldTotalBlocks, pSaveCfg->totalBlocks, oldCfg.minRowsPerFileBlock,
pSaveCfg->minRowsPerFileBlock);
int err = tsdbExpandPool(pRepo, oldTotalBlocks);
if (!TAOS_SUCCEEDED(err)) {
......
src/tsdb/src/tsdbMain.c
浏览文件 @ 574bfd96
......@@ -233,7 +233,7 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
ASSERT(pRCfg->tsdbId == pCfg->tsdbId);
ASSERT(pRCfg->cacheBlockSize == pCfg->cacheBlockSize);
ASSERT(pRCfg->daysPerFile == pCfg->daysPerFile);
ASSERT(pRCfg->minRowsPerFileBlock == pCfg->minRowsPerFileBlock);
// ASSERT(pRCfg->minRowsPerFileBlock == pCfg->minRowsPerFileBlock);
ASSERT(pRCfg->maxRowsPerFileBlock == pCfg->maxRowsPerFileBlock);
ASSERT(pRCfg->precision == pCfg->precision);
......@@ -256,6 +256,9 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
if (pRCfg->totalBlocks != pCfg->totalBlocks) {
configChanged = true;
}
if (pRCfg->minRowsPerFileBlock != pCfg->minRowsPerFileBlock) {
configChanged = true;
}
if (!configChanged) {
tsdbError("vgId:%d no config changed", REPO_ID(repo));
......@@ -277,15 +280,16 @@ int32_t tsdbConfigRepo(STsdbRepo *repo, STsdbCfg *pCfg) {
pSaveCfg->keep2 = pCfg->keep2;
pSaveCfg->cacheLastRow = pCfg->cacheLastRow;
pSaveCfg->totalBlocks = pCfg->totalBlocks;
pSaveCfg->minRowsPerFileBlock = pCfg->minRowsPerFileBlock;
tsdbInfo("vgId:%d old config: compression(%d), keep(%d,%d,%d), cacheLastRow(%d),totalBlocks(%d)",
tsdbInfo("vgId:%d old config: compression(%d),keep(%d,%d,%d),cacheLastRow(%d),totalBlocks(%d),minRows(%d)",
REPO_ID(repo),
pRCfg->compression, pRCfg->keep, pRCfg->keep1,pRCfg->keep2,
pRCfg->cacheLastRow, pRCfg->totalBlocks);
tsdbInfo("vgId:%d new config: compression(%d), keep(%d,%d,%d), cacheLastRow(%d),totalBlocks(%d)",
pRCfg->cacheLastRow, pRCfg->totalBlocks, pRCfg->minRowsPerFileBlock);
tsdbInfo("vgId:%d new config: compression(%d),keep(%d,%d,%d),cacheLastRow(%d),totalBlocks(%d),minRows(%d)",
REPO_ID(repo),
pSaveCfg->compression, pSaveCfg->keep,pSaveCfg->keep1, pSaveCfg->keep2,
pSaveCfg->cacheLastRow,pSaveCfg->totalBlocks);
pSaveCfg->cacheLastRow,pSaveCfg->totalBlocks,pSaveCfg->minRowsPerFileBlock);
repo->config_changed = true;
......@@ -641,6 +645,7 @@ static void tsdbFreeRepo(STsdbRepo *pRepo) {
// tsdbFreeMemTable(pRepo->imem);
tsem_destroy(&(pRepo->readyToCommit));
pthread_mutex_destroy(&pRepo->mutex);
pthread_mutex_destroy(&pRepo->save_mutex);
free(pRepo);
}
}
......
src/tsdb/src/tsdbRead.c
浏览文件 @ 574bfd96
......@@ -836,9 +836,11 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
if (ASCENDING_TRAVERSE(order)) {
assert(pCheckInfo->lastKey <= key);
} else {
assert(pCheckInfo->lastKey >= key);
}
// mem data tskey maybe large than pCheckInfo->lastKey
//} else {
// assert(pCheckInfo->lastKey >= key);
//}
} else {
tsdbDebug("%p uid:%"PRId64", tid:%d no data in mem, 0x%"PRIx64, pHandle, pCheckInfo->tableId.uid, pCheckInfo->tableId.tid,
......@@ -858,9 +860,11 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
if (ASCENDING_TRAVERSE(order)) {
assert(pCheckInfo->lastKey <= key);
} else {
assert(pCheckInfo->lastKey >= key);
}
// imem data tskey maybe large than pCheckInfo->lastKey
//} else {
// assert(pCheckInfo->lastKey >= key);
//}
} else {
tsdbDebug("%p uid:%"PRId64", tid:%d no data in imem, 0x%"PRIx64, pHandle, pCheckInfo->tableId.uid, pCheckInfo->tableId.tid,
pHandle->qId);
......
src/util/inc/tarray.h
浏览文件 @ 574bfd96
......@@ -36,10 +36,10 @@ typedef struct SArray {
} SArray;
/**
*
* @param size
* @param elemSize
* @return
* Initializes a heap-allocated array with `size` elements, the size of element is `elemSize`.
* @param size the number of element.
* @param elemSize the size of element.
* @return the pointer points to the array.
*/
void* taosArrayInit(size_t size, size_t elemSize);
......
src/util/inc/tconfig.h
浏览文件 @ 574bfd96
......@@ -20,7 +20,7 @@
extern "C" {
#endif
#define TSDB_CFG_MAX_NUM 139
#define TSDB_CFG_MAX_NUM 140
#define TSDB_CFG_PRINT_LEN 23
#define TSDB_CFG_OPTION_LEN 24
#define TSDB_CFG_VALUE_LEN 41
......
tests/develop-test/2-query/function_mode.py
浏览文件 @ 574bfd96
......@@ -57,10 +57,10 @@ class TDTestCase:
tdSql.checkData(0, 1, 2)
tdSql.query('select mode(num) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, -32767)
tdSql.query('select mode(dbool) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, False)
tdSql.query('select mode(dtiny) from d001')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
......@@ -87,7 +87,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 1)
tdSql.query('select mode(num),mode(voltage) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.checkData(0, 1, 1)
tdSql.query('select mode(dbool) from smode')
tdSql.checkRows(1)
......@@ -97,7 +97,7 @@ class TDTestCase:
tdSql.checkData(0, 0, None)
tdSql.query('select mode(dfloat) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3.3232219219207764)
tdSql.query('select mode(ddouble) from smode')
tdSql.checkRows(1)
tdSql.checkData(0, 0, 4.982392323)
......@@ -112,16 +112,16 @@ class TDTestCase:
#group by column
tdSql.query('select mode(num) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.checkData(1, 0, None)
tdSql.execute('insert into D002 values("2021-11-17 20:31:31", 1, 3276, true, NULL, 3.32322, 4.982392323, "你好吗", "sdf", 333)')
tdSql.query('select mode(num) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, 3276)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3276)
tdSql.query('select mode(dfloat) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, 3.3232200145721436)
tdSql.query('select mode(dchar) from d002 group by dbinary')
tdSql.checkRows(2)
tdSql.checkData(1, 0, "你好吗")
......@@ -138,7 +138,7 @@ class TDTestCase:
#group by tbname
tdSql.query('select mode(dchar) from smode group by tbname')
tdSql.checkRows(2)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, "你好吗")
tdSql.checkData(0, 1, "d001")
tdSql.checkData(1, 0, "你好吗")
tdSql.checkData(1, 1, "d002")
......@@ -148,7 +148,7 @@ class TDTestCase:
tdSql.checkRows(2)
tdSql.checkData(0, 0, 4.982392323)
tdSql.checkData(0, 1, "Beijing.haidian")
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, 4.982392323)
tdSql.checkData(1, 1, "Beijing.Chaoyang")
#group by tag,column
......@@ -169,7 +169,7 @@ class TDTestCase:
tdSql.checkRows(2)
tdSql.checkData(0, 0, "你好吗")
tdSql.checkData(0, 1, "d002")
tdSql.checkData(1, 0, None)
tdSql.checkData(1, 0, "你好吗")
tdSql.checkData(1, 1, "d001")
#where
......@@ -178,7 +178,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 19)
tdSql.query('select mode(voltage) from smode where num > 9')
tdSql.checkRows(1)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 19)
#interval
tdSql.query('select mode(voltage) from smode interval(1s)')
......@@ -188,7 +188,7 @@ class TDTestCase:
tdSql.checkData(0, 0, "2021-01-01 00:00:00")
tdSql.checkData(0, 1, 1)
tdSql.checkData(1, 0, "2022-01-01 00:00:00")
tdSql.checkData(1, 1, None)
tdSql.checkData(1, 1, 19)
tdSql.query('select mode(voltage) from smode interval(1n)')
tdSql.checkRows(4)
tdSql.checkData(0, 0, "2021-10-01 00:00:00")
......@@ -198,7 +198,7 @@ class TDTestCase:
tdSql.checkData(2, 0, "2021-12-01 00:00:00")
tdSql.checkData(2, 1, 2)
tdSql.checkData(3, 0, "2022-01-01 00:00:00")
tdSql.checkData(3, 1, None)
tdSql.checkData(3, 1, 19)
tdSql.query('select mode(voltage) from smode where ts > "2021-09-01 00:00:00" and ts <"2022-02-02 00:00:00" interval(1n) fill(prev)')
tdSql.checkRows(6)
......@@ -207,7 +207,7 @@ class TDTestCase:
tdSql.checkData(3, 0, "2021-12-01 00:00:00")
tdSql.checkData(3, 1, 2)
tdSql.checkData(5, 0, "2022-02-01 00:00:00")
tdSql.checkData(5, 1, 2)
tdSql.checkData(5, 1, 19)
#session
tdSql.query('select mode(voltage) from d002 session(ts,1w)')
......@@ -222,7 +222,7 @@ class TDTestCase:
#state_window
tdSql.query('select mode(dfloat) from d002 state_window(voltage)')
tdSql.checkRows(3)
tdSql.checkData(0, 0, None)
tdSql.checkData(0, 0, 3.3232200145721436)
tdSql.checkData(1, 0, None)
#slimit/soffset
......@@ -237,9 +237,9 @@ class TDTestCase:
#having
tdSql.query('select mode(ddouble) from smode group by location having mode(ddouble)>3')
tdSql.checkRows(1)
tdSql.checkRows(2)
tdSql.checkData(0, 0, 4.982392323)
tdSql.checkData(0, 1, "Beijing.haidian")
tdSql.checkData(0, 1, "Beijing.Chaoyang")
#subquery
tdSql.query('select mode(ddouble) from (select * from smode where voltage = 1)')
......
tests/develop-test/5-taos-tools/taosbenchmark/default_json.py
浏览文件 @ 574bfd96
......@@ -19,10 +19,9 @@ from util.dnodes import *
class TDTestCase:
def caseDescription(self):
'''
"""
[TD-11510] taosBenchmark test cases
'''
return
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
......@@ -31,19 +30,23 @@ class TDTestCase:
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
else:
projPath = selfPath[:selfPath.find("tests")]
projPath = selfPath[: selfPath.find("tests")]
paths = []
for root, dirs, files in os.walk(projPath):
if ((tool) in files):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
if (len(paths) == 0):
if len(paths) == 0:
tdLog.exit("taosBenchmark not found!")
return
else:
......@@ -52,12 +55,12 @@ class TDTestCase:
def run(self):
binPath = self.getPath()
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/default.json" %binPath
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/default.json" % binPath
tdLog.info("%s" % cmd)
os.system("%s" % cmd)
tdSql.execute("reset query cache")
tdSql.query("select count(tbname) from db.stb")
tdSql.checkData(0, 0, 10)
tdSql.query("show db.tables")
tdSql.checkRows(10)
tdSql.query("select count(*) from db.stb")
tdSql.checkData(0, 0, 100)
......@@ -67,4 +70,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
tests/develop-test/5-taos-tools/taosbenchmark/json/default.json
浏览文件 @ 574bfd96
......@@ -20,8 +20,10 @@
"super_tables": [{
"name": "stb",
"childtable_prefix": "stb_",
"childtable_count": 10,
"insert_rows": 10,
"columns": [{"type": "INT"}],
"tags": [{"type": "INT"}]
}]
}]
}
\ No newline at end of file
}
tests/perftest-scripts/coverage_test.sh
浏览文件 @ 574bfd96
......@@ -64,6 +64,21 @@ function runGeneralCaseOneByOne {
done < $1
}
function runTestNGCaseOneByOne {
while read -r line; do
if [[ $line =~ ^./taostest* ]]; then
case=`echo $line | cut -d' ' -f 3 | cut -d'=' -f 2`
yaml=`echo $line | cut -d' ' -f 2`
if [ -n "$case" ]; then
date +%F\ %T | tee -a $TDENGINE_COVERAGE_REPORT && ./taostest $yaml --case=$case --keep > /dev/null 2>&1 && \
echo -e "${GREEN}$case success${NC}" | tee -a $TDENGINE_COVERAGE_REPORT \
|| echo -e "${RED}$case failed${NC}" | tee -a $TDENGINE_COVERAGE_REPORT
fi
fi
done < $1
}
function runTest {
echo "run Test"
......@@ -87,6 +102,13 @@ function runTest {
else
sed -i '3i\\n' $TDENGINE_COVERAGE_REPORT
fi
# run TestNG cases
stopTaosd
$TDENGINE_DIR/debug/build/bin/taosd -c $TDENGINE_DIR/debug/test/cfg > /dev/null &
sleep 10
cd $TDENGINE_DIR/../TestNG/cases
runTestNGCaseOneByOne ../scripts/cases.txt
cd $TDENGINE_DIR/tests
rm -rf ../sim
......
tests/pytest/query/query1970YearsAf.py
浏览文件 @ 574bfd96
......@@ -25,8 +25,8 @@ from util.cases import *
from util.dnodes import *
from util.dnodes import TDDnode
class TDTestCase:
class TDTestCase:
def __init__(self):
self.path = ""
......@@ -68,7 +68,7 @@ class TDTestCase:
"cachelast": 0,
"quorum": 1,
"fsync": 3000,
"update": 0
"update": 0,
}
# set stable schema
......@@ -99,8 +99,7 @@ class TDTestCase:
{"type": "TINYINT", "count": 2},
{"type": "BOOL", "count": 2},
{"type": "NCHAR", "len": 3, "count": 1},
{"type": "BINARY", "len": 8, "count": 1}
{"type": "BINARY", "len": 8, "count": 1},
],
"tags": [
{"type": "INT", "count": 2},
......@@ -111,17 +110,14 @@ class TDTestCase:
{"type": "TINYINT", "count": 2},
{"type": "BOOL", "count": 2},
{"type": "NCHAR", "len": 3, "count": 1},
{"type": "BINARY", "len": 8, "count": 1}
]
{"type": "BINARY", "len": 8, "count": 1},
],
}
# create different stables like stable1 and add to list super_tables
super_tables = []
super_tables.append(stable1)
database = {
"dbinfo": dbinfo,
"super_tables": super_tables
}
database = {"dbinfo": dbinfo, "super_tables": super_tables}
cfgdir = self.getCfgDir()
create_table = {
......@@ -137,33 +133,60 @@ class TDTestCase:
"confirm_parameter_prompt": "no",
"insert_interval": 0,
"num_of_records_per_req": 100,
"databases": [database]
"databases": [database],
}
return create_table
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
elif "/tests/" in selfPath:
projPath = selfPath[: selfPath.find("/tests/")]
else:
tdLog.exit("path: %s is not supported" % selfPath)
paths = []
for root, dirs, files in os.walk(projPath):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
if len(paths) == 0:
return ""
return paths[0]
def createinsertfile(self):
create_table = self.creatcfg()
date = datetime.datetime.now().strftime("%Y%m%d%H%M")
file_create_table = f"/tmp/insert_{date}.json"
with open(file_create_table, 'w') as f:
with open(file_create_table, "w") as f:
json.dump(create_table, f)
return file_create_table
def inserttable(self, filepath):
create_table_cmd = f"taosdemo -f {filepath} > /dev/null 2>&1"
binPath = self.getPath("taosBenchmark")
if binPath == "":
tdLog.exit("taosBenchmark not found!")
else:
tdLog.info("taosBenchmark found in %s" % binPath)
create_table_cmd = "%s -f %s > /dev/null 2>&1" % (binPath, filepath)
_ = subprocess.check_output(create_table_cmd, shell=True).decode("utf-8")
def sqlsquery(self):
# stable query
tdSql.query(
"select * from stb2 where stb2.ts < '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from stb2 where stb2.ts < '1970-01-01 00:00:00.000' ")
tdSql.checkRows(43200)
tdSql.query(
"select * from stb2 where stb2.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from stb2 where stb2.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(6800)
tdSql.query(
......@@ -172,14 +195,10 @@ class TDTestCase:
tdSql.checkRows(3590)
# child-tables query
tdSql.query(
"select * from t0 where t0.ts < '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from t0 where t0.ts < '1970-01-01 00:00:00.000' ")
tdSql.checkRows(4320)
tdSql.query(
"select * from t1 where t1.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select * from t1 where t1.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(680)
tdSql.query(
......@@ -192,9 +211,7 @@ class TDTestCase:
)
tdSql.checkRows(680)
tdSql.query(
"select diff(c1) from t0 where t0.ts >= '1970-01-01 00:00:00.000' "
)
tdSql.query("select diff(c1) from t0 where t0.ts >= '1970-01-01 00:00:00.000' ")
tdSql.checkRows(679)
tdSql.query(
......@@ -203,9 +220,7 @@ class TDTestCase:
tdSql.checkRows(43200)
# query with timestamp in 'where ...'
tdSql.query(
"select * from stb2 where stb2.ts > -28800000 "
)
tdSql.query("select * from stb2 where stb2.ts > -28800000 ")
tdSql.checkRows(6790)
tdSql.query(
......@@ -219,14 +234,16 @@ class TDTestCase:
tdSql.checkRows(3590)
def run(self):
s = 'reset query cache'
s = "reset query cache"
tdSql.execute(s)
s = 'create database if not exists db'
s = "create database if not exists db"
tdSql.execute(s)
s = 'use db'
s = "use db"
tdSql.execute(s)
tdLog.info("==========step1:create table stable and child table,then insert data automatically")
tdLog.info(
"==========step1:create table stable and child table,then insert data automatically"
)
insertfile = self.createinsertfile()
self.inserttable(insertfile)
......
tests/pytest/query/queryInterval.py
浏览文件 @ 574bfd96
......@@ -197,6 +197,25 @@ class TDTestCase:
else:
tdLog.exit("sql:%s, column : ts is not sorted in accending order as expected" % (tdSql.sql))
# TS-1582
tdSql.execute("create table stb1(ts timestamp, c1 int) tags(t1 int)")
tdSql.execute("insert into tb1 using stb1 tags(1) values(%d, 1)(%d, 5)(%d, 10)" % (self.ts, self.ts + 1000, self.ts + 2000))
tdSql.execute("insert into tb2 using stb1 tags(2) values(%d, 1)(%d, 5)(%d, 10)" % (self.ts, self.ts + 1000, self.ts + 2000))
tdSql.query("select t1, last(c1) from stb1 where ts between 1593548685000 and 1593548688000 interval(2s) fill(NULL) group by tbname")
tdSql.checkRows(6)
tdSql.checkData(2, 2, None)
tdSql.checkData(2, 3, "tb1")
tdSql.checkData(5, 2, None)
tdSql.checkData(5, 3, "tb2")
tdSql.query("select t1, last(c1) - 1 from stb1 where ts between 1593548685000 and 1593548688000 interval(2s) fill(NULL) group by tbname")
tdSql.checkRows(6)
tdSql.checkData(2, 2, None)
tdSql.checkData(2, 3, "tb1")
tdSql.checkData(5, 2, None)
tdSql.checkData(5, 3, "tb2")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
tests/pytest/tools/taosdumpTestBenchmark.py 已删除 100644 → 0
浏览文件 @ 8f5b3f48
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.ts = 1538548685000
self.numberOfTables = 10000
self.numberOfRecords = 100
def checkCommunity(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
return False
else:
return True
def getPath(self, tool="taosdump"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
paths = []
for root, dirs, files in os.walk(projPath):
if ((tool) in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
paths.append(os.path.join(root, tool))
break
if (len(paths) == 0):
return ""
return paths[0]
def insert_data(self, tbname, ts_start, count):
pre_insert = "insert into %s values" % tbname
sql = pre_insert
tdLog.debug("doing insert table %s rows=%d ..." % (tbname, count))
for i in range(count):
sql += " (%d,%d)" % (ts_start + i * 1000, i)
if i > 0 and i % 30000 == 0:
tdSql.execute(sql)
sql = pre_insert
# end sql
if sql != pre_insert:
tdSql.execute(sql)
tdLog.debug("INSERT TABLE DATA ............ [OK]")
return
def run(self):
if not os.path.exists("./taosdumptest"):
os.makedirs("./taosdumptest")
else:
os.system("rm -rf ./taosdumptest")
os.makedirs("./taosdumptest")
for i in range(2):
if not os.path.exists("./taosdumptest/tmp%d" % i):
os.makedirs("./taosdumptest/tmp%d" % i)
else:
os.system("rm -rf ./taosdumptest/tmp%d" % i)
os.makedirs("./taosdumptest/tmp%d" % i)
binPath = self.getPath("taosdump")
if (binPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found: %s" % binPath)
# create db1 , one stables and one table ; create general tables
tdSql.execute("drop database if exists dp1")
tdSql.execute("drop database if exists dp2")
tdSql.execute("create database if not exists dp1")
tdSql.execute("use dp1")
tdSql.execute(
'''create table st0(ts timestamp, c1 tinyint, c2 smallint, c3 int, c4 bigint, c5 float, c6 double,
c7 bool, c8 binary(20), c9 nchar(20), c11 tinyint unsigned, c12 smallint unsigned, c13 int unsigned, c14 bigint unsigned, c15 timestamp )
tags(t1 tinyint, t2 smallint, t3 int, t4 bigint, t5 float, t6 double, t7 bool, t8 binary(20), t9 nchar(20), t11 tinyint unsigned,
t12 smallint unsigned, t13 int unsigned, t14 bigint unsigned, t15 timestamp)''')
tdSql.execute(
'''create table st1(ts timestamp, c1 tinyint, c2 smallint, c3 int, c4 bigint, c5 float, c6 double,
c7 bool, c8 binary(20), c9 nchar(20), c11 tinyint unsigned, c12 smallint unsigned, c13 int unsigned, c14 bigint unsigned, c15 timestamp ) tags(jtag json)''')
intData = []
floatData = []
rowNum = 10
tabNum = 10
ts = 1537146000000
for j in range(tabNum):
tdSql.execute(
"create table st0_%d using st0 tags( %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d);" %
(j,
j +
1,
j +
1,
j +
1,
j +
1,
j +
0.1,
j +
0.1,
j %
2,
j +
1,
j +
1,
j +
1,
j +
1,
j +
1,
j +
1,
ts))
for i in range(rowNum):
tdSql.execute(
"insert into st0_%d values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d)" %
(j, ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i %
2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, ts))
intData.append(i + 1)
floatData.append(i + 0.1)
rowNum = 20
tabNum = 20
for j in range(tabNum):
tdSql.execute(
"create table st1_%d using st1 tags('{\"nv\":null,\"tea\":true,\"\":false,\" \":123%d,\"tea\":false}');" %
(j, j + 1))
for i in range(rowNum):
tdSql.execute(
"insert into st1_%d values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d, %d)" %
(j, self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i %
2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1, self.ts))
intData.append(i + 1)
floatData.append(i + 0.1)
# os.system("%staosBenchmark -f tools/taosdump-insert-dp1.json -y " % benchBinPath)
# create db1 , three stables:stb0,include ctables stb0_0 \ stb0_1,stb1 include ctables stb1_0 and stb1_1
# \stb3,include ctables stb3_0 and stb3_1
# create general three tables gt0 gt1 gt2
tdSql.execute("create database if not exists dp2")
tdSql.execute("use dp2")
tdSql.execute(
"create stable st0(ts timestamp, c01 int, c02 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st0_0 using st0 tags(0) st0_1 using st0 tags(1) ")
tdSql.execute(
"insert into st0_0 values(1614218412000,8600,'R')(1614218422000,8600,'E')")
tdSql.execute(
"insert into st0_1 values(1614218413000,8601,'A')(1614218423000,8601,'D')")
tdSql.execute(
"create stable st1(ts timestamp, c11 float, c12 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st1_0 using st1 tags(0) st1_1 using st1 tags(1) ")
tdSql.execute(
"insert into st1_0 values(1614218412000,8610.1,'R')(1614218422000,8610.1,'E')")
tdSql.execute(
"insert into st1_1 values(1614218413000,8611.2,'A')(1614218423000,8611.1,'D')")
tdSql.execute(
"create stable st2(ts timestamp, c21 float, c22 nchar(10)) tags(t1 int)")
tdSql.execute(
"create table st20 using st2 tags(0) st21 using st2 tags(1) ")
tdSql.execute(
"insert into st20 values(1614218412000,8620.3,'R')(1614218422000,8620.3,'E')")
tdSql.execute(
"insert into st21 values(1614218413000,8621.4,'A')(1614218423000,8621.4,'D')")
tdSql.execute(
"create table if not exists gt0 (ts timestamp, c00 int, c01 float) ")
tdSql.execute(
"create table if not exists gt1 (ts timestamp, c10 int, c11 double) ")
tdSql.execute(
"create table if not exists gt2 (ts timestamp, c20 int, c21 float) ")
tdSql.execute("insert into gt0 values(1614218412700,8637,78.86155)")
tdSql.execute(
"insert into gt1 values(1614218413800,8638,78.862020199)")
tdSql.execute("insert into gt2 values(1614218413900,8639,78.863)")
# self.insert_data("t", self.ts, 300*10000);
# os.system("%staosBenchmark -f tools/taosdump-insert-dp2.json -y " % benchBinPath)
# # taosdump data
# os.system("%s -o ./taosdumptest/tmp1 taosdump -h -ptaosdata -P 6030 -u root -o taosdumptest \
# -D dp1,dp3 -N -c /home/chr/TDinternal/community/sim/dnode1/cfg/taos.cfg -s -d deflate" % binPath)
os.system(
"%s -o ./taosdumptest/tmp0 -D dp2,dp1 -T 8" %
binPath)
os.system(
"%s -o ./taosdumptest/tmp1 dp2 st0 st1_0 gt0 -T 8" %
binPath)
# check taosdumptest/tmp0
tdSql.execute("drop database dp1")
tdSql.execute("drop database dp2")
os.system("%s -i ./taosdumptest/tmp0 -T 8 " % binPath)
tdSql.execute("reset query cache")
tdSql.execute("use dp1")
tdSql.query("show stables")
tdSql.checkRows(3)
for i in range(3):
for j in range(3):
if j < 2:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, (j + 1) * 10)
else:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, 100002)
tdSql.query("select count(*) from st0")
tdSql.checkData(0, 0, 100)
tdSql.query("select count(*) from st1")
tdSql.checkData(0, 0, 400)
tdSql.query("select count(*) from st2")
tdSql.checkData(0, 0, 1000020)
tdSql.execute("use dp2")
tdSql.query("show stables")
tdSql.checkRows(3)
for i in range(3):
for j in range(3):
if j < 2:
if tdSql.queryResult[i][0] == 'st%d' % j:
# print(i,"stb%d"%j)
tdSql.checkData(i, 4, 2)
else:
if tdSql.queryResult[i][0] == 'st%d' % j:
tdSql.checkData(i, 4, 100002)
tdSql.query("select count(*) from st0")
tdSql.checkData(0, 0, 4)
tdSql.query("select count(*) from st1")
tdSql.checkData(0, 0, 4)
tdSql.query("select count(*) from st2")
tdSql.checkData(0, 0, 1000024)
tdSql.query("select ts from gt0")
tdSql.checkData(0, 0, '2021-02-25 10:00:12.700')
tdSql.query("select c10 from gt1")
tdSql.checkData(0, 0, 8638)
tdSql.query("select c20 from gt2")
tdSql.checkData(0, 0, 8639)
# check taosdumptest/tmp1
tdSql.execute("drop database dp1")
tdSql.execute("drop database dp2")
os.system("%s -i ./taosdumptest/tmp1 -T 8 " % binPath)
tdSql.execute("reset query cache")
tdSql.execute("use dp2")
tdSql.query("show stables")
tdSql.checkRows(2)
tdSql.query("show tables")
tdSql.checkRows(4)
tdSql.query("select count(*) from st1_0")
tdSql.checkData(0, 0, 2)
tdSql.query("select ts from gt0")
tdSql.checkData(0, 0, '2021-02-25 10:00:12.700')
tdSql.error("use dp1")
tdSql.error("select count(*) from st2_0")
tdSql.error("select count(*) from gt2")
# #check taosdumptest/tmp2
# tdSql.execute("drop database dp1")
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp2 -T 8 " % binPath)
# tdSql.execute("use dp1")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(3)
# tdSql.query("select c1 from st0_0 order by ts")
# tdSql.checkData(0,0,8537)
# tdSql.query("select c2 from st0_1 order by ts")
# tdSql.checkData(1,0,"D")
# tdSql.query("select * from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000')
# tdSql.checkData(0,1,637)
# tdSql.error("select count(*) from gt1")
# tdSql.error("use dp2")
# #check taosdumptest/tmp3
# tdSql.execute("drop database dp1")
# os.system("%s -i ./taosdumptest/tmp3 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(2)
# tdSql.query("show tables")
# tdSql.checkRows(4)
# tdSql.query("select count(*) from st1_0")
# tdSql.checkData(0,0,2)
# tdSql.query("select ts from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.700')
# tdSql.error("use dp1")
# tdSql.error("select count(*) from st2_0")
# tdSql.error("select count(*) from gt2")
# #check taosdumptest/tmp4
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp4 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(2)
# tdSql.query("show tables")
# tdSql.checkRows(6)
# tdSql.query("select c20 from gt2")
# tdSql.checkData(0, 0, 8639)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st0_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_0")
# tdSql.checkData(0, 0, 2)
# tdSql.error("use dp1")
# tdSql.error("select count(*) from st1_0")
# tdSql.error("select count(*) from st1_1")
# tdSql.error("select count(*) from gt3")
# #check taosdumptest/tmp5
# tdSql.execute("drop database dp2")
# os.system("%s -i ./taosdumptest/tmp5 -T 8 " % binPath)
# tdSql.execute("use dp2")
# tdSql.query("show stables")
# tdSql.checkRows(3)
# tdSql.query("show tables")
# tdSql.checkRows(9)
# tdSql.query("select c20 from gt2")
# tdSql.checkData(0, 0, 8639)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st0_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st2_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st1_1")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select count(*) from st1_0")
# tdSql.checkData(0, 0, 2)
# tdSql.execute("use dp1")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(4)
# tdSql.query("select c1 from st0_0 order by ts")
# tdSql.checkData(0,0,8537)
# tdSql.query("select c2 from st0_1 order by ts")
# tdSql.checkData(1,0,"D")
# tdSql.query("select * from gt0")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000')
# tdSql.checkData(0,1,637)
# # check taosdumptest/tmp6
# tdSql.execute("drop database dp1")
# tdSql.execute("drop database dp2")
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp6 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show databases")
# tdSql.checkRows(1)
# tdSql.checkData(0,16,'ns')
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkData(0, 0, 2)
# tdSql.query("select * from st0 order by ts")
# tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# tdSql.checkData(0,1,8600)
# # check taosdumptest/tmp7
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp7 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show databases")
# tdSql.checkRows(1)
# tdSql.checkData(0,16,'ms')
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkRows(0)
# # tdSql.query("select * from st0 order by ts")
# # tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# # tdSql.checkData(0,1,8600)
# # check taosdumptest/tmp8
# tdSql.execute("drop database dp3")
# os.system("%s -i ./taosdumptest/tmp8 -T 8 " % binPath)
# tdSql.execute("use dp3")
# tdSql.query("show stables")
# tdSql.checkRows(1)
# tdSql.query("show tables")
# tdSql.checkRows(1)
# tdSql.query("select count(*) from st0_0")
# tdSql.checkRows(0)
# # tdSql.query("select * from st0 order by ts")
# # tdSql.checkData(0,0,'2021-02-25 10:00:12.000000001')
# # tdSql.checkData(0,1,8600)
# os.system("rm -rf ./taosdumptest/tmp1")
# os.system("rm -rf ./taosdumptest/tmp2")
# os.system("rm -rf ./taosdumptest/tmp3")
# os.system("rm -rf ./taosdumptest/tmp4")
# os.system("rm -rf ./taosdumptest/tmp5")
# os.system("rm -rf ./dump_result.txt")
# os.system("rm -rf ./db.csv")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
tests/script/general/db/alter_option.sim
浏览文件 @ 574bfd96
......@@ -9,7 +9,7 @@ sleep 2000
sql connect
print ============= create database
sql create database db cache 2 blocks 4 days 10 keep 20 minRows 300 maxRows 400 ctime 120 precision 'ms' comp 2 wal 1 replica 1
sql create database db cache 2 blocks 4 days 10 keep 20 minRows 300 maxRows 1200 ctime 120 precision 'ms' comp 2 wal 1 replica 1
sql show databases
if $data00 != db then
return -1
......@@ -182,9 +182,13 @@ sql_error alter database db blocks -1
sql_error alter database db blocks 10001
print ============== step minrows
sql_error alter database db minrows 0
sql_error alter database db minrows 1
sql_error alter database db minrows 100
sql_error alter database db minrows 1000
sql_error alter database db minrows -1
sql_error alter database db minrows 9
sql_error alter database db minrows 1001
sql alter database db minrows 100
sql alter database db minrows 1000
print ============== step maxrows
sql_error alter database db maxrows 1
......
tests/script/general/db/topic1.sim
浏览文件 @ 574bfd96
......@@ -471,8 +471,9 @@ sql_error alter topic db blocks 10001
print ============== step minrows
sql_error alter database db minrows 1
sql_error alter database db minrows 100
sql_error alter database db minrows 1000
sql alter database db minrows 100
sql alter database db minrows 399
sql_error alter database db minrows 1001
sql_error alter topic db minrows 1
sql_error alter topic db minrows 100
......
tests/system-test/5-taos-tools/TD-12478.py
浏览文件 @ 574bfd96
......@@ -28,13 +28,13 @@ from itertools import combinations
from faker import Faker
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<xyguo>[TD-12434]:taosdump null nchar/binary length can cause core:taos-tools/src/taosdump.c
case2<xyguo>[TD-12478]:taos_stmt_execute() failed! reason: WAL size exceeds limit
'''
return
"""
case1<xyguo>[TD-12434]:taosdump null nchar/binary length can cause core:taos-tools/src/taosdump.c
case2<xyguo>[TD-12478]:taos_stmt_execute() failed! reason: WAL size exceeds limit
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
......@@ -43,104 +43,184 @@ class TDTestCase:
os.system("rm -rf 5-taos-tools/TD-12478.py.sql")
os.system("rm db*")
os.system("rm dump_result.txt*")
os.system("rm -rf taosdump.*")
def restartDnodes(self):
tdDnodes.stop(1)
tdDnodes.start(1)
def dropandcreateDB_random(self,n):
def dropandcreateDB_random(self, n):
self.ts = 1630000000000
fake = Faker('zh_CN')
fake = Faker("zh_CN")
self.num_random = fake.random_int(min=1000, max=5000, step=1)
print(self.num_random)
for i in range(n):
tdSql.execute('''drop database if exists db ;''')
tdSql.execute('''create database db keep 36500;''')
tdSql.execute('''use db;''')
tdSql.execute("""drop database if exists db ;""")
tdSql.execute("""create database db keep 36500;""")
tdSql.execute("""use db;""")
tdSql.execute('''create stable stable_1 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
tdSql.execute(
"""create stable stable_1 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);''')
tdSql.execute('''create stable stable_2 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);"""
)
tdSql.execute(
"""create stable stable_2 (ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);''')
tdSql.execute('''create table table_1 using stable_1 tags('table_1', '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')''')
tdSql.execute('''create table table_2 using stable_1 tags('table_2', '2147483647' , '9223372036854775807' , '32767' , '127' , 1 , 'binary2' , 'nchar2' , '2' , '22' , \'1999-09-09 09:09:09.090\')''')
tdSql.execute('''create table table_3 using stable_1 tags('table_3', '-2147483647' , '-9223372036854775807' , '-32767' , '-127' , false , 'binary3' , 'nchar3nchar3' , '-3.3' , '-33.33' , \'2099-09-09 09:09:09.090\')''')
tdSql.execute('''create table table_21 using stable_2 tags('table_21' , '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')''')
#regular table
tdSql.execute('''create table regular_table_1 \
tags(loc nchar(100) , t_int int , t_bigint bigint , t_smallint smallint , t_tinyint tinyint, t_bool bool , t_binary binary(100) , t_nchar nchar(100) ,t_float float , t_double double , t_ts timestamp);"""
)
tdSql.execute(
"""create table table_1 using stable_1 tags('table_1', '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')"""
)
tdSql.execute(
"""create table table_2 using stable_1 tags('table_2', '2147483647' , '9223372036854775807' , '32767' , '127' , 1 , 'binary2' , 'nchar2' , '2' , '22' , \'1999-09-09 09:09:09.090\')"""
)
tdSql.execute(
"""create table table_3 using stable_1 tags('table_3', '-2147483647' , '-9223372036854775807' , '-32767' , '-127' , false , 'binary3' , 'nchar3nchar3' , '-3.3' , '-33.33' , \'2099-09-09 09:09:09.090\')"""
)
tdSql.execute(
"""create table table_21 using stable_2 tags('table_21' , '0' , '0' , '0' , '0' , 0 , '0' , '0' , '0' , '0' ,'0')"""
)
# regular table
tdSql.execute(
"""create table regular_table_1 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
tdSql.execute('''create table regular_table_2 \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
tdSql.execute(
"""create table regular_table_2 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
tdSql.execute('''create table regular_table_3 \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
tdSql.execute(
"""create table regular_table_3 \
(ts timestamp , q_int int , q_bigint bigint , q_smallint smallint , q_tinyint tinyint , q_float float , q_double double , q_bool bool , q_binary binary(100) , q_nchar nchar(100) , q_ts timestamp , \
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;''')
for i in range(self.num_random):
tdSql.execute('''insert into table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double , q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(min=-9223372036854775807, max=9223372036854775807, step=1),
fake.random_int(min=-32767, max=32767, step=1) , fake.random_int(min=-127, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=2147483647, step=1) ,
fake.random_int(min=-9223372036854775807, max=9223372036854775807, step=1) ,
fake.random_int(min=-32767, max=32767, step=1) , fake.random_int(min=-127, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1) , fake.random_int(min=0, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1) , fake.random_int(min=0, max=127, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1) , fake.random_int(min=-127, max=0, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
tdSql.execute('''insert into regular_table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)'''
% (self.ts + i*1000, fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1) , fake.random_int(min=-127, max=0, step=1) ,
fake.pyfloat() , fake.pyfloat() , fake.pystr() , fake.address() , self.ts + i))
q_int_null int , q_bigint_null bigint , q_smallint_null smallint , q_tinyint_null tinyint, q_float_null float , q_double_null double , q_bool_null bool , q_binary_null binary(20) , q_nchar_null nchar(20) , q_ts_null timestamp) ;"""
)
for i in range(self.num_random):
tdSql.execute(
"""insert into table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double , q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(
min=-9223372036854775807, max=9223372036854775807, step=1
),
fake.random_int(min=-32767, max=32767, step=1),
fake.random_int(min=-127, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_1 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=2147483647, step=1),
fake.random_int(
min=-9223372036854775807, max=9223372036854775807, step=1
),
fake.random_int(min=-32767, max=32767, step=1),
fake.random_int(min=-127, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1),
fake.random_int(min=0, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_2 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=0, max=2147483647, step=1),
fake.random_int(min=0, max=9223372036854775807, step=1),
fake.random_int(min=0, max=32767, step=1),
fake.random_int(min=0, max=127, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 0, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1),
fake.random_int(min=-127, max=0, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.execute(
"""insert into regular_table_3 (ts , q_int , q_bigint , q_smallint , q_tinyint , q_float , q_double, q_bool , q_binary , q_nchar, q_ts) values(%d, %d, %d, %d, %d, %f, %f, 1, 'binary.%s', 'nchar.%s', %d)"""
% (
self.ts + i * 1000,
fake.random_int(min=-2147483647, max=0, step=1),
fake.random_int(min=-9223372036854775807, max=0, step=1),
fake.random_int(min=-32767, max=0, step=1),
fake.random_int(min=-127, max=0, step=1),
fake.pyfloat(),
fake.pyfloat(),
fake.pystr(),
fake.address(),
self.ts + i,
)
)
tdSql.query("select count(*) from stable_1;")
tdSql.checkData(0,0,3*self.num_random)
tdSql.checkData(0, 0, 3 * self.num_random)
tdSql.query("select count(*) from regular_table_1;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
def run(self):
tdSql.prepare()
dcDB = self.dropandcreateDB_random(1)
assert os.system("taosdump -D db") == 0
assert os.system("taosdump -i . -g") == 0
tdSql.query("select count(*) from stable_1;")
tdSql.checkData(0,0,3*self.num_random)
tdSql.checkData(0, 0, 3 * self.num_random)
tdSql.query("select count(*) from regular_table_1;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
tdSql.query("select count(*) from regular_table_2;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
tdSql.query("select count(*) from regular_table_3;")
tdSql.checkData(0,0,self.num_random)
tdSql.checkData(0, 0, self.num_random)
def stop(self):
tdSql.close()
......@@ -148,4 +228,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
tests/system-test/5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson.py
浏览文件 @ 574bfd96
......@@ -24,36 +24,44 @@ class TDTestCase:
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def getBuildPath(self):
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
elif "/tests/" in selfPath:
projPath = selfPath[: selfPath.find("/tests/")]
else:
projPath = selfPath[:selfPath.find("tests")]
tdLog.exit("path: %s is not supported" % selfPath)
paths = []
for root, dirs, files in os.walk(projPath):
if ("taosd" in files):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root)-len("/build/bin")]
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
return buildPath
if len(paths) == 0:
return ""
return paths[0]
def run(self):
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosd not found!")
binPath = self.getPath()
if binPath == "":
tdLog.exit("taosBenchmark not found!")
else:
tdLog.info("taosd found in %s" % buildPath)
binPath = buildPath+ "/build/bin/"
tdLog.info("taosBenchmark found in %s" % binPath)
testcaseFilename = os.path.split(__file__)[-1]
os.system("rm -rf ./insert*_res.txt*")
os.system("rm -rf 5-taos-tools/taosbenchmark/%s.sql" % testcaseFilename )
os.system("rm -rf 5-taos-tools/taosbenchmark/%s.sql" % testcaseFilename)
# insert: create one or mutiple tables per sql and insert multiple rows per sql
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-1s1tnt1r.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insert-1s1tnt1r.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 11)
......@@ -69,7 +77,7 @@ class TDTestCase:
tdSql.checkData(0, 0, 2000)
# restful connector insert data
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insertRestful.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insertRestful.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 10)
......@@ -84,19 +92,19 @@ class TDTestCase:
tdSql.query("select count(*) from stb1")
tdSql.checkData(0, 0, 200)
# default values json files
tdSql.execute("drop database if exists db")
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-default.json -y " % binPath)
# default values json files
tdSql.execute("drop database if exists db")
os.system("%s -f 5-taos-tools/taosbenchmark/insert-default.json -y " % binPath)
tdSql.query("show databases;")
for i in range(tdSql.queryRows):
if tdSql.queryResult[i][0] == 'db':
tdSql.checkData(i, 2, 100)
tdSql.checkData(i, 4, 1)
tdSql.checkData(i, 6, 10)
tdSql.checkData(i, 16, 'ms')
if tdSql.queryResult[i][0] == "db":
tdSql.checkData(i, 2, 100)
tdSql.checkData(i, 4, 1)
tdSql.checkData(i, 6, 10)
tdSql.checkData(i, 16, "ms")
# insert: create mutiple tables per sql and insert one rows per sql .
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-1s1tntmr.json -y " % binPath)
os.system("%s -f 5-taos-tools/taosbenchmark/insert-1s1tntmr.json -y " % binPath)
tdSql.execute("use db")
tdSql.query("select count (tbname) from stb0")
tdSql.checkData(0, 0, 10)
......@@ -113,7 +121,9 @@ class TDTestCase:
# insert: using parament "insert_interval to controls spped of insert.
# but We need to have accurate methods to control the speed, such as getting the speed value, checking the count and so on。
os.system("%staosBenchmark -f 5-taos-tools/taosbenchmark/insert-interval-speed.json -y" % binPath)
os.system(
"%s -f 5-taos-tools/taosbenchmark/insert-interval-speed.json -y" % binPath
)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkData(0, 4, 10)
......@@ -131,11 +141,6 @@ class TDTestCase:
# rm useless files
os.system("rm -rf ./insert*_res.txt*")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或