提交 9f5cf8e7 编写于 作者: S shenglian zhou

:erge branch '2.6' of github.com:taosdata/TDengine into szhou/fix-ts2851

......@@ -39,7 +39,7 @@ IF (TD_LINUX_64 AND JEMALLOC_ENABLED)
PREFIX "jemalloc"
SOURCE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/jemalloc
BUILD_IN_SOURCE 1
CONFIGURE_COMMAND ./autogen.sh COMMAND ./configure --prefix=${CMAKE_BINARY_DIR}/build/
CONFIGURE_COMMAND ./autogen.sh COMMAND ./configure --prefix=${CMAKE_BINARY_DIR}/build/ --disable-initial-exec-tls
BUILD_COMMAND ${MAKE}
)
INCLUDE_DIRECTORIES(${CMAKE_BINARY_DIR}/build/include)
......
......@@ -105,12 +105,11 @@ As a high-performance, scalable and SQL supported time-series database, TDengine
## Comparison with other databases
- [Writing Performance Comparison of TDengine and InfluxDB ](https://tdengine.com/2022/02/23/4975.html)
- [Query Performance Comparison of TDengine and InfluxDB](https://tdengine.com/2022/02/24/5120.html)
- [TDengine vs InfluxDB、OpenTSDB、Cassandra、MySQL、ClickHouse](https://www.tdengine.com/downloads/TDengine_Testing_Report_en.pdf)
- [TDengine vs OpenTSDB](https://tdengine.com/2019/09/12/710.html)
- [TDengine vs Cassandra](https://tdengine.com/2019/09/12/708.html)
- [TDengine vs InfluxDB](https://tdengine.com/2019/09/12/706.html)
- [Writing Performance Comparison of TDengine and InfluxDB ](https://tdengine.com/performance-comparison-of-tdengine-and-influxdb/)
- [Query Performance Comparison of TDengine and InfluxDB](https://tdengine.com/query-performance-comparison-test-report-tdengine-vs-influxdb/)
- [TDengine vs OpenTSDB](https://tdengine.com/performance-tdengine-vs-opentsdb/)
- [TDengine vs Cassandra](https://tdengine.com/performance-tdengine-vs-cassandra/)
- [TDengine vs InfluxDB](https://tdengine.com/performance-tdengine-vs-influxdb/)
If you want to learn some basics about time-series databases, please check [here](https://tdengine.com/tsdb).
If you want to learn some basics about time-series databases, please check [here](https://tdengine.com/tsdb/).
......@@ -26,7 +26,7 @@ In TDengine, the data types below can be used when specifying a column or tag.
| 3 | BIGINT | 8 | Long integer, the value range is [-2^63+1, 2^63-1], while -2^63 is treated as NULL |
| 4 | FLOAT | 4 | Floating point number, the effective number of digits is 6-7, the value range is [-3.4E38, 3.4E38] |
| 5 | DOUBLE | 8 | Double precision floating point number, the effective number of digits is 15-16, the value range is [-1.7E308, 1.7E308] |
| 6 | BINARY | User Defined | Single-byte string for ASCII visible characters. Length must be specified when defining a column or tag of binary type. The string length can be up to 16374 bytes. The string value must be quoted with single quotes. The literal single quote inside the string must be preceded with back slash like `\'` |
| 6 | BINARY | User Defined | Single-byte string for ASCII visible characters. Length must be specified when defining a column or tag of binary type. The string length can be up to 16374 bytes (65517 bytes since version 2.6.0.34). The string value must be quoted with single quotes. The literal single quote inside the string must be preceded with back slash like `\'` |
| 7 | SMALLINT | 2 | Short integer, the value range is [-32767, 32767], while -32768 is treated as NULL |
| 8 | TINYINT | 1 | Single-byte integer, the value range is [-127, 127], while -128 is treated as NULL |
| 9 | BOOL | 1 | Bool, the value range is {true, false} |
......
......@@ -14,7 +14,7 @@ CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_nam
1. The first column of a table MUST be of type TIMESTAMP. It is automatically set as the primary key.
2. The maximum length of the table name is 192 bytes.
3. The maximum length of each row is 48k bytes, please note that the extra 2 bytes used by each BINARY/NCHAR column are also counted.
3. The maximum length of each row is 48k bytes (64k bytes since version 2.6.0.34), please note that the extra 2 bytes used by each BINARY/NCHAR column are also counted.
4. The name of the subtable can only consist of characters from the English alphabet, digits and underscore. Table names can't start with a digit. Table names are case insensitive.
5. The maximum length in bytes must be specified when using BINARY or NCHAR types.
6. Escape character "\`" can be used to avoid the conflict between table names and reserved keywords, above rules will be bypassed when using escape character on table names, but the upper limit for the name length is still valid. The table names specified using escape character are case sensitive. Only ASCII visible characters can be used with escape character.
......
......@@ -569,18 +569,23 @@ Query OK, 2 row(s) in set (0.000793s)
### PERCENTILE
```
SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
SELECT PERCENTILE(field_name, P [, P1] ...) FROM { tb_name } [WHERE clause];
```
**Description**: The value whose rank in a specific column matches the specified percentage. If such a value matching the specified percentage doesn't exist in the column, an interpolation value will be returned.
**Return value type**: Double precision floating point
**Return value type**: This function takes 2 minumum and 11 maximum parameters, and it can simultaneously return 10 percentiles at most. If 2 parameters are given, a single percentile is returned and the value type is DOUBLE.
If more than 2 parameters are given, the return value type is a VARCHAR string, the format of which is a JSON ARRAY containing all return values.
**Applicable column types**: Data types except for timestamp, binary, nchar and bool
**Applicable table types**: table
**More explanations**: _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
**More explanations**:
- _P_ is in range [0,100], when _P_ is 0, the result is same as using function MIN; when _P_ is 100, the result is same as function MAX.
- When calculating multiple percentiles of a specific column, a single PERCENTILE function with multiple parameters is adviced, as this can largely reduce the query response time.
For example, using SELECT percentile(col, 90, 95, 99) FROM table will perform better than SELECT percentile(col, 90), percentile(col, 95), percentile(col, 99) from table.
**Examples**:
......
......@@ -33,7 +33,7 @@ We recommend using the latest version of `taospy`, regardless of the version of
### Preparation
1. Install Python. Python >= 3.6 is recommended. If Python is not available on your system, refer to the [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) to install it.
1. Install Python. Python >= 3.6.2 is recommended. If Python is not available on your system, refer to the [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) to install it.
2. Install [pip](https://pypi.org/project/pip/). In most cases, the Python installer comes with the pip utility. If not, please refer to [pip documentation](https://pip.pypa.io/en/stable/installation/) to install it.
If you use a native connection, you will also need to [Install Client Driver](/reference/connector#Install-Client-Driver). The client install package includes the TDengine client dynamic link library (`libtaos.so` or `taos.dll`) and the TDengine CLI.
......
......@@ -186,6 +186,16 @@ TDengine uses 13 continuous ports, both TCP and UDP, starting with the port spec
| Default Value | 100,000 |
| Note | After version 2.3.0.0 |
### maxSqlGroups
| 属性 | 说明 |
| -------- | -------------------------------- | --- |
| 适用范围 | Server and Client |
| 含义 | The maximus number of groups when group by or interval |
| 取值范围 | [500,000 - 10,000,000] |
| 缺省值 | 1,000,000 |
| 补充说明 | After version 2.6.0.21 | |
## Locale Parameters
### timezone
......
......@@ -76,7 +76,7 @@ Development: false
### Install from source code
```
git clone https://github.com:taosdata/kafka-connect-tdengine.git
git clone --branch master https://github.com:taosdata/kafka-connect-tdengine.git
cd kafka-connect-tdengine
mvn clean package
unzip -d $CONFLUENT_HOME/share/java/ target/components/packages/taosdata-kafka-connect-tdengine-*.zip
......
......@@ -72,7 +72,7 @@ sudo systemctl start telegraf
Log in to the Grafana interface using a web browser at `IP:3000`, with the system's initial username and password being `admin/admin`.
Click on the gear icon on the left and select `Plugins`, you should find the TDengine data source plugin icon.
Click on the plus icon on the left and select `Import` to get the data from `https://github.com/taosdata/grafanaplugin/blob/master/examples/telegraf/grafana/dashboards/telegraf-dashboard-v0.1.0.json`, download the dashboard JSON file and import it. You will then see the dashboard in the following screen.
Click on the plus icon on the left and select `Import` to get the data from `https://github.com/taosdata/grafanaplugin/blob/master/examples/telegraf/grafana/dashboards/telegraf-dashboard-v2.json`, download the dashboard JSON file and import it. You will then see the dashboard in the following screen.
![TDengine Database IT-DevOps-Solutions-telegraf-dashboard](./IT-DevOps-Solutions-telegraf-dashboard.webp)
......
......@@ -27,7 +27,7 @@ CREATE DATABASE db_name PRECISION 'ns';
| 3 | BIGINT | 8 | 长整型,范围 [-2^63+1, 2^63-1], -2^63 用作 NULL |
| 4 | FLOAT | 4 | 浮点型,有效位数 6-7,范围 [-3.4E38, 3.4E38] |
| 5 | DOUBLE | 8 | 双精度浮点型,有效位数 15-16,范围 [-1.7E308, 1.7E308] |
| 6 | BINARY | 自定义 | 记录单字节字符串,建议只用于处理 ASCII 可见字符,中文等多字节字符需使用 nchar。理论上,最长可以有 16374 字节。binary 仅支持字符串输入,字符串两端需使用单引号引用。使用时须指定大小,如 binary(20) 定义了最长为 20 个单字节字符的字符串,每个字符占 1 byte 的存储空间,总共固定占用 20 bytes 的空间,此时如果用户字符串超出 20 字节将会报错。对于字符串内的单引号,可以用转义字符反斜线加单引号来表示,即 `\’`。 |
| 6 | BINARY | 自定义 | 记录单字节字符串,建议只用于处理 ASCII 可见字符,中文等多字节字符需使用 nchar。理论上,最长可以有 16374 字节(从 2.6.0.34 版本开始最长支持 65517 字节)。binary 仅支持字符串输入,字符串两端需使用单引号引用。使用时须指定大小,如 binary(20) 定义了最长为 20 个单字节字符的字符串,每个字符占 1 byte 的存储空间,总共固定占用 20 bytes 的空间,此时如果用户字符串超出 20 字节将会报错。对于字符串内的单引号,可以用转义字符反斜线加单引号来表示,即 `\’`。 |
| 7 | SMALLINT | 2 | 短整型, 范围 [-32767, 32767], -32768 用作 NULL |
| 8 | TINYINT | 1 | 单字节整型,范围 [-127, 127], -128 用作 NULL |
| 9 | BOOL | 1 | 布尔型,{true, false} |
......
......@@ -12,7 +12,7 @@ CREATE TABLE [IF NOT EXISTS] tb_name (timestamp_field_name TIMESTAMP, field1_nam
1. 表的第一个字段必须是 TIMESTAMP,并且系统自动将其设为主键;
2. 表名最大长度为 192;
3. 表的每行长度不能超过 48KB;(注意:每个 BINARY/NCHAR 类型的列还会额外占用 2 个字节的存储位置)
3. 表的每行长度不能超过 48KB;(注意:每个 BINARY/NCHAR 类型的列还会额外占用 2 个字节的存储位置。从 2.6.0.34 版本开始支持 64KB
4. 子表名只能由字母、数字和下划线组成,且不能以数字开头,不区分大小写
5. 使用数据类型 binary 或 nchar,需指定其最长的字节数,如 binary(20),表示 20 字节;
6. 为了兼容支持更多形式的表名,TDengine 引入新的转义符 "\`",可以让表名与关键词不冲突,同时不受限于上述表名称合法性约束检查。但是同样具有长度限制要求。使用转义字符以后,不再对转义字符中的内容进行大小写统一。
......
......@@ -564,18 +564,22 @@ Query OK, 2 row(s) in set (0.000793s)
### PERCENTILE
```
SELECT PERCENTILE(field_name, P) FROM { tb_name } [WHERE clause];
SELECT PERCENTILE(field_name, P [, P1] ...) FROM { tb_name } [WHERE clause];
```
**功能说明**:统计表中某列的值百分比分位数。
**返回数据类型** 双精度浮点数 Double
**返回数据类型**该函数最小参数个数为 2 个,最大参数个数为 11 个。可以最多同时返回 10 个百分比分位数。当参数个数为 2 时, 返回一个分位数,类型为DOUBLE,当参数个数大于 2 时,返回类型为VARCHAR, 格式为包含多个返回值的JSON数组
**应用字段**:不能应用在 timestamp、binary、nchar、bool 类型字段。
**适用于**:表。
**使用说明***P*值取值范围 0≤*P*≤100,为 0 的时候等同于 MIN,为 100 的时候等同于 MAX。
**使用说明**
- *P*值取值范围 0≤*P*≤100,为 0 的时候等同于 MIN,为 100 的时候等同于 MAX。
- 同时计算针对同一列的多个分位数时,建议使用一个PERCENTILE函数和多个参数的方式,能很大程度上降低查询的响应时间。
比如,使用查询SELECT percentile(col, 90, 95, 99) FROM table, 性能会优于SELECT percentile(col, 90), percentile(col, 95), percentile(col, 99) from table。
**示例**
......
......@@ -33,7 +33,7 @@ Python 连接器的源码托管在 [GitHub](https://github.com/taosdata/taos-con
### 准备
1. 安装 Python。建议使用 Python >= 3.6。如果系统上还没有 Python 可参考 [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) 安装。
1. 安装 Python。建议使用 Python >= 3.6.2。如果系统上还没有 Python 可参考 [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) 安装。
2. 安装 [pip](https://pypi.org/project/pip/)。大部分情况下 Python 的安装包都自带了 pip 工具, 如果没有请参考 [pip docuemntation](https://pip.pypa.io/en/stable/installation/) 安装。
3. 如果使用原生连接,还需[安装客户端驱动](../#安装客户端驱动)。客户端软件包含了 TDengine 客户端动态链接库(libtaos.so 或 taos.dll) 和 TDengine CLI。
......
......@@ -186,6 +186,16 @@ taos --dump-config
| 缺省值 | 10 万 |
| 补充说明 | 2.3 版本新增。 | |
### maxSqlGroups
| 属性 | 说明 |
| -------- | -------------------------------- | --- |
| 适用范围 | 服务端和客户端均适用 |
| 含义 | GROUP BY最大支持的分组数或者INTERVAL最大的时间窗口数 |
| 取值范围 | 最小值为 50 万,最大值 1000 万 |
| 缺省值 | 100 万 |
| 补充说明 | 2.6 版本新增。 | |
## 区域相关
### timezone
......
......@@ -78,7 +78,7 @@ Development: false
### 从源码安装
```
git clone https://github.com:taosdata/kafka-connect-tdengine.git
git clone --branch master https://github.com:taosdata/kafka-connect-tdengine.git
cd kafka-connect-tdengine
mvn clean package
unzip -d $CONFLUENT_HOME/share/java/ target/components/packages/taosdata-kafka-connect-tdengine-*.zip
......
......@@ -72,7 +72,7 @@ sudo systemctl start telegraf
使用 Web 浏览器访问 `IP:3000` 登录 Grafana 界面,系统初始用户名密码为 admin/admin。
点击左侧齿轮图标并选择 `Plugins`,应该可以找到 TDengine data source 插件图标。
点击左侧加号图标并选择 `Import`,从 `https://github.com/taosdata/grafanaplugin/blob/master/examples/telegraf/grafana/dashboards/telegraf-dashboard-v0.1.0.json` 下载 dashboard JSON 文件后导入。之后可以看到如下界面的仪表盘:
点击左侧加号图标并选择 `Import`,从 `https://github.com/taosdata/grafanaplugin/blob/master/examples/telegraf/grafana/dashboards/telegraf-dashboard-v2.json` 下载 dashboard JSON 文件后导入。之后可以看到如下界面的仪表盘:
![TDengine Database IT-DevOps-Solutions-telegraf-dashboard](./IT-DevOps-Solutions-telegraf-dashboard.webp)
......
......@@ -267,7 +267,7 @@ if [ "$osType" != "Darwin" ]; then
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-deb.sh ${top_dir} \
${compile_dir} ${output_dir} ${taos_tools_ver} ${cpuType} ${osType} ${verMode} ${verType}
${compile_dir} ${output_dir} ${taos_tools_ver} ${cpuType} ${osType} ${verMode} ${verType} ${verNumberComp}
fi
fi
else
......@@ -292,7 +292,7 @@ if [ "$osType" != "Darwin" ]; then
[ -z "$taos_tools_ver" ] && taos_tools_ver="0.1.0"
${csudo}./make-taos-tools-rpm.sh ${top_dir} \
${compile_dir} ${output_dir} ${taos_tools_ver} ${cpuType} ${osType} ${verMode} ${verType}
${compile_dir} ${output_dir} ${taos_tools_ver} ${cpuType} ${osType} ${verMode} ${verType} ${verNumberComp}
fi
fi
else
......
......@@ -211,8 +211,8 @@ function install_bin() {
[ -x ${install_main_dir}/bin/${clientName} ] && ${csudo}ln -s ${install_main_dir}/bin/${clientName} ${bin_link_dir}/${clientName} || :
[ -x ${install_main_dir}/bin/${serverName} ] && ${csudo}ln -s ${install_main_dir}/bin/${serverName} ${bin_link_dir}/${serverName} || :
[ -x ${install_main_dir}/bin/${adapterName} ] && ${csudo}ln -s ${install_main_dir}/bin/${adapterName} ${bin_link_dir}/${adapterName} || :
[ -x ${install_main_dir}/bin/${benchmarkName} ] && ${csudo}ln -s ${install_main_dir}/bin/${benchmarkName} ${bin_link_dir}/${demoName} || :
[ -x ${install_main_dir}/bin/${benchmarkName} ] && ${csudo}ln -s ${install_main_dir}/bin/${benchmarkName} ${bin_link_dir}/${benchmarkName} || :
[ -x ${install_main_dir}/bin/${benchmarkName} ] && ${csudo}ln -sf ${install_main_dir}/bin/${benchmarkName} ${bin_link_dir}/${demoName} || :
[ -x ${install_main_dir}/bin/${benchmarkName} ] && ${csudo}ln -sf ${install_main_dir}/bin/${benchmarkName} ${bin_link_dir}/${benchmarkName} || :
# [ -x ${install_main_dir}/bin/${dumpName} ] && ${csudo}ln -s ${install_main_dir}/bin/${dumpName} ${bin_link_dir}/${dumpName} || :
[ -x ${install_main_dir}/bin/TDinsight.sh ] && ${csudo}ln -s ${install_main_dir}/bin/TDinsight.sh ${bin_link_dir}/TDinsight.sh || :
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo}ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/${uninstallScript} || :
......@@ -767,6 +767,34 @@ function is_version_compatible() {
esac
}
deb_erase() {
confirm=""
while [ "" == "${confirm}" ]; do
echo -e -n "${RED}Existing TDengine deb is detected, do you want to remove it? [yes|no] ${NC}:"
read confirm
if [ "yes" == "$confirm" ]; then
${csudo}dpkg --remove tdengine ||:
break
elif [ "no" == "$confirm" ]; then
break
fi
done
}
rpm_erase() {
confirm=""
while [ "" == "${confirm}" ]; do
echo -e -n "${RED}Existing TDengine rpm is detected, do you want to remove it? [yes|no] ${NC}:"
read confirm
if [ "yes" == "$confirm" ]; then
${csudo}rpm -e tdengine ||:
break
elif [ "no" == "$confirm" ]; then
break
fi
done
}
function updateProduct() {
# Check if version compatible
if ! is_version_compatible; then
......@@ -779,6 +807,13 @@ function updateProduct() {
echo "File ${tarName} does not exist"
exit 1
fi
if echo $osinfo | grep -qwi "centos"; then
rpm -q tdengine 2>&1 > /dev/null && rpm_erase tdengine ||:
elif echo $osinfo | grep -qwi "ubuntu"; then
dpkg -l tdengine 2>&1 |grep ii > /dev/null && deb_erase tdengine ||:
fi
tar -zxf ${tarName}
install_jemalloc
......
......@@ -336,7 +336,8 @@ cd ${release_dir}
# install_dir has been distinguishes cluster from edege, so comments this code
pkg_name=${install_dir}-${osType}-${cpuType}
taostools_pkg_name=${taostools_install_dir}-${osType}-${cpuType}
versionCompFirst=$(echo ${versionComp} | awk -F '.' '{print $1}')
taostools_pkg_name=${taostools_install_dir}-${osType}-${cpuType}-comp${versionCompFirst}
# if [ "$verMode" == "cluster" ]; then
# pkg_name=${install_dir}-${osType}-${cpuType}
......
......@@ -907,7 +907,7 @@ static FORCE_INLINE int32_t tsParseOneColumnKV(SSchema *pSchema, SStrToken *pTok
} else {
int64_t tmpVal;
if (tsParseTime(pToken, &tmpVal, str, msg, timePrec) != TSDB_CODE_SUCCESS) {
return tscInvalidOperationMsg(msg, "invalid timestamp", pToken->z);
return TSDB_CODE_TSC_INVALID_OPERATION;
}
tdAppendMemRowColVal(row, &tmpVal, true, colId, pSchema->type, toffset);
}
......
......@@ -259,13 +259,14 @@ static inline char *insertTags(char *sql, char *tags) {
// nest call
part2 = insertTags(sub_sql, tags);
free(sub_sql);
if (part2 == NULL) {
// unknown format, can not insert tags
tscError("TAGS insertTags sub select sql failed. subsql=%s sql=%s", sub_sql, sql);
free(sub_sql);
free(buf);
return NULL;
}
free(sub_sql);
// new string is part1 + part2 + part 3
strncpy(buf, p, part1_end - p + 1);
......
......@@ -78,6 +78,9 @@ int tsParseTime(SStrToken *pToken, int64_t *pTime, char **next, char *err, int16
// do nothing
} else if (pToken->type == TK_INTEGER) {
useconds = taosStr2int64(pToken->z);
if (errno == ERANGE) {
return tscInvalidOperationMsg(err, "timestamp is out of range", pToken->z);
}
} else {
// strptime("2001-11-12 18:31:01", "%Y-%m-%d %H:%M:%S", &tm);
if (taosParseTime(pToken->z, pTime, pToken->n, timePrec, tsDaylight) != TSDB_CODE_SUCCESS) {
......@@ -118,7 +121,7 @@ int tsParseTime(SStrToken *pToken, int64_t *pTime, char **next, char *err, int16
char unit = 0;
if (parseAbsoluteDuration(valueToken.z, valueToken.n, &interval, &unit, timePrec) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
return tscInvalidOperationMsg(err, "invalid timestamp", pToken->z);
}
if (sToken.type == TK_PLUS) {
......@@ -382,7 +385,7 @@ int32_t tsParseOneColumn(SSchema *pSchema, SStrToken *pToken, char *payload, cha
} else {
int64_t temp;
if (tsParseTime(pToken, &temp, str, msg, timePrec) != TSDB_CODE_SUCCESS) {
return tscInvalidOperationMsg(msg, "invalid timestamp", pToken->z);
return TSDB_CODE_TSC_INVALID_OPERATION;
}
*((int64_t *)payload) = temp;
......@@ -400,6 +403,9 @@ int32_t tsParseOneColumn(SSchema *pSchema, SStrToken *pToken, char *payload, cha
* Do not employ sort operation is not involved if server time is used.
*/
int32_t tsCheckTimestamp(STableDataBlocks *pDataBlocks, const char *start) {
if (isNull(start, TSDB_DATA_TYPE_TIMESTAMP)) {
return TSDB_CODE_TSC_VALUE_OUT_OF_RANGE;
}
// once the data block is disordered, we do NOT keep previous timestamp any more
if (!pDataBlocks->ordered) {
return TSDB_CODE_SUCCESS;
......
......@@ -723,7 +723,7 @@ static FORCE_INLINE int doBindParam(STableDataBlocks* pBlock, char* data, SParam
if ((*bind->length) > (uintptr_t)param->bytes) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
short size = (short)*bind->length;
uint16_t size = (uint16_t)*bind->length;
STR_WITH_SIZE_TO_VARSTR(data + param->offset, bind->buffer, size);
return TSDB_CODE_SUCCESS;
}
......@@ -777,7 +777,7 @@ static FORCE_INLINE int doBindParam(STableDataBlocks* pBlock, char* data, SParam
return TSDB_CODE_TSC_INVALID_VALUE;
}
short size = 0;
uint16_t size = 0;
switch(param->type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
......@@ -808,7 +808,7 @@ static FORCE_INLINE int doBindParam(STableDataBlocks* pBlock, char* data, SParam
tscError("column length is too big");
return TSDB_CODE_TSC_INVALID_VALUE;
}
size = (short)*pBind->length;
size = (uint16_t)*pBind->length;
STR_WITH_SIZE_TO_VARSTR(data + param->offset, pBind->buffer, size);
return TSDB_CODE_SUCCESS;
......@@ -905,6 +905,12 @@ static int doBindBatchParam(STableDataBlocks* pBlock, SParamInfo* param, TAOS_MU
if (pBind->is_null != NULL && pBind->is_null[i]) {
setNull(data + param->offset, param->type, param->bytes);
if (param->offset == 0) {
if (tsCheckTimestamp(pBlock, data) != TSDB_CODE_SUCCESS) {
tscError("invalid timestamp");
return TSDB_CODE_TSC_INVALID_VALUE;
}
}
continue;
}
......
......@@ -1941,7 +1941,9 @@ static int32_t handleScalarTypeExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32
size_t numOfNode = taosArrayGetSize(colList);
for(int32_t k = 0; k < numOfNode; ++k) {
SColIndex* pIndex = taosArrayGet(colList, k);
if (TSDB_COL_IS_TAG(pIndex->flag)) {
if (TSDB_COL_IS_TAG(pIndex->flag) || (strcasecmp(pIndex->name, TSQL_TBNAME_L) == 0 && pQueryInfo->pUpstream &&
taosArrayGetSize(pQueryInfo->pUpstream) == 0)) {
tExprTreeDestroy(pNode, NULL);
taosArrayDestroy(&colList);
......@@ -2682,7 +2684,7 @@ static int32_t setExprInfoForFunctions(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SS
pExpr->base.param[0].i64 = TSDB_ORDER_DESC;
pExpr->base.param[0].nType = TSDB_DATA_TYPE_INT;
}
// for all queries, the timestamp column needs to be loaded
SSchema s = {.colId = PRIMARYKEY_TIMESTAMP_COL_INDEX, .bytes = TSDB_KEYSIZE, .type = TSDB_DATA_TYPE_TIMESTAMP,};
tscColumnListInsert(pQueryInfo->colList, PRIMARYKEY_TIMESTAMP_COL_INDEX, pExpr->base.uid, &s);
......@@ -3287,9 +3289,12 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
} else if (functionId == TSDB_FUNC_APERCT || functionId == TSDB_FUNC_TAIL) {
size_t cnt = taosArrayGetSize(pItem->pNode->Expr.paramList);
if (cnt != 2 && cnt != 3) valid = false;
} else if (functionId == TSDB_FUNC_PERCT) {
size_t cnt = taosArrayGetSize(pItem->pNode->Expr.paramList);
if (cnt < 2 || cnt > 11) valid = false;
} else if (functionId == TSDB_FUNC_UNIQUE) {
if (taosArrayGetSize(pItem->pNode->Expr.paramList) != 1) valid = false;
}else {
} else {
if (taosArrayGetSize(pItem->pNode->Expr.paramList) != 2) valid = false;
}
if (!valid) {
......@@ -3330,7 +3335,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
}
tVariant* pVariant = NULL;
if (functionId != TSDB_FUNC_UNIQUE) {
if (functionId != TSDB_FUNC_UNIQUE && functionId != TSDB_FUNC_PERCT) {
// 3. valid the parameters
if (pParamElem[1].pNode->tokenId == TK_ID) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg2);
......@@ -3346,7 +3351,31 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
char val[8] = {0};
SExprInfo* pExpr = NULL;
if (functionId == TSDB_FUNC_PERCT || functionId == TSDB_FUNC_APERCT) {
if (functionId == TSDB_FUNC_PERCT) {
int32_t numOfParams = (int32_t)taosArrayGetSize(pItem->pNode->Expr.paramList);
getResultDataInfo(pSchema->type, pSchema->bytes, functionId, numOfParams - 1, &resultType, &resultSize, &interResult, 0,
false, pUdfInfo);
pExpr = tscExprAppend(pQueryInfo, functionId, &idx, resultType, resultSize, getNewResColId(pCmd),
interResult, false);
for (int32_t i = 1; i < numOfParams; ++i) {
pVariant = &pParamElem[i].pNode->value;
if (pVariant->nType != TSDB_DATA_TYPE_DOUBLE && pVariant->nType != TSDB_DATA_TYPE_BIGINT) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
}
tVariantDump(pVariant, val, TSDB_DATA_TYPE_DOUBLE, true);
double dp = GET_DOUBLE_VAL(val);
if (dp < 0 || dp > TOP_BOTTOM_QUERY_LIMIT) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
}
tscExprAddParams(&pExpr->base, val, TSDB_DATA_TYPE_DOUBLE, sizeof(double));
}
tscInsertPrimaryTsSourceColumn(pQueryInfo, pTableMetaInfo->pTableMeta->id.uid);
colIndex += 1; // the first column is ts
} else if (functionId == TSDB_FUNC_APERCT) {
// param1 double
if (pVariant->nType != TSDB_DATA_TYPE_DOUBLE && pVariant->nType != TSDB_DATA_TYPE_BIGINT) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg5);
......@@ -3986,9 +4015,7 @@ int32_t doGetColumnIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColum
const char* msg1 = "invalid column name";
int16_t tsWinColumnIndex;
if (isTablenameToken(pToken)) {
pIndex->columnIndex = TSDB_TBNAME_COLUMN_INDEX;
} else if (strlen(DEFAULT_PRIMARY_TIMESTAMP_COL_NAME) == pToken->n &&
if (strlen(DEFAULT_PRIMARY_TIMESTAMP_COL_NAME) == pToken->n &&
strncmp(pToken->z, DEFAULT_PRIMARY_TIMESTAMP_COL_NAME, pToken->n) == 0) {
pIndex->columnIndex = PRIMARYKEY_TIMESTAMP_COL_INDEX; // just make runtime happy, need fix java test case InsertSpecialCharacterJniTest
} else if (isTimeWindowToken(pToken, &tsWinColumnIndex)) {
......@@ -4015,6 +4042,11 @@ int32_t doGetColumnIndexByName(SStrToken* pToken, SQueryInfo* pQueryInfo, SColum
}
}
// check tbname
if(pIndex->columnIndex == COLUMN_INDEX_INITIAL_VAL && isTablenameToken(pToken)) {
pIndex->columnIndex = TSDB_TBNAME_COLUMN_INDEX;
}
if (pIndex->columnIndex == COLUMN_INDEX_INITIAL_VAL) {
return invalidOperationMsg(msg, msg1);
}
......@@ -9675,7 +9707,7 @@ int32_t checkQueryRangeForFill(SSqlCmd* pCmd, SQueryInfo* pQueryInfo) {
intervalRange = pQueryInfo->interval.interval;
}
// number of result is not greater than 10,000,000
if ((timeRange == 0) || (timeRange / intervalRange) >= MAX_INTERVAL_TIME_WINDOW) {
if ((timeRange / intervalRange) >= MAX_INTERVAL_TIME_WINDOW) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg3);
}
......@@ -10490,6 +10522,7 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
const char* msg8 = "condition missing for join query";
const char* msg9 = "not support 3 level select";
const char* msg10 = "limit user forbid query normal or child table, you can query from stable.";
const char* msg11 = "only support join on tables of the same database";
int32_t code = TSDB_CODE_SUCCESS;
SSqlCmd* pCmd = &pSql->cmd;
......@@ -10684,6 +10717,17 @@ int32_t validateSqlNode(SSqlObj* pSql, SSqlNode* pSqlNode, SQueryInfo* pQueryInf
int32_t joinQuery = (pSqlNode->from != NULL && taosArrayGetSize(pSqlNode->from->list) > 1);
if (joinQuery) {
// the code must be done after all table meta is loaded.
for (int32_t i = 0; i < (int32_t)numOfTables - 1; ++i) {
STableMetaInfo *tmi1 = pQueryInfo->pTableMetaInfo[i];
STableMetaInfo *tmi2 = pQueryInfo->pTableMetaInfo[i + 1];
if (tmi1 != NULL && tmi2 != NULL && strcmp(tmi1->name.dbname, tmi2->name.dbname) != 0) {
return invalidOperationMsg(tscGetErrorMsgPayload(pCmd), msg11);
}
}
}
// parse the group by clause in the first place
if (validateGroupbyNode(pQueryInfo, pSqlNode->pGroupby, pCmd) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_OPERATION;
......
......@@ -1252,11 +1252,28 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->tsBuf.tsOffset = htonl((int32_t)(pMsg - pCmd->payload));
if (pQueryInfo->tsBuf != NULL) {
// note: here used the idx instead of actual vnode id.
int32_t vnodeIndex = pTableMetaInfo->vgroupIndex;
code = dumpFileBlockByGroupId(pQueryInfo->tsBuf, vnodeIndex, pMsg, &pQueryMsg->tsBuf.tsLen, &pQueryMsg->tsBuf.tsNumOfBlocks);
if (code != TSDB_CODE_SUCCESS) {
goto _end;
bool qType = tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0);
if (qType) {
dumpFileBlockByGroupIndex(pQueryInfo->tsBuf, pTableMetaInfo->vgroupIndex, pMsg, &pQueryMsg->tsBuf.tsLen, &pQueryMsg->tsBuf.tsNumOfBlocks);
if (code != TSDB_CODE_SUCCESS) {
goto _end;
}
} else {
// note: here used the idx instead of actual vnode id.
int32_t vgId = 0;
if (pTableMetaInfo->pVgroupTables != NULL) {
int32_t vnodeIndex = pTableMetaInfo->vgroupIndex;
SVgroupTableInfo *pTableInfo = taosArrayGet(pTableMetaInfo->pVgroupTables, vnodeIndex);
vgId = pTableInfo->vgInfo.vgId;
} else {
vgId = query.vgId;
}
code = dumpFileBlockByGroupId(pQueryInfo->tsBuf, vgId, pMsg, &pQueryMsg->tsBuf.tsLen,
&pQueryMsg->tsBuf.tsNumOfBlocks);
if (code != TSDB_CODE_SUCCESS) {
goto _end;
}
}
pMsg += pQueryMsg->tsBuf.tsLen;
......@@ -1317,8 +1334,6 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
memcpy(pMsg, pSql->sqlstr, sqlLen);
pMsg += sqlLen;
pQueryMsg->extend = 1;
STLV *tlv = (STLV *)pMsg;
......
......@@ -1631,6 +1631,8 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
SJoinSupporter* pSupporter = (SJoinSupporter*)param;
int64_t handle = pSupporter->pObj;
tscDebug("***enter joinRetrieveFinalResCallback");
SSqlObj* pParentSql = (SSqlObj*)taosAcquireRef(tscObjRef, handle);
if (pParentSql == NULL) return;
......@@ -1639,7 +1641,7 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
SSqlRes* pRes = &pSql->res;
SQueryInfo* pQueryInfo = tscGetQueryInfo(pCmd);
if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
tscError("0x%"PRIx64" abort query due to other subquery failure. code:%d, global code:%d", pSql->self, numOfRows, pParentSql->res.code);
if (quitAllSubquery(pSql, pParentSql, pSupporter)) {
......@@ -1682,10 +1684,16 @@ static void joinRetrieveFinalResCallback(void* param, TAOS_RES* tres, int numOfR
numOfVgroups = pTableMetaInfo->vgroupList->numOfVgroups;
}
if ((++pTableMetaInfo->vgroupIndex) < numOfVgroups) {
tscDebug("**clauseLimit:%" PRId64 " numOfClauseTotal:%" PRId64 " vgIdx:%d numOfVgroups:%d",
pParentSql->cmd.active->clauseLimit, pParentSql->res.numOfClauseTotal, pTableMetaInfo->vgroupIndex, numOfVgroups);
if ((++pTableMetaInfo->vgroupIndex) < numOfVgroups && (pParentSql->cmd.active->clauseLimit < 0 || pParentSql->cmd.active->clauseLimit > pParentSql->res.numOfClauseTotal)) {
tscDebug("0x%"PRIx64" no result in current vnode anymore, try next vnode, vgIndex:%d", pSql->self, pTableMetaInfo->vgroupIndex);
pSql->cmd.command = TSDB_SQL_SELECT;
pSql->fp = tscJoinQueryCallback;
pSql->cmd.active->clauseLimit = pParentSql->cmd.active->clauseLimit;
pSql->cmd.active->limit.limit = pParentSql->cmd.active->clauseLimit - pParentSql->res.numOfClauseTotal;
pSql->cmd.active->limit.offset = pSql->res.offset;
tscBuildAndSendRequest(pSql, NULL);
goto _return;
......@@ -1744,6 +1752,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
bool hasData = true;
bool reachLimit = false;
tscDebug("***enter tscFetchDatablockForSubquery");
{ pthread_mutex_lock(&pSql->subState.mutex);
assert(pSql->subState.numOfSub >= 1);
......@@ -1782,6 +1791,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
// has data remains in client side, and continue to return data to app
if (hasData) {
tscDebug("*hasData");
tscBuildResFromSubqueries(pSql);
return;
}
......@@ -1789,6 +1799,7 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
// If at least one subquery is completed in current vnode, try the next vnode in case of multi-vnode
// super table projection query.
if (reachLimit) {
tscDebug("*reachLimit");
pSql->res.completed = true;
freeJoinSubqueryObj(pSql);
......@@ -1847,8 +1858,11 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
SQueryInfo* pQueryInfo = tscGetQueryInfo(&pSub->cmd);
tscDebug("**nonorderedPrj:%d resRow:%d numOfRows:%d com:%d numOfClauseTotal:%"PRId64 " clauseLimit:%" PRId64,
tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0), pSub->res.row, pSub->res.numOfRows,
pSub->res.completed, pSql->res.numOfClauseTotal, pSql->cmd.active->clauseLimit);
if (tscNonOrderedProjectionQueryOnSTable(pQueryInfo, 0) && pSub->res.row >= pSub->res.numOfRows &&
pSub->res.completed) {
pSub->res.completed && (pSql->cmd.active->clauseLimit < 0 || pSql->res.numOfClauseTotal < pSql->cmd.active->clauseLimit)) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
assert(pQueryInfo->numOfTables == 1);
......@@ -1865,6 +1879,9 @@ void tscFetchDatablockForSubquery(SSqlObj* pSql) {
pTableMetaInfo->vgroupIndex);
pSub->cmd.command = TSDB_SQL_SELECT;
pSub->fp = tscJoinQueryCallback;
pSub->cmd.active->clauseLimit = pSql->cmd.active->clauseLimit;
pSub->cmd.active->limit.limit = pSql->cmd.active->clauseLimit - pSql->res.numOfClauseTotal;
pSub->cmd.active->limit.offset = pSub->res.offset;
tscBuildAndSendRequest(pSub, NULL);
tryNextVnode = true;
......@@ -2132,6 +2149,8 @@ int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter
SQueryInfo *pNewQueryInfo = tscGetQueryInfo(&pNew->cmd);
assert(pNewQueryInfo != NULL);
pNewQueryInfo->clauseLimit = -1;
pSupporter->colList = pNewQueryInfo->colList;
pNewQueryInfo->colList = NULL;
......@@ -3027,6 +3046,11 @@ void tscHandleSubqueryError(SRetrieveSupport *trsupport, SSqlObj *pSql, int numO
}
SSqlObj *pParentSql = trsupport->pParentSql;
if (pParentSql->signature != pParentSql) {
tscError("query 0x%"PRIx64" parent sql obj %p not valid. parent id: 0x%"PRIx64, pSql->self, pParentSql, pParentSql->self);
return;
}
int32_t subqueryIndex = trsupport->subqueryIndex;
assert(pSql != NULL);
......@@ -3133,6 +3157,11 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p
int32_t idx = trsupport->subqueryIndex;
SSqlObj * pParentSql = trsupport->pParentSql;
tOrderDescriptor *pDesc = trsupport->pOrderDescriptor;
if (pParentSql->signature != pParentSql) {
tscError("query 0x%"PRIx64" parent sql obj %p not valid. parent id: 0x%"PRIx64, pSql->self, pParentSql, pParentSql->self);
return;
}
SSubqueryState* pState = &pParentSql->subState;
SQueryInfo *pQueryInfo = tscGetQueryInfo(&pSql->cmd);
......@@ -3794,6 +3823,7 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) {
pthread_mutex_unlock(&pSql->subState.mutex); }
if (numOfRes == 0) { // no result any more, free all subquery objects
tscDebug("*******query complete");
pSql->res.completed = true;
freeJoinSubqueryObj(pSql);
return;
......@@ -3861,7 +3891,7 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) {
pthread_mutex_unlock(&pSql->subState.mutex); }
pRes->numOfRows = numOfRes;
pRes->numOfClauseTotal += numOfRes;
//pRes->numOfClauseTotal += numOfRes;
int32_t finalRowSize = 0;
for(int32_t i = 0; i < tscNumOfFields(pQueryInfo); ++i) {
......@@ -3878,6 +3908,7 @@ static void doBuildResFromSubqueries(SSqlObj* pSql) {
void tscBuildResFromSubqueries(SSqlObj *pSql) {
SSqlRes* pRes = &pSql->res;
tscDebug("*enter tscBuildResFromSubqueries");
if (pRes->code != TSDB_CODE_SUCCESS) {
tscAsyncResultOnError(pSql);
return;
......
......@@ -282,14 +282,15 @@ void taos_cleanup(void) {
#endif
}
int32_t id = tscObjRef;
tscObjRef = -1;
taosCloseRef(id);
void* p = tscQhandle;
tscQhandle = NULL;
taosCleanUpScheduler(p);
int32_t id = tscObjRef;
tscObjRef = -1;
taosCloseRef(id);
id = tscRefId;
tscRefId = -1;
taosCloseRef(id);
......
......@@ -2780,7 +2780,7 @@ void tscExprAddParams(SSqlExpr* pExpr, char* argument, int32_t type, int32_t byt
tVariantCreateFromBinary(&pExpr->param[pExpr->numOfParams], argument, bytes, type);
pExpr->numOfParams += 1;
assert(pExpr->numOfParams <= 3);
assert(pExpr->numOfParams <= 10);
}
SExprInfo* tscExprGet(SQueryInfo* pQueryInfo, int32_t idx) {
......@@ -4561,7 +4561,9 @@ int32_t tscErrorMsgWithCode(int32_t code, char* dstBuffer, const char* errMsg, c
bool tscHasReachLimitation(SQueryInfo* pQueryInfo, SSqlRes* pRes) {
assert(pQueryInfo != NULL && pQueryInfo->clauseLimit != 0);
return (pQueryInfo->clauseLimit > 0 && pRes->numOfClauseTotal >= pQueryInfo->clauseLimit);
bool reachLimit = (pQueryInfo->clauseLimit > 0 && pRes->numOfClauseTotal >= pQueryInfo->clauseLimit);
tscDebug("reachLimit:%d, limit:%" PRId64 " total:%" PRId64, reachLimit, pQueryInfo->clauseLimit, pRes->numOfClauseTotal);
return reachLimit;
}
char* tscGetErrorMsgPayload(SSqlCmd* pCmd) { return pCmd->payload; }
......
......@@ -59,7 +59,7 @@ typedef struct SSqlExpr {
// pQueryAttr->interBytesForGlobal
int16_t numOfParams; // argument value of each function
tVariant param[3]; // parameters are not more than 3
tVariant param[10]; // parameters are not more than 10
int32_t offset; // sub result column value of arithmetic expression.
int16_t resColId; // result column id
......
......@@ -467,7 +467,11 @@ static void tdAppendDataRowToDataCol(SDataRow row, STSchema *pSchema, SDataCols
while (dcol < pCols->numOfCols) {
SDataCol *pDataCol = &(pCols->cols[dcol]);
if (rcol >= schemaNCols(pSchema)) {
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints, rowOffset);
if (forceSetNull) {
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints, rowOffset);
} else {
break;
}
dcol++;
continue;
}
......@@ -510,7 +514,11 @@ static void tdAppendKvRowToDataCol(SKVRow row, STSchema *pSchema, SDataCols *pCo
while (dcol < pCols->numOfCols) {
SDataCol *pDataCol = &(pCols->cols[dcol]);
if (rcol >= nRowCols || rcol >= schemaNCols(pSchema)) {
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints, rowOffset);
if (forceSetNull) {
dataColAppendVal(pDataCol, getNullValue(pDataCol->type), pCols->numOfRows, pCols->maxPoints, rowOffset);
} else {
break;
}
++dcol;
continue;
}
......
......@@ -24,6 +24,7 @@ extern "C" {
#include "tlog.h"
#include "trpc.h"
#include "tglobal.h"
#include "tfs.h"
#include "dnode.h"
#include "vnode.h"
......
......@@ -27,6 +27,7 @@ static void sigintHandler(int32_t signum, void *sigInfo, void *context);
int32_t main(int32_t argc, char *argv[]) {
int dump_config = 0;
int dump_cluster = 0;
// Set global configuration file
for (int32_t i = 1; i < argc; ++i) {
......@@ -61,8 +62,10 @@ int32_t main(int32_t argc, char *argv[]) {
printf("buildinfo: %s\n", buildinfo);
exit(EXIT_SUCCESS);
} else if (strcmp(argv[i], "-k") == 0) {
grantParseParameter();
grantParseParameter(argv[i]);
exit(EXIT_SUCCESS);
} else if (strcmp(argv[i], "-u") == 0) {
dump_cluster = 1;
} else if (strcmp(argv[i], "-A") == 0) {
tsPrintAuth = 1;
}
......@@ -115,6 +118,23 @@ int32_t main(int32_t argc, char *argv[]) {
exit(EXIT_SUCCESS);
}
if (0 != dump_cluster) {
taosInitGlobalCfg();
if (!taosReadGlobalCfg()) {
printf("TDengine read global config failed\n");
exit(EXIT_FAILURE);
}
if (tsDiskCfgNum > 0) {
if (tfsInit(tsDiskCfg, tsDiskCfgNum) < 0) {
printf("failed to init TFS since %s\n", tstrerror(terrno));
exit(EXIT_FAILURE);
}
strncpy(tsDataDir, TFS_PRIMARY_PATH(), TSDB_FILENAME_LEN);
}
grantParseParameter("-u");
exit(EXIT_SUCCESS);
}
if (tsem_init(&exitSem, 0, 0) != 0) {
printf("failed to create exit semphore\n");
exit(EXIT_FAILURE);
......
......@@ -38,7 +38,7 @@ typedef enum {
int32_t grantInit();
void grantCleanUp();
void grantParseParameter();
void grantParseParameter(const char* param);
int32_t grantCheck(EGrantType grant);
void grantReset(EGrantType grant, uint64_t value);
void grantAdd(EGrantType grant, uint64_t value);
......
Subproject commit 261fcca698d0fe171bb6fed1913c0a922de388f3
Subproject commit 41affde86d4ae7be340e31725650970326ac5b0d
......@@ -22,7 +22,7 @@
int32_t grantInit() { return TSDB_CODE_SUCCESS; }
void grantCleanUp() {}
void grantParseParameter() { mError("can't parsed parameter k"); }
void grantParseParameter(const char* param) { mError("can't parse parameter %s", param); }
int32_t grantCheck(EGrantType grant) { return TSDB_CODE_SUCCESS; }
void grantReset(EGrantType grant, uint64_t value) {}
void grantAdd(EGrantType grant, uint64_t value) {}
......
Subproject commit f3278c55e68b4b6195e3ef8841a1d81df6f50e08
Subproject commit 251a4aa70270966258dd26f90ac3e7c39ca90f1a
......@@ -196,6 +196,7 @@ typedef struct SQLFunctionCtx {
uint32_t order; // asc|desc
int16_t inputType;
int32_t inputBytes;
bool hasColDataInput; // the input data block has column data
int16_t outputType;
int32_t outputBytes; // size of results, determined by function and input column data type
......@@ -209,7 +210,7 @@ typedef struct SQLFunctionCtx {
int64_t startTs; // timestamp range of current query when function is executed on a specific data block
int64_t endTs;
int32_t numOfParams;
tVariant param[4]; // input parameter, e.g., top(k, 20), the number of results for top query is kept in param
tVariant param[10]; // input parameter, e.g., top(k, 20), the number of results for top query is kept in param
int64_t *ptsList; // corresponding timestamp array list
void *ptsOutputBuf; // corresponding output buffer for timestamp of each result, e.g., top/bottom*/
SQLPreAggVal preAggVals;
......
......@@ -110,7 +110,7 @@ int32_t tsBufMerge(STSBuf* pDestBuf, const STSBuf* pSrcBuf);
STSBuf* tsBufClone(STSBuf* pTSBuf);
STSGroupBlockInfo* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id);
SArray* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id);
void tsBufFlush(STSBuf* pTSBuf);
void tsBufResetPos(STSBuf* pTSBuf);
......@@ -136,6 +136,8 @@ void tsBufGetGroupIdList(STSBuf* pTSBuf, int32_t* num, int32_t** id);
int32_t dumpFileBlockByGroupId(STSBuf* pTSBuf, int32_t id, void* buf, int32_t* len, int32_t* numOfBlocks);
int32_t dumpFileBlockByGroupIndex(STSBuf* pTSBuf, int32_t groupIndex, void* pBuf, int32_t* len, int32_t* numOfBlocks);
STSElem tsBufFindElemStartPosByTag(STSBuf* pTSBuf, tVariant* pTag);
bool tsBufIsValidElem(STSElem* pElem);
......
......@@ -791,6 +791,9 @@ expr(A) ::= ID(X) LP exprlist(Y) RP(E). { tStrTokenAppend(pInfo->funcs, &X); A =
// for parsing sql functions with wildcard for parameters. e.g., count(*)/first(*)/last(*) operation
expr(A) ::= ID(X) LP STAR RP(Y). { tStrTokenAppend(pInfo->funcs, &X); A = tSqlExprCreateFunction(NULL, &X, &Y, X.type); }
// for parsing sql functions without parameters
expr(A) ::= ID(X) LP RP(Y). { tStrTokenAppend(pInfo->funcs, &X); A = tSqlExprCreateFunction(tSqlExprListAppend(0, 0, 0, 0), &X, &Y, X.type); }
// for parsing sql function CAST(column as typename)
expr(A) ::= ID(X) LP expr(B) AS typename(C) RP(Y). { tStrTokenAppend(pInfo->funcs, &X); A = tSqlExprCreateFuncWithParams(pInfo, B, &C, &X, &Y, X.type); }
......@@ -855,7 +858,6 @@ expr(A) ::= expr(X) IN LP exprlist(Y) RP. {A = tSqlExprCreate(X, (tSqlExpr*)Y,
exprlist(A) ::= exprlist(X) COMMA expritem(Y). {A = tSqlExprListAppend(X,Y,0, 0);}
exprlist(A) ::= expritem(X). {A = tSqlExprListAppend(0,X,0, 0);}
expritem(A) ::= expr(X). {A = X;}
expritem(A) ::= . {A = 0;}
///////////////////////////////////reset query cache//////////////////////////////////////
cmd ::= RESET QUERY CACHE. { setDCLSqlElems(pInfo, TSDB_SQL_RESET_CACHE, 0);}
......
......@@ -700,8 +700,13 @@ int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionI
*bytes = sizeof(double);
*interBytes = sizeof(SSpreadInfo);
} else if (functionId == TSDB_FUNC_PERCT) {
*type = (int16_t)TSDB_DATA_TYPE_DOUBLE;
*bytes = sizeof(double);
if (param > 1) {
*type = (int16_t)TSDB_DATA_TYPE_BINARY;
*bytes = 512;
} else {
*type = (int16_t)TSDB_DATA_TYPE_DOUBLE;
*bytes = sizeof(double);
}
*interBytes = sizeof(SPercentileInfo);
} else if (functionId == TSDB_FUNC_LEASTSQR) {
*type = TSDB_DATA_TYPE_BINARY;
......@@ -3096,7 +3101,7 @@ static void percentile_function(SQLFunctionCtx *pCtx) {
}
static void percentile_finalizer(SQLFunctionCtx *pCtx) {
double v = pCtx->param[0].nType == TSDB_DATA_TYPE_INT ? pCtx->param[0].i64 : pCtx->param[0].dKey;
double v = 0;
SResultRowCellInfo *pResInfo = GET_RES_INFO(pCtx);
SPercentileInfo* ppInfo = (SPercentileInfo *) GET_ROWCELL_INTERBUF(pResInfo);
......@@ -3107,7 +3112,25 @@ static void percentile_finalizer(SQLFunctionCtx *pCtx) {
assert(ppInfo->numOfElems == 0);
setNull(pCtx->pOutput, pCtx->outputType, pCtx->outputBytes);
} else {
SET_DOUBLE_VAL((double *)pCtx->pOutput, getPercentile(pMemBucket, v));
if (pCtx->numOfParams > 1) {
((char *)varDataVal(pCtx->pOutput))[0] = '[';
size_t len = 1;
size_t maxBufLen = 512;
for (int32_t i = 0; i < pCtx->numOfParams; ++i) {
v = pCtx->param[i].nType == TSDB_DATA_TYPE_INT ? pCtx->param[i].i64 : pCtx->param[i].dKey;
if (i == pCtx->numOfParams - 1) {
len += snprintf((char *)varDataVal(pCtx->pOutput) + len, maxBufLen - len, "%lf]", getPercentile(pMemBucket, v));
} else {
len += snprintf((char *)varDataVal(pCtx->pOutput) + len, maxBufLen - len, "%lf, ", getPercentile(pMemBucket, v));
}
}
varDataSetLen(pCtx->pOutput, len);
} else {
v = pCtx->param[0].nType == TSDB_DATA_TYPE_INT ? pCtx->param[0].i64 : pCtx->param[0].dKey;
SET_DOUBLE_VAL((double *)pCtx->pOutput, getPercentile(pMemBucket, v));
}
}
tMemBucketDestroy(pMemBucket);
......
......@@ -52,6 +52,7 @@ enum {
TS_JOIN_TS_EQUAL = 0,
TS_JOIN_TS_NOT_EQUALS = 1,
TS_JOIN_TAG_NOT_EQUALS = 2,
TS_JOIN_BLOCK_IGNORE = 3,
};
typedef enum SResultTsInterpType {
......@@ -1322,6 +1323,7 @@ static void doSetInputDataBlockInfo(SOperatorInfo* pOperator, SQLFunctionCtx* pC
pCtx[i].order = order;
pCtx[i].size = pBlock->info.rows;
pCtx[i].currentStage = (uint8_t)pOperator->pRuntimeEnv->scanFlag;
pCtx[i].hasColDataInput = false;
setBlockStatisInfo(&pCtx[i], pBlock, &pOperator->pExpr[i].base.colInfo);
}
......@@ -1349,6 +1351,7 @@ static void doSetInputDataBlock(SOperatorInfo* pOperator, SQLFunctionCtx* pCtx,
pCtx[i].order = order;
pCtx[i].size = pBlock->info.rows;
pCtx[i].currentStage = (uint8_t)pOperator->pRuntimeEnv->scanFlag;
pCtx[i].hasColDataInput = (pBlock->pDataBlock != NULL);
setBlockStatisInfo(&pCtx[i], pBlock, &pOperator->pExpr[i].base.colInfo);
......@@ -2129,6 +2132,9 @@ static bool functionNeedToExecute(SQueryRuntimeEnv* pRuntimeEnv, SQLFunctionCtx*
}
if (functionId == TSDB_FUNC_FIRST_DST || functionId == TSDB_FUNC_FIRST) {
if (pCtx->hasColDataInput == false) {
return false;
}
// if param[2] is set value, input data come from client, order is no relation with pQueryAttr->order, so always
// return true
if (pCtx->param[2].nType == TSDB_DATA_TYPE_INT) return true;
......@@ -2137,6 +2143,9 @@ static bool functionNeedToExecute(SQueryRuntimeEnv* pRuntimeEnv, SQLFunctionCtx*
// denote the order type
if ((functionId == TSDB_FUNC_LAST_DST || functionId == TSDB_FUNC_LAST)) {
if (pCtx->hasColDataInput == false) {
return false;
}
// if param[2] is set value, input data come from client, order is no relation with pQueryAttr->order, so always
// return true
if (pCtx->param[2].nType == TSDB_DATA_TYPE_INT) return true;
......@@ -2259,6 +2268,7 @@ static SQLFunctionCtx* createSQLFunctionCtx(SQueryRuntimeEnv* pRuntimeEnv, SExpr
} else {
pCtx->inputBytes = pSqlExpr->colBytes;
}
pCtx->hasColDataInput = false;
pCtx->ptsOutputBuf = NULL;
......@@ -3222,13 +3232,13 @@ static int32_t doTSJoinFilter(SQueryRuntimeEnv* pRuntimeEnv, TSKEY key, tVariant
if (key < elem.ts) {
return TS_JOIN_TS_NOT_EQUALS;
} else if (key > elem.ts) {
longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_INCONSISTAN);
return TS_JOIN_BLOCK_IGNORE;
}
} else {
if (key > elem.ts) {
return TS_JOIN_TS_NOT_EQUALS;
} else if (key < elem.ts) {
longjmp(pRuntimeEnv->env, TSDB_CODE_QRY_INCONSISTAN);
return TS_JOIN_BLOCK_IGNORE;
}
}
......@@ -3413,6 +3423,9 @@ void filterColRowsInDataBlock(SQueryRuntimeEnv* pRuntimeEnv, SSDataBlock* pBlock
} else if (ret == TS_JOIN_TS_NOT_EQUALS) {
all = false;
continue;
} else if (ret == TS_JOIN_BLOCK_IGNORE) {
all = false;
break;
} else {
assert(ret == TS_JOIN_TS_EQUAL);
p[offset] = true;
......@@ -4542,9 +4555,9 @@ int32_t setTimestampListJoinInfo(SQueryRuntimeEnv* pRuntimeEnv, tVariant* pTag,
if (!tsBufIsValidElem(&elem)) {
if (pTag->nType == TSDB_DATA_TYPE_BINARY || pTag->nType == TSDB_DATA_TYPE_NCHAR ||
pTag->nType == TSDB_DATA_TYPE_JSON) {
qError("QInfo:0x%" PRIx64 " failed to find tag:%s in ts_comp", GET_QID(pRuntimeEnv), pTag->pz);
qDebug("QInfo:0x%" PRIx64 " tag not found in:%s in ts_comp", GET_QID(pRuntimeEnv), pTag->pz);
} else {
qError("QInfo:0x%" PRIx64 " failed to find tag:%" PRId64 " in ts_comp", GET_QID(pRuntimeEnv), pTag->i64);
qDebug("QInfo:0x%" PRIx64 " tag not found in:%" PRId64 " in ts_comp", GET_QID(pRuntimeEnv), pTag->i64);
}
return -1;
......@@ -4966,8 +4979,10 @@ static void doOperatorExecProfOnce(SOperatorStackItem* item, SQueryProfEvent* ev
}
void calculateOperatorProfResults(SQInfo* pQInfo) {
if (pQInfo->summary.queryProfEvents == NULL) {
qDebug("QInfo:0x%" PRIx64 " query prof events array is null", pQInfo->qId);
if (pQInfo->summary.queryProfEvents == NULL ||
pQInfo->summary.queryProfEvents->pData == NULL ||
pQInfo->summary.queryProfEvents->size == 0) {
qDebug("QInfo:0x%" PRIx64 " query prof events array is null or array data invalid", pQInfo->qId);
return;
}
......@@ -9764,7 +9779,13 @@ int32_t createQueryFunc(SQueriedTableInfo* pTableInfo, int32_t numOfOutput, SExp
}
}
int32_t param = (int32_t)pExprs[i].base.param[0].i64;
int32_t param;
if (pExprs[i].base.functionId != TSDB_FUNC_PERCT) {
param = (int32_t)pExprs[i].base.param[0].i64;
} else {
param = pExprs[i].base.numOfParams;
}
if (pExprs[i].base.functionId > 0 && pExprs[i].base.functionId != TSDB_FUNC_SCALAR_EXPR &&
!isTimeWindowFunction(pExprs[i].base.functionId) &&
(type != pExprs[i].base.colType || bytes != pExprs[i].base.colBytes)) {
......
......@@ -399,6 +399,8 @@ void taosResetFillInfo(SFillInfo* pFillInfo, TSKEY startTimestamp) {
pFillInfo->numOfRows = 0;
pFillInfo->numOfCurrent = 0;
pFillInfo->numOfTotal = 0;
//free prevValues so that the new group has no prevValues then fill(linear) of new group won't use old group values.
tfree(pFillInfo->prevValues);
}
void* taosDestroyFillInfo(SFillInfo* pFillInfo) {
......
......@@ -3732,7 +3732,7 @@ int32_t filterGetTimeRange(SFilterInfo *info, STimeWindow *win) {
SFilterRange tra;
filterGetRangeRes(prev, &tra);
win->skey = tra.s;
win->skey = tra.s;
win->ekey = tra.e;
}
......@@ -3786,7 +3786,7 @@ int32_t filterConverNcharColumns(SFilterInfo* info, int32_t rows, bool *gotNchar
}
fi->data = nfi.data;
*gotNchar = true;
}
}
......@@ -3820,22 +3820,27 @@ int32_t filterIsIndexedColumnQuery(SFilterInfo* info, int32_t idxId, bool *res)
CHK_JMP(FILTER_GET_COL_FIELD_ID(FILTER_GET_COL_FIELD(info, 0)) != idxId);
int32_t optr = FILTER_UNIT_OPTR(info->units);
CHK_JMP(optr == TSDB_RELATION_LIKE || optr == TSDB_RELATION_IN || optr == TSDB_RELATION_MATCH
|| optr == TSDB_RELATION_ISNULL || optr == TSDB_RELATION_NOTNULL || optr == TSDB_RELATION_CONTAINS);
*res = true;
return TSDB_CODE_SUCCESS;
_return:
*res = false;
return TSDB_CODE_SUCCESS;
}
int32_t filterGetIndexedColumnInfo(SFilterInfo* info, char** val, int32_t *order, int32_t *flag) {
if (FILTER_GET_FLAG(info->status, FI_STATUS_EMPTY)) {
*order = 0;
return TSDB_CODE_SUCCESS;
}
SFilterComUnit *cunit = info->cunits;
uint8_t optr = cunit->optr;
......@@ -3853,7 +3858,7 @@ int32_t filterGetIndexedColumnInfo(SFilterInfo* info, char** val, int32_t *order
if (cunit->valData2 == cunit->valData && optr != TSDB_RELATION_EQUAL) {
FILTER_SET_FLAG(*flag, FI_ACTION_NO_NEED);
}
return TSDB_CODE_SUCCESS;
}
......
......@@ -593,22 +593,22 @@ static void tsBufGetBlock(STSBuf* pTSBuf, int32_t groupIndex, int32_t blockIndex
}
STSCursor* pCur = &pTSBuf->cur;
if (pCur->vgroupIndex == groupIndex && ((pCur->blockIndex <= blockIndex && pCur->order == TSDB_ORDER_ASC) ||
(pCur->blockIndex >= blockIndex && pCur->order == TSDB_ORDER_DESC))) {
int32_t i = 0;
bool decomp = false;
int32_t step = abs(blockIndex - pCur->blockIndex);
while ((++i) <= step) {
if (readDataFromDisk(pTSBuf, pCur->order, decomp) == NULL) {
return;
}
}
} else {
// if (pCur->vgroupIndex == groupIndex && ((pCur->blockIndex <= blockIndex && pCur->order == TSDB_ORDER_ASC) ||
// (pCur->blockIndex >= blockIndex && pCur->order == TSDB_ORDER_DESC))) {
// int32_t i = 0;
// bool decomp = false;
// int32_t step = abs(blockIndex - pCur->blockIndex);
//
// while ((++i) <= step) {
// if (readDataFromDisk(pTSBuf, pCur->order, decomp) == NULL) {
// return;
// }
// }
// } else {
if (tsBufFindBlock(pTSBuf, pBlockInfo, blockIndex) == -1) {
assert(false);
}
}
// }
STSBlock* pBlock = &pTSBuf->block;
......@@ -651,13 +651,16 @@ static int32_t doUpdateGroupInfo(STSBuf* pTSBuf, int64_t offset, STSGroupBlockIn
return 0;
}
STSGroupBlockInfo* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id) {
int32_t j = tsBufFindGroupById(pTSBuf->pData, pTSBuf->numOfGroups, id);
if (j == -1) {
return NULL;
SArray* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id) {
SArray* pList = taosArrayInit(4, sizeof(STSGroupBlockInfo));
for(int32_t i = 0; i < pTSBuf->numOfGroups; ++i) {
if (pTSBuf->pData[i].info.id == id) {
taosArrayPush(pList, &pTSBuf->pData[i].info);
}
}
return &pTSBuf->pData[j].info;
return pList;
}
int32_t STSBufUpdateHeader(STSBuf* pTSBuf, STSBufFileHeader* pHeader) {
......@@ -1099,23 +1102,54 @@ void tsBufGetGroupIdList(STSBuf* pTSBuf, int32_t* num, int32_t** id) {
}
}
int32_t dumpFileBlockByGroupId(STSBuf* pTSBuf, int32_t groupIndex, void* buf, int32_t* len, int32_t* numOfBlocks) {
int32_t dumpFileBlockByGroupId(STSBuf* pTSBuf, int32_t groupId, void* buf, int32_t* len, int32_t* numOfBlocks) {
SArray* pList = tsBufGetGroupBlockInfo(pTSBuf, groupId);
*len = 0;
*numOfBlocks = 0;
char* p = buf;
for(int32_t i = 0; i < taosArrayGetSize(pList); ++i) {
STSGroupBlockInfo* pBlockInfo = taosArrayGet(pList, i);
if (fseek(pTSBuf->f, pBlockInfo->offset, SEEK_SET) != 0) {
int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
// qError("%p: fseek failed: %s", pSql, tstrerror(code));
return code;
}
size_t s = fread(p, 1, pBlockInfo->compLen, pTSBuf->f);
if (s != pBlockInfo->compLen) {
int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
// tscError("%p: fread didn't return expected data: %s", pSql, tstrerror(code));
return code;
}
*len += pBlockInfo->compLen;
*numOfBlocks += pBlockInfo->numOfBlocks;
p += pBlockInfo->compLen;
}
return TSDB_CODE_SUCCESS;
}
int32_t dumpFileBlockByGroupIndex(STSBuf* pTSBuf, int32_t groupIndex, void* pBuf, int32_t* len, int32_t* numOfBlocks) {
assert(groupIndex >= 0 && groupIndex < pTSBuf->numOfGroups);
STSGroupBlockInfo *pBlockInfo = &pTSBuf->pData[groupIndex].info;
STSGroupBlockInfo* pBlockInfo = &pTSBuf->pData[groupIndex].info;
*len = 0;
*numOfBlocks = 0;
if (fseek(pTSBuf->f, pBlockInfo->offset, SEEK_SET) != 0) {
int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
// qError("%p: fseek failed: %s", pSql, tstrerror(code));
// qError("%p: fseek failed: %s", pSql, tstrerror(code));
return code;
}
size_t s = fread(buf, 1, pBlockInfo->compLen, pTSBuf->f);
size_t s = fread(pBuf, 1, pBlockInfo->compLen, pTSBuf->f);
if (s != pBlockInfo->compLen) {
int code = TAOS_SYSTEM_ERROR(ferror(pTSBuf->f));
// tscError("%p: fread didn't return expected data: %s", pSql, tstrerror(code));
// tscError("%p: fread didn't return expected data: %s", pSql, tstrerror(code));
return code;
}
......
此差异已折叠。
......@@ -30,11 +30,11 @@ IF (HEADER_GTEST_INCLUDE_DIR AND (LIB_GTEST_STATIC_DIR OR LIB_GTEST_SHARED_DIR))
ENDIF()
ENDIF()
SET_SOURCE_FILES_PROPERTIES(./astTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./histogramTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./percentileTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./apercentileTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./resultBufferTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./astTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./histogramTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./percentileTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./apercentileTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./resultBufferTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./tsBufTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./unitTest.cpp PROPERTIES COMPILE_FLAGS -w)
SET_SOURCE_FILES_PROPERTIES(./rangeMergeTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./unitTest.cpp PROPERTIES COMPILE_FLAGS -w)
#SET_SOURCE_FILES_PROPERTIES(./rangeMergeTest.cpp PROPERTIES COMPILE_FLAGS -w)
......@@ -291,13 +291,13 @@ void tdigestTest() {
for (int32_t j = 0; j < typeTimes; ++j) {
printf("DMode:%d,Type:%d,randP:%d -", dataMode[i], dataTypes[j], randPers[p]);
for (int32_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[0][i][j][m][p]);
printf(" %ld:%f", totalNum[m], useTime[0][i][j][m][p]);
}
printf("\n");
printf("HMode:%d,Type:%d,randP:%d -", dataMode[i], dataTypes[j], randPers[p]);
for (int32_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[1][i][j][m][p]);
printf(" %ld:%f", totalNum[m], useTime[1][i][j][m][p]);
}
printf("\n");
}
......@@ -307,13 +307,13 @@ void tdigestTest() {
for (int32_t j = 0; j < typeTimes; ++j) {
printf("DMode:%d,Type:%d,randL:%d -", dataMode[i], dataTypes[j], randLimits[p]);
for (int64_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[0][i][j][m][p]);
printf(" %ld:%f", totalNum[m], useTime[0][i][j][m][p]);
}
printf("\n");
printf("HMode:%d,Type:%d,randL:%d -", dataMode[i], dataTypes[j], randLimits[p]);
for (int64_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[1][i][j][m][p]);
printf(" %ld:%f", totalNum[m], useTime[1][i][j][m][p]);
}
printf("\n");
}
......@@ -322,13 +322,13 @@ void tdigestTest() {
for (int32_t j = 0; j < typeTimes; ++j) {
printf("DMode:%d,Type:%d -", dataMode[i], dataTypes[j]);
for (int64_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[0][i][j][m][0]);
printf(" %ld:%f", totalNum[m], useTime[0][i][j][m][0]);
}
printf("\n");
printf("HMode:%d,Type:%d -", dataMode[i], dataTypes[j]);
for (int64_t m = 0; m < numTimes; ++m) {
printf(" %d:%f", totalNum[m], useTime[1][i][j][m][0]);
printf(" %ld:%f", totalNum[m], useTime[1][i][j][m][0]);
}
printf("\n");
}
......
......@@ -35,7 +35,7 @@ void doHistogramAddTest() {
(int64_t)systemTime.tv_sec * 1000L + (uint64_t)systemTime.tv_usec / 1000;
printf("total elapsed time: %ld\n", et - st);
printf("elements: %d, slot:%d \n", pHisto->numOfElems, pHisto->numOfEntries);
printf("elements: %ld, slot:%d \n", pHisto->numOfElems, pHisto->numOfEntries);
tHistogramPrint(pHisto);
printf("%ld\n", tHistogramSum(pHisto, 1.5));
......
......@@ -497,19 +497,103 @@ void mergeIdenticalVnodeBufferTest() {
tsBufDestroy(pTSBuf1);
tsBufDestroy(pTSBuf2);
}
void mergeMultiBlockFromOneGroupTest() {
STSBuf* pTSBuf1 = tsBufCreate(true, TSDB_ORDER_ASC);
STSBuf* pTSBuf2 = tsBufCreate(true, TSDB_ORDER_ASC);
tVariant t = {0};
t.nType = TSDB_DATA_TYPE_BIGINT;
int32_t step = 30;
int32_t num = 1000;
int32_t numOfTags = 2;
// vnodeId:0
int64_t start = 10000;
for (int32_t i = 0; i < numOfTags; ++i) {
int64_t* list = createTsList(num, start, step);
t.i64 = i;
tsBufAppend(pTSBuf1, 12, &t, (const char*)list, num * sizeof(int64_t));
free(list);
start += step * num;
}
tsBufFlush(pTSBuf1);
for (int32_t i = numOfTags; i < numOfTags * 2; ++i) {
int64_t* list = createTsList(num, start, step);
t.i64 = i;
tsBufAppend(pTSBuf1, 77, &t, (const char*)list, num * sizeof(int64_t));
free(list);
start += step * num;
}
tsBufFlush(pTSBuf1);
start = 10000;
for (int32_t i = 911; i < 912; ++i) {
int64_t* list = createTsList(num, start, step);
t.i64 = i;
tsBufAppend(pTSBuf1, 12, &t, (const char*)list, num * sizeof(int64_t));
free(list);
start += step * num;
}
tsBufFlush(pTSBuf1);
char* p = (char*) malloc(1024768);
int32_t len = 0;
int32_t numOfBlocks = 0;
dumpFileBlockByGroupId(pTSBuf1, 12, p, &len, &numOfBlocks);
STSBuf* pNew = tsBufCreateFromCompBlocks(p, numOfBlocks, len, 1, 12);
printf("%p\n", pNew);
tsBufDisplay(pNew);
// tsBufMerge(pTSBuf1, pTSBuf2);
// EXPECT_EQ(pTSBuf1->numOfGroups, 2);
// EXPECT_EQ(pTSBuf1->numOfTotal, numOfTags * 2 * num);
//
// tsBufResetPos(pTSBuf1);
//
// int32_t count = 0;
// while (tsBufNextPos(pTSBuf1)) {
// STSElem elem = tsBufGetElem(pTSBuf1);
//
// if (count++ < numOfTags * num) {
// EXPECT_EQ(elem.id, 12);
// } else {
// EXPECT_EQ(elem.id, 77);
// }
//
// printf("%d-%" PRIu64 "-%" PRIu64 "\n", elem.id, elem.tag->i64, elem.ts);
// }
//
// tsBufDestroy(pTSBuf1);
// tsBufDestroy(pTSBuf2);
}
} // namespace
//TODO add binary tag value test case
TEST(testCase, tsBufTest) {
simpleTest();
largeTSTest();
multiTagsTest();
multiVnodeTagsTest();
loadDataTest();
invalidFileTest();
// simpleTest();
// largeTSTest();
// multiTagsTest();
// multiVnodeTagsTest();
// loadDataTest();
// invalidFileTest();
// randomIncTsTest();
TSTraverse();
mergeDiffVnodeBufferTest();
mergeIdenticalVnodeBufferTest();
// TSTraverse();
// mergeDiffVnodeBufferTest();
// mergeIdenticalVnodeBufferTest();
mergeMultiBlockFromOneGroupTest();
}
......@@ -270,7 +270,8 @@ void *rpcOpen(const SRpcInit *pInit) {
if (pInit->label) tstrncpy(pRpc->label, pInit->label, sizeof(pRpc->label));
pRpc->connType = pInit->connType;
if (pRpc->connType == TAOS_CONN_CLIENT) {
pRpc->numOfThreads = pInit->numOfThreads;
pRpc->numOfThreads =
pInit->numOfThreads > TSDB_MAX_RPC_THREADS * 2 ? TSDB_MAX_RPC_THREADS * 2 : pInit->numOfThreads;
} else {
pRpc->numOfThreads = pInit->numOfThreads > TSDB_MAX_RPC_THREADS ? TSDB_MAX_RPC_THREADS : pInit->numOfThreads;
}
......
###################################################################
# Copyright (c) 2021 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
from util.log import *
from util.cases import *
from util.sql import *
class TDTestCase:
def caseDescription(self):
'''
case1<shenglian zhou>: [TS-2523]join not supported on tables of different databases
'''
return
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self._conn = conn
def run(self):
print("running {}".format(__file__))
tdSql.execute("drop database if exists d1")
tdSql.execute("create database if not exists d1")
tdSql.execute('use d1')
tdSql.execute('create stable st(ts timestamp , value int ) tags (ind int)')
tdSql.execute('insert into tb1 using st tags(1) values(now ,1)')
tdSql.execute('insert into tb1 using st tags(1) values(now+1s ,2)')
tdSql.execute('insert into tb1 using st tags(1) values(now+2s ,3)')
tdSql.execute("drop database if exists d2")
tdSql.execute("create database if not exists d2")
tdSql.execute('use d2')
tdSql.execute('create stable st(ts timestamp , value int ) tags (ind int)')
tdSql.execute('insert into tb1 using st tags(1) values(now ,1)')
tdSql.execute('insert into tb1 using st tags(1) values(now+1s ,2)')
tdSql.execute('insert into tb1 using st tags(1) values(now+2s ,3)')
tdSql.error('select t1.value, t2.value from d1.st t1, d2.st t2 where t1.ts=t2.ts and t1.ind = t2.ind');
tdSql.execute('drop database d1')
tdSql.execute('drop database d2')
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
......@@ -19,10 +19,9 @@ from util.dnodes import *
class TDTestCase:
def caseDescription(self):
'''
"""
[TD-11510] taosBenchmark test cases
'''
return
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
......@@ -31,19 +30,26 @@ class TDTestCase:
def getPath(self, tool="taosBenchmark"):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
if "community" in selfPath:
projPath = selfPath[: selfPath.find("community")]
elif "src" in selfPath:
projPath = selfPath[: selfPath.find("src")]
elif "/tools/" in selfPath:
projPath = selfPath[: selfPath.find("/tools/")]
elif "/tests/" in selfPath:
projPath = selfPath[: selfPath.find("/tests/")]
else:
projPath = selfPath[:selfPath.find("tests")]
tdLog.info("cannot found %s in path: %s, use system's" % (tool, selfPath))
projPath = "/usr/local/taos/bin/"
paths = []
for root, dirs, files in os.walk(projPath):
if ((tool) in files):
for root, dummy, files in os.walk(projPath):
if (tool) in files:
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
if "packaging" not in rootRealPath:
paths.append(os.path.join(root, tool))
break
if (len(paths) == 0):
if len(paths) == 0:
tdLog.exit("taosBenchmark not found!")
return
else:
......@@ -51,31 +57,45 @@ class TDTestCase:
return paths[0]
def run(self):
tdSql.query("select client_version()")
client_ver = "".join(tdSql.queryResult[0])
major_ver = client_ver.split(".")[0]
binPath = self.getPath()
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/sml_json_alltypes.json" %binPath
cmd = "%s -f ./5-taos-tools/taosbenchmark/json/sml_json_alltypes.json" % binPath
tdLog.info("%s" % cmd)
os.system("%s" % cmd)
tdSql.execute("reset query cache")
tdSql.query("describe db.stb1")
tdSql.checkData(1, 1, "BOOL")
tdSql.query("describe db.stb2")
tdSql.checkData(1, 1, "TINYINT")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb3")
tdSql.checkData(1, 1, "SMALLINT")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb4")
tdSql.checkData(1, 1, "INT")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb5")
tdSql.checkData(1, 1, "BIGINT")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb6")
tdSql.checkData(1, 1, "FLOAT")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb7")
tdSql.checkData(1, 1, "DOUBLE")
tdSql.query("describe db.stb8")
tdSql.checkData(1, 1, "BINARY")
tdSql.checkData(1, 2, 8)
if major_ver == "3":
tdSql.checkData(1, 1, "VARCHAR")
tdSql.checkData(1, 2, 16)
else:
tdSql.checkData(1, 1, "NCHAR")
tdSql.checkData(1, 2, 8)
tdSql.query("describe db.stb9")
tdSql.checkData(1, 1, "NCHAR")
tdSql.checkData(1, 2, 8)
if major_ver == "3":
tdSql.checkData(1, 1, "VARCHAR")
tdSql.checkData(1, 2, 16)
else:
tdSql.checkData(1, 1, "NCHAR")
tdSql.checkData(1, 2, 8)
tdSql.query("select count(*) from db.stb1")
tdSql.checkData(0, 0, 160)
tdSql.query("select count(*) from db.stb2")
......@@ -101,4 +121,4 @@ class TDTestCase:
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
tdCases.addLinux(__file__, TDTestCase())
......@@ -4,7 +4,7 @@
290,,pytest,python3 test.py -f update/merge_commit_data.py
241,,pytest,python3 test.py -f update/merge_commit_data2.py
224,,pytest,python3 test.py -f query/queryNullValueTest.py
221,,pytest,python3 test.py -f query/nestedQuery/nestedQuery.py
#221,,pytest,python3 test.py -f query/nestedQuery/nestedQuery.py
220,,pytest,python3 test.py -f update/merge_commit_data2_update0.py
208,,pytest,python3 test.py -f update/merge_commit_last.py
203,,pytest,python3 test.py -f update/merge_commit_last-0.py
......@@ -191,13 +191,14 @@
38,,script,./test.sh -f unique/arbitrator/sync_replica2_dropDb.sim
38,,script,./test.sh -f general/table/delete_reuse2.sim
38,,script,./test.sh -f general/insert/insert_drop.sim
38,,script,./test.sh -f general/insert/insert_update2.sim
37,,script,./test.sh -f unique/mnode/mgmt30.sim
37,,script,./test.sh -f unique/column/replica3.sim
37,,script,./test.sh -f unique/arbitrator/offline_replica2_createTable_online.sim
37,,script,./test.sh -f unique/arbitrator/offline_replica2_alterTable_online.sim
37,,script,./test.sh -f general/table/delete_reuse1.sim
37,,script,./test.sh -f general/db/delete_reuse2.sim
36,,docs-examples-test,eval sh -c \"if [ `uname -m` != aarch64 ]; then ./test_rust.sh; fi\"
# 36,,docs-examples-test,eval sh -c \"if [ `uname -m` != aarch64 ]; then ./test_rust.sh; fi\"
36,,script,./test.sh -f unique/stable/replica3_vnode3.sim
36,,script,./test.sh -f unique/stable/dnode2_stop.sim
36,,script,./test.sh -f unique/arbitrator/sync_replica2_alterTable_add.sim
......@@ -561,10 +562,10 @@
11,,pytest,python3 test.py -f import_merge/importBlock1Sub.py
10,,docs-examples-test,./test_python.sh
10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonStmt-otherPara.py
10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonStmt.py
#10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJsonStmt.py
10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson-autoCreate.py
10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson-otherPara.py
10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson-childTable.py
#10,,system-test,python3 ./test.py -f 5-taos-tools/taosbenchmark/taosdemoTestInsertWithJson-childTable.py
10,,system-test,python3 ./test.py -f 2-query/TD-12344.py
10,,script,./test.sh -f unique/stable/replica2_vnode3.sim
10,,pytest,python3 testCompress.py
......@@ -641,6 +642,7 @@
6,,pytest,python3 test.py -f functions/function_elapsed.py
6,,pytest,python3 test.py -f alter/alterColMultiTimes.py
6,,develop-test,python3 ./test.py -f 2-query/ts_2016.py
6,,develop-test,python3 ./test.py -f 2-query/ts_2523.py
6,,develop-test,python3 ./test.py -f 2-query/escape.py
6,,develop-test,python3 ./test.py -f 2-query/TD-14763.py
5,,system-test,python3 ./test.py -f 4-taosAdapter/taosAdapter_insert.py
......
......@@ -50,6 +50,12 @@ mkdir -p /var/lib/taos/subscribe
cd $CONTAINER_TESTDIR/tests/$exec_dir
ulimit -c unlimited
#define taospy 2.7.3
pip3 list|grep taospy
pip3 uninstall taospy -y
pip3 install taospy==2.7.3
$TIMEOUT_CMD $cmd
RET=$?
......
......@@ -96,7 +96,6 @@ docker run \
-v $REPDIR/packaging:$CONTAINER_TESTDIR/packaging:ro \
-v $REPDIR/README.md:$CONTAINER_TESTDIR/README.md:ro \
-v $REPDIR/docs:$CONTAINER_TESTDIR/docs \
-v $REPDIR/src/connector/python/taos:/usr/local/lib/python3.8/site-packages/taos:ro \
-e LD_LIBRARY_PATH=/home/debug/build/lib:/home/debug/build/lib64 \
--rm --ulimit core=-1 taos_test:v1.0 $CONTAINER_TESTDIR/tests/parallel_test/run_case.sh -d "$exec_dir" -c "$cmd" $timeout_param
ret=$?
......
......@@ -49,8 +49,8 @@
"sample_format": "csv",
"sample_file": "./sample.csv",
"tags_file": "",
"columns": [{"type": "INT"}],
"tags": [{"type": "TINYINT", "count":1}]
"columns": [{"type": "INT"}, {"type": "DOUBLE", "count":10}, {"type": "BINARY", "len": 16, "count":3}, {"type": "BINARY", "len": 32, "count":6}],
"tags": [{"type": "TINYINT", "count":2}, {"type": "BINARY", "len": 16, "count":5}]
}]
}]
}
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
此差异已折叠。
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册