提交 936eb835 编写于 作者: S shenglian zhou

(other):Merge remote-tracking branch 'origin/develop' into szhou/feature/meta-sver-tver

......@@ -381,7 +381,7 @@ pipeline {
println gitlog
if (!(gitlog =~ /\((.*?)\)/)){
autoCancelled = true
error('Aborting the build.')
error('Please fill in the scope information correctly.\neg. [TD-xxxx]<fix>(query,insert):xxxxxxxxxxxxxxxxxx ')
}
temp = (gitlog =~ /\((.*?)\)/)
temp = temp[0].remove(1)
......
......@@ -83,7 +83,7 @@ TDengine是一个高效的存储、查询、分析时序大数据的平台,专
* [Windows客户端](https://www.taosdata.com/blog/2019/07/26/514.html):自行编译windows客户端,Windows环境的各种连接器都需要它
* [Rust Connector](/connector/rust): Rust语言下通过libtaos客户端或RESTful接口,连接TDengine服务器。
## [TDengine 组件与工具](/cn/documentation/)
## TDengine 组件与工具
* [taosAdapter 用户手册](/tools/adapter)
* [TDinsight 用户手册](/tools/insight)
......
......@@ -20,7 +20,7 @@ INSERT INTO d1001 VALUES (1538548685000, 10.3, 219, 0.31) (1538548695000, 12.6,
详细的SQL INSERT语法规则请见 [TAOS SQL 的数据写入](https://www.taosdata.com/cn/documentation/taos-sql#insert) 章节。
**Tips:**
**Tips:**
- 要提高写入效率,需要批量写入。一批写入的记录条数越多,插入效率就越高。但一条记录不能超过16K,一条SQL语句总长度不能超过1M 。
- TDengine支持多线程同时写入,要进一步提高写入速度,一个客户端需要打开20个以上的线程同时写。但线程数达到一定数量后,无法再提高,甚至还会下降,因为线程频繁切换,带来额外开销。
......@@ -56,7 +56,7 @@ tag_set 中的所有的数据自动转化为 nchar 数据类型,并不需要
* 对空格、等号(=)、逗号(,)、双引号("),前面需要使用反斜杠(\)进行转义。(都指的是英文半角符号)
* 数值类型将通过后缀来区分数据类型:
| **序号** | **后缀** | **映射类型** | **大小(字节)** |
| **序号** | **后缀** | **映射类型** | **大小(字节)** |
| -- | ------- | ---------| ------ |
| 1 | 无或f64 | double | 8 |
| 2 | f32 | float | 4 |
......@@ -231,16 +231,16 @@ prometheus产生的数据格式如下:
```json
{
Timestamp: 1576466279341,
Value: 37.000000,
Value: 37.000000,
apiserver_request_latencies_bucket {
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes",
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes",
scope="cluster",
verb="LIST",
version="v1"
verb="LIST",
version="v1"
}
}
```
......@@ -251,6 +251,7 @@ select * from apiserver_request_latencies_bucket;
```
## <a class="anchor" id="telegraf"></a> Telegraf 直接写入(通过 taosAdapter)
安装 Telegraf 请参考[官方文档](https://portal.influxdata.com/downloads/)
TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 Telegraf 的多种应用的数据写入。
......@@ -276,6 +277,7 @@ sudo systemctl start telegraf
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter)
安装 collectd,请参考[官方文档](https://collectd.org/download.shtml)
TDengine 新版本(2.3.0.0+)包含一个 taosAdapter 独立程序,负责接收包括 collectd 的多种应用的数据写入。
......@@ -294,6 +296,7 @@ sudo systemctl start collectd
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter)
安装 StatsD
请参考[官方文档](https://github.com/statsd/statsd)
......@@ -316,6 +319,30 @@ port: 8125
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
icinga2 可以收集监控和性能数据并写入 OpenTSDB,taosAdapter 可以支持接收 icinga2 的数据并写入到 TDengine 中。
## <a class="anchor" id="icinga2"></a> icinga2 直接写入(通过 taosAdapter)
* 参考链接 https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer 使能 opentsdb-writer
* 使能 taosAdapter 配置项 opentsdb_telnet.enable
* 修改配置文件 /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="tcollector"></a> TCollector 直接写入(通过 taosAdapter)
TCollector 是一个在客户侧收集本地收集器并发送数据到 OpenTSDB 的进程,taosAdaapter 可以支持接收 TCollector 的数据并写入到 TDengine 中。
使能 taosAdapter 配置项 opentsdb_telnet.enable
修改 TCollector 配置文件,修改 OpenTSDB 宿主机地址为 taosAdapter 被部署的地址,并修改端口号为 taosAdapter 使用的端口(默认6049)。
taosAdapter 相关配置参数请参考 taosadapter --help 命令输出以及相关文档。
## <a class="anchor" id="taosadapter2-telegraf"></a> 使用 Bailongma 2.0 接入 Telegraf 数据写入
......
......@@ -8,14 +8,14 @@ The following article explains how to quickly build a single-node TDengine runti
The Docker tools themselves can be downloaded from [Docker official site](https://docs.docker.com/get-docker/).
After installation, you can check the Docker version in the command line terminal. If the version number is output properly, the Docker environment has been installed successfully.
After installation, you can check the Docker version in the command-line terminal. If the version number is output properly, the Docker environment has been installed successfully.
```bash
$ docker -v
Docker version 20.10.3, build 48d30b5
```
## Using Docker to run TDengine
## How to use Docker to run TDengine
### running TDengine server inside Docker
......@@ -215,7 +215,7 @@ column[0]:FLOAT column[1]:INT column[2]:FLOAT
Press enter key to continue or Ctrl-C to stop
```
After enter, this command will automatically create a super table meters under the database test, there are 10,000 tables under this super table, the table name is "d0" to "d9999", each table has 10,000 records, each record has four fields (ts, current, voltage, phase), the time stamp is from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999", each table has a tag location and groupId, groupId is set from 1 to 10 and location is set to "beijing" or "shanghai".
After enter, this command will automatically create a super table `meters` under the database test, there are 10,000 tables under this super table, the table name is "d0" to "d9999", each table has 10,000 records, each record has four fields (ts, current, voltage, phase), the time stamp is from "2017-07-14 10:40:00 000" to "2017-07-14 10:40:09 999", each table has a tag location and groupId, groupId is set from 1 to 10 and location is set to "beijing" or "shanghai".
It takes about a few minutes to execute this command and ends up inserting a total of 100 million records.
......
......@@ -2,7 +2,7 @@
TDengine supports multiple ways to write data, including SQL, Prometheus, Telegraf, collectd, StatsD, EMQ MQTT Broker, HiveMQ Broker, CSV file, etc. Kafka, OPC and other interfaces will be provided in the future. Data can be inserted in one single record or in batches, data from one or multiple data collection points can be inserted at the same time. TDengine supports multi-thread insertion, out-of-order data insertion, and also historical data insertion.
## <a class="anchor" id="sql"></a> Data Writing via SQL
## <a class="anchor" id="sql"></a> Data Writing via SQL
Applications insert data by executing SQL insert statements through C/C++, Java, Go, C#, Python, Node.js Connectors, and users can manually enter SQL insert statements to insert data through TAOS Shell. For example, the following insert writes a record to table d1001:
......@@ -119,16 +119,16 @@ The format of generated data by Prometheus is as follows:
```json
{
Timestamp: 1576466279341,
Value: 37.000000,
Value: 37.000000,
apiserver_request_latencies_bucket {
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
component="apiserver",
instance="192.168.99.116:8443",
job="kubernetes-apiservers",
le="125000",
resource="persistentvolumes", s
cope="cluster",
verb="LIST",
version=“v1"
verb="LIST",
version=“v1"
}
}
```
......@@ -167,13 +167,13 @@ Now you can query the metrics data of Telegraf from TDengine.
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="collectd"></a> collectd 直接写入(通过 taosAdapter)
## <a class="anchor" id="collectd"></a> Data Writing via collectd and taosAdapter
Please refer to [official document](https://collectd.org/download.shtml) for collectd installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from collectd.
Configuration:
Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using.
Please add following words in /etc/collectd/collectd.conf. Please fill the value 'host' and 'port' with what the TDengine and taosAdapter using.
```
LoadPlugin network
<Plugin network>
......@@ -186,12 +186,12 @@ sudo systemctl start collectd
```
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="statsd"></a> StatsD 直接写入(通过 taosAdapter)
## <a class="anchor" id="statsd"></a> Data Writting via StatsD and taosAdapter
Please refer to [official document](https://github.com/statsd/statsd) for StatsD installation.
TDengine version 2.3.0.0+ includes a stand-alone application taosAdapter in charge of receive data insertion from StatsD.
Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using.
Please add following words in the config.js file. Please fill the value to 'host' and 'port' with what the TDengine and taosAdapter using.
```
add "./backends/repeater" to backends section.
add { host:'<TDengine server/cluster host>', port: <port for StatsD>} to repeater section.
......@@ -206,8 +206,30 @@ port: 8125
}
```
## <a class="anchor" id="cinga2"></a> Data Writting via icinga2 and taosAdapter
Use icinga2 to collect check result metrics and performance data
* Follow the doc to enable opentsdb-writer https://icinga.com/docs/icinga-2/latest/doc/14-features/#opentsdb-writer
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the configuration file /etc/icinga2/features-enabled/opentsdb.conf
```
object OpenTsdbWriter "opentsdb" {
host = "host to taosAdapter"
port = 6048
}
```
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="tcollector"></a> Data Writting via TCollector and taosAdapter
TCollector is a client-side process that gathers data from local collectors and pushes the data to OpenTSDB. You run it on all your hosts, and it does the work of sending each host’s data to the TSD (OpenTSDB backend process).
* Enable taosAdapter configuration opentsdb_telnet.enable
* Modify the TCollector configuration file, modify the OpenTSDB host to the host where taosAdapter is deployed, and modify the port to 6049
Please find taosAdapter configuration and usage from `taosadapter --help` output.
## <a class="anchor" id="taosadapter2-telegraf"></a> Insert data via Bailongma 2.0 and Telegraf
......
......@@ -199,7 +199,7 @@ function install_bin() {
[ -x ${install_main_dir}/bin/taos ] && ${csudo}ln -s ${install_main_dir}/bin/taos ${bin_link_dir}/taos || :
[ -x ${install_main_dir}/bin/taosd ] && ${csudo}ln -s ${install_main_dir}/bin/taosd ${bin_link_dir}/taosd || :
[ -x ${install_main_dir}/bin/taosadapter ] && ${csudo}ln -s ${install_main_dir}/bin/taosadapter ${bin_link_dir}/taosadapter || :
[ -x ${install_main_dir}/bin/taosdemo ] && ${csudo}ln -s ${install_main_dir}/bin/taosdemo ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/taosBenchmark ] && ${csudo}ln -s ${install_main_dir}/bin/taosBenchmark ${bin_link_dir}/taosdemo || :
[ -x ${install_main_dir}/bin/taosdump ] && ${csudo}ln -s ${install_main_dir}/bin/taosdump ${bin_link_dir}/taosdump || :
[ -x ${install_main_dir}/bin/remove.sh ] && ${csudo}ln -s ${install_main_dir}/bin/remove.sh ${bin_link_dir}/rmtaos || :
[ -x ${install_main_dir}/bin/set_core.sh ] && ${csudo}ln -s ${install_main_dir}/bin/set_core.sh ${bin_link_dir}/set_core || :
......
......@@ -123,9 +123,9 @@ if [ -n "${taostools_bin_files}" ]; then
mkdir -p ${taostools_install_dir}/bin \
&& cp ${taostools_bin_files} ${taostools_install_dir}/bin \
&& chmod a+x ${taostools_install_dir}/bin/* || :
[ -f ${taostools_install_dir}/bin/taosBenchmark ] && \
ln -sf ${taostools_install_dir}/bin/taosBenchmark \
${taostools_install_dir}/bin/taosdemo
# [ -f ${taostools_install_dir}/bin/taosBenchmark ] && \
# ln -sf ${taostools_install_dir}/bin/taosBenchmark \
# ${taostools_install_dir}/bin/taosdemo
if [ -f ${top_dir}/src/kit/taos-tools/packaging/tools/install-taostools.sh ]; then
cp ${top_dir}/src/kit/taos-tools/packaging/tools/install-taostools.sh \
......
......@@ -287,7 +287,7 @@ static void dnodeSendStatusMsg(void *handle, void *tmrId) {
dnodeGetCfg(&pStatus->dnodeId, pStatus->clusterId);
pStatus->dnodeId = htonl(dnodeGetDnodeId());
pStatus->version = htonl(tsVersion);
pStatus->version = htonl(tsVersion >> 8);
pStatus->lastReboot = htonl(tsRebootTime);
pStatus->numOfCores = htons((uint16_t) tsNumOfCores);
pStatus->diskAvailable = tsAvailDataDirGB;
......
Subproject commit 59f00a69f36b08cea86a70a22c29b2c27ef506ae
Subproject commit 5c944c1c93c63112890c8c65e1f57cfa6bd89179
......@@ -530,7 +530,7 @@ static int32_t mnodeProcessDnodeStatusMsg(SMnodeMsg *pMsg) {
pStatus->numOfCores = htons(pStatus->numOfCores);
uint32_t _version = htonl(pStatus->version);
if (_version != tsVersion) {
if (_version != tsVersion >> 8) {
pDnode = mnodeGetDnodeByEp(pStatus->dnodeEp);
if (pDnode != NULL && pDnode->status != TAOS_DN_STATUS_READY) {
pDnode->offlineReason = TAOS_DN_OFF_VERSION_NOT_MATCH;
......
......@@ -370,8 +370,11 @@ int32_t taosFsync(FileFd fd) {
}
HANDLE h = (HANDLE)_get_osfhandle(fd);
return FlushFileBuffers(h);
//If the function succeeds, the return value is nonzero.
//If the function fails, the return value is zero. To get extended error information, call GetLastError.
//The function fails if hFile is a handle to the console output. That is because the console output is not buffered. The function returns FALSE, and GetLastError returns ERROR_INVALID_HANDLE.
return FlushFileBuffers(h)-1;
}
int32_t taosRename(char *oldName, char *newName) {
......
......@@ -524,11 +524,11 @@ int32_t compareWStrPatternComp(const void* pLeft, const void* pRight) {
assert(varDataLen(pRight) <= TSDB_MAX_FIELD_LEN * TSDB_NCHAR_SIZE);
wchar_t *pattern = calloc(varDataLen(pRight) + 1, sizeof(wchar_t));
wchar_t *str = calloc(size + 1, sizeof(wchar_t));
char *pattern = calloc(varDataLen(pRight) + TSDB_NCHAR_SIZE, 1);
char *str = calloc(varDataLen(pLeft) + TSDB_NCHAR_SIZE, 1);
memcpy(pattern, varDataVal(pRight), varDataLen(pRight));
memcpy(str, varDataVal(pLeft), size * sizeof(wchar_t));
memcpy(str, varDataVal(pLeft), varDataLen(pLeft));
int32_t ret = WCSPatternMatch((uint32_t *)pattern, (uint32_t *)str, size, &pInfo);
......
......@@ -8,10 +8,9 @@
3. mkdir debug; cd debug; cmake ..; make ; sudo make install
4. pip install ../src/connector/python ; pip3 install
../src/connector/python
4. cd ../tests && pip3 install -r requirements.txt
5. pip install numpy; pip3 install numpy fabric2 psutil pandas(numpy is required only if you need to run querySort.py)
> Note: Both Python2 and Python3 are currently supported by the Python test
> framework. Since Python2 is no longer officially supported by Python Software
......
......@@ -40,6 +40,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
......@@ -40,6 +40,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
......@@ -41,6 +41,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
......@@ -41,6 +41,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
......@@ -40,6 +40,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
......@@ -40,6 +40,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......@@ -85,11 +86,11 @@ class TDTestCase:
os.system("rm -rf %s" % self.tmpdir)
os.makedirs(self.tmpdir)
os.system("%staosdump --databases db -o %s" % (binPath, self.tmpdir))
os.system("%staosdump --databases db -o %s -g" % (binPath, self.tmpdir))
tdSql.execute("drop database db")
os.system("%staosdump -i %s" % (binPath, self.tmpdir))
os.system("%staosdump -i %s -g" % (binPath, self.tmpdir))
tdSql.query("show databases")
tdSql.checkRows(1)
......
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<sdsang>: [TD-12655] taosdump supports unsigned big int
'''
return
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.tmpdir = "tmp"
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root) - len("/build/bin")]
break
return buildPath
def run(self):
tdSql.prepare()
tdSql.execute("drop database if exists db")
tdSql.execute("create database db days 11 keep 3649 blocks 8 ")
tdSql.execute("use db")
tdSql.execute(
"create table st(ts timestamp, c1 BIGINT UNSIGNED) tags(ubntag BIGINT UNSIGNED)")
tdSql.execute("create table t1 using st tags(0)")
tdSql.execute("insert into t1 values(1640000000000, 0)")
tdSql.execute("create table t2 using st tags(18446744073709551614)")
tdSql.execute("insert into t2 values(1640000000000, 18446744073709551614)")
tdSql.execute("create table t3 using st tags(NULL)")
tdSql.execute("insert into t3 values(1640000000000, NULL)")
# sys.exit(1)
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found in %s" % buildPath)
binPath = buildPath + "/build/bin/"
if not os.path.exists(self.tmpdir):
os.makedirs(self.tmpdir)
else:
print("directory exists")
os.system("rm -rf %s" % self.tmpdir)
os.makedirs(self.tmpdir)
os.system(
"%staosdump --databases db -o %s -T 1 -g" %
(binPath, self.tmpdir))
# sys.exit(1)
tdSql.execute("drop database db")
os.system("%staosdump -i %s -T 1 -g" % (binPath, self.tmpdir))
tdSql.query("show databases")
tdSql.checkRows(1)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 'st')
tdSql.query("show tables")
tdSql.checkRows(3)
tdSql.query("select * from st where ubntag = 0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 0)
tdSql.checkData(0, 2, 0)
tdSql.query("select * from st where ubntag = 18446744073709551614")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 18446744073709551614)
tdSql.checkData(0, 2, 18446744073709551614)
tdSql.query("select * from st where ubntag is null")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
tdSql.checkData(0, 1, None)
tdSql.checkData(0, 2, None)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<sdsang>: [TD-12526] taosdump supports unsigned int
'''
return
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.tmpdir = "tmp"
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root) - len("/build/bin")]
break
return buildPath
def run(self):
tdSql.prepare()
tdSql.execute("drop database if exists db")
tdSql.execute("create database db days 11 keep 3649 blocks 8 ")
tdSql.execute("use db")
tdSql.execute(
"create table st(ts timestamp, c1 INT UNSIGNED) tags(untag INT UNSIGNED)")
tdSql.execute("create table t1 using st tags(0)")
tdSql.execute("insert into t1 values(1640000000000, 0)")
tdSql.execute("create table t2 using st tags(4294967294)")
tdSql.execute("insert into t2 values(1640000000000, 4294967294)")
tdSql.execute("create table t3 using st tags(NULL)")
tdSql.execute("insert into t3 values(1640000000000, NULL)")
# sys.exit(1)
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found in %s" % buildPath)
binPath = buildPath + "/build/bin/"
if not os.path.exists(self.tmpdir):
os.makedirs(self.tmpdir)
else:
print("directory exists")
os.system("rm -rf %s" % self.tmpdir)
os.makedirs(self.tmpdir)
os.system(
"%staosdump --databases db -o %s -T 1 -g" %
(binPath, self.tmpdir))
# sys.exit(1)
tdSql.execute("drop database db")
os.system("%staosdump -i %s -T 1 -g" % (binPath, self.tmpdir))
tdSql.query("show databases")
tdSql.checkRows(1)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 'st')
tdSql.query("show tables")
tdSql.checkRows(3)
tdSql.query("select * from st where untag = 0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 0)
tdSql.checkData(0, 2, 0)
tdSql.query("select * from st where untag = 4294967294")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 4294967294)
tdSql.checkData(0, 2, 4294967294)
tdSql.query("select * from st where untag is null")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
tdSql.checkData(0, 1, None)
tdSql.checkData(0, 2, None)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<sdsang>: [TD-12526] taosdump supports unsigned small int
'''
return
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.tmpdir = "tmp"
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root) - len("/build/bin")]
break
return buildPath
def run(self):
tdSql.prepare()
tdSql.execute("drop database if exists db")
tdSql.execute("create database db days 11 keep 3649 blocks 8 ")
tdSql.execute("use db")
tdSql.execute(
"create table st(ts timestamp, c1 SMALLINT UNSIGNED) tags(usntag SMALLINT UNSIGNED)")
tdSql.execute("create table t1 using st tags(0)")
tdSql.execute("insert into t1 values(1640000000000, 0)")
tdSql.execute("create table t2 using st tags(65534)")
tdSql.execute("insert into t2 values(1640000000000, 65534)")
tdSql.execute("create table t3 using st tags(NULL)")
tdSql.execute("insert into t3 values(1640000000000, NULL)")
# sys.exit(1)
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found in %s" % buildPath)
binPath = buildPath + "/build/bin/"
if not os.path.exists(self.tmpdir):
os.makedirs(self.tmpdir)
else:
print("directory exists")
os.system("rm -rf %s" % self.tmpdir)
os.makedirs(self.tmpdir)
os.system(
"%staosdump --databases db -o %s -T 1 -g" %
(binPath, self.tmpdir))
# sys.exit(1)
tdSql.execute("drop database db")
os.system("%staosdump -i %s -T 1 -g" % (binPath, self.tmpdir))
tdSql.query("show databases")
tdSql.checkRows(1)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 'st')
tdSql.query("show tables")
tdSql.checkRows(3)
tdSql.query("select * from st where usntag = 0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 0)
tdSql.checkData(0, 2, 0)
tdSql.query("select * from st where usntag = 65534")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 65534)
tdSql.checkData(0, 2, 65534)
tdSql.query("select * from st where usntag is null")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
tdSql.checkData(0, 1, None)
tdSql.checkData(0, 2, None)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import subprocess
class TDTestCase:
def caseDescription(self):
'''
case1<sdsang>: [TD-12526] taosdump supports unsigned tiny int
'''
return
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.tmpdir = "tmp"
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root) - len("/build/bin")]
break
return buildPath
def run(self):
tdSql.prepare()
tdSql.execute("drop database if exists db")
tdSql.execute("create database db days 11 keep 3649 blocks 8 ")
tdSql.execute("use db")
tdSql.execute(
"create table st(ts timestamp, c1 TINYINT UNSIGNED) tags(utntag TINYINT UNSIGNED)")
tdSql.execute("create table t1 using st tags(0)")
tdSql.execute("insert into t1 values(1640000000000, 0)")
tdSql.execute("create table t2 using st tags(254)")
tdSql.execute("insert into t2 values(1640000000000, 254)")
tdSql.execute("create table t3 using st tags(NULL)")
tdSql.execute("insert into t3 values(1640000000000, NULL)")
# sys.exit(1)
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosdump not found!")
else:
tdLog.info("taosdump found in %s" % buildPath)
binPath = buildPath + "/build/bin/"
if not os.path.exists(self.tmpdir):
os.makedirs(self.tmpdir)
else:
print("directory exists")
os.system("rm -rf %s" % self.tmpdir)
os.makedirs(self.tmpdir)
os.system(
"%staosdump --databases db -o %s -T 1 -g" %
(binPath, self.tmpdir))
# sys.exit(1)
tdSql.execute("drop database db")
os.system("%staosdump -i %s -T 1 -g" % (binPath, self.tmpdir))
tdSql.query("show databases")
tdSql.checkRows(1)
tdSql.execute("use db")
tdSql.query("show stables")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 'st')
tdSql.query("show tables")
tdSql.checkRows(3)
tdSql.query("select * from st where utntag = 0")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 0)
tdSql.checkData(0, 2, 0)
tdSql.query("select * from st where utntag = 254")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 1640000000000)
tdSql.checkData(0, 1, 254)
tdSql.checkData(0, 2, 254)
tdSql.query("select * from st where utntag is null")
tdSql.checkRows(1)
tdSql.checkData(0, 0, 0)
tdSql.checkData(0, 1, None)
tdSql.checkData(0, 2, None)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeJson.py
\ No newline at end of file
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeBigInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeBool.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeDouble.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeFloat.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeJson.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeSmallInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeTinyInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeUnsignedBigInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeUnsignedInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeUnsignedSmallInt.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestTypeUnsignedTinyInt.py
......@@ -28,7 +28,7 @@ import taos
if __name__ == "__main__":
fileName = "all"
deployPath = ""
masterIp = ""
......@@ -55,7 +55,7 @@ if __name__ == "__main__":
tdLog.printNoPrefix('-w taos on windows')
sys.exit(0)
if key in ['-r', '--restart']:
if key in ['-r', '--restart']:
restart = True
if key in ['-f', '--file']:
......@@ -117,7 +117,7 @@ if __name__ == "__main__":
time.sleep(2)
tdLog.info('stop All dnodes')
if masterIp == "":
host = '127.0.0.1'
else:
......@@ -129,11 +129,11 @@ if __name__ == "__main__":
tdLog.info("Procedures for testing self-deployment")
td_clinet = TDSimClient("C:\\TDengine")
td_clinet.deploy()
remote_conn = Connection("root@%s"%host)
remote_conn = Connection("root@%s" % host)
with remote_conn.cd('/var/lib/jenkins/workspace/TDinternal/community/tests/pytest'):
remote_conn.run("python3 ./test.py")
conn = taos.connect(
host="%s"%(host),
host="%s" % (host),
config=td_clinet.cfgDir)
tdCases.runOneWindows(conn, fileName)
else:
......@@ -146,22 +146,21 @@ if __name__ == "__main__":
try:
if key_word in open(fileName).read():
is_test_framework = 1
except:
except BaseException:
pass
if is_test_framework:
moduleName = fileName.replace(".py", "").replace("/", ".")
uModule = importlib.import_module(moduleName)
try:
ucase = uModule.TDTestCase()
tdDnodes.deploy(1,ucase.updatecfgDict)
except :
tdDnodes.deploy(1,{})
tdDnodes.deploy(1, ucase.updatecfgDict)
except BaseException:
tdDnodes.deploy(1, {})
else:
pass
tdDnodes.deploy(1,{})
tdDnodes.start(1)
tdDnodes.deploy(1, {})
tdDnodes.start(1)
tdCases.logSql(logSql)
......@@ -179,18 +178,20 @@ if __name__ == "__main__":
if fileName == "all":
tdCases.runAllLinux(conn)
else:
tdCases.runOneWindows(conn, fileName)
tdCases.runOneLinux(conn, fileName)
if restart:
if fileName == "all":
tdLog.info("not need to query ")
else:
else:
sp = fileName.rsplit(".", 1)
if len(sp) == 2 and sp[1] == "py":
tdDnodes.stopAll()
tdDnodes.start(1)
time.sleep(1)
conn = taos.connect( host, config=tdDnodes.getSimCfgPath())
tdLog.info("Procedures for tdengine deployed in %s" % (host))
time.sleep(1)
conn = taos.connect(host, config=tdDnodes.getSimCfgPath())
tdLog.info(
"Procedures for tdengine deployed in %s" %
(host))
tdLog.info("query test after taosd restart")
tdCases.runOneLinux(conn, sp[0] + "_" + "restart.py")
else:
......
......@@ -45,8 +45,7 @@ class BuildDockerCluser:
"qdebugFlag":"135",
"maxSQLLength":"1048576"
}
cmd = "mkdir -p %s" % self.dockerDir
self.execCmd(cmd)
os.makedirs(self.dockerDir, exist_ok=True) # like "mkdir -p"
cmd = "cp *.yml %s" % self.dockerDir
self.execCmd(cmd)
......@@ -100,8 +99,7 @@ class BuildDockerCluser:
self.removeFile(self.dockerDir, i, self.dirs[2])
def createDir(self, rootDir, index, dir):
cmd = "mkdir -p %s/node%d/%s" % (rootDir, index, dir)
self.execCmd(cmd)
os.makedirs("%s/node%d/%s" % (rootDir, index, dir), exist_ok=True) # like "mkdir -p"
def createDirs(self):
for i in range(1, self.numOfNodes + 1):
......
......@@ -34,7 +34,8 @@ class TDTestCase:
path = tdDnodes.dnodes[1].getDnodeRootDir(1)
print(path)
tdLog.info("sudo mkdir -p %s/data/vnode/vnode2/wal/old" % path)
os.system("sudo mkdir -p %s/data/vnode/vnode2/wal/old" % path)
os.makedirs("%s/data/vnode/vnode2/wal/old" % path, exist_ok=True) # like "mkdir -p"
def run(self):
# os.system("rm -rf %s/ " % tdDnodes.getDnodesRootDir())
......
......@@ -54,11 +54,11 @@ if __name__ == "__main__":
tdLog.printNoPrefix('-w taos on windows')
sys.exit(0)
if key in ['-r', '--restart']:
if key in ['-r', '--restart']:
restart = True
if key in ['-f', '--file']:
fileName = value
fileName = os.path.normpath(value)
if key in ['-p', '--path']:
deployPath = value
......@@ -116,23 +116,48 @@ if __name__ == "__main__":
time.sleep(2)
tdLog.info('stop All dnodes')
if masterIp == "":
host = '127.0.0.1'
else:
host = masterIp
tdLog.info("Procedures for tdengine deployed in %s" % (host))
tdLog.info("Procedures for tdengine deployed in %s" % (host))
if windows:
tdCases.logSql(logSql)
tdLog.info("Procedures for testing self-deployment")
td_clinet = TDSimClient("C:\\TDengine")
td_clinet.deploy()
remote_conn = Connection("root@%s"%host)
with remote_conn.cd('/var/lib/jenkins/workspace/TDinternal/community/tests/pytest'):
remote_conn.run("python3 ./test.py")
if masterIp == "" or masterIp == "localhost":
tdDnodes.init(deployPath)
tdDnodes.setTestCluster(testCluster)
tdDnodes.setValgrind(valgrind)
tdDnodes.stopAll()
is_test_framework = 0
key_word = 'tdCases.addWindows'
try:
if key_word in open(fileName).read():
is_test_framework = 1
except:
pass
if is_test_framework:
moduleName = fileName.replace(".py", "").replace(os.sep, ".")
uModule = importlib.import_module(moduleName)
try:
ucase = uModule.TDTestCase()
tdDnodes.deploy(1,ucase.updatecfgDict)
except :
tdDnodes.deploy(1,{})
else:
pass
tdDnodes.deploy(1,{})
tdDnodes.startWin(1)
else:
remote_conn = Connection("root@%s"%host)
with remote_conn.cd('/var/lib/jenkins/workspace/TDinternal/community/tests/pytest'):
remote_conn.run("python3 ./test.py")
conn = taos.connect(
host="%s"%(host),
host="%s" % (host),
config=td_clinet.cfgDir)
tdCases.runOneWindows(conn, fileName)
else:
......@@ -145,23 +170,21 @@ if __name__ == "__main__":
try:
if key_word in open(fileName).read():
is_test_framework = 1
except:
except BaseException:
pass
if is_test_framework:
moduleName = fileName.replace(".py", "").replace("/", ".")
moduleName = fileName.replace(".py", "").replace(os.sep, ".")
uModule = importlib.import_module(moduleName)
try:
ucase = uModule.TDTestCase()
tdDnodes.deploy(1,ucase.updatecfgDict)
except :
tdDnodes.deploy(1,{})
tdDnodes.deploy(1, ucase.updatecfgDict)
except BaseException:
tdDnodes.deploy(1, {})
else:
pass
tdDnodes.deploy(1,{})
tdDnodes.deploy(1, {})
tdDnodes.start(1)
tdCases.logSql(logSql)
if testCluster:
......@@ -178,18 +201,20 @@ if __name__ == "__main__":
if fileName == "all":
tdCases.runAllLinux(conn)
else:
tdCases.runOneWindows(conn, fileName)
tdCases.runOneLinux(conn, fileName)
if restart:
if fileName == "all":
tdLog.info("not need to query ")
else:
else:
sp = fileName.rsplit(".", 1)
if len(sp) == 2 and sp[1] == "py":
tdDnodes.stopAll()
tdDnodes.start(1)
time.sleep(1)
conn = taos.connect( host, config=tdDnodes.getSimCfgPath())
tdLog.info("Procedures for tdengine deployed in %s" % (host))
time.sleep(1)
conn = taos.connect(host, config=tdDnodes.getSimCfgPath())
tdLog.info(
"Procedures for tdengine deployed in %s" %
(host))
tdLog.info("query test after taosd restart")
tdCases.runOneLinux(conn, sp[0] + "_" + "restart.py")
else:
......
......@@ -33,6 +33,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosd" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......@@ -49,7 +50,7 @@ class TDTestCase:
else:
tdLog.info("taosd found in %s" % buildPath)
binPath = buildPath + "/build/bin/"
taosdemoCmd = "%staosBenchmark -f tools/insert-interlace.json -PP 2>&1 | grep sleep | wc -l" % binPath
taosdemoCmd = "%staosBenchmark -f tools/insert-interlace.json -G 2>&1 | grep sleep | wc -l" % binPath
sleepTimes = subprocess.check_output(
taosdemoCmd, shell=True).decode("utf-8")
print("sleep times: %d" % int(sleepTimes))
......
......@@ -53,7 +53,7 @@ class TDCases:
# TODO: load all Linux cases here
runNum = 0
for tmp in self.linuxCases:
if tmp.name.find(fileName) != -1:
if tmp.name.find(os.path.normcase(fileName)) != -1:
case = testModule.TDTestCase()
case.init(conn)
case.run()
......@@ -68,7 +68,7 @@ class TDCases:
runNum = 0
for tmp in self.linuxCases:
if tmp.name.find(fileName) != -1:
if tmp.name.find(os.path.normcase(fileName)) != -1:
case = testModule.TDTestCase()
case.init(conn, self._logSql)
try:
......@@ -84,7 +84,7 @@ class TDCases:
# TODO: load all Windows cases here
runNum = 0
for tmp in self.windowsCases:
if tmp.name.find(fileName) != -1:
if tmp.name.find(os.path.normcase(fileName)) != -1:
case = testModule.TDTestCase()
case.init(conn)
case.run()
......@@ -118,7 +118,7 @@ class TDCases:
runNum = 0
for tmp in self.clusterCases:
if tmp.name.find(fileName) != -1:
if tmp.name.find(os.path.normcase(fileName)) != -1:
tdLog.notice("run cases like %s" % (fileName))
case = testModule.TDTestCase()
case.init()
......@@ -134,7 +134,7 @@ class TDCases:
runNum = 0
for tmp in self.clusterCases:
if tmp.name.find(fileName) != -1:
if tmp.name.find(os.path.normcase(fileName)) != -1:
tdLog.notice("run cases like %s" % (fileName))
case = testModule.TDTestCase()
case.init()
......
......@@ -73,17 +73,13 @@ class TDSimClient:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "rm -rf " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......@@ -149,17 +145,11 @@ class TDDnode:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.dataDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.dataDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......
......@@ -71,17 +71,13 @@ class TDSimClient:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "rm -rf " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......@@ -147,17 +143,11 @@ class TDDnode:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.dataDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.dataDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......
......@@ -71,17 +71,13 @@ class TDSimClient:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "rm -rf " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......@@ -147,17 +143,11 @@ class TDDnode:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.dataDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.dataDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......
......@@ -200,17 +200,11 @@ class TDDnode:
if os.system(cmd) != 0:
tdLog.exit(cmd)
cmd = "mkdir -p " + self.dataDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.dataDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.logDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.logDir, exist_ok=True) # like "mkdir -p"
cmd = "mkdir -p " + self.cfgDir
if os.system(cmd) != 0:
tdLog.exit(cmd)
os.makedirs(self.cfgDir, exist_ok=True) # like "mkdir -p"
cmd = "touch " + self.cfgPath
if os.system(cmd) != 0:
......@@ -231,7 +225,7 @@ class TDDnode:
# self.cfg("logDir",self.logDir)
# print(updatecfgDict)
isFirstDir = 1
if updatecfgDict[0] and updatecfgDict[0][0]:
if bool(updatecfgDict) and updatecfgDict[0] and updatecfgDict[0][0]:
print(updatecfgDict[0][0])
for key,value in updatecfgDict[0][0].items():
if value == 'dataDir' :
......@@ -332,6 +326,59 @@ class TDDnode:
# time.sleep(5)
def startWin(self):
buildPath = self.getBuildPath("taosd.exe")
if (buildPath == ""):
tdLog.exit("taosd.exe not found!")
else:
tdLog.info("taosd.exe found in %s" % buildPath)
binPath = buildPath + "/build/bin/taosd.exe"
taosadapterBinPath = buildPath + "/build/bin/taosadapter.exe"
if self.deployed == 0:
tdLog.exit("dnode:%d is not deployed" % (self.index))
cmd = "mintty -h never -w hide %s -c %s" % (
binPath, self.cfgDir)
taosadapterCmd = "mintty -h never -w hide %s " % (
taosadapterBinPath)
if os.system(taosadapterCmd) != 0:
tdLog.exit(taosadapterCmd)
if os.system(cmd) != 0:
tdLog.exit(cmd)
self.running = 1
tdLog.debug("dnode:%d is running with %s " % (self.index, cmd))
if self.valgrind == 0:
time.sleep(0.1)
key = 'from offline to online'
bkey = bytes(key,encoding="utf8")
logFile = self.logDir + "/taosdlog.0"
i = 0
while not os.path.exists(logFile):
sleep(0.1)
i += 1
if i>50:
break
popen = subprocess.Popen('tail -n +0 -f ' + logFile, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
pid = popen.pid
# print('Popen.pid:' + str(pid))
timeout = time.time() + 60*2
while True:
line = popen.stdout.readline().strip()
if bkey in line:
popen.kill()
break
if time.time() > timeout:
tdLog.exit('wait too long for taosd start')
tdLog.debug("the dnode:%d has been started." % (self.index))
else:
tdLog.debug("wait 10 seconds for the dnode:%d to start." % (self.index))
time.sleep(10)
def startWithoutSleep(self):
buildPath = self.getBuildPath()
......@@ -547,6 +594,10 @@ class TDDnodes:
def start(self, index):
self.check(index)
self.dnodes[index - 1].start()
def startWin(self, index):
self.check(index)
self.dnodes[index - 1].startWin()
def startWithoutSleep(self, index):
self.check(index)
......
......@@ -27,7 +27,7 @@ class TDTestCase:
def createOldDir(self):
oldDir = tdDnodes.getDnodesRootDir() + "dnode1/data/vnode/vnode2/wal/old"
os.system("sudo mkdir -p %s" % oldDir)
os.makedirs(oldDir, exist_ok=True) # like "mkdir -p"
def createOldDirAndAddWal(self):
oldDir = tdDnodes.getDnodesRootDir() + "dnode1/data/vnode/vnode2/wal/old"
......
../src/connector/python
numpy
fabric2
psutil
pandas
\ No newline at end of file
###################################################################
# Copyright (c) 2020 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
from posixpath import split
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import subprocess
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
for root, dirs, files in os.walk(projPath):
if ("taosd" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root) - len("/build/bin")]
break
return buildPath
def caseDescription(self):
'''
case1 <wenzhouwww>: [TD-12275] :
this test case is an long query crash for elapsed function .
'''
return
def run(self):
tdSql.prepare()
build_path = self.getBuildPath()+"/build/bin/"
prepare_cmd = "%staosBenchmark -t 100 -n 100000 -S 10000 -y " % (build_path)
# only taos -s for shell can generate this issue
print(prepare_cmd)
_ = subprocess.check_output(prepare_cmd, shell=True).decode("utf-8")
cmd1 = "taos -s 'select elapsed(ts) from test.meters interval(10s) sliding(5s) group by tbname' "
print(cmd1)
_ = subprocess.check_output(cmd1, shell=True).decode("utf-8")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
......@@ -44,6 +44,7 @@ class TDTestCase:
else:
projPath = selfPath[:selfPath.find("tests")]
buildPath = ""
for root, dirs, files in os.walk(projPath):
if ("taosdump" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
......
python3 ./test.py -f 5-taos-tools/basic.py
python3 ./test.py -f 5-taos-tools/TD-12478.py
python3 ./test.py -f 5-taos-tools/dump_col_tag.py
python3 ./test.py -f 5-taos-tools/taosdump/taosdumpTestColTag.py
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册