Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
CSDN产品汪
TDengine
提交
22f0046f
T
TDengine
项目概览
CSDN产品汪
/
TDengine
与 Fork 源项目一致
Fork自
taosdata / TDengine
通知
5
Star
0
Fork
0
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
0
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
T
TDengine
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
0
Issue
0
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
体验新版 GitCode,发现更多精彩内容 >>
提交
22f0046f
编写于
5月 07, 2020
作者:
H
hzcheng
浏览文件
操作
浏览文件
下载
差异文件
Merge branch 'develop' into feature/2.0tsdb
上级
03f2ca28
0c68821a
变更
5
隐藏空白更改
内联
并排
Showing
5 changed file
with
255 addition
and
7 deletion
+255
-7
README.md
README.md
+245
-0
src/connector/python/linux/python3/taos/cinterface.py
src/connector/python/linux/python3/taos/cinterface.py
+4
-1
src/mnode/src/mgmtDServer.c
src/mnode/src/mgmtDServer.c
+2
-2
src/mnode/src/mgmtUser.c
src/mnode/src/mgmtUser.c
+3
-3
src/tsdb/src/tsdbFile.c
src/tsdb/src/tsdbFile.c
+1
-1
未找到文件。
README.md
浏览文件 @
22f0046f
...
...
@@ -115,6 +115,251 @@ TDengine provides abundant developing tools for users to develop on TDengine. Fo
-
[
RESTful API
](
https://www.taosdata.com/en/documentation/connector/#RESTful-Connector
)
-
[
Node.js
](
https://www.taosdata.com/en/documentation/connector/#Node.js-Connector
)
# How to run the test cases and how to add a new test case?
### Prepare development environment
1.
sudo apt install
build-essential cmake net-tools python-pip python-setuptools python3-pip
python3-setuptools valgrind
2.
git clone
<https://github.com/taosdata/TDengine>
; cd TDengine
3.
mkdir debug; cd debug; cmake ..; make ; sudo make install
4.
pip install src/connector/python/linux/python2 ; pip3 install
src/connector/python/linux/python3
### How to run TSIM test suite
1.
cd
\<
TDengine
\>
/tests/script
2.
sudo ./test.sh
### How to run Python test suite
1.
cd
\<
TDengine
\>
/tests/pytest
2.
./smoketest.sh
\#
for smoke test
3.
./smoketest.sh -g
\#
for memory leak detection test with valgrind
4.
./fulltest.sh
\#
for full test
> Note1: TDengine daemon's configuration and data files are stored in
> \<TDengine\>/sim directory. As a historical design, it's same place with
> TSIM script. So after the TSIM script ran with sudo privilege, the directory
> has been used by TSIM then the python script cannot write it by a normal
> user. You need to remove the directory completely first before running the
> Python test case. We should consider using two different locations to store
> for TSIM and Python script.
> Note2: if you need to debug crash problem with a core dump, you need
> manually edit smoketest.sh or fulltest.sh to add "ulimit -c unlimited"
> before the script line. Then you can look for the core file in
> \<TDengine\>/tests/pytest after the program crash.
### How to add a new test case
**1. add a new TSIM test cases:**
TSIM test cases are now included in the new development branch and can be
added to the TDengine/tests/script/test.sh script based on the manual test
methods necessary to add test cases as described above.
**2. add a new Python test cases:**
**
2.1 Please refer to
\<
TDengine
\>
/tests/pytest/insert/basic.py to add a new
test case.
**
The new test case must implement 3 functions, where self.init()
and self.stop() simply copy the contents of insert/basic.py and the test
logic is implemented in self.run(). You can refer to the code in the util
directory for more information.
**2.2 Edit smoketest.sh to add the path and filename of the new test case**
Note: The Python test framework may continue to be improved in the future,
hopefully, to provide more functionality and ease of writing test cases. The
method of writing the test case above does not exclude that it will also be
affected.
**2.3 What test.py does in detail:**
test.py is the entry program for test case execution and monitoring.
test.py has the following functions.
\-
f --file, Specifies the test case file name to be executed
-p --path, Specifies deployment path
\-
m --master, Specifies the master server IP for cluster deployment
-c--cluster, test cluster function
-s--stop, terminates all running nodes
\-
g--valgrind, load valgrind for memory leak detection test
\-
h--help, display help
**2.4 What util/log.py does in detail:**
log.py is quite simple, the main thing is that you can print the output in
different colors as needed. The success() should be called for successful
test case execution and the success() will print green text. The exit() will
print red text and exit the program, exit() should be called for test
failure.
**util/log.py**
...
def info(self, info):
printf("%s %s" % (datetime.datetime.now(), info))
def sleep(self, sec):
printf("%s sleep %d seconds" % (datetime.datetime.now(), sec))
time.sleep(sec)
def debug(self, err):
printf("
\\
033
[
1;36m%s %s\\033[0m" % (datetime.datetime.now(), err))
def success(self, info):
printf("\\033[1;32m%s %s\\033[0m" % (datetime.datetime.now(), info))
def notice(self, err):
printf("\\033[1;33m%s %s\\033[0m" % (datetime.datetime.now(), err))
def exit(self, err):
printf("\\033[1;31m%s %s\\033[0m" % (datetime.datetime.now(), err))
sys.exit(1)
def printNoPrefix(self, info):
printf("\\033[1;36m%s\\033[0m" % (info)
...
**2.5 What util/sql.py does in detail:**
SQL.py is mainly used to execute SQL statements to manipulate the database,
and the code is extracted and commented as follows:
**util/sql.py**
\# prepare() is mainly used to set up the environment for testing table and
data, and to set up the database db for testing. do not call prepare() if you
need to test the database operation command.
def prepare(self):
tdLog.info("prepare database:db")
self.cursor.execute('reset query cache')
self.cursor.execute('drop database if exists db')
self.cursor.execute('create database db')
self.cursor.execute('use db')
...
\# query() is mainly used to execute select statements for normal syntax input
def query(self, sql):
...
\# error() is mainly used to execute the select statement with the wrong syntax
input, the error will be caught as a reasonable behavior, if not caught it will
prove that the test failed
def error()
...
\# checkRows() is used to check the number of returned lines after calling
query(select ...) after calling the query(select ...) to check the number of
rows of returned results.
def checkRows(self, expectRows):
...
\# checkData() is used to check the returned result data after calling
query(select ...) after the query(select ...) is called, failure to meet
expectation is
def checkData(self, row, col, data):
...
\# getData() returns the result data after calling query(select ...) to return
the resulting data after calling query(select ...)
def getData(self, row, col):
...
\# execute() used to execute sql and return the number of affected rows
def execute(self, sql):
...
\# executeTimes() Multiple executions of the same sql statement
def executeTimes(self, sql, times):
...
\# CheckAffectedRows() Check if the number of affected rows is as expected
def checkAffectedRows(self, expectAffectedRows):
...
> Note: Both Python2 and Python3 are currently supported by the Python test
> case. Since Python2 is no longer officially supported by January 1, 2020, it
> is recommended that subsequent test case development be guaranteed to run
> correctly on Python3. For Python2, please consider being compatible if
> appropriate without additional
> burden. <https://nakedsecurity.sophos.com/2020/01/03/python-is-dead-long-live-python/>
### CI Covenant submission adoption principle.
- Every commit / PR compilation must pass. Currently, the warning is treated
as an error, so the warning must also be resolved.
- Test cases that already exist must pass.
- Because CI is very important to support build and automatically test
procedure, it is necessary to manually test the test case before adding it
and do as many iterations as possible to ensure that the test case provides
stable and reliable test results when added.
> Note: In the future, according to the requirements and test development
> progress will add stress testing, performance testing, code style,
> and other features based on functional testing.
### Third Party Connectors
The TDengine community has also kindly built some of their own connectors! Follow the links below to find the source code for them.
...
...
src/connector/python/linux/python3/taos/cinterface.py
浏览文件 @
22f0046f
...
...
@@ -146,6 +146,7 @@ class CTaosInterface(object):
libtaos
.
taos_errstr
.
restype
=
ctypes
.
c_char_p
libtaos
.
taos_subscribe
.
restype
=
ctypes
.
c_void_p
libtaos
.
taos_consume
.
restype
=
ctypes
.
c_void_p
libtaos
.
taos_fetch_lengths
.
restype
=
ctypes
.
c_void_p
def
__init__
(
self
,
config
=
None
):
'''
...
...
@@ -314,6 +315,8 @@ class CTaosInterface(object):
isMicro
=
(
CTaosInterface
.
libtaos
.
taos_result_precision
(
result
)
==
FieldType
.
C_TIMESTAMP_MICRO
)
blocks
=
[
None
]
*
len
(
fields
)
fieldL
=
CTaosInterface
.
libtaos
.
taos_fetch_lengths
(
result
)
fieldLen
=
[
ele
for
ele
in
ctypes
.
cast
(
fieldL
,
ctypes
.
POINTER
(
ctypes
.
c_int
))[:
len
(
fields
)]]
for
i
in
range
(
len
(
fields
)):
data
=
ctypes
.
cast
(
pblock
,
ctypes
.
POINTER
(
ctypes
.
c_void_p
))[
i
]
if
data
==
None
:
...
...
@@ -323,7 +326,7 @@ class CTaosInterface(object):
if
fields
[
i
][
'type'
]
not
in
_CONVERT_FUNC
:
raise
DatabaseError
(
"Invalid data type returned from database"
)
blocks
[
i
]
=
_CONVERT_FUNC
[
fields
[
i
][
'type'
]](
data
,
num_of_rows
,
field
s
[
i
][
'bytes'
],
isMicro
)
blocks
[
i
]
=
_CONVERT_FUNC
[
fields
[
i
][
'type'
]](
data
,
num_of_rows
,
field
Len
[
i
],
isMicro
)
return
blocks
,
abs
(
num_of_rows
)
...
...
src/mnode/src/mgmtDServer.c
浏览文件 @
22f0046f
...
...
@@ -52,14 +52,14 @@ int32_t mgmtInitDServer() {
rpcInit
.
idleTime
=
tsShellActivityTimer
*
1000
;
rpcInit
.
afp
=
mgmtDServerRetrieveAuth
;
tsMgmtDServerQhandle
=
taosInitScheduler
(
tsMaxShellConns
,
1
,
"MS"
);
tsMgmtDServerRpc
=
rpcOpen
(
&
rpcInit
);
if
(
tsMgmtDServerRpc
==
NULL
)
{
mError
(
"failed to init server connection to dnode"
);
return
-
1
;
}
tsMgmtDServerQhandle
=
taosInitScheduler
(
tsMaxShellConns
,
1
,
"MS"
);
mPrint
(
"server connection to dnode is opened"
);
return
0
;
}
...
...
src/mnode/src/mgmtUser.c
浏览文件 @
22f0046f
...
...
@@ -396,7 +396,7 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) {
code
=
mgmtUpdateUser
(
pUser
);
mLPrint
(
"user:%s, password is altered by %s, result:%s"
,
pUser
->
user
,
pOperUser
->
user
,
tstrerror
(
code
));
}
else
{
mError
(
"user:%s, no rights to ater user"
,
pOperUser
->
user
);
mError
(
"user:%s, no rights to a
l
ter user"
,
pOperUser
->
user
);
code
=
TSDB_CODE_NO_RIGHTS
;
}
...
...
@@ -439,13 +439,13 @@ static void mgmtProcessAlterUserMsg(SQueuedMsg *pMsg) {
code
=
mgmtUpdateUser
(
pUser
);
mLPrint
(
"user:%s, privilege is altered by %s, result:%s"
,
pUser
->
user
,
pOperUser
->
user
,
tstrerror
(
code
));
}
else
{
mError
(
"user:%s, no rights to ater user"
,
pOperUser
->
user
);
mError
(
"user:%s, no rights to a
l
ter user"
,
pOperUser
->
user
);
code
=
TSDB_CODE_NO_RIGHTS
;
}
mgmtSendSimpleResp
(
pMsg
->
thandle
,
code
);
}
else
{
mError
(
"user:%s, no rights to ater user"
,
pOperUser
->
user
);
mError
(
"user:%s, no rights to a
l
ter user"
,
pOperUser
->
user
);
mgmtSendSimpleResp
(
pMsg
->
thandle
,
TSDB_CODE_NO_RIGHTS
);
}
...
...
src/tsdb/src/tsdbFile.c
浏览文件 @
22f0046f
...
...
@@ -180,7 +180,7 @@ void tsdbFitRetention(STsdbRepo *pRepo) {
int
mfid
=
tsdbGetKeyFileId
(
taosGetTimestamp
(
pRepo
->
config
.
precision
),
pRepo
->
config
.
daysPerFile
,
pRepo
->
config
.
precision
);
while
(
pGroup
[
0
].
fileId
<
mfid
)
{
while
(
p
FileH
->
numOfFGroups
>
0
&&
p
Group
[
0
].
fileId
<
mfid
)
{
tsdbRemoveFileGroup
(
pFileH
,
pGroup
[
0
].
fileId
);
}
}
...
...
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录