Skip to content

T
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1192
Star 22018
Fork 4786
T
TDengine
未验证 提交 59790c4d 编写于 作者: W wu champion 提交者： GitHub
浏览文件
操作

Merge pull request #7 from wu-champion/develop 

更新分支1380
上级 2f0f9006 fbed83c4
展开全部 隐藏空白更改
内联 并排
Showing with 2931 addition and 2496 deletion
README-CN.md 0 → 100644
浏览文件 @ 59790c4d
[![Build Status](https://travis-ci.org/taosdata/TDengine.svg?branch=master)](https://travis-ci.org/taosdata/TDengine)
[![Build status](https://ci.appveyor.com/api/projects/status/kf3pwh2or5afsgl9/branch/master?svg=true)](https://ci.appveyor.com/project/sangshuduo/tdengine-2n8ge/branch/master)
[![Coverage Status](https://coveralls.io/repos/github/taosdata/TDengine/badge.svg?branch=develop)](https://coveralls.io/github/taosdata/TDengine?branch=develop)
[![CII Best Practices](https://bestpractices.coreinfrastructure.org/projects/4201/badge)](https://bestpractices.coreinfrastructure.org/projects/4201)
[![tdengine](https://snapcraft.io//tdengine/badge.svg)](https://snapcraft.io/tdengine)
[![TDengine](TDenginelogo.png)](https://www.taosdata.com)
简体中文 | [English](./README.md)
# TDengine 简介
TDengine是涛思数据专为物联网、车联网、工业互联网、IT运维等设计和优化的大数据平台。除核心的快10倍以上的时序数据库功能外，还提供缓存、数据订阅、流式计算等功能，最大程度减少研发和运维的复杂度，且核心代码，包括集群功能全部开源（开源协议，AGPL v3.0）。
- 10 倍以上性能提升。定义了创新的数据存储结构，单核每秒就能处理至少2万次请求，插入数百万个数据点，读出一千万以上数据点，比现有通用数据库快了十倍以上。
- 硬件或云服务成本降至1/5。由于超强性能，计算资源不到通用大数据方案的1/5；通过列式存储和先进的压缩算法，存储空间不到通用数据库的1/10。
- 全栈时序数据处理引擎。将数据库、消息队列、缓存、流式计算等功能融合一起，应用无需再集成Kafka/Redis/HBase/Spark等软件，大幅降低应用开发和维护成本。
- 强大的分析功能。无论是十年前还是一秒钟前的数据，指定时间范围即可查询。数据可在时间轴上或多个设备上进行聚合。即席查询可通过Shell/Python/R/Matlab随时进行。
- 与第三方工具无缝连接。不用一行代码，即可与Telegraf, Grafana, EMQ X, Prometheus, Matlab, R集成。后续还将支持MQTT, OPC, Hadoop，Spark等, BI工具也将无缝连接。
- 零运维成本、零学习成本。安装、集群一秒搞定，无需分库分表，实时备份。标准SQL，支持JDBC,RESTful，支持Python/Java/C/C++/Go/Node.JS, 与MySQL相似，零学习成本。
# 文档
TDengine是一个高效的存储、查询、分析时序大数据的平台，专为物联网、车联网、工业互联网、运维监测等优化而设计。您可以像使用关系型数据库MySQL一样来使用它，但建议您在使用前仔细阅读一遍下面的文档，特别是 [数据模型](https://www.taosdata.com/cn/documentation/architecture) 与 [数据建模](https://www.taosdata.com/cn/documentation/model)。除本文档之外，欢迎 [下载产品白皮书](https://www.taosdata.com/downloads/TDengine%20White%20Paper.pdf)。
# 生成
TDengine目前2.0版服务器仅能在Linux系统上安装和运行，后续会支持Windows、macOS等系统。客户端可以在Windows或Linux上安装和运行。任何OS的应用也可以选择RESTful接口连接服务器taosd。CPU支持X64/ARM64/MIPS64/Alpha64，后续会支持ARM32、RISC-V等CPU架构。用户可根据需求选择通过[源码](https://www.taosdata.com/cn/getting-started/#通过源码安装)或者[安装包](https://www.taosdata.com/cn/getting-started/#通过安装包安装)来安装。本快速指南仅适用于通过源码安装。
## 安装工具
### Ubuntu 16.04 及以上版本 & Debian：
```bash
sudo apt-get install -y gcc cmake build-essential git
```
### Ubuntu 14.04：
```bash
sudo apt-get install -y gcc cmake3 build-essential git binutils-2.26
export PATH=/usr/lib/binutils-2.26/bin:$PATH
```
编译或打包 JDBC 驱动源码，需安装 Java JDK 8 或以上版本和 Apache Maven 2.7 或以上版本。
安装 OpenJDK 8：
```bash
sudo apt-get install -y openjdk-8-jdk
```
安装 Apache Maven：
```bash
sudo apt-get install -y maven
```
### CentOS 7：
```bash
sudo yum install -y gcc gcc-c++ make cmake git
```
安装 OpenJDK 8：
```bash
sudo yum install -y java-1.8.0-openjdk
```
安装 Apache Maven：
```bash
sudo yum install -y maven
```
### CentOS 8 & Fedora:
```bash
sudo dnf install -y gcc gcc-c++ make cmake epel-release git
```
安装 OpenJDK 8：
```bash
sudo dnf install -y java-1.8.0-openjdk
```
安装 Apache Maven：
```bash
sudo dnf install -y maven
```
## 获取源码
首先，你需要从 GitHub 克隆源码：
```bash
git clone https://github.com/taosdata/TDengine.git
cd TDengine
```
Go 连接器和 Grafana 插件在其他独立仓库，如果安装它们的话，需要在 TDengine 目录下通过此命令安装：
```bash
git submodule update --init --recursive
```
## 生成 TDengine
### Linux 系统
```bash
mkdir debug && cd debug
cmake .. && cmake --build .
```
在X86-64、X86、arm64 和 arm32 平台上，TDengine 生成脚本可以自动检测机器架构。也可以手动配置 CPUTYPE 参数来指定 CPU 类型，如 aarch64 或 aarch32 等。
aarch64：
```bash
cmake .. -DCPUTYPE=aarch64 && cmake --build .
```
aarch32：
```bash
cmake .. -DCPUTYPE=aarch32 && cmake --build .
```
### Windows 系统
如果你使用的是 Visual Studio 2013 版本：
打开 cmd.exe，执行 vcvarsall.bat 时，为 64 位操作系统指定“x86_amd64”，为 32 位操作系统指定“x86”。
```bash
mkdir debug && cd debug
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\VC\vcvarsall.bat" < x86_amd64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
如果你使用的是 Visual Studio 2019 或 2017 版本：
打开cmd.exe，执行 vcvarsall.bat 时，为 64 位操作系统指定“x64”，为 32 位操作系统指定“x86”。
```bash
mkdir debug && cd debug
"c:\Program Files (x86)\Microsoft Visual Studio\2019\Community\VC\Auxiliary\Build\vcvarsall.bat" < x64 | x86 >
cmake .. -G "NMake Makefiles"
nmake
```
你也可以从开始菜单中找到"Visual Studio < 2019 | 2017 >"菜单项，根据你的系统选择"x64 Native Tools Command Prompt for VS < 2019 | 2017 >"或"x86 Native Tools Command Prompt for VS < 2019 | 2017 >"，打开命令行窗口，执行：
```bash
mkdir debug && cd debug
cmake .. -G "NMake Makefiles"
nmake
```
### Mac OS X 系统
安装 Xcode 命令行工具和 cmake. 在 Catalina 和 Big Sur 操作系统上，需要安装 XCode 11.4+ 版本。
```bash
mkdir debug && cd debug
cmake .. && cmake --build .
```
# 安装
如果你不想安装，可以直接在shell中运行。生成完成后，安装 TDengine：
```bash
make install
```
用户可以在[文件目录结构](https://www.taosdata.com/cn/documentation/administrator#directories)中了解更多在操作系统中生成的目录或文件。
安装成功后，在终端中启动 TDengine 服务：
```bash
taosd
```
用户可以使用 TDengine Shell 来连接 TDengine 服务，在终端中，输入：
```bash
taos
```
如果 TDengine Shell 连接服务成功，将会打印出欢迎消息和版本信息。如果失败，则会打印出错误消息。
## 快速运行
TDengine 生成后，在终端执行以下命令：
```bash
./build/bin/taosd -c test/cfg
```
在另一个终端，使用 TDengine shell 连接服务器：
```bash
./build/bin/taos -c test/cfg
```
"-c test/cfg"指定系统配置文件所在目录。
# 体验 TDengine
在TDengine终端中，用户可以通过SQL命令来创建/删除数据库、表等，并进行插入查询操作。
```bash
create database demo;
use demo;
create table t (ts timestamp, speed int);
insert into t values ('2019-07-15 00:00:00', 10);
insert into t values ('2019-07-15 01:00:00', 20);
select * from t;
ts | speed |
===================================
19-07-15 00:00:00.000| 10|
19-07-15 01:00:00.000| 20|
Query OK, 2 row(s) in set (0.001700s)
```
# 应用开发
## 官方连接器
TDengine 提供了丰富的应用程序开发接口，其中包括C/C++、Java、Python、Go、Node.js、C# 、RESTful 等，便于用户快速开发应用：
- Java
- C/C++
- Python
- Go
- RESTful API
- Node.js
## 第三方连接器
TDengine 社区生态中也有一些非常友好的第三方连接器，可以通过以下链接访问它们的源码。
- [Rust Connector](https://github.com/taosdata/TDengine/tree/master/tests/examples/rust)
- [.Net Core Connector](https://github.com/maikebing/Maikebing.EntityFrameworkCore.Taos)
- [Lua Connector](https://github.com/taosdata/TDengine/tree/develop/tests/examples/lua)
# 运行和添加测试例
TDengine 的测试框架和所有测试例全部开源。
点击[这里](tests/How-To-Run-Test-And-How-To-Add-New-Test-Case.md)，了解如何运行测试例和添加新的测试例。
# 成为社区贡献者
点击[这里](https://www.taosdata.com/cn/contributor/)，了解如何成为 TDengine 的贡献者。
#加入技术交流群
TDengine官方社群「物联网大数据群」对外开放，欢迎您加入讨论。搜索微信号 "tdengine"，加小T为好友，即可入群。
README.md
浏览文件 @ 59790c4d
......@@ -6,6 +6,8 @@
[![TDengine](TDenginelogo.png)](https://www.taosdata.com)
English | [简体中文](./README-CN.md)
# What is TDengine？
TDengine is an open-sourced big data platform under [GNU AGPL v3.0](http://www.gnu.org/licenses/agpl-3.0.html), designed and optimized for the Internet of Things (IoT), Connected Cars, Industrial IoT, and IT Infrastructure and Application Monitoring. Besides the 10x faster time-series database, it provides caching, stream computing, message queuing and other functionalities to reduce the complexity and cost of development and operation.
......@@ -160,24 +162,13 @@ mkdir debug && cd debug
cmake .. && cmake --build .
```
# Quick Run
# Quick Run
To quickly start a TDengine server after building, run the command below in terminal:
```bash
./build/bin/taosd -c test/cfg
```
In another terminal, use the TDengine shell to connect the server:
```bash
./build/bin/taos -c test/cfg
```
option "-c test/cfg" specifies the system configuration file directory.
# Installing
After building successfully, TDengine can be installed by:
```bash
sudo make install
```
Users can find more information about directories installed on the system in the [directory and files](https://www.taosdata.com/en/documentation/administrator/#Directory-and-Files) section. Since version 2.0, installing from source code will also configure service management for TDengine.
Users can also choose to [install from packages](https://www.taosdata.com/en/getting-started/#Install-from-Package) for it.
......@@ -193,6 +184,20 @@ taos
If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
## Quick Run
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal:
```bash
./build/bin/taosd -c test/cfg
```
In another terminal, use the TDengine shell to connect the server:
```bash
./build/bin/taos -c test/cfg
```
option "-c test/cfg" specifies the system configuration file directory.
# Try TDengine
It is easy to run SQL commands from TDengine shell which is the same as other SQL databases.
```sql
......
documentation20/cn/03.architecture/docs.md
浏览文件 @ 59790c4d
......@@ -145,7 +145,7 @@ TDengine 建议用数据采集点的名字(如上表中的D1001)来做表名。
在TDengine的设计里，**表用来代表一个具体的数据采集点，超级表用来代表一组相同类型的数据采集点集合**。当为某个具体数据采集点创建表时，用户使用超级表的定义做模板，同时指定该具体采集点（表）的标签值。与传统的关系型数据库相比，表（一个数据采集点）是带有静态标签的，而且这些标签可以事后增加、删除、修改。**一张超级表包含有多张表，这些表具有相同的时序数据schema，但带有不同的标签值**。
当对多个具有相同数据类型的数据采集点进行聚合操作时，TDengine将先把满足标签过滤条件的表从超级表的中查找出来，然后再扫描这些表的时序数据，进行聚合操作，这样能将需要扫描的数据集大幅减少，从而大幅提高聚合计算的性能。
当对多个具有相同数据类型的数据采集点进行聚合操作时，TDengine会先把满足标签过滤条件的表从超级表中找出来，然后再扫描这些表的时序数据，进行聚合操作，这样需要扫描的数据集会大幅减少，从而显著提高聚合计算的性能。
## <a class="anchor" id="cluster"></a>集群与基本逻辑单元
......
documentation20/cn/12.taos-sql/docs.md
浏览文件 @ 59790c4d
......@@ -125,7 +125,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
```mysql
ALTER DATABASE db_name CACHELAST 0;
```
CACHELAST 参数控制是否在内存中缓存数据子表的 last_row。缺省值为 0，取值范围 [0, 1]。其中 0 表示不启用、1 表示启用。（从 2.0.11 版本开始支持）
CACHELAST 参数控制是否在内存中缓存数据子表的 last_row。缺省值为 0，取值范围 [0, 1]。其中 0 表示不启用、1 表示启用。（从 2.0.11 版本开始支持，修改后需要重启服务器生效。）
**Tips**: 以上所有参数修改后都可以用show databases来确认是否修改成功。
......
documentation20/cn/13.faq/docs.md
浏览文件 @ 59790c4d
......@@ -16,13 +16,13 @@
## 1. TDengine2.0之前的版本升级到2.0及以上的版本应该注意什么？☆☆☆
2.0版本在之前版本的基础上，进行了完全的重构，配置文件和数据文件是不兼容的。在升级之前务必进行如下操作：
2.0版在之前版本的基础上，进行了完全的重构，配置文件和数据文件是不兼容的。在升级之前务必进行如下操作：
1. 删除配置文件，执行 <code> sudo rm -rf /etc/taos/taos.cfg </code>
2. 删除日志文件，执行 <code> sudo rm -rf /var/log/taos/ </code>
3. 确保数据已经不再需要的前提下，删除数据文件，执行 <code> sudo rm -rf /var/lib/taos/ </code>
4. 安装最新稳定版本的TDengine
5. 如果数据需要迁移数据或者数据文件损坏，请联系涛思数据官方技术支持团队，进行协助解决
1. 删除配置文件，执行 `sudo rm -rf /etc/taos/taos.cfg`
2. 删除日志文件，执行 `sudo rm -rf /var/log/taos/`
3. 确保数据已经不再需要的前提下，删除数据文件，执行 `sudo rm -rf /var/lib/taos/`
4. 安装最新稳定版本的 TDengine
5. 如果需要迁移数据或者数据文件损坏，请联系涛思数据官方技术支持团队，进行协助解决
## 2. Windows平台下JDBCDriver找不到动态链接库，怎么办？
......
packaging/tools/remove.sh
浏览文件 @ 59790c4d
......@@ -213,10 +213,10 @@ fi
if echo $osinfo | grep -qwi "ubuntu" ; then
# echo "this is ubuntu system"
${csudo} rm -f /var/lib/dpkg/info/tdengine* || :
${csudo} dpkg --force-all -P tdengine || :
elif echo $osinfo | grep -qwi "debian" ; then
# echo "this is debian system"
${csudo} rm -f /var/lib/dpkg/info/tdengine* || :
${csudo} dpkg --force-all -P tdengine || :
elif echo $osinfo | grep -qwi "centos" ; then
# echo "this is centos system"
${csudo} rpm -e --noscripts tdengine || :
......
src/balance/src/bnMain.c
浏览文件 @ 59790c4d
......@@ -425,7 +425,7 @@ static bool bnMonitorVgroups() {
while (1) {
pIter = mnodeGetNextVgroup(pIter, &pVgroup);
if (pVgroup == NULL) break;
if (pVgroup == NULL || pVgroup->pDb == NULL) break;
int32_t dbReplica = pVgroup->pDb->cfg.replications;
int32_t vgReplica = pVgroup->numOfVnodes;
......@@ -721,4 +721,4 @@ int32_t bnAlterDnode(struct SDnodeObj *pSrcDnode, int32_t vnodeId, int32_t dnode
mnodeDecDnodeRef(pDestDnode);
return code;
}
\ No newline at end of file
}
src/client/src/tscAsync.c
浏览文件 @ 59790c4d
......@@ -281,7 +281,7 @@ void tscQueueAsyncError(void(*fp), void *param, int32_t code) {
}
static void tscAsyncResultCallback(SSchedMsg *pMsg) {
SSqlObj* pSql = pMsg->ahandle;
SSqlObj* pSql = (SSqlObj*)taosAcquireRef(tscObjRef, (int64_t)pMsg->ahandle);
if (pSql == NULL || pSql->signature != pSql) {
tscDebug("%p SqlObj is freed, not add into queue async res", pSql);
return;
......@@ -292,23 +292,26 @@ static void tscAsyncResultCallback(SSchedMsg *pMsg) {
SSqlRes *pRes = &pSql->res;
if (pSql->fp == NULL || pSql->fetchFp == NULL){
taosReleaseRef(tscObjRef, pSql->self);
return;
}
pSql->fp = pSql->fetchFp;
(*pSql->fp)(pSql->param, pSql, pRes->code);
taosReleaseRef(tscObjRef, pSql->self);
}
void tscAsyncResultOnError(SSqlObj* pSql) {
SSchedMsg schedMsg = {0};
schedMsg.fp = tscAsyncResultCallback;
schedMsg.ahandle = pSql;
schedMsg.ahandle = (void *)pSql->self;
schedMsg.thandle = (void *)1;
schedMsg.msg = 0;
taosScheduleTask(tscQhandle, &schedMsg);
}
int tscSendMsgToServer(SSqlObj *pSql);
void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
......
src/client/src/tscSQLParser.c
浏览文件 @ 59790c4d
......@@ -2223,7 +2223,7 @@ int32_t addExprAndResultField(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, int32_t col
bool multiColOutput = taosArrayGetSize(pItem->pNode->pParam) > 1;
setResultColName(name, pItem, cvtFunc.originFuncId, &pParamElem->pNode->colInfo, multiColOutput);
if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, name, colIndex + i, &index, finalResult) != 0) {
if (setExprInfoForFunctions(pCmd, pQueryInfo, pSchema, cvtFunc, name, colIndex++, &index, finalResult) != 0) {
return TSDB_CODE_TSC_INVALID_SQL;
}
}
......@@ -3092,18 +3092,26 @@ static int32_t doExtractColumnFilterInfo(SSqlCmd* pCmd, SQueryInfo* pQueryInfo,
}
int32_t retVal = TSDB_CODE_SUCCESS;
int32_t bufLen = 0;
if (IS_NUMERIC_TYPE(pRight->value.nType)) {
bufLen = 60;
} else {
bufLen = pRight->value.nLen + 1;
}
if (pExpr->tokenId == TK_LE || pExpr->tokenId == TK_LT) {
retVal = tVariantDump(&pRight->value, (char*)&pColumnFilter->upperBndd, colType, false);
// TK_GT,TK_GE,TK_EQ,TK_NE are based on the pColumn->lowerBndd
} else if (colType == TSDB_DATA_TYPE_BINARY) {
pColumnFilter->pz = (int64_t)calloc(1, pRight->value.nLen + TSDB_NCHAR_SIZE);
pColumnFilter->pz = (int64_t)calloc(1, bufLen * TSDB_NCHAR_SIZE);
pColumnFilter->len = pRight->value.nLen;
retVal = tVariantDump(&pRight->value, (char*)pColumnFilter->pz, colType, false);
} else if (colType == TSDB_DATA_TYPE_NCHAR) {
// pRight->value.nLen + 1 is larger than the actual nchar string length
pColumnFilter->pz = (int64_t)calloc(1, (pRight->value.nLen + 1) * TSDB_NCHAR_SIZE);
pColumnFilter->pz = (int64_t)calloc(1, bufLen * TSDB_NCHAR_SIZE);
retVal = tVariantDump(&pRight->value, (char*)pColumnFilter->pz, colType, false);
size_t len = twcslen((wchar_t*)pColumnFilter->pz);
pColumnFilter->len = len * TSDB_NCHAR_SIZE;
......@@ -4138,13 +4146,21 @@ static int32_t validateTagCondExpr(SSqlCmd* pCmd, tExprNode *p) {
}
int32_t retVal = TSDB_CODE_SUCCESS;
int32_t bufLen = 0;
if (IS_NUMERIC_TYPE(vVariant->nType)) {
bufLen = 60; // The maximum length of string that a number is converted to.
} else {
bufLen = vVariant->nLen + 1;
}
if (schemaType == TSDB_DATA_TYPE_BINARY) {
char *tmp = calloc(1, vVariant->nLen + TSDB_NCHAR_SIZE);
char *tmp = calloc(1, bufLen * TSDB_NCHAR_SIZE);
retVal = tVariantDump(vVariant, tmp, schemaType, false);
free(tmp);
} else if (schemaType == TSDB_DATA_TYPE_NCHAR) {
// pRight->value.nLen + 1 is larger than the actual nchar string length
char *tmp = calloc(1, (vVariant->nLen + 1) * TSDB_NCHAR_SIZE);
char *tmp = calloc(1, bufLen * TSDB_NCHAR_SIZE);
retVal = tVariantDump(vVariant, tmp, schemaType, false);
free(tmp);
} else {
......@@ -4155,7 +4171,7 @@ static int32_t validateTagCondExpr(SSqlCmd* pCmd, tExprNode *p) {
if (retVal != TSDB_CODE_SUCCESS) {
return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg2);
}
}while (0);
} while (0);
return TSDB_CODE_SUCCESS;
}
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/rs/RestfulResultSet.java
浏览文件 @ 59790c4d
......@@ -136,7 +136,8 @@ public class RestfulResultSet extends AbstractResultSet implements ResultSet {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_PARAMETER_INDEX_OUT_RANGE, "Column Index out of range, " + columnIndex + " > " + resultSet.get(pos).size());
columnIndex = getTrueColumnIndex(columnIndex);
return resultSet.get(pos).get(columnIndex).toString();
Object value = resultSet.get(pos).get(columnIndex);
return value == null ? null : value.toString();
}
@Override
......
src/inc/tp.h
浏览文件 @ 59790c4d
......@@ -22,7 +22,7 @@ extern "C" {
int32_t tpInit();
void tpCleanUp();
void tpUpdateTs(int32_t *seq, void *pMsg);
void tpUpdateTs(int32_t vgId, int64_t *seq, void *pMsg);
#ifdef __cplusplus
}
......
src/inc/tsdb.h
浏览文件 @ 59790c4d
......@@ -25,6 +25,8 @@
#include "tdataformat.h"
#include "tname.h"
#include "hash.h"
#include "tlockfree.h"
#include "tlist.h"
#ifdef __cplusplus
extern "C" {
......@@ -172,10 +174,32 @@ typedef struct STsdbQueryCond {
int32_t type; // data block load type:
} STsdbQueryCond;
typedef struct STableData STableData;
typedef struct {
T_REF_DECLARE()
SRWLatch latch;
TSKEY keyFirst;
TSKEY keyLast;
int64_t numOfRows;
int32_t maxTables;
STableData **tData;
SList * actList;
SList * extraBuffList;
SList * bufBlockList;
int64_t pointsAdd; // TODO
int64_t storageAdd; // TODO
} SMemTable;
typedef struct {
SMemTable* mem;
SMemTable* imem;
SMemTable mtable;
SMemTable* omem;
} SMemSnapshot;
typedef struct SMemRef {
int32_t ref;
void * mem;
void * imem;
int32_t ref;
SMemSnapshot snapshot;
} SMemRef;
typedef struct SDataBlockInfo {
......
src/inc/ttokendef.h
浏览文件 @ 59790c4d
......@@ -197,13 +197,11 @@
#define TK_PREV 178
#define TK_LINEAR 179
#define TK_IMPORT 180
#define TK_METRIC 181
#define TK_TBNAME 182
#define TK_JOIN 183
#define TK_METRICS 184
#define TK_INSERT 185
#define TK_INTO 186
#define TK_VALUES 187
#define TK_TBNAME 181
#define TK_JOIN 182
#define TK_INSERT 183
#define TK_INTO 184
#define TK_VALUES 185
#define TK_SPACE 300
......
src/kit/taosdemo/insert-interlace.json 0 → 100644
浏览文件 @ 59790c4d
{
"filetype": "insert",
"cfgdir": "/etc/taos",
"host": "127.0.0.1",
"port": 6030,
"user": "root",
"password": "taosdata",
"thread_count": 4,
"thread_count_create_tbl": 4,
"result_file": "./insert_res.txt",
"confirm_parameter_prompt": "no",
"insert_interval": 1000,
"num_of_records_per_req": 100,
"max_sql_len": 1024000,
"databases": [{
"dbinfo": {
"name": "db",
"drop": "yes",
"replica": 1,
"days": 10,
"cache": 16,
"blocks": 8,
"precision": "ms",
"keep": 365,
"minRows": 100,
"maxRows": 4096,
"comp":2,
"walLevel":1,
"cachelast":0,
"quorum":1,
"fsync":3000,
"update": 0
},
"super_tables": [{
"name": "stb",
"child_table_exists":"no",
"childtable_count": 100,
"childtable_prefix": "stb_",
"auto_create_table": "no",
"data_source": "rand",
"insert_mode": "taosc",
"insert_rows": 1000,
"multi_thread_write_one_tbl": "no",
"rows_per_tbl": 20,
"max_sql_len": 1024000,
"disorder_ratio": 0,
"disorder_range": 1000,
"timestamp_step": 1,
"start_timestamp": "2020-10-01 00:00:00.000",
"sample_format": "csv",
"sample_file": "./sample.csv",
"tags_file": "",
"columns": [{"type": "INT"}],
"tags": [{"type": "TINYINT", "count":1}]
}]
}]
}
src/kit/taosdemo/taosdemo.c
浏览文件 @ 59790c4d
此差异已折叠。
src/mnode/inc/mnodeVgroup.h
浏览文件 @ 59790c4d
......@@ -44,7 +44,7 @@ void mnodeDropVgroup(SVgObj *pVgroup, void *ahandle);
void mnodeAlterVgroup(SVgObj *pVgroup, void *ahandle);
int32_t mnodeGetAvailableVgroup(struct SMnodeMsg *pMsg, SVgObj **pVgroup, int32_t *sid);
void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SCTableObj *pTable);
int32_t mnodeAddTableIntoVgroup(SVgObj *pVgroup, SCTableObj *pTable, bool needCheck);
void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SCTableObj *pTable);
void mnodeSendDropVnodeMsg(int32_t vgId, SRpcEpSet *epSet, void *ahandle);
void mnodeSendCreateVgroupMsg(SVgObj *pVgroup, void *ahandle);
......
src/mnode/src/mnodeDb.c
浏览文件 @ 59790c4d
......@@ -55,7 +55,7 @@ int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg);
#ifndef _TOPIC
int32_t tpInit() { return 0; }
void tpCleanUp() {}
void tpUpdateTs(int32_t *seq, void *pMsg) {}
void tpUpdateTs(int32_t vgId, int64_t *seq, void *pMsg) {}
#endif
static void mnodeDestroyDb(SDbObj *pDb) {
......
src/mnode/src/mnodeSdb.c
浏览文件 @ 59790c4d
......@@ -552,7 +552,7 @@ static int32_t sdbInsertHash(SSdbTable *pTable, SSdbRow *pRow) {
int32_t code = (*pTable->fpInsert)(pRow);
if (code != TSDB_CODE_SUCCESS) {
sdbError("vgId:1, sdb:%s, failed to insert key:%s to hash, remove it", pTable->name,
sdbError("vgId:1, sdb:%s, failed to perform insert action for key:%s, remove it", pTable->name,
sdbGetRowStr(pTable, pRow->pObj));
sdbDeleteHash(pTable, pRow);
}
......
src/mnode/src/mnodeTable.c
浏览文件 @ 59790c4d
......@@ -108,10 +108,12 @@ static int32_t mnodeChildTableActionDestroy(SSdbRow *pRow) {
static int32_t mnodeChildTableActionInsert(SSdbRow *pRow) {
SCTableObj *pTable = pRow->pObj;
int32_t code = 0;
SVgObj *pVgroup = mnodeGetVgroup(pTable->vgId);
if (pVgroup == NULL) {
mError("ctable:%s, not in vgId:%d", pTable->info.tableId, pTable->vgId);
code = -1;
}
SDbObj *pDb = NULL;
......@@ -119,6 +121,7 @@ static int32_t mnodeChildTableActionInsert(SSdbRow *pRow) {
pDb = mnodeGetDb(pVgroup->dbName);
if (pDb == NULL) {
mError("ctable:%s, vgId:%d not in db:%s", pTable->info.tableId, pVgroup->vgId, pVgroup->dbName);
code = -1;
}
}
......@@ -127,6 +130,7 @@ static int32_t mnodeChildTableActionInsert(SSdbRow *pRow) {
pAcct = mnodeGetAcct(pDb->acct);
if (pAcct == NULL) {
mError("ctable:%s, acct:%s not exists", pTable->info.tableId, pDb->acct);
code = -1;
}
}
......@@ -139,6 +143,7 @@ static int32_t mnodeChildTableActionInsert(SSdbRow *pRow) {
if (pAcct) pAcct->acctInfo.numOfTimeSeries += (pTable->superTable->numOfColumns - 1);
} else {
mError("table:%s:%p, correspond stable not found suid:%" PRIu64, pTable->info.tableId, pTable, pTable->suid);
code = -1;
}
} else {
grantAdd(TSDB_GRANT_TIMESERIES, pTable->numOfColumns - 1);
......@@ -146,18 +151,31 @@ static int32_t mnodeChildTableActionInsert(SSdbRow *pRow) {
}
if (pDb) mnodeAddTableIntoDb(pDb);
if (pVgroup) mnodeAddTableIntoVgroup(pVgroup, pTable);
if (pVgroup) {
if (mnodeAddTableIntoVgroup(pVgroup, pTable, pRow->pMsg == NULL) != 0) {
mError("table:%s, vgId:%d tid:%d, failed to perform insert action, uid:%" PRIu64 " suid:%" PRIu64,
pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid, pTable->suid);
code = -1;
}
}
mnodeDecVgroupRef(pVgroup);
mnodeDecDbRef(pDb);
mnodeDecAcctRef(pAcct);
return TSDB_CODE_SUCCESS;
if (code == 0) {
mTrace("table:%s, vgId:%d tid:%d, perform insert action, uid:%" PRIu64 " suid:%" PRIu64, pTable->info.tableId,
pTable->vgId, pTable->tid, pTable->uid, pTable->suid);
}
return code;
}
static int32_t mnodeChildTableActionDelete(SSdbRow *pRow) {
SCTableObj *pTable = pRow->pObj;
if (pTable->vgId == 0) {
mError("table:%s, vgId:%d tid:%d, failed to perform delete action, uid:%" PRIu64 " suid:%" PRIu64,
pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid, pTable->suid);
return TSDB_CODE_MND_VGROUP_NOT_EXIST;
}
......@@ -188,6 +206,8 @@ static int32_t mnodeChildTableActionDelete(SSdbRow *pRow) {
mnodeDecDbRef(pDb);
mnodeDecAcctRef(pAcct);
mTrace("table:%s, vgId:%d tid:%d, perform delete action, uid:%" PRIu64 " suid:%" PRIu64, pTable->info.tableId,
pTable->vgId, pTable->tid, pTable->uid, pTable->suid);
return TSDB_CODE_SUCCESS;
}
......@@ -399,13 +419,13 @@ static void mnodeAddTableIntoStable(SSTableObj *pStable, SCTableObj *pCtable) {
if (pStable->vgHash == NULL) {
pStable->vgHash = taosHashInit(64, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), true, HASH_ENTRY_LOCK);
mDebug("table:%s, create hash:%p", pStable->info.tableId, pStable->vgHash);
mDebug("stable:%s, create vgId hash:%p", pStable->info.tableId, pStable->vgHash);
}
if (pStable->vgHash != NULL) {
if (taosHashGet(pStable->vgHash, &pCtable->vgId, sizeof(pCtable->vgId)) == NULL) {
taosHashPut(pStable->vgHash, &pCtable->vgId, sizeof(pCtable->vgId), &pCtable->vgId, sizeof(pCtable->vgId));
mDebug("table:%s, vgId:%d is put into stable hash:%p, sizeOfVgList:%d", pStable->info.tableId, pCtable->vgId,
mDebug("stable:%s, vgId:%d is put into stable vgId hash:%p, sizeOfVgList:%d", pStable->info.tableId, pCtable->vgId,
pStable->vgHash, taosHashGetSize(pStable->vgHash));
}
}
......@@ -443,19 +463,21 @@ static int32_t mnodeSuperTableActionDestroy(SSdbRow *pRow) {
static int32_t mnodeSuperTableActionInsert(SSdbRow *pRow) {
SSTableObj *pStable = pRow->pObj;
SDbObj *pDb = mnodeGetDbByTableName(pStable->info.tableId);
SDbObj * pDb = mnodeGetDbByTableName(pStable->info.tableId);
if (pDb != NULL && pDb->status == TSDB_DB_STATUS_READY) {
mnodeAddSuperTableIntoDb(pDb);
}
mnodeDecDbRef(pDb);
taosHashPut(tsSTableUidHash, &pStable->uid, sizeof(int64_t), &pStable, sizeof(int64_t));
mTrace("stable:%s, perform insert action, uid:%" PRIu64, pStable->info.tableId, pStable->uid);
return TSDB_CODE_SUCCESS;
}
static int32_t mnodeSuperTableActionDelete(SSdbRow *pRow) {
SSTableObj *pStable = pRow->pObj;
SDbObj *pDb = mnodeGetDbByTableName(pStable->info.tableId);
SDbObj * pDb = mnodeGetDbByTableName(pStable->info.tableId);
if (pDb != NULL) {
mnodeRemoveSuperTableFromDb(pDb);
mnodeDropAllChildTablesInStable((SSTableObj *)pStable);
......@@ -463,6 +485,8 @@ static int32_t mnodeSuperTableActionDelete(SSdbRow *pRow) {
mnodeDecDbRef(pDb);
taosHashRemove(tsSTableUidHash, &pStable->uid, sizeof(int64_t));
mTrace("stable:%s, perform delete action, uid:%" PRIu64, pStable->info.tableId, pStable->uid);
return TSDB_CODE_SUCCESS;
}
......@@ -1888,15 +1912,14 @@ static int32_t mnodeDoCreateChildTableCb(SMnodeMsg *pMsg, int32_t code) {
pMsg->rpcMsg.handle);
if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->successed;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
}
++pMsg->pBatchMasterMsg->successed;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
}
mnodeDestroySubMsg(pMsg);
mnodeDestroySubMsg(pMsg);
return TSDB_CODE_MND_ACTION_IN_PROGRESS;
return TSDB_CODE_MND_ACTION_IN_PROGRESS;
}
dnodeSendRpcMWriteRsp(pMsg, TSDB_CODE_SUCCESS);
......@@ -1911,9 +1934,8 @@ static int32_t mnodeDoCreateChildTableCb(SMnodeMsg *pMsg, int32_t code) {
if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->received;
pMsg->pBatchMasterMsg->code = code;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, code);
}
mnodeDestroySubMsg(pMsg);
......@@ -2662,9 +2684,8 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) {
if (pMsg->pBatchMasterMsg) {
++pMsg->pBatchMasterMsg->successed;
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received
>= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, rpcMsg->code);
if (pMsg->pBatchMasterMsg->successed + pMsg->pBatchMasterMsg->received >= pMsg->pBatchMasterMsg->expected) {
dnodeSendRpcMWriteRsp(pMsg->pBatchMasterMsg, rpcMsg->code);
}
mnodeDestroySubMsg(pMsg);
......
src/mnode/src/mnodeVgroup.c
浏览文件 @ 59790c4d
......@@ -443,6 +443,7 @@ int32_t mnodeGetAvailableVgroup(SMnodeMsg *pMsg, SVgObj **ppVgroup, int32_t *pSi
mDebug("msg:%p, app:%p db:%s, no enough sid in vgId:%d", pMsg, pMsg->rpcMsg.ahandle, pDb->name, pVgroup->vgId);
continue;
}
mTrace("vgId:%d, alloc tid:%d", pVgroup->vgId, sid);
*pSid = sid;
*ppVgroup = pVgroup;
......@@ -507,6 +508,7 @@ int32_t mnodeGetAvailableVgroup(SMnodeMsg *pMsg, SVgObj **ppVgroup, int32_t *pSi
pDb->vgListIndex = 0;
pthread_mutex_unlock(&pDb->mutex);
mTrace("vgId:%d, alloc tid:%d", pVgroup->vgId, sid);
return TSDB_CODE_SUCCESS;
}
......@@ -832,26 +834,37 @@ static int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, v
return numOfRows;
}
void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SCTableObj *pTable) {
int32_t mnodeAddTableIntoVgroup(SVgObj *pVgroup, SCTableObj *pTable, bool needCheck) {
int32_t idPoolSize = taosIdPoolMaxSize(pVgroup->idPool);
if (pTable->tid > idPoolSize) {
mnodeAllocVgroupIdPool(pVgroup);
}
if (pTable->tid >= 1) {
taosIdPoolMarkStatus(pVgroup->idPool, pTable->tid);
pVgroup->numOfTables++;
// The create vgroup message may be received later than the create table message
// and the writing order in sdb is therefore uncertain
// which will cause the reference count of the vgroup to be incorrect when restarting
// mnodeIncVgroupRef(pVgroup);
if (taosIdPoolMarkStatus(pVgroup->idPool, pTable->tid) || !needCheck) {
pVgroup->numOfTables++;
mTrace("table:%s, vgId:%d tid:%d, mark tid used, uid:%" PRIu64, pTable->info.tableId, pTable->vgId, pTable->tid,
pTable->uid);
// The create vgroup message may be received later than the create table message
// and the writing order in sdb is therefore uncertain
// which will cause the reference count of the vgroup to be incorrect when restarting
// mnodeIncVgroupRef(pVgroup);
} else {
mError("table:%s, vgId:%d tid:%d, failed to mark tid, uid:%" PRIu64, pTable->info.tableId, pTable->vgId,
pTable->tid, pTable->uid);
return -1;
}
}
return 0;
}
void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SCTableObj *pTable) {
if (pTable->tid >= 1) {
taosFreeId(pVgroup->idPool, pTable->tid);
pVgroup->numOfTables--;
mTrace("table:%s, vgId:%d tid:%d, put tid back uid:%" PRIu64, pTable->info.tableId, pTable->vgId, pTable->tid,
pTable->uid);
// The create vgroup message may be received later than the create table message
// and the writing order in sdb is therefore uncertain
// which will cause the reference count of the vgroup to be incorrect when restarting
......
src/query/src/qExecutor.c
浏览文件 @ 59790c4d
......@@ -1840,7 +1840,7 @@ static void doFreeQueryHandle(SQueryRuntimeEnv* pRuntimeEnv) {
pRuntimeEnv->pQueryHandle = NULL;
SMemRef* pMemRef = &pQuery->memRef;
assert(pMemRef->ref == 0 && pMemRef->imem == NULL && pMemRef->mem == NULL);
assert(pMemRef->ref == 0 && pMemRef->snapshot.imem == NULL && pMemRef->snapshot.mem == NULL);
}
static void teardownQueryRuntimeEnv(SQueryRuntimeEnv *pRuntimeEnv) {
......
src/query/src/qTokenizer.c
浏览文件 @ 59790c4d
......@@ -206,10 +206,8 @@ static SKeyword keywordTable[] = {
{"PREV", TK_PREV},
{"LINEAR", TK_LINEAR},
{"IMPORT", TK_IMPORT},
{"METRIC", TK_METRIC},
{"TBNAME", TK_TBNAME},
{"JOIN", TK_JOIN},
{"METRICS", TK_METRICS},
{"STABLE", TK_STABLE},
{"FILE", TK_FILE},
{"VNODES", TK_VNODES},
......
src/query/src/queryMain.c
浏览文件 @ 59790c4d
......@@ -430,7 +430,7 @@ void qQueryMgmtNotifyClosed(void* pQMgmt) {
}
SQueryMgmt* pQueryMgmt = pQMgmt;
qDebug("vgId:%d, set querymgmt closed, wait for all queries cancelled", pQueryMgmt->vgId);
qInfo("vgId:%d, set querymgmt closed, wait for all queries cancelled", pQueryMgmt->vgId);
pthread_mutex_lock(&pQueryMgmt->lock);
pQueryMgmt->closed = true;
......@@ -445,7 +445,7 @@ void qQueryMgmtReOpen(void *pQMgmt) {
}
SQueryMgmt *pQueryMgmt = pQMgmt;
qDebug("vgId:%d, set querymgmt reopen", pQueryMgmt->vgId);
qInfo("vgId:%d, set querymgmt reopen", pQueryMgmt->vgId);
pthread_mutex_lock(&pQueryMgmt->lock);
pQueryMgmt->closed = false;
......
src/query/src/sql.c
浏览文件 @ 59790c4d
此差异已折叠。
src/sync/src/syncRetrieve.c
浏览文件 @ 59790c4d
......@@ -29,7 +29,7 @@ static int32_t syncGetWalVersion(SSyncNode *pNode, SSyncPeer *pPeer) {
uint64_t fver, wver;
int32_t code = (*pNode->getVersionFp)(pNode->vgId, &fver, &wver);
if (code != 0) {
sDebug("%s, vnode is commiting while retrieve, last wver:%" PRIu64, pPeer->id, pPeer->lastWalVer);
sInfo("%s, vnode is commiting while retrieve, last wver:%" PRIu64, pPeer->id, pPeer->lastWalVer);
return -1;
}
......@@ -41,12 +41,12 @@ static bool syncIsWalModified(SSyncNode *pNode, SSyncPeer *pPeer) {
uint64_t fver, wver;
int32_t code = (*pNode->getVersionFp)(pNode->vgId, &fver, &wver);
if (code != 0) {
sDebug("%s, vnode is commiting while retrieve, last wver:%" PRIu64, pPeer->id, pPeer->lastWalVer);
sInfo("%s, vnode is commiting while retrieve, last wver:%" PRIu64, pPeer->id, pPeer->lastWalVer);
return true;
}
if (wver != pPeer->lastWalVer) {
sDebug("%s, wal is modified while retrieve, wver:%" PRIu64 ", last:%" PRIu64, pPeer->id, wver, pPeer->lastWalVer);
sInfo("%s, wal is modified while retrieve, wver:%" PRIu64 ", last:%" PRIu64, pPeer->id, wver, pPeer->lastWalVer);
return true;
}
......@@ -57,7 +57,7 @@ static int32_t syncGetFileVersion(SSyncNode *pNode, SSyncPeer *pPeer) {
uint64_t fver, wver;
int32_t code = (*pNode->getVersionFp)(pNode->vgId, &fver, &wver);
if (code != 0) {
sDebug("%s, vnode is commiting while get fver for retrieve, last fver:%" PRIu64, pPeer->id, pPeer->lastFileVer);
sInfo("%s, vnode is commiting while get fver for retrieve, last fver:%" PRIu64, pPeer->id, pPeer->lastFileVer);
return -1;
}
......@@ -69,13 +69,13 @@ static bool syncAreFilesModified(SSyncNode *pNode, SSyncPeer *pPeer) {
uint64_t fver, wver;
int32_t code = (*pNode->getVersionFp)(pNode->vgId, &fver, &wver);
if (code != 0) {
sDebug("%s, vnode is commiting while retrieve, last fver:%" PRIu64, pPeer->id, pPeer->lastFileVer);
sInfo("%s, vnode is commiting while retrieve, last fver:%" PRIu64, pPeer->id, pPeer->lastFileVer);
pPeer->fileChanged = 1;
return true;
}
if (fver != pPeer->lastFileVer) {
sDebug("%s, files are modified while retrieve, fver:%" PRIu64 ", last:%" PRIu64, pPeer->id, fver, pPeer->lastFileVer);
sInfo("%s, files are modified while retrieve, fver:%" PRIu64 ", last:%" PRIu64, pPeer->id, fver, pPeer->lastFileVer);
pPeer->fileChanged = 1;
return true;
}
......@@ -143,13 +143,13 @@ static int32_t syncReadOneWalRecord(int32_t sfd, SWalHead *pHead) {
}
if (ret == 0) {
sDebug("sfd:%d, read to the end of file, ret:%d", sfd, ret);
sInfo("sfd:%d, read to the end of file, ret:%d", sfd, ret);
return 0;
}
if (ret != sizeof(SWalHead)) {
// file is not at end yet, it shall be reloaded
sDebug("sfd:%d, a partial wal head is read out, ret:%d", sfd, ret);
sInfo("sfd:%d, a partial wal head is read out, ret:%d", sfd, ret);
return 0;
}
......@@ -163,7 +163,7 @@ static int32_t syncReadOneWalRecord(int32_t sfd, SWalHead *pHead) {
if (ret != pHead->len) {
// file is not at end yet, it shall be reloaded
sDebug("sfd:%d, a partial wal conetnt is read out, ret:%d", sfd, ret);
sInfo("sfd:%d, a partial wal conetnt is read out, ret:%d", sfd, ret);
return 0;
}
......@@ -184,7 +184,7 @@ static int64_t syncRetrieveLastWal(SSyncPeer *pPeer, char *name, uint64_t fversi
return -1;
}
sDebug("%s, retrieve last wal:%s, offset:%" PRId64 " fver:%" PRIu64, pPeer->id, name, offset, fversion);
sInfo("%s, retrieve last wal:%s, offset:%" PRId64 " fver:%" PRIu64, pPeer->id, name, offset, fversion);
SWalHead *pHead = malloc(SYNC_MAX_SIZE);
int64_t bytes = 0;
......@@ -198,7 +198,7 @@ static int64_t syncRetrieveLastWal(SSyncPeer *pPeer, char *name, uint64_t fversi
if (code == 0) {
code = bytes;
sDebug("%s, read to the end of wal, bytes:%" PRId64, pPeer->id, bytes);
sInfo("%s, read to the end of wal, bytes:%" PRId64, pPeer->id, bytes);
break;
}
......@@ -217,7 +217,7 @@ static int64_t syncRetrieveLastWal(SSyncPeer *pPeer, char *name, uint64_t fversi
if (pHead->version >= fversion && fversion > 0) {
code = 0;
sDebug("%s, retrieve wal finished, hver:%" PRIu64 " fver:%" PRIu64, pPeer->id, pHead->version, fversion);
sInfo("%s, retrieve wal finished, hver:%" PRIu64 " fver:%" PRIu64, pPeer->id, pHead->version, fversion);
break;
}
}
......@@ -237,7 +237,7 @@ static int64_t syncProcessLastWal(SSyncPeer *pPeer, char *wname, int64_t index)
// get full path to wal file
snprintf(fname, sizeof(fname), "%s/%s", pNode->path, wname);
sDebug("%s, start to retrieve last wal:%s", pPeer->id, fname);
sInfo("%s, start to retrieve last wal:%s", pPeer->id, fname);
while (1) {
if (syncAreFilesModified(pNode, pPeer)) return -1;
......@@ -245,7 +245,7 @@ static int64_t syncProcessLastWal(SSyncPeer *pPeer, char *wname, int64_t index)
int64_t bytes = syncRetrieveLastWal(pPeer, fname, fversion, offset);
if (bytes < 0) {
sDebug("%s, failed to retrieve last wal, bytes:%" PRId64, pPeer->id, bytes);
sInfo("%s, failed to retrieve last wal, bytes:%" PRId64, pPeer->id, bytes);
return bytes;
}
......@@ -257,13 +257,13 @@ static int64_t syncProcessLastWal(SSyncPeer *pPeer, char *wname, int64_t index)
if (fversion == 0) {
pPeer->sstatus = TAOS_SYNC_STATUS_CACHE; // start to forward pkt
fversion = nodeVersion; // must read data to fversion
sDebug("%s, set sstatus:%s and fver:%" PRIu64, pPeer->id, syncStatus[pPeer->sstatus], fversion);
sInfo("%s, set sstatus:%s and fver:%" PRIu64, pPeer->id, syncStatus[pPeer->sstatus], fversion);
}
}
// if all data up to fversion is read out, it is over
if (pPeer->sversion >= fversion && fversion > 0) {
sDebug("%s, data up to fver:%" PRIu64 " has been read out, bytes:%" PRId64 " sver:%" PRIu64, pPeer->id, fversion, bytes,
sInfo("%s, data up to fver:%" PRIu64 " has been read out, bytes:%" PRId64 " sver:%" PRIu64, pPeer->id, fversion, bytes,
pPeer->sversion);
return 0;
}
......@@ -277,7 +277,7 @@ static int64_t syncProcessLastWal(SSyncPeer *pPeer, char *wname, int64_t index)
// if bytes > 0, file is updated, or fversion is not reached but file still open, read again
once = 1;
offset += bytes;
sDebug("%s, continue retrieve last wal, bytes:%" PRId64 " offset:%" PRId64 " sver:%" PRIu64 " fver:%" PRIu64, pPeer->id,
sInfo("%s, continue retrieve last wal, bytes:%" PRId64 " offset:%" PRId64 " sver:%" PRIu64 " fver:%" PRIu64, pPeer->id,
bytes, offset, pPeer->sversion, fversion);
}
......@@ -303,7 +303,7 @@ static int64_t syncRetrieveWal(SSyncPeer *pPeer) {
if (wname[0] == 0) { // no wal file
code = 0;
sDebug("%s, no wal file anymore", pPeer->id);
sInfo("%s, no wal file anymore", pPeer->id);
break;
}
......@@ -320,12 +320,12 @@ static int64_t syncRetrieveWal(SSyncPeer *pPeer) {
struct stat fstat;
if (stat(fname, &fstat) < 0) {
code = -1;
sDebug("%s, failed to stat wal:%s for retrieve since %s, code:0x%" PRIx64, pPeer->id, fname, strerror(errno), code);
sInfo("%s, failed to stat wal:%s for retrieve since %s, code:0x%" PRIx64, pPeer->id, fname, strerror(errno), code);
break;
}
size = fstat.st_size;
sDebug("%s, retrieve wal:%s size:%d", pPeer->id, fname, size);
sInfo("%s, retrieve wal:%s size:%d", pPeer->id, fname, size);
int32_t sfd = open(fname, O_RDONLY | O_BINARY);
if (sfd < 0) {
......@@ -334,7 +334,7 @@ static int64_t syncRetrieveWal(SSyncPeer *pPeer) {
break;
}
code = (int32_t)taosSendFile(pPeer->syncFd, sfd, NULL, size);
code = taosSendFile(pPeer->syncFd, sfd, NULL, size);
close(sfd);
if (code < 0) {
sError("%s, failed to send wal:%s for retrieve since %s, code:0x%" PRIx64, pPeer->id, fname, strerror(errno), code);
......@@ -374,7 +374,7 @@ static int32_t syncRetrieveFirstPkt(SSyncPeer *pPeer) {
sError("%s, failed to send sync-data msg since %s, tranId:%u", pPeer->id, strerror(errno), msg.tranId);
return -1;
}
sDebug("%s, send sync-data msg to peer, tranId:%u", pPeer->id, msg.tranId);
sInfo("%s, send sync-data msg to peer, tranId:%u", pPeer->id, msg.tranId);
SSyncRsp rsp;
if (taosReadMsg(pPeer->syncFd, &rsp, sizeof(SSyncRsp)) != sizeof(SSyncRsp)) {
......@@ -382,7 +382,7 @@ static int32_t syncRetrieveFirstPkt(SSyncPeer *pPeer) {
return -1;
}
sDebug("%s, recv sync-data rsp from peer, tranId:%u rsp-tranId:%u", pPeer->id, msg.tranId, rsp.tranId);
sInfo("%s, recv sync-data rsp from peer, tranId:%u rsp-tranId:%u", pPeer->id, msg.tranId, rsp.tranId);
return 0;
}
......
src/tsdb/inc/tsdbMemTable.h
浏览文件 @ 59790c4d
......@@ -31,29 +31,14 @@ typedef struct {
SSkipListIterator *pIter;
} SCommitIter;
typedef struct {
struct STableData {
uint64_t uid;
TSKEY keyFirst;
TSKEY keyLast;
int64_t numOfRows;
SSkipList* pData;
T_REF_DECLARE()
} STableData;
typedef struct {
T_REF_DECLARE()
SRWLatch latch;
TSKEY keyFirst;
TSKEY keyLast;
int64_t numOfRows;
int32_t maxTables;
STableData** tData;
SList* actList;
SList* extraBuffList;
SList* bufBlockList;
int64_t pointsAdd; // TODO
int64_t storageAdd; // TODO
} SMemTable;
};
enum { TSDB_UPDATE_META, TSDB_DROP_META };
......@@ -77,8 +62,8 @@ typedef struct {
int tsdbRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable);
int tsdbUnRefMemTable(STsdbRepo* pRepo, SMemTable* pMemTable);
int tsdbTakeMemSnapshot(STsdbRepo* pRepo, SMemTable** pMem, SMemTable** pIMem, SArray* pATable);
void tsdbUnTakeMemSnapShot(STsdbRepo* pRepo, SMemTable* pMem, SMemTable* pIMem);
int tsdbTakeMemSnapshot(STsdbRepo* pRepo, SMemSnapshot* pSnapshot, SArray* pATable);
void tsdbUnTakeMemSnapShot(STsdbRepo* pRepo, SMemSnapshot* pSnapshot);
void* tsdbAllocBytes(STsdbRepo* pRepo, int bytes);
int tsdbAsyncCommit(STsdbRepo* pRepo);
int tsdbLoadDataFromCache(STable* pTable, SSkipListIterator* pIter, TSKEY maxKey, int maxRowsToRead, SDataCols* pCols,
......
src/tsdb/src/tsdbMemTable.c
浏览文件 @ 59790c4d
......@@ -124,88 +124,80 @@ int tsdbUnRefMemTable(STsdbRepo *pRepo, SMemTable *pMemTable) {
return 0;
}
int tsdbTakeMemSnapshot(STsdbRepo *pRepo, SMemTable **pMem, SMemTable **pIMem, SArray *pATable) {
SMemTable *tmem;
int tsdbTakeMemSnapshot(STsdbRepo *pRepo, SMemSnapshot *pSnapshot, SArray *pATable) {
memset(pSnapshot, 0, sizeof(*pSnapshot));
// Get snap object
if (tsdbLockRepo(pRepo) < 0) return -1;
tmem = pRepo->mem;
*pIMem = pRepo->imem;
tsdbRefMemTable(pRepo, tmem);
tsdbRefMemTable(pRepo, *pIMem);
pSnapshot->omem = pRepo->mem;
pSnapshot->imem = pRepo->imem;
tsdbRefMemTable(pRepo, pRepo->mem);
tsdbRefMemTable(pRepo, pRepo->imem);
if (tsdbUnlockRepo(pRepo) < 0) return -1;
// Copy mem objects and ref needed STableData
if (tmem) {
taosRLockLatch(&(tmem->latch));
if (pSnapshot->omem) {
taosRLockLatch(&(pSnapshot->omem->latch));
*pMem = (SMemTable *)calloc(1, sizeof(**pMem));
if (*pMem == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
taosRUnLockLatch(&(tmem->latch));
tsdbUnRefMemTable(pRepo, tmem);
tsdbUnRefMemTable(pRepo, *pIMem);
*pMem = NULL;
*pIMem = NULL;
return -1;
}
pSnapshot->mem = &(pSnapshot->mtable);
(*pMem)->tData = (STableData **)calloc(tmem->maxTables, sizeof(STableData *));
if ((*pMem)->tData == NULL) {
pSnapshot->mem->tData = (STableData **)calloc(pSnapshot->omem->maxTables, sizeof(STableData *));
if (pSnapshot->mem->tData == NULL) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
taosRUnLockLatch(&(tmem->latch));
free(*pMem);
tsdbUnRefMemTable(pRepo, tmem);
tsdbUnRefMemTable(pRepo, *pIMem);
*pMem = NULL;
*pIMem = NULL;
taosRUnLockLatch(&(pSnapshot->omem->latch));
tsdbUnRefMemTable(pRepo, pSnapshot->omem);
tsdbUnRefMemTable(pRepo, pSnapshot->imem);
pSnapshot->mem = NULL;
pSnapshot->imem = NULL;
pSnapshot->omem = NULL;
return -1;
}
(*pMem)->keyFirst = tmem->keyFirst;
(*pMem)->keyLast = tmem->keyLast;
(*pMem)->numOfRows = tmem->numOfRows;
(*pMem)->maxTables = tmem->maxTables;
pSnapshot->mem->keyFirst = pSnapshot->omem->keyFirst;
pSnapshot->mem->keyLast = pSnapshot->omem->keyLast;
pSnapshot->mem->numOfRows = pSnapshot->omem->numOfRows;
pSnapshot->mem->maxTables = pSnapshot->omem->maxTables;
for (size_t i = 0; i < taosArrayGetSize(pATable); i++) {
STable * pTable = *(STable **)taosArrayGet(pATable, i);
int32_t tid = TABLE_TID(pTable);
STableData *pTableData = (tid < tmem->maxTables) ? tmem->tData[tid] : NULL;
STableData *pTableData = (tid < pSnapshot->omem->maxTables) ? pSnapshot->omem->tData[tid] : NULL;
if ((pTableData == NULL) || (TABLE_UID(pTable) != pTableData->uid)) continue;
(*pMem)->tData[tid] = tmem->tData[tid];
T_REF_INC(tmem->tData[tid]);
pSnapshot->mem->tData[tid] = pTableData;
T_REF_INC(pTableData);
}
taosRUnLockLatch(&(tmem->latch));
taosRUnLockLatch(&(pSnapshot->omem->latch));
}
tsdbUnRefMemTable(pRepo, tmem);
tsdbDebug("vgId:%d take memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), *pMem, *pIMem);
tsdbDebug("vgId:%d take memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), pSnapshot->omem, pSnapshot->imem);
return 0;
}
void tsdbUnTakeMemSnapShot(STsdbRepo *pRepo, SMemTable *pMem, SMemTable *pIMem) {
tsdbDebug("vgId:%d untake memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), pMem, pIMem);
void tsdbUnTakeMemSnapShot(STsdbRepo *pRepo, SMemSnapshot *pSnapshot) {
tsdbDebug("vgId:%d untake memory snapshot, pMem %p pIMem %p", REPO_ID(pRepo), pSnapshot->omem, pSnapshot->imem);
if (pMem != NULL) {
for (size_t i = 0; i < pMem->maxTables; i++) {
STableData *pTableData = pMem->tData[i];
if (pSnapshot->mem) {
ASSERT(pSnapshot->omem != NULL);
for (size_t i = 0; i < pSnapshot->mem->maxTables; i++) {
STableData *pTableData = pSnapshot->mem->tData[i];
if (pTableData) {
tsdbFreeTableData(pTableData);
}
}
free(pMem->tData);
free(pMem);
}
tfree(pSnapshot->mem->tData);
if (pIMem != NULL) {
tsdbUnRefMemTable(pRepo, pIMem);
tsdbUnRefMemTable(pRepo, pSnapshot->omem);
}
tsdbUnRefMemTable(pRepo, pSnapshot->imem);
pSnapshot->mem = NULL;
pSnapshot->imem = NULL;
pSnapshot->omem = NULL;
}
void *tsdbAllocBytes(STsdbRepo *pRepo, int bytes) {
......
src/tsdb/src/tsdbMeta.c
浏览文件 @ 59790c4d
......@@ -213,9 +213,9 @@ void *tsdbGetTableTagVal(const void* pTable, int32_t colId, int16_t type, int16_
char *val = tdGetKVRowValOfCol(((STable*)pTable)->tagVal, colId);
assert(type == pCol->type && bytes == pCol->bytes);
if (val != NULL && IS_VAR_DATA_TYPE(type)) {
assert(varDataLen(val) < pCol->bytes);
}
// if (val != NULL && IS_VAR_DATA_TYPE(type)) {
// assert(varDataLen(val) < pCol->bytes);
// }
return val;
}
......
src/tsdb/src/tsdbRead.c
浏览文件 @ 59790c4d
......@@ -194,7 +194,7 @@ static void tsdbMayTakeMemSnapshot(STsdbQueryHandle* pQueryHandle, SArray* psTab
SMemRef* pMemRef = pQueryHandle->pMemRef;
if (pQueryHandle->pMemRef->ref++ == 0) {
tsdbTakeMemSnapshot(pQueryHandle->pTsdb, (SMemTable**)&(pMemRef->mem), (SMemTable**)&(pMemRef->imem), psTable);
tsdbTakeMemSnapshot(pQueryHandle->pTsdb, &(pMemRef->snapshot), psTable);
}
taosArrayDestroy(psTable);
......@@ -208,9 +208,7 @@ static void tsdbMayUnTakeMemSnapshot(STsdbQueryHandle* pQueryHandle) {
}
if (--pMemRef->ref == 0) {
tsdbUnTakeMemSnapShot(pQueryHandle->pTsdb, pMemRef->mem, pMemRef->imem);
pMemRef->mem = NULL;
pMemRef->imem = NULL;
tsdbUnTakeMemSnapShot(pQueryHandle->pTsdb, &(pMemRef->snapshot));
}
pQueryHandle->pMemRef = NULL;
......@@ -229,10 +227,10 @@ int64_t tsdbGetNumOfRowsInMemTable(TsdbQueryHandleT* pHandle) {
if (pMemRef == NULL) { return rows; }
STableData* pMem = NULL;
STableData* pIMem = NULL;
STableData* pIMem = NULL;
SMemTable *pMemT = (SMemTable *)(pMemRef->mem);
SMemTable *pIMemT = (SMemTable *)(pMemRef->imem);
SMemTable* pMemT = pMemRef->snapshot.mem;
SMemTable* pIMemT = pMemRef->snapshot.imem;
if (pMemT && pCheckInfo->tableId.tid < pMemT->maxTables) {
pMem = pMemT->tData[pCheckInfo->tableId.tid];
......@@ -605,7 +603,7 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
int32_t order = pHandle->order;
// no data in buffer, abort
if (pHandle->pMemRef->mem == NULL && pHandle->pMemRef->imem == NULL) {
if (pHandle->pMemRef->snapshot.mem == NULL && pHandle->pMemRef->snapshot.imem == NULL) {
return false;
}
......@@ -614,8 +612,8 @@ static bool initTableMemIterator(STsdbQueryHandle* pHandle, STableCheckInfo* pCh
STableData* pMem = NULL;
STableData* pIMem = NULL;
SMemTable* pMemT = pHandle->pMemRef->mem;
SMemTable* pIMemT = pHandle->pMemRef->imem;
SMemTable* pMemT = pHandle->pMemRef->snapshot.mem;
SMemTable* pIMemT = pHandle->pMemRef->snapshot.imem;
if (pMemT && pCheckInfo->tableId.tid < pMemT->maxTables) {
pMem = pMemT->tData[pCheckInfo->tableId.tid];
......
src/tsdb/src/tsdbSync.c
浏览文件 @ 59790c4d
......@@ -152,14 +152,14 @@ static int32_t tsdbSyncSendMeta(SSyncH *pSynch) {
return -1;
}
int32_t writeLen = (int32_t)mf.info.size;
tsdbInfo("vgId:%d, metafile:%s will be sent, size:%d", REPO_ID(pRepo), mf.f.aname, writeLen);
int64_t writeLen = mf.info.size;
tsdbInfo("vgId:%d, metafile:%s will be sent, size:%" PRId64, REPO_ID(pRepo), mf.f.aname, writeLen);
int32_t ret = (int32_t)taosSendFile(pSynch->socketFd, TSDB_FILE_FD(&mf), 0, writeLen);
int64_t ret = taosSendFile(pSynch->socketFd, TSDB_FILE_FD(&mf), 0, writeLen);
if (ret != writeLen) {
terrno = TAOS_SYSTEM_ERROR(errno);
tsdbError("vgId:%d, failed to send metafile since %s, ret:%d writeLen:%d", REPO_ID(pRepo), tstrerror(terrno), ret,
writeLen);
tsdbError("vgId:%d, failed to send metafile since %s, ret:%" PRId64 " writeLen:%" PRId64, REPO_ID(pRepo),
tstrerror(terrno), ret, writeLen);
tsdbCloseMFile(&mf);
return -1;
}
......@@ -217,18 +217,18 @@ static int32_t tsdbSyncRecvMeta(SSyncH *pSynch) {
tsdbInfo("vgId:%d, metafile:%s is created", REPO_ID(pRepo), mf.f.aname);
int32_t readLen = (int32_t)pSynch->pmf->info.size;
int32_t ret = taosCopyFds(pSynch->socketFd, TSDB_FILE_FD(&mf), readLen);
int64_t readLen = pSynch->pmf->info.size;
int64_t ret = taosCopyFds(pSynch->socketFd, TSDB_FILE_FD(&mf), readLen);
if (ret != readLen) {
terrno = TAOS_SYSTEM_ERROR(errno);
tsdbError("vgId:%d, failed to recv metafile since %s, ret:%d readLen:%d", REPO_ID(pRepo), tstrerror(terrno), ret,
readLen);
tsdbError("vgId:%d, failed to recv metafile since %s, ret:%" PRId64 " readLen:%" PRId64, REPO_ID(pRepo),
tstrerror(terrno), ret, readLen);
tsdbCloseMFile(&mf);
tsdbRemoveMFile(&mf);
return -1;
}
tsdbInfo("vgId:%d, metafile is received, size:%d", REPO_ID(pRepo), readLen);
tsdbInfo("vgId:%d, metafile is received, size:%" PRId64, REPO_ID(pRepo), readLen);
mf.info = pSynch->pmf->info;
tsdbCloseMFile(&mf);
......@@ -463,12 +463,12 @@ static int32_t tsdbSyncRecvDFileSetArray(SSyncH *pSynch) {
tsdbInfo("vgId:%d, file:%s will be received, osize:%" PRIu64 " rsize:%" PRIu64, REPO_ID(pRepo),
pDFile->f.aname, pDFile->info.size, pRDFile->info.size);
int32_t writeLen = (int32_t)pRDFile->info.size;
int32_t ret = taosCopyFds(pSynch->socketFd, pDFile->fd, writeLen);
int64_t writeLen = pRDFile->info.size;
int64_t ret = taosCopyFds(pSynch->socketFd, pDFile->fd, writeLen);
if (ret != writeLen) {
terrno = TAOS_SYSTEM_ERROR(errno);
tsdbError("vgId:%d, failed to recv file:%s since %s, ret:%d writeLen:%d", REPO_ID(pRepo), pDFile->f.aname,
tstrerror(terrno), ret, writeLen);
tsdbError("vgId:%d, failed to recv file:%s since %s, ret:%" PRId64 " writeLen:%" PRId64, REPO_ID(pRepo),
pDFile->f.aname, tstrerror(terrno), ret, writeLen);
tsdbCloseDFileSet(&fset);
tsdbRemoveDFileSet(&fset);
return -1;
......@@ -476,7 +476,7 @@ static int32_t tsdbSyncRecvDFileSetArray(SSyncH *pSynch) {
// Update new file info
pDFile->info = pRDFile->info;
tsdbInfo("vgId:%d, file:%s is received, size:%d", REPO_ID(pRepo), pDFile->f.aname, writeLen);
tsdbInfo("vgId:%d, file:%s is received, size:%" PRId64, REPO_ID(pRepo), pDFile->f.aname, writeLen);
}
tsdbCloseDFileSet(&fset);
......@@ -575,14 +575,14 @@ static int32_t tsdbSyncSendDFileSet(SSyncH *pSynch, SDFileSet *pSet) {
return -1;
}
int32_t writeLen = (int32_t)df.info.size;
tsdbInfo("vgId:%d, file:%s will be sent, size:%d", REPO_ID(pRepo), df.f.aname, writeLen);
int64_t writeLen = df.info.size;
tsdbInfo("vgId:%d, file:%s will be sent, size:%" PRId64, REPO_ID(pRepo), df.f.aname, writeLen);
int32_t ret = (int32_t)taosSendFile(pSynch->socketFd, TSDB_FILE_FD(&df), 0, writeLen);
int64_t ret = taosSendFile(pSynch->socketFd, TSDB_FILE_FD(&df), 0, writeLen);
if (ret != writeLen) {
terrno = TAOS_SYSTEM_ERROR(errno);
tsdbError("vgId:%d, failed to send file:%s since %s, ret:%d writeLen:%d", REPO_ID(pRepo), df.f.aname,
tstrerror(terrno), ret, writeLen);
tsdbError("vgId:%d, failed to send file:%s since %s, ret:%" PRId64 " writeLen:%" PRId64, REPO_ID(pRepo),
df.f.aname, tstrerror(terrno), ret, writeLen);
tsdbCloseDFile(&df);
return -1;
}
......@@ -677,13 +677,13 @@ static int32_t tsdbRecvDFileSetInfo(SSyncH *pSynch) {
static int tsdbReload(STsdbRepo *pRepo, bool isMfChanged) {
// TODO: may need to stop and restart stream
if (isMfChanged) {
tsdbCloseMeta(pRepo);
tsdbFreeMeta(pRepo->tsdbMeta);
pRepo->tsdbMeta = tsdbNewMeta(REPO_CFG(pRepo));
tsdbOpenMeta(pRepo);
tsdbLoadMetaCache(pRepo, true);
}
// if (isMfChanged) {
tsdbCloseMeta(pRepo);
tsdbFreeMeta(pRepo->tsdbMeta);
pRepo->tsdbMeta = tsdbNewMeta(REPO_CFG(pRepo));
tsdbOpenMeta(pRepo);
tsdbLoadMetaCache(pRepo, true);
// }
tsdbUnRefMemTable(pRepo, pRepo->mem);
tsdbUnRefMemTable(pRepo, pRepo->imem);
......
src/util/inc/tidpool.h
浏览文件 @ 59790c4d
......@@ -34,7 +34,7 @@ void taosIdPoolCleanUp(void *handle);
int taosIdPoolNumOfUsed(void *handle);
void taosIdPoolMarkStatus(void *handle, int id);
bool taosIdPoolMarkStatus(void *handle, int id);
#ifdef __cplusplus
}
......
src/util/inc/tsocket.h
浏览文件 @ 59790c4d
......@@ -28,7 +28,7 @@ int32_t taosReadn(SOCKET sock, char *buffer, int32_t len);
int32_t taosWriteMsg(SOCKET fd, void *ptr, int32_t nbytes);
int32_t taosReadMsg(SOCKET fd, void *ptr, int32_t nbytes);
int32_t taosNonblockwrite(SOCKET fd, char *ptr, int32_t nbytes);
int32_t taosCopyFds(SOCKET sfd, int32_t dfd, int64_t len);
int64_t taosCopyFds(SOCKET sfd, int32_t dfd, int64_t len);
int32_t taosSetNonblocking(SOCKET sock, int32_t on);
SOCKET taosOpenUdpSocket(uint32_t localIp, uint16_t localPort);
......
src/util/src/tidpool.c
浏览文件 @ 59790c4d
......@@ -104,10 +104,16 @@ void taosIdPoolCleanUp(void *handle) {
int taosIdPoolNumOfUsed(void *handle) {
id_pool_t *pIdPool = handle;
return pIdPool->maxId - pIdPool->numOfFree;
pthread_mutex_lock(&pIdPool->mutex);
int ret = pIdPool->maxId - pIdPool->numOfFree;
pthread_mutex_unlock(&pIdPool->mutex);
return ret;
}
void taosIdPoolMarkStatus(void *handle, int id) {
bool taosIdPoolMarkStatus(void *handle, int id) {
bool ret = false;
id_pool_t *pIdPool = handle;
pthread_mutex_lock(&pIdPool->mutex);
......@@ -115,9 +121,13 @@ void taosIdPoolMarkStatus(void *handle, int id) {
if (!pIdPool->freeList[slot]) {
pIdPool->freeList[slot] = true;
pIdPool->numOfFree--;
ret = true;
} else {
ret = false;
}
pthread_mutex_unlock(&pIdPool->mutex);
return ret;
}
int taosUpdateIdPool(id_pool_t *handle, int maxId) {
......@@ -147,6 +157,11 @@ int taosUpdateIdPool(id_pool_t *handle, int maxId) {
}
int taosIdPoolMaxSize(void *handle) {
id_pool_t *pIdPool = (id_pool_t*)handle;
return pIdPool->maxId;
id_pool_t *pIdPool = (id_pool_t *)handle;
pthread_mutex_lock(&pIdPool->mutex);
int ret = pIdPool->maxId;
pthread_mutex_unlock(&pIdPool->mutex);
return ret;
}
\ No newline at end of file
src/util/src/tsocket.c
浏览文件 @ 59790c4d
......@@ -465,36 +465,36 @@ void tinet_ntoa(char *ipstr, uint32_t ip) {
#define COPY_SIZE 32768
// sendfile shall be used
int32_t taosCopyFds(SOCKET sfd, int32_t dfd, int64_t len) {
int64_t taosCopyFds(SOCKET sfd, int32_t dfd, int64_t len) {
int64_t leftLen;
int32_t readLen, writeLen;
int64_t readLen, writeLen;
char temp[COPY_SIZE];
leftLen = len;
while (leftLen > 0) {
if (leftLen < COPY_SIZE)
readLen = (int32_t)leftLen;
readLen = leftLen;
else
readLen = COPY_SIZE; // 4K
int32_t retLen = taosReadMsg(sfd, temp, (int32_t)readLen);
int64_t retLen = taosReadMsg(sfd, temp, (int32_t)readLen);
if (readLen != retLen) {
uError("read error, readLen:%d retLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, retLen, len,
leftLen, strerror(errno));
uError("read error, readLen:%" PRId64 " retLen:%" PRId64 " len:%" PRId64 " leftLen:%" PRId64 ", reason:%s",
readLen, retLen, len, leftLen, strerror(errno));
return -1;
}
writeLen = taosWriteMsg(dfd, temp, readLen);
writeLen = taosWriteMsg(dfd, temp, (int32_t)readLen);
if (readLen != writeLen) {
uError("copy error, readLen:%d writeLen:%d len:%" PRId64 " leftLen:%" PRId64 ", reason:%s", readLen, writeLen,
len, leftLen, strerror(errno));
uError("copy error, readLen:%" PRId64 " writeLen:%" PRId64 " len:%" PRId64 " leftLen:%" PRId64 ", reason:%s",
readLen, writeLen, len, leftLen, strerror(errno));
return -1;
}
leftLen -= readLen;
}
return (int32_t)len;
return len;
}
src/vnode/inc/vnodeInt.h
浏览文件 @ 59790c4d
......@@ -40,7 +40,7 @@ typedef struct {
int32_t queuedWMsg;
int32_t queuedRMsg;
int32_t flowctrlLevel;
int32_t sequence; // for topic
int64_t sequence; // for topic
int8_t status;
int8_t role;
int8_t accessState;
......
src/vnode/src/vnodeMain.c
浏览文件 @ 59790c4d
......@@ -147,7 +147,7 @@ static int32_t vnodeAlterImp(SVnodeObj *pVnode, SCreateVnodeMsg *pVnodeCfg) {
vDebug("vgId:%d, tsdbchanged:%d syncchanged:%d while alter vnode", pVnode->vgId, tsdbCfgChanged, syncCfgChanged);
if (tsdbCfgChanged || syncCfgChanged) {
if (/*tsdbCfgChanged || */syncCfgChanged) {
// vnode in non-ready state and still needs to return success instead of TSDB_CODE_VND_INVALID_STATUS
// dbCfgVersion can be corrected by status msg
if (!vnodeSetUpdatingStatus(pVnode)) {
......@@ -482,7 +482,7 @@ static int32_t vnodeProcessTsdbStatus(void *arg, int32_t status, int32_t eno) {
if (status == TSDB_STATUS_COMMIT_START) {
pVnode->isCommiting = 1;
pVnode->cversion = pVnode->version;
vDebug("vgId:%d, start commit, fver:%" PRIu64 " vver:%" PRIu64, pVnode->vgId, pVnode->fversion, pVnode->version);
vInfo("vgId:%d, start commit, fver:%" PRIu64 " vver:%" PRIu64, pVnode->vgId, pVnode->fversion, pVnode->version);
if (!vnodeInInitStatus(pVnode)) {
return walRenew(pVnode->wal);
}
......@@ -493,7 +493,7 @@ static int32_t vnodeProcessTsdbStatus(void *arg, int32_t status, int32_t eno) {
pVnode->isCommiting = 0;
pVnode->isFull = 0;
pVnode->fversion = pVnode->cversion;
vDebug("vgId:%d, commit over, fver:%" PRIu64 " vver:%" PRIu64, pVnode->vgId, pVnode->fversion, pVnode->version);
vInfo("vgId:%d, commit over, fver:%" PRIu64 " vver:%" PRIu64, pVnode->vgId, pVnode->fversion, pVnode->version);
if (!vnodeInInitStatus(pVnode)) {
walRemoveOneOldFile(pVnode->wal);
}
......
src/vnode/src/vnodeMgmt.c
浏览文件 @ 59790c4d
......@@ -128,7 +128,7 @@ static void vnodeBuildVloadMsg(SVnodeObj *pVnode, SStatusMsg *pStatus) {
int64_t compStorage = 0;
int64_t pointsWritten = 0;
if (!vnodeInReadyStatus(pVnode)) return;
if (vnodeInClosingStatus(pVnode)) return;
if (pStatus->openVnodes >= TSDB_MAX_VNODES) return;
if (pVnode->tsdb) {
......@@ -194,4 +194,4 @@ void vnodeSetAccess(SVgroupAccess *pAccess, int32_t numOfVnodes) {
vnodeRelease(pVnode);
}
}
}
\ No newline at end of file
}
src/vnode/src/vnodeStatus.c
浏览文件 @ 59790c4d
......@@ -109,6 +109,8 @@ bool vnodeSetResetStatus(SVnodeObj* pVnode) {
taosMsleep(1);
}
vInfo("vgId:%d, set to reset status", pVnode->vgId);
// release local resources only after cutting off outside connections
qQueryMgmtNotifyClosed(pVnode->qMgmt);
vnodeWaitReadCompleted(pVnode);
......@@ -153,6 +155,18 @@ bool vnodeInReadyOrUpdatingStatus(SVnodeObj* pVnode) {
return in;
}
bool vnodeInClosingStatus(SVnodeObj* pVnode) {
bool in = false;
pthread_mutex_lock(&pVnode->statusMutex);
if (pVnode->status == TAOS_VN_STATUS_CLOSING) {
in = true;
}
pthread_mutex_unlock(&pVnode->statusMutex);
return in;
}
bool vnodeInResetStatus(SVnodeObj* pVnode) {
bool in = false;
pthread_mutex_lock(&pVnode->statusMutex);
......
src/vnode/src/vnodeSync.c
浏览文件 @ 59790c4d
......@@ -91,7 +91,7 @@ void vnodeStartSyncFile(int32_t vgId) {
return;
}
vDebug("vgId:%d, datafile will be synced", vgId);
vInfo("vgId:%d, datafile will be synced", vgId);
vnodeSetResetStatus(pVnode);
vnodeRelease(pVnode);
......@@ -147,7 +147,7 @@ int32_t vnodeGetVersion(int32_t vgId, uint64_t *fver, uint64_t *wver) {
int32_t code = 0;
if (pVnode->isCommiting) {
vDebug("vgId:%d, vnode is commiting while get version", vgId);
vInfo("vgId:%d, vnode is commiting while get version", vgId);
code = -1;
} else {
*fver = pVnode->fversion;
......@@ -168,7 +168,7 @@ int32_t vnodeResetVersion(int32_t vgId, uint64_t fver) {
pVnode->fversion = fver;
pVnode->version = fver;
walResetVersion(pVnode->wal, fver);
vDebug("vgId:%d, version reset to %" PRIu64, vgId, fver);
vInfo("vgId:%d, version reset to %" PRIu64, vgId, fver);
vnodeRelease(pVnode);
return 0;
......
src/vnode/src/vnodeWrite.c
浏览文件 @ 59790c4d
......@@ -142,7 +142,7 @@ static int32_t vnodeProcessSubmitMsg(SVnodeObj *pVnode, void *pCont, SRspRet *pR
vTrace("vgId:%d, submit msg is processed", pVnode->vgId);
if (pVnode->dbType == TSDB_DB_TYPE_TOPIC && pVnode->role == TAOS_SYNC_ROLE_MASTER) {
tpUpdateTs(&pVnode->sequence, pCont);
tpUpdateTs(pVnode->vgId, &pVnode->sequence, pCont);
}
// save insert result into item
......
src/wal/src/walWrite.c
浏览文件 @ 59790c4d
......@@ -455,7 +455,7 @@ uint64_t walGetVersion(twalh param) {
void walResetVersion(twalh param, uint64_t newVer) {
SWal *pWal = param;
if (pWal == 0) return;
wDebug("vgId:%d, version reset from %" PRIu64 " to %" PRIu64, pWal->vgId, pWal->version, newVer);
wInfo("vgId:%d, version reset from %" PRIu64 " to %" PRIu64, pWal->vgId, pWal->version, newVer);
pWal->version = newVer;
}
\ No newline at end of file
tests/pytest/cluster/clusterEnvSetup/basic.py
浏览文件 @ 59790c4d
......@@ -18,13 +18,15 @@ import argparse
class BuildDockerCluser:
def __init__(self, hostName, user, password, configDir, numOfNodes, clusterVersion):
def __init__(self, hostName, user, password, configDir, numOfNodes, clusterVersion, dockerDir, removeFlag):
self.hostName = hostName
self.user = user
self.password = password
self.configDir = configDir
self.numOfNodes = numOfNodes
self.clusterVersion = clusterVersion
self.clusterVersion = clusterVersion
self.dockerDir = dockerDir
self.removeFlag = removeFlag
def getConnection(self):
self.conn = taos.connect(
......@@ -42,14 +44,17 @@ class BuildDockerCluser:
print("start arbitrator")
os.system("docker exec -d $(docker ps|grep tdnode1|awk '{print $1}') tarbitrator")
def run(self):
def run(self):
if self.numOfNodes < 2 or self.numOfNodes > 5:
print("the number of nodes must be between 2 and 5")
exit(0)
os.system("./buildClusterEnv.sh -n %d -v %s" % (self.numOfNodes, self.clusterVersion))
exit(0)
print("remove Flag value %s" % self.removeFlag)
if self.removeFlag == False:
os.system("./cleanClusterEnv.sh -d %s" % self.dockerDir)
os.system("./buildClusterEnv.sh -n %d -v %s -d %s" % (self.numOfNodes, self.clusterVersion, self.dockerDir))
self.getConnection()
self.createDondes()
self.startArbitrator()
self.startArbitrator()
parser = argparse.ArgumentParser()
parser.add_argument(
......@@ -91,10 +96,24 @@ parser.add_argument(
'-v',
'--version',
action='store',
default='2.0.14.1',
default='2.0.17.1',
type=str,
help='the version of the cluster to be build, Default is 2.0.14.1')
help='the version of the cluster to be build, Default is 2.0.17.1')
parser.add_argument(
'-d',
'--docker-dir',
action='store',
default='/data',
type=str,
help='the data dir for docker, default is /data')
parser.add_argument(
'--flag',
action='store_true',
help='remove docker containers flag, default: True')
args = parser.parse_args()
cluster = BuildDockerCluser(args.host, args.user, args.password, args.config_dir, args.num_of_nodes, args.version)
cluster.run()
\ No newline at end of file
cluster = BuildDockerCluser(args.host, args.user, args.password, args.config_dir, args.num_of_nodes, args.version, args.docker_dir, args.flag)
cluster.run()
# usage 1: python3 basic.py -n 2 --flag (flag is True)
# usage 2: python3 basic.py -n 2 (flag should be False when it is not specified)
\ No newline at end of file
tests/pytest/cluster/clusterEnvSetup/buildClusterEnv.sh
浏览文件 @ 59790c4d
#!/bin/bash
echo "Executing buildClusterEnv.sh"
DOCKER_DIR=/data
CURR_DIR=`pwd`
if [ $# != 4 ]; then
if [ $# != 6 ]; then
echo "argument list need input : "
echo " -n numOfNodes"
echo " -v version"
echo " -v version"
echo " -d docker dir"
exit 1
fi
NUM_OF_NODES=
VERSION=
while getopts "n:v:" arg
DOCKER_DIR=
while getopts "n:v:d:" arg
do
case $arg in
n)
......@@ -20,6 +21,9 @@ do
;;
v)
VERSION=$OPTARG
;;
d)
DOCKER_DIR=$OPTARG
;;
?)
echo "unkonwn argument"
......@@ -30,30 +34,29 @@ done
function addTaoscfg {
for i in {1..5}
do
touch /data/node$i/cfg/taos.cfg
echo 'firstEp tdnode1:6030' > /data/node$i/cfg/taos.cfg
echo 'fqdn tdnode'$i >> /data/node$i/cfg/taos.cfg
echo 'arbitrator tdnode1:6042' >> /data/node$i/cfg/taos.cfg
touch $DOCKER_DIR/node$i/cfg/taos.cfg
echo 'firstEp tdnode1:6030' > $DOCKER_DIR/node$i/cfg/taos.cfg
echo 'fqdn tdnode'$i >> $DOCKER_DIR/node$i/cfg/taos.cfg
echo 'arbitrator tdnode1:6042' >> $DOCKER_DIR/node$i/cfg/taos.cfg
done
}
function createDIR {
for i in {1..5}
do
mkdir -p /data/node$i/data
mkdir -p /data/node$i/log
mkdir -p /data/node$i/cfg
mkdir -p /data/node$i/core
mkdir -p $DOCKER_DIR/node$i/data
mkdir -p $DOCKER_DIR/node$i/log
mkdir -p $DOCKER_DIR/node$i/cfg
mkdir -p $DOCKER_DIR/node$i/core
done
}
function cleanEnv {
function cleanEnv {
echo "Clean up docker environment"
for i in {1..5}
do
echo /data/node$i/data/*
rm -rf /data/node$i/data/*
echo /data/node$i/log/*
rm -rf /data/node$i/log/*
do
rm -rf $DOCKER_DIR/node$i/data/*
rm -rf $DOCKER_DIR/node$i/log/*
done
}
......@@ -98,19 +101,19 @@ function clusterUp {
if [ $NUM_OF_NODES -eq 2 ]; then
echo "create 2 dnodes"
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION docker-compose up -d
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz TARBITRATORPKG=TDengine-arbitrator-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION DATADIR=$DOCKER_DIR docker-compose up -d
fi
if [ $NUM_OF_NODES -eq 3 ]; then
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION docker-compose -f docker-compose.yml -f node3.yml up -d
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz TARBITRATORPKG=TDengine-arbitrator-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION DATADIR=$DOCKER_DIR docker-compose -f docker-compose.yml -f node3.yml up -d
fi
if [ $NUM_OF_NODES -eq 4 ]; then
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION docker-compose -f docker-compose.yml -f node3.yml -f node4.yml up -d
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz TARBITRATORPKG=TDengine-arbitrator-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION DATADIR=$DOCKER_DIR docker-compose -f docker-compose.yml -f node3.yml -f node4.yml up -d
fi
if [ $NUM_OF_NODES -eq 5 ]; then
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION docker-compose -f docker-compose.yml -f node3.yml -f node4.yml -f node5.yml up -d
PACKAGE=TDengine-server-$VERSION-Linux-x64.tar.gz TARBITRATORPKG=TDengine-arbitrator-$VERSION-Linux-x64.tar.gz DIR=TDengine-server-$VERSION DIR2=TDengine-arbitrator-$VERSION VERSION=$VERSION DATADIR=$DOCKER_DIR docker-compose -f docker-compose.yml -f node3.yml -f node4.yml -f node5.yml up -d
fi
echo "docker compose finish"
......
tests/pytest/cluster/clusterEnvSetup/cleanClusterEnv.sh 0 → 100755
浏览文件 @ 59790c4d
#!/bin/bash
echo "Executing cleanClusterEnv.sh"
CURR_DIR=`pwd`
if [ $# != 2 ]; then
echo "argument list need input : "
echo " -d docker dir"
exit 1
fi
DOCKER_DIR=
while getopts "d:" arg
do
case $arg in
d)
DOCKER_DIR=$OPTARG
;;
?)
echo "unkonwn argument"
;;
esac
done
function removeDockerContainers {
cd $DOCKER_DIR
docker-compose down --remove-orphans
}
function cleanEnv {
echo "Clean up docker environment"
for i in {1..5}
do
rm -rf $DOCKER_DIR/node$i/data/*
rm -rf $DOCKER_DIR/node$i/log/*
done
}
removeDockerContainers
cleanEnv
\ No newline at end of file
tests/pytest/cluster/clusterEnvSetup/docker-compose.yml
浏览文件 @ 59790c4d
......@@ -9,6 +9,7 @@ services:
- TARBITRATORPKG=${TARBITRATORPKG}
- EXTRACTDIR=${DIR}
- EXTRACTDIR2=${DIR2}
- DATADIR=${DATADIR}
image: 'tdengine:${VERSION}'
container_name: 'tdnode1'
cap_add:
......@@ -32,19 +33,19 @@ services:
volumes:
# bind data directory
- type: bind
source: /data/node1/data
source: ${DATADIR}/node1/data
target: /var/lib/taos
# bind log directory
- type: bind
source: /data/node1/log
source: ${DATADIR}/node1/log
target: /var/log/taos
# bind configuration
- type: bind
source: /data/node1/cfg
source: ${DATADIR}/node1/cfg
target: /etc/taos
# bind core dump path
- type: bind
source: /data/node1/core
source: ${DATADIR}/node1/core
target: /coredump
- type: bind
source: /data
......@@ -61,6 +62,7 @@ services:
args:
- PACKAGE=${PACKAGE}
- EXTRACTDIR=${DIR}
- DATADIR=${DATADIR}
image: 'tdengine:${VERSION}'
container_name: 'tdnode2'
cap_add:
......@@ -84,22 +86,22 @@ services:
volumes:
# bind data directory
- type: bind
source: /data/node2/data
source: ${DATADIR}/node2/data
target: /var/lib/taos
# bind log directory
- type: bind
source: /data/node2/log
source: ${DATADIR}/node2/log
target: /var/log/taos
# bind configuration
- type: bind
source: /data/node2/cfg
source: ${DATADIR}/node2/cfg
target: /etc/taos
# bind core dump path
- type: bind
source: /data/node2/core
source: ${DATADIR}/node2/core
target: /coredump
- type: bind
source: /data
source: ${DATADIR}
target: /root
hostname: tdnode2
networks:
......
tests/pytest/cluster/clusterEnvSetup/node3.yml
浏览文件 @ 59790c4d
......@@ -7,6 +7,7 @@ services:
args:
- PACKAGE=${PACKAGE}
- EXTRACTDIR=${DIR}
- DATADIR=${DATADIR}
image: 'tdengine:${VERSION}'
container_name: 'tdnode3'
cap_add:
......@@ -30,22 +31,22 @@ services:
volumes:
# bind data directory
- type: bind
source: /data/node3/data
source: ${DATADIR}/node3/data
target: /var/lib/taos
# bind log directory
- type: bind
source: /data/node3/log
source: ${DATADIR}/node3/log
target: /var/log/taos
# bind configuration
- type: bind
source: /data/node3/cfg
source: ${DATADIR}/node3/cfg
target: /etc/taos
# bind core dump path
- type: bind
source: /data/node3/core
source: ${DATADIR}/node3/core
target: /coredump
- type: bind
source: /data
source: ${DATADIR}
target: /root
hostname: tdnode3
networks:
......
tests/pytest/cluster/clusterEnvSetup/node4.yml
浏览文件 @ 59790c4d
......@@ -7,6 +7,7 @@ services:
args:
- PACKAGE=${PACKAGE}
- EXTRACTDIR=${DIR}
- DATADIR=${DATADIR}
image: 'tdengine:${VERSION}'
container_name: 'tdnode4'
cap_add:
......@@ -30,23 +31,23 @@ services:
volumes:
# bind data directory
- type: bind
source: /data/node4/data
source: ${DATADIR}/node4/data
target: /var/lib/taos
# bind log directory
- type: bind
source: /data/node4/log
source: ${DATADIR}/node4/log
target: /var/log/taos
# bind configuration
- type: bind
source: /data/node4/cfg
source: ${DATADIR}/node4/cfg
target: /etc/taos
# bind core dump path
# bind core dump path
- type: bind
source: /data/node4/core
source: ${DATADIR}/node4/core
target: /coredump
- type: bind
source: /data
target: /root
source: ${DATADIR}
target: /root
hostname: tdnode4
networks:
taos_update_net:
......
tests/pytest/cluster/clusterEnvSetup/node5.yml
浏览文件 @ 59790c4d
......@@ -7,6 +7,7 @@ services:
args:
- PACKAGE=${PACKAGE}
- EXTRACTDIR=${DIR}
- DATADIR=${DATADIR}
image: 'tdengine:${VERSION}'
container_name: 'tdnode5'
cap_add:
......@@ -30,22 +31,22 @@ services:
volumes:
# bind data directory
- type: bind
source: /data/node5/data
source: ${DATADIR}/node5/data
target: /var/lib/taos
# bind log directory
- type: bind
source: /data/node5/log
source: ${DATADIR}/node5/log
target: /var/log/taos
# bind configuration
- type: bind
source: /data/node5/cfg
source: ${DATADIR}/node5/cfg
target: /etc/taos
# bind core dump path
- type: bind
source: /data/node5/core
source: ${DATADIR}/node5/core
target: /coredump
- type: bind
source: /data
source: ${DATADIR}
target: /root
hostname: tdnode5
networks:
......
tests/pytest/fulltest.sh
浏览文件 @ 59790c4d
#!/bin/bash
ulimit -c unlimited
#======================p1-start===============
python3 ./test.py -f insert/basic.py
python3 ./test.py -f insert/int.py
python3 ./test.py -f insert/unsignedInt.py
python3 ./test.py -f insert/float.py
python3 ./test.py -f insert/bigint.py
python3 ./test.py -f insert/unsignedBigint.py
python3 ./test.py -f insert/bool.py
python3 ./test.py -f insert/double.py
python3 ./test.py -f insert/smallint.py
python3 ./test.py -f insert/unsignedSmallint.py
python3 ./test.py -f insert/tinyint.py
python3 ./test.py -f insert/unsignedTinyint.py
python3 ./test.py -f insert/date.py
python3 ./test.py -f insert/binary.py
python3 ./test.py -f insert/nchar.py
......@@ -23,10 +20,8 @@ python3 ./test.py -f insert/randomNullCommit.py
python3 insert/retentionpolicy.py
python3 ./test.py -f insert/alterTableAndInsert.py
python3 ./test.py -f insert/insertIntoTwoTables.py
#python3 ./test.py -f insert/before_1970.py
python3 ./test.py -f insert/metadataUpdate.py
python3 ./test.py -f insert/before_1970.py
python3 bug2265.py
python3 ./test.py -f insert/boundary2.py
#table
python3 ./test.py -f table/alter_wal0.py
......@@ -71,7 +66,7 @@ python3 ./test.py -f tag_lite/int.py
python3 ./test.py -f tag_lite/set.py
python3 ./test.py -f tag_lite/smallint.py
python3 ./test.py -f tag_lite/tinyint.py
python3 ./test.py -f tag_lite/alter_tag.py
#python3 ./test.py -f dbmgmt/database-name-boundary.py
python3 ./test.py -f import_merge/importBlock1HO.py
......@@ -142,6 +137,26 @@ python3 ./test.py -f import_merge/importTPORestart.py
python3 ./test.py -f import_merge/importTRestart.py
python3 ./test.py -f import_merge/importInsertThenImport.py
python3 ./test.py -f import_merge/importCSV.py
#======================p1-end===============
#======================p2-start===============
# update
python3 ./test.py -f update/allow_update.py
python3 ./test.py -f update/allow_update-0.py
python3 ./test.py -f update/append_commit_data.py
python3 ./test.py -f update/append_commit_last-0.py
python3 ./test.py -f update/append_commit_last.py
python3 ./test.py -f update/merge_commit_data.py
python3 ./test.py -f update/merge_commit_data2.py
python3 ./test.py -f update/merge_commit_data2_update0.py
python3 ./test.py -f update/merge_commit_last-0.py
python3 ./test.py -f update/merge_commit_last.py
python3 ./test.py -f update/bug_td2279.py
#======================p2-end===============
#======================p3-start===============
# user
python3 ./test.py -f user/user_create.py
python3 ./test.py -f user/pass_len.py
......@@ -155,7 +170,6 @@ python3 ./test.py -f query/filterCombo.py
python3 ./test.py -f query/queryNormal.py
python3 ./test.py -f query/queryError.py
python3 ./test.py -f query/filterAllIntTypes.py
python3 ./test.py -f query/filterAllUnsignedIntTypes.py
python3 ./test.py -f query/filterFloatAndDouble.py
python3 ./test.py -f query/filterOtherTypes.py
python3 ./test.py -f query/querySort.py
......@@ -182,8 +196,6 @@ python3 ./test.py -f query/bug2119.py
python3 ./test.py -f query/isNullTest.py
python3 ./test.py -f query/queryWithTaosdKilled.py
python3 ./test.py -f query/floatCompare.py
python3 ./test.py -f query/queryGroupbySort.py
python3 ./test.py -f query/queryBetweenAnd.py
#stream
python3 ./test.py -f stream/metric_1.py
......@@ -199,8 +211,6 @@ python3 ./test.py -f stream/table_n.py
#alter table
python3 ./test.py -f alter/alter_table_crash.py
python3 ./test.py -f alter/alter_table.py
python3 ./test.py -f alter/alter_debugFlag.py
# client
python3 ./test.py -f client/client.py
......@@ -213,6 +223,37 @@ python3 testCompress.py
python3 testNoCompress.py
python3 testMinTablesPerVnode.py
python3 queryCount.py
python3 ./test.py -f query/queryGroupbyWithInterval.py
python3 client/twoClients.py
python3 test.py -f query/queryInterval.py
python3 test.py -f query/queryFillTest.py
# tools
python3 test.py -f tools/taosdemoTest.py
python3 test.py -f tools/taosdemoTestWithoutMetric.py
python3 test.py -f tools/taosdemoTestWithJson.py
python3 test.py -f tools/taosdemoTestLimitOffset.py
python3 test.py -f tools/taosdumpTest.py
python3 test.py -f tools/taosdemoTest2.py
python3 test.py -f tools/taosdemoTestSampleData.py
# subscribe
python3 test.py -f subscribe/singlemeter.py
#python3 test.py -f subscribe/stability.py
python3 test.py -f subscribe/supertable.py
#======================p3-end===============
#======================p4-start===============
python3 ./test.py -f update/merge_commit_data-0.py
# wal
python3 ./test.py -f wal/addOldWalTest.py
# function
python3 ./test.py -f functions/all_null_value.py
# functions
python3 ./test.py -f functions/function_avg.py -r 1
python3 ./test.py -f functions/function_bottom.py -r 1
......@@ -232,47 +273,24 @@ python3 ./test.py -f functions/function_sum.py -r 1
python3 ./test.py -f functions/function_top.py -r 1
python3 ./test.py -f functions/function_twa.py -r 1
python3 ./test.py -f functions/function_twa_test2.py
python3 ./test.py -f functions/all_null_value.py
python3 ./test.py -f functions/function_percentile2.py
python3 queryCount.py
python3 ./test.py -f query/queryGroupbyWithInterval.py
python3 client/twoClients.py
python3 ./test.py -f query/queryInterval.py
python3 ./test.py -f query/queryFillTest.py
python3 ./test.py -f functions/function_stddev_td2555.py
python3 ./test.py -f insert/metadataUpdate.py
#python3 ./test.py -f query/last_cache.py
python3 ./test.py -f query/last_row_cache.py
python3 ./test.py -f query/last_cache.py
# tools
python3 test.py -f tools/taosdumpTest.py
python3 test.py -f tools/lowaTest.py
python3 test.py -f tools/taosdemoTest.py
python3 test.py -f tools/taosdemoTest2.py
python3 test.py -f tools/taosdemoTestWithoutMetric.py
python3 test.py -f tools/taosdemoTestLimitOffset.py
python3 test.py -f tools/taosdemoTestSampleData.py
# subscribe
python3 test.py -f subscribe/singlemeter.py
#python3 test.py -f subscribe/stability.py
python3 test.py -f subscribe/supertable.py
python3 ./test.py -f account/account_create.py
python3 ./test.py -f alter/alter_table.py
python3 ./test.py -f query/queryGroupbySort.py
# update
python3 ./test.py -f update/allow_update.py
python3 ./test.py -f update/allow_update-0.py
python3 ./test.py -f update/append_commit_data.py
python3 ./test.py -f update/append_commit_last-0.py
python3 ./test.py -f update/append_commit_last.py
python3 ./test.py -f update/merge_commit_data.py
python3 ./test.py -f update/merge_commit_data-0.py
python3 ./test.py -f update/merge_commit_data2.py
python3 ./test.py -f update/merge_commit_data2_update0.py
python3 ./test.py -f update/merge_commit_last-0.py
python3 ./test.py -f update/merge_commit_last.py
python3 ./test.py -f update/bug_td2279.py
python3 ./test.py -f insert/unsignedInt.py
python3 ./test.py -f insert/unsignedBigint.py
python3 ./test.py -f insert/unsignedSmallint.py
python3 ./test.py -f insert/unsignedTinyint.py
python3 ./test.py -f query/filterAllUnsignedIntTypes.py
# wal
python3 ./test.py -f wal/addOldWalTest.py
python3 ./test.py -f functions/function_percentile2.py
python3 ./test.py -f insert/boundary2.py
python3 ./test.py -f alter/alter_debugFlag.py
python3 ./test.py -f query/queryBetweenAnd.py
python3 ./test.py -f tag_lite/alter_tag.py
# account
python3 ./test.py -f account/account_create.py
#======================p4-end===============
tests/pytest/functions/function_operations.py
浏览文件 @ 59790c4d
......@@ -92,14 +92,16 @@ class TDTestCase:
col_list = [ 'col1' , 'col2' , 'col3' , 'col4' , 'col5' , 'col6' , 'col7' , 'col8' , 'col9' , 'col11' , 'col12' , 'col13' , 'col14' , '1' , '1.1' , 'NULL' ]
op_list = [ '+' , '-' , '*' , '/' , '%' ]
err_list = [ 'col7' , 'col8' , 'col9' , 'NULL' ]
order_lsit = [ ' order by ts ', ' order by ts desc ', ' order by ts asc ']
for i in col_list :
for j in col_list :
for k in op_list :
sql = " select %s %s %s from test1 " % ( i , k , j )
if i in err_list or j in err_list:
tdSql.error(sql)
else:
tdSql.query(sql)
for l in order_lsit :
sql = " select %s %s %s from test1 %s" % ( i , k , j , l )
if i in err_list or j in err_list:
tdSql.error(sql)
else:
tdSql.query(sql)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
tests/pytest/functions/function_stddev.py
浏览文件 @ 59790c4d
......@@ -118,7 +118,11 @@ class TDTestCase:
if i == 1 or i == 5 or i == 6 or i == 7 or i == 9 or i == 8 :continue
tdSql.query('select stddev(c%d),stddev(c%d) from s group by c%d' %( i+1 , i+1 , i+1 ) )
#add for TD-3318
tdSql.execute('create table t1(ts timestamp, k int, b binary(12));')
tdSql.execute("insert into t1 values(now, 1, 'abc');")
tdLog.info("select stddev(k) from t1 where b <> 'abc' interval(1s);")
tdSql.query("select stddev(k) from t1 where b <> 'abc' interval(1s);")
def stop(self):
......
tests/pytest/handle_crash_gen_val_log.sh
浏览文件 @ 59790c4d
......@@ -16,7 +16,7 @@ TOP_DIR=`pwd`
TAOSD_DIR=`find . -name "taosd"|grep -v community|head -n1`
nohup $TAOSD_DIR >/dev/null &
cd -
./crash_gen.sh --valgrind -p -t 10 -s 500 -b 4
./crash_gen.sh --valgrind -p -t 10 -s 1000 -b 4
pidof taosd|xargs kill -9
grep 'start to execute\|ERROR SUMMARY' valgrind.err|grep -v 'grep'|uniq|tee crash_gen_mem_err.log
......
tests/pytest/pytest_1.sh 已删除 100755 → 0
浏览文件 @ 2f0f9006
#!/bin/bash
ulimit -c unlimited
python3 ./test.py -f insert/basic.py
python3 ./test.py -f insert/int.py
python3 ./test.py -f insert/float.py
python3 ./test.py -f insert/bigint.py
python3 ./test.py -f insert/bool.py
python3 ./test.py -f insert/double.py
python3 ./test.py -f insert/smallint.py
python3 ./test.py -f insert/tinyint.py
python3 ./test.py -f insert/date.py
python3 ./test.py -f insert/binary.py
python3 ./test.py -f insert/nchar.py
#python3 ./test.py -f insert/nchar-boundary.py
python3 ./test.py -f insert/nchar-unicode.py
python3 ./test.py -f insert/multi.py
python3 ./test.py -f insert/randomNullCommit.py
python3 insert/retentionpolicy.py
python3 ./test.py -f insert/alterTableAndInsert.py
python3 ./test.py -f insert/insertIntoTwoTables.py
python3 ./test.py -f insert/before_1970.py
python3 bug2265.py
#table
python3 ./test.py -f table/alter_wal0.py
python3 ./test.py -f table/column_name.py
python3 ./test.py -f table/column_num.py
python3 ./test.py -f table/db_table.py
python3 ./test.py -f table/create_sensitive.py
#python3 ./test.py -f table/tablename-boundary.py
python3 ./test.py -f table/max_table_length.py
python3 ./test.py -f table/alter_column.py
python3 ./test.py -f table/boundary.py
python3 ./test.py -f table/create.py
python3 ./test.py -f table/del_stable.py
# tag
python3 ./test.py -f tag_lite/filter.py
python3 ./test.py -f tag_lite/create-tags-boundary.py
python3 ./test.py -f tag_lite/3.py
python3 ./test.py -f tag_lite/4.py
python3 ./test.py -f tag_lite/5.py
python3 ./test.py -f tag_lite/6.py
python3 ./test.py -f tag_lite/add.py
python3 ./test.py -f tag_lite/bigint.py
python3 ./test.py -f tag_lite/binary_binary.py
python3 ./test.py -f tag_lite/binary.py
python3 ./test.py -f tag_lite/bool_binary.py
python3 ./test.py -f tag_lite/bool_int.py
python3 ./test.py -f tag_lite/bool.py
python3 ./test.py -f tag_lite/change.py
python3 ./test.py -f tag_lite/column.py
python3 ./test.py -f tag_lite/commit.py
python3 ./test.py -f tag_lite/create.py
python3 ./test.py -f tag_lite/datatype.py
python3 ./test.py -f tag_lite/datatype-without-alter.py
python3 ./test.py -f tag_lite/delete.py
python3 ./test.py -f tag_lite/double.py
python3 ./test.py -f tag_lite/float.py
python3 ./test.py -f tag_lite/int_binary.py
python3 ./test.py -f tag_lite/int_float.py
python3 ./test.py -f tag_lite/int.py
python3 ./test.py -f tag_lite/set.py
python3 ./test.py -f tag_lite/smallint.py
python3 ./test.py -f tag_lite/tinyint.py
#python3 ./test.py -f dbmgmt/database-name-boundary.py
python3 ./test.py -f import_merge/importBlock1HO.py
python3 ./test.py -f import_merge/importBlock1HPO.py
python3 ./test.py -f import_merge/importBlock1H.py
python3 ./test.py -f import_merge/importBlock1S.py
python3 ./test.py -f import_merge/importBlock1Sub.py
python3 ./test.py -f import_merge/importBlock1TO.py
python3 ./test.py -f import_merge/importBlock1TPO.py
python3 ./test.py -f import_merge/importBlock1T.py
python3 ./test.py -f import_merge/importBlock2HO.py
python3 ./test.py -f import_merge/importBlock2HPO.py
python3 ./test.py -f import_merge/importBlock2H.py
python3 ./test.py -f import_merge/importBlock2S.py
python3 ./test.py -f import_merge/importBlock2Sub.py
python3 ./test.py -f import_merge/importBlock2TO.py
python3 ./test.py -f import_merge/importBlock2TPO.py
python3 ./test.py -f import_merge/importBlock2T.py
python3 ./test.py -f import_merge/importBlockbetween.py
python3 ./test.py -f import_merge/importCacheFileHO.py
python3 ./test.py -f import_merge/importCacheFileHPO.py
python3 ./test.py -f import_merge/importCacheFileH.py
python3 ./test.py -f import_merge/importCacheFileS.py
python3 ./test.py -f import_merge/importCacheFileSub.py
python3 ./test.py -f import_merge/importCacheFileTO.py
python3 ./test.py -f import_merge/importCacheFileTPO.py
python3 ./test.py -f import_merge/importCacheFileT.py
python3 ./test.py -f import_merge/importDataH2.py
python3 ./test.py -f import_merge/importDataHO2.py
python3 ./test.py -f import_merge/importDataHO.py
python3 ./test.py -f import_merge/importDataHPO.py
python3 ./test.py -f import_merge/importDataLastHO.py
python3 ./test.py -f import_merge/importDataLastHPO.py
python3 ./test.py -f import_merge/importDataLastH.py
python3 ./test.py -f import_merge/importDataLastS.py
python3 ./test.py -f import_merge/importDataLastSub.py
python3 ./test.py -f import_merge/importDataLastTO.py
python3 ./test.py -f import_merge/importDataLastTPO.py
python3 ./test.py -f import_merge/importDataLastT.py
python3 ./test.py -f import_merge/importDataS.py
python3 ./test.py -f import_merge/importDataSub.py
python3 ./test.py -f import_merge/importDataTO.py
python3 ./test.py -f import_merge/importDataTPO.py
python3 ./test.py -f import_merge/importDataT.py
python3 ./test.py -f import_merge/importHeadOverlap.py
python3 ./test.py -f import_merge/importHeadPartOverlap.py
python3 ./test.py -f import_merge/importHead.py
python3 ./test.py -f import_merge/importHORestart.py
python3 ./test.py -f import_merge/importHPORestart.py
python3 ./test.py -f import_merge/importHRestart.py
python3 ./test.py -f import_merge/importLastHO.py
python3 ./test.py -f import_merge/importLastHPO.py
python3 ./test.py -f import_merge/importLastH.py
python3 ./test.py -f import_merge/importLastS.py
python3 ./test.py -f import_merge/importLastSub.py
python3 ./test.py -f import_merge/importLastTO.py
python3 ./test.py -f import_merge/importLastTPO.py
python3 ./test.py -f import_merge/importLastT.py
python3 ./test.py -f import_merge/importSpan.py
python3 ./test.py -f import_merge/importSRestart.py
python3 ./test.py -f import_merge/importSubRestart.py
python3 ./test.py -f import_merge/importTailOverlap.py
python3 ./test.py -f import_merge/importTailPartOverlap.py
python3 ./test.py -f import_merge/importTail.py
python3 ./test.py -f import_merge/importToCommit.py
python3 ./test.py -f import_merge/importTORestart.py
python3 ./test.py -f import_merge/importTPORestart.py
python3 ./test.py -f import_merge/importTRestart.py
python3 ./test.py -f import_merge/importInsertThenImport.py
python3 ./test.py -f import_merge/importCSV.py
\ No newline at end of file
tests/pytest/pytest_2.sh 已删除 100755 → 0
浏览文件 @ 2f0f9006
# update
python3 ./test.py -f update/allow_update.py
python3 ./test.py -f update/allow_update-0.py
python3 ./test.py -f update/append_commit_data.py
python3 ./test.py -f update/append_commit_last-0.py
python3 ./test.py -f update/append_commit_last.py
python3 ./test.py -f update/merge_commit_data.py
python3 ./test.py -f update/merge_commit_data2.py
python3 ./test.py -f update/merge_commit_data2_update0.py
python3 ./test.py -f update/merge_commit_last-0.py
python3 ./test.py -f update/merge_commit_last.py
python3 ./test.py -f update/bug_td2279.py
tests/pytest/pytest_3.sh 已删除 100755 → 0
浏览文件 @ 2f0f9006
#!/bin/bash
ulimit -c unlimited
python3 ./test.py -f insert/randomNullCommit.py
# user
python3 ./test.py -f user/user_create.py
python3 ./test.py -f user/pass_len.py
# stable
python3 ./test.py -f stable/query_after_reset.py
#query
python3 ./test.py -f query/filter.py
python3 ./test.py -f query/filterCombo.py
python3 ./test.py -f query/queryNormal.py
python3 ./test.py -f query/queryError.py
python3 ./test.py -f query/filterAllIntTypes.py
python3 ./test.py -f query/filterFloatAndDouble.py
python3 ./test.py -f query/filterOtherTypes.py
python3 ./test.py -f query/querySort.py
python3 ./test.py -f query/queryJoin.py
python3 ./test.py -f query/select_last_crash.py
python3 ./test.py -f query/queryNullValueTest.py
python3 ./test.py -f query/queryInsertValue.py
python3 ./test.py -f query/queryConnection.py
python3 ./test.py -f query/queryCountCSVData.py
python3 ./test.py -f query/natualInterval.py
python3 ./test.py -f query/bug1471.py
#python3 ./test.py -f query/dataLossTest.py
python3 ./test.py -f query/bug1874.py
python3 ./test.py -f query/bug1875.py
python3 ./test.py -f query/bug1876.py
python3 ./test.py -f query/bug2218.py
python3 ./test.py -f query/bug2117.py
python3 ./test.py -f query/bug2118.py
python3 ./test.py -f query/bug2143.py
python3 ./test.py -f query/sliding.py
python3 ./test.py -f query/unionAllTest.py
python3 ./test.py -f query/bug2281.py
python3 ./test.py -f query/bug2119.py
python3 ./test.py -f query/isNullTest.py
python3 ./test.py -f query/queryWithTaosdKilled.py
python3 ./test.py -f query/floatCompare.py
#stream
python3 ./test.py -f stream/metric_1.py
python3 ./test.py -f stream/metric_n.py
python3 ./test.py -f stream/new.py
python3 ./test.py -f stream/stream1.py
python3 ./test.py -f stream/stream2.py
#python3 ./test.py -f stream/parser.py
python3 ./test.py -f stream/history.py
python3 ./test.py -f stream/sys.py
python3 ./test.py -f stream/table_1.py
python3 ./test.py -f stream/table_n.py
#alter table
python3 ./test.py -f alter/alter_table_crash.py
# client
python3 ./test.py -f client/client.py
python3 ./test.py -f client/version.py
python3 ./test.py -f client/alterDatabase.py
python3 ./test.py -f client/noConnectionErrorTest.py
# Misc
python3 testCompress.py
python3 testNoCompress.py
python3 testMinTablesPerVnode.py
python3 queryCount.py
python3 ./test.py -f query/queryGroupbyWithInterval.py
python3 client/twoClients.py
python3 test.py -f query/queryInterval.py
python3 test.py -f query/queryFillTest.py
# tools
python3 test.py -f tools/lowaTest.py
python3 test.py -f tools/taosdemoTest.py
python3 test.py -f tools/taosdemoTestWithoutMetric.py
python3 test.py -f tools/taosdemoTestLimitOffset.py
python3 test.py -f tools/taosdemoTestSampleData.py
python3 test.py -f tools/taosdumpTest.py
#python3 test.py -f tools/taosdemoTest2.py
# subscribe
python3 test.py -f subscribe/singlemeter.py
#python3 test.py -f subscribe/stability.py
python3 test.py -f subscribe/supertable.py
tests/pytest/pytest_4.sh 已删除 100755 → 0
浏览文件 @ 2f0f9006
python3 ./test.py -f update/merge_commit_data-0.py
# wal
python3 ./test.py -f wal/addOldWalTest.py
# function
python3 ./test.py -f functions/all_null_value.py
# functions
python3 ./test.py -f functions/function_avg.py -r 1
python3 ./test.py -f functions/function_bottom.py -r 1
python3 ./test.py -f functions/function_count.py -r 1
python3 ./test.py -f functions/function_diff.py -r 1
python3 ./test.py -f functions/function_first.py -r 1
python3 ./test.py -f functions/function_last.py -r 1
python3 ./test.py -f functions/function_last_row.py -r 1
python3 ./test.py -f functions/function_leastsquares.py -r 1
python3 ./test.py -f functions/function_max.py -r 1
python3 ./test.py -f functions/function_min.py -r 1
python3 ./test.py -f functions/function_operations.py -r 1
python3 ./test.py -f functions/function_percentile.py -r 1
python3 ./test.py -f functions/function_spread.py -r 1
python3 ./test.py -f functions/function_stddev.py -r 1
python3 ./test.py -f functions/function_sum.py -r 1
python3 ./test.py -f functions/function_top.py -r 1
python3 ./test.py -f functions/function_twa.py -r 1
python3 ./test.py -f functions/function_twa_test2.py
python3 ./test.py -f functions/function_stddev_td2555.py
python3 ./test.py -f insert/metadataUpdate.py
python3 ./test.py -f tools/taosdemoTest2.py
python3 ./test.py -f query/last_cache.py
python3 ./test.py -f query/last_row_cache.py
python3 ./test.py -f account/account_create.py
python3 ./test.py -f alter/alter_table.py
python3 ./test.py -f query/queryGroupbySort.py
python3 ./test.py -f insert/unsignedInt.py
python3 ./test.py -f insert/unsignedBigint.py
python3 ./test.py -f insert/unsignedSmallint.py
python3 ./test.py -f insert/unsignedTinyint.py
python3 ./test.py -f query/filterAllUnsignedIntTypes.py
\ No newline at end of file
tests/pytest/query/filter.py
浏览文件 @ 59790c4d
......@@ -28,18 +28,18 @@ class TDTestCase:
print("==============step1")
tdSql.execute(
"create table if not exists st (ts timestamp, tagtype int, name nchar(16)) tags(dev nchar(50))")
"create table if not exists st (ts timestamp, tagtype int, name nchar(16), col4 binary(16)) tags(dev nchar(50), tag2 binary(16))")
tdSql.execute(
'CREATE TABLE if not exists dev_001 using st tags("dev_01")')
'CREATE TABLE if not exists dev_001 using st tags("dev_01", "tag_01")')
tdSql.execute(
'CREATE TABLE if not exists dev_002 using st tags("dev_02")')
'CREATE TABLE if not exists dev_002 using st tags("dev_02", "tag_02")')
print("==============step2")
tdSql.execute(
"""INSERT INTO dev_001(ts, tagtype, name) VALUES('2020-05-13 10:00:00.000', 1, 'first'),('2020-05-13 10:00:00.001', 2, 'second'),
('2020-05-13 10:00:00.002', 3, 'third') dev_002 VALUES('2020-05-13 10:00:00.003', 1, 'first'), ('2020-05-13 10:00:00.004', 2, 'second'),
('2020-05-13 10:00:00.005', 3, 'third')""")
"""INSERT INTO dev_001 VALUES('2020-05-13 10:00:00.000', 1, 'first', 'binary1'),('2020-05-13 10:00:00.001', 2, 'second', 'binary2'),
('2020-05-13 10:00:00.002', 3, 'third' , 'binary3') dev_002 VALUES('2020-05-13 10:00:00.003', 1, 'first', 'binary4'), ('2020-05-13 10:00:00.004', 2, 'second', 'binary5'),
('2020-05-13 10:00:00.005', 3, 'third', 'binary6')""")
# > for timestamp type
tdSql.query("select * from db.st where ts > '2020-05-13 10:00:00.002'")
......@@ -85,6 +85,12 @@ class TDTestCase:
tdSql.query("select * from db.st where name = 'first'")
tdSql.checkRows(2)
tdSql.query("select * from db.st where col4 = 1231231")
tdSql.checkRows(0)
tdSql.query("select * from db.st where name = 1231231")
tdSql.checkRows(0)
# <> for timestamp type
tdSql.query("select * from db.st where ts <> '2020-05-13 10:00:00.002'")
# tdSql.checkRows(4)
......@@ -105,6 +111,13 @@ class TDTestCase:
tdSql.query("select * from db.st where name like '_econd'")
tdSql.checkRows(2)
# for tag
tdSql.query("select * from db.st where dev=1")
tdSql.checkRows(0)
tdSql.query("select * from db.st where tag2=1")
tdSql.checkRows(0)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
tests/pytest/tools/insert-interlace.json 0 → 100644
浏览文件 @ 59790c4d
{
"filetype": "insert",
"cfgdir": "/etc/taos",
"host": "127.0.0.1",
"port": 6030,
"user": "root",
"password": "taosdata",
"thread_count": 4,
"thread_count_create_tbl": 4,
"result_file": "./insert_res.txt",
"confirm_parameter_prompt": "no",
"insert_interval": 5000,
"rows_per_tbl": 50,
"num_of_records_per_req": 100,
"max_sql_len": 1024000,
"databases": [{
"dbinfo": {
"name": "db",
"drop": "yes",
"replica": 1,
"days": 10,
"cache": 16,
"blocks": 8,
"precision": "ms",
"keep": 365,
"minRows": 100,
"maxRows": 4096,
"comp":2,
"walLevel":1,
"cachelast":0,
"quorum":1,
"fsync":3000,
"update": 0
},
"super_tables": [{
"name": "stb",
"child_table_exists":"no",
"childtable_count": 9,
"childtable_prefix": "stb_",
"auto_create_table": "no",
"data_source": "rand",
"insert_mode": "taosc",
"insert_rows": 250,
"multi_thread_write_one_tbl": "no",
"rows_per_tbl": 80,
"max_sql_len": 1024000,
"disorder_ratio": 0,
"disorder_range": 1000,
"timestamp_step": 1,
"start_timestamp": "2020-10-01 00:00:00.000",
"sample_format": "csv",
"sample_file": "./sample.csv",
"tags_file": "",
"columns": [{"type": "INT"}],
"tags": [{"type": "INT", "count":1}]
}]
}]
}
tests/pytest/tools/insert-tblimit-tboffset0.json
浏览文件 @ 59790c4d
......@@ -44,7 +44,6 @@
"childtable_offset": 0,
"multi_thread_write_one_tbl": "no",
"number_of_tbl_in_one_sql": 0,
"rows_per_tbl": 100,
"max_sql_len": 1024000,
"disorder_ratio": 0,
"disorder_range": 1000,
......
tests/pytest/tools/insert-tblimit1-tboffset.json 0 → 100644
浏览文件 @ 59790c4d
{
"filetype": "insert",
"cfgdir": "/etc/taos",
"host": "127.0.0.1",
"port": 6030,
"user": "root",
"password": "taosdata",
"thread_count": 4,
"thread_count_create_tbl": 4,
"result_file": "./insert_res.txt",
"confirm_parameter_prompt": "no",
"insert_interval": 0,
"num_of_records_per_req": 100,
"max_sql_len": 1024000,
"databases": [{
"dbinfo": {
"name": "db",
"drop": "yes",
"replica": 1,
"days": 10,
"cache": 16,
"blocks": 8,
"precision": "ms",
"keep": 365,
"minRows": 100,
"maxRows": 4096,
"comp":2,
"walLevel":1,
"cachelast":0,
"quorum":1,
"fsync":3000,
"update": 0
},
"super_tables": [{
"name": "stb",
"child_table_exists":"no",
"childtable_count": 100,
"childtable_prefix": "stb_",
"auto_create_table": "no",
"data_source": "rand",
"insert_mode": "taosc",
"insert_rows": 1000,
"childtable_limit": 1,
"childtable_offset": 50,
"multi_thread_write_one_tbl": "no",
"number_of_tbl_in_one_sql": 0,
"max_sql_len": 1024000,
"disorder_ratio": 0,
"disorder_range": 1000,
"timestamp_step": 1,
"start_timestamp": "2020-10-01 00:00:00.000",
"sample_format": "csv",
"sample_file": "./sample.csv",
"tags_file": "",
"columns": [{"type": "INT"}],
"tags": [{"type": "TINYINT", "count":1}]
}]
}]
}
tests/pytest/tools/insert.json
浏览文件 @ 59790c4d
{
"filetype":"insert",
"cfgdir": "/etc/taos",
"host": "127.0.0.1",
"port": 6030,
"user": "root",
"password": "taosdata",
"thread_count": 1,
"databases": [{
"dbinfo": {
"name": "db01",
"replica": 1,
"days": 10,
"cache": 16,
"blocks": 8,
"precision": "ms",
"host": "127.0.0.1",
"port": 6030,
"user": "root",
"password": "taosdata",
"thread_count": 4,
"thread_count_create_tbl": 4,
"databases": [{
"dbinfo": {
"name": "db01",
"replica": 1,
"days": 10,
"cache": 16,
"blocks": 8,
"precision": "ms",
"update": 0,
"maxtablesPerVnode": 1000
},
"super_tables": [{
"name": "stb01",
"childtable_count": 100,
"childtable_prefix": "stb01_",
"auto_create_table": "no",
"data_source": "rand",
"insert_mode": "taosc",
"insert_rate": 0,
"insert_rows": 1000,
"timestamp_step": 1000,
"start_timestamp": "2020-10-01 00:00:00.000",
"sample_format": "csv",
"sample_file": "/home/data/sample.csv",
"tags_file": "",
"columns": [{
"type": "SMALLINT"
}, {
"type": "BOOL"
}, {
"type": "BINARY",
"len": 6
}],
"tags": [{
"type": "INT"
},{
"type": "BINARY",
"len": 4
}]
}]
}]
"maxtablesPerVnode": 1000
},
"super_tables": [{
"name": "stb01",
"childtable_count": 100,
"childtable_prefix": "stb01_",
"auto_create_table": "no",
"data_source": "rand",
"insert_mode": "taosc",
"insert_rate": 0,
"insert_rows": 1000,
"timestamp_step": 1000,
"start_timestamp": "2020-10-01 00:00:00.000",
"sample_format": "csv",
"sample_file": "/home/data/sample.csv",
"tags_file": "",
"columns": [{
"type": "SMALLINT"
}, {
"type": "BOOL"
}, {
"type": "BINARY",
"len": 6
}],
"tags": [{
"type": "INT"
},{
"type": "BINARY",
"len": 4
}]
}]
}]
}
tests/pytest/tools/taosdemoTestInterlace.py 0 → 100644
浏览文件 @ 59790c4d
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import os
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.numberOfTables = 10000
self.numberOfRecords = 100
def getBuildPath(self):
selfPath = os.path.dirname(os.path.realpath(__file__))
if ("community" in selfPath):
projPath = selfPath[:selfPath.find("community")]
else:
projPath = selfPath[:selfPath.find("tests")]
for root, dirs, files in os.walk(projPath):
if ("taosd" in files):
rootRealPath = os.path.dirname(os.path.realpath(root))
if ("packaging" not in rootRealPath):
buildPath = root[:len(root)-len("/build/bin")]
break
return buildPath
def run(self):
tdSql.prepare()
buildPath = self.getBuildPath()
if (buildPath == ""):
tdLog.exit("taosd not found!")
else:
tdLog.info("taosd found in %s" % buildPath)
binPath = buildPath+ "/build/bin/"
os.system("%staosdemo -f tools/insert-interlace.json" % binPath)
tdSql.execute("use db")
tdSql.query("select count(tbname) from db.stb")
tdSql.checkData(0, 0, 100)
tdSql.query("select count(*) from db.stb")
tdSql.checkData(0, 0, 33000)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
tests/pytest/tools/taosdemoTestLimitOffset.py
浏览文件 @ 59790c4d
......@@ -68,6 +68,15 @@ class TDTestCase:
tdSql.query("select count(*) from db.stb")
tdSql.checkData(0, 0, 20000)
os.system("%staosdemo -f tools/insert-tblimit1-tboffset.json" % binPath)
tdSql.execute("reset query cache")
tdSql.execute("use db")
tdSql.query("select count(tbname) from db.stb")
tdSql.checkData(0, 0, 100)
tdSql.query("select count(*) from db.stb")
tdSql.checkData(0, 0, 1000)
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
......
tests/script/general/parser/create_mt.sim
浏览文件 @ 59790c4d
......@@ -126,8 +126,7 @@ $tb_ = table
$tbs = tables
$db_ = database
$dbs = databases
$mt_ = metric
$mts = metrics
$ses = session
$int = int
$bint = bigint
$binary = binary
......@@ -145,8 +144,7 @@ sql_error create table $mt (ts timestamp, col1 int) tags ( $tb_ int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $tbs int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $db_ int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $dbs int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $mt_ int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $mts int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $ses int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $int int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $bint int)
sql_error create table $mt (ts timestamp, col1 int) tags ( $binary int)
......
tests/script/general/parser/create_tb.sim
浏览文件 @ 59790c4d
......@@ -90,8 +90,7 @@ $tb_ = table
$tbs = tables
$db_ = database
$dbs = databases
$mt_ = metric
$mts = metrics
$ses = session
$int = int
$bint = bigint
$binary = binary
......@@ -105,9 +104,8 @@ $nchar = nchar
sql_error create table $tb (ts timestamp, $tb_ int)
sql_error create table $tb (ts timestamp, $tbs int)
sql_error create table $tb (ts timestamp, $db_ int)
sql_error create table $tb (ts timestamp, $dbs int)
sql_error create table $tb (ts timestamp, $mt_ int)
sql_error create table $tb (ts timestamp, $mts int)
sql_error create table $tb (ts timestamp, $dbs int)
sql_error create table $tb (ts timestamp, $ses int)
sql_error create table $tb (ts timestamp, $int int)
sql_error create table $tb (ts timestamp, $bint int)
sql_error create table $tb (ts timestamp, $binary int)
......
tests/script/jenkins/basic.txt
浏览文件 @ 59790c4d
此差异已折叠。
tests/script/jenkins/basic_1.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_2.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_3.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_4.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_5.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_6.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/script/jenkins/basic_7.txt 已删除 100644 → 0
浏览文件 @ 2f0f9006
此差异已折叠。
tests/test-all.sh
浏览文件 @ 59790c4d
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或