Merge branch '3.0' into feat/sangshuduo/TD-14141-update-taostools-for3.0

docs/zh/12-taos-sql/02-database.md

@@ -8,17 +8,17 @@ description: "创建、删除数据库，查看、修改数据库参数"
 
 ```sql
 CREATE DATABASE [IF NOT EXISTS] db_name [database_options]
-
+ 
 database_options:
     database_option ...
-
+ 
 database_option: {
     BUFFER value
   | CACHEMODEL {'none' | 'last_row' | 'last_value' | 'both'}
   | CACHESIZE value
   | COMP {0 | 1 | 2}
   | DURATION value
-  | FSYNC value
+  | WAL_FSYNC_PERIOD value
   | MAXROWS value
   | MINROWS value
   | KEEP value
@@ -28,7 +28,7 @@ database_option: {
   | REPLICA value
   | RETENTIONS ingestion_duration:keep_duration ...
   | STRICT {'off' | 'on'}
-  | WAL {1 | 2}
+  | WAL_LEVEL {1 | 2}
   | VGROUPS value
   | SINGLE_STABLE {0 | 1}
   | WAL_RETENTION_PERIOD value
@@ -46,7 +46,7 @@ database_option: {
   - last_row：表示缓存子表最近一行数据。这将显著改善 LAST_ROW 函数的性能表现。
   - last_value：表示缓存子表每一列的最近的非 NULL 值。这将显著改善无特殊影响（WHERE、ORDER BY、GROUP BY、INTERVAL）下的 LAST 函数的性能表现。
   - both：表示同时打开缓存最近行和列功能。
-- CACHESIZE：表示缓存子表最近数据的内存大小。默认为 1 ，范围是[1, 65536]，单位是 MB。
+- CACHESIZE：表示每个 vnode 中用于缓存子表最近数据的内存大小。默认为 1 ，范围是[1, 65536]，单位是 MB。
 - COMP：表示数据库文件压缩标志位，缺省值为 2，取值范围为 [0, 2]。
   - 0：表示不压缩。
   - 1：表示一阶段压缩。

docs/zh/12-taos-sql/21-node.md

@@ -40,7 +40,6 @@ ALTER ALL DNODES dnode_option
 
 dnode_option: {
     'resetLog'
-  | 'resetQueryCache'
   | 'balance' value
   | 'monitor' value
   | 'debugFlag' value

docs/zh/14-reference/14-taosx.md

+---
+sidebar_label: taosX
+title: 使用 taosX 在集群间复制数据
+---
\ No newline at end of file

docs/zh/14-reference/_category_.yml

-label: 参考指南
+label: 参考手册
\ No newline at end of file

docs/zh/14-reference/index.md

 ---
-sidebar_label: 参考手册
 title: 参考手册
 ---
 

docs/zh/17-operation/02-planning.mdx

@@ -5,33 +5,29 @@ title: 容量规划
 
 使用 TDengine 来搭建一个物联网大数据平台，计算资源、存储资源需要根据业务场景进行规划。下面分别讨论系统运行所需要的内存、CPU 以及硬盘空间。
 
-## 内存需求
+## 服务端内存需求
 
-每个 Database 可以创建固定数目的 vgroup，默认与 CPU 核数相同，可通过 maxVgroupsPerDb 配置；vgroup 中的每个副本会是一个 vnode；每个 vnode 会占用固定大小的内存（大小与数据库的配置参数 blocks 和 cache 有关)；每个 Table 会占用与标签总长度有关的内存；此外，系统会有一些固定的内存开销。因此，每个 DB 需要的系统内存可通过如下公式计算：
+每个 Database 可以创建固定数目的 vgroup，默认 2 个 vgroup，在创建数据库时可以通过`vgroups <num>`参数来指定，其副本数由参数`replica <num>`指定。vgroup 中的每个副本会是一个 vnode；所以每个数据库占用的内存由以下几个参数决定：
 
-```
-Database Memory Size = maxVgroupsPerDb * replica * (blocks * cache + 10MB) + numOfTables * (tagSizePerTable + 0.5KB)
-```
+- vgroups
+- replica
+- buffer
+- pages
+- pagesize
+- cachesize
 
-示例：假设 maxVgroupPerDB 是缺省值 64，cache 是缺省大小 16M, blocks 是缺省值 6，并且一个 DB 中有 10 万张表，单副本，标签总长度是 256 字节，则这个 DB 总的内存需求为：64 \* 1 \* (16 \* 6 + 10) + 100000 \* (0.25 + 0.5) / 1000 = 6792M。
+关于这些参数的详细说明请参考 [数据库管理](../taos-sql/database)。
 
-在实际的系统运维中，我们通常会更关心 TDengine 服务进程（taosd）会占用的内存量。
+一个数据库所需要的内存大小等于 
 
 ```
-taosd 内存总量 = vnode 内存 + mnode 内存 + 查询内存
+vgroups * replica * (buffer + pages * pagesize + cachesize)
 ```
 
-其中：
-
-1. “vnode 内存”指的是集群中所有的 Database 存储分摊到当前 taosd 节点上所占用的内存资源。可以按上文“Database Memory Size”计算公式估算每个 DB 的内存占用量进行加总，再按集群中总共的 TDengine 节点数做平均（如果设置为多副本，则还需要乘以对应的副本倍数）。
-2. “mnode 内存”指的是集群中管理节点所占用的资源。如果一个 taosd 节点上分布有 mnode 管理节点，则内存消耗还需要增加“0.2KB \* 集群中数据表总数”。
-3. “查询内存”指的是服务端处理查询请求时所需要占用的内存。单条查询语句至少会占用“0.2KB \* 查询涉及的数据表总数”的内存量。
+但要注意的是这些内存并不需要由单一服务器提供，而是由整个集群中所有数据节点共同负担，相当于由这些数据节点所在的服务器共同负担。如果集群中有不止一个数据库，则所需内存要累加，并由集群中所有服务器共同负担。更复杂的场景是如果集群中的数据节点并非在最初就一次性全部建立，而是随着使用中系统负载的增加逐步增加服务器并增加数据节点，则新创建的数据库会导致新旧数据节点上的负载并不均衡，此时简单的理论计算并不能直接使用，要结合各数据节点的负载情况。
 
-注意：以上内存估算方法，主要讲解了系统的“必须内存需求”，而不是“内存总数上限”。在实际运行的生产环境中，由于操作系统缓存、资源管理调度等方面的原因，内存规划应当在估算结果的基础上保留一定冗余，以维持系统状态和系统性能的稳定性。并且，生产环境通常会配置系统资源的监控工具，以便及时发现硬件资源的紧缺情况。
 
-最后，如果内存充裕，可以考虑加大 Blocks 的配置，这样更多数据将保存在内存里，提高写入和查询速度。
-
-### 客户端内存需求
+## 客户端内存需求
 
 客户端应用采用 taosc 客户端驱动连接服务端，会有内存需求的开销。
 
@@ -77,5 +73,3 @@ Raw DataSize = numOfTables * rowSizePerTable * rowsPerTable
 根据上面的内存、CPU、存储的预估，就可以知道整个系统需要多少核、多少内存、多少存储空间。如果数据副本数不为 1，总需求量需要再乘以副本数。
 
 因为 TDengine 具有很好的水平扩展能力，根据总量，再根据单个物理机或虚拟机的资源，就可以轻松决定需要购置多少台物理机或虚拟机了。
-
-**立即计算 CPU、内存、存储，请参见：[资源估算方法](https://www.taosdata.com/config/config.html)。**

docs/zh/17-operation/03-tolerance.md

@@ -10,19 +10,21 @@ TDengine 接收到应用的请求数据包时，先将请求的原始数据包
 
 涉及的系统配置参数有两个：
 
-- walLevel：WAL 级别，0：不写 WAL; 1：写 WAL, 但不执行 fsync; 2：写 WAL, 而且执行 fsync。
-- fsync：当 walLevel 设置为 2 时，执行 fsync 的周期。设置为 0，表示每次写入，立即执行 fsync。
+- wal_level：WAL 级别，1：写WAL，但不执行fsync。2：写WAL，而且执行fsync。默认值为 1。
+- wal_fsync_period：当 wal_evel 设置为 2 时，执行 fsync 的周期。设置为 0，表示每次写入，立即执行 fsync。
 
-如果要 100%的保证数据不丢失，需要将 walLevel 设置为 2，fsync 设置为 0。这时写入速度将会下降。但如果应用侧启动的写数据的线程数达到一定的数量(超过 50)，那么写入数据的性能也会很不错，只会比 fsync 设置为 3000 毫秒下降 30%左右。
+如果要 100%的保证数据不丢失，需要将 wal_level 设置为 2，fsync 设置为 0。这时写入速度将会下降。但如果应用侧启动的写数据的线程数达到一定的数量(超过 50)，那么写入数据的性能也会很不错，只会比 fsync 设置为 3000 毫秒下降 30%左右。
 
 ## 灾备
 
-TDengine 的集群通过多个副本的机制，来提供系统的高可用性，实现灾备能力。
+TDengine 的集群通过多个副本的机制，来提供系统的高可用性，同时具备一定的灾备能力。
 
-TDengine 集群是由 mnode 负责管理的，为保证 mnode 的高可靠，可以配置多个 mnode 副本，副本数由系统配置参数 numOfMnodes 决定，为了支持高可靠，需要设置大于 1。为保证元数据的强一致性，mnode 副本之间通过同步方式进行数据复制，保证了元数据的强一致性。
+TDengine 集群是由 mnode 负责管理的，为保证 mnode 的高可靠，可以配置 三个 mnode 副本。为保证元数据的强一致性，mnode 副本之间通过同步方式进行数据复制，保证了元数据的强一致性。
 
-TDengine 集群中的时序数据的副本数是与数据库关联的，一个集群里可以有多个数据库，每个数据库可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数。为了支持高可靠，需要设置副本数大于 1。
+TDengine 集群中的时序数据的副本数是与数据库关联的，一个集群里可以有多个数据库，每个数据库可以配置不同的副本数。创建数据库时，通过参数 replica 指定副本数。为了支持高可靠，需要设置副本数为 3。
 
 TDengine 集群的节点数必须大于等于副本数，否则创建表时将报错。
 
 当 TDengine 集群中的节点部署在不同的物理机上，并设置多个副本数时，就实现了系统的高可靠性，无需再使用其他软件或工具。TDengine 企业版还可以将副本部署在不同机房，从而实现异地容灾。
+
+另外一种灾备方式是通过 `taosX` 将一个 TDengine 集群的数据同步复制到物理上位于不同数据中心的另一个 TDengine 集群。其详细使用方法请参考 [taosX 参考手册](../reference/taosX)

docs/zh/17-operation/06-admin.md

----
-title: 用户管理
----
-
-系统管理员可以在 CLI 界面里添加、删除用户，也可以修改密码。CLI 里 SQL 语法如下：
-
-```sql
-CREATE USER <user_name> PASS <'password'>;
-```
-
-创建用户，并指定用户名和密码，密码需要用单引号引起来，单引号为英文半角
-
-```sql
-DROP USER <user_name>;
-```
-
-删除用户，限 root 用户使用
-
-```sql
-ALTER USER <user_name> PASS <'password'>;
-```
-
-修改用户密码，为避免被转换为小写，密码需要用单引号引用，单引号为英文半角
-
-```sql
-ALTER USER <user_name> PRIVILEGE <write|read>;
-```
-
-修改用户权限为：write 或 read，不需要添加单引号
-
-说明：系统内共有 super/write/read 三种权限级别，但目前不允许通过 alter 指令把 super 权限赋予用户。
-
-```sql
-SHOW USERS;
-```
-
-显示所有用户
-
-:::note
-SQL 语法中，< >表示需要用户输入的部分，但请不要输入< >本身。
-
-:::

docs/zh/17-operation/09-status.md

----
-title: 系统连接、任务查询管理
----
-
-系统管理员可以从 CLI 查询系统的连接、正在进行的查询、流式计算，并且可以关闭连接、停止正在进行的查询和流式计算。
-
-## 显示数据库的连接
-
-```sql
-SHOW CONNECTIONS;
-```
-
-其结果中的一列显示 ip:port, 为连接的 IP 地址和端口号。
-
-## 强制关闭数据库连接
-
-```sql
-KILL CONNECTION <connection-id>;
-```
-
-其中的 connection-id 是 SHOW CONNECTIONS 中显示的第一列的数字。
-
-## 显示数据查询
-
-```sql
-SHOW QUERIES;
-```
-
-其中第一列显示的以冒号隔开的两个数字为 query-id，为发起该 query 应用连接的 connection-id 和查询次数。
-
-## 强制关闭数据查询
-
-```sql
-KILL QUERY <query-id>;
-```
-
-其中 query-id 是 SHOW QUERIES 中显示的 connection-id:query-no 字串，如“105:2”，拷贝粘贴即可。
-
-## 显示连续查询
-
-```sql
-SHOW STREAMS;
-```
-
-其中第一列显示的以冒号隔开的两个数字为 stream-id, 为启动该 stream 应用连接的 connection-id 和发起 stream 的次数。
-
-## 强制关闭连续查询
-
-```sql
-KILL STREAM <stream-id>;
-```
-
-其中的 stream-id 是 SHOW STREAMS 中显示的 connection-id:stream-no 字串，如 103:2，拷贝粘贴即可。

include/common/tmsg.h

@@ -3044,12 +3044,41 @@ typedef struct SDeleteRes {
   int64_t  skey;
   int64_t  ekey;
   int64_t  affectedRows;
-  char     tableFName[TSDB_TABLE_FNAME_LEN];
+  char     tableFName[TSDB_TABLE_NAME_LEN];
+  char     tsColName[TSDB_COL_NAME_LEN];
 } SDeleteRes;
 
 int32_t tEncodeDeleteRes(SEncoder* pCoder, const SDeleteRes* pRes);
 int32_t tDecodeDeleteRes(SDecoder* pCoder, SDeleteRes* pRes);
 
+typedef struct {
+  int32_t msgType;
+  int32_t msgLen;
+  void*   msg;
+} SBatchMsg;
+
+typedef struct {
+  SMsgHead  header;
+  int32_t   msgNum;
+  SBatchMsg msg[];
+} SBatchReq;
+
+typedef struct {
+  int32_t reqType;
+  int32_t msgLen;
+  int32_t rspCode;
+  void*   msg;
+} SBatchRsp;
+
+static FORCE_INLINE void tFreeSBatchRsp(void *p) {
+  if (NULL == p) {
+    return;
+  }
+
+  SBatchRsp* pRsp = (SBatchRsp*)p;
+  taosMemoryFree(pRsp->msg);
+}
+
 #pragma pack(pop)
 
 #ifdef __cplusplus

include/common/tmsgdef.h

@@ -136,6 +136,7 @@ enum {
   TD_DEF_MSG_TYPE(TDMT_MND_DROP_INDEX, "drop-index", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_MND_GET_INDEX, "get-index", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_MND_GET_TABLE_INDEX, "get-table-index", NULL, NULL)
+  TD_DEF_MSG_TYPE(TDMT_MND_BATCH_META, "batch-meta", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_MND_TABLE_CFG, "table-cfg", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_MND_CREATE_TOPIC, "create-topic", SMCreateTopicReq, SMCreateTopicRsp)
   TD_DEF_MSG_TYPE(TDMT_MND_ALTER_TOPIC, "alter-topic", NULL, NULL)
@@ -180,6 +181,7 @@ enum {
   TD_DEF_MSG_TYPE(TDMT_VND_TABLE_META, "vnode-table-meta", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_VND_TABLES_META, "vnode-tables-meta", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_VND_TABLE_CFG, "vnode-table-cfg", NULL, NULL)
+  TD_DEF_MSG_TYPE(TDMT_VND_BATCH_META, "vnode-batch-meta", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_VND_CREATE_STB, "vnode-create-stb", SVCreateStbReq, NULL)
   TD_DEF_MSG_TYPE(TDMT_VND_ALTER_STB, "vnode-alter-stb", NULL, NULL)
   TD_DEF_MSG_TYPE(TDMT_VND_DROP_STB, "vnode-drop-stb", SVDropStbReq, NULL)

include/libs/executor/dataSinkMgt.h

@@ -38,7 +38,8 @@ typedef struct SDeleterRes {
   int64_t  skey;
   int64_t  ekey;
   int64_t  affectedRows;
-  char     tableFName[TSDB_TABLE_FNAME_LEN];
+  char     tableName[TSDB_TABLE_NAME_LEN];
+  char     tsColName[TSDB_COL_NAME_LEN];
 } SDeleterRes;
 
 typedef struct SDeleterParam {

include/libs/executor/executor.h

@@ -64,17 +64,7 @@ qTaskInfo_t qCreateStreamExecTaskInfo(void* msg, SReadHandle* readers);
  * @param SReadHandle
  * @return
  */
-qTaskInfo_t qCreateQueueExecTaskInfo(void* msg, SReadHandle* readers, int32_t* numOfCols,
-                                     SSchemaWrapper** pSchema);
-
-/**
- * Set the input data block for the stream scan.
- * @param tinfo
- * @param input
- * @param type
- * @return
- */
-int32_t qSetStreamInput(qTaskInfo_t tinfo, const void* input, int32_t type, bool assignUid);
+qTaskInfo_t qCreateQueueExecTaskInfo(void* msg, SReadHandle* readers, int32_t* numOfCols, SSchemaWrapper** pSchema);
 
 /**
  * Set multiple input data blocks for the stream scan.
@@ -84,7 +74,7 @@ int32_t qSetStreamInput(qTaskInfo_t tinfo, const void* input, int32_t type, bool
  * @param type
  * @return
  */
-int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numOfBlocks, int32_t type, bool assignUid);
+int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numOfBlocks, int32_t type);
 
 /**
  * Update the table id list, add or remove.

include/libs/nodes/plannodes.h

@@ -151,7 +151,8 @@ typedef struct SVnodeModifyLogicNode {
   uint64_t         tableId;
   uint64_t         stableId;
   int8_t           tableType;  // table type
-  char             tableFName[TSDB_TABLE_FNAME_LEN];
+  char             tableName[TSDB_TABLE_NAME_LEN];
+  char             tsColName[TSDB_COL_NAME_LEN];
   STimeWindow      deleteTimeRange;
   SVgroupsInfo*    pVgroupList;
   SNodeList*       pInsertCols;
@@ -494,7 +495,7 @@ typedef struct SQueryInserterNode {
   uint64_t      tableId;
   uint64_t      stableId;
   int8_t        tableType;  // table type
-  char          tableFName[TSDB_TABLE_FNAME_LEN];
+  char          tableName[TSDB_TABLE_NAME_LEN];
   int32_t       vgId;
   SEpSet        epSet;
 } SQueryInserterNode;
@@ -503,7 +504,7 @@ typedef struct SDataDeleterNode {
   SDataSinkNode sink;
   uint64_t      tableId;
   int8_t        tableType;  // table type
-  char          tableFName[TSDB_TABLE_FNAME_LEN];
+  char          tableFName[TSDB_TABLE_NAME_LEN];
   char          tsColName[TSDB_COL_NAME_LEN];
   STimeWindow   deleteTimeRange;
   SNode*        pAffectedRows;

include/libs/stream/tstream.h

@@ -270,7 +270,7 @@ typedef struct SStreamTask {
   int64_t startVer;
   int64_t checkpointVer;
   int64_t processedVer;
-  int32_t numOfVgroups;
+  // int32_t numOfVgroups;
 
   // children info
   SArray* childEpInfo;  // SArray<SStreamChildEpInfo*>

source/client/src/tmq.c

@@ -2823,35 +2823,35 @@ end:
 
 // delete from db.tabl where ..       -> delete from tabl where ..
 // delete from db    .tabl where ..   -> delete from tabl where ..
-static void getTbName(char *sql){
-  char *ch = sql;
-
-  bool inBackQuote = false;
-  int8_t dotIndex = 0;
-  while(*ch != '\0'){
-    if(!inBackQuote && *ch == '`'){
-      inBackQuote = true;
-      ch++;
-      continue;
-    }
-
-    if(inBackQuote && *ch == '`'){
-      inBackQuote = false;
-      ch++;
-
-      continue;
-    }
-
-    if(!inBackQuote && *ch == '.'){
-      dotIndex ++;
-      if(dotIndex == 2){
-        memmove(sql, ch + 1, strlen(ch + 1) + 1);
-        break;
-      }
-    }
-    ch++;
-  }
-}
+//static void getTbName(char *sql){
+//  char *ch = sql;
+//
+//  bool inBackQuote = false;
+//  int8_t dotIndex = 0;
+//  while(*ch != '\0'){
+//    if(!inBackQuote && *ch == '`'){
+//      inBackQuote = true;
+//      ch++;
+//      continue;
+//    }
+//
+//    if(inBackQuote && *ch == '`'){
+//      inBackQuote = false;
+//      ch++;
+//
+//      continue;
+//    }
+//
+//    if(!inBackQuote && *ch == '.'){
+//      dotIndex ++;
+//      if(dotIndex == 2){
+//        memmove(sql, ch + 1, strlen(ch + 1) + 1);
+//        break;
+//      }
+//    }
+//    ch++;
+//  }
+//}
 
 static int32_t taosDeleteData(TAOS* taos, void* meta, int32_t metaLen) {
   SDeleteRes     req = {0};
@@ -2867,9 +2867,9 @@ static int32_t taosDeleteData(TAOS* taos, void* meta, int32_t metaLen) {
     goto end;
   }
 
-  getTbName(req.tableFName);
+//  getTbName(req.tableFName);
   char sql[256] = {0};
-  sprintf(sql, "delete from `%s` where `%s` >= %" PRId64" and `%s` <= %" PRId64, req.tableFName, "ts", req.skey, "ts", req.ekey);
+  sprintf(sql, "delete from `%s` where `%s` >= %" PRId64" and `%s` <= %" PRId64, req.tableFName, req.tsColName, req.skey, req.tsColName, req.ekey);
   printf("delete sql:%s\n", sql);
 
   TAOS_RES* res = taos_query(taos, sql);

source/common/src/tmsg.c

@@ -5682,6 +5682,7 @@ int32_t tEncodeDeleteRes(SEncoder *pCoder, const SDeleteRes *pRes) {
   if (tEncodeI64v(pCoder, pRes->affectedRows) < 0) return -1;
 
   if (tEncodeCStr(pCoder, pRes->tableFName) < 0) return -1;
+  if (tEncodeCStr(pCoder, pRes->tsColName) < 0) return -1;
   return 0;
 }
 
@@ -5700,6 +5701,7 @@ int32_t tDecodeDeleteRes(SDecoder *pCoder, SDeleteRes *pRes) {
   if (tDecodeI64v(pCoder, &pRes->affectedRows) < 0) return -1;
 
   if (tDecodeCStrTo(pCoder, pRes->tableFName) < 0) return -1;
+  if (tDecodeCStrTo(pCoder, pRes->tsColName) < 0) return -1;
   return 0;
 }
 int32_t tEncodeSMqDataRsp(SEncoder *pEncoder, const SMqDataRsp *pRsp) {

source/dnode/mgmt/mgmt_mnode/src/mmHandle.c

@@ -184,6 +184,7 @@ SArray *mmGetMsgHandles() {
   if (dmSetMgmtHandle(pArray, TDMT_MND_ALTER_STB, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_MND_DROP_STB, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_MND_TABLE_META, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;
+  if (dmSetMgmtHandle(pArray, TDMT_MND_BATCH_META, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_MND_TABLE_CFG, mmPutMsgToReadQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_MND_CREATE_SMA, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_MND_DROP_SMA, mmPutMsgToWriteQueue, 0) == NULL) goto _OVER;

source/dnode/mgmt/mgmt_vnode/src/vmHandle.c

@@ -343,6 +343,7 @@ SArray *vmGetMsgHandles() {
   if (dmSetMgmtHandle(pArray, TDMT_VND_UPDATE_TAG_VAL, vmPutMsgToWriteQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_VND_TABLE_META, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_VND_TABLE_CFG, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;
+  if (dmSetMgmtHandle(pArray, TDMT_VND_BATCH_META, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_VND_TABLES_META, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_SCH_CANCEL_TASK, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;
   if (dmSetMgmtHandle(pArray, TDMT_SCH_DROP_TASK, vmPutMsgToFetchQueue, 0) == NULL) goto _OVER;

source/dnode/mgmt/mgmt_vnode/src/vmWorker.c

@@ -89,7 +89,7 @@ static void vmProcessStreamQueue(SQueueInfo *pInfo, SRpcMsg *pMsg) {
   int32_t code = vnodeProcessFetchMsg(pVnode->pImpl, pMsg, pInfo);
   if (code != 0) {
     if (terrno != 0) code = terrno;
-    dGError("vgId:%d, msg:%p failed to stream since %s", pVnode->vgId, pMsg, terrstr());
+    dGError("vgId:%d, msg:%p failed to process stream since %s", pVnode->vgId, pMsg, terrstr());
     vmSendRsp(pMsg, code);
   }
 

source/dnode/mnode/impl/src/mndQuery.c

@@ -63,6 +63,106 @@ int32_t mndProcessQueryMsg(SRpcMsg *pMsg) {
   return code;
 }
 
+int32_t mndProcessBatchMetaMsg(SRpcMsg *pMsg) {
+  int32_t code = 0;
+  int32_t offset = 0;
+  int32_t rspSize = 0;
+  SBatchReq *batchReq = (SBatchReq*)pMsg->pCont;
+  int32_t msgNum = ntohl(batchReq->msgNum);
+  offset += sizeof(SBatchReq);
+  SBatchMsg req = {0};
+  SBatchRsp rsp = {0};
+  SRpcMsg reqMsg = *pMsg;
+  SRpcMsg rspMsg = {0};
+  void* pRsp = NULL;
+  SMnode *pMnode = pMsg->info.node;
+
+  SArray* batchRsp = taosArrayInit(msgNum, sizeof(SBatchRsp));
+  if (NULL == batchRsp) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _exit;
+  }
+  
+  for (int32_t i = 0; i < msgNum; ++i) {
+    req.msgType = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    offset += sizeof(req.msgType);
+
+    req.msgLen = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    offset += sizeof(req.msgLen);
+
+    req.msg = (char*)pMsg->pCont + offset;
+    offset += req.msgLen;
+
+    reqMsg.msgType = req.msgType;
+    reqMsg.pCont = req.msg;
+    reqMsg.contLen = req.msgLen;
+    reqMsg.info.rsp = NULL;
+    reqMsg.info.rspLen = 0;
+
+    MndMsgFp fp = pMnode->msgFp[TMSG_INDEX(req.msgType)];
+    if (fp == NULL) {
+      mError("msg:%p, failed to get msg handle, app:%p type:%s", pMsg, pMsg->info.ahandle, TMSG_INFO(pMsg->msgType));
+      terrno = TSDB_CODE_MSG_NOT_PROCESSED;
+      return -1;
+    }
+
+    if ((*fp)(&reqMsg)) {
+      rsp.rspCode = terrno;
+    } else {
+      rsp.rspCode = 0;
+    }
+    rsp.reqType = reqMsg.msgType;
+    rsp.msgLen = reqMsg.info.rspLen;
+    rsp.msg = reqMsg.info.rsp;
+    
+    taosArrayPush(batchRsp, &rsp);
+
+    rspSize += sizeof(rsp) + rsp.msgLen - POINTER_BYTES;
+  }
+
+  rspSize += sizeof(int32_t);
+  offset = 0;
+  
+  pRsp = rpcMallocCont(rspSize);
+  if (pRsp == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _exit;
+  }
+
+  *(int32_t*)((char*)pRsp + offset) = htonl(msgNum);
+  offset += sizeof(msgNum);
+  for (int32_t i = 0; i < msgNum; ++i) {
+    SBatchRsp *p = taosArrayGet(batchRsp, i);
+    
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->reqType);
+    offset += sizeof(p->reqType);
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->msgLen);
+    offset += sizeof(p->msgLen);
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->rspCode);
+    offset += sizeof(p->rspCode);
+    memcpy((char*)pRsp + offset, p->msg, p->msgLen);
+    offset += p->msgLen;
+
+    rpcFreeCont(p->msg);
+  }
+
+  taosArrayDestroy(batchRsp);
+  batchRsp = NULL;
+
+_exit:
+
+  pMsg->info.rsp = pRsp;
+  pMsg->info.rspLen = rspSize;
+
+  if (code) {
+    mError("mnd get batch meta failed cause of %s", tstrerror(code));
+  }
+
+  taosArrayDestroyEx(batchRsp, tFreeSBatchRsp);
+
+  return code;
+}
+
 int32_t mndInitQuery(SMnode *pMnode) {
   if (qWorkerInit(NODE_TYPE_MNODE, MNODE_HANDLE, NULL, (void **)&pMnode->pQuery, &pMnode->msgCb) != 0) {
     mError("failed to init qworker in mnode since %s", terrstr());
@@ -76,6 +176,7 @@ int32_t mndInitQuery(SMnode *pMnode) {
   mndSetMsgHandle(pMnode, TDMT_SCH_MERGE_FETCH, mndProcessQueryMsg);
   mndSetMsgHandle(pMnode, TDMT_SCH_DROP_TASK, mndProcessQueryMsg);
   mndSetMsgHandle(pMnode, TDMT_SCH_QUERY_HEARTBEAT, mndProcessQueryMsg);
+  mndSetMsgHandle(pMnode, TDMT_MND_BATCH_META, mndProcessBatchMetaMsg);
 
   return 0;
 }

source/dnode/mnode/impl/src/mndScheduler.c

@@ -391,10 +391,12 @@ int32_t mndScheduleStream(SMnode* pMnode, SStreamObj* pStream) {
       // exec
       pInnerTask->execType = TASK_EXEC__PIPE;
 
+#if 0
       SDbObj* pSourceDb = mndAcquireDb(pMnode, pStream->sourceDb);
       ASSERT(pDbObj != NULL);
       sdbRelease(pSdb, pSourceDb);
       pInnerTask->numOfVgroups = pSourceDb->cfg.numOfVgroups;
+#endif
 
       if (tsSchedStreamToSnode) {
         SSnodeObj* pSnode = mndSchedFetchOneSnode(pMnode);

source/dnode/vnode/src/inc/vnd.h

@@ -78,8 +78,9 @@ void    vnodeBufPoolReset(SVBufPool* pPool);
 // vnodeQuery.c
 int32_t vnodeQueryOpen(SVnode* pVnode);
 void    vnodeQueryClose(SVnode* pVnode);
-int32_t vnodeGetTableMeta(SVnode* pVnode, SRpcMsg* pMsg);
-int     vnodeGetTableCfg(SVnode* pVnode, SRpcMsg* pMsg);
+int32_t vnodeGetTableMeta(SVnode* pVnode, SRpcMsg* pMsg, bool direct);
+int     vnodeGetTableCfg(SVnode* pVnode, SRpcMsg* pMsg, bool direct);
+int32_t vnodeGetBatchMeta(SVnode *pVnode, SRpcMsg *pMsg);
 
 // vnodeCommit.c
 int32_t vnodeBegin(SVnode* pVnode);

source/dnode/vnode/src/sma/smaRollup.c

@@ -611,8 +611,8 @@ static int32_t tdRSmaFetchAndSubmitResult(SRSmaInfoItem *pItem, STSchema *pTSche
         goto _err;
       }
 
-      smaDebug("vgId:%d, process submit req for rsma table %" PRIi64 " level %" PRIi8 " version:%"PRIi64, SMA_VID(pSma),
-               suid, pItem->level, output->info.version);
+      smaDebug("vgId:%d, process submit req for rsma table %" PRIi64 " level %" PRIi8 " version:%" PRIi64,
+               SMA_VID(pSma), suid, pItem->level, output->info.version);
 
       taosMemoryFreeClear(pReq);
       taosArrayClear(pResult);
@@ -644,7 +644,7 @@ static int32_t tdExecuteRSmaImpl(SSma *pSma, const void *pMsg, int32_t inputType
   smaDebug("vgId:%d, execute rsma %" PRIi8 " task for qTaskInfo:%p suid:%" PRIu64, SMA_VID(pSma), level,
            pItem->taskInfo, suid);
 
-  if (qSetStreamInput(pItem->taskInfo, pMsg, inputType, true) < 0) {  // INPUT__DATA_SUBMIT
+  if (qSetMultiStreamInput(pItem->taskInfo, pMsg, 1, inputType) < 0) {  // INPUT__DATA_SUBMIT
     smaError("vgId:%d, rsma % " PRIi8 " qSetStreamInput failed since %s", SMA_VID(pSma), level, tstrerror(terrno));
     return TSDB_CODE_FAILED;
   }
@@ -1329,7 +1329,7 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
       tdRefRSmaInfo(pSma, pRSmaInfo);
 
       SSDataBlock dataBlock = {.info.type = STREAM_GET_ALL};
-      qSetStreamInput(pItem->taskInfo, &dataBlock, STREAM_INPUT__DATA_BLOCK, false);
+      qSetMultiStreamInput(pItem->taskInfo, &dataBlock, 1, STREAM_INPUT__DATA_BLOCK);
       tdRSmaFetchAndSubmitResult(pItem, pRSmaInfo->pTSchema, pRSmaInfo->suid, pStat, STREAM_INPUT__DATA_BLOCK);
 
       tdUnRefRSmaInfo(pSma, pRSmaInfo);
@@ -1356,4 +1356,4 @@ static void tdRSmaFetchTrigger(void *param, void *tmrId) {
 
 _end:
   tdReleaseSmaRef(smaMgmt.rsetId, pItem->refId, __func__, __LINE__);
-}
+}
\ No newline at end of file

source/dnode/vnode/src/tq/tq.c

@@ -653,7 +653,7 @@ int32_t tqProcessTaskDeployReq(STQ* pTq, char* msg, int32_t msgLen) {
     } else {
       SReadHandle mgHandle = {
           .vnode = NULL,
-          .numOfVgroups = pTask->numOfVgroups,
+          .numOfVgroups = (int32_t)taosArrayGetSize(pTask->childEpInfo),
       };
       pTask->exec.executor = qCreateStreamExecTaskInfo(pTask->exec.qmsg, &mgHandle);
     }

source/dnode/vnode/src/vnd/vnodeQuery.c

@@ -21,7 +21,7 @@ int vnodeQueryOpen(SVnode *pVnode) {
 
 void vnodeQueryClose(SVnode *pVnode) { qWorkerDestroy((void **)&pVnode->pQuery); }
 
-int vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg) {
+int vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
   STableInfoReq  infoReq = {0};
   STableMetaRsp  metaRsp = {0};
   SMetaReader    mer1 = {0};
@@ -99,7 +99,12 @@ int vnodeGetTableMeta(SVnode *pVnode, SRpcMsg *pMsg) {
     goto _exit;
   }
 
-  pRsp = rpcMallocCont(rspLen);
+  if (direct) {
+    pRsp = rpcMallocCont(rspLen);
+  } else {
+    pRsp = taosMemoryCalloc(1, rspLen);
+  }
+  
   if (pRsp == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;
     goto _exit;
@@ -117,15 +122,19 @@ _exit:
     qError("get table %s meta failed cause of %s", infoReq.tbName, tstrerror(code));
   }
 
-  tmsgSendRsp(&rpcMsg);
-
+  if (direct) {
+    tmsgSendRsp(&rpcMsg);
+  } else {
+    *pMsg = rpcMsg;
+  }
+  
   taosMemoryFree(metaRsp.pSchemas);
   metaReaderClear(&mer2);
   metaReaderClear(&mer1);
   return TSDB_CODE_SUCCESS;
 }
 
-int vnodeGetTableCfg(SVnode *pVnode, SRpcMsg *pMsg) {
+int vnodeGetTableCfg(SVnode *pVnode, SRpcMsg *pMsg, bool direct) {
   STableCfgReq   cfgReq = {0};
   STableCfgRsp   cfgRsp = {0};
   SMetaReader    mer1 = {0};
@@ -209,7 +218,12 @@ int vnodeGetTableCfg(SVnode *pVnode, SRpcMsg *pMsg) {
     goto _exit;
   }
 
-  pRsp = rpcMallocCont(rspLen);
+  if (direct) {
+    pRsp = rpcMallocCont(rspLen);
+  } else {
+    pRsp = taosMemoryCalloc(1, rspLen);
+  }
+
   if (pRsp == NULL) {
     code = TSDB_CODE_OUT_OF_MEMORY;
     goto _exit;
@@ -227,14 +241,124 @@ _exit:
     qError("get table %s cfg failed cause of %s", cfgReq.tbName, tstrerror(code));
   }
 
-  tmsgSendRsp(&rpcMsg);
-
+  if (direct) {
+    tmsgSendRsp(&rpcMsg);
+  } else {
+    *pMsg = rpcMsg;
+  }
+  
   tFreeSTableCfgRsp(&cfgRsp);
   metaReaderClear(&mer2);
   metaReaderClear(&mer1);
   return TSDB_CODE_SUCCESS;
 }
 
+int32_t vnodeGetBatchMeta(SVnode *pVnode, SRpcMsg *pMsg) {
+  int32_t code = 0;
+  int32_t offset = 0;
+  int32_t rspSize = 0;
+  SBatchReq *batchReq = (SBatchReq*)pMsg->pCont;
+  int32_t msgNum = ntohl(batchReq->msgNum);
+  offset += sizeof(SBatchReq);
+  SBatchMsg req = {0};
+  SBatchRsp rsp = {0};
+  SRpcMsg reqMsg = *pMsg;
+  SRpcMsg rspMsg = {0};
+  void* pRsp = NULL;
+
+  SArray* batchRsp = taosArrayInit(msgNum, sizeof(SBatchRsp));
+  if (NULL == batchRsp) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _exit;
+  }
+  
+  for (int32_t i = 0; i < msgNum; ++i) {
+    req.msgType = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    offset += sizeof(req.msgType);
+
+    req.msgLen = ntohl(*(int32_t*)((char*)pMsg->pCont + offset));
+    offset += sizeof(req.msgLen);
+
+    req.msg = (char*)pMsg->pCont + offset;
+    offset += req.msgLen;
+
+    reqMsg.msgType = req.msgType;
+    reqMsg.pCont = req.msg;
+    reqMsg.contLen = req.msgLen;
+    
+    switch (req.msgType) {
+      case TDMT_VND_TABLE_META:
+        vnodeGetTableMeta(pVnode, &reqMsg, false);
+        break;
+      case TDMT_VND_TABLE_CFG:
+        vnodeGetTableCfg(pVnode, &reqMsg, false);
+        break;
+      default:
+        qError("invalid req msgType %d", req.msgType);
+        reqMsg.code = TSDB_CODE_INVALID_MSG;
+        reqMsg.pCont = NULL;
+        reqMsg.contLen = 0;
+        break;
+    }
+
+    rsp.reqType = reqMsg.msgType;
+    rsp.msgLen = reqMsg.contLen;
+    rsp.rspCode = reqMsg.code;
+    rsp.msg = reqMsg.pCont;
+    
+    taosArrayPush(batchRsp, &rsp);
+
+    rspSize += sizeof(rsp) + rsp.msgLen - POINTER_BYTES;
+  }
+
+  rspSize += sizeof(int32_t);
+  offset = 0;
+  
+  pRsp = rpcMallocCont(rspSize);
+  if (pRsp == NULL) {
+    code = TSDB_CODE_OUT_OF_MEMORY;
+    goto _exit;
+  }
+
+  *(int32_t*)((char*)pRsp + offset) = htonl(msgNum);
+  offset += sizeof(msgNum);
+  for (int32_t i = 0; i < msgNum; ++i) {
+    SBatchRsp *p = taosArrayGet(batchRsp, i);
+    
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->reqType);
+    offset += sizeof(p->reqType);
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->msgLen);
+    offset += sizeof(p->msgLen);
+    *(int32_t*)((char*)pRsp + offset) = htonl(p->rspCode);
+    offset += sizeof(p->rspCode);
+    memcpy((char*)pRsp + offset, p->msg, p->msgLen);
+    offset += p->msgLen;
+
+    taosMemoryFreeClear(p->msg);
+  }
+
+  taosArrayDestroy(batchRsp);
+  batchRsp = NULL;
+
+_exit:
+
+  rspMsg.info = pMsg->info;
+  rspMsg.pCont = pRsp;
+  rspMsg.contLen = rspSize;
+  rspMsg.code = code;
+  rspMsg.msgType = pMsg->msgType;
+
+  if (code) {
+    qError("vnd get batch meta failed cause of %s", tstrerror(code));
+  }
+
+  taosArrayDestroyEx(batchRsp, tFreeSBatchRsp);
+
+  tmsgSendRsp(&rspMsg);
+
+  return code;
+}
+
 int32_t vnodeGetLoad(SVnode *pVnode, SVnodeLoad *pLoad) {
   pLoad->vgId = TD_VID(pVnode);
   pLoad->syncState = syncGetMyRole(pVnode->sync);

source/dnode/vnode/src/vnd/vnodeSvr.c

@@ -298,7 +298,7 @@ int32_t vnodeProcessQueryMsg(SVnode *pVnode, SRpcMsg *pMsg) {
 int32_t vnodeProcessFetchMsg(SVnode *pVnode, SRpcMsg *pMsg, SQueueInfo *pInfo) {
   vTrace("message in fetch queue is processing");
   if ((pMsg->msgType == TDMT_SCH_FETCH || pMsg->msgType == TDMT_VND_TABLE_META ||
-       pMsg->msgType == TDMT_VND_TABLE_CFG) &&
+       pMsg->msgType == TDMT_VND_TABLE_CFG || pMsg->msgType == TDMT_VND_BATCH_META) &&
       !vnodeIsLeader(pVnode)) {
     vnodeRedirectRpcMsg(pVnode, pMsg);
     return 0;
@@ -320,9 +320,11 @@ int32_t vnodeProcessFetchMsg(SVnode *pVnode, SRpcMsg *pMsg, SQueueInfo *pInfo) {
     case TDMT_SCH_QUERY_HEARTBEAT:
       return qWorkerProcessHbMsg(pVnode, pVnode->pQuery, pMsg, 0);
     case TDMT_VND_TABLE_META:
-      return vnodeGetTableMeta(pVnode, pMsg);
+      return vnodeGetTableMeta(pVnode, pMsg, true);
     case TDMT_VND_TABLE_CFG:
-      return vnodeGetTableCfg(pVnode, pMsg);
+      return vnodeGetTableCfg(pVnode, pMsg, true);
+    case TDMT_VND_BATCH_META:
+      return vnodeGetBatchMeta(pVnode, pMsg);
     case TDMT_VND_CONSUME:
       return tqProcessPollReq(pVnode->pTq, pMsg);
     case TDMT_STREAM_TASK_RUN:

source/libs/catalog/inc/catalogInt.h

@@ -31,6 +31,7 @@ extern "C" {
 #define CTG_DEFAULT_RENT_SECOND 10
 #define CTG_DEFAULT_RENT_SLOT_SIZE 10
 #define CTG_DEFAULT_MAX_RETRY_TIMES 3
+#define CTG_DEFAULT_BATCH_NUM 64
 
 #define CTG_RENT_SLOT_SECOND 1.5
 
@@ -38,6 +39,8 @@ extern "C" {
 
 #define CTG_ERR_CODE_TABLE_NOT_EXIST TSDB_CODE_PAR_TABLE_NOT_EXIST
 
+#define CTG_BATCH_FETCH 1
+
 enum {
   CTG_READ = 1,
   CTG_WRITE,
@@ -200,8 +203,20 @@ typedef struct SCatalog {
   SCtgRentMgmt     stbRent;
 } SCatalog;
 
+typedef struct SCtgBatch {
+  int32_t batchId;
+  int32_t msgType;
+  int32_t msgSize;
+  SArray* pMsgs;
+  SRequestConnInfo conn;
+  char    dbFName[TSDB_DB_FNAME_LEN];
+  SArray* pTaskIds;
+} SCtgBatch;
+
 typedef struct SCtgJob {
   int64_t           refId;
+  int32_t           batchId;
+  SHashObj*         pBatchs;
   SArray*           pTasks;
   int32_t           taskDone;
   SMetaData         jobRes;
@@ -236,6 +251,16 @@ typedef struct SCtgMsgCtx {
   char* target;
 } SCtgMsgCtx;
 
+
+typedef struct SCtgTaskCallbackParam {
+  uint64_t                queryId;
+  int64_t                 refId;
+  SArray*                 taskId;
+  int32_t                 reqType;
+  int32_t                 batchId;
+} SCtgTaskCallbackParam;
+
+
 typedef struct SCtgTask SCtgTask;
 typedef int32_t (*ctgSubTaskCbFp)(SCtgTask*);
 
@@ -258,6 +283,7 @@ typedef struct SCtgTask {
   SRWLatch        lock;
   SArray*         pParents;
   SCtgSubRes      subRes;
+  SHashObj*       pBatchs;
 } SCtgTask;
 
 typedef int32_t (*ctgInitTaskFp)(SCtgJob*, int32_t, void*);
@@ -618,6 +644,7 @@ int32_t ctgGetTbMetaFromVnode(SCatalog* pCtg, SRequestConnInfo *pConn, const SNa
 int32_t ctgGetTableCfgFromVnode(SCatalog* pCtg, SRequestConnInfo *pConn, const SName* pTableName, SVgroupInfo *vgroupInfo, STableCfg **out, SCtgTask* pTask);
 int32_t ctgGetTableCfgFromMnode(SCatalog* pCtg, SRequestConnInfo *pConn, const SName* pTableName, STableCfg **out, SCtgTask* pTask);
 int32_t ctgGetSvrVerFromMnode(SCatalog* pCtg, SRequestConnInfo *pConn, char **out, SCtgTask* pTask);
+int32_t ctgLaunchBatchs(SCatalog* pCtg, SCtgJob *pJob, SHashObj* pBatchs);
 
 int32_t ctgInitJob(SCatalog* pCtg, SRequestConnInfo *pConn, SCtgJob** job, const SCatalogReq* pReq, catalogCallback fp, void* param);
 int32_t ctgLaunchJob(SCtgJob *pJob);
@@ -626,6 +653,9 @@ int32_t ctgLaunchSubTask(SCtgTask *pTask, CTG_TASK_TYPE type, ctgSubTaskCbFp fp,
 int32_t ctgGetTbCfgCb(SCtgTask *pTask);
 void    ctgFreeHandle(SCatalog* pCatalog);
 
+void    ctgFreeMsgSendParam(void* param);
+void    ctgFreeBatch(SCtgBatch *pBatch);
+void    ctgFreeBatchs(SHashObj *pBatchs);
 int32_t ctgCloneVgInfo(SDBVgInfo *src, SDBVgInfo **dst);
 int32_t ctgCloneMetaOutput(STableMetaOutput *output, STableMetaOutput **pOutput);
 int32_t ctgGenerateVgList(SCatalog *pCtg, SHashObj *vgHash, SArray** pList);
@@ -642,7 +672,7 @@ int32_t ctgDbVgVersionSearchCompare(const void* key1, const void* key2);
 void    ctgFreeSTableMetaOutput(STableMetaOutput* pOutput);
 int32_t ctgUpdateMsgCtx(SCtgMsgCtx* pCtx, int32_t reqType, void* out, char* target);
 char *  ctgTaskTypeStr(CTG_TASK_TYPE type);
-int32_t ctgUpdateSendTargetInfo(SMsgSendInfo *pMsgSendInfo, int32_t msgType, SCtgTask* pTask);
+int32_t ctgUpdateSendTargetInfo(SMsgSendInfo *pMsgSendInfo, int32_t msgType, char* dbFName, int32_t vgId);
 int32_t ctgCloneTableIndex(SArray* pIndex, SArray** pRes);
 void    ctgFreeSTableIndex(void *info);
 void    ctgClearSubTaskRes(SCtgSubRes *pRes);

source/libs/catalog/inc/ctgRemote.h

@@ -20,12 +20,6 @@
 extern "C" {
 #endif
 
-typedef struct SCtgTaskCallbackParam {
-  uint64_t                queryId;
-  int64_t                 refId;
-  uint64_t                taskId;
-  int32_t                 reqType;
-} SCtgTaskCallbackParam;
 
 
 #ifdef __cplusplus

source/libs/catalog/src/ctgAsync.c

@@ -473,8 +473,15 @@ int32_t ctgInitJob(SCatalog* pCtg, SRequestConnInfo *pConn, SCtgJob** job, const
   pJob->tbCfgNum = tbCfgNum;
   pJob->svrVerNum = svrVerNum;
 
-  pJob->pTasks = taosArrayInit(taskNum, sizeof(SCtgTask));
+#if CTG_BATCH_FETCH
+  pJob->pBatchs = taosHashInit(CTG_DEFAULT_BATCH_NUM, taosGetDefaultHashFunction(TSDB_DATA_TYPE_INT), false, HASH_NO_LOCK);
+  if (NULL == pJob->pBatchs) {
+    ctgError("taosHashInit %d batch failed", CTG_DEFAULT_BATCH_NUM);
+    CTG_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY);
+  }
+#endif
 
+  pJob->pTasks = taosArrayInit(taskNum, sizeof(SCtgTask));
   if (NULL == pJob->pTasks) {
     ctgError("taosArrayInit %d tasks failed", taskNum);
     CTG_ERR_JRET(TSDB_CODE_OUT_OF_MEMORY);
@@ -560,7 +567,7 @@ int32_t ctgInitJob(SCatalog* pCtg, SRequestConnInfo *pConn, SCtgJob** job, const
 
 _return:
 
-  taosMemoryFreeClear(*job);
+  ctgFreeJob(*job);
   CTG_RET(code);
 }
 
@@ -776,7 +783,8 @@ int32_t ctgCallSubCb(SCtgTask *pTask) {
         pParent->subRes.code = code;
       }
     }
-    
+
+    pParent->pBatchs = pTask->pBatchs;    
     CTG_ERR_JRET(pParent->subRes.fp(pParent));
   }
   
@@ -872,7 +880,7 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTask* pTask, int32_t reqType, const SDataBuf *
             SVgroupInfo vgInfo = {0};
             CTG_ERR_JRET(ctgGetVgInfoFromHashValue(pCtg, dbCache->vgCache.vgInfo, ctx->pName, &vgInfo));
           
-            ctgDebug("will refresh tbmeta, not supposed to be stb, tbName:%s, flag:%d", tNameGetTableName(ctx->pName), ctx->flag);
+            ctgDebug("will refresh tbmeta, supposed to be stb, tbName:%s, flag:%d", tNameGetTableName(ctx->pName), ctx->flag);
 
             ctx->vgId = vgInfo.vgId;          
             CTG_ERR_JRET(ctgGetTbMetaFromVnode(pCtg, pConn, ctx->pName, &vgInfo, NULL, pTask));
@@ -890,7 +898,7 @@ int32_t ctgHandleGetTbMetaRsp(SCtgTask* pTask, int32_t reqType, const SDataBuf *
           return TSDB_CODE_SUCCESS;
         }
         
-        ctgError("no tbmeta got, tbNmae:%s", tNameGetTableName(ctx->pName));
+        ctgError("no tbmeta got, tbName:%s", tNameGetTableName(ctx->pName));
         ctgRemoveTbMetaFromCache(pCtg, ctx->pName, false);
         
         CTG_ERR_JRET(CTG_ERR_CODE_TABLE_NOT_EXIST);
@@ -1653,6 +1661,7 @@ int32_t ctgSetSubTaskCb(SCtgTask *pSub, SCtgTask *pTask) {
   if (CTG_TASK_DONE == pSub->status) {
     pTask->subRes.code = pSub->code;
     CTG_ERR_JRET((*gCtgAsyncFps[pTask->type].cloneFp)(pSub, &pTask->subRes.res));
+    pTask->pBatchs = pSub->pBatchs;
     CTG_ERR_JRET(pTask->subRes.fp(pTask));
   } else {
     if (NULL == pSub->pParents) {
@@ -1690,6 +1699,7 @@ int32_t ctgLaunchSubTask(SCtgTask *pTask, CTG_TASK_TYPE type, ctgSubTaskCbFp fp,
   CTG_ERR_RET(ctgSetSubTaskCb(pSub, pTask));
 
   if (newTask) {
+    pSub->pBatchs = pTask->pBatchs;
     CTG_ERR_RET((*gCtgAsyncFps[pSub->type].launchFp)(pSub));
     pSub->status = CTG_TASK_LAUNCHED;
   }
@@ -1702,9 +1712,11 @@ int32_t ctgLaunchJob(SCtgJob *pJob) {
   
   for (int32_t i = 0; i < taskNum; ++i) {
     SCtgTask *pTask = taosArrayGet(pJob->pTasks, i);
+    pTask->pBatchs = pJob->pBatchs;
 
     qDebug("QID:0x%" PRIx64 " ctg launch [%dth] task", pJob->queryId, pTask->taskId);
     CTG_ERR_RET((*gCtgAsyncFps[pTask->type].launchFp)(pTask));
+    
     pTask->status = CTG_TASK_LAUNCHED;
   }
 
@@ -1712,6 +1724,10 @@ int32_t ctgLaunchJob(SCtgJob *pJob) {
     qDebug("QID:0x%" PRIx64 " ctg call user callback with rsp %s", pJob->queryId, tstrerror(pJob->jobResCode));
     
     taosAsyncExec(ctgCallUserCb, pJob, NULL);
+#if CTG_BATCH_FETCH    
+  } else {
+    ctgLaunchBatchs(pJob->pCtg, pJob, pJob->pBatchs);
+#endif
   }
 
   return TSDB_CODE_SUCCESS;

source/libs/catalog/src/ctgUtil.c

@@ -19,6 +19,39 @@
 #include "catalogInt.h"
 #include "systable.h"
 
+void ctgFreeMsgSendParam(void* param) {
+  if (NULL == param) {
+    return;
+  }
+
+  SCtgTaskCallbackParam* pParam = (SCtgTaskCallbackParam*)param;
+  taosArrayDestroy(pParam->taskId);
+
+  taosMemoryFree(param);
+}
+
+void ctgFreeBatch(SCtgBatch *pBatch) {
+  if (NULL == pBatch) {
+    return;
+  }
+  
+  taosArrayDestroy(pBatch->pMsgs);
+  taosArrayDestroy(pBatch->pTaskIds);
+}
+
+void ctgFreeBatchs(SHashObj *pBatchs) {
+  void* p = taosHashIterate(pBatchs, NULL);
+  while (NULL != p) {
+    SCtgBatch* pBatch = (SCtgBatch*)p;
+
+    ctgFreeBatch(pBatch);
+
+    p = taosHashIterate(pBatchs, p);
+  }
+
+  taosHashCleanup(pBatchs);
+}
+
 char *ctgTaskTypeStr(CTG_TASK_TYPE type) {
   switch (type) {
     case CTG_TASK_GET_QNODE:
@@ -612,6 +645,7 @@ void ctgFreeJob(void* job) {
   uint64_t qid = pJob->queryId;
 
   ctgFreeTasks(pJob->pTasks);
+  ctgFreeBatchs(pJob->pBatchs);
 
   ctgFreeSMetaData(&pJob->jobRes);
 
@@ -867,14 +901,10 @@ int32_t ctgCloneTableIndex(SArray* pIndex, SArray** pRes) {
 }
 
 
-int32_t ctgUpdateSendTargetInfo(SMsgSendInfo *pMsgSendInfo, int32_t msgType, SCtgTask* pTask) {
-  if (msgType == TDMT_VND_TABLE_META) {
-    SCtgTbMetaCtx* ctx = (SCtgTbMetaCtx*)pTask->taskCtx;
-    char dbFName[TSDB_DB_FNAME_LEN];
-    tNameGetFullDbName(ctx->pName, dbFName);
-    
+int32_t ctgUpdateSendTargetInfo(SMsgSendInfo *pMsgSendInfo, int32_t msgType, char* dbFName, int32_t vgId) {
+  if (msgType == TDMT_VND_TABLE_META || msgType == TDMT_VND_TABLE_CFG || msgType == TDMT_VND_BATCH_META) {
     pMsgSendInfo->target.type = TARGET_TYPE_VNODE;
-    pMsgSendInfo->target.vgId = ctx->vgId;
+    pMsgSendInfo->target.vgId = vgId;
     pMsgSendInfo->target.dbFName = strdup(dbFName);
   } else {
     pMsgSendInfo->target.type = TARGET_TYPE_MNODE;

source/libs/executor/src/dataDeleter.c

@@ -90,7 +90,8 @@ static void toDataCacheEntry(SDataDeleterHandle* pHandle, const SInputData* pInp
   pRes->uidList = pHandle->pParam->pUidList;
   pRes->skey = pHandle->pDeleter->deleteTimeRange.skey;
   pRes->ekey = pHandle->pDeleter->deleteTimeRange.ekey;
-  strcpy(pRes->tableFName, pHandle->pDeleter->tableFName);
+  strcpy(pRes->tableName, pHandle->pDeleter->tableFName);
+  strcpy(pRes->tsColName, pHandle->pDeleter->tsColName);
   pRes->affectedRows = *(int64_t*)pColRes->pData;
 
   pBuf->useSize += pEntry->dataLen;

source/libs/executor/src/executor.c

@@ -30,8 +30,7 @@ static void cleanupRefPool() {
   taosCloseRef(ref);
 }
 
-static int32_t doSetStreamBlock(SOperatorInfo* pOperator, void* input, size_t numOfBlocks, int32_t type, bool assignUid,
-                                char* id) {
+static int32_t doSetStreamBlock(SOperatorInfo* pOperator, void* input, size_t numOfBlocks, int32_t type, char* id) {
   ASSERT(pOperator != NULL);
   if (pOperator->operatorType != QUERY_NODE_PHYSICAL_PLAN_STREAM_SCAN) {
     if (pOperator->numOfDownstream == 0) {
@@ -44,12 +43,12 @@ static int32_t doSetStreamBlock(SOperatorInfo* pOperator, void* input, size_t nu
       return TSDB_CODE_QRY_APP_ERROR;
     }
     pOperator->status = OP_NOT_OPENED;
-    return doSetStreamBlock(pOperator->pDownstream[0], input, numOfBlocks, type, assignUid, id);
+    return doSetStreamBlock(pOperator->pDownstream[0], input, numOfBlocks, type, id);
   } else {
     pOperator->status = OP_NOT_OPENED;
 
     SStreamScanInfo* pInfo = pOperator->info;
-    pInfo->assignBlockUid = assignUid;
+    /*pInfo->assignBlockUid = assignUid;*/
 
     // TODO: if a block was set but not consumed,
     // prevent setting a different type of block
@@ -95,11 +94,7 @@ static int32_t doSetStreamBlock(SOperatorInfo* pOperator, void* input, size_t nu
   }
 }
 
-int32_t qSetStreamInput(qTaskInfo_t tinfo, const void* input, int32_t type, bool assignUid) {
-  return qSetMultiStreamInput(tinfo, input, 1, type, assignUid);
-}
-
-int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numOfBlocks, int32_t type, bool assignUid) {
+int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numOfBlocks, int32_t type) {
   if (tinfo == NULL) {
     return TSDB_CODE_QRY_APP_ERROR;
   }
@@ -110,8 +105,7 @@ int32_t qSetMultiStreamInput(qTaskInfo_t tinfo, const void* pBlocks, size_t numO
 
   SExecTaskInfo* pTaskInfo = (SExecTaskInfo*)tinfo;
 
-  int32_t code =
-      doSetStreamBlock(pTaskInfo->pRoot, (void**)pBlocks, numOfBlocks, type, assignUid, GET_TASKID(pTaskInfo));
+  int32_t code = doSetStreamBlock(pTaskInfo->pRoot, (void**)pBlocks, numOfBlocks, type, GET_TASKID(pTaskInfo));
   if (code != TSDB_CODE_SUCCESS) {
     qError("%s failed to set the stream block data", GET_TASKID(pTaskInfo));
   } else {
@@ -337,7 +331,7 @@ int32_t qCreateExecTask(SReadHandle* readHandle, int32_t vgId, uint64_t taskId,
     }
 
     code = dsCreateDataSinker(pSubplan->pDataSink, handle, pSinkParam);
-    if(code != TSDB_CODE_SUCCESS){
+    if (code != TSDB_CODE_SUCCESS) {
       taosMemoryFreeClear(pSinkParam);
     }
   }

source/libs/nodes/src/nodesCloneFuncs.c

@@ -401,7 +401,8 @@ static int32_t logicVnodeModifCopy(const SVnodeModifyLogicNode* pSrc, SVnodeModi
   COPY_SCALAR_FIELD(tableId);
   COPY_SCALAR_FIELD(stableId);
   COPY_SCALAR_FIELD(tableType);
-  COPY_CHAR_ARRAY_FIELD(tableFName);
+  COPY_CHAR_ARRAY_FIELD(tableName);
+  COPY_CHAR_ARRAY_FIELD(tsColName);
   COPY_OBJECT_FIELD(deleteTimeRange, sizeof(STimeWindow));
   CLONE_OBJECT_FIELD(pVgroupList, vgroupsInfoClone);
   CLONE_NODE_LIST_FIELD(pInsertCols);

source/libs/nodes/src/nodesCodeFuncs.c

@@ -2332,7 +2332,7 @@ static int32_t physiQueryInsertNodeToJson(const void* pObj, SJson* pJson) {
     code = tjsonAddIntegerToObject(pJson, jkQueryInsertPhysiPlanTableType, pNode->tableType);
   }
   if (TSDB_CODE_SUCCESS == code) {
-    code = tjsonAddStringToObject(pJson, jkQueryInsertPhysiPlanTableFName, pNode->tableFName);
+    code = tjsonAddStringToObject(pJson, jkQueryInsertPhysiPlanTableFName, pNode->tableName);
   }
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonAddIntegerToObject(pJson, jkQueryInsertPhysiPlanVgId, pNode->vgId);
@@ -2361,7 +2361,7 @@ static int32_t jsonToPhysiQueryInsertNode(const SJson* pJson, void* pObj) {
     code = tjsonGetTinyIntValue(pJson, jkQueryInsertPhysiPlanTableType, &pNode->tableType);
   }
   if (TSDB_CODE_SUCCESS == code) {
-    code = tjsonGetStringValue(pJson, jkQueryInsertPhysiPlanTableFName, pNode->tableFName);
+    code = tjsonGetStringValue(pJson, jkQueryInsertPhysiPlanTableFName, pNode->tableName);
   }
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonGetIntValue(pJson, jkQueryInsertPhysiPlanVgId, &pNode->vgId);
@@ -2376,6 +2376,7 @@ static int32_t jsonToPhysiQueryInsertNode(const SJson* pJson, void* pObj) {
 static const char* jkDeletePhysiPlanTableId = "TableId";
 static const char* jkDeletePhysiPlanTableType = "TableType";
 static const char* jkDeletePhysiPlanTableFName = "TableFName";
+static const char* jkDeletePhysiPlanTsColName = "TsColName";
 static const char* jkDeletePhysiPlanDeleteTimeRangeStartKey = "DeleteTimeRangeStartKey";
 static const char* jkDeletePhysiPlanDeleteTimeRangeEndKey = "DeleteTimeRangeEndKey";
 static const char* jkDeletePhysiPlanAffectedRows = "AffectedRows";
@@ -2393,6 +2394,9 @@ static int32_t physiDeleteNodeToJson(const void* pObj, SJson* pJson) {
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonAddStringToObject(pJson, jkDeletePhysiPlanTableFName, pNode->tableFName);
   }
+  if (TSDB_CODE_SUCCESS == code) {
+    code = tjsonAddStringToObject(pJson, jkDeletePhysiPlanTsColName, pNode->tsColName);
+  }
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonAddIntegerToObject(pJson, jkDeletePhysiPlanDeleteTimeRangeStartKey, pNode->deleteTimeRange.skey);
   }
@@ -2419,6 +2423,9 @@ static int32_t jsonToPhysiDeleteNode(const SJson* pJson, void* pObj) {
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonGetStringValue(pJson, jkDeletePhysiPlanTableFName, pNode->tableFName);
   }
+  if (TSDB_CODE_SUCCESS == code) {
+    code = tjsonGetStringValue(pJson, jkDeletePhysiPlanTsColName, pNode->tsColName);
+  }
   if (TSDB_CODE_SUCCESS == code) {
     code = tjsonGetBigIntValue(pJson, jkDeletePhysiPlanDeleteTimeRangeStartKey, &pNode->deleteTimeRange.skey);
   }

source/libs/planner/src/planLogicCreater.c

@@ -1292,8 +1292,8 @@ static int32_t createVnodeModifLogicNodeByDelete(SLogicPlanContext* pCxt, SDelet
   pModify->modifyType = MODIFY_TABLE_TYPE_DELETE;
   pModify->tableId = pRealTable->pMeta->uid;
   pModify->tableType = pRealTable->pMeta->tableType;
-  snprintf(pModify->tableFName, sizeof(pModify->tableFName), "%d.%s.%s", pCxt->pPlanCxt->acctId,
-           pRealTable->table.dbName, pRealTable->table.tableName);
+  snprintf(pModify->tableName, sizeof(pModify->tableName), "%s", pRealTable->table.tableName);
+  strcpy(pModify->tsColName, pRealTable->pMeta->schema->name);
   pModify->deleteTimeRange = pDelete->timeRange;
   pModify->pAffectedRows = nodesCloneNode(pDelete->pCountFunc);
   if (NULL == pModify->pAffectedRows) {
@@ -1342,8 +1342,7 @@ static int32_t createVnodeModifLogicNodeByInsert(SLogicPlanContext* pCxt, SInser
   pModify->tableId = pRealTable->pMeta->uid;
   pModify->stableId = pRealTable->pMeta->suid;
   pModify->tableType = pRealTable->pMeta->tableType;
-  snprintf(pModify->tableFName, sizeof(pModify->tableFName), "%d.%s.%s", pCxt->pPlanCxt->acctId,
-           pRealTable->table.dbName, pRealTable->table.tableName);
+  snprintf(pModify->tableName, sizeof(pModify->tableName), "%s", pRealTable->table.tableName);
   TSWAP(pModify->pVgroupList, pRealTable->pVgroupList);
   pModify->pInsertCols = nodesCloneList(pInsert->pCols);
   if (NULL == pModify->pInsertCols) {

source/libs/planner/src/planPhysiCreater.c

@@ -1588,7 +1588,7 @@ static int32_t createQueryInserter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNod
   pInserter->tableId = pModify->tableId;
   pInserter->stableId = pModify->stableId;
   pInserter->tableType = pModify->tableType;
-  strcpy(pInserter->tableFName, pModify->tableFName);
+  strcpy(pInserter->tableName, pModify->tableName);
   pInserter->vgId = pModify->pVgroupList->vgroups[0].vgId;
   pInserter->epSet = pModify->pVgroupList->vgroups[0].epSet;
   vgroupInfoToNodeAddr(pModify->pVgroupList->vgroups, &pSubplan->execNode);
@@ -1638,7 +1638,8 @@ static int32_t createDataDeleter(SPhysiPlanContext* pCxt, SVnodeModifyLogicNode*
 
   pDeleter->tableId = pModify->tableId;
   pDeleter->tableType = pModify->tableType;
-  strcpy(pDeleter->tableFName, pModify->tableFName);
+  strcpy(pDeleter->tableFName, pModify->tableName);
+  strcpy(pDeleter->tsColName, pModify->tsColName);
   pDeleter->deleteTimeRange = pModify->deleteTimeRange;
 
   int32_t code = setNodeSlotId(pCxt, pRoot->pOutputDataBlockDesc->dataBlockId, -1, pModify->pAffectedRows,

source/libs/qworker/src/qworker.c

@@ -283,7 +283,8 @@ int32_t qwGetDeleteResFromSink(QW_FPARAMS_DEF, SQWTaskCtx *ctx, SDeleteRes *pRes
   pRes->skey = pDelRes->skey;
   pRes->ekey = pDelRes->ekey;
   pRes->affectedRows = pDelRes->affectedRows;
-  strcpy(pRes->tableFName, pDelRes->tableFName);
+  strcpy(pRes->tableFName, pDelRes->tableName);
+  strcpy(pRes->tsColName, pDelRes->tsColName);
   taosMemoryFree(output.pData);
   
   return TSDB_CODE_SUCCESS;

source/libs/stream/src/streamDispatch.c

@@ -227,6 +227,8 @@ int32_t streamDispatchOneReq(SStreamTask* pTask, const SStreamDispatchReq* pReq,
   msg.pCont = buf;
   msg.msgType = pTask->dispatchMsgType;
 
+  qDebug("dispatch from task %d to task %d node %d", pTask->taskId, pReq->taskId, vgId);
+
   tmsgSendReq(pEpSet, &msg);
 
   code = 0;
@@ -281,8 +283,10 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
     return code;
 
   } else if (pTask->dispatchType == TASK_DISPATCH__SHUFFLE) {
-    SArray* vgInfo = pTask->shuffleDispatcher.dbInfo.pVgroupInfos;
-    ASSERT(pTask->shuffleDispatcher.waitingRspCnt == 0);
+    int32_t rspCnt = atomic_load_32(&pTask->shuffleDispatcher.waitingRspCnt);
+    ASSERT(rspCnt == 0);
+
+    SArray*             vgInfo = pTask->shuffleDispatcher.dbInfo.pVgroupInfos;
     int32_t             vgSz = taosArrayGetSize(vgInfo);
     SStreamDispatchReq* pReqs = taosMemoryCalloc(vgSz, sizeof(SStreamDispatchReq));
     if (pReqs == NULL) {
@@ -301,7 +305,10 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
       if (pReqs[i].data == NULL || pReqs[i].dataLen == NULL) {
         goto FAIL_SHUFFLE_DISPATCH;
       }
+      SVgroupInfo* pVgInfo = taosArrayGet(vgInfo, i);
+      pReqs[i].taskId = pVgInfo->taskId;
     }
+
     for (int32_t i = 0; i < blockNum; i++) {
       SSDataBlock* pDataBlock = taosArrayGet(pData->blocks, i);
       char*        ctbName = buildCtbNameByGroupId(pTask->shuffleDispatcher.stbFullName, pDataBlock->info.groupId);
@@ -309,6 +316,9 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
       // TODO: get hash function by hashMethod
       uint32_t hashValue = MurmurHash3_32(ctbName, strlen(ctbName));
 
+      taosMemoryFree(ctbName);
+
+      bool found = false;
       // TODO: optimize search
       int32_t j;
       for (j = 0; j < vgSz; j++) {
@@ -318,12 +328,17 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
           if (streamAddBlockToDispatchMsg(pDataBlock, &pReqs[j]) < 0) {
             goto FAIL_SHUFFLE_DISPATCH;
           }
-          pReqs[j].taskId = pVgInfo->taskId;
+          if (pReqs[j].blockNum == 0) {
+            atomic_add_fetch_32(&pTask->shuffleDispatcher.waitingRspCnt, 1);
+          }
           pReqs[j].blockNum++;
+          found = true;
           break;
         }
       }
+      ASSERT(found);
     }
+
     for (int32_t i = 0; i < vgSz; i++) {
       if (pReqs[i].blockNum > 0) {
         // send
@@ -331,7 +346,6 @@ int32_t streamDispatchAllBlocks(SStreamTask* pTask, const SStreamDataBlock* pDat
         if (streamDispatchOneReq(pTask, &pReqs[i], pVgInfo->vgId, &pVgInfo->epSet) < 0) {
           goto FAIL_SHUFFLE_DISPATCH;
         }
-        pTask->shuffleDispatcher.waitingRspCnt++;
       }
     }
     code = 0;

source/libs/stream/src/streamExec.c

@@ -22,22 +22,22 @@ static int32_t streamTaskExecImpl(SStreamTask* pTask, void* data, SArray* pRes)
   SStreamQueueItem* pItem = (SStreamQueueItem*)data;
   if (pItem->type == STREAM_INPUT__GET_RES) {
     SStreamTrigger* pTrigger = (SStreamTrigger*)data;
-    qSetMultiStreamInput(exec, pTrigger->pBlock, 1, STREAM_INPUT__DATA_BLOCK, false);
+    qSetMultiStreamInput(exec, pTrigger->pBlock, 1, STREAM_INPUT__DATA_BLOCK);
   } else if (pItem->type == STREAM_INPUT__DATA_SUBMIT) {
     ASSERT(pTask->isDataScan);
     SStreamDataSubmit* pSubmit = (SStreamDataSubmit*)data;
     qDebug("task %d %p set submit input %p %p %d 1", pTask->taskId, pTask, pSubmit, pSubmit->data, *pSubmit->dataRef);
-    qSetStreamInput(exec, pSubmit->data, STREAM_INPUT__DATA_SUBMIT, false);
+    qSetMultiStreamInput(exec, pSubmit->data, 1, STREAM_INPUT__DATA_SUBMIT);
   } else if (pItem->type == STREAM_INPUT__DATA_BLOCK || pItem->type == STREAM_INPUT__DATA_RETRIEVE) {
     SStreamDataBlock* pBlock = (SStreamDataBlock*)data;
     SArray*           blocks = pBlock->blocks;
     qDebug("task %d %p set ssdata input", pTask->taskId, pTask);
-    qSetMultiStreamInput(exec, blocks->pData, blocks->size, STREAM_INPUT__DATA_BLOCK, false);
+    qSetMultiStreamInput(exec, blocks->pData, blocks->size, STREAM_INPUT__DATA_BLOCK);
   } else if (pItem->type == STREAM_INPUT__MERGED_SUBMIT) {
     SStreamMergedSubmit* pMerged = (SStreamMergedSubmit*)data;
     SArray*              blocks = pMerged->reqs;
     qDebug("task %d %p set submit input (merged), batch num: %d", pTask->taskId, pTask, (int32_t)blocks->size);
-    qSetMultiStreamInput(exec, blocks->pData, blocks->size, STREAM_INPUT__MERGED_SUBMIT, false);
+    qSetMultiStreamInput(exec, blocks->pData, blocks->size, STREAM_INPUT__MERGED_SUBMIT);
   } else {
     ASSERT(0);
   }

source/libs/stream/src/streamTask.c

@@ -64,7 +64,7 @@ int32_t tEncodeSStreamTask(SEncoder* pEncoder, const SStreamTask* pTask) {
   if (tEncodeI32(pEncoder, pTask->selfChildId) < 0) return -1;
   if (tEncodeI32(pEncoder, pTask->nodeId) < 0) return -1;
   if (tEncodeSEpSet(pEncoder, &pTask->epSet) < 0) return -1;
-  if (tEncodeI32(pEncoder, pTask->numOfVgroups) < 0) return -1;
+  /*if (tEncodeI32(pEncoder, pTask->numOfVgroups) < 0) return -1;*/
 
   int32_t epSz = taosArrayGetSize(pTask->childEpInfo);
   if (tEncodeI32(pEncoder, epSz) < 0) return -1;
@@ -119,7 +119,7 @@ int32_t tDecodeSStreamTask(SDecoder* pDecoder, SStreamTask* pTask) {
   if (tDecodeI32(pDecoder, &pTask->selfChildId) < 0) return -1;
   if (tDecodeI32(pDecoder, &pTask->nodeId) < 0) return -1;
   if (tDecodeSEpSet(pDecoder, &pTask->epSet) < 0) return -1;
-  if (tDecodeI32(pDecoder, &pTask->numOfVgroups) < 0) return -1;
+  /*if (tDecodeI32(pDecoder, &pTask->numOfVgroups) < 0) return -1;*/
 
   int32_t epSz;
   if (tDecodeI32(pDecoder, &epSz) < 0) return -1;

source/libs/wal/inc/walInt.h

@@ -61,26 +61,31 @@ static inline int32_t compareWalFileInfo(const void* pLeft, const void* pRight)
 }
 
 static inline int64_t walGetLastFileSize(SWal* pWal) {
+  if (taosArrayGetSize(pWal->fileInfoSet) == 0) return 0;
   SWalFileInfo* pInfo = (SWalFileInfo*)taosArrayGetLast(pWal->fileInfoSet);
   return pInfo->fileSize;
 }
 
 static inline int64_t walGetLastFileFirstVer(SWal* pWal) {
+  if (taosArrayGetSize(pWal->fileInfoSet) == 0) return -1;
   SWalFileInfo* pInfo = (SWalFileInfo*)taosArrayGetLast(pWal->fileInfoSet);
   return pInfo->firstVer;
 }
 
 static inline int64_t walGetCurFileFirstVer(SWal* pWal) {
+  if (pWal->writeCur == -1) return -1;
   SWalFileInfo* pInfo = (SWalFileInfo*)taosArrayGet(pWal->fileInfoSet, pWal->writeCur);
   return pInfo->firstVer;
 }
 
 static inline int64_t walGetCurFileLastVer(SWal* pWal) {
+  if (pWal->writeCur == -1) return -1;
   SWalFileInfo* pInfo = (SWalFileInfo*)taosArrayGet(pWal->fileInfoSet, pWal->writeCur);
   return pInfo->firstVer;
 }
 
 static inline int64_t walGetCurFileOffset(SWal* pWal) {
+  if (pWal->writeCur == -1) return -1;
   SWalFileInfo* pInfo = (SWalFileInfo*)taosArrayGet(pWal->fileInfoSet, pWal->writeCur);
   return pInfo->fileSize;
 }
@@ -88,6 +93,7 @@ static inline int64_t walGetCurFileOffset(SWal* pWal) {
 static inline bool walCurFileClosed(SWal* pWal) { return taosArrayGetSize(pWal->fileInfoSet) != pWal->writeCur; }
 
 static inline SWalFileInfo* walGetCurFileInfo(SWal* pWal) {
+  if (pWal->writeCur == -1) return NULL;
   return (SWalFileInfo*)taosArrayGet(pWal->fileInfoSet, pWal->writeCur);
 }
 

tests/pytest/util/common.py

@@ -18,7 +18,7 @@ import time
 import socket
 import json
 import toml
-from .boundary import DataBoundary
+from util.boundary import DataBoundary
 import taos
 from util.log import *
 from util.sql import *
@@ -80,23 +80,18 @@ class DataSet:
             self.bool_data.append( bool((i + bool_start) % 2 ))
             self.vchar_data.append( f"{vchar_prefix}_{i * vchar_step}" )
             self.nchar_data.append( f"{nchar_prefix}_{i * nchar_step}")
-            self.ts_data.append( int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000 - i * ts_step))
-
-    def get_disorder_set(self,
-        rows,
-        int_low     :int    = INT_MIN,
-        int_up      :int    = INT_MAX,
-        bint_low    :int    = BIGINT_MIN,
-        bint_up     :int    = BIGINT_MAX,
-        sint_low    :int    = SMALLINT_MIN,
-        sint_up     :int    = SMALLINT_MAX,
-        tint_low    :int    = TINYINT_MIN,
-        tint_up     :int    = TINYINT_MAX,
-        ubint_low   :int    = BIGINT_UN_MIN,
-        ubint_up    :int    = BIGINT_UN_MAX,
-
-
-    ):
+            self.ts_data.append( int(datetime.timestamp(datetime.now()) * 1000 - i * ts_step))
+
+    def get_disorder_set(self, rows, **kwargs):
+        for k, v in kwargs.items():
+            int_low = v if k == "int_low" else INT_MIN
+            int_up = v if k == "int_up" else INT_MAX
+            bint_low = v if k == "bint_low" else BIGINT_MIN
+            bint_up = v if k == "bint_up" else BIGINT_MAX
+            sint_low = v if k == "sint_low" else SMALLINT_MIN
+            sint_up = v if k == "sint_up" else SMALLINT_MAX
+            tint_low = v if k == "tint_low" else TINYINT_MIN
+            tint_up = v if k == "tint_up" else TINYINT_MAX
         pass
 
 

tests/pytest/util/sql.py

@@ -49,18 +49,23 @@ class TDSql:
     def close(self):
         self.cursor.close()
 
-    def prepare(self):
-        tdLog.info("prepare database:db")
+    def prepare(self, dbname="db", drop=True, **kwargs):
+        tdLog.info(f"prepare database:{dbname}")
         s = 'reset query cache'
         try:
             self.cursor.execute(s)
         except:
             tdLog.notice("'reset query cache' is not supported")
-        s = 'drop database if exists db'
-        self.cursor.execute(s)
-        s = 'create database db duration 300'
+        if drop:
+            s = f'drop database if exists {dbname}'
+            self.cursor.execute(s)
+        s = f'create database {dbname}'
+        for k, v in kwargs.items():
+            s += f" {k} {v}"
+        if "duration" not in kwargs:
+            s += " duration 300"
         self.cursor.execute(s)
-        s = 'use db'
+        s = f'use {dbname}'
         self.cursor.execute(s)
         time.sleep(2)
 
@@ -106,7 +111,7 @@ class TDSql:
                 if row_tag:
                     return self.queryResult
                 return self.queryRows
-            except Exception as e:    
+            except Exception as e:
                 caller = inspect.getframeinfo(inspect.stack()[1][0])
                 args = (caller.filename, caller.lineno, sql, repr(e))
                 tdLog.notice("%s(%d) failed: sql:%s, %s" % args)
@@ -304,7 +309,7 @@ class TDSql:
                 tdLog.notice("Try to execute sql again, query times: %d "%i)
                 time.sleep(1)
                 pass
-        else:               
+        else:
             try:
                 tdLog.notice("Try the last execute sql ")
                 self.affectedRows = self.cursor.execute(sql)

tests/script/jenkins/basic.txt

 
 #======================b1-start===============
 
-# ---- user
+# ---- user ----
 ./test.sh -f tsim/user/basic.sim
 ./test.sh -f tsim/user/password.sim
 ./test.sh -f tsim/user/privilege_db.sim
@@ -58,13 +58,13 @@
 # unsupport ./test.sh -f tsim/dnode/vnode_clean.sim
 ./test.sh -f tsim/dnode/use_dropped_dnode.sim 
 
-# ---- import
+# ---- import ----
 ./test.sh -f tsim/import/basic.sim
 ./test.sh -f tsim/import/commit.sim
 ./test.sh -f tsim/import/large.sim
 ./test.sh -f tsim/import/replica1.sim
 
-# ---- insert
+# ---- insert ----
 ./test.sh -f tsim/insert/backquote.sim
 ./test.sh -f tsim/insert/basic.sim
 ./test.sh -f tsim/insert/basic0.sim
@@ -164,7 +164,7 @@
 # TD-17704 ./test.sh -f tsim/parser/union_sysinfo.sim
 # TD-17661 ./test.sh -f tsim/parser/where.sim
 
-# ---- query
+# ---- query ----
 ./test.sh -f tsim/query/charScalarFunction.sim
 # ./test.sh -f tsim/query/explain.sim
 ./test.sh -f tsim/query/interval-offset.sim
@@ -187,10 +187,10 @@
 ./test.sh -f tsim/mnode/basic1.sim
 ./test.sh -f tsim/mnode/basic2.sim
 ./test.sh -f tsim/mnode/basic3.sim
-./test.sh -f tsim/mnode/basic4.sim
+# TD-17919 ./test.sh -f tsim/mnode/basic4.sim
 ./test.sh -f tsim/mnode/basic5.sim
 
-# ---- show
+# ---- show ----
 ./test.sh -f tsim/show/basic.sim
 
 # ---- table
@@ -260,7 +260,7 @@
 ./test.sh -f tsim/tmq/snapshot.sim
 ./test.sh -f tsim/tmq/snapshot1.sim
 
-# --- stable
+# --- stable ----
 ./test.sh -f tsim/stable/alter_comment.sim
 ./test.sh -f tsim/stable/alter_count.sim
 ./test.sh -f tsim/stable/alter_import.sim
@@ -274,7 +274,6 @@
 ./test.sh -f tsim/stable/dnode3.sim
 ./test.sh -f tsim/stable/metrics.sim
 ./test.sh -f tsim/stable/refcount.sim
-./test.sh -f tsim/stable/show.sim
 ./test.sh -f tsim/stable/tag_add.sim
 ./test.sh -f tsim/stable/tag_drop.sim
 ./test.sh -f tsim/stable/tag_filter.sim
@@ -298,8 +297,9 @@
 # --- sma
 ./test.sh -f tsim/sma/drop_sma.sim
 ./test.sh -f tsim/sma/tsmaCreateInsertQuery.sim
-./test.sh -f tsim/sma/rsmaCreateInsertQuery.sim
-./test.sh -f tsim/sma/rsmaPersistenceRecovery.sim
+# temp disable
+#./test.sh -f tsim/sma/rsmaCreateInsertQuery.sim
+#./test.sh -f tsim/sma/rsmaPersistenceRecovery.sim
 
 # --- valgrind
 ./test.sh -f tsim/valgrind/checkError1.sim
@@ -308,6 +308,7 @@
 ./test.sh -f tsim/valgrind/checkError4.sim
 ./test.sh -f tsim/valgrind/checkError5.sim
 ./test.sh -f tsim/valgrind/checkError6.sim
+./test.sh -f tsim/valgrind/checkError7.sim
 
 # --- vnode
 # unsupport ./test.sh -f tsim/vnode/replica3_basic.sim
@@ -328,35 +329,35 @@
 ./test.sh -f tsim/sync/oneReplica1VgElect.sim
 ./test.sh -f tsim/sync/oneReplica5VgElect.sim
 
-# --- catalog
+# --- catalog ----
 ./test.sh -f tsim/catalog/alterInCurrent.sim
 
 # --- scalar
 ./test.sh -f tsim/scalar/in.sim
 ./test.sh -f tsim/scalar/scalar.sim
 
-# ---- alter
+# ---- alter ----
 ./test.sh -f tsim/alter/cached_schema_after_alter.sim
 ./test.sh -f tsim/alter/dnode.sim
 ./test.sh -f tsim/alter/table.sim
 
-# ---- cache
+# ---- cache ----
 ./test.sh -f tsim/cache/new_metrics.sim
 ./test.sh -f tsim/cache/restart_table.sim
 ./test.sh -f tsim/cache/restart_metrics.sim
 
-# ---- column
+# ---- column ----
 ./test.sh -f tsim/column/commit.sim
 ./test.sh -f tsim/column/metrics.sim
 ./test.sh -f tsim/column/table.sim
 
-# ---- compress
+# ---- compress ----
 ./test.sh -f tsim/compress/commitlog.sim
 ./test.sh -f tsim/compress/compress2.sim
 ./test.sh -f tsim/compress/compress.sim
 ./test.sh -f tsim/compress/uncompress.sim
 
-# ---- compute
+# ---- compute ----
 ./test.sh -f tsim/compute/avg.sim
 ./test.sh -f tsim/compute/block_dist.sim
 ./test.sh -f tsim/compute/bottom.sim

tests/script/tsim/alter/cached_schema_after_alter.sim

@@ -50,11 +50,8 @@ endi
 
 print ================== restart server to commit data into disk
 system sh/exec.sh -n dnode1 -s stop -x SIGINT
-sleep 3000
 system sh/exec.sh -n dnode1 -s start
 print ================== server restart completed
-sleep 3000
-sql connect
 
 sql use $db
 sql select * from $stb

tests/script/tsim/mnode/basic4.sim

@@ -45,8 +45,11 @@ endi
 if $data(2)[4] != ready then
   goto step2
 endi
+if $data(3)[4] != ready then
+  goto step2
+endi
 
-system sh/exec.sh -n dnode3 -s stop
+system sh/exec.sh -n dnode3 -s stop -x SIGKILL
 sql_error create mnode on dnode 3
 
 print =============== step3: show mnodes

tests/script/tsim/parser/limit1_stb.sim

@@ -51,40 +51,10 @@ endi
 ##TBASE-352
 $offset = $tbNum * $rowNum
 $offset = $offset - 1
-sql select * from $stb limit 2 offset $offset
+sql select * from $stb order by ts limit 2 offset $offset
 if $rows != 1 then
   return -1
 endi
-if $data00 != @18-11-25 19:30:00.000@ then
-  return -1
-endi
-if $data01 != 9 then
-  return -1
-endi
-if $data02 != NULL then
-  return -1
-endi
-if $data03 != 9.00000 then
-  return -1
-endi
-if $data04 != NULL then
-  return -1
-endi
-if $data05 != 9 then
-  return -1
-endi
-if $data06 != 9 then
-  return -1
-endi
-if $data07 != 1 then
-  return -1
-endi
-if $data08 != binary9 then
-  return -1
-endi
-if $data09 != nchar9 then
-  return -1
-endi
 
 $offset = $tbNum * $rowNum
 $offset = $offset / 2
@@ -207,21 +177,6 @@ sql select * from $stb where ts > '2018-09-17 09:30:00.000' and ts < '2018-09-17
 if $rows != 5 then
   return -1
 endi
-if $data01 != 5 then
-  return -1
-endi
-if $data11 != 6 then
-  return -1
-endi
-if $data21 != 7 then
-  return -1
-endi
-if $data31 != 8 then
-  return -1
-endi
-if $data41 != 4 then
-  return -1
-endi
 
 $offset = $totalNum / 2
 sql select * from $stb where ts >= $ts0 and ts <= $tsu limit 5 offset $offset

tests/script/tsim/query/udf.sim

 system sh/stop_dnodes.sh
-
 system sh/deploy.sh -n dnode1 -i 1
-system sh/cfg.sh -n dnode1 -c wallevel -v 2
-system sh/cfg.sh -n dnode1 -c numOfMnodes -v 1
 system sh/cfg.sh -n dnode1 -c udf -v 1
 
 print ========= start dnode1 as leader
 system sh/exec.sh -n dnode1 -s start
-sleep 1000
 sql connect
 
 print ======== step1 udf 

tests/script/tsim/stable/show.sim

-system sh/stop_dnodes.sh
-system sh/deploy.sh -n dnode1 -i 1
-system sh/exec.sh -n dnode1 -s start
-sql connect
-
-print ======================== create stable
-sql create database d1
-sql use d1
-
-$x = 0
-while $x < 128
-  $tb = d1.s . $x
-  sql create table $tb (ts timestamp, i int) tags (j int)
-  $x = $x + 1
-endw
-
-print ======================== describe stables
-# TODO : create stable error
-$m = 0
-while $m < 128
-  $tb = s . $m
-  $filter = ' . $tb
-  $filter = $filter . '
-  sql show stables like $filter
-  print sql : show stables like $filter
-  if $rows != 1 then
-    print expect 1, actual: $rows
-    return -1
-  endi
-  $m = $m + 1
-endw
-
-
-print ======================== show stables
-
-sql show d1.stables
-
-print num of stables is $rows
-if $rows != 128 then
-  return -1
-endi
-
-print ======================== create table
-
-$x = 0
-while $x < 424
-  $tb = d1.t . $x
-  sql create table $tb using d1.s0 tags( $x )
-  $x = $x + 1
-endw
-
-print ======================== show stables
-
-sql show d1.tables
-
-print num of tables is $rows
-if $rows != 424 then
-  return -1
-endi
-
-system sh/exec.sh -n dnode1 -s stop -x SIGINT

tests/script/tsim/valgrind/checkError6.sim

@@ -63,13 +63,15 @@ sql show table distributed tb1
 sql select count(1) from tb1
 sql select count(tbcol) as b from tb1 where ts <= 1601481840000 interval(1m)
 sql select diff(tbcol) from tb1 where ts <= 1601481840000 
-sql select diff(tbcol) from tb1 where tbcol > 5 and tbcol < 20
+sql select diff(tbcol) from tb1 where tbcol > 5 and tbcol < 20 order by ts
 sql select first(tbcol), last(tbcol) as b from tb1 where ts <= 1601481840000 interval(1m)
-sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), sum(tbcol), stddev(tbcol) from tb1 where ts <= 1601481840000 partition by tgcol interval(1m)
+sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), sum(tbcol), stddev(tbcol) from tb1 where ts <= 1601481840000 partition by tgcol order by tgcol
+sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), sum(tbcol), stddev(tbcol) from tb1 where ts <= 1601481840000 partition by tgcol interval(1m) 
 sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), count(tbcol) from tb1 where ts <= 1601481840000 and ts >= 1601481800000 partition by tgcol interval(1m) fill(value, 0)
 sql select last_row(*) from tb1 where tbcol > 5 and tbcol < 20
 sql select _wstart, _wend, _wduration, _qstart, _qend,  count(*) from tb1 interval(10s, 2s) sliding(10s)
 sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), count(tbcol) from tb1 where ts <= 1601481840000 and ts >= 1601481800000 partition by tgcol interval(1m) fill(value, 0)
+sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), count(tbcol) from tb1 where ts <= 1601481840000 and ts >= 1601481800000 partition by tgcol interval(1m) fill(value, 0) order by tgcol desc
 sql select log(tbcol), abs(tbcol), pow(tbcol, 2), sqrt(tbcol), sin(tbcol), cos(tbcol), tan(tbcol), asin(tbcol), acos(tbcol), atan(tbcol), ceil(tbcol), floor(tbcol), round(tbcol), atan(tbcol) from tb1
 sql select length("abcd1234"), char_length("abcd1234=-+*") from tb1
 sql select tbcol4, length(tbcol4), lower(tbcol4), upper(tbcol4), ltrim(tbcol4), rtrim(tbcol4), concat(tbcol4, tbcol5), concat_ws('_', tbcol4, tbcol5), substr(tbcol4, 1, 4)  from tb1
@@ -94,6 +96,7 @@ sql select first(tbcol), last(tbcol) as b from stb where ts <= 1601481840000 and
 sql select first(tbcol), last(tbcol) as b from stb where ts <= 1601481840000 partition by tgcol interval(1m)
 sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), sum(tbcol), stddev(tbcol) from stb where ts <= 1601481840000 partition by tgcol interval(1m)
 sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), count(tbcol) from stb where ts <= 1601481840000 and ts >= 1601481800000 partition by tgcol interval(1m) fill(value, 0)
+sql select count(tbcol), avg(tbcol), max(tbcol), min(tbcol), count(tbcol) from stb where ts <= 1601481840000 and ts >= 1601481800000 and tgcol = 1 partition by tgcol interval(1m) fill(value, 0) order by tgcol desc
 sql select last_row(tbcol), stddev(tbcol) from stb where tbcol > 5 and tbcol < 20 group by tgcol
 sql select _wstart, _wend, _wduration, _qstart, _qend,  count(*) from stb interval(10s, 2s) sliding(10s)
 sql select log(tbcol), abs(tbcol), pow(tbcol, 2), sqrt(tbcol), sin(tbcol), cos(tbcol), tan(tbcol), asin(tbcol), acos(tbcol), atan(tbcol), ceil(tbcol), floor(tbcol), round(tbcol), atan(tbcol) from stb

tests/system-test/1-insert/mutil_stage.py

+from datetime import datetime
+import time
+
+from typing import List, Any, Tuple
+from util.log import *
+from util.sql import *
+from util.cases import *
+from util.dnodes import *
+from util.common import *
+
+PRIMARY_COL = "ts"
+
+INT_COL = "c_int"
+BINT_COL = "c_bint"
+SINT_COL = "c_sint"
+TINT_COL = "c_tint"
+FLOAT_COL = "c_float"
+DOUBLE_COL = "c_double"
+BOOL_COL = "c_bool"
+TINT_UN_COL = "c_utint"
+SINT_UN_COL = "c_usint"
+BINT_UN_COL = "c_ubint"
+INT_UN_COL = "c_uint"
+BINARY_COL = "c_binary"
+NCHAR_COL = "c_nchar"
+TS_COL = "c_ts"
+
+NUM_COL = [INT_COL, BINT_COL, SINT_COL, TINT_COL, FLOAT_COL, DOUBLE_COL, ]
+CHAR_COL = [BINARY_COL, NCHAR_COL, ]
+BOOLEAN_COL = [BOOL_COL, ]
+TS_TYPE_COL = [TS_COL, ]
+
+INT_TAG = "t_int"
+
+TAG_COL = [INT_TAG]
+# insert data args：
+TIME_STEP = 10000
+NOW = int(datetime.timestamp(datetime.now()) * 1000)
+
+# init db/table
+DBNAME  = "db"
+STBNAME = "stb1"
+CTB_PRE = "ct"
+NTB_PRE = "nt"
+
+L0 = 0
+L1 = 1
+L2 = 2
+
+PRIMARY_DIR = 1
+NON_PRIMARY_DIR = 0
+
+DATA_PRE0 = f"data0"
+DATA_PRE1 = f"data1"
+DATA_PRE2 = f"data2"
+
+class TDTestCase:
+
+    def init(self, conn, logSql):
+        tdLog.debug(f"start to excute {__file__}")
+        tdSql.init(conn.cursor(), False)
+        self.taos_cfg_path = tdDnodes.dnodes[0].cfgPath
+        self.taos_data_dir = tdDnodes.dnodes[0].dataDir
+
+
+    def cfg(self, filename, **update_dict):
+        cmd = "echo "
+        for k, v in update_dict.items():
+            cmd += f"{k} {v}\n"
+
+        cmd += f" >> {filename}"
+        if os.system(cmd) != 0:
+            tdLog.exit(cmd)
+
+    def cfg_str(self, filename, update_str):
+        cmd = f'echo "{update_str}" >> {filename}'
+        if os.system(cmd) != 0:
+            tdLog.exit(cmd)
+
+    def cfg_str_list(self, filename, update_list):
+        for update_str in update_list:
+            self.cfg_str(filename, update_str)
+
+    def del_old_datadir(self, filename):
+        cmd = f"sed -i '/^dataDir/d' {filename}"
+        if os.system(cmd) != 0:
+            tdLog.exit(cmd)
+
+    @property
+    def __err_cfg(self):
+        cfg_list = []
+        err_case1 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE1}1 {L1} {PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L2} {NON_PRIMARY_DIR}"
+        ]
+        err_case2 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE1}1 {L1} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L2} {PRIMARY_DIR}"
+        ]
+        err_case3 = [
+            f"dataDir {self.taos_data_dir}/data33 3 {NON_PRIMARY_DIR}"
+        ]
+        err_case4 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE1}1 {L1} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L2} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L1} {NON_PRIMARY_DIR}"
+        ]
+        err_case5 = [f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {PRIMARY_DIR}"]
+        for i in range(16):
+            err_case5.append(f"dataDir {self.taos_data_dir}/{DATA_PRE0}{i+1} {L0} {NON_PRIMARY_DIR}")
+
+        err_case6 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}1 {L0} {PRIMARY_DIR}",
+        ]
+        err_case7 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L2} {PRIMARY_DIR}",
+        ]
+        err_case8 = [
+            f"dataDir {self.taos_data_dir}/data33 3 {PRIMARY_DIR}"
+        ]
+        err_case9 = [
+            f"dataDir {self.taos_data_dir}/data33 -1 {NON_PRIMARY_DIR}"
+        ]
+
+        cfg_list.append(err_case1)
+        cfg_list.append(err_case2)
+        cfg_list.append(err_case3)
+        cfg_list.append(err_case4)
+        cfg_list.append(err_case5)
+        cfg_list.append(err_case6)
+        cfg_list.append(err_case7)
+        cfg_list.append(err_case8)
+        cfg_list.append(err_case9)
+
+        return cfg_list
+
+    @property
+    def __current_cfg(self):
+        cfg_list = []
+        current_case1 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}1 {L0} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE1}1 {L1} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}2 {L2} {NON_PRIMARY_DIR}"
+        ]
+
+        current_case2 = [f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 {L0} {PRIMARY_DIR}"]
+        for i in range(9):
+            current_case2.append(f"dataDir {self.taos_data_dir}/{DATA_PRE0}{i+1} {L0} {NON_PRIMARY_DIR}")
+
+        # TD-17773bug
+        current_case3 = [
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}0 ",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE0}1 {L0} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE1}0 {L1} {NON_PRIMARY_DIR}",
+            f"dataDir {self.taos_data_dir}/{DATA_PRE2}0 {L2} {NON_PRIMARY_DIR}",
+        ]
+        cfg_list.append(current_case1)
+        cfg_list.append(current_case3)
+
+        # case2 must in last of least, because use this cfg as data uniformity test
+        cfg_list.append(current_case2)
+
+        return cfg_list
+
+    def cfg_check(self):
+        for cfg_case in self.__err_cfg:
+            self.del_old_datadir(filename=self.taos_cfg_path)
+            tdDnodes.stop(1)
+            tdDnodes.deploy(1)
+            self.cfg_str_list(filename=self.taos_cfg_path, update_list=cfg_case)
+            tdDnodes.starttaosd(1)
+            time.sleep(2)
+            tdSql.error(f"show databases")
+
+        for cfg_case in self.__current_cfg:
+            self.del_old_datadir(filename=self.taos_cfg_path)
+            tdDnodes.stop(1)
+            tdDnodes.deploy(1)
+            self.cfg_str_list(filename=self.taos_cfg_path, update_list=cfg_case)
+            tdDnodes.start(1)
+            tdSql.query(f"show databases")
+
+    def __create_tb(self, stb=STBNAME, ctb_pre = CTB_PRE, ctb_num=20, ntb_pre=NTB_PRE, ntbnum=1, dbname=DBNAME):
+        tdLog.printNoPrefix("==========step: create table")
+        create_stb_sql = f'''create table {dbname}.{stb}(
+                ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
+                {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
+                {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp,
+                {TINT_UN_COL} tinyint unsigned, {SINT_UN_COL} smallint unsigned,
+                {INT_UN_COL} int unsigned, {BINT_UN_COL} bigint unsigned
+            ) tags ({INT_TAG} int)
+            '''
+        tdSql.execute(create_stb_sql)
+
+        for i in range(ntbnum):
+            create_ntb_sql = f'''create table {dbname}.{ntb_pre}{i+1}(
+                    ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
+                    {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
+                    {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp,
+                    {TINT_UN_COL} tinyint unsigned, {SINT_UN_COL} smallint unsigned,
+                    {INT_UN_COL} int unsigned, {BINT_UN_COL} bigint unsigned
+                )
+                '''
+            tdSql.execute(create_ntb_sql)
+
+        for i in range(ctb_num):
+            tdSql.execute(f'create table {dbname}.{ctb_pre}{i+1} using {dbname}.{stb} tags ( {i+1} )')
+
+    def __insert_data(self, rows, dbname=DBNAME, ctb_num=20):
+        data = DataSet()
+        data.get_order_set(rows)
+
+        tdLog.printNoPrefix("==========step: start inser data into tables now.....")
+        for i in range(self.rows):
+            row_data = f'''
+                {data.int_data[i]}, {data.bint_data[i]}, {data.sint_data[i]}, {data.tint_data[i]}, {data.float_data[i]}, {data.double_data[i]},
+                {data.bool_data[i]}, '{data.vchar_data[i]}', '{data.nchar_data[i]}', {data.ts_data[i]}, {data.utint_data[i]},
+                {data.usint_data[i]}, {data.uint_data[i]}, {data.ubint_data[i]}
+            '''
+            neg_row_data = f'''
+                {-1 * data.int_data[i]}, {-1 * data.bint_data[i]}, {-1 * data.sint_data[i]}, {-1 * data.tint_data[i]}, {-1 * data.float_data[i]}, {-1 * data.double_data[i]},
+                {data.bool_data[i]}, '{data.vchar_data[i]}', '{data.nchar_data[i]}', {data.ts_data[i]}, {1 * data.utint_data[i]},
+                {1 * data.usint_data[i]}, {1 * data.uint_data[i]}, {1 * data.ubint_data[i]}
+            '''
+
+            for j in range(ctb_num):
+                tdSql.execute(
+                    f"insert into {dbname}.{CTB_PRE}{j + 1} values ( {NOW - i * TIME_STEP}, {row_data} )")
+
+            # tdSql.execute(
+            #     f"insert into {dbname}.{CTB_PRE}2 values ( {NOW - i * int(TIME_STEP * 0.6)}, {neg_row_data} )")
+            # tdSql.execute(
+            #     f"insert into {dbname}.{CTB_PRE}4 values ( {NOW - i * int(TIME_STEP * 0.8) }, {row_data} )")
+            tdSql.execute(
+                f"insert into {dbname}.{NTB_PRE}1 values ( {NOW - i * int(TIME_STEP * 1.2)}, {row_data} )")
+
+    def run(self):
+        self.rows = 10
+        self.cfg_check()
+        tdSql.prepare(dbname=DBNAME, **{"keep": "1d, 1500m, 26h", "duration":"1h", "vgroups": 10})
+        self.__create_tb(dbname=DBNAME)
+        self.__insert_data(rows=self.rows, dbname=DBNAME)
+        tdSql.query(f"select count(*) from {DBNAME}.{NTB_PRE}1")
+        tdSql.execute(f"flush database {DBNAME}")
+
+    def stop(self):
+        tdSql.close()
+        tdLog.success(f"{__file__} successfully executed")
+
+tdCases.addLinux(__file__, TDTestCase())
+tdCases.addWindows(__file__, TDTestCase())

tests/system-test/1-insert/time_range_wise.py

 import datetime
+import time
 
 from dataclasses import dataclass
 from typing import List, Any, Tuple
@@ -328,11 +329,15 @@ class TDTestCase:
         tdSql.query("select database()")
         dbname =  tdSql.getData(0,0)
         tdSql.query("show databases")
+        for index , value in enumerate(tdSql.cursor.description):
+            if value[0] == "retention":
+                r_index = index
+                break
         for row in tdSql.queryResult:
             if row[0] == dbname:
-                if row[-1] is None:
+                if row[r_index] is None:
                     continue
-                if ":" in row[-1]:
+                if ":" in row[r_index]:
                     sma.rollup_db = True
         if sma.rollup_db :
             return False
@@ -393,8 +398,6 @@ class TDTestCase:
         else:
             tdSql.error(self.__create_sma_index(sma))
 
-
-
     def __drop_sma_index(self, sma:SMAschema):
         sql = f"{sma.drop} {sma.drop_flag} {sma.index_name}"
         return sql
@@ -416,8 +419,7 @@ class TDTestCase:
             self.sma_created_index = list(filter(lambda x: x != sma.index_name, self.sma_created_index))
             tdSql.query("show streams")
             tdSql.checkRows(self.sma_count)
-
-
+            time.sleep(1)
         else:
             tdSql.error(self.__drop_sma_index(sma))
 

tests/system-test/2-query/concat.py

@@ -136,23 +136,23 @@ class TDTestCase:
 
         return sqls
 
-    def __test_current(self):  # sourcery skip: use-itertools-product
+    def __test_current(self, dbname="db"):  # sourcery skip: use-itertools-product
         tdLog.printNoPrefix("==========current sql condition check , must return query ok==========")
         tbname = [
-            "ct1",
-            "ct2",
-            "ct4",
+            f"{dbname}.ct1",
+            f"{dbname}.ct2",
+            f"{dbname}.ct4",
         ]
         for tb in tbname:
             for i in range(2,8):
                 self.__concat_check(tb,i)
                 tdLog.printNoPrefix(f"==========current sql condition check in {tb}, col num: {i} over==========")
 
-    def __test_error(self):
+    def __test_error(self, dbname="db"):
         tdLog.printNoPrefix("==========err sql condition check , must return error==========")
         tbname = [
-            "t1",
-            "stb1",
+            f"{dbname}.t1",
+            f"{dbname}.stb1",
         ]
 
         for tb in tbname:
@@ -163,22 +163,20 @@ class TDTestCase:
             tdLog.printNoPrefix(f"==========err sql condition check in {tb} over==========")
 
 
-    def all_test(self):
-        self.__test_current()
-        self.__test_error()
+    def all_test(self, dbname="db"):
+        self.__test_current(dbname)
+        self.__test_error(dbname)
 
-
-    def __create_tb(self):
-        tdSql.prepare()
+    def __create_tb(self, dbname="db"):
 
         tdLog.printNoPrefix("==========step1:create table")
-        create_stb_sql  =  f'''create table stb1(
+        create_stb_sql  =  f'''create table {dbname}.stb1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
             ) tags (t1 int)
             '''
-        create_ntb_sql = f'''create table t1(
+        create_ntb_sql = f'''create table {dbname}.t1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
@@ -188,29 +186,29 @@ class TDTestCase:
         tdSql.execute(create_ntb_sql)
 
         for i in range(4):
-            tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )')
+            tdSql.execute(f'create table {dbname}.ct{i+1} using {dbname}.stb1 tags ( {i+1} )')
 
-    def __insert_data(self, rows):
+    def __insert_data(self, rows, dbname="db"):
         now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000)
         for i in range(rows):
             tdSql.execute(
-                f"insert into ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
         tdSql.execute(
-            f'''insert into ct1 values
+            f'''insert into {dbname}.ct1 values
             ( { now_time - rows * 5 }, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar_测试_0', { now_time + 8 } )
             ( { now_time + 10000 }, { rows }, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar_测试_9', { now_time + 9 } )
             '''
         )
 
         tdSql.execute(
-            f'''insert into ct4 values
+            f'''insert into {dbname}.ct4 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time +  7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -226,7 +224,7 @@ class TDTestCase:
         )
 
         tdSql.execute(
-            f'''insert into ct2 values
+            f'''insert into {dbname}.ct2 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time + 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -242,13 +240,13 @@ class TDTestCase:
         )
 
         for i in range(rows):
-            insert_data = f'''insert into t1 values
+            insert_data = f'''insert into {dbname}.t1 values
                 ( { now_time - i * 3600000 }, {i}, {i * 11111}, { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2},
                 "binary_{i}", "nchar_测试_{i}", { now_time - 1000 * i } )
                 '''
             tdSql.execute(insert_data)
         tdSql.execute(
-            f'''insert into t1 values
+            f'''insert into {dbname}.t1 values
             ( { now_time + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - (( rows // 2 ) * 60 + 30) * 60000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3600000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -268,22 +266,23 @@ class TDTestCase:
         tdSql.prepare()
 
         tdLog.printNoPrefix("==========step1:create table")
-        self.__create_tb()
+        self.__create_tb(dbname="db")
 
         tdLog.printNoPrefix("==========step2:insert data")
         self.rows = 10
-        self.__insert_data(self.rows)
+        self.__insert_data(self.rows, dbname="db")
 
         tdLog.printNoPrefix("==========step3:all check")
-        self.all_test()
+        self.all_test(dbname="db")
 
-        tdDnodes.stop(1)
-        tdDnodes.start(1)
+        # tdDnodes.stop(1)
+        # tdDnodes.start(1)
+        tdSql.execute("flush database db")
 
         tdSql.execute("use db")
 
         tdLog.printNoPrefix("==========step4:after wal, all check again ")
-        self.all_test()
+        self.all_test(dbname="db")
 
     def stop(self):
         tdSql.close()

tests/system-test/2-query/concat2.py

@@ -137,22 +137,22 @@ class TDTestCase:
 
         return sqls
 
-    def __test_current(self):  # sourcery skip: use-itertools-product
+    def __test_current(self, dbname="db"):
         tdLog.printNoPrefix("==========current sql condition check , must return query ok==========")
         tbname = [
-            "t1",
-            "stb1",
+            f"{dbname}.t1",
+            f"{dbname}.stb1",
         ]
         for tb in tbname:
             for i in range(2,8):
                 self.__concat_check(tb,i)
                 tdLog.printNoPrefix(f"==========current sql condition check in {tb}, col num: {i} over==========")
 
-    def __test_error(self):
+    def __test_error(self, dbname="db"):
         tdLog.printNoPrefix("==========err sql condition check , must return error==========")
         tbname = [
-            "ct1",
-            "ct4",
+            f"{dbname}.ct1",
+            f"{dbname}.ct4",
         ]
 
         for tb in tbname:
@@ -163,22 +163,20 @@ class TDTestCase:
             tdLog.printNoPrefix(f"==========err sql condition check in {tb} over==========")
 
 
-    def all_test(self):
-        self.__test_current()
-        self.__test_error()
+    def all_test(self, dbname="db"):
+        self.__test_current(dbname)
+        self.__test_error(dbname)
 
-
-    def __create_tb(self):
-        tdSql.prepare()
+    def __create_tb(self, dbname="db"):
 
         tdLog.printNoPrefix("==========step1:create table")
-        create_stb_sql  =  f'''create table stb1(
+        create_stb_sql  =  f'''create table {dbname}.stb1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
             ) tags (t1 int)
             '''
-        create_ntb_sql = f'''create table t1(
+        create_ntb_sql = f'''create table {dbname}.t1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
@@ -188,29 +186,29 @@ class TDTestCase:
         tdSql.execute(create_ntb_sql)
 
         for i in range(4):
-            tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )')
+            tdSql.execute(f'create table {dbname}.ct{i+1} using {dbname}.stb1 tags ( {i+1} )')
 
-    def __insert_data(self, rows):
+    def __insert_data(self, rows, dbname="db"):
         now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000)
         for i in range(rows):
             tdSql.execute(
-                f"insert into ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
         tdSql.execute(
-            f'''insert into ct1 values
+            f'''insert into {dbname}.ct1 values
             ( { now_time - rows * 5 }, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar_测试_0', { now_time + 8 } )
             ( { now_time + 10000 }, { rows }, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar_测试_9', { now_time + 9 } )
             '''
         )
 
         tdSql.execute(
-            f'''insert into ct4 values
+            f'''insert into {dbname}.ct4 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time +  7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -226,7 +224,7 @@ class TDTestCase:
         )
 
         tdSql.execute(
-            f'''insert into ct2 values
+            f'''insert into {dbname}.ct2 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time + 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -242,13 +240,13 @@ class TDTestCase:
         )
 
         for i in range(rows):
-            insert_data = f'''insert into t1 values
+            insert_data = f'''insert into {dbname}.t1 values
                 ( { now_time - i * 3600000 }, {i}, {i * 11111}, { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2},
                 "binary_{i}", "nchar_测试_{i}", { now_time - 1000 * i } )
                 '''
             tdSql.execute(insert_data)
         tdSql.execute(
-            f'''insert into t1 values
+            f'''insert into {dbname}.t1 values
             ( { now_time + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - (( rows // 2 ) * 60 + 30) * 60000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3600000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -268,23 +266,23 @@ class TDTestCase:
         tdSql.prepare()
 
         tdLog.printNoPrefix("==========step1:create table")
-        self.__create_tb()
+        self.__create_tb(dbname="db")
 
         tdLog.printNoPrefix("==========step2:insert data")
         self.rows = 10
-        self.__insert_data(self.rows)
+        self.__insert_data(self.rows, dbname="db")
 
         tdLog.printNoPrefix("==========step3:all check")
-        self.all_test()
+        self.all_test(dbname="db")
 
-        tdDnodes.stop(1)
-        tdDnodes.start(1)
+        # tdDnodes.stop(1)
+        # tdDnodes.start(1)
+        tdSql.execute("flush database db")
 
         tdSql.execute("use db")
 
         tdLog.printNoPrefix("==========step4:after wal, all check again ")
-        self.all_test()
-
+        self.all_test(dbname="db")
     def stop(self):
         tdSql.close()
         tdLog.success(f"{__file__} successfully executed")

tests/system-test/2-query/concat_ws.py

@@ -137,23 +137,23 @@ class TDTestCase:
 
         return sqls
 
-    def __test_current(self):  # sourcery skip: use-itertools-product
+    def __test_current(self,dbname="db"):  # sourcery skip: use-itertools-product
         tdLog.printNoPrefix("==========current sql condition check , must return query ok==========")
         tbname = [
-            "t1",
-            "stb1"
+            f"{dbname}.t1",
+            f"{dbname}.stb1"
         ]
         for tb in tbname:
             for i in range(2,8):
                 self.__concat_ws_check(tb,i)
                 tdLog.printNoPrefix(f"==========current sql condition check in {tb}, col num: {i} over==========")
 
-    def __test_error(self):
+    def __test_error(self, dbname="db"):
         tdLog.printNoPrefix("==========err sql condition check , must return error==========")
         tbname = [
-            "ct1",
-            "ct2",
-            "ct4",
+            f"{dbname}.ct1",
+            f"{dbname}.ct2",
+            f"{dbname}.ct4",
         ]
 
         for tb in tbname:
@@ -164,22 +164,21 @@ class TDTestCase:
             tdLog.printNoPrefix(f"==========err sql condition check in {tb} over==========")
 
 
-    def all_test(self):
-        self.__test_current()
-        self.__test_error()
+    def all_test(self,dbname="db"):
+        self.__test_current(dbname)
+        self.__test_error(dbname)
 
 
-    def __create_tb(self):
-        tdSql.prepare()
+    def __create_tb(self, dbname="db"):
 
         tdLog.printNoPrefix("==========step1:create table")
-        create_stb_sql  =  f'''create table stb1(
+        create_stb_sql  =  f'''create table {dbname}.stb1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
             ) tags (t1 int)
             '''
-        create_ntb_sql = f'''create table t1(
+        create_ntb_sql = f'''create table {dbname}.t1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
@@ -189,29 +188,29 @@ class TDTestCase:
         tdSql.execute(create_ntb_sql)
 
         for i in range(4):
-            tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )')
+            tdSql.execute(f'create table {dbname}.ct{i+1} using {dbname}.stb1 tags ( {i+1} )')
 
-    def __insert_data(self, rows):
+    def __insert_data(self, rows, dbname="db"):
         now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000)
         for i in range(rows):
             tdSql.execute(
-                f"insert into ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
         tdSql.execute(
-            f'''insert into ct1 values
+            f'''insert into {dbname}.ct1 values
             ( { now_time - rows * 5 }, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar_测试_0', { now_time + 8 } )
             ( { now_time + 10000 }, { rows }, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar_测试_9', { now_time + 9 } )
             '''
         )
 
         tdSql.execute(
-            f'''insert into ct4 values
+            f'''insert into {dbname}.ct4 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time +  7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -227,7 +226,7 @@ class TDTestCase:
         )
 
         tdSql.execute(
-            f'''insert into ct2 values
+            f'''insert into {dbname}.ct2 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time + 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -243,13 +242,13 @@ class TDTestCase:
         )
 
         for i in range(rows):
-            insert_data = f'''insert into t1 values
+            insert_data = f'''insert into {dbname}.t1 values
                 ( { now_time - i * 3600000 }, {i}, {i * 11111}, { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2},
                 "binary_{i}", "nchar_测试_{i}", { now_time - 1000 * i } )
                 '''
             tdSql.execute(insert_data)
         tdSql.execute(
-            f'''insert into t1 values
+            f'''insert into {dbname}.t1 values
             ( { now_time + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - (( rows // 2 ) * 60 + 30) * 60000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3600000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -269,22 +268,23 @@ class TDTestCase:
         tdSql.prepare()
 
         tdLog.printNoPrefix("==========step1:create table")
-        self.__create_tb()
+        self.__create_tb(dbname="db")
 
         tdLog.printNoPrefix("==========step2:insert data")
         self.rows = 10
-        self.__insert_data(self.rows)
+        self.__insert_data(self.rows, dbname="db")
 
         tdLog.printNoPrefix("==========step3:all check")
-        self.all_test()
+        self.all_test(dbname="db")
 
-        tdDnodes.stop(1)
-        tdDnodes.start(1)
+        # tdDnodes.stop(1)
+        # tdDnodes.start(1)
+        tdSql.execute("flush database db")
 
         tdSql.execute("use db")
 
         tdLog.printNoPrefix("==========step4:after wal, all check again ")
-        self.all_test()
+        self.all_test(dbname="db")
 
     def stop(self):
         tdSql.close()

tests/system-test/2-query/concat_ws2.py

@@ -137,23 +137,23 @@ class TDTestCase:
 
         return sqls
 
-    def __test_current(self):  # sourcery skip: use-itertools-product
+    def __test_current(self, dbname="db"):  # sourcery skip: use-itertools-product
         tdLog.printNoPrefix("==========current sql condition check , must return query ok==========")
         tbname = [
-            "ct1",
-            "ct2",
-            "ct4",
+            f"{dbname}.ct1",
+            f"{dbname}.ct2",
+            f"{dbname}.ct4",
         ]
         for tb in tbname:
             for i in range(2,8):
                 self.__concat_ws_check(tb,i)
                 tdLog.printNoPrefix(f"==========current sql condition check in {tb}, col num: {i} over==========")
 
-    def __test_error(self):
+    def __test_error(self, dbname="db"):
         tdLog.printNoPrefix("==========err sql condition check , must return error==========")
         tbname = [
-            "t1",
-            "stb1"
+            f"{dbname}.t1",
+            f"{dbname}.stb1"
         ]
 
         for tb in tbname:
@@ -164,22 +164,21 @@ class TDTestCase:
             tdLog.printNoPrefix(f"==========err sql condition check in {tb} over==========")
 
 
-    def all_test(self):
-        self.__test_current()
-        self.__test_error()
+    def all_test(self, dbname="db"):
+        self.__test_current(dbname="db")
+        self.__test_error(dbname="db")
 
 
-    def __create_tb(self):
-        tdSql.prepare()
+    def __create_tb(self, dbname="db"):
 
         tdLog.printNoPrefix("==========step1:create table")
-        create_stb_sql  =  f'''create table stb1(
+        create_stb_sql  =  f'''create table {dbname}.stb1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
             ) tags (t1 int)
             '''
-        create_ntb_sql = f'''create table t1(
+        create_ntb_sql = f'''create table {dbname}.t1(
                 ts timestamp, {INT_COL} int, {BINT_COL} bigint, {SINT_COL} smallint, {TINT_COL} tinyint,
                  {FLOAT_COL} float, {DOUBLE_COL} double, {BOOL_COL} bool,
                  {BINARY_COL} binary(16), {NCHAR_COL} nchar(32), {TS_COL} timestamp
@@ -189,29 +188,29 @@ class TDTestCase:
         tdSql.execute(create_ntb_sql)
 
         for i in range(4):
-            tdSql.execute(f'create table ct{i+1} using stb1 tags ( {i+1} )')
+            tdSql.execute(f'create table {dbname}.ct{i+1} using {dbname}.stb1 tags ( {i+1} )')
 
-    def __insert_data(self, rows):
+    def __insert_data(self, rows, dbname="db"):
         now_time = int(datetime.datetime.timestamp(datetime.datetime.now()) * 1000)
         for i in range(rows):
             tdSql.execute(
-                f"insert into ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct1 values ( { now_time - i * 1000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct4 values ( { now_time - i * 7776000000 }, {i}, {11111 * i}, {111 * i % 32767 }, {11 * i % 127}, {1.11*i}, {1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
             tdSql.execute(
-                f"insert into ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
+                f"insert into {dbname}.ct2 values ( { now_time - i * 7776000000 }, {-i},  {-11111 * i}, {-111 * i % 32767 }, {-11 * i % 127}, {-1.11*i}, {-1100.0011*i}, {i%2}, 'binary{i}', 'nchar_测试_{i}', { now_time + 1 * i } )"
             )
         tdSql.execute(
-            f'''insert into ct1 values
+            f'''insert into {dbname}.ct1 values
             ( { now_time - rows * 5 }, 0, 0, 0, 0, 0, 0, 0, 'binary0', 'nchar_测试_0', { now_time + 8 } )
             ( { now_time + 10000 }, { rows }, -99999, -999, -99, -9.99, -99.99, 1, 'binary9', 'nchar_测试_9', { now_time + 9 } )
             '''
         )
 
         tdSql.execute(
-            f'''insert into ct4 values
+            f'''insert into {dbname}.ct4 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time +  7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -227,7 +226,7 @@ class TDTestCase:
         )
 
         tdSql.execute(
-            f'''insert into ct2 values
+            f'''insert into {dbname}.ct2 values
             ( { now_time - rows * 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3888000000 + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time + 7776000000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -243,13 +242,13 @@ class TDTestCase:
         )
 
         for i in range(rows):
-            insert_data = f'''insert into t1 values
+            insert_data = f'''insert into {dbname}.t1 values
                 ( { now_time - i * 3600000 }, {i}, {i * 11111}, { i % 32767 }, { i % 127}, { i * 1.11111 }, { i * 1000.1111 }, { i % 2},
                 "binary_{i}", "nchar_测试_{i}", { now_time - 1000 * i } )
                 '''
             tdSql.execute(insert_data)
         tdSql.execute(
-            f'''insert into t1 values
+            f'''insert into {dbname}.t1 values
             ( { now_time + 10800000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - (( rows // 2 ) * 60 + 30) * 60000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
             ( { now_time - rows * 3600000 }, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL )
@@ -269,22 +268,23 @@ class TDTestCase:
         tdSql.prepare()
 
         tdLog.printNoPrefix("==========step1:create table")
-        self.__create_tb()
+        self.__create_tb(dbname="db")
 
         tdLog.printNoPrefix("==========step2:insert data")
         self.rows = 10
-        self.__insert_data(self.rows)
+        self.__insert_data(self.rows, dbname="db")
 
         tdLog.printNoPrefix("==========step3:all check")
-        self.all_test()
+        self.all_test(dbname="db")
 
-        tdDnodes.stop(1)
-        tdDnodes.start(1)
+        # tdDnodes.stop(1)
+        # tdDnodes.start(1)
+        tdSql.execute("flush database db")
 
         tdSql.execute("use db")
 
         tdLog.printNoPrefix("==========step4:after wal, all check again ")
-        self.all_test()
+        self.all_test(dbname="db")
 
     def stop(self):
         tdSql.close()

tests/system-test/2-query/count.py

@@ -5,13 +5,14 @@ from util.sqlset import *
 class TDTestCase:
     def init(self, conn, logSql):
         tdLog.debug("start to execute %s" % __file__)
-        tdSql.init(conn.cursor(),logSql)
+        tdSql.init(conn.cursor(),False)
         self.setsql = TDSetSql()
         self.rowNum = 10
         self.ts = 1537146000000
 
-        self.ntbname = 'ntb'
-        self.stbname = 'stb'
+        dbname = "db"
+        self.ntbname = f'{dbname}.ntb'
+        self.stbname = f'{dbname}.stb'
         self.column_dict = {
             'ts':'timestamp',
             'c1':'int',

tests/system-test/2-query/count_partition.py

@@ -11,17 +11,17 @@ class TDTestCase:
         self.row_nums = 10
         self.tb_nums = 10
         self.ts = 1537146000000
-    
-    def prepare_datas(self, stb_name , tb_nums , row_nums ):
-        tdSql.execute(" use db ")
-        tdSql.execute(f" create stable {stb_name} (ts timestamp , c1 int , c2 bigint , c3 float , c4 double , c5 smallint , c6 tinyint , c7 bool , c8 binary(36) , c9 nchar(36) , uc1 int unsigned,\
+
+    def prepare_datas(self, stb_name , tb_nums , row_nums, dbname="db" ):
+        tdSql.execute(f" use {dbname} ")
+        tdSql.execute(f" create stable {dbname}.{stb_name} (ts timestamp , c1 int , c2 bigint , c3 float , c4 double , c5 smallint , c6 tinyint , c7 bool , c8 binary(36) , c9 nchar(36) , uc1 int unsigned,\
             uc2 bigint unsigned ,uc3 smallint unsigned , uc4 tinyint unsigned ) tags(t1 timestamp , t2 int , t3 bigint , t4 float , t5 double , t6 smallint , t7 tinyint , t8 bool , t9 binary(36)\
                 , t10 nchar(36) , t11 int unsigned , t12 bigint unsigned ,t13 smallint unsigned , t14 tinyint unsigned ) ")
-        
+
         for i in range(tb_nums):
-            tbname = f"sub_{stb_name}_{i}"
+            tbname = f"{dbname}.sub_{stb_name}_{i}"
             ts = self.ts + i*10000
-            tdSql.execute(f"create table {tbname} using {stb_name} tags ({ts} , {i} , {i}*10 ,{i}*1.0,{i}*1.0 , 1 , 2, 'true', 'binary_{i}' ,'nchar_{i}',{i},{i},10,20 )")
+            tdSql.execute(f"create table {tbname} using {dbname}.{stb_name} tags ({ts} , {i} , {i}*10 ,{i}*1.0,{i}*1.0 , 1 , 2, 'true', 'binary_{i}' ,'nchar_{i}',{i},{i},10,20 )")
 
             for row in range(row_nums):
                 ts = self.ts + row*1000
@@ -30,143 +30,144 @@ class TDTestCase:
             for null in range(5):
                 ts =  self.ts + row_nums*1000 + null*1000
                 tdSql.execute(f"insert into {tbname} values({ts} , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL , NULL )")
-            
-    def basic_query(self):
-        tdSql.query("select count(*) from stb")
+
+    def basic_query(self, dbname="db"):
+        tdSql.query(f"select count(*) from {dbname}.stb")
         tdSql.checkData(0,0,(self.row_nums + 5 )*self.tb_nums)
-        tdSql.query("select count(c1) from stb")
+        tdSql.query(f"select count(c1) from {dbname}.stb")
         tdSql.checkData(0,0,(self.row_nums )*self.tb_nums)
-        tdSql.query(" select tbname , count(*) from stb partition by tbname ")
+        tdSql.query(f"select tbname , count(*) from {dbname}.stb partition by tbname ")
         tdSql.checkRows(self.tb_nums)
-        tdSql.query(" select count(c1) from stb group by t1 order by t1 ")
+        tdSql.query(f"select count(c1) from {dbname}.stb group by t1 order by t1 ")
         tdSql.checkRows(self.tb_nums)
-        tdSql.error(" select count(c1) from stb group by c1 order by t1 ")
-        tdSql.error(" select count(t1) from stb group by c1 order by t1 ")
-        tdSql.query(" select count(c1) from stb group by tbname order by tbname ")
+        tdSql.error(f"select count(c1) from {dbname}.stb group by c1 order by t1 ")
+        tdSql.error(f"select count(t1) from {dbname}.stb group by c1 order by t1 ")
+        tdSql.query(f"select count(c1) from {dbname}.stb group by tbname order by tbname ")
         tdSql.checkRows(self.tb_nums)
-        # bug need fix 
-        # tdSql.query(" select count(t1) from stb group by t2 order by t2 ")
+        # bug need fix
+        # tdSql.query(f"select count(t1) from {dbname}.stb group by t2 order by t2 ")
         # tdSql.checkRows(self.tb_nums)
-        tdSql.query(" select count(c1) from stb group by c1 order by c1 ")
+        tdSql.query(f"select count(c1) from {dbname}.stb group by c1 order by c1 ")
         tdSql.checkRows(self.row_nums+1)
 
-        tdSql.query(" select c1 , count(c1) from stb group by c1 order by c1 ")
+        tdSql.query(f"select c1 , count(c1) from {dbname}.stb group by c1 order by c1 ")
         tdSql.checkRows(self.row_nums+1)
 
-        tdSql.query("select count(c1) from stb group by abs(c1) order by abs(c1)")
+        tdSql.query(f"select count(c1) from {dbname}.stb group by abs(c1) order by abs(c1)")
         tdSql.checkRows(self.row_nums+1)
-        tdSql.query("select abs(c1+c3), count(c1+c3) from stb group by abs(c1+c3) order by abs(c1+c3)")
+        tdSql.query(f"select abs(c1+c3), count(c1+c3) from {dbname}.stb group by abs(c1+c3) order by abs(c1+c3)")
         tdSql.checkRows(self.row_nums+1)
-        tdSql.query("select count(c1+c3)+max(c2) ,abs(c1) from stb group by abs(c1) order by abs(c1)")
+        tdSql.query(f"select count(c1+c3)+max(c2) ,abs(c1) from {dbname}.stb group by abs(c1) order by abs(c1)")
         tdSql.checkRows(self.row_nums+1)
-        tdSql.error("select count(c1+c3)+max(c2) ,abs(c1) ,abs(t1) from stb group by abs(c1) order by abs(t1)+c2")
-        tdSql.error("select count(c1+c3)+max(c2) ,abs(c1) from stb group by abs(c1) order by abs(c1)+c2")
-        tdSql.query("select abs(c1+c3)+abs(c2) , count(c1+c3)+count(c2) from stb group by abs(c1+c3)+abs(c2) order by abs(c1+c3)+abs(c2)")
+        tdSql.error(f"select count(c1+c3)+max(c2) ,abs(c1) ,abs(t1) from {dbname}.stb group by abs(c1) order by abs(t1)+c2")
+        tdSql.error(f"select count(c1+c3)+max(c2) ,abs(c1) from {dbname}.stb group by abs(c1) order by abs(c1)+c2")
+        tdSql.query(f"select abs(c1+c3)+abs(c2) , count(c1+c3)+count(c2) from {dbname}.stb group by abs(c1+c3)+abs(c2) order by abs(c1+c3)+abs(c2)")
         tdSql.checkRows(self.row_nums+1)
 
-        tdSql.query("select count(c1) , count(t2) from stb where abs(c1+t2)=1 partition by tbname")
+        tdSql.query(f"select count(c1) , count(t2) from {dbname}.stb where abs(c1+t2)=1 partition by tbname")
         tdSql.checkRows(2)
-        tdSql.query("select count(c1) from stb where abs(c1+t2)=1 partition by tbname")
+        tdSql.query(f"select count(c1) from {dbname}.stb where abs(c1+t2)=1 partition by tbname")
         tdSql.checkRows(2)
-        
-        tdSql.query("select tbname , count(c1) from stb partition by tbname order by tbname")
+
+        tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname order by tbname")
         tdSql.checkRows(self.tb_nums)
         tdSql.checkData(0,1,self.row_nums)
 
-        tdSql.error("select tbname , count(c1) from stb partition by t1 order by t1")
-        tdSql.error("select tbname , count(t1) from stb partition by t1 order by t1")
-        tdSql.error("select tbname , count(t1) from stb partition by t2 order by t2")
+        tdSql.error(f"select tbname , count(c1) from {dbname}.stb partition by t1 order by t1")
+        tdSql.error(f"select tbname , count(t1) from {dbname}.stb partition by t1 order by t1")
+        tdSql.error(f"select tbname , count(t1) from {dbname}.stb partition by t2 order by t2")
 
-        # # bug need fix 
-        # tdSql.query("select t2 , count(t1) from stb partition by t2 order by t2")
+        # # bug need fix
+        # tdSql.query(f"select t2 , count(t1) from {dbname}.stb partition by t2 order by t2")
         # tdSql.checkRows(self.tb_nums)
 
-        tdSql.query("select tbname , count(c1) from stb partition by tbname order by tbname")
+        tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname order by tbname")
         tdSql.checkRows(self.tb_nums)
         tdSql.checkData(0,1,self.row_nums)
 
-        
-        tdSql.error("select tbname , count(c1) from stb partition by t2 order by t2")
 
-        tdSql.query("select c2, count(c1) from stb partition by c2 order by c2 desc")
+        tdSql.error(f"select tbname , count(c1) from {dbname}.stb partition by t2 order by t2")
+
+        tdSql.query(f"select c2, count(c1) from {dbname}.stb partition by c2 order by c2 desc")
         tdSql.checkRows(self.tb_nums+1)
         tdSql.checkData(0,1,self.tb_nums)
 
-        tdSql.error("select tbname , count(c1) from stb partition by c1 order by c2")
+        tdSql.error(f"select tbname , count(c1) from {dbname}.stb partition by c1 order by c2")
 
 
-        tdSql.query("select tbname , abs(t2) from stb partition by c2 order by t2")
+        tdSql.query(f"select tbname , abs(t2) from {dbname}.stb partition by c2 order by t2")
         tdSql.checkRows(self.tb_nums*(self.row_nums+5))
 
-        tdSql.query("select count(c1) , count(t2) from stb partition by c2 ")
+        tdSql.query(f"select count(c1) , count(t2) from {dbname}.stb partition by c2 ")
         tdSql.checkRows(self.row_nums+1)
         tdSql.checkData(0,1,self.row_nums)
 
-        tdSql.query("select count(c1) , count(t2) ,c2 from stb partition by c2 order by c2")
+        tdSql.query(f"select count(c1) , count(t2) ,c2 from {dbname}.stb partition by c2 order by c2")
         tdSql.checkRows(self.row_nums+1)
 
-        tdSql.query("select count(c1) , count(t1) ,max(c2) ,tbname  from stb partition by tbname order by tbname")
+        tdSql.query(f"select count(c1) , count(t1) ,max(c2) ,tbname  from {dbname}.stb partition by tbname order by tbname")
         tdSql.checkRows(self.tb_nums)
         tdSql.checkCols(4)
 
-        tdSql.query("select count(c1) , count(t2) ,t1  from stb partition by t1 order by t1")
+        tdSql.query(f"select count(c1) , count(t2) ,t1  from {dbname}.stb partition by t1 order by t1")
         tdSql.checkRows(self.tb_nums)
         tdSql.checkData(0,0,self.row_nums)
 
-        # bug need fix 
-        # tdSql.query("select count(c1) , count(t1) ,abs(c1) from stb partition by abs(c1) order by abs(c1)")
+        # bug need fix
+        # tdSql.query(f"select count(c1) , count(t1) ,abs(c1) from {dbname}.stb partition by abs(c1) order by abs(c1)")
         # tdSql.checkRows(self.row_nums+1)
-   
 
-        tdSql.query("select count(ceil(c2)) , count(floor(t2)) ,count(floor(c2)) from stb partition by abs(c2) order by abs(c2)")
+
+        tdSql.query(f"select count(ceil(c2)) , count(floor(t2)) ,count(floor(c2)) from {dbname}.stb partition by abs(c2) order by abs(c2)")
         tdSql.checkRows(self.row_nums+1)
 
 
-        tdSql.query("select count(ceil(c1-2)) , count(floor(t2+1)) ,max(c2-c1) from stb partition by abs(floor(c1)) order by abs(floor(c1))")
+        tdSql.query(f"select count(ceil(c1-2)) , count(floor(t2+1)) ,max(c2-c1) from {dbname}.stb partition by abs(floor(c1)) order by abs(floor(c1))")
         tdSql.checkRows(self.row_nums+1)
-     
 
-        # interval 
-        tdSql.query("select count(c1) from stb interval(2s) sliding(1s)")
+
+        # interval
+        tdSql.query(f"select count(c1) from {dbname}.stb interval(2s) sliding(1s)")
 
         # bug need fix
 
-        tdSql.query('select max(c1) from stb where ts>="2022-07-06 16:00:00.000 " and ts < "2022-07-06 17:00:00.000 " interval(50s) sliding(30s) fill(NULL)')
+        tdSql.query(f'select max(c1) from {dbname}.stb where ts>="2022-07-06 16:00:00.000 " and ts < "2022-07-06 17:00:00.000 " interval(50s) sliding(30s) fill(NULL)')
 
-        tdSql.query(" select tbname , count(c1) from stb partition by tbname interval(10s) slimit 5 soffset 1 ")
+        tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname interval(10s) slimit 5 soffset 1 ")
 
-        tdSql.query("select tbname , count(c1) from stb partition by tbname interval(10s)")
+        tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname interval(10s)")
 
-        tdSql.query("select tbname , count(c1) from sub_stb_1 partition by tbname interval(10s)")
+        tdSql.query(f"select tbname , count(c1) from {dbname}.sub_stb_1 partition by tbname interval(10s)")
         tdSql.checkData(0,0,'sub_stb_1')
         tdSql.checkData(0,1,self.row_nums)
 
-        # tdSql.query(" select tbname , count(c1) from stb partition by tbname order by tbname slimit 5 soffset 0 ")  
+        # tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname order by tbname slimit 5 soffset 0 ")
+        # tdSql.checkRows(5)
+
+        # tdSql.query(f"select tbname , count(c1) from {dbname}.stb partition by tbname order by tbname slimit 5 soffset 1 ")
         # tdSql.checkRows(5)
-        
-        # tdSql.query(" select tbname , count(c1) from stb partition by tbname order by tbname slimit 5 soffset 1 ")
-        # tdSql.checkRows(5)     
-         
-        tdSql.query(" select tbname , count(c1) from sub_stb_1 partition by tbname interval(10s) sliding(5s) ")   
-       
-        tdSql.query(f'select max(c1) from stb where ts>={self.ts} and ts < {self.ts}+10000 partition by tbname interval(50s) sliding(30s)')
-        tdSql.query(f'select max(c1) from stb where ts>={self.ts} and ts < {self.ts}+10000 interval(50s) sliding(30s)')
-        tdSql.query(f'select tbname , count(c1) from stb where ts>={self.ts} and ts < {self.ts}+10000 partition by tbname interval(50s) sliding(30s)')
+
+        tdSql.query(f"select tbname , count(c1) from {dbname}.sub_stb_1 partition by tbname interval(10s) sliding(5s) ")
+
+        tdSql.query(f'select max(c1) from {dbname}.stb where ts>={self.ts} and ts < {self.ts}+10000 partition by tbname interval(50s) sliding(30s)')
+        tdSql.query(f'select max(c1) from {dbname}.stb where ts>={self.ts} and ts < {self.ts}+10000 interval(50s) sliding(30s)')
+        tdSql.query(f'select tbname , count(c1) from {dbname}.stb where ts>={self.ts} and ts < {self.ts}+10000 partition by tbname interval(50s) sliding(30s)')
 
 
     def run(self):
         tdSql.prepare()
         self.prepare_datas("stb",self.tb_nums,self.row_nums)
         self.basic_query()
+        dbname="db"
+
+        # # coverage case for taosd crash about bug fix
+        tdSql.query(f"select sum(c1) from {dbname}.stb where t2+10 >1 ")
+        tdSql.query(f"select count(c1),count(t1) from {dbname}.stb where -t2<1 ")
+        tdSql.query(f"select tbname ,max(ceil(c1)) from {dbname}.stb group by tbname ")
+        tdSql.query(f"select avg(abs(c1)) , tbname from {dbname}.stb group by tbname ")
+        tdSql.query(f"select t1,c1 from {dbname}.stb where abs(t2+c1)=1 ")
 
-        # # coverage case for taosd crash about bug fix 
-        tdSql.query(" select sum(c1) from stb where t2+10 >1 ")
-        tdSql.query(" select count(c1),count(t1) from stb where -t2<1 ")
-        tdSql.query(" select tbname ,max(ceil(c1)) from stb group by tbname ")
-        tdSql.query(" select avg(abs(c1)) , tbname from stb group by tbname ")
-        tdSql.query(" select t1,c1 from stb where abs(t2+c1)=1 ")
 
-        
     def stop(self):
         tdSql.close()
         tdLog.success("%s successfully executed" % __file__)

tests/system-test/2-query/db.py

@@ -10,9 +10,6 @@ import random
 
 
 class TDTestCase:
-    updatecfgDict = {'debugFlag': 143, "cDebugFlag": 143, "uDebugFlag": 143, "rpcDebugFlag": 143, "tmrDebugFlag": 143,
-                     "jniDebugFlag": 143, "simDebugFlag": 143, "dDebugFlag": 143, "dDebugFlag": 143, "vDebugFlag": 143, "mDebugFlag": 143, "qDebugFlag": 143,
-                     "wDebugFlag": 143, "sDebugFlag": 143, "tsdbDebugFlag": 143, "tqDebugFlag": 143, "fsDebugFlag": 143, "udfDebugFlag": 143}
 
     def init(self, conn, logSql):
         tdLog.debug(f"start to excute {__file__}")
@@ -26,22 +23,22 @@ class TDTestCase:
         tdSql.execute("create table dbms.ntb (ts timestamp, c1 int, c2 bigint)")
         tdSql.execute("create table dbus.ntb (ts timestamp, c1 int, c2 bigint)")
         tdSql.execute("create table dbns.ntb (ts timestamp, c1 int, c2 bigint)")
-        
+
         tdSql.execute("insert into dbms.ntb values ('2022-01-01 08:00:00.001', 1, 2)")
         tdSql.execute("insert into dbms.ntb values ('2022-01-01 08:00:00.002', 3, 4)")
-        
+
         tdSql.execute("insert into dbus.ntb values ('2022-01-01 08:00:00.000001', 1, 2)")
         tdSql.execute("insert into dbus.ntb values ('2022-01-01 08:00:00.000002', 3, 4)")
-        
+
         tdSql.execute("insert into dbns.ntb values ('2022-01-01 08:00:00.000000001', 1, 2)")
         tdSql.execute("insert into dbns.ntb values ('2022-01-01 08:00:00.000000002', 3, 4)")
-        
+
         tdSql.query("select count(c1) from dbms.ntb interval(1a)")
         tdSql.checkRows(2)
-        
+
         tdSql.query("select count(c1) from dbus.ntb interval(1u)")
         tdSql.checkRows(2)
-        
+
         tdSql.query("select count(c1) from dbns.ntb interval(1b)")
         tdSql.checkRows(2)
 

tests/system-test/2-query/diff.py

@@ -18,188 +18,117 @@ class TDTestCase:
 
     def run(self):
         tdSql.prepare()
+        dbname = "db"
         tdSql.execute(
-            "create table ntb(ts timestamp,c1 int,c2 double,c3 float)")
+            f"create table {dbname}.ntb(ts timestamp,c1 int,c2 double,c3 float)")
         tdSql.execute(
-            "insert into ntb values(now,1,1.0,10.5)(now+1s,10,-100.0,5.1)(now+10s,-1,15.1,5.0)")
+            f"insert into {dbname}.ntb values(now,1,1.0,10.5)(now+1s,10,-100.0,5.1)(now+10s,-1,15.1,5.0)")
 
-        tdSql.query("select diff(c1,0) from ntb")
+        tdSql.query(f"select diff(c1,0) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, 9)
         tdSql.checkData(1, 0, -11)
-        tdSql.query("select diff(c1,1) from ntb")
+        tdSql.query(f"select diff(c1,1) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, 9)
         tdSql.checkData(1, 0, None)
 
-        tdSql.query("select diff(c2,0) from ntb")
+        tdSql.query(f"select diff(c2,0) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, -101)
         tdSql.checkData(1, 0, 115.1)
-        tdSql.query("select diff(c2,1) from ntb")
+        tdSql.query(f"select diff(c2,1) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, None)
         tdSql.checkData(1, 0, 115.1)
 
-        tdSql.query("select diff(c3,0) from ntb")
+        tdSql.query(f"select diff(c3,0) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, -5.4)
         tdSql.checkData(1, 0, -0.1)
-        tdSql.query("select diff(c3,1) from ntb")
+        tdSql.query(f"select diff(c3,1) from {dbname}.ntb")
         tdSql.checkRows(2)
         tdSql.checkData(0, 0, None)
         tdSql.checkData(1, 0, None)
 
-        tdSql.execute('''create table stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double,
+        tdSql.execute(f'''create table {dbname}.stb(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double,
                     col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''')
-        tdSql.execute("create table stb_1 using stb tags('beijing')")
+        tdSql.execute(f"create table {dbname}.stb_1 using {dbname}.stb tags('beijing')")
         tdSql.execute(
-            "insert into stb_1 values(%d, 0, 0, 0, 0, 0.0, 0.0, False, ' ', ' ', 0, 0, 0, 0)" % (self.ts - 1))
+            f"insert into {dbname}.stb_1 values(%d, 0, 0, 0, 0, 0.0, 0.0, False, ' ', ' ', 0, 0, 0, 0)" % (self.ts - 1))
 
         # diff verifacation
-        tdSql.query("select diff(col1) from stb_1")
+        tdSql.query(f"select diff(col1) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col2) from stb_1")
+        tdSql.query(f"select diff(col2) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col3) from stb_1")
+        tdSql.query(f"select diff(col3) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col4) from stb_1")
+        tdSql.query(f"select diff(col4) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col5) from stb_1")
+        tdSql.query(f"select diff(col5) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col6) from stb_1")
+        tdSql.query(f"select diff(col6) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
-        tdSql.query("select diff(col7) from stb_1")
+        tdSql.query(f"select diff(col7) from {dbname}.stb_1")
         tdSql.checkRows(0)
 
         for i in range(self.rowNum):
-            tdSql.execute("insert into stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
+            tdSql.execute(f"insert into {dbname}.stb_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
                           % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1))
 
-        tdSql.error("select diff(ts) from stb")
-        tdSql.error("select diff(ts) from stb_1")
+        tdSql.error(f"select diff(ts) from {dbname}.stb")
+        tdSql.error(f"select diff(ts) from {dbname}.stb_1")
 
-        # tdSql.error("select diff(col7) from stb")
+        # tdSql.error(f"select diff(col7) from  {dbname}.stb")
 
-        tdSql.error("select diff(col8) from stb")
-        tdSql.error("select diff(col8) from stb_1")
-        tdSql.error("select diff(col9) from stb")
-        tdSql.error("select diff(col9) from stb_1")
-        tdSql.error("select diff(col11) from stb_1")
-        tdSql.error("select diff(col12) from stb_1")
-        tdSql.error("select diff(col13) from stb_1")
-        tdSql.error("select diff(col14) from stb_1")
+        tdSql.error(f"select diff(col8) from {dbname}.stb")
+        tdSql.error(f"select diff(col8) from {dbname}.stb_1")
+        tdSql.error(f"select diff(col9) from {dbname}.stb")
+        tdSql.error(f"select diff(col9) from {dbname}.stb_1")
+        tdSql.error(f"select diff(col11) from  {dbname}.stb_1")
+        tdSql.error(f"select diff(col12) from  {dbname}.stb_1")
+        tdSql.error(f"select diff(col13) from  {dbname}.stb_1")
+        tdSql.error(f"select diff(col14) from  {dbname}.stb_1")
+        tdSql.query(f"select ts,diff(col1),ts from  {dbname}.stb_1")
 
-        tdSql.query("select diff(col1) from stb_1")
+        tdSql.query(f"select diff(col1) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        tdSql.query("select diff(col2) from stb_1")
+        tdSql.query(f"select diff(col2) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        tdSql.query("select diff(col3) from stb_1")
+        tdSql.query(f"select diff(col3) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        tdSql.query("select diff(col4) from stb_1")
+        tdSql.query(f"select diff(col4) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        tdSql.query("select diff(col5) from stb_1")
+        tdSql.query(f"select diff(col5) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        tdSql.query("select diff(col6) from stb_1")
+        tdSql.query(f"select diff(col6) from  {dbname}.stb_1")
         tdSql.checkRows(10)
 
-        # check selectivity
-        tdSql.query("select ts, diff(col1), col2 from stb_1")
-        tdSql.checkRows(10)
-        tdSql.checkData(0, 0, "2018-09-17 09:00:00.000")
-        tdSql.checkData(1, 0, "2018-09-17 09:00:00.001")
-        tdSql.checkData(2, 0, "2018-09-17 09:00:00.002")
-        tdSql.checkData(3, 0, "2018-09-17 09:00:00.003")
-        tdSql.checkData(4, 0, "2018-09-17 09:00:00.004")
-        tdSql.checkData(5, 0, "2018-09-17 09:00:00.005")
-        tdSql.checkData(6, 0, "2018-09-17 09:00:00.006")
-        tdSql.checkData(7, 0, "2018-09-17 09:00:00.007")
-        tdSql.checkData(8, 0, "2018-09-17 09:00:00.008")
-        tdSql.checkData(9, 0, "2018-09-17 09:00:00.009")
-
-        tdSql.checkData(0, 1, 1)
-        tdSql.checkData(1, 1, 1)
-        tdSql.checkData(2, 1, 1)
-        tdSql.checkData(3, 1, 1)
-        tdSql.checkData(4, 1, 1)
-        tdSql.checkData(5, 1, 1)
-        tdSql.checkData(6, 1, 1)
-        tdSql.checkData(7, 1, 1)
-        tdSql.checkData(8, 1, 1)
-        tdSql.checkData(9, 1, 1)
-
-        tdSql.checkData(0, 2, 0)
-        tdSql.checkData(1, 2, 1)
-        tdSql.checkData(2, 2, 2)
-        tdSql.checkData(3, 2, 3)
-        tdSql.checkData(4, 2, 4)
-        tdSql.checkData(5, 2, 5)
-        tdSql.checkData(6, 2, 6)
-        tdSql.checkData(7, 2, 7)
-        tdSql.checkData(8, 2, 8)
-        tdSql.checkData(9, 2, 9)
-
-        tdSql.query("select ts, diff(col1), col2 from stb order by ts")
-        tdSql.checkRows(10)
-
-        tdSql.checkData(0, 0, "2018-09-17 09:00:00.000")
-        tdSql.checkData(1, 0, "2018-09-17 09:00:00.001")
-        tdSql.checkData(2, 0, "2018-09-17 09:00:00.002")
-        tdSql.checkData(3, 0, "2018-09-17 09:00:00.003")
-        tdSql.checkData(4, 0, "2018-09-17 09:00:00.004")
-        tdSql.checkData(5, 0, "2018-09-17 09:00:00.005")
-        tdSql.checkData(6, 0, "2018-09-17 09:00:00.006")
-        tdSql.checkData(7, 0, "2018-09-17 09:00:00.007")
-        tdSql.checkData(8, 0, "2018-09-17 09:00:00.008")
-        tdSql.checkData(9, 0, "2018-09-17 09:00:00.009")
-
-        tdSql.checkData(0, 1, 1)
-        tdSql.checkData(1, 1, 1)
-        tdSql.checkData(2, 1, 1)
-        tdSql.checkData(3, 1, 1)
-        tdSql.checkData(4, 1, 1)
-        tdSql.checkData(5, 1, 1)
-        tdSql.checkData(6, 1, 1)
-        tdSql.checkData(7, 1, 1)
-        tdSql.checkData(8, 1, 1)
-        tdSql.checkData(9, 1, 1)
-
-        tdSql.checkData(0, 2, 0)
-        tdSql.checkData(1, 2, 1)
-        tdSql.checkData(2, 2, 2)
-        tdSql.checkData(3, 2, 3)
-        tdSql.checkData(4, 2, 4)
-        tdSql.checkData(5, 2, 5)
-        tdSql.checkData(6, 2, 6)
-        tdSql.checkData(7, 2, 7)
-        tdSql.checkData(8, 2, 8)
-        tdSql.checkData(9, 2, 9)
-
-
-        tdSql.execute('''create table stb1(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double,
+        tdSql.execute(f'''create table  {dbname}.stb1(ts timestamp, col1 tinyint, col2 smallint, col3 int, col4 bigint, col5 float, col6 double,
                     col7 bool, col8 binary(20), col9 nchar(20), col11 tinyint unsigned, col12 smallint unsigned, col13 int unsigned, col14 bigint unsigned) tags(loc nchar(20))''')
-        tdSql.execute("create table stb1_1 using stb tags('shanghai')")
+        tdSql.execute(f"create table  {dbname}.stb1_1 using  {dbname}.stb tags('shanghai')")
 
         for i in range(self.rowNum):
-            tdSql.execute("insert into stb1_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
+            tdSql.execute(f"insert into  {dbname}.stb1_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
                           % (self.ts + i, i + 1, i + 1, i + 1, i + 1, i + 0.1, i + 0.1, i % 2, i + 1, i + 1, i + 1, i + 1, i + 1, i + 1))
         for i in range(self.rowNum):
-            tdSql.execute("insert into stb1_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
+            tdSql.execute(f"insert into  {dbname}.stb1_1 values(%d, %d, %d, %d, %d, %f, %f, %d, 'taosdata%d', '涛思数据%d', %d, %d, %d, %d)"
                           % (self.ts - i-1, i-1, i-1, i-1, i-1, -i - 0.1, -i - 0.1, -i % 2, i - 1, i - 1, i + 1, i + 1, i + 1, i + 1))
-        tdSql.query("select diff(col1,0) from stb1_1")
+        tdSql.query(f"select diff(col1,0) from  {dbname}.stb1_1")
         tdSql.checkRows(19)
-        tdSql.query("select diff(col1,1) from stb1_1")
+        tdSql.query(f"select diff(col1,1) from  {dbname}.stb1_1")
         tdSql.checkRows(19)
         tdSql.checkData(0,0,None)
 

tests/system-test/7-tmq/tmqShow.py

@@ -51,32 +51,32 @@ class TDTestCase:
         tdCom.create_ctable(tdSql, dbname=paraDict["dbName"],stbname=paraDict["stbName"],tag_elm_list=paraDict['tagSchema'],count=paraDict["ctbNum"], default_ctbname_prefix=paraDict['ctbPrefix'])
         # tdLog.info("insert data")
         # tmqCom.insert_data(tdSql,paraDict["dbName"],paraDict["ctbPrefix"],paraDict["ctbNum"],paraDict["rowsPerTbl"],paraDict["batchNum"],paraDict["startTs"])
-        
+
         tdLog.info("create 4 topics")
         sqlString = "create topic %s as database %s" %(topicNameList[0], paraDict['dbName'])
         tdLog.info("create topic sql: %s"%sqlString)
         tdSql.execute(sqlString)
-        
+
         sqlString = "create topic %s as stable %s.%s" %(topicNameList[1], paraDict['dbName'], paraDict['stbName'])
         tdLog.info("create topic sql: %s"%sqlString)
-        tdSql.execute(sqlString)        
+        tdSql.execute(sqlString)
 
         queryString = "select * from %s.%s where c1 %% 7 == 0" %(paraDict['dbName'], paraDict['stbName'])
         sqlString = "create topic %s as %s" %(topicNameList[2], queryString)
         tdLog.info("create topic sql: %s"%sqlString)
         tdSql.execute(sqlString)
-        
+
         queryString = "select ts, log(c1), ceil(pow(c1,3)) from %s.%s where c1 %% 7 == 0" %(paraDict['dbName'], paraDict['stbName'])
         sqlString = "create topic %s as %s " %(topicNameList[3], queryString)
         tdLog.info("create topic sql: %s"%sqlString)
         tdSql.execute(sqlString)
 
         tdSql.query("show topics")
-        tdLog.debug(tdSql.queryResult)     
+        tdLog.debug(tdSql.queryResult)
         rows = tdSql.getRows()
         if rows != len(consumerIdList):
             tdLog.exit("topic rows error")
-            
+
         for i in range (rows):
             topicName = tdSql.getData(i,0)
             matchFlag = 0
@@ -87,24 +87,24 @@ class TDTestCase:
                         break
                 if matchFlag == 0:
                     tdLog.exit("topic name: %s is error", topicName)
-        
+
         # init consume info, and start tmq_sim, then check consume result
         tdLog.info("insert consume info to consume processor")
         expectrowcnt = paraDict["rowsPerTbl"] * paraDict["ctbNum"]
         topicList    = topicNameList[0]
         ifcheckdata  = 0
-        ifManualCommit = 0        
+        ifManualCommit = 0
         keyList      = 'group.id:%s, enable.auto.commit:false, auto.commit.interval.ms:6000, auto.offset.reset:earliest'%consumeGroupIdList[0]
         tmqCom.insertConsumerInfo(consumerIdList[0], expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit)
-                
+
         topicList    = topicNameList[1]
         keyList      = 'group.id:%s, enable.auto.commit:false, auto.commit.interval.ms:6000, auto.offset.reset:earliest'%consumeGroupIdList[1]
         tmqCom.insertConsumerInfo(consumerIdList[1], expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit)
-        
+
         topicList    = topicNameList[2]
         keyList      = 'group.id:%s, enable.auto.commit:false, auto.commit.interval.ms:6000, auto.offset.reset:earliest'%consumeGroupIdList[2]
         tmqCom.insertConsumerInfo(consumerIdList[2], expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit)
-        
+
         topicList    = topicNameList[3]
         keyList      = 'group.id:%s, enable.auto.commit:false, auto.commit.interval.ms:6000, auto.offset.reset:earliest'%consumeGroupIdList[3]
         tmqCom.insertConsumerInfo(consumerIdList[3], expectrowcnt,topicList,keyList,ifcheckdata,ifManualCommit)
@@ -118,27 +118,27 @@ class TDTestCase:
         time.sleep(5)
         tdLog.info("check show consumers")
         tdSql.query("show consumers")
-        # tdLog.info(tdSql.queryResult)     
+        # tdLog.info(tdSql.queryResult)
         rows = tdSql.getRows()
         tdLog.info("show consumers rows: %d"%rows)
         if rows != len(topicNameList):
             tdLog.exit("show consumers rows error")
-        
-        tdLog.info("check show subscriptions")      
+
+        tdLog.info("check show subscriptions")
         tdSql.query("show subscriptions")
-        # tdLog.debug(tdSql.queryResult)     
+        # tdLog.debug(tdSql.queryResult)
         rows = tdSql.getRows()
         tdLog.info("show subscriptions rows: %d"%rows)
         if rows != paraDict['vgroups'] * len(topicNameList):
             tdLog.exit("show subscriptions rows error")
 
         pThread.join()
-        
+
         tdLog.info("insert process end, and start to check consume result")
         expectRows = len(consumerIdList)
         _ = tmqCom.selectConsumeResult(expectRows)
-        
-        time.sleep(10)        
+
+        time.sleep(10)
         for i in range(len(topicNameList)):
             tdSql.query("drop topic %s"%topicNameList[i])
 

tests/test/c/CMakeLists.txt

 add_executable(tmq_demo tmqDemo.c)
 add_executable(tmq_sim tmqSim.c)
 add_executable(create_table createTable.c)
+add_executable(tmq_taosx_ci tmq_taosx_ci.c)
 target_link_libraries(
     create_table
     PUBLIC taos_static
@@ -22,6 +23,13 @@ target_link_libraries(
     PUBLIC common
     PUBLIC os
 )
+target_link_libraries(
+    tmq_taosx_ci
+    PUBLIC taos_static
+    PUBLIC util
+    PUBLIC common
+    PUBLIC os
+)
 
 add_executable(sdbDump sdbDump.c)
 target_link_libraries(