Skip to content

TDengine
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1192
Star 22018
Fork 4786
TDengine
TDengine
未验证 提交 4ec4714e 编写于 作者: W wu champion 提交者： GitHub
浏览文件
操作

Merge pull request #2 from taosdata/develop 

[3.15更新]获取3.15TDengine内容
上级 96c8e5f3 78eda385
展开全部 隐藏空白更改
内联 并排
Showing with 10059 addition and 8021 deletion
CMakeLists.txt
浏览文件 @ 4ec4714e
...@@ -16,6 +16,7 @@ SET(TD_GRANT FALSE) ...@@ -16,6 +16,7 @@ SET(TD_GRANT FALSE)
SET(TD_MQTT FALSE) SET(TD_MQTT FALSE)
SET(TD_TSDB_PLUGINS FALSE) SET(TD_TSDB_PLUGINS FALSE)
SET(TD_STORAGE FALSE) SET(TD_STORAGE FALSE)
SET(TD_TOPIC FALSE)
SET(TD_COVER FALSE) SET(TD_COVER FALSE)
SET(TD_MEM_CHECK FALSE) SET(TD_MEM_CHECK FALSE)
......
Jenkinsfile
浏览文件 @ 4ec4714e
...@@ -46,6 +46,7 @@ def pre_test(){ ...@@ -46,6 +46,7 @@ def pre_test(){
git fetch origin +refs/pull/${CHANGE_ID}/merge git fetch origin +refs/pull/${CHANGE_ID}/merge
git checkout -qf FETCH_HEAD git checkout -qf FETCH_HEAD
git --no-pager diff --name-only FETCH_HEAD $(git merge-base FETCH_HEAD develop)|grep -v -E '.*md|//src//connector|Jenkinsfile' || exit 0 git --no-pager diff --name-only FETCH_HEAD $(git merge-base FETCH_HEAD develop)|grep -v -E '.*md|//src//connector|Jenkinsfile' || exit 0
find ${WKC}/tests/pytest -name \'*\'.sql -exec rm -rf {} \\;
cd ${WK} cd ${WK}
git reset --hard HEAD~10 git reset --hard HEAD~10
git checkout develop git checkout develop
...@@ -115,7 +116,6 @@ pipeline { ...@@ -115,7 +116,6 @@ pipeline {
sh ''' sh '''
date date
cd ${WKC}/tests cd ${WKC}/tests
find pytest -name '*'sql|xargs rm -rf
./test-all.sh p1 ./test-all.sh p1
date''' date'''
} }
...@@ -131,7 +131,6 @@ pipeline { ...@@ -131,7 +131,6 @@ pipeline {
sh ''' sh '''
date date
cd ${WKC}/tests cd ${WKC}/tests
find pytest -name '*'sql|xargs rm -rf
./test-all.sh p2 ./test-all.sh p2
date''' date'''
} }
......
cmake/define.inc
浏览文件 @ 4ec4714e
...@@ -25,6 +25,10 @@ IF (TD_STORAGE) ...@@ -25,6 +25,10 @@ IF (TD_STORAGE)
ADD_DEFINITIONS(-D_STORAGE) ADD_DEFINITIONS(-D_STORAGE)
ENDIF () ENDIF ()
IF (TD_TOPIC)
ADD_DEFINITIONS(-D_TOPIC)
ENDIF ()
IF (TD_GODLL) IF (TD_GODLL)
ADD_DEFINITIONS(-D_TD_GO_DLL_) ADD_DEFINITIONS(-D_TD_GO_DLL_)
ENDIF () ENDIF ()
......
cmake/input.inc
浏览文件 @ 4ec4714e
...@@ -9,6 +9,14 @@ ELSEIF (${ACCOUNT} MATCHES "false") ...@@ -9,6 +9,14 @@ ELSEIF (${ACCOUNT} MATCHES "false")
MESSAGE(STATUS "Build without account plugins") MESSAGE(STATUS "Build without account plugins")
ENDIF () ENDIF ()
IF (${TOPIC} MATCHES "true")
SET(TD_TOPIC TRUE)
MESSAGE(STATUS "Build with topic plugins")
ELSEIF (${TOPIC} MATCHES "false")
SET(TD_TOPIC FALSE)
MESSAGE(STATUS "Build without topic plugins")
ENDIF ()
IF (${COVER} MATCHES "true") IF (${COVER} MATCHES "true")
SET(TD_COVER TRUE) SET(TD_COVER TRUE)
MESSAGE(STATUS "Build with test coverage") MESSAGE(STATUS "Build with test coverage")
......
documentation20/cn/03.architecture/docs.md
浏览文件 @ 4ec4714e
...@@ -166,7 +166,7 @@ TDengine 分布式架构的逻辑结构图如下： ...@@ -166,7 +166,7 @@ TDengine 分布式架构的逻辑结构图如下：
**虚拟节点(vnode)**: 为更好的支持数据分片、负载均衡，防止数据过热或倾斜，数据节点被虚拟化成多个虚拟节点(vnode，图中V2, V3, V4等)。每个 vnode 都是一个相对独立的工作单元，是时序数据存储的基本单元，具有独立的运行线程、内存空间与持久化存储的路径。一个 vnode 包含一定数量的表（数据采集点）。当创建一张新表时，系统会检查是否需要创建新的 vnode。一个数据节点上能创建的 vnode 的数量取决于该数据节点所在物理节点的硬件资源。一个 vnode 只属于一个DB，但一个DB可以有多个 vnode。一个 vnode 除存储的时序数据外，也保存有所包含的表的schema、标签值等。一个虚拟节点由所属的数据节点的EP，以及所属的VGroup ID在系统内唯一标识，由管理节点创建并管理。 **虚拟节点(vnode)**: 为更好的支持数据分片、负载均衡，防止数据过热或倾斜，数据节点被虚拟化成多个虚拟节点(vnode，图中V2, V3, V4等)。每个 vnode 都是一个相对独立的工作单元，是时序数据存储的基本单元，具有独立的运行线程、内存空间与持久化存储的路径。一个 vnode 包含一定数量的表（数据采集点）。当创建一张新表时，系统会检查是否需要创建新的 vnode。一个数据节点上能创建的 vnode 的数量取决于该数据节点所在物理节点的硬件资源。一个 vnode 只属于一个DB，但一个DB可以有多个 vnode。一个 vnode 除存储的时序数据外，也保存有所包含的表的schema、标签值等。一个虚拟节点由所属的数据节点的EP，以及所属的VGroup ID在系统内唯一标识，由管理节点创建并管理。
**管理节点(mnode):** 一个虚拟的逻辑单元，负责所有数据节点运行状态的监控和维护，以及节点之间的负载均衡(图中M)。同时，管理节点也负责元数据(包括用户、数据库、表、静态标签等)的存储和管理，因此也称为 Meta Node。TDengine 集群中可配置多个(最多不超过5个) mnode，它们自动构建成为一个虚拟管理节点组(图中M0, M1, M2)。mnode 间采用 master/slave 的机制进行管理，而且采取强一致方式进行数据同步, 任何数据更新操作只能在 Master 上进行。mnode 集群的创建由系统自动完成，无需人工干预。每个dnode上至多有一个mnode，由所属的数据节点的EP来唯一标识。每个dnode通过内部消息交互自动获取整个集群中所有 mnode 所在的 dnode 的EP。 **管理节点(mnode):** 一个虚拟的逻辑单元，负责所有数据节点运行状态的监控和维护，以及节点之间的负载均衡(图中M)。同时，管理节点也负责元数据(包括用户、数据库、表、静态标签等)的存储和管理，因此也称为 Meta Node。TDengine 集群中可配置多个(开源版最多不超过3个) mnode，它们自动构建成为一个虚拟管理节点组(图中M0, M1, M2)。mnode 间采用 master/slave 的机制进行管理，而且采取强一致方式进行数据同步, 任何数据更新操作只能在 Master 上进行。mnode 集群的创建由系统自动完成，无需人工干预。每个dnode上至多有一个mnode，由所属的数据节点的EP来唯一标识。每个dnode通过内部消息交互自动获取整个集群中所有 mnode 所在的 dnode 的EP。
**虚拟节点组(VGroup):** 不同数据节点上的 vnode 可以组成一个虚拟节点组(vnode group)来保证系统的高可靠。虚拟节点组内采取master/slave的方式进行管理。写操作只能在 master vnode 上进行，系统采用异步复制的方式将数据同步到 slave vnode，这样确保了一份数据在多个物理节点上有拷贝。一个 vgroup 里虚拟节点个数就是数据的副本数。如果一个DB的副本数为N，系统必须有至少N个数据节点。副本数在创建DB时通过参数 replica 可以指定，缺省为1。使用 TDengine 的多副本特性，可以不再需要昂贵的磁盘阵列等存储设备，就可以获得同样的数据高可靠性。虚拟节点组由管理节点创建、管理，并且由管理节点分配一个系统唯一的ID，VGroup ID。如果两个虚拟节点的vnode group ID相同，说明他们属于同一个组，数据互为备份。虚拟节点组里虚拟节点的个数是可以动态改变的，容许只有一个，也就是没有数据复制。VGroup ID是永远不变的，即使一个虚拟节点组被删除，它的ID也不会被收回重复利用。 **虚拟节点组(VGroup):** 不同数据节点上的 vnode 可以组成一个虚拟节点组(vnode group)来保证系统的高可靠。虚拟节点组内采取master/slave的方式进行管理。写操作只能在 master vnode 上进行，系统采用异步复制的方式将数据同步到 slave vnode，这样确保了一份数据在多个物理节点上有拷贝。一个 vgroup 里虚拟节点个数就是数据的副本数。如果一个DB的副本数为N，系统必须有至少N个数据节点。副本数在创建DB时通过参数 replica 可以指定，缺省为1。使用 TDengine 的多副本特性，可以不再需要昂贵的磁盘阵列等存储设备，就可以获得同样的数据高可靠性。虚拟节点组由管理节点创建、管理，并且由管理节点分配一个系统唯一的ID，VGroup ID。如果两个虚拟节点的vnode group ID相同，说明他们属于同一个组，数据互为备份。虚拟节点组里虚拟节点的个数是可以动态改变的，容许只有一个，也就是没有数据复制。VGroup ID是永远不变的，即使一个虚拟节点组被删除，它的ID也不会被收回重复利用。
......
documentation20/cn/11.administrator/docs.md
浏览文件 @ 4ec4714e
...@@ -2,52 +2,52 @@ ...@@ -2,52 +2,52 @@
## <a class="anchor" id="planning"></a>容量规划 ## <a class="anchor" id="planning"></a>容量规划
使用TDengine来搭建一个物联网大数据平台，计算资源、存储资源需要根据业务场景进行规划。下面分别讨论系统运行所需要的内存、CPU以及硬盘空间。 使用 TDengine 来搭建一个物联网大数据平台，计算资源、存储资源需要根据业务场景进行规划。下面分别讨论系统运行所需要的内存、CPU 以及硬盘空间。
### 内存需求 ### 内存需求
每个DB可以创建固定数目的vgroup，默认与CPU核数相同，可通过maxVgroupsPerDb配置；vgroup中的每个副本会是一个vnode；每个vnode会占用固定大小的内存（大小与数据库的配置参数blocks和cache有关)；每个Table会占用与标签总长度有关的内存；此外，系统会有一些固定的内存开销。因此，每个DB需要的系统内存可通过如下公式计算： 每个 DB 可以创建固定数目的 vgroup，默认与 CPU 核数相同，可通过 maxVgroupsPerDb 配置；vgroup 中的每个副本会是一个 vnode；每个 vnode 会占用固定大小的内存（大小与数据库的配置参数 blocks 和 cache 有关)；每个 Table 会占用与标签总长度有关的内存；此外，系统会有一些固定的内存开销。因此，每个 DB 需要的系统内存可通过如下公式计算：
``` ```
Memory Size = maxVgroupsPerDb * (blocks * cache + 10Mb) + numOfTables * (tagSizePerTable + 0.5Kb) Memory Size = maxVgroupsPerDb * (blocks * cache + 10MB) + numOfTables * (tagSizePerTable + 0.5KB)
``` ```
示例：假设是4核机器，cache是缺省大小16M, blocks是缺省值6，假设有10万张表，标签总长度是256字节，则总的内存需求为：4\*(16\*6+10) + 100000\*(0.25+0.5)/1000 = 499M。 示例：假设是 4 核机器，cache 是缺省大小 16M, blocks 是缺省值 6，假设有 10 万张表，标签总长度是 256 字节，则总的内存需求为：4 \* (16 \* 6 + 10) + 100000 \* (0.25 + 0.5) / 1000 = 499M。
实际运行的系统往往会根据数据特点的不同，将数据存放在不同的DB里。因此做规划时，也需要考虑。 实际运行的系统往往会根据数据特点的不同，将数据存放在不同的 DB 里。因此做规划时，也需要考虑。
如果内存充裕，可以加大Blocks的配置，这样更多数据将保存在内存里，提高查询速度。 如果内存充裕，可以加大 Blocks 的配置，这样更多数据将保存在内存里，提高查询速度。
### CPU需求 ### CPU 需求
CPU的需求取决于如下两方面： CPU 的需求取决于如下两方面：
* __数据插入__ TDengine单核每秒能至少处理一万个插入请求。每个插入请求可以带多条记录，一次插入一条记录与插入10条记录，消耗的计算资源差别很小。因此每次插入，条数越大，插入效率越高。如果一个插入请求带200条以上记录，单核就能达到每秒插入100万条记录的速度。但对前端数据采集的要求越高，因为需要缓存记录，然后一批插入。 * __数据插入__ TDengine 单核每秒能至少处理一万个插入请求。每个插入请求可以带多条记录，一次插入一条记录与插入 10 条记录，消耗的计算资源差别很小。因此每次插入，条数越大，插入效率越高。如果一个插入请求带 200 条以上记录，单核就能达到每秒插入 100 万条记录的速度。但对前端数据采集的要求越高，因为需要缓存记录，然后一批插入。
* __查询需求__ TDengine提供高效的查询，但是每个场景的查询差异很大，查询频次变化也很大，难以给出客观数字。需要用户针对自己的场景，写一些查询语句，才能确定。 * __查询需求__ TDengine 提供高效的查询，但是每个场景的查询差异很大，查询频次变化也很大，难以给出客观数字。需要用户针对自己的场景，写一些查询语句，才能确定。
因此仅对数据插入而言，CPU是可以估算出来的，但查询所耗的计算资源无法估算。在实际运营过程中，不建议CPU使用率超过50%，超过后，需要增加新的节点，以获得更多计算资源。 因此仅对数据插入而言，CPU 是可以估算出来的，但查询所耗的计算资源无法估算。在实际运营过程中，不建议 CPU 使用率超过 50%，超过后，需要增加新的节点，以获得更多计算资源。
### 存储需求 ### 存储需求
TDengine相对于通用数据库，有超高的压缩比，在绝大多数场景下，TDengine的压缩比不会低于5倍，有的场合，压缩比可达到10倍以上，取决于实际场景的数据特征。压缩前的原始数据大小可通过如下方式计算： TDengine 相对于通用数据库，有超高的压缩比，在绝大多数场景下，TDengine 的压缩比不会低于 5 倍，有的场合，压缩比可达到 10 倍以上，取决于实际场景的数据特征。压缩前的原始数据大小可通过如下方式计算：
``` ```
Raw DataSize = numOfTables * rowSizePerTable * rowsPerTable Raw DataSize = numOfTables * rowSizePerTable * rowsPerTable
``` ```
示例：1000万台智能电表，每台电表每15分钟采集一次数据，每次采集的数据128字节，那么一年的原始数据量是：10000000\*128\*24\*60/15\*365 = 44.8512T。TDengine大概需要消耗44.851/5=8.97024T空间。 示例：1000 万台智能电表，每台电表每 15 分钟采集一次数据，每次采集的数据 128 字节，那么一年的原始数据量是：10000000 \* 128 \* 24 \* 60 / 15 \* 365 = 44.8512T。TDengine大概需要消耗 44.851 / 5 = 8.97024T 空间。
用户可以通过参数keep，设置数据在磁盘中的最大保存时长。为进一步减少存储成本，TDengine还提供多级存储，最冷的数据可以存放在最廉价的存储介质上，应用的访问不用做任何调整，只是读取速度降低了。 用户可以通过参数 keep，设置数据在磁盘中的最大保存时长。为进一步减少存储成本，TDengine 还提供多级存储，最冷的数据可以存放在最廉价的存储介质上，应用的访问不用做任何调整，只是读取速度降低了。
为提高速度，可以配置多块硬盘，这样可以并发写入或读取数据。需要提醒的是，TDengine采取多副本的方式提供数据的高可靠，因此不再需要采用昂贵的磁盘阵列。 为提高速度，可以配置多块硬盘，这样可以并发写入或读取数据。需要提醒的是，TDengine 采取多副本的方式提供数据的高可靠，因此不再需要采用昂贵的磁盘阵列。
### 物理机或虚拟机台数 ### 物理机或虚拟机台数
根据上面的内存、CPU、存储的预估，就可以知道整个系统需要多少核、多少内存、多少存储空间。如果数据副本数不为1，总需求量需要再乘以副本数。 根据上面的内存、CPU、存储的预估，就可以知道整个系统需要多少核、多少内存、多少存储空间。如果数据副本数不为 1，总需求量需要再乘以副本数。
因为TDengine具有很好的水平扩展能力，根据总量，再根据单个物理机或虚拟机的资源，就可以轻松决定需要购置多少台物理机或虚拟机了。 因为 TDengine 具有很好的水平扩展能力，根据总量，再根据单个物理机或虚拟机的资源，就可以轻松决定需要购置多少台物理机或虚拟机了。
**立即计算CPU、内存、存储，请参见：[资源估算方法](https://www.taosdata.com/config/config.html)** **立即计算 CPU、内存、存储，请参见：[资源估算方法](https://www.taosdata.com/config/config.html)**
## <a class="anchor" id="tolerance"></a>容错和灾备 ## <a class="anchor" id="tolerance"></a>容错和灾备
......
documentation20/cn/12.taos-sql/docs.md
浏览文件 @ 4ec4714e
...@@ -115,7 +115,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic ...@@ -115,7 +115,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic
```mysql ```mysql
ALTER DATABASE db_name QUORUM 2; ALTER DATABASE db_name QUORUM 2;
``` ```
QUORUM 参数是指数据写入成功所需要的确认数，取值范围 [1, 3]。对于异步复制，quorum 设为 1，具有 master 角色的虚拟节点自己确认即可。对于同步复制，需要至少大于等于 2。原则上，Quorum >= 1 并且 Quorum <= replica(副本数)，这个参数在启动一个同步模块实例时需要提供。 QUORUM 参数是指数据写入成功所需要的确认数，取值范围 [1, 2]。对于异步复制，quorum 设为 1，具有 master 角色的虚拟节点自己确认即可。对于同步复制，需要至少大于等于 2。原则上，Quorum >= 1 并且 Quorum <= replica(副本数)，这个参数在启动一个同步模块实例时需要提供。
```mysql ```mysql
ALTER DATABASE db_name BLOCKS 100; ALTER DATABASE db_name BLOCKS 100;
......
src/client/inc/tscUtil.h
浏览文件 @ 4ec4714e
...@@ -123,6 +123,7 @@ int32_t tscGetDataBlockFromList(SHashObj* pHashList, int64_t id, int32_t size, i ...@@ -123,6 +123,7 @@ int32_t tscGetDataBlockFromList(SHashObj* pHashList, int64_t id, int32_t size, i
bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo); bool tscIsPointInterpQuery(SQueryInfo* pQueryInfo);
bool tscIsTWAQuery(SQueryInfo* pQueryInfo); bool tscIsTWAQuery(SQueryInfo* pQueryInfo);
bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo); bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo);
bool tscGroupbyColumn(SQueryInfo* pQueryInfo);
bool tscNonOrderedProjectionQueryOnSTable(SQueryInfo *pQueryInfo, int32_t tableIndex); bool tscNonOrderedProjectionQueryOnSTable(SQueryInfo *pQueryInfo, int32_t tableIndex);
bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscOrderedProjectionQueryOnSTable(SQueryInfo* pQueryInfo, int32_t tableIndex);
...@@ -133,6 +134,7 @@ bool tscIsProjectionQuery(SQueryInfo* pQueryInfo); ...@@ -133,6 +134,7 @@ bool tscIsProjectionQuery(SQueryInfo* pQueryInfo);
bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex);
bool tscQueryTags(SQueryInfo* pQueryInfo); bool tscQueryTags(SQueryInfo* pQueryInfo);
bool tscMultiRoundQuery(SQueryInfo* pQueryInfo, int32_t tableIndex); bool tscMultiRoundQuery(SQueryInfo* pQueryInfo, int32_t tableIndex);
bool tscQueryBlockInfo(SQueryInfo* pQueryInfo);
SSqlExpr* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId, SSqlExpr* tscAddFuncInSelectClause(SQueryInfo* pQueryInfo, int32_t outputColIndex, int16_t functionId,
SColumnIndex* pIndex, SSchema* pColSchema, int16_t colType); SColumnIndex* pIndex, SSchema* pColSchema, int16_t colType);
...@@ -152,7 +154,6 @@ SInternalField* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_F ...@@ -152,7 +154,6 @@ SInternalField* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_F
SInternalField* tscFieldInfoGetInternalField(SFieldInfo* pFieldInfo, int32_t index); SInternalField* tscFieldInfoGetInternalField(SFieldInfo* pFieldInfo, int32_t index);
TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index); TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index);
void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo);
void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo); void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo);
int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index); int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index);
......
src/client/inc/tsclient.h
浏览文件 @ 4ec4714e
...@@ -198,9 +198,10 @@ typedef struct STableDataBlocks { ...@@ -198,9 +198,10 @@ typedef struct STableDataBlocks {
typedef struct SQueryInfo { typedef struct SQueryInfo {
int16_t command; // the command may be different for each subclause, so keep it seperately. int16_t command; // the command may be different for each subclause, so keep it seperately.
uint32_t type; // query/insert type uint32_t type; // query/insert type
STimeWindow window; // the whole query time window
STimeWindow window; // query time window SInterval interval; // tumble time window
SInterval interval; SSessionWindow sessionWindow; // session time window
SSqlGroupbyExpr groupbyExpr; // group by tags info SSqlGroupbyExpr groupbyExpr; // group by tags info
SArray * colList; // SArray<SColumn*> SArray * colList; // SArray<SColumn*>
...@@ -232,6 +233,7 @@ typedef struct SQueryInfo { ...@@ -232,6 +233,7 @@ typedef struct SQueryInfo {
typedef struct { typedef struct {
int command; int command;
uint8_t msgType; uint8_t msgType;
char reserve1[3]; // fix bus error on arm32
bool autoCreated; // create table if it is not existed during retrieve table meta in mnode bool autoCreated; // create table if it is not existed during retrieve table meta in mnode
union { union {
...@@ -244,8 +246,10 @@ typedef struct { ...@@ -244,8 +246,10 @@ typedef struct {
char * curSql; // current sql, resume position of sql after parsing paused char * curSql; // current sql, resume position of sql after parsing paused
int8_t parseFinished; int8_t parseFinished;
char reserve2[3]; // fix bus error on arm32
int16_t numOfCols; int16_t numOfCols;
char reserve3[2]; // fix bus error on arm32
uint32_t allocSize; uint32_t allocSize;
char * payload; char * payload;
int32_t payloadLen; int32_t payloadLen;
...@@ -255,7 +259,9 @@ typedef struct { ...@@ -255,7 +259,9 @@ typedef struct {
int32_t numOfParams; int32_t numOfParams;
int8_t dataSourceType; // load data from file or not int8_t dataSourceType; // load data from file or not
char reserve4[3]; // fix bus error on arm32
int8_t submitSchema; // submit block is built with table schema int8_t submitSchema; // submit block is built with table schema
char reserve5[3]; // fix bus error on arm32
STagData tagData; // NOTE: pTagData->data is used as a variant length array STagData tagData; // NOTE: pTagData->data is used as a variant length array
SName **pTableNameList; // all involved tableMeta list of current insert sql statement. SName **pTableNameList; // all involved tableMeta list of current insert sql statement.
...@@ -397,7 +403,6 @@ typedef struct SSqlStream { ...@@ -397,7 +403,6 @@ typedef struct SSqlStream {
void tscSetStreamDestTable(SSqlStream* pStream, const char* dstTable); void tscSetStreamDestTable(SSqlStream* pStream, const char* dstTable);
int tscAcquireRpc(const char *key, const char *user, const char *secret,void **pRpcObj); int tscAcquireRpc(const char *key, const char *user, const char *secret,void **pRpcObj);
void tscReleaseRpc(void *param); void tscReleaseRpc(void *param);
void tscInitMsgsFp(); void tscInitMsgsFp();
......
src/client/src/tscLocalMerge.c
浏览文件 @ 4ec4714e
...@@ -100,6 +100,10 @@ static void tscInitSqlContext(SSqlCmd *pCmd, SLocalMerger *pReducer, tOrderDescr ...@@ -100,6 +100,10 @@ static void tscInitSqlContext(SSqlCmd *pCmd, SLocalMerger *pReducer, tOrderDescr
} else if (functionId == TSDB_FUNC_APERCT) { } else if (functionId == TSDB_FUNC_APERCT) {
pCtx->param[0].i64 = pExpr->param[0].i64; pCtx->param[0].i64 = pExpr->param[0].i64;
pCtx->param[0].nType = pExpr->param[0].nType; pCtx->param[0].nType = pExpr->param[0].nType;
} else if (functionId == TSDB_FUNC_BLKINFO) {
pCtx->param[0].i64 = pExpr->param[0].i64;
pCtx->param[0].nType = pExpr->param[0].nType;
pCtx->numOfParams = 1;
} }
pCtx->interBufBytes = pExpr->interBytes; pCtx->interBufBytes = pExpr->interBytes;
...@@ -951,10 +955,10 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp ...@@ -951,10 +955,10 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp
// todo extract function // todo extract function
int64_t actualETime = (pQueryInfo->order.order == TSDB_ORDER_ASC)? pQueryInfo->window.ekey: pQueryInfo->window.skey; int64_t actualETime = (pQueryInfo->order.order == TSDB_ORDER_ASC)? pQueryInfo->window.ekey: pQueryInfo->window.skey;
tFilePage **pResPages = malloc(POINTER_BYTES * pQueryInfo->fieldsInfo.numOfOutput); void** pResPages = malloc(POINTER_BYTES * pQueryInfo->fieldsInfo.numOfOutput);
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) { for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i); TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i);
pResPages[i] = calloc(1, sizeof(tFilePage) + pField->bytes * pLocalMerge->resColModel->capacity); pResPages[i] = calloc(1, pField->bytes * pLocalMerge->resColModel->capacity);
} }
while (1) { while (1) {
...@@ -966,7 +970,7 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp ...@@ -966,7 +970,7 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp
if (pQueryInfo->limit.offset > 0) { if (pQueryInfo->limit.offset > 0) {
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) { for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i); TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i);
memmove(pResPages[i]->data, pResPages[i]->data + pField->bytes * pQueryInfo->limit.offset, memmove(pResPages[i], ((char*)pResPages[i]) + pField->bytes * pQueryInfo->limit.offset,
(size_t)(newRows * pField->bytes)); (size_t)(newRows * pField->bytes));
} }
} }
...@@ -1010,7 +1014,7 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp ...@@ -1010,7 +1014,7 @@ static void doFillResult(SSqlObj *pSql, SLocalMerger *pLocalMerge, bool doneOutp
int32_t offset = 0; int32_t offset = 0;
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) { for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i); TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, i);
memcpy(pRes->data + offset * pRes->numOfRows, pResPages[i]->data, (size_t)(pField->bytes * pRes->numOfRows)); memcpy(pRes->data + offset * pRes->numOfRows, pResPages[i], (size_t)(pField->bytes * pRes->numOfRows));
offset += pField->bytes; offset += pField->bytes;
} }
......
src/client/src/tscParseInsert.c
浏览文件 @ 4ec4714e
...@@ -307,7 +307,8 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload, ...@@ -307,7 +307,8 @@ int32_t tsParseOneColumnData(SSchema *pSchema, SStrToken *pToken, char *payload,
return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z); return tscInvalidSQLErrMsg(msg, "illegal float data", pToken->z);
} }
*((float *)payload) = (float)dv; // *((float *)payload) = (float)dv;
SET_FLOAT_VAL(payload, dv);
} }
break; break;
......
src/client/src/tscSQLParser.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/client/src/tscServer.c
浏览文件 @ 4ec4714e
...@@ -497,8 +497,6 @@ int tscProcessSql(SSqlObj *pSql) { ...@@ -497,8 +497,6 @@ int tscProcessSql(SSqlObj *pSql) {
return pSql->res.code; return pSql->res.code;
} }
} else if (pCmd->command >= TSDB_SQL_LOCAL) { } else if (pCmd->command >= TSDB_SQL_LOCAL) {
//pSql->epSet = tscMgmtEpSet;
// } else { // local handler
return (*tscProcessMsgRsp[pCmd->command])(pSql); return (*tscProcessMsgRsp[pCmd->command])(pSql);
} }
...@@ -645,7 +643,6 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char ...@@ -645,7 +643,6 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char
} }
pSql->epSet.inUse = rand()%pSql->epSet.numOfEps; pSql->epSet.inUse = rand()%pSql->epSet.numOfEps;
pQueryMsg->head.vgId = htonl(vgId); pQueryMsg->head.vgId = htonl(vgId);
STableIdInfo *pTableIdInfo = (STableIdInfo *)pMsg; STableIdInfo *pTableIdInfo = (STableIdInfo *)pMsg;
...@@ -660,8 +657,6 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char ...@@ -660,8 +657,6 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char
int32_t numOfVgroups = (int32_t)taosArrayGetSize(pTableMetaInfo->pVgroupTables); int32_t numOfVgroups = (int32_t)taosArrayGetSize(pTableMetaInfo->pVgroupTables);
assert(index >= 0 && index < numOfVgroups); assert(index >= 0 && index < numOfVgroups);
tscDebug("%p query on stable, vgIndex:%d, numOfVgroups:%d", pSql, index, numOfVgroups);
SVgroupTableInfo* pTableIdList = taosArrayGet(pTableMetaInfo->pVgroupTables, index); SVgroupTableInfo* pTableIdList = taosArrayGet(pTableMetaInfo->pVgroupTables, index);
// set the vgroup info // set the vgroup info
...@@ -670,7 +665,10 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char ...@@ -670,7 +665,10 @@ static char *doSerializeTableInfo(SQueryTableMsg* pQueryMsg, SSqlObj *pSql, char
int32_t numOfTables = (int32_t)taosArrayGetSize(pTableIdList->itemList); int32_t numOfTables = (int32_t)taosArrayGetSize(pTableIdList->itemList);
pQueryMsg->numOfTables = htonl(numOfTables); // set the number of tables pQueryMsg->numOfTables = htonl(numOfTables); // set the number of tables
tscDebug("%p query on stable, vgId:%d, numOfTables:%d, vgIndex:%d, numOfVgroups:%d", pSql,
pTableIdList->vgInfo.vgId, numOfTables, index, numOfVgroups);
// serialize each table id info // serialize each table id info
for(int32_t i = 0; i < numOfTables; ++i) { for(int32_t i = 0; i < numOfTables; ++i) {
STableIdInfo* pItem = taosArrayGet(pTableIdList->itemList, i); STableIdInfo* pItem = taosArrayGet(pTableIdList->itemList, i);
...@@ -705,7 +703,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -705,7 +703,7 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
STableMeta * pTableMeta = pTableMetaInfo->pTableMeta; STableMeta * pTableMeta = pTableMetaInfo->pTableMeta;
size_t numOfSrcCols = taosArrayGetSize(pQueryInfo->colList); size_t numOfSrcCols = taosArrayGetSize(pQueryInfo->colList);
if (numOfSrcCols <= 0 && !tscQueryTags(pQueryInfo)) { if (numOfSrcCols <= 0 && !tscQueryTags(pQueryInfo) && !tscQueryBlockInfo(pQueryInfo)) {
tscError("%p illegal value of numOfCols in query msg: %" PRIu64 ", table cols:%d", pSql, (uint64_t)numOfSrcCols, tscError("%p illegal value of numOfCols in query msg: %" PRIu64 ", table cols:%d", pSql, (uint64_t)numOfSrcCols,
tscGetNumOfColumns(pTableMeta)); tscGetNumOfColumns(pTableMeta));
...@@ -756,6 +754,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -756,6 +754,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pQueryMsg->vgroupLimit = htobe64(pQueryInfo->vgroupLimit); pQueryMsg->vgroupLimit = htobe64(pQueryInfo->vgroupLimit);
pQueryMsg->sqlstrLen = htonl(sqlLen); pQueryMsg->sqlstrLen = htonl(sqlLen);
pQueryMsg->prevResultLen = htonl(pQueryInfo->bufLen); pQueryMsg->prevResultLen = htonl(pQueryInfo->bufLen);
pQueryMsg->sw.gap = htobe64(pQueryInfo->sessionWindow.gap);
pQueryMsg->sw.primaryColId = htonl(PRIMARYKEY_TIMESTAMP_COL_INDEX);
size_t numOfOutput = tscSqlExprNumOfExprs(pQueryInfo); size_t numOfOutput = tscSqlExprNumOfExprs(pQueryInfo);
pQueryMsg->numOfOutput = htons((int16_t)numOfOutput); // this is the stage one output column number pQueryMsg->numOfOutput = htons((int16_t)numOfOutput); // this is the stage one output column number
...@@ -835,13 +835,31 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -835,13 +835,31 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pSqlFuncExpr->colInfo.colIndex = htons(pExpr->colInfo.colIndex); pSqlFuncExpr->colInfo.colIndex = htons(pExpr->colInfo.colIndex);
pSqlFuncExpr->colInfo.flag = htons(pExpr->colInfo.flag); pSqlFuncExpr->colInfo.flag = htons(pExpr->colInfo.flag);
if (TSDB_COL_IS_UD_COL(pExpr->colInfo.flag)) {
pSqlFuncExpr->colType = htons(pExpr->resType);
pSqlFuncExpr->colBytes = htons(pExpr->resBytes);
} else if (pExpr->colInfo.colId == TSDB_TBNAME_COLUMN_INDEX) {
SSchema *s = tGetTbnameColumnSchema();
pSqlFuncExpr->colType = htons(s->type);
pSqlFuncExpr->colBytes = htons(s->bytes);
} else if (pExpr->colInfo.colId == TSDB_BLOCK_DIST_COLUMN_INDEX) {
SSchema s = tGetBlockDistColumnSchema();
pSqlFuncExpr->colType = htons(s.type);
pSqlFuncExpr->colBytes = htons(s.bytes);
} else {
SSchema* s = tscGetColumnSchemaById(pTableMeta, pExpr->colInfo.colId);
pSqlFuncExpr->colType = htons(s->type);
pSqlFuncExpr->colBytes = htons(s->bytes);
}
pSqlFuncExpr->functionId = htons(pExpr->functionId); pSqlFuncExpr->functionId = htons(pExpr->functionId);
pSqlFuncExpr->numOfParams = htons(pExpr->numOfParams); pSqlFuncExpr->numOfParams = htons(pExpr->numOfParams);
pSqlFuncExpr->resColId = htons(pExpr->resColId); pSqlFuncExpr->resColId = htons(pExpr->resColId);
pMsg += sizeof(SSqlFuncMsg); pMsg += sizeof(SSqlFuncMsg);
for (int32_t j = 0; j < pExpr->numOfParams; ++j) { for (int32_t j = 0; j < pExpr->numOfParams; ++j) { // todo add log
// todo add log
pSqlFuncExpr->arg[j].argType = htons((uint16_t)pExpr->param[j].nType); pSqlFuncExpr->arg[j].argType = htons((uint16_t)pExpr->param[j].nType);
pSqlFuncExpr->arg[j].argBytes = htons(pExpr->param[j].nLen); pSqlFuncExpr->arg[j].argBytes = htons(pExpr->param[j].nLen);
...@@ -866,6 +884,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -866,6 +884,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
for (int32_t i = 0; i < output; ++i) { for (int32_t i = 0; i < output; ++i) {
SInternalField* pField = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, i); SInternalField* pField = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, i);
SSqlExpr *pExpr = pField->pSqlExpr; SSqlExpr *pExpr = pField->pSqlExpr;
// this should be switched to projection query
if (pExpr != NULL) { if (pExpr != NULL) {
// the queried table has been removed and a new table with the same name has already been created already // the queried table has been removed and a new table with the same name has already been created already
// return error msg // return error msg
...@@ -879,33 +899,31 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -879,33 +899,31 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
return TSDB_CODE_TSC_INVALID_SQL; return TSDB_CODE_TSC_INVALID_SQL;
} }
pSqlFuncExpr1->colInfo.colId = htons(pExpr->colInfo.colId); pSqlFuncExpr1->numOfParams = 0; // no params for projection query
pSqlFuncExpr1->colInfo.colIndex = htons(pExpr->colInfo.colIndex); pSqlFuncExpr1->functionId = htons(TSDB_FUNC_PRJ);
pSqlFuncExpr1->colInfo.flag = htons(pExpr->colInfo.flag); pSqlFuncExpr1->colInfo.colId = htons(pExpr->resColId);
pSqlFuncExpr1->colInfo.flag = htons(TSDB_COL_NORMAL);
pSqlFuncExpr1->functionId = htons(pExpr->functionId);
pSqlFuncExpr1->numOfParams = htons(pExpr->numOfParams); bool assign = false;
pMsg += sizeof(SSqlFuncMsg); for (int32_t f = 0; f < tscSqlExprNumOfExprs(pQueryInfo); ++f) {
SSqlExpr *pe = tscSqlExprGet(pQueryInfo, f);
for (int32_t j = 0; j < pExpr->numOfParams; ++j) { if (pe == pExpr) {
// todo add log pSqlFuncExpr1->colInfo.colIndex = htons(f);
pSqlFuncExpr1->arg[j].argType = htons((uint16_t)pExpr->param[j].nType); pSqlFuncExpr1->colType = htons(pe->resType);
pSqlFuncExpr1->arg[j].argBytes = htons(pExpr->param[j].nLen); pSqlFuncExpr1->colBytes = htons(pe->resBytes);
assign = true;
if (pExpr->param[j].nType == TSDB_DATA_TYPE_BINARY) { break;
memcpy(pMsg, pExpr->param[j].pz, pExpr->param[j].nLen);
pMsg += pExpr->param[j].nLen;
} else {
pSqlFuncExpr1->arg[j].argValue.i64 = htobe64(pExpr->param[j].i64);
} }
} }
assert(assign);
pMsg += sizeof(SSqlFuncMsg);
pSqlFuncExpr1 = (SSqlFuncMsg *)pMsg; pSqlFuncExpr1 = (SSqlFuncMsg *)pMsg;
} else { } else {
assert(pField->pArithExprInfo != NULL); assert(pField->pArithExprInfo != NULL);
SExprInfo* pExprInfo = pField->pArithExprInfo; SExprInfo* pExprInfo = pField->pArithExprInfo;
pSqlFuncExpr1->colInfo.colId = htons(pExprInfo->base.colInfo.colId); pSqlFuncExpr1->colInfo.colId = htons(pExprInfo->base.colInfo.colId);
pSqlFuncExpr1->functionId = htons(pExprInfo->base.functionId); pSqlFuncExpr1->functionId = htons(pExprInfo->base.functionId);
pSqlFuncExpr1->numOfParams = htons(pExprInfo->base.numOfParams); pSqlFuncExpr1->numOfParams = htons(pExprInfo->base.numOfParams);
pMsg += sizeof(SSqlFuncMsg); pMsg += sizeof(SSqlFuncMsg);
...@@ -1055,7 +1073,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -1055,7 +1073,8 @@ int tscBuildQueryMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
int32_t tscBuildCreateDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { int32_t tscBuildCreateDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
pCmd->payloadLen = sizeof(SCreateDbMsg); pCmd->payloadLen = sizeof(SCreateDbMsg);
pCmd->msgType = TSDB_MSG_TYPE_CM_CREATE_DB;
pCmd->msgType = (pInfo->pMiscInfo->dbOpt.dbType == TSDB_DB_TYPE_DEFAULT) ? TSDB_MSG_TYPE_CM_CREATE_DB : TSDB_MSG_TYPE_CM_CREATE_TP;
SCreateDbMsg *pCreateDbMsg = (SCreateDbMsg *)pCmd->payload; SCreateDbMsg *pCreateDbMsg = (SCreateDbMsg *)pCmd->payload;
...@@ -1187,7 +1206,7 @@ int32_t tscBuildDropDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { ...@@ -1187,7 +1206,7 @@ int32_t tscBuildDropDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
pDropDbMsg->ignoreNotExists = pInfo->pMiscInfo->existsCheck ? 1 : 0; pDropDbMsg->ignoreNotExists = pInfo->pMiscInfo->existsCheck ? 1 : 0;
pCmd->msgType = TSDB_MSG_TYPE_CM_DROP_DB; pCmd->msgType = (pInfo->pMiscInfo->dbType == TSDB_DB_TYPE_DEFAULT) ? TSDB_MSG_TYPE_CM_DROP_DB : TSDB_MSG_TYPE_CM_DROP_TP;
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
...@@ -1514,9 +1533,11 @@ int tscBuildUpdateTagMsg(SSqlObj* pSql, SSqlInfo *pInfo) { ...@@ -1514,9 +1533,11 @@ int tscBuildUpdateTagMsg(SSqlObj* pSql, SSqlInfo *pInfo) {
int tscAlterDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) { int tscAlterDbMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
pCmd->payloadLen = sizeof(SAlterDbMsg); pCmd->payloadLen = sizeof(SAlterDbMsg);
pCmd->msgType = TSDB_MSG_TYPE_CM_ALTER_DB; pCmd->msgType = (pInfo->pMiscInfo->dbOpt.dbType == TSDB_DB_TYPE_DEFAULT) ? TSDB_MSG_TYPE_CM_ALTER_DB : TSDB_MSG_TYPE_CM_ALTER_TP;
SAlterDbMsg *pAlterDbMsg = (SAlterDbMsg* )pCmd->payload; SAlterDbMsg *pAlterDbMsg = (SAlterDbMsg* )pCmd->payload;
pAlterDbMsg->dbType = -1;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
tNameExtractFullName(&pTableMetaInfo->name, pAlterDbMsg->db); tNameExtractFullName(&pTableMetaInfo->name, pAlterDbMsg->db);
......
src/client/src/tscSub.c
浏览文件 @ 4ec4714e
...@@ -503,9 +503,19 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) { ...@@ -503,9 +503,19 @@ TAOS_RES *taos_consume(TAOS_SUB *tsub) {
SSqlCmd *pCmd = &pSql->cmd; SSqlCmd *pCmd = &pSql->cmd;
STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0); STableMetaInfo *pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, 0); SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
if (taosArrayGetSize(pSub->progress) > 0) { // fix crash in single tabel subscription if (taosArrayGetSize(pSub->progress) > 0) { // fix crash in single table subscription
pQueryInfo->window.skey = ((SSubscriptionProgress*)taosArrayGet(pSub->progress, 0))->key;
tscDebug("subscribe:%s set subscribe skey:%"PRId64, pSub->topic, pQueryInfo->window.skey); size_t size = taosArrayGetSize(pSub->progress);
TSKEY s = INT64_MAX;
for(int32_t i = 0; i < size; ++i) {
TSKEY k = ((SSubscriptionProgress*)taosArrayGet(pSub->progress, i))->key;
if (s > k) {
s = k;
}
}
pQueryInfo->window.skey = s;
tscDebug("subscribe:%s set next round subscribe skey:%"PRId64, pSub->topic, pQueryInfo->window.skey);
} }
if (pSub->pTimer == NULL) { if (pSub->pTimer == NULL) {
......
src/client/src/tscSubquery.c
浏览文件 @ 4ec4714e
...@@ -74,14 +74,14 @@ static bool allSubqueryDone(SSqlObj *pParentSql) { ...@@ -74,14 +74,14 @@ static bool allSubqueryDone(SSqlObj *pParentSql) {
SSubqueryState *subState = &pParentSql->subState; SSubqueryState *subState = &pParentSql->subState;
//lock in caller //lock in caller
tscDebug("%p total subqueries: %d", pParentSql, subState->numOfSub);
for (int i = 0; i < subState->numOfSub; i++) { for (int i = 0; i < subState->numOfSub; i++) {
if (0 == subState->states[i]) { if (0 == subState->states[i]) {
tscDebug("%p subquery:%p,%d is NOT finished, total:%d", pParentSql, pParentSql->pSubs[i], i, subState->numOfSub); tscDebug("%p subquery:%p, index: %d NOT finished, abort query completion check", pParentSql, pParentSql->pSubs[i], i);
done = false; done = false;
break; break;
} else { } else {
tscDebug("%p subquery:%p,%d is finished, total:%d", pParentSql, pParentSql->pSubs[i], i, subState->numOfSub); tscDebug("%p subquery:%p, index: %d finished", pParentSql, pParentSql->pSubs[i], i);
} }
} }
...@@ -453,7 +453,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { ...@@ -453,7 +453,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
pSubQueryInfo->tsBuf = NULL; pSubQueryInfo->tsBuf = NULL;
// free result for async object will also free sqlObj // free result for async object will also free sqlObj
assert(tscSqlExprNumOfExprs(pSubQueryInfo) == 1); // ts_comp query only requires one resutl columns assert(tscSqlExprNumOfExprs(pSubQueryInfo) == 1); // ts_comp query only requires one result columns
taos_free_result(pPrevSub); taos_free_result(pPrevSub);
SSqlObj *pNew = createSubqueryObj(pSql, (int16_t) i, tscJoinQueryCallback, pSupporter, TSDB_SQL_SELECT, NULL); SSqlObj *pNew = createSubqueryObj(pSql, (int16_t) i, tscJoinQueryCallback, pSupporter, TSDB_SQL_SELECT, NULL);
...@@ -507,6 +507,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) { ...@@ -507,6 +507,7 @@ static int32_t tscLaunchRealSubqueries(SSqlObj* pSql) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, 0); SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, 0);
int16_t funcId = pExpr->functionId; int16_t funcId = pExpr->functionId;
// add the invisible timestamp column
if ((pExpr->colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX) || if ((pExpr->colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX) ||
(funcId != TSDB_FUNC_TS && funcId != TSDB_FUNC_TS_DUMMY && funcId != TSDB_FUNC_PRJ)) { (funcId != TSDB_FUNC_TS && funcId != TSDB_FUNC_TS_DUMMY && funcId != TSDB_FUNC_PRJ)) {
...@@ -847,6 +848,8 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -847,6 +848,8 @@ static void tidTagRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
SSqlRes* pRes = &pSql->res; SSqlRes* pRes = &pSql->res;
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex); SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
// todo, the type may not include TSDB_QUERY_TYPE_TAG_FILTER_QUERY
assert(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY)); assert(TSDB_QUERY_HAS_TYPE(pQueryInfo->type, TSDB_QUERY_TYPE_TAG_FILTER_QUERY));
if (pParentSql->res.code != TSDB_CODE_SUCCESS) { if (pParentSql->res.code != TSDB_CODE_SUCCESS) {
...@@ -1059,7 +1062,6 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow ...@@ -1059,7 +1062,6 @@ static void tsCompRetrieveCallback(void* param, TAOS_RES* tres, int32_t numOfRow
tscError("%p invalid ts comp file from vnode, abort subquery, file size:%d", pSql, numOfRows); tscError("%p invalid ts comp file from vnode, abort subquery, file size:%d", pSql, numOfRows);
pParentSql->res.code = TAOS_SYSTEM_ERROR(errno); pParentSql->res.code = TAOS_SYSTEM_ERROR(errno);
if (quitAllSubquery(pSql, pParentSql, pSupporter)){ if (quitAllSubquery(pSql, pParentSql, pSupporter)){
return; return;
} }
...@@ -1880,6 +1882,13 @@ void doAppendData(SInterResult* pInterResult, TAOS_ROW row, int32_t numOfCols, S ...@@ -1880,6 +1882,13 @@ void doAppendData(SInterResult* pInterResult, TAOS_ROW row, int32_t numOfCols, S
} }
} }
if (p && taosArrayGetSize(p) > 0) {
SResPair *l = taosArrayGetLast(p);
if (l->key == key && key == INT64_MIN) {
continue;
}
}
//append a new column //append a new column
if (p == NULL) { if (p == NULL) {
SStddevInterResult t = {.colId = id, .pResult = taosArrayInit(10, sizeof(SResPair)),}; SStddevInterResult t = {.colId = id, .pResult = taosArrayInit(10, sizeof(SResPair)),};
...@@ -1941,7 +1950,11 @@ void tscFirstRoundRetrieveCallback(void* param, TAOS_RES* tres, int numOfRows) { ...@@ -1941,7 +1950,11 @@ void tscFirstRoundRetrieveCallback(void* param, TAOS_RES* tres, int numOfRows) {
// tag or group by column // tag or group by column
if (TSDB_COL_IS_TAG(pExpr->colInfo.flag) || pExpr->functionId == TSDB_FUNC_PRJ) { if (TSDB_COL_IS_TAG(pExpr->colInfo.flag) || pExpr->functionId == TSDB_FUNC_PRJ) {
memcpy(p + offset, row[i], length[i]); if (row[i] == NULL) {
setNull(p + offset, pExpr->resType, pExpr->resBytes);
} else {
memcpy(p + offset, row[i], length[i]);
}
offset += pExpr->resBytes; offset += pExpr->resBytes;
} }
} }
...@@ -2639,12 +2652,17 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo ...@@ -2639,12 +2652,17 @@ static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsuppo
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pNew->cmd, 0); SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(&pNew->cmd, 0);
pQueryInfo->type |= TSDB_QUERY_TYPE_STABLE_SUBQUERY; pQueryInfo->type |= TSDB_QUERY_TYPE_STABLE_SUBQUERY;
// clear the limit/offset info, since it should not be sent to vnode to be executed.
pQueryInfo->limit.limit = -1;
pQueryInfo->limit.offset = 0;
assert(pQueryInfo->numOfTables == 1 && pNew->cmd.numOfClause == 1 && trsupport->subqueryIndex < pSql->subState.numOfSub); assert(pQueryInfo->numOfTables == 1 && pNew->cmd.numOfClause == 1 && trsupport->subqueryIndex < pSql->subState.numOfSub);
// launch subquery for each vnode, so the subquery index equals to the vgroupIndex. // launch subquery for each vnode, so the subquery index equals to the vgroupIndex.
STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, table_index); STableMetaInfo *pTableMetaInfo = tscGetMetaInfo(pQueryInfo, table_index);
pTableMetaInfo->vgroupIndex = trsupport->subqueryIndex; pTableMetaInfo->vgroupIndex = trsupport->subqueryIndex;
pSql->pSubs[trsupport->subqueryIndex] = pNew; pSql->pSubs[trsupport->subqueryIndex] = pNew;
} }
...@@ -3098,30 +3116,6 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) { ...@@ -3098,30 +3116,6 @@ void tscBuildResFromSubqueries(SSqlObj *pSql) {
} }
} }
static UNUSED_FUNC void transferNcharData(SSqlObj *pSql, int32_t columnIndex, TAOS_FIELD *pField) {
SSqlRes *pRes = &pSql->res;
if (pRes->tsrow[columnIndex] != NULL && pField->type == TSDB_DATA_TYPE_NCHAR) {
// convert unicode to native code in a temporary buffer extra one byte for terminated symbol
if (pRes->buffer[columnIndex] == NULL) {
pRes->buffer[columnIndex] = malloc(pField->bytes + TSDB_NCHAR_SIZE);
}
/* string terminated char for binary data*/
memset(pRes->buffer[columnIndex], 0, pField->bytes + TSDB_NCHAR_SIZE);
int32_t length = taosUcs4ToMbs(pRes->tsrow[columnIndex], pRes->length[columnIndex], pRes->buffer[columnIndex]);
if ( length >= 0 ) {
pRes->tsrow[columnIndex] = (unsigned char*)pRes->buffer[columnIndex];
pRes->length[columnIndex] = length;
} else {
tscError("%p charset:%s to %s. val:%s convert failed.", pSql, DEFAULT_UNICODE_ENCODEC, tsCharset, (char*)pRes->tsrow[columnIndex]);
pRes->tsrow[columnIndex] = NULL;
pRes->length[columnIndex] = 0;
}
}
}
char *getArithmeticInputSrc(void *param, const char *name, int32_t colId) { char *getArithmeticInputSrc(void *param, const char *name, int32_t colId) {
SArithmeticSupport *pSupport = (SArithmeticSupport *) param; SArithmeticSupport *pSupport = (SArithmeticSupport *) param;
......
src/client/src/tscUtil.c
浏览文件 @ 4ec4714e
...@@ -97,6 +97,22 @@ bool tscQueryTags(SQueryInfo* pQueryInfo) { ...@@ -97,6 +97,22 @@ bool tscQueryTags(SQueryInfo* pQueryInfo) {
return true; return true;
} }
bool tscQueryBlockInfo(SQueryInfo* pQueryInfo) {
int32_t numOfCols = (int32_t) tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < numOfCols; ++i) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
int32_t functId = pExpr->functionId;
// "select count(tbname)" query
if (functId == TSDB_FUNC_BLKINFO) {
return true;
}
}
return false;
}
bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex) { bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex) {
if (pQueryInfo == NULL) { if (pQueryInfo == NULL) {
return false; return false;
...@@ -223,6 +239,21 @@ bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo) { ...@@ -223,6 +239,21 @@ bool tscIsSecondStageQuery(SQueryInfo* pQueryInfo) {
return false; return false;
} }
bool tscGroupbyColumn(SQueryInfo* pQueryInfo) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, 0);
int32_t numOfCols = tscGetNumOfColumns(pTableMetaInfo->pTableMeta);
SSqlGroupbyExpr* pGroupbyExpr = &pQueryInfo->groupbyExpr;
for (int32_t k = 0; k < pGroupbyExpr->numOfGroupCols; ++k) {
SColIndex* pIndex = taosArrayGet(pGroupbyExpr->columnInfo, k);
if (!TSDB_COL_IS_TAG(pIndex->flag) && pIndex->colIndex < numOfCols) { // group by normal columns
return true;
}
}
return false;
}
bool tscIsTWAQuery(SQueryInfo* pQueryInfo) { bool tscIsTWAQuery(SQueryInfo* pQueryInfo) {
size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo); size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < numOfExprs; ++i) { for (int32_t i = 0; i < numOfExprs; ++i) {
...@@ -1722,10 +1753,15 @@ void tscInitQueryInfo(SQueryInfo* pQueryInfo) { ...@@ -1722,10 +1753,15 @@ void tscInitQueryInfo(SQueryInfo* pQueryInfo) {
pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField)); pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField));
assert(pQueryInfo->exprList == NULL); assert(pQueryInfo->exprList == NULL);
pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES);
pQueryInfo->colList = taosArrayInit(4, POINTER_BYTES); pQueryInfo->colList = taosArrayInit(4, POINTER_BYTES);
pQueryInfo->udColumnId = TSDB_UD_COLUMN_INDEX; pQueryInfo->udColumnId = TSDB_UD_COLUMN_INDEX;
pQueryInfo->resColumnId= -1000; pQueryInfo->resColumnId = -1000;
pQueryInfo->limit.limit = -1;
pQueryInfo->limit.offset = 0;
pQueryInfo->slimit.limit = -1;
pQueryInfo->slimit.offset = 0;
} }
int32_t tscAddSubqueryInfo(SSqlCmd* pCmd) { int32_t tscAddSubqueryInfo(SSqlCmd* pCmd) {
......
src/common/inc/tdataformat.h
浏览文件 @ 4ec4714e
...@@ -283,12 +283,37 @@ typedef struct { ...@@ -283,12 +283,37 @@ typedef struct {
#define keyCol(pCols) (&((pCols)->cols[0])) // Key column #define keyCol(pCols) (&((pCols)->cols[0])) // Key column
#define dataColsTKeyAt(pCols, idx) ((TKEY *)(keyCol(pCols)->pData))[(idx)] #define dataColsTKeyAt(pCols, idx) ((TKEY *)(keyCol(pCols)->pData))[(idx)]
#define dataColsKeyAt(pCols, idx) tdGetKey(dataColsTKeyAt(pCols, idx)) #define dataColsKeyAt(pCols, idx) tdGetKey(dataColsTKeyAt(pCols, idx))
#define dataColsTKeyFirst(pCols) (((pCols)->numOfRows == 0) ? TKEY_INVALID : dataColsTKeyAt(pCols, 0)) static FORCE_INLINE TKEY dataColsTKeyFirst(SDataCols *pCols) {
#define dataColsKeyFirst(pCols) (((pCols)->numOfRows == 0) ? TSDB_DATA_TIMESTAMP_NULL : dataColsKeyAt(pCols, 0)) if (pCols->numOfRows) {
#define dataColsTKeyLast(pCols) \ return dataColsTKeyAt(pCols, 0);
(((pCols)->numOfRows == 0) ? TKEY_INVALID : dataColsTKeyAt(pCols, (pCols)->numOfRows - 1)) } else {
#define dataColsKeyLast(pCols) \ return TKEY_INVALID;
(((pCols)->numOfRows == 0) ? TSDB_DATA_TIMESTAMP_NULL : dataColsKeyAt(pCols, (pCols)->numOfRows - 1)) }
}
static FORCE_INLINE TSKEY dataColsKeyFirst(SDataCols *pCols) {
if (pCols->numOfRows) {
return dataColsKeyAt(pCols, 0);
} else {
return TSDB_DATA_TIMESTAMP_NULL;
}
}
static FORCE_INLINE TKEY dataColsTKeyLast(SDataCols *pCols) {
if (pCols->numOfRows) {
return dataColsTKeyAt(pCols, pCols->numOfRows - 1);
} else {
return TKEY_INVALID;
}
}
static FORCE_INLINE TSKEY dataColsKeyLast(SDataCols *pCols) {
if (pCols->numOfRows) {
return dataColsKeyAt(pCols, pCols->numOfRows - 1);
} else {
return TSDB_DATA_TIMESTAMP_NULL;
}
}
SDataCols *tdNewDataCols(int maxRowSize, int maxCols, int maxRows); SDataCols *tdNewDataCols(int maxRowSize, int maxCols, int maxRows);
void tdResetDataCols(SDataCols *pCols); void tdResetDataCols(SDataCols *pCols);
......
src/common/inc/tglobal.h
浏览文件 @ 4ec4714e
...@@ -95,6 +95,7 @@ extern int8_t tsCompression; ...@@ -95,6 +95,7 @@ extern int8_t tsCompression;
extern int8_t tsWAL; extern int8_t tsWAL;
extern int32_t tsFsyncPeriod; extern int32_t tsFsyncPeriod;
extern int32_t tsReplications; extern int32_t tsReplications;
extern int16_t tsPartitons;
extern int32_t tsQuorum; extern int32_t tsQuorum;
extern int8_t tsUpdate; extern int8_t tsUpdate;
extern int8_t tsCacheLastRow; extern int8_t tsCacheLastRow;
......
src/common/inc/tname.h
浏览文件 @ 4ec4714e
...@@ -33,7 +33,7 @@ typedef struct SDataStatis { ...@@ -33,7 +33,7 @@ typedef struct SDataStatis {
typedef struct SColumnInfoData { typedef struct SColumnInfoData {
SColumnInfo info; SColumnInfo info;
void* pData; // the corresponding block data in memory char* pData; // the corresponding block data in memory
} SColumnInfoData; } SColumnInfoData;
typedef struct SResPair { typedef struct SResPair {
......
src/common/src/tglobal.c
浏览文件 @ 4ec4714e
...@@ -126,8 +126,9 @@ int8_t tsWAL = TSDB_DEFAULT_WAL_LEVEL; ...@@ -126,8 +126,9 @@ int8_t tsWAL = TSDB_DEFAULT_WAL_LEVEL;
int32_t tsFsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD; int32_t tsFsyncPeriod = TSDB_DEFAULT_FSYNC_PERIOD;
int32_t tsReplications = TSDB_DEFAULT_DB_REPLICA_OPTION; int32_t tsReplications = TSDB_DEFAULT_DB_REPLICA_OPTION;
int32_t tsQuorum = TSDB_DEFAULT_DB_QUORUM_OPTION; int32_t tsQuorum = TSDB_DEFAULT_DB_QUORUM_OPTION;
int16_t tsPartitons = TSDB_DEFAULT_DB_PARTITON_OPTION;
int8_t tsUpdate = TSDB_DEFAULT_DB_UPDATE_OPTION; int8_t tsUpdate = TSDB_DEFAULT_DB_UPDATE_OPTION;
int8_t tsCacheLastRow = TSDB_DEFAULT_CACHE_BLOCK_SIZE; int8_t tsCacheLastRow = TSDB_DEFAULT_CACHE_LAST_ROW;
int32_t tsMaxVgroupsPerDb = 0; int32_t tsMaxVgroupsPerDb = 0;
int32_t tsMinTablePerVnode = TSDB_TABLES_STEP; int32_t tsMinTablePerVnode = TSDB_TABLES_STEP;
int32_t tsMaxTablePerVnode = TSDB_DEFAULT_TABLES; int32_t tsMaxTablePerVnode = TSDB_DEFAULT_TABLES;
...@@ -853,6 +854,16 @@ static void doInitGlobalConfig(void) { ...@@ -853,6 +854,16 @@ static void doInitGlobalConfig(void) {
cfg.unitType = TAOS_CFG_UTYPE_NONE; cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg); taosInitConfigOption(cfg);
cfg.option = "partitions";
cfg.ptr = &tsPartitons;
cfg.valType = TAOS_CFG_VTYPE_INT16;
cfg.cfgType = TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_SHOW;
cfg.minValue = TSDB_MIN_DB_PARTITON_OPTION;
cfg.maxValue = TSDB_MAX_DB_PARTITON_OPTION;
cfg.ptrLength = 0;
cfg.unitType = TAOS_CFG_UTYPE_NONE;
taosInitConfigOption(cfg);
cfg.option = "quorum"; cfg.option = "quorum";
cfg.ptr = &tsQuorum; cfg.ptr = &tsQuorum;
cfg.valType = TAOS_CFG_VTYPE_INT32; cfg.valType = TAOS_CFG_VTYPE_INT32;
......
src/connector/jdbc/pom.xml
浏览文件 @ 4ec4714e
...@@ -102,6 +102,7 @@ ...@@ -102,6 +102,7 @@
<include>**/*Test.java</include> <include>**/*Test.java</include>
</includes> </includes>
<excludes> <excludes>
<exclude>**/DatetimeBefore1970Test.java</exclude>
<exclude>**/AppMemoryLeakTest.java</exclude> <exclude>**/AppMemoryLeakTest.java</exclude>
<exclude>**/AuthenticationTest.java</exclude> <exclude>**/AuthenticationTest.java</exclude>
<exclude>**/TaosInfoMonitorTest.java</exclude> <exclude>**/TaosInfoMonitorTest.java</exclude>
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/AbstractDriver.java
浏览文件 @ 4ec4714e
package com.taosdata.jdbc; package com.taosdata.jdbc;
import java.io.*;
import java.sql.Driver; import java.sql.Driver;
import java.sql.DriverPropertyInfo; import java.sql.DriverPropertyInfo;
import java.util.ArrayList;
import java.util.List;
import java.util.Properties; import java.util.Properties;
import java.util.StringTokenizer; import java.util.StringTokenizer;
public abstract class AbstractDriver implements Driver { public abstract class AbstractDriver implements Driver {
private static final String TAOS_CFG_FILENAME = "taos.cfg";
/**
* @param cfgDirPath
* @return return the config dir
**/
protected File loadConfigDir(String cfgDirPath) {
if (cfgDirPath == null)
return loadDefaultConfigDir();
File cfgDir = new File(cfgDirPath);
if (!cfgDir.exists())
return loadDefaultConfigDir();
return cfgDir;
}
/**
* @return search the default config dir, if the config dir is not exist will return null
*/
protected File loadDefaultConfigDir() {
File cfgDir;
File cfgDir_linux = new File("/etc/taos");
cfgDir = cfgDir_linux.exists() ? cfgDir_linux : null;
File cfgDir_windows = new File("C:\\TDengine\\cfg");
cfgDir = (cfgDir == null && cfgDir_windows.exists()) ? cfgDir_windows : cfgDir;
return cfgDir;
}
protected List<String> loadConfigEndpoints(File cfgFile) {
List<String> endpoints = new ArrayList<>();
try (BufferedReader reader = new BufferedReader(new FileReader(cfgFile))) {
String line = null;
while ((line = reader.readLine()) != null) {
if (line.trim().startsWith("firstEp") || line.trim().startsWith("secondEp")) {
endpoints.add(line.substring(line.indexOf('p') + 1).trim());
}
if (endpoints.size() > 1)
break;
}
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
return endpoints;
}
protected void loadTaosConfig(Properties info) {
if ((info.getProperty(TSDBDriver.PROPERTY_KEY_HOST) == null || info.getProperty(TSDBDriver.PROPERTY_KEY_HOST).isEmpty()) && (
info.getProperty(TSDBDriver.PROPERTY_KEY_PORT) == null || info.getProperty(TSDBDriver.PROPERTY_KEY_PORT).isEmpty())) {
File cfgDir = loadConfigDir(info.getProperty(TSDBDriver.PROPERTY_KEY_CONFIG_DIR));
File cfgFile = cfgDir.listFiles((dir, name) -> TAOS_CFG_FILENAME.equalsIgnoreCase(name))[0];
List<String> endpoints = loadConfigEndpoints(cfgFile);
if (!endpoints.isEmpty()) {
info.setProperty(TSDBDriver.PROPERTY_KEY_HOST, endpoints.get(0).split(":")[0]);
info.setProperty(TSDBDriver.PROPERTY_KEY_PORT, endpoints.get(0).split(":")[1]);
}
}
}
protected DriverPropertyInfo[] getPropertyInfo(Properties info) { protected DriverPropertyInfo[] getPropertyInfo(Properties info) {
DriverPropertyInfo hostProp = new DriverPropertyInfo(TSDBDriver.PROPERTY_KEY_HOST, info.getProperty(TSDBDriver.PROPERTY_KEY_HOST)); DriverPropertyInfo hostProp = new DriverPropertyInfo(TSDBDriver.PROPERTY_KEY_HOST, info.getProperty(TSDBDriver.PROPERTY_KEY_HOST));
hostProp.required = false; hostProp.required = false;
...@@ -154,6 +92,4 @@ public abstract class AbstractDriver implements Driver { ...@@ -154,6 +92,4 @@ public abstract class AbstractDriver implements Driver {
return urlProps; return urlProps;
} }
} }
src/connector/jdbc/src/main/java/com/taosdata/jdbc/ColumnMetaData.java
浏览文件 @ 4ec4714e
...@@ -52,4 +52,14 @@ public class ColumnMetaData { ...@@ -52,4 +52,14 @@ public class ColumnMetaData {
public void setColIndex(int colIndex) { public void setColIndex(int colIndex) {
this.colIndex = colIndex; this.colIndex = colIndex;
} }
@Override
public String toString() {
return "ColumnMetaData{" +
"colType=" + colType +
", colName='" + colName + '\'' +
", colSize=" + colSize +
", colIndex=" + colIndex +
'}';
}
} }
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBConnection.java
浏览文件 @ 4ec4714e
...@@ -87,11 +87,10 @@ public class TSDBConnection extends AbstractConnection { ...@@ -87,11 +87,10 @@ public class TSDBConnection extends AbstractConnection {
} }
public void close() throws SQLException { public void close() throws SQLException {
if (isClosed()) { if (isClosed)
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_CONNECTION_CLOSED); return;
}
this.isClosed = true;
this.connector.closeConnection(); this.connector.closeConnection();
this.isClosed = true;
} }
public boolean isClosed() throws SQLException { public boolean isClosed() throws SQLException {
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBDriver.java
浏览文件 @ 4ec4714e
...@@ -112,8 +112,6 @@ public class TSDBDriver extends AbstractDriver { ...@@ -112,8 +112,6 @@ public class TSDBDriver extends AbstractDriver {
if ((props = parseURL(url, info)) == null) { if ((props = parseURL(url, info)) == null) {
return null; return null;
} }
//load taos.cfg start
loadTaosConfig(info);
try { try {
TSDBJNIConnector.init((String) props.get(PROPERTY_KEY_CONFIG_DIR), (String) props.get(PROPERTY_KEY_LOCALE), TSDBJNIConnector.init((String) props.get(PROPERTY_KEY_CONFIG_DIR), (String) props.get(PROPERTY_KEY_LOCALE),
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBJNIConnector.java
浏览文件 @ 4ec4714e
/*************************************************************************** /**
* *************************************************************************
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com> * Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
* *
* This program is free software: you can use, redistribute, and/or modify * This program is free software: you can use, redistribute, and/or modify
...@@ -11,7 +12,7 @@ ...@@ -11,7 +12,7 @@
* *
* You should have received a copy of the GNU Affero General Public License * You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*****************************************************************************/ **************************************************************************** */
package com.taosdata.jdbc; package com.taosdata.jdbc;
import com.taosdata.jdbc.utils.TaosInfo; import com.taosdata.jdbc.utils.TaosInfo;
...@@ -20,6 +21,9 @@ import java.sql.SQLException; ...@@ -20,6 +21,9 @@ import java.sql.SQLException;
import java.sql.SQLWarning; import java.sql.SQLWarning;
import java.util.List; import java.util.List;
/**
* JNI connector
* */
public class TSDBJNIConnector { public class TSDBJNIConnector {
private static volatile Boolean isInitialized = false; private static volatile Boolean isInitialized = false;
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBResultSet.java
浏览文件 @ 4ec4714e
...@@ -20,18 +20,16 @@ import java.util.ArrayList; ...@@ -20,18 +20,16 @@ import java.util.ArrayList;
import java.util.List; import java.util.List;
public class TSDBResultSet extends AbstractResultSet implements ResultSet { public class TSDBResultSet extends AbstractResultSet implements ResultSet {
private TSDBJNIConnector jniConnector; private final TSDBJNIConnector jniConnector;
private final TSDBStatement statement; private final TSDBStatement statement;
private long resultSetPointer = 0L; private final long resultSetPointer;
private List<ColumnMetaData> columnMetaDataList = new ArrayList<>(); private List<ColumnMetaData> columnMetaDataList = new ArrayList<>();
private final TSDBResultSetRowData rowData;
private TSDBResultSetRowData rowData; private final TSDBResultSetBlockData blockData;
private TSDBResultSetBlockData blockData;
private boolean batchFetch = false; private boolean batchFetch = false;
private boolean lastWasNull = false; private boolean lastWasNull = false;
private final int COLUMN_INDEX_START_VALUE = 1; private boolean isClosed;
public void setBatchFetch(boolean batchFetch) { public void setBatchFetch(boolean batchFetch) {
this.batchFetch = batchFetch; this.batchFetch = batchFetch;
...@@ -56,13 +54,13 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -56,13 +54,13 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
int code = this.jniConnector.getSchemaMetaData(this.resultSetPointer, this.columnMetaDataList); int code = this.jniConnector.getSchemaMetaData(this.resultSetPointer, this.columnMetaDataList);
if (code == TSDBConstants.JNI_CONNECTION_NULL) { if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} }
if (code == TSDBConstants.JNI_RESULT_SET_NULL) { if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_RESULT_SET_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} }
if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) { if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_NUM_OF_FIELDS_0)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} }
this.rowData = new TSDBResultSetRowData(this.columnMetaDataList.size()); this.rowData = new TSDBResultSetRowData(this.columnMetaDataList.size());
this.blockData = new TSDBResultSetBlockData(this.columnMetaDataList, this.columnMetaDataList.size()); this.blockData = new TSDBResultSetBlockData(this.columnMetaDataList, this.columnMetaDataList.size());
...@@ -78,16 +76,12 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -78,16 +76,12 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
this.blockData.reset(); this.blockData.reset();
if (code == TSDBConstants.JNI_CONNECTION_NULL) { if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} else if (code == TSDBConstants.JNI_RESULT_SET_NULL) { } else if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_RESULT_SET_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) { } else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_NUM_OF_FIELDS_0)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} else if (code == TSDBConstants.JNI_FETCH_END) { } else return code != TSDBConstants.JNI_FETCH_END;
return false;
}
return true;
} else { } else {
if (rowData != null) { if (rowData != null) {
this.rowData.clear(); this.rowData.clear();
...@@ -95,11 +89,11 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -95,11 +89,11 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
int code = this.jniConnector.fetchRow(this.resultSetPointer, this.rowData); int code = this.jniConnector.fetchRow(this.resultSetPointer, this.rowData);
if (code == TSDBConstants.JNI_CONNECTION_NULL) { if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} else if (code == TSDBConstants.JNI_RESULT_SET_NULL) { } else if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_RESULT_SET_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) { } else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_NUM_OF_FIELDS_0)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} else if (code == TSDBConstants.JNI_FETCH_END) { } else if (code == TSDBConstants.JNI_FETCH_END) {
return false; return false;
} else { } else {
...@@ -109,14 +103,17 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -109,14 +103,17 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
} }
public void close() throws SQLException { public void close() throws SQLException {
if (isClosed)
return;
if (this.jniConnector != null) { if (this.jniConnector != null) {
int code = this.jniConnector.freeResultSet(this.resultSetPointer); int code = this.jniConnector.freeResultSet(this.resultSetPointer);
if (code == TSDBConstants.JNI_CONNECTION_NULL) { if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} else if (code == TSDBConstants.JNI_RESULT_SET_NULL) { } else if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_RESULT_SET_NULL)); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} }
} }
isClosed = true;
} }
public boolean wasNull() throws SQLException { public boolean wasNull() throws SQLException {
...@@ -415,8 +412,8 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -415,8 +412,8 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
} }
public boolean isClosed() throws SQLException { public boolean isClosed() throws SQLException {
//TODO: check if need release resources if (isClosed)
boolean isClosed = true; return true;
if (jniConnector != null) { if (jniConnector != null) {
isClosed = jniConnector.isResultsetClosed(); isClosed = jniConnector.isResultsetClosed();
} }
...@@ -429,14 +426,12 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet { ...@@ -429,14 +426,12 @@ public class TSDBResultSet extends AbstractResultSet implements ResultSet {
} }
private int getTrueColumnIndex(int columnIndex) throws SQLException { private int getTrueColumnIndex(int columnIndex) throws SQLException {
if (columnIndex < this.COLUMN_INDEX_START_VALUE) { if (columnIndex < 1)
throw new SQLException("Column Index out of range, " + columnIndex + " < " + this.COLUMN_INDEX_START_VALUE); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_PARAMETER_INDEX_OUT_RANGE, "columnIndex(" + columnIndex + "): < 1");
}
int numOfCols = this.columnMetaDataList.size(); int numOfCols = this.columnMetaDataList.size();
if (columnIndex > numOfCols) { if (columnIndex > numOfCols)
throw new SQLException("Column Index out of range, " + columnIndex + " > " + numOfCols); throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_PARAMETER_INDEX_OUT_RANGE, "columnIndex: " + columnIndex);
}
return columnIndex - 1; return columnIndex - 1;
} }
} }
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBStatement.java
浏览文件 @ 4ec4714e
...@@ -73,11 +73,11 @@ public class TSDBStatement extends AbstractStatement { ...@@ -73,11 +73,11 @@ public class TSDBStatement extends AbstractStatement {
} }
public void close() throws SQLException { public void close() throws SQLException {
if (!isClosed) { if (isClosed)
if (this.resultSet != null) return;
this.resultSet.close(); if (this.resultSet != null && !this.resultSet.isClosed())
isClosed = true; this.resultSet.close();
} isClosed = true;
} }
public boolean execute(String sql) throws SQLException { public boolean execute(String sql) throws SQLException {
......
src/connector/jdbc/src/test/java/com/taosdata/jdbc/TSDBJNIConnectorTest.java 0 → 100644
浏览文件 @ 4ec4714e
package com.taosdata.jdbc;
import org.junit.Test;
import java.sql.SQLException;
import java.sql.SQLWarning;
import java.util.ArrayList;
import java.util.List;
public class TSDBJNIConnectorTest {
private static TSDBResultSetRowData rowData;
@Test
public void test() {
try {
// init
TSDBJNIConnector.init(null, null, null, null);
// connect
TSDBJNIConnector connector = new TSDBJNIConnector();
connector.connect("127.0.0.1", 6030, null, "root", "taosdata");
// executeQuery
long pSql = connector.executeQuery("show variables");
if (connector.isUpdateQuery(pSql)) {
connector.freeResultSet(pSql);
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_INVALID_WITH_EXECUTEQUERY);
}
// get schema
List<ColumnMetaData> columnMetaDataList = new ArrayList<>();
int code = connector.getSchemaMetaData(pSql, columnMetaDataList);
if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
}
if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_RESULT_SET_NULL));
}
if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_NUM_OF_FIELDS_0));
}
int columnSize = columnMetaDataList.size();
// print metadata
for (int i = 0; i < columnSize; i++) {
System.out.println(columnMetaDataList.get(i));
}
rowData = new TSDBResultSetRowData(columnSize);
// iterate resultSet
for (int i = 0; next(connector, pSql); i++) {
System.out.println("col[" + i + "] size: " + rowData.getColSize());
rowData.getData().stream().forEach(col -> System.out.print(col + "\t"));
System.out.println();
}
// close resultSet
code = connector.freeResultSet(pSql);
if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} else if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
}
// close statement
// close connection
connector.closeConnection();
} catch (SQLWarning throwables) {
throwables.printStackTrace();
} catch (SQLException e) {
e.printStackTrace();
}
}
private static boolean next(TSDBJNIConnector connector, long pSql) throws SQLException {
if (rowData != null)
rowData.clear();
int code = connector.fetchRow(pSql, rowData);
if (code == TSDBConstants.JNI_CONNECTION_NULL) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_CONNECTION_NULL);
} else if (code == TSDBConstants.JNI_RESULT_SET_NULL) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_RESULT_SET_NULL);
} else if (code == TSDBConstants.JNI_NUM_OF_FIELDS_0) {
throw TSDBError.createSQLException(TSDBErrorNumbers.ERROR_JNI_NUM_OF_FIELDS_0);
} else if (code == TSDBConstants.JNI_FETCH_END) {
return false;
} else {
return true;
}
}
}
\ No newline at end of file
src/connector/jdbc/src/test/java/com/taosdata/jdbc/cases/FailOverTest.java
浏览文件 @ 4ec4714e
...@@ -4,7 +4,6 @@ import org.junit.Test; ...@@ -4,7 +4,6 @@ import org.junit.Test;
import java.sql.*; import java.sql.*;
import java.text.SimpleDateFormat; import java.text.SimpleDateFormat;
import java.util.Date;
import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeUnit;
public class FailOverTest { public class FailOverTest {
...@@ -18,13 +17,17 @@ public class FailOverTest { ...@@ -18,13 +17,17 @@ public class FailOverTest {
long end = System.currentTimeMillis() + 1000 * 60 * 5; long end = System.currentTimeMillis() + 1000 * 60 * 5;
while (System.currentTimeMillis() < end) { while (System.currentTimeMillis() < end) {
try (Connection conn = DriverManager.getConnection(url)) { try (Connection conn = DriverManager.getConnection(url); Statement stmt = conn.createStatement()) {
Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("show dnodes");
ResultSet resultSet = stmt.executeQuery("select server_status()"); ResultSetMetaData meta = rs.getMetaData();
resultSet.next(); while (rs.next()) {
int status = resultSet.getInt("server_status()"); for (int i = 1; i <= meta.getColumnCount(); i++) {
System.out.println(">>>>>>>>>" + sdf.format(new Date()) + " status : " + status); System.out.print(meta.getColumnLabel(i) + ": " + rs.getString(i) + "\t");
stmt.close(); }
System.out.println();
}
System.out.println("=======================");
rs.close();
TimeUnit.SECONDS.sleep(5); TimeUnit.SECONDS.sleep(5);
} catch (SQLException | InterruptedException e) { } catch (SQLException | InterruptedException e) {
e.printStackTrace(); e.printStackTrace();
......
src/connector/python/linux/python2/setup.py
浏览文件 @ 4ec4714e
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.6", version="2.0.7",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
src/connector/python/linux/python2/taos/cinterface.py
浏览文件 @ 4ec4714e
...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
if num_of_rows > 0: if num_of_rows > 0:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
else: else:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False):
""" """
if num_of_rows > 0: if num_of_rows > 0:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
else: else:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
def _crow_bigint_unsigned_to_python( def _crow_bigint_unsigned_to_python(
...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( ...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python(
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
else: else:
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
...@@ -600,7 +600,7 @@ class CTaosInterface(object): ...@@ -600,7 +600,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE):
...@@ -608,7 +608,7 @@ class CTaosInterface(object): ...@@ -608,7 +608,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY):
# return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00') # return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00')
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR):
# return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00') # return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00')
......
src/connector/python/linux/python3/setup.py
浏览文件 @ 4ec4714e
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.5", version="2.0.7",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
src/connector/python/linux/python3/taos/cinterface.py
浏览文件 @ 4ec4714e
...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
if num_of_rows > 0: if num_of_rows > 0:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
else: else:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False):
""" """
if num_of_rows > 0: if num_of_rows > 0:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
else: else:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
def _crow_bigint_unsigned_to_python( def _crow_bigint_unsigned_to_python(
...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( ...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python(
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
else: else:
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
...@@ -600,7 +600,7 @@ class CTaosInterface(object): ...@@ -600,7 +600,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE):
...@@ -608,7 +608,7 @@ class CTaosInterface(object): ...@@ -608,7 +608,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY):
# return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00') # return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00')
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR):
# return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00') # return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00')
......
src/connector/python/linux/python3/taos/cursor.py
浏览文件 @ 4ec4714e
from .cinterface import CTaosInterface from .cinterface import CTaosInterface
from .error import * from .error import *
from .constants import FieldType from .constants import FieldType
import threading
# querySeqNum = 0 # querySeqNum = 0
...@@ -38,7 +37,6 @@ class TDengineCursor(object): ...@@ -38,7 +37,6 @@ class TDengineCursor(object):
self._block_iter = 0 self._block_iter = 0
self._affected_rows = 0 self._affected_rows = 0
self._logfile = "" self._logfile = ""
self._threadId = threading.get_ident()
if connection is not None: if connection is not None:
self._connection = connection self._connection = connection
...@@ -105,12 +103,6 @@ class TDengineCursor(object): ...@@ -105,12 +103,6 @@ class TDengineCursor(object):
def execute(self, operation, params=None): def execute(self, operation, params=None):
"""Prepare and execute a database operation (query or command). """Prepare and execute a database operation (query or command).
""" """
# if threading.get_ident() != self._threadId:
# info ="Cursor execute:Thread ID not match,creater:"+str(self._threadId)+" caller:"+str(threading.get_ident())
# raise OperationalError(info)
# print(info)
# return None
if not operation: if not operation:
return None return None
...@@ -280,12 +272,6 @@ class TDengineCursor(object): ...@@ -280,12 +272,6 @@ class TDengineCursor(object):
def _handle_result(self): def _handle_result(self):
"""Handle the return result from query. """Handle the return result from query.
""" """
# if threading.get_ident() != self._threadId:
# info = "Cursor handleresult:Thread ID not match,creater:"+str(self._threadId)+" caller:"+str(threading.get_ident())
# raise OperationalError(info)
# print(info)
# return None
self._description = [] self._description = []
for ele in self._fields: for ele in self._fields:
self._description.append( self._description.append(
......
src/connector/python/osx/python3/setup.py
浏览文件 @ 4ec4714e
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.5", version="2.0.7",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
src/connector/python/osx/python3/taos/cinterface.py
浏览文件 @ 4ec4714e
...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
if num_of_rows > 0: if num_of_rows > 0:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
else: else:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False):
""" """
if num_of_rows > 0: if num_of_rows > 0:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
else: else:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
def _crow_bigint_unsigned_to_python( def _crow_bigint_unsigned_to_python(
...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( ...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python(
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
else: else:
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
...@@ -600,7 +600,7 @@ class CTaosInterface(object): ...@@ -600,7 +600,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE):
...@@ -608,7 +608,7 @@ class CTaosInterface(object): ...@@ -608,7 +608,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY):
# return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00') # return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00')
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR):
# return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00') # return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00')
......
src/connector/python/osx/python3/taos/cursor.py
浏览文件 @ 4ec4714e
from .cinterface import CTaosInterface from .cinterface import CTaosInterface
from .error import * from .error import *
from .constants import FieldType from .constants import FieldType
import threading
# querySeqNum = 0 # querySeqNum = 0
...@@ -38,7 +37,6 @@ class TDengineCursor(object): ...@@ -38,7 +37,6 @@ class TDengineCursor(object):
self._block_iter = 0 self._block_iter = 0
self._affected_rows = 0 self._affected_rows = 0
self._logfile = "" self._logfile = ""
self._threadId = threading.get_ident()
if connection is not None: if connection is not None:
self._connection = connection self._connection = connection
...@@ -105,12 +103,6 @@ class TDengineCursor(object): ...@@ -105,12 +103,6 @@ class TDengineCursor(object):
def execute(self, operation, params=None): def execute(self, operation, params=None):
"""Prepare and execute a database operation (query or command). """Prepare and execute a database operation (query or command).
""" """
# if threading.get_ident() != self._threadId:
# info ="Cursor execute:Thread ID not match,creater:"+str(self._threadId)+" caller:"+str(threading.get_ident())
# raise OperationalError(info)
# print(info)
# return None
if not operation: if not operation:
return None return None
...@@ -280,12 +272,6 @@ class TDengineCursor(object): ...@@ -280,12 +272,6 @@ class TDengineCursor(object):
def _handle_result(self): def _handle_result(self):
"""Handle the return result from query. """Handle the return result from query.
""" """
# if threading.get_ident() != self._threadId:
# info = "Cursor handleresult:Thread ID not match,creater:"+str(self._threadId)+" caller:"+str(threading.get_ident())
# raise OperationalError(info)
# print(info)
# return None
self._description = [] self._description = []
for ele in self._fields: for ele in self._fields:
self._description.append( self._description.append(
......
src/connector/python/windows/python2/setup.py
浏览文件 @ 4ec4714e
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.4", version="2.0.7",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
src/connector/python/windows/python2/taos/cinterface.py
浏览文件 @ 4ec4714e
...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
if num_of_rows > 0: if num_of_rows > 0:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
else: else:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False):
""" """
if num_of_rows > 0: if num_of_rows > 0:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
else: else:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
def _crow_bigint_unsigned_to_python( def _crow_bigint_unsigned_to_python(
...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( ...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python(
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
else: else:
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
...@@ -600,7 +600,7 @@ class CTaosInterface(object): ...@@ -600,7 +600,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE):
...@@ -608,7 +608,7 @@ class CTaosInterface(object): ...@@ -608,7 +608,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY):
# return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00') # return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00')
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR):
# return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00') # return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00')
......
src/connector/python/windows/python2/taos/cursor.py
浏览文件 @ 4ec4714e
from .cinterface import CTaosInterface from .cinterface import CTaosInterface
from .error import * from .error import *
from .constants import FieldType from .constants import FieldType
import threading
# querySeqNum = 0 # querySeqNum = 0
...@@ -38,7 +37,6 @@ class TDengineCursor(object): ...@@ -38,7 +37,6 @@ class TDengineCursor(object):
self._block_iter = 0 self._block_iter = 0
self._affected_rows = 0 self._affected_rows = 0
self._logfile = "" self._logfile = ""
self._threadId = threading.get_ident()
if connection is not None: if connection is not None:
self._connection = connection self._connection = connection
......
src/connector/python/windows/python3/setup.py
浏览文件 @ 4ec4714e
...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh: ...@@ -5,7 +5,7 @@ with open("README.md", "r") as fh:
setuptools.setup( setuptools.setup(
name="taos", name="taos",
version="2.0.4", version="2.0.7",
author="Taosdata Inc.", author="Taosdata Inc.",
author_email="support@taosdata.com", author_email="support@taosdata.com",
description="TDengine python client package", description="TDengine python client package",
......
src/connector/python/windows/python3/taos/cinterface.py
浏览文件 @ 4ec4714e
...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -22,10 +22,10 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False):
if num_of_rows > 0: if num_of_rows > 0:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
else: else:
return list(map(_timestamp_converter, ctypes.cast( return list(map(_timestamp_converter, ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]))
def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False):
...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): ...@@ -145,10 +145,10 @@ def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False):
""" """
if num_of_rows > 0: if num_of_rows > 0:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
else: else:
return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] data, ctypes.POINTER(ctypes.c_int64))[:abs(num_of_rows)]]
def _crow_bigint_unsigned_to_python( def _crow_bigint_unsigned_to_python(
...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( ...@@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python(
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
else: else:
return [ return [
None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast(
data, ctypes.POINTER( data, ctypes.POINTER(
ctypes.c_ulong))[ ctypes.c_uint64))[
:abs(num_of_rows)]] :abs(num_of_rows)]]
...@@ -600,7 +600,7 @@ class CTaosInterface(object): ...@@ -600,7 +600,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_INT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BIGINT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_FLOAT):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_DOUBLE):
...@@ -608,7 +608,7 @@ class CTaosInterface(object): ...@@ -608,7 +608,7 @@ class CTaosInterface(object):
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_BINARY):
# return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00') # return (ctypes.cast(data, ctypes.POINTER(ctypes.c_char))[0:byte]).rstrip('\x00')
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_TIMESTAMP):
# return ctypes.cast(data, ctypes.POINTER(ctypes.c_long))[0] # return ctypes.cast(data, ctypes.POINTER(ctypes.c_int64))[0]
# elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR): # elif (dtype == CTaosInterface.TSDB_DATA_TYPE_NCHAR):
# return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00') # return (ctypes.cast(data, ctypes.c_char_p).value).rstrip('\x00')
......
src/connector/python/windows/python3/taos/cursor.py
浏览文件 @ 4ec4714e
from .cinterface import CTaosInterface from .cinterface import CTaosInterface
from .error import * from .error import *
from .constants import FieldType from .constants import FieldType
import threading
# querySeqNum = 0 # querySeqNum = 0
...@@ -38,7 +37,6 @@ class TDengineCursor(object): ...@@ -38,7 +37,6 @@ class TDengineCursor(object):
self._block_iter = 0 self._block_iter = 0
self._affected_rows = 0 self._affected_rows = 0
self._logfile = "" self._logfile = ""
self._threadId = threading.get_ident()
if connection is not None: if connection is not None:
self._connection = connection self._connection = connection
......
src/dnode/CMakeLists.txt
浏览文件 @ 4ec4714e
...@@ -31,6 +31,10 @@ IF (TD_MQTT) ...@@ -31,6 +31,10 @@ IF (TD_MQTT)
TARGET_LINK_LIBRARIES(taosd mqtt) TARGET_LINK_LIBRARIES(taosd mqtt)
ENDIF () ENDIF ()
IF (TD_TOPIC)
TARGET_LINK_LIBRARIES(taosd topic)
ENDIF ()
SET(PREPARE_ENV_CMD "prepare_env_cmd") SET(PREPARE_ENV_CMD "prepare_env_cmd")
SET(PREPARE_ENV_TARGET "prepare_env_target") SET(PREPARE_ENV_TARGET "prepare_env_target")
ADD_CUSTOM_COMMAND(OUTPUT ${PREPARE_ENV_CMD} ADD_CUSTOM_COMMAND(OUTPUT ${PREPARE_ENV_CMD}
......
src/dnode/src/dnodeMWrite.c
浏览文件 @ 4ec4714e
...@@ -146,10 +146,10 @@ void dnodeSendRpcMWriteRsp(void *pMsg, int32_t code) { ...@@ -146,10 +146,10 @@ void dnodeSendRpcMWriteRsp(void *pMsg, int32_t code) {
} }
dTrace("msg:%p, app:%p type:%s master:%p will be responsed", pWrite, pWrite->rpcMsg.ahandle, dTrace("msg:%p, app:%p type:%s master:%p will be responsed", pWrite, pWrite->rpcMsg.ahandle,
taosMsg[pWrite->rpcMsg.msgType], pWrite->pBatchMasterMsg); taosMsg[pWrite->rpcMsg.msgType], pWrite->pBatchMasterMsg);
if (pWrite->pBatchMasterMsg && pWrite != pWrite->pBatchMasterMsg) { if (pWrite->pBatchMasterMsg && pWrite != pWrite->pBatchMasterMsg) {
dError("msg:%p, app:%p type:%s master:%p sub message should not response!", pWrite, pWrite->rpcMsg.ahandle, dError("msg:%p, app:%p type:%s master:%p sub message should not response!", pWrite, pWrite->rpcMsg.ahandle,
taosMsg[pWrite->rpcMsg.msgType], pWrite->pBatchMasterMsg); taosMsg[pWrite->rpcMsg.msgType], pWrite->pBatchMasterMsg);
return; return;
} }
......
src/dnode/src/dnodeMain.c
浏览文件 @ 4ec4714e
...@@ -189,6 +189,11 @@ static void dnodeCheckDataDirOpenned(char *dir) { ...@@ -189,6 +189,11 @@ static void dnodeCheckDataDirOpenned(char *dir) {
} }
static int32_t dnodeInitStorage() { static int32_t dnodeInitStorage() {
if (tsDiskCfgNum == 1 && dnodeCreateDir(tsDataDir) < 0) {
dError("failed to create dir: %s, reason: %s", tsDataDir, strerror(errno));
return -1;
}
if (tfsInit(tsDiskCfg, tsDiskCfgNum) < 0) { if (tfsInit(tsDiskCfg, tsDiskCfgNum) < 0) {
dError("failed to init TFS since %s", tstrerror(terrno)); dError("failed to init TFS since %s", tstrerror(terrno));
return -1; return -1;
......
src/dnode/src/dnodeShell.c
浏览文件 @ 4ec4714e
...@@ -47,8 +47,11 @@ int32_t dnodeInitShell() { ...@@ -47,8 +47,11 @@ int32_t dnodeInitShell() {
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DNODE]= dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DNODE] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_DB] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TP] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_DB] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TP] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_DB] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TP] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_CREATE_TABLE]= dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_DROP_TABLE] = dnodeDispatchToMWriteQueue;
dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = dnodeDispatchToMWriteQueue; dnodeProcessShellMsgFp[TSDB_MSG_TYPE_CM_ALTER_TABLE] = dnodeDispatchToMWriteQueue;
......
src/inc/taosdef.h
浏览文件 @ 4ec4714e
...@@ -222,6 +222,9 @@ do { \ ...@@ -222,6 +222,9 @@ do { \
#define TSDB_MQTT_TOPIC_LEN 64 #define TSDB_MQTT_TOPIC_LEN 64
#define TSDB_MQTT_CLIENT_ID_LEN 32 #define TSDB_MQTT_CLIENT_ID_LEN 32
#define TSDB_DB_TYPE_DEFAULT 0
#define TSDB_DB_TYPE_TOPIC 1
#define TSDB_DEFAULT_PKT_SIZE 65480 //same as RPC_MAX_UDP_SIZE #define TSDB_DEFAULT_PKT_SIZE 65480 //same as RPC_MAX_UDP_SIZE
#define TSDB_PAYLOAD_SIZE TSDB_DEFAULT_PKT_SIZE #define TSDB_PAYLOAD_SIZE TSDB_DEFAULT_PKT_SIZE
...@@ -306,6 +309,10 @@ do { \ ...@@ -306,6 +309,10 @@ do { \
#define TSDB_MAX_DB_REPLICA_OPTION 3 #define TSDB_MAX_DB_REPLICA_OPTION 3
#define TSDB_DEFAULT_DB_REPLICA_OPTION 1 #define TSDB_DEFAULT_DB_REPLICA_OPTION 1
#define TSDB_MIN_DB_PARTITON_OPTION 0
#define TSDB_MAX_DB_PARTITON_OPTION 1000
#define TSDB_DEFAULT_DB_PARTITON_OPTION 4
#define TSDB_MIN_DB_QUORUM_OPTION 1 #define TSDB_MIN_DB_QUORUM_OPTION 1
#define TSDB_MAX_DB_QUORUM_OPTION 2 #define TSDB_MAX_DB_QUORUM_OPTION 2
#define TSDB_DEFAULT_DB_QUORUM_OPTION 1 #define TSDB_DEFAULT_DB_QUORUM_OPTION 1
......
src/inc/taoserror.h
浏览文件 @ 4ec4714e
...@@ -185,6 +185,9 @@ int32_t* taosGetErrno(); ...@@ -185,6 +185,9 @@ int32_t* taosGetErrno();
#define TSDB_CODE_MND_INVALID_DB_OPTION_DAYS TAOS_DEF_ERROR_CODE(0, 0x0390) //"Invalid database option: days out of range") #define TSDB_CODE_MND_INVALID_DB_OPTION_DAYS TAOS_DEF_ERROR_CODE(0, 0x0390) //"Invalid database option: days out of range")
#define TSDB_CODE_MND_INVALID_DB_OPTION_KEEP TAOS_DEF_ERROR_CODE(0, 0x0391) //"Invalid database option: keep >= keep1 >= keep0 >= days") #define TSDB_CODE_MND_INVALID_DB_OPTION_KEEP TAOS_DEF_ERROR_CODE(0, 0x0391) //"Invalid database option: keep >= keep1 >= keep0 >= days")
#define TSDB_CODE_MND_INVALID_TOPIC TAOS_DEF_ERROR_CODE(0, 0x0392) //"Invalid topic name)
#define TSDB_CODE_MND_INVALID_TOPIC_OPTION TAOS_DEF_ERROR_CODE(0, 0x0393) //"Invalid topic option)
// dnode // dnode
#define TSDB_CODE_DND_MSG_NOT_PROCESSED TAOS_DEF_ERROR_CODE(0, 0x0400) //"Message not processed") #define TSDB_CODE_DND_MSG_NOT_PROCESSED TAOS_DEF_ERROR_CODE(0, 0x0400) //"Message not processed")
#define TSDB_CODE_DND_OUT_OF_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0401) //"Dnode out of memory") #define TSDB_CODE_DND_OUT_OF_MEMORY TAOS_DEF_ERROR_CODE(0, 0x0401) //"Dnode out of memory")
...@@ -235,6 +238,7 @@ int32_t* taosGetErrno(); ...@@ -235,6 +238,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_TDB_IVD_CREATE_TABLE_INFO TAOS_DEF_ERROR_CODE(0, 0x0612) //"Invalid information to create table") #define TSDB_CODE_TDB_IVD_CREATE_TABLE_INFO TAOS_DEF_ERROR_CODE(0, 0x0612) //"Invalid information to create table")
#define TSDB_CODE_TDB_NO_AVAIL_DISK TAOS_DEF_ERROR_CODE(0, 0x0613) //"No available disk") #define TSDB_CODE_TDB_NO_AVAIL_DISK TAOS_DEF_ERROR_CODE(0, 0x0613) //"No available disk")
#define TSDB_CODE_TDB_MESSED_MSG TAOS_DEF_ERROR_CODE(0, 0x0614) //"TSDB messed message") #define TSDB_CODE_TDB_MESSED_MSG TAOS_DEF_ERROR_CODE(0, 0x0614) //"TSDB messed message")
#define TSDB_CODE_TDB_IVLD_TAG_VAL TAOS_DEF_ERROR_CODE(0, 0x0615) //"TSDB invalid tag value")
// query // query
#define TSDB_CODE_QRY_INVALID_QHANDLE TAOS_DEF_ERROR_CODE(0, 0x0700) //"Invalid handle") #define TSDB_CODE_QRY_INVALID_QHANDLE TAOS_DEF_ERROR_CODE(0, 0x0700) //"Invalid handle")
...@@ -249,7 +253,7 @@ int32_t* taosGetErrno(); ...@@ -249,7 +253,7 @@ int32_t* taosGetErrno();
#define TSDB_CODE_QRY_IN_EXEC TAOS_DEF_ERROR_CODE(0, 0x0709) //"Multiple retrieval of this query") #define TSDB_CODE_QRY_IN_EXEC TAOS_DEF_ERROR_CODE(0, 0x0709) //"Multiple retrieval of this query")
#define TSDB_CODE_QRY_TOO_MANY_TIMEWINDOW TAOS_DEF_ERROR_CODE(0, 0x070A) //"Too many time window in query") #define TSDB_CODE_QRY_TOO_MANY_TIMEWINDOW TAOS_DEF_ERROR_CODE(0, 0x070A) //"Too many time window in query")
#define TSDB_CODE_QRY_NOT_ENOUGH_BUFFER TAOS_DEF_ERROR_CODE(0, 0x070B) //"Query buffer limit has reached") #define TSDB_CODE_QRY_NOT_ENOUGH_BUFFER TAOS_DEF_ERROR_CODE(0, 0x070B) //"Query buffer limit has reached")
#define TSDB_CODE_QRY_INCONSISTAN TAOS_DEF_ERROR_CODE(0, 0x070C) //"File inconsistance in replica") #define TSDB_CODE_QRY_INCONSISTAN TAOS_DEF_ERROR_CODE(0, 0x070C) //"File inconsistency in replica")
// grant // grant
......
src/inc/taosmsg.h
浏览文件 @ 4ec4714e
...@@ -107,6 +107,12 @@ TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_DUMMY13, "dummy13" ) ...@@ -107,6 +107,12 @@ TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_DUMMY13, "dummy13" )
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_DUMMY14, "dummy14" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_DUMMY14, "dummy14" )
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_NETWORK_TEST, "nettest" ) TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_NETWORK_TEST, "nettest" )
// message for topic
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_CREATE_TP, "create-tp" )
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_DROP_TP, "drop-tp" )
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_USE_TP, "use-tp" )
TAOS_DEFINE_MESSAGE_TYPE( TSDB_MSG_TYPE_CM_ALTER_TP, "alter-tp" )
#ifndef TAOS_MESSAGE_C #ifndef TAOS_MESSAGE_C
TSDB_MSG_TYPE_MAX // 105 TSDB_MSG_TYPE_MAX // 105
#endif #endif
...@@ -141,6 +147,7 @@ enum _mgmt_table { ...@@ -141,6 +147,7 @@ enum _mgmt_table {
TSDB_MGMT_TABLE_VNODES, TSDB_MGMT_TABLE_VNODES,
TSDB_MGMT_TABLE_STREAMTABLES, TSDB_MGMT_TABLE_STREAMTABLES,
TSDB_MGMT_TABLE_CLUSTER, TSDB_MGMT_TABLE_CLUSTER,
TSDB_MGMT_TABLE_TP,
TSDB_MGMT_TABLE_MAX, TSDB_MGMT_TABLE_MAX,
}; };
...@@ -387,7 +394,7 @@ typedef struct SColIndex { ...@@ -387,7 +394,7 @@ typedef struct SColIndex {
int16_t colId; // column id int16_t colId; // column id
int16_t colIndex; // column index in colList if it is a normal column or index in tagColList if a tag int16_t colIndex; // column index in colList if it is a normal column or index in tagColList if a tag
uint16_t flag; // denote if it is a tag or a normal column uint16_t flag; // denote if it is a tag or a normal column
char name[TSDB_COL_NAME_LEN]; char name[TSDB_COL_NAME_LEN]; // TODO remove it
} SColIndex; } SColIndex;
/* sql function msg, to describe the message to vnode about sql function /* sql function msg, to describe the message to vnode about sql function
...@@ -395,7 +402,10 @@ typedef struct SColIndex { ...@@ -395,7 +402,10 @@ typedef struct SColIndex {
typedef struct SSqlFuncMsg { typedef struct SSqlFuncMsg {
int16_t functionId; int16_t functionId;
int16_t numOfParams; int16_t numOfParams;
int16_t resColId; // result column id, id of the current output column int16_t resColId; // result column id, id of the current output column
int16_t colType;
int16_t colBytes;
SColIndex colInfo; SColIndex colInfo;
struct ArgElem { struct ArgElem {
...@@ -475,12 +485,13 @@ typedef struct { ...@@ -475,12 +485,13 @@ typedef struct {
int16_t orderColId; int16_t orderColId;
int16_t numOfCols; // the number of columns will be load from vnode int16_t numOfCols; // the number of columns will be load from vnode
SInterval interval; SInterval interval;
SSessionWindow sw; // session window
uint16_t tagCondLen; // tag length in current query uint16_t tagCondLen; // tag length in current query
uint32_t tbnameCondLen; // table name filter condition string length uint32_t tbnameCondLen; // table name filter condition string length
int16_t numOfGroupCols; // num of group by columns int16_t numOfGroupCols; // num of group by columns
int16_t orderByIdx; int16_t orderByIdx;
int16_t orderType; // used in group by xx order by xxx int16_t orderType; // used in group by xx order by xxx
int64_t vgroupLimit; // limit the number of rows for each table, used in order by + limit in stable projection query. int64_t vgroupLimit; // limit the number of rows for each table, used in order by + limit in stable projection query.
int16_t prjOrder; // global order in super table projection query. int16_t prjOrder; // global order in super table projection query.
int64_t limit; int64_t limit;
int64_t offset; int64_t offset;
...@@ -555,7 +566,9 @@ typedef struct { ...@@ -555,7 +566,9 @@ typedef struct {
int8_t ignoreExist; int8_t ignoreExist;
int8_t update; int8_t update;
int8_t cacheLastRow; int8_t cacheLastRow;
int8_t reserve[8]; int8_t dbType;
int16_t partitions;
int8_t reserve[5];
} SCreateDbMsg, SAlterDbMsg; } SCreateDbMsg, SAlterDbMsg;
typedef struct { typedef struct {
...@@ -616,6 +629,7 @@ typedef struct { ...@@ -616,6 +629,7 @@ typedef struct {
int32_t maxtablesPerVnode; int32_t maxtablesPerVnode;
int32_t maxVgroupsPerDb; int32_t maxVgroupsPerDb;
char arbitrator[TSDB_EP_LEN]; // tsArbitrator char arbitrator[TSDB_EP_LEN]; // tsArbitrator
char reserve[2]; // to solve arm32 bus error
char timezone[64]; // tsTimezone char timezone[64]; // tsTimezone
int64_t checkTime; // 1970-01-01 00:00:00.000 int64_t checkTime; // 1970-01-01 00:00:00.000
char locale[TSDB_LOCALE_LEN]; // tsLocale char locale[TSDB_LOCALE_LEN]; // tsLocale
...@@ -674,7 +688,8 @@ typedef struct { ...@@ -674,7 +688,8 @@ typedef struct {
int8_t cacheLastRow; int8_t cacheLastRow;
int32_t vgCfgVersion; int32_t vgCfgVersion;
int8_t dbReplica; int8_t dbReplica;
int8_t reserved[9]; int8_t dbType;
int8_t reserved[8];
} SVnodeCfg; } SVnodeCfg;
typedef struct { typedef struct {
......
src/inc/tp.h 0 → 100644
浏览文件 @ 4ec4714e
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef TDENGINE_TP
#define TDENGINE_TP
#ifdef __cplusplus
extern "C" {
#endif
int32_t tpInit();
void tpCleanUp();
void tpUpdateTs(int32_t *seq, void *pMsg);
#ifdef __cplusplus
}
#endif
#endif
src/inc/tsdb.h
浏览文件 @ 4ec4714e
...@@ -158,13 +158,18 @@ int32_t tsdbInsertData(STsdbRepo *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg *pR ...@@ -158,13 +158,18 @@ int32_t tsdbInsertData(STsdbRepo *repo, SSubmitMsg *pMsg, SShellSubmitRspMsg *pR
typedef void *TsdbQueryHandleT; // Use void to hide implementation details typedef void *TsdbQueryHandleT; // Use void to hide implementation details
// query condition to build vnode iterator #define BLOCK_LOAD_OFFSET_SEQ_ORDER 1
#define BLOCK_LOAD_TABLE_SEQ_ORDER 2
#define BLOCK_LOAD_TABLE_RR_ORDER 3
// query condition to build multi-table data block iterator
typedef struct STsdbQueryCond { typedef struct STsdbQueryCond {
STimeWindow twindow; STimeWindow twindow;
int32_t order; // desc|asc order to iterate the data block int32_t order; // desc|asc order to iterate the data block
int32_t numOfCols; int32_t numOfCols;
SColumnInfo *colList; SColumnInfo *colList;
bool loadExternalRows; // load external rows or not bool loadExternalRows; // load external rows or not
int32_t type; // data block load type:
} STsdbQueryCond; } STsdbQueryCond;
typedef struct SMemRef { typedef struct SMemRef {
...@@ -181,17 +186,31 @@ typedef struct SDataBlockInfo { ...@@ -181,17 +186,31 @@ typedef struct SDataBlockInfo {
int32_t tid; int32_t tid;
} SDataBlockInfo; } SDataBlockInfo;
typedef struct SFileBlockInfo {
int32_t numOfRows;
} SFileBlockInfo;
typedef struct { typedef struct {
void *pTable; void *pTable;
TSKEY lastKey; TSKEY lastKey;
} STableKeyInfo; } STableKeyInfo;
typedef struct { typedef struct {
size_t numOfTables; uint32_t numOfTables;
SArray * pGroupList; SArray * pGroupList;
SHashObj *map; // speedup acquire the tableQueryInfo by table uid SHashObj *map; // speedup acquire the tableQueryInfo by table uid
} STableGroupInfo; } STableGroupInfo;
typedef struct {
uint16_t rowSize;
uint16_t numOfFiles;
uint32_t numOfTables;
uint64_t totalSize;
int32_t firstSeekTimeUs;
uint32_t numOfRowsInMemTable;
SArray *dataBlockInfos;
} STableBlockDist;
/** /**
* Get the data block iterator, starting from position according to the query condition * Get the data block iterator, starting from position according to the query condition
* *
...@@ -252,16 +271,7 @@ int64_t tsdbGetNumOfRowsInMemTable(TsdbQueryHandleT* pHandle); ...@@ -252,16 +271,7 @@ int64_t tsdbGetNumOfRowsInMemTable(TsdbQueryHandleT* pHandle);
* @param pQueryHandle * @param pQueryHandle
* @return * @return
*/ */
bool tsdbNextDataBlock(TsdbQueryHandleT *pQueryHandle); bool tsdbNextDataBlock(TsdbQueryHandleT pQueryHandle);
/**
* move to next block if exists but not merge data in memtable
*
* @param pQueryHandle
* @return
*/
bool tsdbNextDataBlockWithoutMerge(TsdbQueryHandleT *pQueryHandle);
SArray* tsdbGetExternalRow(TsdbQueryHandleT *pHandle, SMemRef* pMemRef, int16_t type);
/** /**
* Get current data block information * Get current data block information
...@@ -306,7 +316,7 @@ int32_t tsdbQuerySTableByTagCond(STsdbRepo *tsdb, uint64_t uid, TSKEY key, const ...@@ -306,7 +316,7 @@ int32_t tsdbQuerySTableByTagCond(STsdbRepo *tsdb, uint64_t uid, TSKEY key, const
SColIndex *pColIndex, int32_t numOfCols); SColIndex *pColIndex, int32_t numOfCols);
/** /**
* destory the created table group list, which is generated by tag query * destroy the created table group list, which is generated by tag query
* @param pGroupList * @param pGroupList
*/ */
void tsdbDestroyTableGroup(STableGroupInfo *pGroupList); void tsdbDestroyTableGroup(STableGroupInfo *pGroupList);
...@@ -336,6 +346,12 @@ int32_t tsdbGetTableGroupFromIdList(STsdbRepo *tsdb, SArray *pTableIdList, STabl ...@@ -336,6 +346,12 @@ int32_t tsdbGetTableGroupFromIdList(STsdbRepo *tsdb, SArray *pTableIdList, STabl
*/ */
void tsdbCleanupQueryHandle(TsdbQueryHandleT queryHandle); void tsdbCleanupQueryHandle(TsdbQueryHandleT queryHandle);
void tsdbResetQueryHandle(TsdbQueryHandleT queryHandle, STsdbQueryCond *pCond);
void tsdbResetQueryHandleForNewTable(TsdbQueryHandleT queryHandle, STsdbQueryCond *pCond, STableGroupInfo* groupList);
int32_t tsdbGetFileBlocksDistInfo(TsdbQueryHandleT* queryHandle, STableBlockDist* pTableBlockInfo);
/** /**
* get the statistics of repo usage * get the statistics of repo usage
* @param repo. point to the tsdbrepo * @param repo. point to the tsdbrepo
......
src/inc/ttokendef.h
浏览文件 @ 4ec4714e
...@@ -62,175 +62,148 @@ ...@@ -62,175 +62,148 @@
#define TK_BITNOT 43 #define TK_BITNOT 43
#define TK_SHOW 44 #define TK_SHOW 44
#define TK_DATABASES 45 #define TK_DATABASES 45
#define TK_MNODES 46 #define TK_TOPICS 46
#define TK_DNODES 47 #define TK_MNODES 47
#define TK_ACCOUNTS 48 #define TK_DNODES 48
#define TK_USERS 49 #define TK_ACCOUNTS 49
#define TK_MODULES 50 #define TK_USERS 50
#define TK_QUERIES 51 #define TK_MODULES 51
#define TK_CONNECTIONS 52 #define TK_QUERIES 52
#define TK_STREAMS 53 #define TK_CONNECTIONS 53
#define TK_VARIABLES 54 #define TK_STREAMS 54
#define TK_SCORES 55 #define TK_VARIABLES 55
#define TK_GRANTS 56 #define TK_SCORES 56
#define TK_VNODES 57 #define TK_GRANTS 57
#define TK_IPTOKEN 58 #define TK_VNODES 58
#define TK_DOT 59 #define TK_IPTOKEN 59
#define TK_CREATE 60 #define TK_DOT 60
#define TK_TABLE 61 #define TK_CREATE 61
#define TK_DATABASE 62 #define TK_TABLE 62
#define TK_TABLES 63 #define TK_DATABASE 63
#define TK_STABLES 64 #define TK_TABLES 64
#define TK_VGROUPS 65 #define TK_STABLES 65
#define TK_DROP 66 #define TK_VGROUPS 66
#define TK_STABLE 67 #define TK_DROP 67
#define TK_DNODE 68 #define TK_STABLE 68
#define TK_USER 69 #define TK_TOPIC 69
#define TK_ACCOUNT 70 #define TK_DNODE 70
#define TK_USE 71 #define TK_USER 71
#define TK_DESCRIBE 72 #define TK_ACCOUNT 72
#define TK_ALTER 73 #define TK_USE 73
#define TK_PASS 74 #define TK_DESCRIBE 74
#define TK_PRIVILEGE 75 #define TK_ALTER 75
#define TK_LOCAL 76 #define TK_PASS 76
#define TK_IF 77 #define TK_PRIVILEGE 77
#define TK_EXISTS 78 #define TK_LOCAL 78
#define TK_PPS 79 #define TK_IF 79
#define TK_TSERIES 80 #define TK_EXISTS 80
#define TK_DBS 81 #define TK_PPS 81
#define TK_STORAGE 82 #define TK_TSERIES 82
#define TK_QTIME 83 #define TK_DBS 83
#define TK_CONNS 84 #define TK_STORAGE 84
#define TK_STATE 85 #define TK_QTIME 85
#define TK_KEEP 86 #define TK_CONNS 86
#define TK_CACHE 87 #define TK_STATE 87
#define TK_REPLICA 88 #define TK_KEEP 88
#define TK_QUORUM 89 #define TK_CACHE 89
#define TK_DAYS 90 #define TK_REPLICA 90
#define TK_MINROWS 91 #define TK_QUORUM 91
#define TK_MAXROWS 92 #define TK_DAYS 92
#define TK_BLOCKS 93 #define TK_MINROWS 93
#define TK_CTIME 94 #define TK_MAXROWS 94
#define TK_WAL 95 #define TK_BLOCKS 95
#define TK_FSYNC 96 #define TK_CTIME 96
#define TK_COMP 97 #define TK_WAL 97
#define TK_PRECISION 98 #define TK_FSYNC 98
#define TK_UPDATE 99 #define TK_COMP 99
#define TK_CACHELAST 100 #define TK_PRECISION 100
#define TK_LP 101 #define TK_UPDATE 101
#define TK_RP 102 #define TK_CACHELAST 102
#define TK_UNSIGNED 103 #define TK_PARTITIONS 103
#define TK_TAGS 104 #define TK_LP 104
#define TK_USING 105 #define TK_RP 105
#define TK_COMMA 106 #define TK_UNSIGNED 106
#define TK_AS 107 #define TK_TAGS 107
#define TK_NULL 108 #define TK_USING 108
#define TK_SELECT 109 #define TK_COMMA 109
#define TK_UNION 110 #define TK_AS 110
#define TK_ALL 111 #define TK_NULL 111
#define TK_DISTINCT 112 #define TK_SELECT 112
#define TK_FROM 113 #define TK_UNION 113
#define TK_VARIABLE 114 #define TK_ALL 114
#define TK_INTERVAL 115 #define TK_DISTINCT 115
#define TK_FILL 116 #define TK_FROM 116
#define TK_SLIDING 117 #define TK_VARIABLE 117
#define TK_ORDER 118 #define TK_INTERVAL 118
#define TK_BY 119 #define TK_SESSION 119
#define TK_ASC 120 #define TK_FILL 120
#define TK_DESC 121 #define TK_SLIDING 121
#define TK_GROUP 122 #define TK_ORDER 122
#define TK_HAVING 123 #define TK_BY 123
#define TK_LIMIT 124 #define TK_ASC 124
#define TK_OFFSET 125 #define TK_DESC 125
#define TK_SLIMIT 126 #define TK_GROUP 126
#define TK_SOFFSET 127 #define TK_HAVING 127
#define TK_WHERE 128 #define TK_LIMIT 128
#define TK_NOW 129 #define TK_OFFSET 129
#define TK_RESET 130 #define TK_SLIMIT 130
#define TK_QUERY 131 #define TK_SOFFSET 131
#define TK_ADD 132 #define TK_WHERE 132
#define TK_COLUMN 133 #define TK_NOW 133
#define TK_TAG 134 #define TK_RESET 134
#define TK_CHANGE 135 #define TK_QUERY 135
#define TK_SET 136 #define TK_ADD 136
#define TK_KILL 137 #define TK_COLUMN 137
#define TK_CONNECTION 138 #define TK_TAG 138
#define TK_STREAM 139 #define TK_CHANGE 139
#define TK_COLON 140 #define TK_SET 140
#define TK_ABORT 141 #define TK_KILL 141
#define TK_AFTER 142 #define TK_CONNECTION 142
#define TK_ATTACH 143 #define TK_STREAM 143
#define TK_BEFORE 144 #define TK_COLON 144
#define TK_BEGIN 145 #define TK_ABORT 145
#define TK_CASCADE 146 #define TK_AFTER 146
#define TK_CLUSTER 147 #define TK_ATTACH 147
#define TK_CONFLICT 148 #define TK_BEFORE 148
#define TK_COPY 149 #define TK_BEGIN 149
#define TK_DEFERRED 150 #define TK_CASCADE 150
#define TK_DELIMITERS 151 #define TK_CLUSTER 151
#define TK_DETACH 152 #define TK_CONFLICT 152
#define TK_EACH 153 #define TK_COPY 153
#define TK_END 154 #define TK_DEFERRED 154
#define TK_EXPLAIN 155 #define TK_DELIMITERS 155
#define TK_FAIL 156 #define TK_DETACH 156
#define TK_FOR 157 #define TK_EACH 157
#define TK_IGNORE 158 #define TK_END 158
#define TK_IMMEDIATE 159 #define TK_EXPLAIN 159
#define TK_INITIALLY 160 #define TK_FAIL 160
#define TK_INSTEAD 161 #define TK_FOR 161
#define TK_MATCH 162 #define TK_IGNORE 162
#define TK_KEY 163 #define TK_IMMEDIATE 163
#define TK_OF 164 #define TK_INITIALLY 164
#define TK_RAISE 165 #define TK_INSTEAD 165
#define TK_REPLACE 166 #define TK_MATCH 166
#define TK_RESTRICT 167 #define TK_KEY 167
#define TK_ROW 168 #define TK_OF 168
#define TK_STATEMENT 169 #define TK_RAISE 169
#define TK_TRIGGER 170 #define TK_REPLACE 170
#define TK_VIEW 171 #define TK_RESTRICT 171
#define TK_COUNT 172 #define TK_ROW 172
#define TK_SUM 173 #define TK_STATEMENT 173
#define TK_AVG 174 #define TK_TRIGGER 174
#define TK_MIN 175 #define TK_VIEW 175
#define TK_MAX 176 #define TK_SEMI 176
#define TK_FIRST 177 #define TK_NONE 177
#define TK_LAST 178 #define TK_PREV 178
#define TK_TOP 179 #define TK_LINEAR 179
#define TK_BOTTOM 180 #define TK_IMPORT 180
#define TK_STDDEV 181 #define TK_METRIC 181
#define TK_PERCENTILE 182 #define TK_TBNAME 182
#define TK_APERCENTILE 183 #define TK_JOIN 183
#define TK_LEASTSQUARES 184 #define TK_METRICS 184
#define TK_HISTOGRAM 185 #define TK_INSERT 185
#define TK_DIFF 186 #define TK_INTO 186
#define TK_SPREAD 187 #define TK_VALUES 187
#define TK_TWA 188
#define TK_INTERP 189
#define TK_LAST_ROW 190
#define TK_RATE 191
#define TK_IRATE 192
#define TK_SUM_RATE 193
#define TK_SUM_IRATE 194
#define TK_AVG_RATE 195
#define TK_AVG_IRATE 196
#define TK_TBID 197
#define TK_SEMI 198
#define TK_NONE 199
#define TK_PREV 200
#define TK_LINEAR 201
#define TK_IMPORT 202
#define TK_METRIC 203
#define TK_TBNAME 204
#define TK_JOIN 205
#define TK_METRICS 206
#define TK_INSERT 207
#define TK_INTO 208
#define TK_VALUES 209
#define TK_SPACE 300 #define TK_SPACE 300
......
src/inc/ttype.h
浏览文件 @ 4ec4714e
...@@ -171,10 +171,10 @@ extern tDataTypeDescriptor tDataTypes[15]; ...@@ -171,10 +171,10 @@ extern tDataTypeDescriptor tDataTypes[15];
bool isValidDataType(int32_t type); bool isValidDataType(int32_t type);
void setVardataNull(char* val, int32_t type); void setVardataNull(char* val, int32_t type);
void setNull(char *val, int32_t type, int32_t bytes); void setNull(char *val, int32_t type, int32_t bytes);
void setNullN(char *val, int32_t type, int32_t bytes, int32_t numOfElems); void setNullN(char *val, int32_t type, int32_t bytes, int32_t numOfElems);
void* getNullValue(int32_t type); void *getNullValue(int32_t type);
void assignVal(char *val, const char *src, int32_t len, int32_t type); void assignVal(char *val, const char *src, int32_t len, int32_t type);
void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size, void* buf); void tsDataSwap(void *pLeft, void *pRight, int32_t type, int32_t size, void* buf);
......
src/kit/taosdemo/insert.json
浏览文件 @ 4ec4714e
...@@ -11,6 +11,7 @@ ...@@ -11,6 +11,7 @@
"confirm_parameter_prompt": "no", "confirm_parameter_prompt": "no",
"insert_interval": 0, "insert_interval": 0,
"num_of_records_per_req": 100, "num_of_records_per_req": 100,
"max_sql_len": 1024000,
"databases": [{ "databases": [{
"dbinfo": { "dbinfo": {
"name": "db", "name": "db",
...@@ -38,7 +39,9 @@ ...@@ -38,7 +39,9 @@
"auto_create_table": "no", "auto_create_table": "no",
"data_source": "rand", "data_source": "rand",
"insert_mode": "taosc", "insert_mode": "taosc",
"insert_rows": 100000, "childtable_limit": 33,
"childtable_offset": 33,
"insert_rows": 1000,
"multi_thread_write_one_tbl": "no", "multi_thread_write_one_tbl": "no",
"number_of_tbl_in_one_sql": 0, "number_of_tbl_in_one_sql": 0,
"rows_per_tbl": 100, "rows_per_tbl": 100,
......
src/kit/taosdemo/taosdemo.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/mnode/inc/mnodeDef.h
浏览文件 @ 4ec4714e
...@@ -175,7 +175,9 @@ typedef struct { ...@@ -175,7 +175,9 @@ typedef struct {
int8_t quorum; int8_t quorum;
int8_t update; int8_t update;
int8_t cacheLastRow; int8_t cacheLastRow;
int8_t reserved[10]; int8_t dbType;
int16_t partitions;
int8_t reserved[7];
} SDbCfg; } SDbCfg;
typedef struct SDbObj { typedef struct SDbObj {
......
src/mnode/src/mnodeDb.c
浏览文件 @ 4ec4714e
...@@ -22,6 +22,7 @@ ...@@ -22,6 +22,7 @@
#include "tname.h" #include "tname.h"
#include "tbn.h" #include "tbn.h"
#include "tdataformat.h" #include "tdataformat.h"
#include "tp.h"
#include "mnode.h" #include "mnode.h"
#include "mnodeDef.h" #include "mnodeDef.h"
#include "mnodeInt.h" #include "mnodeInt.h"
...@@ -38,8 +39,8 @@ ...@@ -38,8 +39,8 @@
#include "mnodeVgroup.h" #include "mnodeVgroup.h"
#define VG_LIST_SIZE 8 #define VG_LIST_SIZE 8
int64_t tsDbRid = -1; int64_t tsDbRid = -1;
static void * tsDbSdb = NULL; void * tsDbSdb = NULL;
static int32_t tsDbUpdateSize; static int32_t tsDbUpdateSize;
static int32_t mnodeCreateDb(SAcctObj *pAcct, SCreateDbMsg *pCreate, SMnodeMsg *pMsg); static int32_t mnodeCreateDb(SAcctObj *pAcct, SCreateDbMsg *pCreate, SMnodeMsg *pMsg);
...@@ -48,8 +49,14 @@ static int32_t mnodeSetDbDropping(SDbObj *pDb); ...@@ -48,8 +49,14 @@ static int32_t mnodeSetDbDropping(SDbObj *pDb);
static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn); static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn);
static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn); static int32_t mnodeRetrieveDbs(SShowObj *pShow, char *data, int32_t rows, void *pConn);
static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg); static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg);
static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg);
static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg); static int32_t mnodeProcessDropDbMsg(SMnodeMsg *pMsg);
int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg);
#ifndef _TOPIC
int32_t tpInit() { return 0; }
void tpCleanUp() {}
void tpUpdateTs(int32_t *seq, void *pMsg) {}
#endif
static void mnodeDestroyDb(SDbObj *pDb) { static void mnodeDestroyDb(SDbObj *pDb) {
pthread_mutex_destroy(&pDb->mutex); pthread_mutex_destroy(&pDb->mutex);
...@@ -176,7 +183,7 @@ int32_t mnodeInitDbs() { ...@@ -176,7 +183,7 @@ int32_t mnodeInitDbs() {
mnodeAddShowFreeIterHandle(TSDB_MGMT_TABLE_DB, mnodeCancelGetNextDb); mnodeAddShowFreeIterHandle(TSDB_MGMT_TABLE_DB, mnodeCancelGetNextDb);
mDebug("table:dbs table is created"); mDebug("table:dbs table is created");
return 0; return tpInit();
} }
void *mnodeGetNextDb(void *pIter, SDbObj **pDb) { void *mnodeGetNextDb(void *pIter, SDbObj **pDb) {
...@@ -332,6 +339,17 @@ static int32_t mnodeCheckDbCfg(SDbCfg *pCfg) { ...@@ -332,6 +339,17 @@ static int32_t mnodeCheckDbCfg(SDbCfg *pCfg) {
return TSDB_CODE_MND_INVALID_DB_OPTION; return TSDB_CODE_MND_INVALID_DB_OPTION;
} }
if (pCfg->dbType < 0 || pCfg->dbType > 1) {
mError("invalid db option dbType:%d valid range: [%d, %d]", pCfg->dbType, 0, 1);
return TSDB_CODE_MND_INVALID_DB_OPTION;
}
if (pCfg->partitions < TSDB_MIN_DB_PARTITON_OPTION || pCfg->partitions > TSDB_MAX_DB_PARTITON_OPTION) {
mError("invalid db option partitions:%d valid range: [%d, %d]", pCfg->partitions, TSDB_MIN_DB_PARTITON_OPTION,
TSDB_MAX_DB_PARTITON_OPTION);
return TSDB_CODE_MND_INVALID_DB_OPTION;
}
return TSDB_CODE_SUCCESS; return TSDB_CODE_SUCCESS;
} }
...@@ -354,6 +372,8 @@ static void mnodeSetDefaultDbCfg(SDbCfg *pCfg) { ...@@ -354,6 +372,8 @@ static void mnodeSetDefaultDbCfg(SDbCfg *pCfg) {
if (pCfg->quorum < 0) pCfg->quorum = tsQuorum; if (pCfg->quorum < 0) pCfg->quorum = tsQuorum;
if (pCfg->update < 0) pCfg->update = tsUpdate; if (pCfg->update < 0) pCfg->update = tsUpdate;
if (pCfg->cacheLastRow < 0) pCfg->cacheLastRow = tsCacheLastRow; if (pCfg->cacheLastRow < 0) pCfg->cacheLastRow = tsCacheLastRow;
if (pCfg->dbType < 0) pCfg->dbType = 0;
if (pCfg->partitions < 0) pCfg->partitions = tsPartitons;
} }
static int32_t mnodeCreateDbCb(SMnodeMsg *pMsg, int32_t code) { static int32_t mnodeCreateDbCb(SMnodeMsg *pMsg, int32_t code) {
...@@ -408,7 +428,9 @@ static int32_t mnodeCreateDb(SAcctObj *pAcct, SCreateDbMsg *pCreate, SMnodeMsg * ...@@ -408,7 +428,9 @@ static int32_t mnodeCreateDb(SAcctObj *pAcct, SCreateDbMsg *pCreate, SMnodeMsg *
.replications = pCreate->replications, .replications = pCreate->replications,
.quorum = pCreate->quorum, .quorum = pCreate->quorum,
.update = pCreate->update, .update = pCreate->update,
.cacheLastRow = pCreate->cacheLastRow .cacheLastRow = pCreate->cacheLastRow,
.dbType = pCreate->dbType,
.partitions = pCreate->partitions
}; };
mnodeSetDefaultDbCfg(&pDb->cfg); mnodeSetDefaultDbCfg(&pDb->cfg);
...@@ -501,6 +523,7 @@ void mnodeRemoveVgroupFromDb(SVgObj *pVgroup) { ...@@ -501,6 +523,7 @@ void mnodeRemoveVgroupFromDb(SVgObj *pVgroup) {
} }
void mnodeCleanupDbs() { void mnodeCleanupDbs() {
tpCleanUp();
sdbCloseTable(tsDbRid); sdbCloseTable(tsDbRid);
tsDbSdb = NULL; tsDbSdb = NULL;
} }
...@@ -660,7 +683,7 @@ static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn ...@@ -660,7 +683,7 @@ static int32_t mnodeGetDbMeta(STableMetaMsg *pMeta, SShowObj *pShow, void *pConn
return 0; return 0;
} }
static char *mnodeGetDbStr(char *src) { char *mnodeGetDbStr(char *src) {
char *pos = strstr(src, TS_PATH_DELIMITER); char *pos = strstr(src, TS_PATH_DELIMITER);
if (pos != NULL) ++pos; if (pos != NULL) ++pos;
...@@ -852,6 +875,7 @@ static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg) { ...@@ -852,6 +875,7 @@ static int32_t mnodeProcessCreateDbMsg(SMnodeMsg *pMsg) {
pCreate->daysToKeep2 = htonl(pCreate->daysToKeep2); pCreate->daysToKeep2 = htonl(pCreate->daysToKeep2);
pCreate->commitTime = htonl(pCreate->commitTime); pCreate->commitTime = htonl(pCreate->commitTime);
pCreate->fsyncPeriod = htonl(pCreate->fsyncPeriod); pCreate->fsyncPeriod = htonl(pCreate->fsyncPeriod);
pCreate->partitions = htons(pCreate->partitions);
pCreate->minRowsPerFileBlock = htonl(pCreate->minRowsPerFileBlock); pCreate->minRowsPerFileBlock = htonl(pCreate->minRowsPerFileBlock);
pCreate->maxRowsPerFileBlock = htonl(pCreate->maxRowsPerFileBlock); pCreate->maxRowsPerFileBlock = htonl(pCreate->maxRowsPerFileBlock);
...@@ -887,6 +911,8 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) { ...@@ -887,6 +911,8 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) {
int8_t precision = pAlter->precision; int8_t precision = pAlter->precision;
int8_t update = pAlter->update; int8_t update = pAlter->update;
int8_t cacheLastRow = pAlter->cacheLastRow; int8_t cacheLastRow = pAlter->cacheLastRow;
int8_t dbType = pAlter->dbType;
int16_t partitions = htons(pAlter->partitions);
terrno = TSDB_CODE_SUCCESS; terrno = TSDB_CODE_SUCCESS;
...@@ -1004,6 +1030,16 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) { ...@@ -1004,6 +1030,16 @@ static SDbCfg mnodeGetAlterDbOption(SDbObj *pDb, SAlterDbMsg *pAlter) {
newCfg.cacheLastRow = cacheLastRow; newCfg.cacheLastRow = cacheLastRow;
} }
if (dbType >= 0 && dbType != pDb->cfg.dbType) {
mDebug("db:%s, dbType:%d change to %d", pDb->name, pDb->cfg.dbType, dbType);
newCfg.dbType = dbType;
}
if (partitions >= 0 && partitions != pDb->cfg.partitions) {
mDebug("db:%s, partitions:%d change to %d", pDb->name, pDb->cfg.partitions, partitions);
newCfg.partitions = partitions;
}
return newCfg; return newCfg;
} }
...@@ -1031,6 +1067,8 @@ static int32_t mnodeAlterDbCb(SMnodeMsg *pMsg, int32_t code) { ...@@ -1031,6 +1067,8 @@ static int32_t mnodeAlterDbCb(SMnodeMsg *pMsg, int32_t code) {
} }
static int32_t mnodeAlterDb(SDbObj *pDb, SAlterDbMsg *pAlter, void *pMsg) { static int32_t mnodeAlterDb(SDbObj *pDb, SAlterDbMsg *pAlter, void *pMsg) {
mDebug("db:%s, type:%d do alter operation", pDb->name, pDb->cfg.dbType);
SDbCfg newCfg = mnodeGetAlterDbOption(pDb, pAlter); SDbCfg newCfg = mnodeGetAlterDbOption(pDb, pAlter);
if (terrno != TSDB_CODE_SUCCESS) { if (terrno != TSDB_CODE_SUCCESS) {
return terrno; return terrno;
...@@ -1061,9 +1099,9 @@ static int32_t mnodeAlterDb(SDbObj *pDb, SAlterDbMsg *pAlter, void *pMsg) { ...@@ -1061,9 +1099,9 @@ static int32_t mnodeAlterDb(SDbObj *pDb, SAlterDbMsg *pAlter, void *pMsg) {
return code; return code;
} }
static int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) { int32_t mnodeProcessAlterDbMsg(SMnodeMsg *pMsg) {
SAlterDbMsg *pAlter = pMsg->rpcMsg.pCont; SAlterDbMsg *pAlter = pMsg->rpcMsg.pCont;
mDebug("db:%s, alter db msg is received from thandle:%p", pAlter->db, pMsg->rpcMsg.handle); mDebug("db:%s, alter db msg is received from thandle:%p, dbType:%d", pAlter->db, pMsg->rpcMsg.handle, pAlter->dbType);
if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pAlter->db); if (pMsg->pDb == NULL) pMsg->pDb = mnodeGetDb(pAlter->db);
if (pMsg->pDb == NULL) { if (pMsg->pDb == NULL) {
......
src/mnode/src/mnodeShow.c
浏览文件 @ 4ec4714e
...@@ -109,6 +109,7 @@ static char *mnodeGetShowType(int32_t showType) { ...@@ -109,6 +109,7 @@ static char *mnodeGetShowType(int32_t showType) {
case TSDB_MGMT_TABLE_VNODES: return "show vnodes"; case TSDB_MGMT_TABLE_VNODES: return "show vnodes";
case TSDB_MGMT_TABLE_CLUSTER: return "show clusters"; case TSDB_MGMT_TABLE_CLUSTER: return "show clusters";
case TSDB_MGMT_TABLE_STREAMTABLES : return "show streamtables"; case TSDB_MGMT_TABLE_STREAMTABLES : return "show streamtables";
case TSDB_MGMT_TABLE_TP: return "show topics";
default: return "undefined"; default: return "undefined";
} }
} }
......
src/mnode/src/mnodeVgroup.c
浏览文件 @ 4ec4714e
...@@ -367,6 +367,11 @@ static int32_t mnodeAllocVgroupIdPool(SVgObj *pInputVgroup) { ...@@ -367,6 +367,11 @@ static int32_t mnodeAllocVgroupIdPool(SVgObj *pInputVgroup) {
maxIdPoolSize = MAX(maxIdPoolSize, idPoolSize); maxIdPoolSize = MAX(maxIdPoolSize, idPoolSize);
} }
// create one table each vnode
if (pDb->cfg.dbType == TSDB_DB_TYPE_TOPIC) {
maxIdPoolSize = 1;
}
// new vgroup // new vgroup
if (pInputVgroup->idPool == NULL) { if (pInputVgroup->idPool == NULL) {
pInputVgroup->idPool = taosInitIdPool(maxIdPoolSize); pInputVgroup->idPool = taosInitIdPool(maxIdPoolSize);
...@@ -379,6 +384,11 @@ static int32_t mnodeAllocVgroupIdPool(SVgObj *pInputVgroup) { ...@@ -379,6 +384,11 @@ static int32_t mnodeAllocVgroupIdPool(SVgObj *pInputVgroup) {
} }
} }
// create one table each vnode
if (pDb->cfg.dbType == TSDB_DB_TYPE_TOPIC) {
return TSDB_CODE_SUCCESS;
}
// realloc all vgroups in db // realloc all vgroups in db
int32_t newIdPoolSize; int32_t newIdPoolSize;
if (minIdPoolSize * 4 < tsTableIncStepPerVnode) { if (minIdPoolSize * 4 < tsTableIncStepPerVnode) {
...@@ -449,6 +459,10 @@ int32_t mnodeGetAvailableVgroup(SMnodeMsg *pMsg, SVgObj **ppVgroup, int32_t *pSi ...@@ -449,6 +459,10 @@ int32_t mnodeGetAvailableVgroup(SMnodeMsg *pMsg, SVgObj **ppVgroup, int32_t *pSi
maxVgroupsPerDb = MIN(maxVgroupsPerDb, TSDB_MAX_VNODES_PER_DB); maxVgroupsPerDb = MIN(maxVgroupsPerDb, TSDB_MAX_VNODES_PER_DB);
} }
if (pDb->cfg.dbType == TSDB_DB_TYPE_TOPIC) {
maxVgroupsPerDb = TSDB_MAX_DB_PARTITON_OPTION;
}
int32_t code = TSDB_CODE_MND_NO_ENOUGH_DNODES; int32_t code = TSDB_CODE_MND_NO_ENOUGH_DNODES;
if (pDb->numOfVgroups < maxVgroupsPerDb) { if (pDb->numOfVgroups < maxVgroupsPerDb) {
mDebug("msg:%p, app:%p db:%s, try to create a new vgroup, numOfVgroups:%d maxVgroupsPerDb:%d", pMsg, mDebug("msg:%p, app:%p db:%s, try to create a new vgroup, numOfVgroups:%d maxVgroupsPerDb:%d", pMsg,
...@@ -881,6 +895,7 @@ static SCreateVnodeMsg *mnodeBuildVnodeMsg(SVgObj *pVgroup) { ...@@ -881,6 +895,7 @@ static SCreateVnodeMsg *mnodeBuildVnodeMsg(SVgObj *pVgroup) {
pCfg->update = pDb->cfg.update; pCfg->update = pDb->cfg.update;
pCfg->cacheLastRow = pDb->cfg.cacheLastRow; pCfg->cacheLastRow = pDb->cfg.cacheLastRow;
pCfg->dbReplica = pDb->cfg.replications; pCfg->dbReplica = pDb->cfg.replications;
pCfg->dbType = pDb->cfg.dbType;
SVnodeDesc *pNodes = pVnode->nodes; SVnodeDesc *pNodes = pVnode->nodes;
for (int32_t j = 0; j < pVgroup->numOfVnodes; ++j) { for (int32_t j = 0; j < pVgroup->numOfVnodes; ++j) {
......
src/os/inc/osTime.h
浏览文件 @ 4ec4714e
...@@ -72,6 +72,11 @@ typedef struct SInterval { ...@@ -72,6 +72,11 @@ typedef struct SInterval {
int64_t offset; int64_t offset;
} SInterval; } SInterval;
typedef struct SSessionWindow {
int64_t gap; // gap between two session window(in microseconds)
int32_t primaryColId; // primary timestamp column
} SSessionWindow;
int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision); int64_t taosTimeAdd(int64_t t, int64_t duration, char unit, int32_t precision);
int64_t taosTimeTruncate(int64_t t, const SInterval* pInterval, int32_t precision); int64_t taosTimeTruncate(int64_t t, const SInterval* pInterval, int32_t precision);
int32_t taosTimeCountInterval(int64_t skey, int64_t ekey, int64_t interval, char unit, int32_t precision); int32_t taosTimeCountInterval(int64_t skey, int64_t ekey, int64_t interval, char unit, int32_t precision);
......
src/plugins/http/inc/httpJson.h
浏览文件 @ 4ec4714e
...@@ -63,9 +63,11 @@ void httpJsonString(JsonBuf* buf, char* sVal, int32_t len); ...@@ -63,9 +63,11 @@ void httpJsonString(JsonBuf* buf, char* sVal, int32_t len);
void httpJsonOriginString(JsonBuf* buf, char* sVal, int32_t len); void httpJsonOriginString(JsonBuf* buf, char* sVal, int32_t len);
void httpJsonStringForTransMean(JsonBuf* buf, char* SVal, int32_t maxLen); void httpJsonStringForTransMean(JsonBuf* buf, char* SVal, int32_t maxLen);
void httpJsonInt64(JsonBuf* buf, int64_t num); void httpJsonInt64(JsonBuf* buf, int64_t num);
void httpJsonUInt64(JsonBuf* buf, uint64_t num);
void httpJsonTimestamp(JsonBuf* buf, int64_t t, bool us); void httpJsonTimestamp(JsonBuf* buf, int64_t t, bool us);
void httpJsonUtcTimestamp(JsonBuf* buf, int64_t t, bool us); void httpJsonUtcTimestamp(JsonBuf* buf, int64_t t, bool us);
void httpJsonInt(JsonBuf* buf, int32_t num); void httpJsonInt(JsonBuf* buf, int32_t num);
void httpJsonUInt(JsonBuf* buf, uint32_t num);
void httpJsonFloat(JsonBuf* buf, float num); void httpJsonFloat(JsonBuf* buf, float num);
void httpJsonDouble(JsonBuf* buf, double num); void httpJsonDouble(JsonBuf* buf, double num);
void httpJsonNull(JsonBuf* buf); void httpJsonNull(JsonBuf* buf);
......
src/plugins/http/src/httpJson.c
浏览文件 @ 4ec4714e
...@@ -256,6 +256,12 @@ void httpJsonInt64(JsonBuf* buf, int64_t num) { ...@@ -256,6 +256,12 @@ void httpJsonInt64(JsonBuf* buf, int64_t num) {
buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%" PRId64, num); buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%" PRId64, num);
} }
void httpJsonUInt64(JsonBuf* buf, uint64_t num) {
httpJsonItemToken(buf);
httpJsonTestBuf(buf, MAX_NUM_STR_SZ);
buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%" PRIu64, num);
}
void httpJsonTimestamp(JsonBuf* buf, int64_t t, bool us) { void httpJsonTimestamp(JsonBuf* buf, int64_t t, bool us) {
char ts[35] = {0}; char ts[35] = {0};
struct tm* ptm; struct tm* ptm;
...@@ -303,6 +309,12 @@ void httpJsonInt(JsonBuf* buf, int32_t num) { ...@@ -303,6 +309,12 @@ void httpJsonInt(JsonBuf* buf, int32_t num) {
buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%d", num); buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%d", num);
} }
void httpJsonUInt(JsonBuf* buf, uint32_t num) {
httpJsonItemToken(buf);
httpJsonTestBuf(buf, MAX_NUM_STR_SZ);
buf->lst += snprintf(buf->lst, MAX_NUM_STR_SZ, "%u", num);
}
void httpJsonFloat(JsonBuf* buf, float num) { void httpJsonFloat(JsonBuf* buf, float num) {
httpJsonItemToken(buf); httpJsonItemToken(buf);
httpJsonTestBuf(buf, MAX_NUM_STR_SZ); httpJsonTestBuf(buf, MAX_NUM_STR_SZ);
......
src/plugins/http/src/httpRestJson.c
浏览文件 @ 4ec4714e
...@@ -162,6 +162,18 @@ bool restBuildSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result, ...@@ -162,6 +162,18 @@ bool restBuildSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result,
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
httpJsonInt64(jsonBuf, *((int64_t *)row[i])); httpJsonInt64(jsonBuf, *((int64_t *)row[i]));
break; break;
case TSDB_DATA_TYPE_UTINYINT:
httpJsonUInt(jsonBuf, *((uint8_t *)row[i]));
break;
case TSDB_DATA_TYPE_USMALLINT:
httpJsonUInt(jsonBuf, *((uint16_t *)row[i]));
break;
case TSDB_DATA_TYPE_UINT:
httpJsonUInt(jsonBuf, *((uint32_t *)row[i]));
break;
case TSDB_DATA_TYPE_UBIGINT:
httpJsonUInt64(jsonBuf, *((uint64_t *)row[i]));
break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
httpJsonFloat(jsonBuf, GET_FLOAT_VAL(row[i])); httpJsonFloat(jsonBuf, GET_FLOAT_VAL(row[i]));
break; break;
......
src/query/inc/qAggMain.h
浏览文件 @ 4ec4714e
...@@ -26,6 +26,7 @@ extern "C" { ...@@ -26,6 +26,7 @@ extern "C" {
#include "taosdef.h" #include "taosdef.h"
#include "trpc.h" #include "trpc.h"
#include "tvariant.h" #include "tvariant.h"
#include "tsdb.h"
#define TSDB_FUNC_INVALID_ID -1 #define TSDB_FUNC_INVALID_ID -1
#define TSDB_FUNC_COUNT 0 #define TSDB_FUNC_COUNT 0
...@@ -70,15 +71,17 @@ extern "C" { ...@@ -70,15 +71,17 @@ extern "C" {
#define TSDB_FUNC_AVG_IRATE 34 #define TSDB_FUNC_AVG_IRATE 34
#define TSDB_FUNC_TID_TAG 35 #define TSDB_FUNC_TID_TAG 35
#define TSDB_FUNC_HISTOGRAM 36 #define TSDB_FUNC_BLKINFO 36
#define TSDB_FUNC_HLL 37
#define TSDB_FUNC_MODE 38 #define TSDB_FUNC_HISTOGRAM 37
#define TSDB_FUNC_SAMPLE 39 #define TSDB_FUNC_HLL 38
#define TSDB_FUNC_CEIL 40 #define TSDB_FUNC_MODE 39
#define TSDB_FUNC_FLOOR 41 #define TSDB_FUNC_SAMPLE 40
#define TSDB_FUNC_ROUND 42 #define TSDB_FUNC_CEIL 41
#define TSDB_FUNC_MAVG 43 #define TSDB_FUNC_FLOOR 42
#define TSDB_FUNC_CSUM 44 #define TSDB_FUNC_ROUND 43
#define TSDB_FUNC_MAVG 44
#define TSDB_FUNC_CSUM 45
#define TSDB_FUNCSTATE_SO 0x1u // single output #define TSDB_FUNCSTATE_SO 0x1u // single output
...@@ -214,13 +217,14 @@ typedef struct SAggFunctionInfo { ...@@ -214,13 +217,14 @@ typedef struct SAggFunctionInfo {
void (*xFinalize)(SQLFunctionCtx *pCtx); void (*xFinalize)(SQLFunctionCtx *pCtx);
void (*mergeFunc)(SQLFunctionCtx *pCtx); void (*mergeFunc)(SQLFunctionCtx *pCtx);
int32_t (*dataReqFunc)(SQLFunctionCtx *pCtx, TSKEY start, TSKEY end, int32_t colId); int32_t (*dataReqFunc)(SQLFunctionCtx *pCtx, STimeWindow* w, int32_t colId);
} SAggFunctionInfo; } SAggFunctionInfo;
#define GET_RES_INFO(ctx) ((ctx)->resultInfo) #define GET_RES_INFO(ctx) ((ctx)->resultInfo)
int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionId, int32_t param, int16_t *type, int32_t getResultDataInfo(int32_t dataType, int32_t dataBytes, int32_t functionId, int32_t param, int16_t *type,
int16_t *len, int32_t *interBytes, int16_t extLength, bool isSuperTable); int16_t *len, int32_t *interBytes, int16_t extLength, bool isSuperTable);
int32_t isValidFunction(const char* name, int32_t len);
#define IS_STREAM_QUERY_VALID(x) (((x)&TSDB_FUNCSTATE_STREAM) != 0) #define IS_STREAM_QUERY_VALID(x) (((x)&TSDB_FUNCSTATE_STREAM) != 0)
#define IS_MULTIOUTPUT(x) (((x)&TSDB_FUNCSTATE_MO) != 0) #define IS_MULTIOUTPUT(x) (((x)&TSDB_FUNCSTATE_MO) != 0)
...@@ -242,12 +246,16 @@ typedef struct STwaInfo { ...@@ -242,12 +246,16 @@ typedef struct STwaInfo {
STimeWindow win; STimeWindow win;
} STwaInfo; } STwaInfo;
struct SBufferWriter;
void blockDistInfoToBinary(STableBlockDist* pDist, struct SBufferWriter* bw);
void blockDistInfoFromBinary(const char* data, int32_t len, STableBlockDist* pDist);
/* global sql function array */ /* global sql function array */
extern struct SAggFunctionInfo aAggs[]; extern struct SAggFunctionInfo aAggs[];
extern int32_t functionCompatList[]; // compatible check array list extern int32_t functionCompatList[]; // compatible check array list
bool topbot_datablock_filter(SQLFunctionCtx *pCtx, int32_t functionId, const char *minval, const char *maxval); bool topbot_datablock_filter(SQLFunctionCtx *pCtx, const char *minval, const char *maxval);
/** /**
* the numOfRes should be kept, since it may be used later * the numOfRes should be kept, since it may be used later
...@@ -258,14 +266,14 @@ bool topbot_datablock_filter(SQLFunctionCtx *pCtx, int32_t functionId, const cha ...@@ -258,14 +266,14 @@ bool topbot_datablock_filter(SQLFunctionCtx *pCtx, int32_t functionId, const cha
(_r)->initialized = false; \ (_r)->initialized = false; \
} while (0) } while (0)
static FORCE_INLINE void initResultInfo(SResultRowCellInfo *pResInfo, uint32_t bufLen) { static FORCE_INLINE void initResultInfo(SResultRowCellInfo *pResInfo, int32_t bufLen) {
pResInfo->initialized = true; // the this struct has been initialized flag pResInfo->initialized = true; // the this struct has been initialized flag
pResInfo->complete = false; pResInfo->complete = false;
pResInfo->hasResult = false; pResInfo->hasResult = false;
pResInfo->numOfRes = 0; pResInfo->numOfRes = 0;
memset(GET_ROWCELL_INTERBUF(pResInfo), 0, (size_t)bufLen); memset(GET_ROWCELL_INTERBUF(pResInfo), 0, bufLen);
} }
#ifdef __cplusplus #ifdef __cplusplus
......
src/query/inc/qExecutor.h
浏览文件 @ 4ec4714e
...@@ -12,8 +12,8 @@ ...@@ -12,8 +12,8 @@
* You should have received a copy of the GNU Affero General Public License * You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#ifndef TDENGINE_QUERYEXECUTOR_H #ifndef TDENGINE_QEXECUTOR_H
#define TDENGINE_QUERYEXECUTOR_H #define TDENGINE_QEXECUTOR_H
#include "os.h" #include "os.h"
...@@ -37,30 +37,24 @@ typedef int32_t (*__block_search_fn_t)(char* data, int32_t num, int64_t key, int ...@@ -37,30 +37,24 @@ typedef int32_t (*__block_search_fn_t)(char* data, int32_t num, int64_t key, int
#define Q_STATUS_EQUAL(p, s) (((p) & (s)) != 0u) #define Q_STATUS_EQUAL(p, s) (((p) & (s)) != 0u)
#define QUERY_IS_ASC_QUERY(q) (GET_FORWARD_DIRECTION_FACTOR((q)->order.order) == QUERY_ASC_FORWARD_STEP) #define QUERY_IS_ASC_QUERY(q) (GET_FORWARD_DIRECTION_FACTOR((q)->order.order) == QUERY_ASC_FORWARD_STEP)
#define SET_STABLE_QUERY_OVER(_q) ((_q)->tableIndex = (int32_t)((_q)->tableqinfoGroupInfo.numOfTables))
#define IS_STASBLE_QUERY_OVER(_q) ((_q)->tableIndex >= (int32_t)((_q)->tableqinfoGroupInfo.numOfTables))
#define GET_TABLEGROUP(q, _index) ((SArray*) taosArrayGetP((q)->tableqinfoGroupInfo.pGroupList, (_index))) #define GET_TABLEGROUP(q, _index) ((SArray*) taosArrayGetP((q)->tableqinfoGroupInfo.pGroupList, (_index)))
#define GET_NUM_OF_RESULTS(_r) (((_r)->outputBuf) == NULL? 0:((_r)->outputBuf)->info.rows)
enum { enum {
// when query starts to execute, this status will set // when query starts to execute, this status will set
QUERY_NOT_COMPLETED = 0x1u, QUERY_NOT_COMPLETED = 0x1u,
/* result output buffer is full, current query is paused.
* this status is only exist in group-by clause and diff/add/division/multiply/ query.
*/
QUERY_RESBUF_FULL = 0x2u,
/* query is over /* query is over
* 1. this status is used in one row result query process, e.g., count/sum/first/last/ avg...etc. * 1. this status is used in one row result query process, e.g., count/sum/first/last/ avg...etc.
* 2. when all data within queried time window, it is also denoted as query_completed * 2. when all data within queried time window, it is also denoted as query_completed
*/ */
QUERY_COMPLETED = 0x4u, QUERY_COMPLETED = 0x2u,
/* when the result is not completed return to client, this status will be /* when the result is not completed return to client, this status will be
* usually used in case of interval query with interpolation option * usually used in case of interval query with interpolation option
*/ */
QUERY_OVER = 0x8u, QUERY_OVER = 0x4u,
}; };
typedef struct SResultRowPool { typedef struct SResultRowPool {
...@@ -86,13 +80,13 @@ typedef struct SSqlGroupbyExpr { ...@@ -86,13 +80,13 @@ typedef struct SSqlGroupbyExpr {
typedef struct SResultRow { typedef struct SResultRow {
int32_t pageId; // pageId & rowId is the position of current result in disk-based output buffer int32_t pageId; // pageId & rowId is the position of current result in disk-based output buffer
int32_t rowId:29; // row index in buffer page int32_t offset:29; // row index in buffer page
bool startInterp; // the time window start timestamp has done the interpolation already. bool startInterp; // the time window start timestamp has done the interpolation already.
bool endInterp; // the time window end timestamp has done the interpolation already. bool endInterp; // the time window end timestamp has done the interpolation already.
bool closed; // this result status: closed or opened bool closed; // this result status: closed or opened
uint32_t numOfRows; // number of rows of current time window uint32_t numOfRows; // number of rows of current time window
SResultRowCellInfo* pCellInfo; // For each result column, there is a resultInfo SResultRowCellInfo* pCellInfo; // For each result column, there is a resultInfo
union {STimeWindow win; char* key;}; // start key of current time window union {STimeWindow win; char* key;}; // start key of current result row
} SResultRow; } SResultRow;
typedef struct SGroupResInfo { typedef struct SGroupResInfo {
...@@ -106,12 +100,11 @@ typedef struct SGroupResInfo { ...@@ -106,12 +100,11 @@ typedef struct SGroupResInfo {
* If the number of generated results is greater than this value, * If the number of generated results is greater than this value,
* query query will be halt and return results to client immediate. * query query will be halt and return results to client immediate.
*/ */
typedef struct SResultRec { typedef struct SRspResultInfo {
int64_t total; // total generated result size in rows int64_t total; // total generated result size in rows
int64_t rows; // current result set size in rows int32_t capacity; // capacity of current result output buffer
int64_t capacity; // capacity of current result output buffer
int32_t threshold; // result size threshold in rows. int32_t threshold; // result size threshold in rows.
} SResultRec; } SRspResultInfo;
typedef struct SResultRowInfo { typedef struct SResultRowInfo {
SResultRow** pResult; // result list SResultRow** pResult; // result list
...@@ -138,7 +131,6 @@ typedef struct SSingleColumnFilterInfo { ...@@ -138,7 +131,6 @@ typedef struct SSingleColumnFilterInfo {
typedef struct STableQueryInfo { typedef struct STableQueryInfo {
TSKEY lastKey; TSKEY lastKey;
int32_t groupIndex; // group id in table list int32_t groupIndex; // group id in table list
int16_t queryRangeSet; // denote if the query range is set, only available for interval query
tVariant tag; tVariant tag;
STimeWindow win; STimeWindow win;
STSCursor cur; STSCursor cur;
...@@ -179,82 +171,136 @@ typedef struct { ...@@ -179,82 +171,136 @@ typedef struct {
SArray* pResult; // SArray<SStddevInterResult> SArray* pResult; // SArray<SStddevInterResult>
} SInterResult; } SInterResult;
typedef struct SSDataBlock {
SDataStatis *pBlockStatis;
SArray *pDataBlock;
SDataBlockInfo info;
} SSDataBlock;
typedef struct SQuery { typedef struct SQuery {
SLimitVal limit;
bool stableQuery; // super table query or not
bool topBotQuery; // TODO used bitwise flag
bool groupbyColumn; // denote if this is a groupby normal column query
bool hasTagResults; // if there are tag values in final result or not
bool timeWindowInterpo;// if the time window start/end required interpolation
bool queryBlockDist; // if query data block distribution
bool stabledev; // super table stddev query
int32_t interBufSize; // intermediate buffer sizse
SOrderVal order;
int16_t numOfCols; int16_t numOfCols;
int16_t numOfTags; int16_t numOfTags;
SOrderVal order;
STimeWindow window; STimeWindow window;
SInterval interval; SInterval interval;
SSessionWindow sw;
int16_t precision; int16_t precision;
int16_t numOfOutput; int16_t numOfOutput;
int16_t fillType; int16_t fillType;
int16_t checkResultBuf; // check if the buffer is full during scan each block int16_t checkResultBuf; // check if the buffer is full during scan each block
SLimitVal limit;
int32_t srcRowSize; // todo extract struct int32_t srcRowSize; // todo extract struct
int32_t resultRowSize; int32_t resultRowSize;
int32_t intermediateResultRowSize; // intermediate result row size, in case of top-k query.
int32_t maxSrcColumnSize; int32_t maxSrcColumnSize;
int32_t tagLen; // tag value length of current query int32_t tagLen; // tag value length of current query
SSqlGroupbyExpr* pGroupbyExpr; SSqlGroupbyExpr* pGroupbyExpr;
SExprInfo* pExpr1; SExprInfo* pExpr1;
SExprInfo* pExpr2; SExprInfo* pExpr2;
int32_t numOfExpr2; int32_t numOfExpr2;
SColumnInfo* colList; SColumnInfo* colList;
SColumnInfo* tagColList; SColumnInfo* tagColList;
int32_t numOfFilterCols; int32_t numOfFilterCols;
int64_t* fillVal; int64_t* fillVal;
uint32_t status; // query status
SResultRec rec;
int32_t pos;
tFilePage** sdata;
STableQueryInfo* current;
int32_t numOfCheckedBlocks; // number of check data blocks
SOrderedPrjQueryInfo prjInfo; // limit value for each vgroup, only available in global order projection query. SOrderedPrjQueryInfo prjInfo; // limit value for each vgroup, only available in global order projection query.
SSingleColumnFilterInfo* pFilterInfo; SSingleColumnFilterInfo* pFilterInfo;
STableQueryInfo* current;
void* tsdb;
SMemRef memRef;
STableGroupInfo tableGroupInfo; // table <tid, last_key> list SArray<STableKeyInfo>
int32_t vgId;
} SQuery; } SQuery;
typedef SSDataBlock* (*__operator_fn_t)(void* param);
typedef void (*__optr_cleanup_fn_t)(void* param, int32_t num);
struct SOperatorInfo;
typedef struct SQueryRuntimeEnv { typedef struct SQueryRuntimeEnv {
jmp_buf env; jmp_buf env;
SQuery* pQuery; SQuery* pQuery;
SQLFunctionCtx* pCtx; uint32_t status; // query status
int32_t numOfRowsPerPage; void* qinfo;
uint16_t* offset; uint8_t scanFlag; // denotes reversed scan of data or not
uint16_t scanFlag; // denotes reversed scan of data or not void* pQueryHandle;
SFillInfo* pFillInfo;
SResultRowInfo resultRowInfo; int32_t prevGroupId; // previous executed group id
SDiskbasedResultBuf* pResultBuf; // query result buffer based on blocked-wised disk file
SQueryCostInfo summary; SHashObj* pResultRowHashTable; // quick locate the window object for each result
void* pQueryHandle; char* keyBuf; // window key buffer
void* pSecQueryHandle; // another thread for SResultRowPool* pool; // window result object pool
bool stableQuery; // super table query or not char** prevRow;
bool topBotQuery; // TODO used bitwise flag
bool groupbyColumn; // denote if this is a groupby normal column query SArray* prevResult; // intermediate result, SArray<SInterResult>
bool hasTagResults; // if there are tag values in final result or not STSBuf* pTsBuf; // timestamp filter list
bool timeWindowInterpo;// if the time window start/end required interpolation STSCursor cur;
bool queryWindowIdentical; // all query time windows are identical for all tables in one group
bool queryBlockDist; // if query data block distribution char* tagVal; // tag value of current data block
bool stabledev; // super table stddev query SArithmeticSupport *sasArray;
int32_t interBufSize; // intermediate buffer sizse
int32_t prevGroupId; // previous executed group id SSDataBlock *outputBuf;
SDiskbasedResultBuf* pResultBuf; // query result buffer based on blocked-wised disk file STableGroupInfo tableqinfoGroupInfo; // this is a group array list, including SArray<STableQueryInfo*> structure
SHashObj* pResultRowHashTable; // quick locate the window object for each result struct SOperatorInfo *proot;
char* keyBuf; // window key buffer struct SOperatorInfo *pTableScanner; // table scan operator
SResultRowPool* pool; // window result object pool SGroupResInfo groupResInfo;
int64_t currentOffset; // dynamic offset value
int32_t* rowCellInfoOffset;// offset value for each row result cell info
char** prevRow; SRspResultInfo resultInfo;
SHashObj *pTableRetrieveTsMap;
SArray* prevResult; // intermediate result, SArray<SInterResult>
STSBuf* pTsBuf; // timestamp filter list
STSCursor cur;
char* tagVal; // tag value of current data block
SArithmeticSupport *sasArray;
} SQueryRuntimeEnv; } SQueryRuntimeEnv;
enum {
OP_IN_EXECUTING = 1,
OP_RES_TO_RETURN = 2,
OP_EXEC_DONE = 3,
};
enum OPERATOR_TYPE_E {
OP_TableScan = 1,
OP_DataBlocksOptScan = 2,
OP_TableSeqScan = 3,
OP_TagScan = 4,
OP_TableBlockInfoScan= 5,
OP_Aggregate = 6,
OP_Arithmetic = 7,
OP_Groupby = 8,
OP_Limit = 9,
OP_Offset = 10,
OP_TimeWindow = 11,
OP_SessionWindow = 12,
OP_Fill = 13,
OP_MultiTableAggregate = 14,
OP_MultiTableTimeInterval = 15,
};
typedef struct SOperatorInfo {
uint8_t operatorType;
bool blockingOptr; // block operator or not
uint8_t status; // denote if current operator is completed
int32_t numOfOutput; // number of columns of the current operator results
char *name; // name, used to show the query execution plan
void *info; // extension attribution
SExprInfo *pExpr;
SQueryRuntimeEnv *pRuntimeEnv;
struct SOperatorInfo *upstream;
__operator_fn_t exec;
__optr_cleanup_fn_t cleanup;
} SOperatorInfo;
enum { enum {
QUERY_RESULT_NOT_READY = 1, QUERY_RESULT_NOT_READY = 1,
QUERY_RESULT_READY = 2, QUERY_RESULT_READY = 2,
...@@ -262,23 +308,11 @@ enum { ...@@ -262,23 +308,11 @@ enum {
typedef struct SQInfo { typedef struct SQInfo {
void* signature; void* signature;
int32_t code; // error code to returned to client int32_t code; // error code to returned to client
int64_t owner; // if it is in execution int64_t owner; // if it is in execution
void* tsdb;
SMemRef memRef;
int32_t vgId;
STableGroupInfo tableGroupInfo; // table <tid, last_key> list SArray<STableKeyInfo>
STableGroupInfo tableqinfoGroupInfo; // this is a group array list, including SArray<STableQueryInfo*> structure
SQueryRuntimeEnv runtimeEnv;
SHashObj* arrTableIdInfo;
int32_t groupIndex;
/* SQueryRuntimeEnv runtimeEnv;
* the query is executed position on which meter of the whole list. SQuery query;
* when the index reaches the last one of the list, it means the query is completed.
*/
int32_t tableIndex;
SGroupResInfo groupResInfo;
void* pBuf; // allocated buffer for STableQueryInfo, sizeof(STableQueryInfo)*numOfTables; void* pBuf; // allocated buffer for STableQueryInfo, sizeof(STableQueryInfo)*numOfTables;
pthread_mutex_t lock; // used to synchronize the rsp/query threads pthread_mutex_t lock; // used to synchronize the rsp/query threads
...@@ -287,6 +321,7 @@ typedef struct SQInfo { ...@@ -287,6 +321,7 @@ typedef struct SQInfo {
void* rspContext; // response context void* rspContext; // response context
int64_t startExecTs; // start to exec timestamp int64_t startExecTs; // start to exec timestamp
char* sql; // query sql string char* sql; // query sql string
SQueryCostInfo summary;
} SQInfo; } SQInfo;
typedef struct SQueryParam { typedef struct SQueryParam {
...@@ -305,10 +340,93 @@ typedef struct SQueryParam { ...@@ -305,10 +340,93 @@ typedef struct SQueryParam {
SSqlGroupbyExpr *pGroupbyExpr; SSqlGroupbyExpr *pGroupbyExpr;
} SQueryParam; } SQueryParam;
typedef struct STableScanInfo {
void *pQueryHandle;
int32_t numOfBlocks;
int32_t numOfSkipped;
int32_t numOfBlockStatis;
int64_t numOfRows;
int32_t order; // scan order
int32_t times; // repeat counts
int32_t current;
int32_t reverseTimes; // 0 by default
SQLFunctionCtx *pCtx; // next operator query context
SResultRowInfo *pResultRowInfo;
int32_t *rowCellInfoOffset;
SExprInfo *pExpr;
SSDataBlock block;
bool loadExternalRows; // load external rows (prev & next rows)
int32_t numOfOutput;
int64_t elapsedTime;
int32_t tableIndex;
} STableScanInfo;
typedef struct STagScanInfo {
SColumnInfo* pCols;
SSDataBlock* pRes;
int32_t totalTables;
int32_t currentIndex;
} STagScanInfo;
typedef struct SOptrBasicInfo {
SResultRowInfo resultRowInfo;
int32_t *rowCellInfoOffset; // offset value for each row result cell info
SQLFunctionCtx *pCtx;
SSDataBlock *pRes;
} SOptrBasicInfo;
typedef struct SOptrBasicInfo STableIntervalOperatorInfo;
typedef struct SAggOperatorInfo {
SOptrBasicInfo binfo;
uint32_t seed;
} SAggOperatorInfo;
typedef struct SArithOperatorInfo {
SOptrBasicInfo binfo;
int32_t bufCapacity;
uint32_t seed;
} SArithOperatorInfo;
typedef struct SLimitOperatorInfo {
int64_t limit;
int64_t total;
} SLimitOperatorInfo;
typedef struct SOffsetOperatorInfo {
int64_t offset;
} SOffsetOperatorInfo;
typedef struct SFillOperatorInfo {
SFillInfo *pFillInfo;
SSDataBlock *pRes;
int64_t totalInputRows;
} SFillOperatorInfo;
typedef struct SGroupbyOperatorInfo {
SOptrBasicInfo binfo;
int32_t colIndex;
char *prevData; // previous group by value
} SGroupbyOperatorInfo;
typedef struct SSWindowOperatorInfo {
SOptrBasicInfo binfo;
STimeWindow curWindow; // current time window
TSKEY prevTs; // previous timestamp
int32_t numOfRows; // number of rows
int32_t start; // start row index
} SSWindowOperatorInfo;
void freeParam(SQueryParam *param); void freeParam(SQueryParam *param);
int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param); int32_t convertQueryMsg(SQueryTableMsg *pQueryMsg, SQueryParam* param);
int32_t createQueryFuncExprFromMsg(SQueryTableMsg *pQueryMsg, int32_t numOfOutput, SExprInfo **pExprInfo, SSqlFuncMsg **pExprMsg, int32_t createQueryFuncExprFromMsg(SQueryTableMsg *pQueryMsg, int32_t numOfOutput, SExprInfo **pExprInfo, SSqlFuncMsg **pExprMsg,
SColumnInfo* pTagCols); SColumnInfo* pTagCols);
int32_t createIndirectQueryFuncExprFromMsg(SQueryTableMsg *pQueryMsg, int32_t numOfOutput, SExprInfo **pExprInfo,
SSqlFuncMsg **pExprMsg, SExprInfo *prevExpr);
SSqlGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code); SSqlGroupbyExpr *createGroupbyExprFromMsg(SQueryTableMsg *pQueryMsg, SColIndex *pColIndex, int32_t *code);
SQInfo *createQInfoImpl(SQueryTableMsg *pQueryMsg, SSqlGroupbyExpr *pGroupbyExpr, SExprInfo *pExprs, SQInfo *createQInfoImpl(SQueryTableMsg *pQueryMsg, SSqlGroupbyExpr *pGroupbyExpr, SExprInfo *pExprs,
SExprInfo *pSecExprs, STableGroupInfo *pTableGroupInfo, SColumnInfo* pTagCols, bool stableQuery, char* sql); SExprInfo *pSecExprs, STableGroupInfo *pTableGroupInfo, SColumnInfo* pTagCols, bool stableQuery, char* sql);
...@@ -318,13 +436,9 @@ void freeColumnFilterInfo(SColumnFilterInfo* pFilter, int32_t numOfFilters); ...@@ -318,13 +436,9 @@ void freeColumnFilterInfo(SColumnFilterInfo* pFilter, int32_t numOfFilters);
bool isQueryKilled(SQInfo *pQInfo); bool isQueryKilled(SQInfo *pQInfo);
int32_t checkForQueryBuf(size_t numOfTables); int32_t checkForQueryBuf(size_t numOfTables);
bool doBuildResCheck(SQInfo* pQInfo); bool doBuildResCheck(SQInfo* pQInfo);
void setQueryStatus(SQuery *pQuery, int8_t status); void setQueryStatus(SQueryRuntimeEnv *pRuntimeEnv, int8_t status);
bool onlyQueryTags(SQuery* pQuery); bool onlyQueryTags(SQuery* pQuery);
void buildTagQueryResult(SQInfo *pQInfo);
void stableQueryImpl(SQInfo *pQInfo);
void buildTableBlockDistResult(SQInfo *pQInfo);
void tableQueryImpl(SQInfo *pQInfo);
bool isValidQInfo(void *param); bool isValidQInfo(void *param);
int32_t doDumpQueryResult(SQInfo *pQInfo, char *data); int32_t doDumpQueryResult(SQInfo *pQInfo, char *data);
...@@ -336,4 +450,4 @@ void freeQInfo(SQInfo *pQInfo); ...@@ -336,4 +450,4 @@ void freeQInfo(SQInfo *pQInfo);
int32_t getMaximumIdleDurationSec(); int32_t getMaximumIdleDurationSec();
#endif // TDENGINE_QUERYEXECUTOR_H #endif // TDENGINE_QEXECUTOR_H
src/query/inc/qFill.h
浏览文件 @ 4ec4714e
...@@ -24,6 +24,8 @@ extern "C" { ...@@ -24,6 +24,8 @@ extern "C" {
#include "qExtbuffer.h" #include "qExtbuffer.h"
#include "taosdef.h" #include "taosdef.h"
struct SSDataBlock;
typedef struct { typedef struct {
STColumn col; // column info STColumn col; // column info
int16_t functionId; // sql function id int16_t functionId; // sql function id
...@@ -78,7 +80,7 @@ void* taosDestroyFillInfo(SFillInfo *pFillInfo); ...@@ -78,7 +80,7 @@ void* taosDestroyFillInfo(SFillInfo *pFillInfo);
void taosFillSetStartInfo(SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey); void taosFillSetStartInfo(SFillInfo* pFillInfo, int32_t numOfRows, TSKEY endKey);
void taosFillSetDataBlockFromFilePage(SFillInfo* pFillInfo, const tFilePage** pInput); void taosFillSetInputDataBlock(SFillInfo* pFillInfo, const struct SSDataBlock* pInput);
void taosFillCopyInputDataFromOneFilePage(SFillInfo* pFillInfo, const tFilePage* pInput); void taosFillCopyInputDataFromOneFilePage(SFillInfo* pFillInfo, const tFilePage* pInput);
...@@ -88,7 +90,7 @@ int64_t getNumOfResultsAfterFillGap(SFillInfo* pFillInfo, int64_t ekey, int32_t ...@@ -88,7 +90,7 @@ int64_t getNumOfResultsAfterFillGap(SFillInfo* pFillInfo, int64_t ekey, int32_t
int32_t taosGetLinearInterpolationVal(SPoint* point, int32_t outputType, SPoint* point1, SPoint* point2, int32_t inputType); int32_t taosGetLinearInterpolationVal(SPoint* point, int32_t outputType, SPoint* point1, SPoint* point2, int32_t inputType);
int64_t taosFillResultDataBlock(SFillInfo* pFillInfo, tFilePage** output, int32_t capacity); int64_t taosFillResultDataBlock(SFillInfo* pFillInfo, void** output, int32_t capacity);
#ifdef __cplusplus #ifdef __cplusplus
} }
......
src/query/inc/qResultbuf.h
浏览文件 @ 4ec4714e
...@@ -55,7 +55,6 @@ typedef struct SResultBufStatis { ...@@ -55,7 +55,6 @@ typedef struct SResultBufStatis {
} SResultBufStatis; } SResultBufStatis;
typedef struct SDiskbasedResultBuf { typedef struct SDiskbasedResultBuf {
int32_t numOfRowsPerPage;
int32_t numOfPages; int32_t numOfPages;
int64_t totalBufSize; int64_t totalBufSize;
int64_t fileSize; // disk file size int64_t fileSize; // disk file size
...@@ -77,7 +76,7 @@ typedef struct SDiskbasedResultBuf { ...@@ -77,7 +76,7 @@ typedef struct SDiskbasedResultBuf {
SResultBufStatis statis; SResultBufStatis statis;
} SDiskbasedResultBuf; } SDiskbasedResultBuf;
#define DEFAULT_INTERN_BUF_PAGE_SIZE (256L) // in bytes #define DEFAULT_INTERN_BUF_PAGE_SIZE (1024L) // in bytes
#define PAGE_INFO_INITIALIZER (SPageDiskInfo){-1, -1} #define PAGE_INFO_INITIALIZER (SPageDiskInfo){-1, -1}
/** /**
...@@ -89,8 +88,7 @@ typedef struct SDiskbasedResultBuf { ...@@ -89,8 +88,7 @@ typedef struct SDiskbasedResultBuf {
* @param handle * @param handle
* @return * @return
*/ */
int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t rowSize, int32_t pagesize, int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t pagesize, int32_t inMemBufSize, const void* handle);
int32_t inMemBufSize, const void* handle);
/** /**
* *
...@@ -101,13 +99,6 @@ int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t ro ...@@ -101,13 +99,6 @@ int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t ro
*/ */
tFilePage* getNewDataBuf(SDiskbasedResultBuf* pResultBuf, int32_t groupId, int32_t* pageId); tFilePage* getNewDataBuf(SDiskbasedResultBuf* pResultBuf, int32_t groupId, int32_t* pageId);
/**
*
* @param pResultBuf
* @return
*/
size_t getNumOfRowsPerPage(const SDiskbasedResultBuf* pResultBuf);
/** /**
* *
* @param pResultBuf * @param pResultBuf
......
src/query/inc/qSqlparser.h
浏览文件 @ 4ec4714e
...@@ -45,7 +45,7 @@ typedef struct SLimitVal { ...@@ -45,7 +45,7 @@ typedef struct SLimitVal {
typedef struct SOrderVal { typedef struct SOrderVal {
uint32_t order; uint32_t order;
int32_t orderColId; int32_t orderColId;
} SOrderVal; } SOrderVal;
typedef struct tVariantListItem { typedef struct tVariantListItem {
...@@ -58,14 +58,19 @@ typedef struct SIntervalVal { ...@@ -58,14 +58,19 @@ typedef struct SIntervalVal {
SStrToken offset; SStrToken offset;
} SIntervalVal; } SIntervalVal;
typedef struct SSessionWindowVal {
SStrToken col;
SStrToken gap;
} SSessionWindowVal;
typedef struct SQuerySQL { typedef struct SQuerySQL {
struct tSQLExprList *pSelection; // select clause struct tSQLExprList *pSelection; // select clause
SArray * from; // from clause SArray<tVariantListItem> SArray * from; // from clause SArray<tVariantListItem>
struct tSQLExpr * pWhere; // where clause [optional] struct tSQLExpr * pWhere; // where clause [optional]
SArray * pGroupby; // groupby clause, only for tags[optional], SArray<tVariantListItem> SArray * pGroupby; // groupby clause, only for tags[optional], SArray<tVariantListItem>
SArray * pSortOrder; // orderby [optional], SArray<tVariantListItem> SArray * pSortOrder; // orderby [optional], SArray<tVariantListItem>
SStrToken interval; // interval [optional] SIntervalVal interval; // (interval, interval_offset) [optional]
SStrToken offset; // offset window [optional] SSessionWindowVal sessionVal; // session window [optional]
SStrToken sliding; // sliding window [optional] SStrToken sliding; // sliding window [optional]
SLimitVal limit; // limit offset [optional] SLimitVal limit; // limit offset [optional]
SLimitVal slimit; // group limit offset [optional] SLimitVal slimit; // group limit offset [optional]
...@@ -125,6 +130,8 @@ typedef struct SCreateDbInfo { ...@@ -125,6 +130,8 @@ typedef struct SCreateDbInfo {
int8_t update; int8_t update;
int8_t cachelast; int8_t cachelast;
SArray *keep; SArray *keep;
int8_t dbType;
int16_t partitions;
} SCreateDbInfo; } SCreateDbInfo;
typedef struct SCreateAcctInfo { typedef struct SCreateAcctInfo {
...@@ -155,6 +162,7 @@ typedef struct SUserInfo { ...@@ -155,6 +162,7 @@ typedef struct SUserInfo {
typedef struct SMiscInfo { typedef struct SMiscInfo {
SArray *a; // SArray<SStrToken> SArray *a; // SArray<SStrToken>
bool existsCheck; bool existsCheck;
int16_t dbType;
int16_t tableType; int16_t tableType;
SUserInfo user; SUserInfo user;
union { union {
...@@ -182,19 +190,32 @@ typedef struct SSqlInfo { ...@@ -182,19 +190,32 @@ typedef struct SSqlInfo {
}; };
} SSqlInfo; } SSqlInfo;
#define NON_ARITHMEIC_EXPR 0
#define NORMAL_ARITHMETIC 1
#define AGG_ARIGHTMEIC 2
enum SQL_NODE_TYPE {
SQL_NODE_TABLE_COLUMN= 1,
SQL_NODE_SQLFUNCTION = 2,
SQL_NODE_VALUE = 3,
SQL_NODE_EXPR = 4,
};
typedef struct tSQLExpr { typedef struct tSQLExpr {
uint32_t nSQLOptr; // TK_FUNCTION: sql function, TK_LE: less than(binary expr) uint16_t type; // sql node type
uint32_t tokenId; // TK_FUNCTION: sql function, TK_LE: less than(binary expr)
// the full sql string of function(col, param), which is actually the raw
// field name, since the function name is kept in nSQLOptr already // the whole string of the function(col, param), while the function name is kept in token
SStrToken operand; SStrToken operand;
SStrToken colInfo; // field id uint32_t functionId; // function id
tVariant val; // value only for string, float, int
SStrToken colInfo; // table column info
tVariant value; // the use input value
SStrToken token; // original sql expr string SStrToken token; // original sql expr string
struct tSQLExpr *pLeft; // left child struct tSQLExpr *pLeft; // left child
struct tSQLExpr *pRight; // right child struct tSQLExpr *pRight; // right child
struct tSQLExprList *pParam; // function parameters struct tSQLExprList *pParam; // function parameters list
} tSQLExpr; } tSQLExpr;
// used in select clause. select <tSQLExprList> from xxx // used in select clause. select <tSQLExprList> from xxx
...@@ -240,8 +261,8 @@ tSQLExprList *tSqlExprListAppend(tSQLExprList *pList, tSQLExpr *pNode, SStrToken ...@@ -240,8 +261,8 @@ tSQLExprList *tSqlExprListAppend(tSQLExprList *pList, tSQLExpr *pNode, SStrToken
void tSqlExprListDestroy(tSQLExprList *pList); void tSqlExprListDestroy(tSQLExprList *pList);
SQuerySQL *tSetQuerySqlElems(SStrToken *pSelectToken, tSQLExprList *pSelection, SArray *pFrom, tSQLExpr *pWhere, SQuerySQL *tSetQuerySqlNode(SStrToken *pSelectToken, tSQLExprList *pSelection, SArray *pFrom, tSQLExpr *pWhere,
SArray *pGroupby, SArray *pSortOrder, SIntervalVal *pInterval, SArray *pGroupby, SArray *pSortOrder, SIntervalVal *pInterval, SSessionWindowVal *pSession,
SStrToken *pSliding, SArray *pFill, SLimitVal *pLimit, SLimitVal *pGLimit); SStrToken *pSliding, SArray *pFill, SLimitVal *pLimit, SLimitVal *pGLimit);
SCreateTableSQL *tSetCreateSqlElems(SArray *pCols, SArray *pTags, SQuerySQL *pSelect, int32_t type); SCreateTableSQL *tSetCreateSqlElems(SArray *pCols, SArray *pTags, SQuerySQL *pSelect, int32_t type);
...@@ -265,7 +286,7 @@ void setCreatedTableName(SSqlInfo *pInfo, SStrToken *pTableNameToken, SStrToken ...@@ -265,7 +286,7 @@ void setCreatedTableName(SSqlInfo *pInfo, SStrToken *pTableNameToken, SStrToken
void SqlInfoDestroy(SSqlInfo *pInfo); void SqlInfoDestroy(SSqlInfo *pInfo);
void setDCLSQLElems(SSqlInfo *pInfo, int32_t type, int32_t nParams, ...); void setDCLSQLElems(SSqlInfo *pInfo, int32_t type, int32_t nParams, ...);
void setDropDbTableInfo(SSqlInfo *pInfo, int32_t type, SStrToken* pToken, SStrToken* existsCheck,int16_t tableType); void setDropDbTableInfo(SSqlInfo *pInfo, int32_t type, SStrToken* pToken, SStrToken* existsCheck,int16_t dbType,int16_t tableType);
void setShowOptions(SSqlInfo *pInfo, int32_t type, SStrToken* prefix, SStrToken* pPatterns); void setShowOptions(SSqlInfo *pInfo, int32_t type, SStrToken* prefix, SStrToken* pPatterns);
void setCreateDbInfo(SSqlInfo *pInfo, int32_t type, SStrToken *pToken, SCreateDbInfo *pDB, SStrToken *pIgExists); void setCreateDbInfo(SSqlInfo *pInfo, int32_t type, SStrToken *pToken, SCreateDbInfo *pDB, SStrToken *pIgExists);
...@@ -276,6 +297,7 @@ void setKillSql(SSqlInfo *pInfo, int32_t type, SStrToken *ip); ...@@ -276,6 +297,7 @@ void setKillSql(SSqlInfo *pInfo, int32_t type, SStrToken *ip);
void setAlterUserSql(SSqlInfo *pInfo, int16_t type, SStrToken *pName, SStrToken* pPwd, SStrToken *pPrivilege); void setAlterUserSql(SSqlInfo *pInfo, int16_t type, SStrToken *pName, SStrToken* pPwd, SStrToken *pPrivilege);
void setDefaultCreateDbOption(SCreateDbInfo *pDBInfo); void setDefaultCreateDbOption(SCreateDbInfo *pDBInfo);
void setDefaultCreateTopicOption(SCreateDbInfo *pDBInfo);
// prefix show db.tables; // prefix show db.tables;
void setDbName(SStrToken *pCpxName, SStrToken *pDb); void setDbName(SStrToken *pCpxName, SStrToken *pDb);
...@@ -290,16 +312,6 @@ void tSqlSetColumnType(TAOS_FIELD *pField, SStrToken *type); ...@@ -290,16 +312,6 @@ void tSqlSetColumnType(TAOS_FIELD *pField, SStrToken *type);
void *ParseAlloc(void *(*mallocProc)(size_t)); void *ParseAlloc(void *(*mallocProc)(size_t));
enum {
TSQL_NODE_TYPE_EXPR = 0x1,
TSQL_NODE_TYPE_ID = 0x2,
TSQL_NODE_TYPE_VALUE = 0x4,
};
#define NON_ARITHMEIC_EXPR 0
#define NORMAL_ARITHMETIC 1
#define AGG_ARIGHTMEIC 2
SSqlInfo qSQLParse(const char *str); SSqlInfo qSQLParse(const char *str);
#ifdef __cplusplus #ifdef __cplusplus
......
src/query/inc/qTsbuf.h
浏览文件 @ 4ec4714e
...@@ -112,13 +112,11 @@ STSBuf* tsBufClone(STSBuf* pTSBuf); ...@@ -112,13 +112,11 @@ STSBuf* tsBufClone(STSBuf* pTSBuf);
STSGroupBlockInfo* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id); STSGroupBlockInfo* tsBufGetGroupBlockInfo(STSBuf* pTSBuf, int32_t id);
void tsBufFlush(STSBuf* pTSBuf); void tsBufFlush(STSBuf* pTSBuf);
void tsBufResetPos(STSBuf* pTSBuf); void tsBufResetPos(STSBuf* pTSBuf);
STSElem tsBufGetElem(STSBuf* pTSBuf);
bool tsBufNextPos(STSBuf* pTSBuf); bool tsBufNextPos(STSBuf* pTSBuf);
STSElem tsBufGetElem(STSBuf* pTSBuf);
STSElem tsBufGetElemStartPos(STSBuf* pTSBuf, int32_t id, tVariant* tag); STSElem tsBufGetElemStartPos(STSBuf* pTSBuf, int32_t id, tVariant* tag);
STSCursor tsBufGetCursor(STSBuf* pTSBuf); STSCursor tsBufGetCursor(STSBuf* pTSBuf);
......
src/query/inc/qUtil.h
浏览文件 @ 4ec4714e
...@@ -27,7 +27,7 @@ ...@@ -27,7 +27,7 @@
#define GET_RES_WINDOW_KEY_LEN(_l) ((_l) + sizeof(uint64_t)) #define GET_RES_WINDOW_KEY_LEN(_l) ((_l) + sizeof(uint64_t))
#define curTimeWindowIndex(_winres) ((_winres)->curIndex) #define curTimeWindowIndex(_winres) ((_winres)->curIndex)
#define GET_ROW_PARAM_FOR_MULTIOUTPUT(_q, tbq, sq) (((tbq) && (!sq))? (_q)->pExpr1[1].base.arg->argValue.i64:1) #define GET_ROW_PARAM_FOR_MULTIOUTPUT(_q, tbq, sq) (((tbq) && (!(sq)))? (_q)->pExpr1[1].base.arg->argValue.i64:1)
int32_t getOutputInterResultBufSize(SQuery* pQuery); int32_t getOutputInterResultBufSize(SQuery* pQuery);
...@@ -44,22 +44,18 @@ void closeResultRow(SResultRowInfo* pResultRowInfo, int32_t slot); ...@@ -44,22 +44,18 @@ void closeResultRow(SResultRowInfo* pResultRowInfo, int32_t slot);
bool isResultRowClosed(SResultRowInfo *pResultRowInfo, int32_t slot); bool isResultRowClosed(SResultRowInfo *pResultRowInfo, int32_t slot);
void clearResultRow(SQueryRuntimeEnv* pRuntimeEnv, SResultRow* pResultRow, int16_t type); void clearResultRow(SQueryRuntimeEnv* pRuntimeEnv, SResultRow* pResultRow, int16_t type);
SResultRowCellInfo* getResultCell(SQueryRuntimeEnv* pRuntimeEnv, const SResultRow* pRow, int32_t index); SResultRowCellInfo* getResultCell(const SResultRow* pRow, int32_t index, int32_t* offset);
static FORCE_INLINE SResultRow *getResultRow(SResultRowInfo *pResultRowInfo, int32_t slot) { static FORCE_INLINE SResultRow *getResultRow(SResultRowInfo *pResultRowInfo, int32_t slot) {
assert(pResultRowInfo != NULL && slot >= 0 && slot < pResultRowInfo->size); assert(pResultRowInfo != NULL && slot >= 0 && slot < pResultRowInfo->size);
return pResultRowInfo->pResult[slot]; return pResultRowInfo->pResult[slot];
} }
static FORCE_INLINE char *getPosInResultPage(SQueryRuntimeEnv *pRuntimeEnv, int32_t columnIndex, SResultRow *pResult, static FORCE_INLINE char *getPosInResultPage(SQuery *pQuery, tFilePage* page, int32_t rowOffset, int16_t offset) {
tFilePage* page) { assert(rowOffset >= 0 && pQuery != NULL);
assert(pResult != NULL && pRuntimeEnv != NULL);
SQuery *pQuery = pRuntimeEnv->pQuery; int32_t numOfRows = (int32_t)GET_ROW_PARAM_FOR_MULTIOUTPUT(pQuery, pQuery->topBotQuery, pQuery->stableQuery);
return ((char *)page->data) + rowOffset + offset * numOfRows;
int32_t realRowId = (int32_t)(pResult->rowId * GET_ROW_PARAM_FOR_MULTIOUTPUT(pQuery, pRuntimeEnv->topBotQuery, pRuntimeEnv->stableQuery));
return ((char *)page->data) + pRuntimeEnv->offset[columnIndex] * pRuntimeEnv->numOfRowsPerPage +
pQuery->pExpr1[columnIndex].bytes * realRowId;
} }
bool isNullOperator(SColumnFilterElem *pFilter, const char* minval, const char* maxval, int16_t type); bool isNullOperator(SColumnFilterElem *pFilter, const char* minval, const char* maxval, int16_t type);
...@@ -74,8 +70,6 @@ void* destroyResultRowPool(SResultRowPool* p); ...@@ -74,8 +70,6 @@ void* destroyResultRowPool(SResultRowPool* p);
int32_t getNumOfAllocatedResultRows(SResultRowPool* p); int32_t getNumOfAllocatedResultRows(SResultRowPool* p);
int32_t getNumOfUsedResultRows(SResultRowPool* p); int32_t getNumOfUsedResultRows(SResultRowPool* p);
bool isPointInterpoQuery(SQuery *pQuery);
typedef struct { typedef struct {
SArray* pResult; // SArray<SResPair> SArray* pResult; // SArray<SResPair>
int32_t colId; int32_t colId;
...@@ -85,12 +79,14 @@ void interResToBinary(SBufferWriter* bw, SArray* pRes, int32_t tagLen); ...@@ -85,12 +79,14 @@ void interResToBinary(SBufferWriter* bw, SArray* pRes, int32_t tagLen);
SArray* interResFromBinary(const char* data, int32_t len); SArray* interResFromBinary(const char* data, int32_t len);
void freeInterResult(void* param); void freeInterResult(void* param);
void initGroupResInfo(SGroupResInfo* pGroupResInfo, SResultRowInfo* pResultInfo, int32_t offset); void initGroupResInfo(SGroupResInfo* pGroupResInfo, SResultRowInfo* pResultInfo);
void cleanupGroupResInfo(SGroupResInfo* pGroupResInfo); void cleanupGroupResInfo(SGroupResInfo* pGroupResInfo);
bool hasRemainDataInCurrentGroup(SGroupResInfo* pGroupResInfo);
bool hasRemainData(SGroupResInfo* pGroupResInfo); bool hasRemainData(SGroupResInfo* pGroupResInfo);
bool incNextGroup(SGroupResInfo* pGroupResInfo); bool incNextGroup(SGroupResInfo* pGroupResInfo);
int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo); int32_t getNumOfTotalRes(SGroupResInfo* pGroupResInfo);
int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQInfo *pQInfo); int32_t mergeIntoGroupResult(SGroupResInfo* pGroupResInfo, SQueryRuntimeEnv *pRuntimeEnv, int32_t* offset);
#endif // TDENGINE_QUERYUTIL_H #endif // TDENGINE_QUERYUTIL_H
src/query/inc/sql.y
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qAggMain.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qExecutor.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qFill.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qParserImpl.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qPercentile.c
浏览文件 @ 4ec4714e
...@@ -254,7 +254,7 @@ tMemBucket *tMemBucketCreate(int16_t nElemSize, int16_t dataType, double minval, ...@@ -254,7 +254,7 @@ tMemBucket *tMemBucketCreate(int16_t nElemSize, int16_t dataType, double minval,
resetSlotInfo(pBucket); resetSlotInfo(pBucket);
int32_t ret = createDiskbasedResultBuffer(&pBucket->pBuffer, pBucket->bytes, pBucket->bufPageSize, pBucket->bufPageSize * 512, NULL); int32_t ret = createDiskbasedResultBuffer(&pBucket->pBuffer, pBucket->bufPageSize, pBucket->bufPageSize * 512, NULL);
if (ret != TSDB_CODE_SUCCESS) { if (ret != TSDB_CODE_SUCCESS) {
tMemBucketDestroy(pBucket); tMemBucketDestroy(pBucket);
return NULL; return NULL;
......
src/query/src/qResultbuf.c
浏览文件 @ 4ec4714e
...@@ -9,8 +9,7 @@ ...@@ -9,8 +9,7 @@
#define GET_DATA_PAYLOAD(_p) ((char *)(_p)->pData + POINTER_BYTES) #define GET_DATA_PAYLOAD(_p) ((char *)(_p)->pData + POINTER_BYTES)
#define NO_IN_MEM_AVAILABLE_PAGES(_b) (listNEles((_b)->lruList) >= (_b)->inMemPages) #define NO_IN_MEM_AVAILABLE_PAGES(_b) (listNEles((_b)->lruList) >= (_b)->inMemPages)
int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t rowSize, int32_t pagesize, int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t pagesize, int32_t inMemBufSize, const void* handle) {
int32_t inMemBufSize, const void* handle) {
*pResultBuf = calloc(1, sizeof(SDiskbasedResultBuf)); *pResultBuf = calloc(1, sizeof(SDiskbasedResultBuf));
SDiskbasedResultBuf* pResBuf = *pResultBuf; SDiskbasedResultBuf* pResBuf = *pResultBuf;
...@@ -31,7 +30,6 @@ int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t ro ...@@ -31,7 +30,6 @@ int32_t createDiskbasedResultBuffer(SDiskbasedResultBuf** pResultBuf, int32_t ro
// at least more than 2 pages must be in memory // at least more than 2 pages must be in memory
assert(inMemBufSize >= pagesize * 2); assert(inMemBufSize >= pagesize * 2);
pResBuf->numOfRowsPerPage = (pagesize - sizeof(tFilePage)) / rowSize;
pResBuf->lruList = tdListNew(POINTER_BYTES); pResBuf->lruList = tdListNew(POINTER_BYTES);
// init id hash table // init id hash table
...@@ -387,8 +385,6 @@ void releaseResBufPageInfo(SDiskbasedResultBuf* pResultBuf, SPageInfo* pi) { ...@@ -387,8 +385,6 @@ void releaseResBufPageInfo(SDiskbasedResultBuf* pResultBuf, SPageInfo* pi) {
pResultBuf->statis.releasePages += 1; pResultBuf->statis.releasePages += 1;
} }
size_t getNumOfRowsPerPage(const SDiskbasedResultBuf* pResultBuf) { return pResultBuf->numOfRowsPerPage; }
size_t getNumOfResultBufGroupId(const SDiskbasedResultBuf* pResultBuf) { return taosHashGetSize(pResultBuf->groupSet); } size_t getNumOfResultBufGroupId(const SDiskbasedResultBuf* pResultBuf) { return taosHashGetSize(pResultBuf->groupSet); }
size_t getResBufSize(const SDiskbasedResultBuf* pResultBuf) { return (size_t)pResultBuf->totalBufSize; } size_t getResBufSize(const SDiskbasedResultBuf* pResultBuf) { return (size_t)pResultBuf->totalBufSize; }
......
src/query/src/qTokenizer.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/qUtil.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/queryMain.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/src/sql.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/query/tests/resultBufferTest.cpp
浏览文件 @ 4ec4714e
...@@ -10,7 +10,7 @@ namespace { ...@@ -10,7 +10,7 @@ namespace {
// simple test // simple test
void simpleTest() { void simpleTest() {
SDiskbasedResultBuf* pResultBuf = NULL; SDiskbasedResultBuf* pResultBuf = NULL;
int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 64, 1024, 4096, NULL); int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 1024, 4096, NULL);
int32_t pageId = 0; int32_t pageId = 0;
int32_t groupId = 0; int32_t groupId = 0;
...@@ -52,7 +52,7 @@ void simpleTest() { ...@@ -52,7 +52,7 @@ void simpleTest() {
void writeDownTest() { void writeDownTest() {
SDiskbasedResultBuf* pResultBuf = NULL; SDiskbasedResultBuf* pResultBuf = NULL;
int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 64, 1024, 4*1024, NULL); int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 1024, 4*1024, NULL);
int32_t pageId = 0; int32_t pageId = 0;
int32_t writePageId = 0; int32_t writePageId = 0;
...@@ -99,7 +99,7 @@ void writeDownTest() { ...@@ -99,7 +99,7 @@ void writeDownTest() {
void recyclePageTest() { void recyclePageTest() {
SDiskbasedResultBuf* pResultBuf = NULL; SDiskbasedResultBuf* pResultBuf = NULL;
int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 64, 1024, 4*1024, NULL); int32_t ret = createDiskbasedResultBuffer(&pResultBuf, 1024, 4*1024, NULL);
int32_t pageId = 0; int32_t pageId = 0;
int32_t writePageId = 0; int32_t writePageId = 0;
......
src/tsdb/src/tsdbCommit.c
浏览文件 @ 4ec4714e
...@@ -15,7 +15,13 @@ ...@@ -15,7 +15,13 @@
#include "tsdbint.h" #include "tsdbint.h"
#define TSDB_MAX_SUBBLOCKS 8 #define TSDB_MAX_SUBBLOCKS 8
#define TSDB_KEY_FID(key, days, precision) ((key) / tsMsPerDay[(precision)] / (days)) static FORCE_INLINE int TSDB_KEY_FID(TSKEY key, int32_t days, int8_t precision) {
if (key < 0) {
return (int)(-((-key) / tsMsPerDay[precision] / days + 1));
} else {
return (int)((key / tsMsPerDay[precision] / days));
}
}
typedef struct { typedef struct {
SRtn rtn; // retention snapshot SRtn rtn; // retention snapshot
......
src/tsdb/src/tsdbMeta.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/tsdb/src/tsdbRead.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/util/inc/tarray.h
浏览文件 @ 4ec4714e
...@@ -25,7 +25,8 @@ extern "C" { ...@@ -25,7 +25,8 @@ extern "C" {
#define TARRAY_MIN_SIZE 8 #define TARRAY_MIN_SIZE 8
#define TARRAY_GET_ELEM(array, index) ((void*)((char*)((array)->pData) + (index) * (array)->elemSize)) #define TARRAY_GET_ELEM(array, index) ((void*)((char*)((array)->pData) + (index) * (array)->elemSize))
#define TARRAY_ELEM_IDX(array, ele) (POINTER_DISTANCE(ele, (array)->pData) / (array)->elemSize) #define TARRAY_ELEM_IDX(array, ele) (POINTER_DISTANCE(ele, (array)->pData) / (array)->elemSize)
#define TARRAY_GET_START(array) ((array)->pData)
typedef struct SArray { typedef struct SArray {
size_t size; size_t size;
......
src/util/inc/tbuffer.h
浏览文件 @ 4ec4714e
...@@ -73,14 +73,14 @@ int main( int argc, char** argv ) { ...@@ -73,14 +73,14 @@ int main( int argc, char** argv ) {
} }
*/ */
typedef struct { typedef struct SBufferReader {
bool endian; bool endian;
const char* data; const char* data;
size_t pos; size_t pos;
size_t size; size_t size;
} SBufferReader; } SBufferReader;
typedef struct { typedef struct SBufferWriter {
bool endian; bool endian;
char* data; char* data;
size_t pos; size_t pos;
......
src/util/src/terror.c
浏览文件 @ 4ec4714e
...@@ -197,6 +197,9 @@ TAOS_DEFINE_ERROR(TSDB_CODE_MND_VGROUP_NOT_READY, "Database unsynced") ...@@ -197,6 +197,9 @@ TAOS_DEFINE_ERROR(TSDB_CODE_MND_VGROUP_NOT_READY, "Database unsynced")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_DB_OPTION_DAYS, "Invalid database option: days out of range") TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_DB_OPTION_DAYS, "Invalid database option: days out of range")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_DB_OPTION_KEEP, "Invalid database option: keep >= keep1 >= keep0 >= days") TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_DB_OPTION_KEEP, "Invalid database option: keep >= keep1 >= keep0 >= days")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_TOPIC, "Invalid topic name")
TAOS_DEFINE_ERROR(TSDB_CODE_MND_INVALID_TOPIC_OPTION, "Invalid topic option")
// dnode // dnode
TAOS_DEFINE_ERROR(TSDB_CODE_DND_MSG_NOT_PROCESSED, "Message not processed") TAOS_DEFINE_ERROR(TSDB_CODE_DND_MSG_NOT_PROCESSED, "Message not processed")
TAOS_DEFINE_ERROR(TSDB_CODE_DND_OUT_OF_MEMORY, "Dnode out of memory") TAOS_DEFINE_ERROR(TSDB_CODE_DND_OUT_OF_MEMORY, "Dnode out of memory")
......
src/vnode/inc/vnodeInt.h
浏览文件 @ 4ec4714e
...@@ -40,6 +40,7 @@ typedef struct { ...@@ -40,6 +40,7 @@ typedef struct {
int32_t queuedWMsg; int32_t queuedWMsg;
int32_t queuedRMsg; int32_t queuedRMsg;
int32_t flowctrlLevel; int32_t flowctrlLevel;
int32_t sequence; // for topic
int8_t status; int8_t status;
int8_t role; int8_t role;
int8_t accessState; int8_t accessState;
...@@ -47,7 +48,7 @@ typedef struct { ...@@ -47,7 +48,7 @@ typedef struct {
int8_t isCommiting; int8_t isCommiting;
int8_t dbReplica; int8_t dbReplica;
int8_t dropped; int8_t dropped;
int8_t reserved; int8_t dbType;
uint64_t version; // current version uint64_t version; // current version
uint64_t cversion; // version while commit start uint64_t cversion; // version while commit start
uint64_t fversion; // version on saved data file uint64_t fversion; // version on saved data file
......
src/vnode/src/vnodeCfg.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/vnode/src/vnodeRead.c
浏览文件 @ 4ec4714e
此差异已折叠。
src/vnode/src/vnodeWrite.c
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/fulltest.sh
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/functions/function_percentile2.py 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/pytest_3.sh
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/query/queryBetweenAnd.py 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/tools/taosdemoTestWithoutMetric.py 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/pytest/util/dnodes.py
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/general/db/topic1.sim 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/general/db/topic2.sim 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/general/parser/last_groupby.sim 0 → 100644
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/jenkins/basic.txt
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/jenkins/basic_1.txt
浏览文件 @ 4ec4714e
此差异已折叠。
tests/script/jenkins/basic_7.txt
浏览文件 @ 4ec4714e
此差异已折叠。
tests/test-all.sh
浏览文件 @ 4ec4714e
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或