Skip to content

T
TDengine

慢慢CG / TDengine
与 Fork 源项目一致

Fork自 taosdata / TDengine

通知 1
Star 0
Fork 0
T
TDengine
提交 d74400f3 编写于 作者: L liuyq-617
浏览文件
操作

Merge branch 'develop' into hotfix/testcase

上级 22ec4f4a d16caf31
展开全部 隐藏空白更改
内联 并排
Showing with 6151 addition and 755 deletion
.travis.yml
浏览文件 @ d74400f3
......@@ -32,6 +32,8 @@ matrix:
- python3-setuptools
- valgrind
- psmisc
- unixodbc
- unixodbc-dev
before_script:
- export TZ=Asia/Harbin
......
cmake/install.inc
浏览文件 @ d74400f3
......@@ -31,7 +31,7 @@ ELSEIF (TD_WINDOWS)
#INSTALL(TARGETS taos RUNTIME DESTINATION driver)
#INSTALL(TARGETS shell RUNTIME DESTINATION .)
IF (TD_MVN_INSTALLED)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.0-dist.jar DESTINATION connector/jdbc)
INSTALL(FILES ${LIBRARY_OUTPUT_PATH}/taos-jdbcdriver-2.0.8-dist.jar DESTINATION connector/jdbc)
ENDIF ()
ELSEIF (TD_DARWIN)
SET(TD_MAKE_INSTALL_SH "${TD_COMMUNITY_DIR}/packaging/tools/make_install.sh")
......
documentation/tdenginedocs-cn/advanced-features/index.html
浏览文件 @ d74400f3
......@@ -51,8 +51,8 @@ INTERVAL(1M)</code></pre>
<li><p>mseconds：查询数据库更新的时间间隔，单位为毫秒。一般设置为1000毫秒。返回值为指向TDengine_SUB 结构的指针，如果返回为空，表示失败。</p></li>
</ul><li><p><code>TAOS_ROW taos_consume(TAOS_SUB *tsub)</code>
</p><p>该函数用来获取订阅的结果，用户应用程序将其置于一个无限循环语句。如果数据库有新记录到达，该API将返回该最新的记录。如果没有新的记录，该API将阻塞。如果返回值为空，说明系统出错。参数说明：</p></li><ul><li><p>tsub：taos_subscribe的结构体指针。</p></li></ul><li><p><code>void taos_unsubscribe(TAOS_SUB *tsub)</code></p><p>取消订阅。应用程序退出时，务必调用该函数以避免资源泄露。</p></li>
<li><p><code>int taos_num_subfields(TAOS_SUB *tsub)</code></p><p>获取返回的一行记录中数据包含多少列。</p></li>
<li><p><code>TAOS_FIELD *taos_fetch_subfields(TAOS_SUB *tsub)</code></p><p>获取每列数据的属性（数据类型、名字、长度），与taos_num_subfileds配合使用，可解析返回的每行数据。</p></li></ul>
<li><p><code>int taos_num_fields(TAOS_SUB *tsub)</code></p><p>获取返回的一行记录中数据包含多少列。</p></li>
<li><p><code>TAOS_FIELD *taos_fetch_fields(TAOS_SUB *tsub)</code></p><p>获取每列数据的属性（数据类型、名字、长度），与taos_num_subfileds配合使用，可解析返回的每行数据。</p></li></ul>
<p>示例代码：请看安装包中的的示范程序</p>
<a class='anchor' id='缓存-(Cache)'></a><h2>缓存 (Cache)</h2>
<p>TDengine采用时间驱动缓存管理策略（First-In-First-Out，FIFO），又称为写驱动的缓存管理机制。这种策略有别于读驱动的数据缓存模式（Least-Recent-Use，LRU），直接将最近写入的数据保存在系统的缓存中。当缓存达到临界值的时候，将最早的数据批量写入磁盘。一般意义上来说，对于物联网数据的使用，用户最为关心最近产生的数据，即当前状态。TDengine充分利用了这一特性，将最近到达的（当前状态）数据保存在缓存中。</p>
......
documentation/tdenginedocs-cn/connector/index.html
浏览文件 @ d74400f3
......@@ -64,9 +64,9 @@
<p>该API用来获取最新消息，应用程序一般会将其置于一个无限循环语句中。其中参数tsub是taos_subscribe的返回值。如果数据库有新的记录，该API将返回，返回参数是一行记录。如果没有新的记录，该API将阻塞。如果返回值为空，说明系统出错，需要检查系统是否还在正常运行。</p></li>
<li><p><code>void taos_unsubscribe(TAOS_SUB *tsub)</code></p>
<p>该API用于取消订阅，参数tsub是taos_subscribe的返回值。应用程序退出时，需要调用该API，否则有资源泄露。</p></li>
<li><p><code>int taos_num_subfields(TAOS_SUB *tsub)</code></p>
<li><p><code>int taos_num_fields(TAOS_SUB *tsub)</code></p>
<p>该API用来获取返回的一排数据中数据的列数</p></li>
<li><p><code>TAOS_FIELD *taos_fetch_subfields(TAOS_RES *res)</code></p>
<li><p><code>TAOS_FIELD *taos_fetch_fields(TAOS_RES *res)</code></p>
<p>该API用来获取每列数据的属性（数据类型、名字、字节数），与taos_num_subfileds配合使用，可用来解析返回的一排数据。</p></li>
</ul>
<a class='anchor' id='Java-Connector'></a><h2>Java Connector</h2>
......@@ -259,4 +259,4 @@ conn.close()
_ "taosSql"
)</code></pre>
<p>taosSql驱动包内采用cgo模式，调用了TDengine的C/C++同步接口，与TDengine进行交互，因此，在数据库操作执行完成之前，客户端应用将处于阻塞状态。单个数据库连接，在同一时刻只能有一个线程调用API。客户应用可以建立多个连接，进行多线程的数据写入或查询处理。</p>
<p>更多使用的细节，请参考下载目录中的示例源码。</p><a href='../index.html'>回去</a></section></main></div><?php include($s.'/footer.php'); ?><script>$('pre').addClass('prettyprint linenums');PR.prettyPrint()</script><script src='lib/docs/liner.js'></script></body></html>
\ No newline at end of file
<p>更多使用的细节，请参考下载目录中的示例源码。</p><a href='../index.html'>回去</a></section></main></div><?php include($s.'/footer.php'); ?><script>$('pre').addClass('prettyprint linenums');PR.prettyPrint()</script><script src='lib/docs/liner.js'></script></body></html>
documentation/tdenginedocs-en/connector/index.html
浏览文件 @ d74400f3
......@@ -72,9 +72,9 @@ The API is used to start a subscription session by given a handle. The parameter
The API used to get the new data from a TDengine server. It should be put in an infinite loop. The parameter <em>tsub</em> is the handle returned by <em>taos_subscribe</em>. If new data are updated, the API will return a row of the result. Otherwise, the API is blocked until new data arrives. If <em>NULL</em> pointer is returned, it means an error occurs.</p></li>
<li><p><code>void taos_unsubscribe(TAOS_SUB *tsub)</code>
Stop a subscription session by the handle returned by <em>taos_subscribe</em>.</p></li>
<li><p><code>int taos_num_subfields(TAOS_SUB *tsub)</code>
<li><p><code>int taos_num_fields(TAOS_SUB *tsub)</code>
The API used to get the number of fields in a row.</p></li>
<li><p><code>TAOS_FIELD *taos_fetch_subfields(TAOS_RES *res)</code>
<li><p><code>TAOS_FIELD *taos_fetch_fields(TAOS_RES *res)</code>
The API used to get the description of each column.</p></li>
</ul>
<a class='anchor' id='Java-Connector'></a><h2>Java Connector</h2>
......@@ -351,4 +351,4 @@ promise2.then(function(result) {
})</code></pre>
<h3>Example</h3>
<p>An example of using the NodeJS connector to create a table with weather data and create and execute queries can be found <a href="https://github.com/taosdata/TDengine/tree/master/tests/examples/nodejs/node-example.js">here</a> (The preferred method for using the connector)</p>
<p>An example of using the NodeJS connector to achieve the same things but without all the object wrappers that wrap around the data returned to achieve higher functionality can be found <a href="https://github.com/taosdata/TDengine/tree/master/tests/examples/nodejs/node-example-raw.js">here</a></p><a href='../index.html'>Back</a></section></main></div><?php include($s.'/footer.php'); ?><script>$('pre').addClass('prettyprint linenums');PR.prettyPrint()</script><script src='lib/docs/liner.js'></script></body></html>
\ No newline at end of file
<p>An example of using the NodeJS connector to achieve the same things but without all the object wrappers that wrap around the data returned to achieve higher functionality can be found <a href="https://github.com/taosdata/TDengine/tree/master/tests/examples/nodejs/node-example-raw.js">here</a></p><a href='../index.html'>Back</a></section></main></div><?php include($s.'/footer.php'); ?><script>$('pre').addClass('prettyprint linenums');PR.prettyPrint()</script><script src='lib/docs/liner.js'></script></body></html>
documentation20/webdocs/markdowndocs/architecture-ch.md
浏览文件 @ d74400f3
......@@ -228,7 +228,7 @@ TDengine采用数据驱动的方式让缓存中的数据写入硬盘进行持久
为充分利用时序数据特点，TDengine将一个vnode保存在持久化存储的数据切分成多个文件，每个文件只保存固定天数的数据，这个天数由系统配置参数days决定。切分成多个文件后，给定查询的起止日期，无需任何索引，就可以立即定位需要打开哪些数据文件，大大加快读取速度。
对于采集的数据，一般有保留时长，这个时长由系统配置参数keep决定。超过这个设置天数的数据文件，将被系统将自动删除，释放存储空间。
对于采集的数据，一般有保留时长，这个时长由系统配置参数keep决定。超过这个设置天数的数据文件，将被系统自动删除，释放存储空间。
给定days与keep两个参数，一个vnode总的数据文件数为：keep/days。总的数据文件个数不宜过大，也不宜过小。10到100以内合适。基于这个原则，可以设置合理的days。 目前的版本，参数keep可以修改，但对于参数days，一但设置后，不可修改。
......
documentation20/webdocs/markdowndocs/cluster-ch.md
浏览文件 @ d74400f3
......@@ -213,6 +213,6 @@ SHOW MNODES;
## Arbitrator的使用
如果副本数为偶数，当一个vnode group里一半或超过一半的vnode不工作时，是无法从中选出master的。同理，一半或超过一半的mnode不工作时，是无法选出mnode的master的，因为存在“split brain”问题。为解决这个问题，TDengine引入了arbitrator的概念。Arbitrator模拟一个vnode或mnode在工作，但只简单的负责网络连接，不处理任何数据插入或访问。只要包含arbitrator在内，超过半数的vnode或mnode工作，那么该vnode group或mnode组就可以正常的提供数据插入或查询服务。比如对于副本数为2的情形，如果一个节点A离线，但另外一个节点B正常，而且能连接到arbitrator, 那么节点B就能正常工作。
如果副本数为偶数，当一个vnode group里一半vnode不工作时，是无法从中选出master的。同理，一半mnode不工作时，是无法选出mnode的master的，因为存在“split brain”问题。为解决这个问题，TDengine引入了arbitrator的概念。Arbitrator模拟一个vnode或mnode在工作，但只简单的负责网络连接，不处理任何数据插入或访问。只要包含arbitrator在内，超过半数的vnode或mnode工作，那么该vnode group或mnode组就可以正常的提供数据插入或查询服务。比如对于副本数为2的情形，如果一个节点A离线，但另外一个节点B正常，而且能连接到arbitrator, 那么节点B就能正常工作。
TDengine安装包里带有一个执行程序tarbitrator, 找任何一台Linux服务器运行它即可。该程序对系统资源几乎没有要求，只需要保证有网络连接即可。该应用的命令行参数`-p`可以指定其对外服务的端口号，缺省是6042。配置每个taosd实例时，可以在配置文件taos.cfg里将参数arbitrator设置为arbitrator的End Point。如果该参数配置了，当副本数为偶数数，系统将自动连接配置的arbitrator。
TDengine提供一个执行程序tarbitrator, 找任何一台Linux服务器运行它即可。请点击[安装包下载](https://www.taosdata.com/cn/all-downloads/)，在TDengine Arbitrator Linux一节中，选择适合的版本下载并安装。该程序对系统资源几乎没有要求，只需要保证有网络连接即可。该应用的命令行参数`-p`可以指定其对外服务的端口号，缺省是6042。配置每个taosd实例时，可以在配置文件taos.cfg里将参数arbitrator设置为arbitrator的End Point。如果该参数配置了，当副本数为偶数数，系统将自动连接配置的arbitrator。如果副本数为奇数，即使配置了arbitrator, 系统也不会去建立连接。
documentation20/webdocs/markdowndocs/taosd-ch.md
浏览文件 @ d74400f3
......@@ -61,7 +61,7 @@ vnode与其子模块是通过API直接调用，而不是通过消息队列传递
mnode是整个系统的大脑，负责整个系统的资源调度，负责meta data的管理与存储。
一个运行的系统里，只有一个mnode，但它有多个副本（由系统配置参数numOfMpeers控制）。这些副本分布在不同的dnode里，目的是保证系统的高可靠运行。副本之间的数据复制是采用同步而非异步的方式，以确保数据的一致性，确保数据不会丢失。这些副本会自动选举一个Master，其他副本是slave。所有数据更新类的操作，都只能在master上进行，而查询类的可以在slave节点上进行。代码实现上，同步模块与vnode共享，但mnode被分配一个特殊的vgroup ID: 1，而且quorum大于1。整个集群系统是由多个dnode组成的，运行的mnode的副本数不可能超过dnode的个数，但不会超过配置的副本数。如果某个mnode副本宕机一段时间，只要超过半数的mnode副本仍在运行，运行的mnode会自动根据整个系统的资源情况，在其他dnode里再启动一个mnode, 以保证运行的副本数。
一个运行的系统里，只有一个mnode，但它有多个副本（由系统配置参数numOfMnodes控制）。这些副本分布在不同的dnode里，目的是保证系统的高可靠运行。副本之间的数据复制是采用同步而非异步的方式，以确保数据的一致性，确保数据不会丢失。这些副本会自动选举一个Master，其他副本是slave。所有数据更新类的操作，都只能在master上进行，而查询类的可以在slave节点上进行。代码实现上，同步模块与vnode共享，但mnode被分配一个特殊的vgroup ID: 1，而且quorum大于1。整个集群系统是由多个dnode组成的，运行的mnode的副本数不可能超过dnode的个数，但不会超过配置的副本数。如果某个mnode副本宕机一段时间，只要超过半数的mnode副本仍在运行，运行的mnode会自动根据整个系统的资源情况，在其他dnode里再启动一个mnode, 以保证运行的副本数。
各个dnode通过信息交换，保存有mnode各个副本的End Point列表，并向其中的master节点定时（间隔由系统配置参数statusInterval控制）发送status消息，消息体里包含该dnode的CPU、内存、剩余存储空间、vnode个数，以及各个vnode的状态(存储空间、原始数据大小、记录条数、角色等）。这样mnode就了解整个系统的资源情况，如果用户创建新的表，就可以决定需要在哪个dnode创建；如果增加或删除dnode, 或者监测到某dnode数据过热、或离线太长，就可以决定需要挪动那些vnode，以实现负载均衡。
......
src/CMakeLists.txt
浏览文件 @ d74400f3
......@@ -20,4 +20,6 @@ ADD_SUBDIRECTORY(tsdb)
ADD_SUBDIRECTORY(wal)
ADD_SUBDIRECTORY(cq)
ADD_SUBDIRECTORY(dnode)
ADD_SUBDIRECTORY(connector/odbc)
ADD_SUBDIRECTORY(connector/jdbc)
src/balance/src/balance.c
浏览文件 @ d74400f3
......@@ -216,8 +216,8 @@ static bool balanceCheckVgroupReady(SVgObj *pVgroup, SVnodeGid *pRmVnode) {
SVnodeGid *pVnode = pVgroup->vnodeGid + i;
if (pVnode == pRmVnode) continue;
mTrace("vgId:%d, change vgroup status, dnode:%d status:%d", pVgroup->vgId, pVnode->pDnode->dnodeId,
pVnode->pDnode->status);
mTrace("vgId:%d, check vgroup status, dnode:%d status:%d, vnode role:%s", pVgroup->vgId, pVnode->pDnode->dnodeId,
pVnode->pDnode->status, syncRole[pVnode->role]);
if (pVnode->pDnode->status == TAOS_DN_STATUS_DROPPING) continue;
if (pVnode->pDnode->status == TAOS_DN_STATUS_OFFLINE) continue;
......
src/client/inc/tscSubquery.h
浏览文件 @ d74400f3
......@@ -39,7 +39,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql);
int32_t tscHandleInsertRetry(SSqlObj* pSql);
void tscBuildResFromSubqueries(SSqlObj *pSql);
void **doSetResultRowData(SSqlObj *pSql, bool finalResult);
TAOS_ROW doSetResultRowData(SSqlObj *pSql, bool finalResult);
#ifdef __cplusplus
}
......
src/client/inc/tscUtil.h
浏览文件 @ d74400f3
......@@ -149,14 +149,13 @@ int tscAllocPayload(SSqlCmd* pCmd, int size);
TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes);
SFieldSupInfo* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField);
SFieldSupInfo* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_FIELD* field);
SInternalField* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField);
SInternalField* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_FIELD* field);
SFieldSupInfo* tscFieldInfoGetSupp(SFieldInfo* pFieldInfo, int32_t index);
SInternalField* tscFieldInfoGetInternalField(SFieldInfo* pFieldInfo, int32_t index);
TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index);
void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo);
void tscFieldInfoCopy(SFieldInfo* dst, const SFieldInfo* src);
void tscFieldInfoUpdateOffsetForInterResult(SQueryInfo* pQueryInfo);
int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index);
......@@ -231,10 +230,11 @@ int tscGetTableMeta(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo);
int tscGetMeterMetaEx(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, bool createIfNotExists);
void tscResetForNextRetrieve(SSqlRes* pRes);
void tscAddTimestampColumn(SQueryInfo* pQueryInfo, int16_t functionId, int16_t tableIndex);
void tscDoQuery(SSqlObj* pSql);
SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *pInfo);
void* tscVgroupInfoClear(SVgroupsInfo *pInfo);
void tscSCMVgroupInfoCopy(SCMVgroupInfo* dst, const SCMVgroupInfo* src);
/**
* The create object function must be successful expect for the out of memory issue.
*
......
src/client/inc/tsclient.h
浏览文件 @ d74400f3
......@@ -90,10 +90,10 @@ typedef struct STableComInfo {
} STableComInfo;
typedef struct SCMCorVgroupInfo {
int32_t version;
int8_t inUse;
int8_t numOfEps;
SEpAddr epAddr[TSDB_MAX_REPLICA];
int32_t version;
int8_t inUse;
int8_t numOfEps;
SEpAddr1 epAddr[TSDB_MAX_REPLICA];
} SCMCorVgroupInfo;
typedef struct STableMeta {
......@@ -142,16 +142,17 @@ typedef struct SColumnIndex {
int16_t columnIndex;
} SColumnIndex;
typedef struct SFieldSupInfo {
typedef struct SInternalField {
TAOS_FIELD field;
bool visible;
SExprInfo *pArithExprInfo;
SSqlExpr *pSqlExpr;
} SFieldSupInfo;
} SInternalField;
typedef struct SFieldInfo {
int16_t numOfOutput; // number of column in result
SArray *pFields; // SArray<TAOS_FIELD>
SArray *pSupportInfo; // SArray<SFieldSupInfo>
int16_t numOfOutput; // number of column in result
TAOS_FIELD* final;
SArray *internalField; // SArray<SInternalField>
} SFieldInfo;
typedef struct SColumn {
......@@ -308,7 +309,7 @@ typedef struct {
int32_t numOfGroups;
SResRec * pGroupRec;
char * data;
void ** tsrow;
TAOS_ROW tsrow;
int32_t* length; // length for each field for current row
char ** buffer; // Buffer used to put multibytes encoded using unicode (wchar_t)
SColumnIndex * pColumnIndex;
......@@ -443,14 +444,14 @@ void tscPartiallyFreeSqlObj(SSqlObj *pSql);
*/
void tscFreeSqlObj(SSqlObj *pSql);
void tscFreeRegisteredSqlObj(void *pSql);
void tscFreeTableMetaHelper(void *pTableMeta);
void tscCloseTscObj(STscObj *pObj);
// todo move to taos? or create a new file: taos_internal.h
TAOS *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int),
void *param, void **taos);
void *param, TAOS **taos);
TAOS_RES* taos_query_h(TAOS* taos, const char *sqlstr, TAOS_RES** res);
void waitForQueryRsp(void *param, TAOS_RES *tres, int code);
void doAsyncQuery(STscObj *pObj, SSqlObj *pSql, __async_cb_func_t fp, void *param, const char *sqlstr, size_t sqlLen);
......@@ -468,7 +469,7 @@ int32_t tscSQLSyntaxErrMsg(char* msg, const char* additionalInfo, const char* s
int32_t tscToSQLCmd(SSqlObj *pSql, struct SSqlInfo *pInfo);
static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pFieldInfo, int32_t columnIndex) {
SFieldSupInfo* pInfo = (SFieldSupInfo*) TARRAY_GET_ELEM(pFieldInfo->pSupportInfo, columnIndex);
SInternalField* pInfo = (SInternalField*) TARRAY_GET_ELEM(pFieldInfo->internalField, columnIndex);
assert(pInfo->pSqlExpr != NULL);
int32_t type = pInfo->pSqlExpr->resType;
......@@ -481,11 +482,11 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField
if (type == TSDB_DATA_TYPE_NCHAR || type == TSDB_DATA_TYPE_BINARY) {
pData = pInfo->pSqlExpr->param[1].pz;
pRes->length[columnIndex] = pInfo->pSqlExpr->param[1].nLen;
pRes->tsrow[columnIndex] = (pInfo->pSqlExpr->param[1].nType == TSDB_DATA_TYPE_NULL) ? NULL : pData;
pRes->tsrow[columnIndex] = (pInfo->pSqlExpr->param[1].nType == TSDB_DATA_TYPE_NULL) ? NULL : (unsigned char*)pData;
} else {
assert(bytes == tDataTypeDesc[type].nSize);
pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : &pInfo->pSqlExpr->param[1].i64Key;
pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : (unsigned char*)&pInfo->pSqlExpr->param[1].i64Key;
pRes->length[columnIndex] = bytes;
}
} else {
......@@ -493,7 +494,7 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField
int32_t realLen = varDataLen(pData);
assert(realLen <= bytes - VARSTR_HEADER_SIZE);
pRes->tsrow[columnIndex] = (isNull(pData, type)) ? NULL : ((tstr *)pData)->data;
pRes->tsrow[columnIndex] = (isNull(pData, type)) ? NULL : (unsigned char*)((tstr *)pData)->data;
if (realLen < pInfo->pSqlExpr->resBytes - VARSTR_HEADER_SIZE) { // todo refactor
*(pData + realLen + VARSTR_HEADER_SIZE) = 0;
}
......@@ -502,7 +503,7 @@ static FORCE_INLINE void tscGetResultColumnChr(SSqlRes* pRes, SFieldInfo* pField
} else {
assert(bytes == tDataTypeDesc[type].nSize);
pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : pData;
pRes->tsrow[columnIndex] = isNull(pData, type) ? NULL : (unsigned char*)pData;
pRes->length[columnIndex] = bytes;
}
}
......
src/client/src/tscAsync.c
浏览文件 @ d74400f3
......@@ -327,7 +327,7 @@ void tscAsyncFetchSingleRowProxy(void *param, TAOS_RES *tres, int numOfRows) {
}
for (int i = 0; i < pCmd->numOfCols; ++i){
SFieldSupInfo* pSup = taosArrayGet(pQueryInfo->fieldsInfo.pSupportInfo, i);
SInternalField* pSup = taosArrayGet(pQueryInfo->fieldsInfo.internalField, i);
if (pSup->pSqlExpr != NULL) {
// pRes->tsrow[i] = TSC_GET_RESPTR_BASE(pRes, pQueryInfo, i) + pSup->pSqlExpr->resBytes * pRes->row;
} else {
......@@ -348,7 +348,7 @@ void tscProcessFetchRow(SSchedMsg *pMsg) {
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
for (int i = 0; i < pCmd->numOfCols; ++i) {
SFieldSupInfo* pSup = taosArrayGet(pQueryInfo->fieldsInfo.pSupportInfo, i);
SInternalField* pSup = taosArrayGet(pQueryInfo->fieldsInfo.internalField, i);
if (pSup->pSqlExpr != NULL) {
tscGetResultColumnChr(pRes, &pQueryInfo->fieldsInfo, i);
......@@ -405,11 +405,11 @@ void tscTableMetaCallBack(void *param, TAOS_RES *res, int code) {
SSqlRes *pRes = &pSql->res;
pRes->code = code;
const char* msg = (pCmd->command == TSDB_SQL_STABLEVGROUP)? "vgroup-list":"table-meta";
if (code != TSDB_CODE_SUCCESS) {
tscError("%p get tableMeta failed, code:%s", pSql, tstrerror(code));
tscError("%p get %s failed, code:%s", pSql, msg, tstrerror(code));
goto _error;
} else {
const char* msg = (pCmd->command == TSDB_SQL_STABLEVGROUP)? "vgroup-list":"table-meta";
tscDebug("%p get %s successfully", pSql, msg);
}
......
src/client/src/tscLocal.c
浏览文件 @ d74400f3
......@@ -239,7 +239,7 @@ static int32_t tscBuildTableSchemaResultFields(SSqlObj *pSql, int32_t numOfCols,
TAOS_FIELD f = {.type = TSDB_DATA_TYPE_BINARY, .bytes = (TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE};
tstrncpy(f.name, "Field", sizeof(f.name));
SFieldSupInfo* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
SInternalField* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY,
(TSDB_COL_NAME_LEN - 1) + VARSTR_HEADER_SIZE, (TSDB_COL_NAME_LEN - 1), false);
......@@ -296,7 +296,7 @@ static int32_t tscProcessDescribeTable(SSqlObj *pSql) {
return tscSetValueToResObj(pSql, rowLen);
}
static int32_t tscGetNthFieldResult(TAOS_ROW row, TAOS_FIELD* fields, int *lengths, int idx, char *result) {
const char *val = row[idx];
const char *val = (const char*)row[idx];
if (val == NULL) {
sprintf(result, "%s", TSDB_DATA_NULL_STR);
return -1;
......@@ -485,7 +485,7 @@ static int32_t tscSCreateBuildResultFields(SSqlObj *pSql, BuildType type, const
tstrncpy(f.name, "Database", sizeof(f.name));
}
SFieldSupInfo* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
SInternalField* pInfo = tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index, TSDB_DATA_TYPE_BINARY,
f.bytes, f.bytes - VARSTR_HEADER_SIZE, false);
......@@ -922,19 +922,17 @@ void tscSetLocalQueryResult(SSqlObj *pSql, const char *val, const char *columnNa
pQueryInfo->order.order = TSDB_ORDER_ASC;
tscFieldInfoClear(&pQueryInfo->fieldsInfo);
pQueryInfo->fieldsInfo.pFields = taosArrayInit(1, sizeof(TAOS_FIELD));
pQueryInfo->fieldsInfo.pSupportInfo = taosArrayInit(1, sizeof(SFieldSupInfo));
pQueryInfo->fieldsInfo.internalField = taosArrayInit(1, sizeof(SInternalField));
TAOS_FIELD f = tscCreateField((int8_t)type, columnName, (int16_t)valueLength);
tscFieldInfoAppend(&pQueryInfo->fieldsInfo, &f);
tscInitResObjForLocalQuery(pSql, 1, (int32_t)valueLength);
TAOS_FIELD *pField = tscFieldInfoGetField(&pQueryInfo->fieldsInfo, 0);
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, 0);
SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, 0);
pInfo->pSqlExpr = taosArrayGetP(pQueryInfo->exprList, 0);
memcpy(pRes->data, val, pField->bytes);
memcpy(pRes->data, val, pInfo->field.bytes);
}
int tscProcessLocalCmd(SSqlObj *pSql) {
......
src/client/src/tscLocalMerge.c
浏览文件 @ d74400f3
......@@ -510,7 +510,8 @@ void tscDestroyLocalReducer(SSqlObj *pSql) {
taosTFree(pLocalReducer->pResultBuf);
if (pLocalReducer->pResInfo != NULL) {
for (int32_t i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
size_t num = tscSqlExprNumOfExprs(pQueryInfo);
for (int32_t i = 0; i < num; ++i) {
taosTFree(pLocalReducer->pResInfo[i].interResultBuf);
}
......
src/client/src/tscPrepare.c
浏览文件 @ d74400f3
......@@ -139,7 +139,7 @@ static int normalStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
}
return TSDB_CODE_SUCCESS;
}
......@@ -213,7 +213,7 @@ static char* normalStmtBuildSql(STscStmt* stmt) {
case TSDB_DATA_TYPE_NULL:
taosStringBuilderAppendNull(&sb);
break;
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT:
......@@ -266,6 +266,388 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) {
return TSDB_CODE_SUCCESS;
}
if (1) {
// allow user bind param data with different type
short size = 0;
union {
int8_t v1;
int16_t v2;
int32_t v4;
int64_t v8;
float f4;
double f8;
unsigned char buf[32*1024];
} u;
switch (param->type) {
case TSDB_DATA_TYPE_BOOL: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL: {
u.v1 = *(int8_t*)bind->buffer;
if (u.v1==0 || u.v1==1) break;
} break;
case TSDB_DATA_TYPE_TINYINT: {
u.v1 = *(int8_t*)bind->buffer;
if (u.v1==0 || u.v1==1) break;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
u.v1 = (int8_t)*(int16_t*)bind->buffer;
if (u.v1==0 || u.v1==1) break;
} break;
case TSDB_DATA_TYPE_INT: {
u.v1 = (int8_t)*(int32_t*)bind->buffer;
if (u.v1==0 || u.v1==1) break;
} break;
case TSDB_DATA_TYPE_BIGINT: {
u.v1 = (int8_t)*(int64_t*)bind->buffer;
if (u.v1==0 || u.v1==1) break;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
// "0", "1" convertible
if (strncmp((const char*)bind->buffer, "0", *bind->length)==0) {
u.v1 = 0;
break;
}
if (strncmp((const char*)bind->buffer, "1", *bind->length)==0) {
u.v1 = 1;
break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.v1, sizeof(u.v1));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_TINYINT: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT: {
int8_t v = *(int8_t*)bind->buffer;
u.v1 = v;
if (v >= SCHAR_MIN && v <= SCHAR_MAX) break;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
int16_t v = *(int16_t*)bind->buffer;
u.v1 = (int8_t)v;
if (v >= SCHAR_MIN && v <= SCHAR_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_INT: {
int32_t v = *(int32_t*)bind->buffer;
u.v1 = (int8_t)v;
if (v >= SCHAR_MIN && v <= SCHAR_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BIGINT: {
int64_t v = *(int64_t*)bind->buffer;
u.v1 = (int8_t)v;
if (v >= SCHAR_MIN && v <= SCHAR_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
int64_t v;
int n,r;
r = sscanf((const char*)bind->buffer, "%" PRId64 "%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.v1 = (int8_t)v;
if (v >= SCHAR_MIN && v <= SCHAR_MAX) break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.v1, sizeof(u.v1));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT: {
int v = *(int16_t*)bind->buffer;
u.v2 = (int16_t)v;
} break;
case TSDB_DATA_TYPE_INT: {
int32_t v = *(int32_t*)bind->buffer;
u.v2 = (int16_t)v;
if (v >= SHRT_MIN && v <= SHRT_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BIGINT: {
int64_t v = *(int64_t*)bind->buffer;
u.v2 = (int16_t)v;
if (v >= SHRT_MIN && v <= SHRT_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
int64_t v;
int n,r;
r = sscanf((const char*)bind->buffer, "%" PRId64 "%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.v2 = (int16_t)v;
if (v >= SHRT_MIN && v <= SHRT_MAX) break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.v2, sizeof(u.v2));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_INT: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT:
case TSDB_DATA_TYPE_INT: {
u.v4 = *(int32_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BIGINT: {
int64_t v = *(int64_t*)bind->buffer;
u.v4 = (int32_t)v;
if (v >= INT_MIN && v <= INT_MAX) break;
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
int64_t v;
int n,r;
r = sscanf((const char*)bind->buffer, "%" PRId64 "%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.v4 = (int32_t)v;
if (v >= INT_MIN && v <= INT_MAX) break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.v2, sizeof(u.v2));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_FLOAT: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT: {
u.f4 = *(int8_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
u.f4 = *(int16_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_INT: {
u.f4 = (float)*(int32_t*)bind->buffer;
// shall we check equality?
} break;
case TSDB_DATA_TYPE_BIGINT: {
u.f4 = (float)*(int64_t*)bind->buffer;
// shall we check equality?
} break;
case TSDB_DATA_TYPE_FLOAT: {
u.f4 = *(float*)bind->buffer;
} break;
case TSDB_DATA_TYPE_DOUBLE: {
u.f4 = *(float*)bind->buffer;
// shall we check equality?
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
float v;
int n,r;
r = sscanf((const char*)bind->buffer, "%f%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.f4 = v;
break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.f4, sizeof(u.f4));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_BIGINT: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT: {
u.v8 = *(int8_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
u.v8 = *(int16_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_INT: {
u.v8 = *(int32_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BIGINT: {
u.v8 = *(int64_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
int64_t v;
int n,r;
r = sscanf((const char*)bind->buffer, "%" PRId64 "%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.v8 = v;
break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
memcpy(data + param->offset, &u.v8, sizeof(u.v8));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_DOUBLE: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT: {
u.f8 = *(int8_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_SMALLINT: {
u.f8 = *(int16_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_INT: {
u.f8 = *(int32_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BIGINT: {
u.f8 = (double)*(int64_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_FLOAT: {
u.f8 = *(float*)bind->buffer;
} break;
case TSDB_DATA_TYPE_DOUBLE: {
u.f8 = *(double*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
double v;
int n,r;
r = sscanf((const char*)bind->buffer, "%lf%n", &v, &n);
if (r==1 && n==strlen((const char*)bind->buffer)) {
u.f8 = v;
break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_TIMESTAMP:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
} break;
memcpy(data + param->offset, &u.f8, sizeof(u.f8));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_TIMESTAMP: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_TIMESTAMP: {
u.v8 = *(int64_t*)bind->buffer;
} break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR: {
// is this the correct way to call taosParseTime?
int32_t len = (int32_t)*bind->length;
if (taosParseTime(bind->buffer, &u.v8, len, 3, tsDaylight) == TSDB_CODE_SUCCESS) {
break;
}
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT:
case TSDB_DATA_TYPE_INT:
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_BIGINT:
case TSDB_DATA_TYPE_DOUBLE:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
} break;
memcpy(data + param->offset, &u.v8, sizeof(u.v8));
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_BINARY: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_BINARY: {
if ((*bind->length) > (uintptr_t)param->bytes) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
size = (short)*bind->length;
STR_WITH_SIZE_TO_VARSTR(data + param->offset, bind->buffer, size);
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT:
case TSDB_DATA_TYPE_INT:
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_BIGINT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
case TSDB_DATA_TYPE_NCHAR:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
} break;
case TSDB_DATA_TYPE_NCHAR: {
switch (bind->buffer_type) {
case TSDB_DATA_TYPE_NCHAR: {
size_t output = 0;
if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
varDataSetLen(data + param->offset, output);
return TSDB_CODE_SUCCESS;
} break;
case TSDB_DATA_TYPE_BOOL:
case TSDB_DATA_TYPE_TINYINT:
case TSDB_DATA_TYPE_SMALLINT:
case TSDB_DATA_TYPE_INT:
case TSDB_DATA_TYPE_FLOAT:
case TSDB_DATA_TYPE_BIGINT:
case TSDB_DATA_TYPE_DOUBLE:
case TSDB_DATA_TYPE_TIMESTAMP:
case TSDB_DATA_TYPE_BINARY:
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
} break;
default: {
return TSDB_CODE_TSC_INVALID_VALUE;
} break;
}
}
if (bind->buffer_type != param->type) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
......@@ -299,12 +681,12 @@ static int doBindParam(char* data, SParamInfo* param, TAOS_BIND* bind) {
size = (short)*bind->length;
STR_WITH_SIZE_TO_VARSTR(data + param->offset, bind->buffer, size);
return TSDB_CODE_SUCCESS;
case TSDB_DATA_TYPE_NCHAR: {
size_t output = 0;
if (!taosMbsToUcs4(bind->buffer, *bind->length, varDataVal(data + param->offset), param->bytes - VARSTR_HEADER_SIZE, &output)) {
return TSDB_CODE_TSC_INVALID_VALUE;
}
}
varDataSetLen(data + param->offset, output);
return TSDB_CODE_SUCCESS;
}
......@@ -358,7 +740,7 @@ static int insertStmtBindParam(STscStmt* stmt, TAOS_BIND* bind) {
}
// actual work of all data blocks is done, update block size and numOfRows.
// note we don't do this block by block during the binding process, because
// note we don't do this block by block during the binding process, because
// we cannot recover if something goes wrong.
pCmd->batchSize = binded * 2 + 1;
......@@ -405,7 +787,7 @@ static int insertStmtReset(STscStmt* pStmt) {
}
}
pCmd->batchSize = 0;
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, 0);
pTableMetaInfo->vgroupIndex = 0;
return TSDB_CODE_SUCCESS;
......@@ -447,7 +829,7 @@ static int insertStmtExecute(STscStmt* stmt) {
pRes->numOfRows = 0;
pRes->numOfTotal = 0;
pRes->numOfClauseTotal = 0;
pRes->qhandle = 0;
pSql->cmd.insertType = 0;
......@@ -508,35 +890,35 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) {
SSqlObj* pSql = pStmt->pSql;
size_t sqlLen = strlen(sql);
SSqlCmd *pCmd = &pSql->cmd;
SSqlRes *pRes = &pSql->res;
pSql->param = (void*) pSql;
pSql->fp = waitForQueryRsp;
pSql->cmd.insertType = TSDB_QUERY_TYPE_STMT_INSERT;
if (TSDB_CODE_SUCCESS != tscAllocPayload(pCmd, TSDB_DEFAULT_PAYLOAD_SIZE)) {
tscError("%p failed to malloc payload buffer", pSql);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
pSql->sqlstr = realloc(pSql->sqlstr, sqlLen + 1);
if (pSql->sqlstr == NULL) {
tscError("%p failed to malloc sql string buffer", pSql);
free(pCmd->payload);
return TSDB_CODE_TSC_OUT_OF_MEMORY;
}
pRes->qhandle = 0;
pRes->numOfRows = 1;
strtolower(pSql->sqlstr, sql);
tscDebugL("%p SQL: %s", pSql, pSql->sqlstr);
if (tscIsInsertData(pSql->sqlstr)) {
if (tscIsInsertData(pSql->sqlstr)) {
pStmt->isInsert = true;
pSql->cmd.numOfParams = 0;
pSql->cmd.batchSize = 0;
......@@ -548,7 +930,7 @@ int taos_stmt_prepare(TAOS_STMT* stmt, const char* sql, unsigned long length) {
tsem_wait(&pSql->rspSem);
return pSql->res.code;
}
return code;
}
......@@ -637,3 +1019,80 @@ TAOS_RES *taos_stmt_use_result(TAOS_STMT* stmt) {
pStmt->pSql = NULL;
return result;
}
int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert) {
STscStmt* pStmt = (STscStmt*)stmt;
if (stmt == NULL || pStmt->taos == NULL || pStmt->pSql == NULL) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return TSDB_CODE_TSC_DISCONNECTED;
}
if (insert) *insert = pStmt->isInsert;
return TSDB_CODE_SUCCESS;
}
int taos_stmt_num_params(TAOS_STMT *stmt, int *nums) {
STscStmt* pStmt = (STscStmt*)stmt;
if (stmt == NULL || pStmt->taos == NULL || pStmt->pSql == NULL) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return TSDB_CODE_TSC_DISCONNECTED;
}
if (pStmt->isInsert) {
SSqlObj* pSql = pStmt->pSql;
SSqlCmd *pCmd = &pSql->cmd;
*nums = pCmd->numOfParams;
return TSDB_CODE_SUCCESS;
} else {
SNormalStmt* normal = &pStmt->normal;
*nums = normal->numParams;
return TSDB_CODE_SUCCESS;
}
}
int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes) {
STscStmt* pStmt = (STscStmt*)stmt;
if (stmt == NULL || pStmt->taos == NULL || pStmt->pSql == NULL) {
terrno = TSDB_CODE_TSC_DISCONNECTED;
return TSDB_CODE_TSC_DISCONNECTED;
}
if (pStmt->isInsert) {
SSqlObj* pSql = pStmt->pSql;
SSqlCmd *pCmd = &pSql->cmd;
STableDataBlocks* pBlock = taosArrayGetP(pCmd->pDataBlocks, 0);
assert(pCmd->numOfParams == pBlock->numOfParams);
if (idx < 0 || idx >= pBlock->numOfParams) return -1;
SParamInfo* param = pBlock->params + idx;
if (type) *type = param->type;
if (bytes) *bytes = param->bytes;
return TSDB_CODE_SUCCESS;
} else {
return TSDB_CODE_TSC_APP_ERROR;
}
}
const char *taos_data_type(int type) {
switch (type) {
case TSDB_DATA_TYPE_NULL: return "TSDB_DATA_TYPE_NULL";
case TSDB_DATA_TYPE_BOOL: return "TSDB_DATA_TYPE_BOOL";
case TSDB_DATA_TYPE_TINYINT: return "TSDB_DATA_TYPE_TINYINT";
case TSDB_DATA_TYPE_SMALLINT: return "TSDB_DATA_TYPE_SMALLINT";
case TSDB_DATA_TYPE_INT: return "TSDB_DATA_TYPE_INT";
case TSDB_DATA_TYPE_BIGINT: return "TSDB_DATA_TYPE_BIGINT";
case TSDB_DATA_TYPE_FLOAT: return "TSDB_DATA_TYPE_FLOAT";
case TSDB_DATA_TYPE_DOUBLE: return "TSDB_DATA_TYPE_DOUBLE";
case TSDB_DATA_TYPE_BINARY: return "TSDB_DATA_TYPE_BINARY";
case TSDB_DATA_TYPE_TIMESTAMP: return "TSDB_DATA_TYPE_TIMESTAMP";
case TSDB_DATA_TYPE_NCHAR: return "TSDB_DATA_TYPE_NCHAR";
default: return "UNKNOWN";
}
}
src/client/src/tscProfile.c
浏览文件 @ d74400f3
......@@ -20,8 +20,10 @@
#include "tutil.h"
#include "taosmsg.h"
#include "taos.h"
void tscSaveSlowQueryFp(void *handle, void *tmrId);
void *tscSlowQueryConn = NULL;
TAOS *tscSlowQueryConn = NULL;
bool tscSlowQueryConnInitialized = false;
void tscInitConnCb(void *param, TAOS_RES *result, int code) {
......
src/client/src/tscSQLParser.c
浏览文件 @ d74400f3
......@@ -122,7 +122,7 @@ static int32_t doCheckForCreateTable(SSqlObj* pSql, int32_t subClauseIndex, SSql
static int32_t doCheckForCreateFromStable(SSqlObj* pSql, SSqlInfo* pInfo);
static int32_t doCheckForStream(SSqlObj* pSql, SSqlInfo* pInfo);
static int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index);
static int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSQLExpr* pSqlExpr, SArray* pExprInfo, SQueryInfo* pQueryInfo, SArray* pCols);
static int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSQLExpr* pSqlExpr, SQueryInfo* pQueryInfo, SArray* pCols, int64_t *uid);
/*
* Used during parsing query sql. Since the query sql usually small in length, error position
......@@ -190,7 +190,8 @@ int32_t tscToSQLCmd(SSqlObj* pSql, struct SSqlInfo* pInfo) {
SSqlRes* pRes = &pSql->res;
int32_t code = TSDB_CODE_SUCCESS;
if (!pInfo->valid) {
if (!pInfo->valid || terrno == TSDB_CODE_TSC_SQL_SYNTAX_ERROR) {
terrno = TSDB_CODE_SUCCESS; // clear the error number
return tscSQLSyntaxErrMsg(tscGetErrorMsgPayload(pCmd), NULL, pInfo->pzErrMsg);
}
......@@ -1249,7 +1250,7 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, int32_t clauseIndex, int32_t
tExprNode* pNode = NULL;
SArray* colList = taosArrayInit(10, sizeof(SColIndex));
int32_t ret = exprTreeFromSqlExpr(pCmd, &pNode, pItem->pNode, pQueryInfo->exprList, pQueryInfo, colList);
int32_t ret = exprTreeFromSqlExpr(pCmd, &pNode, pItem->pNode, pQueryInfo, colList, NULL);
if (ret != TSDB_CODE_SUCCESS) {
taosArrayDestroy(colList);
tExprTreeDestroy(&pNode, NULL);
......@@ -1305,17 +1306,17 @@ static int32_t handleArithmeticExpr(SSqlCmd* pCmd, int32_t clauseIndex, int32_t
insertResultField(pQueryInfo, exprIndex, &columnList, sizeof(double), TSDB_DATA_TYPE_DOUBLE, aliasName, NULL);
int32_t slot = tscNumOfFields(pQueryInfo) - 1;
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, slot);
SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, slot);
if (pInfo->pSqlExpr == NULL) {
SExprInfo* pArithExprInfo = calloc(1, sizeof(SExprInfo));
// arithmetic expression always return result in the format of double float
pArithExprInfo->bytes = sizeof(double);
pArithExprInfo->bytes = sizeof(double);
pArithExprInfo->interBytes = sizeof(double);
pArithExprInfo->type = TSDB_DATA_TYPE_DOUBLE;
pArithExprInfo->type = TSDB_DATA_TYPE_DOUBLE;
int32_t ret = exprTreeFromSqlExpr(pCmd, &pArithExprInfo->pExpr, pItem->pNode, pQueryInfo->exprList, pQueryInfo, NULL);
int32_t ret = exprTreeFromSqlExpr(pCmd, &pArithExprInfo->pExpr, pItem->pNode, pQueryInfo, NULL, &pArithExprInfo->uid);
if (ret != TSDB_CODE_SUCCESS) {
tExprTreeDestroy(&pArithExprInfo->pExpr, NULL);
return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), "invalid expression in select clause");
......@@ -1372,7 +1373,7 @@ static void addPrimaryTsColIntoResult(SQueryInfo* pQueryInfo) {
int32_t numOfCols = (int32_t)tscSqlExprNumOfExprs(pQueryInfo);
tscAddSpecialColumnForSelect(pQueryInfo, numOfCols, TSDB_FUNC_PRJ, &index, pSchema, TSDB_COL_NORMAL);
SFieldSupInfo* pSupInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, numOfCols);
SInternalField* pSupInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, numOfCols);
pSupInfo->visible = false;
pQueryInfo->type |= TSDB_QUERY_TYPE_PROJECTION_QUERY;
......@@ -1469,7 +1470,7 @@ int32_t insertResultField(SQueryInfo* pQueryInfo, int32_t outputIndex, SColumnLi
}
TAOS_FIELD f = tscCreateField(type, fieldName, bytes);
SFieldSupInfo* pInfo = tscFieldInfoInsert(&pQueryInfo->fieldsInfo, outputIndex, &f);
SInternalField* pInfo = tscFieldInfoInsert(&pQueryInfo->fieldsInfo, outputIndex, &f);
pInfo->pSqlExpr = pSqlExpr;
return TSDB_CODE_SUCCESS;
......@@ -3384,10 +3385,26 @@ static int32_t validateSQLExpr(SSqlCmd* pCmd, tSQLExpr* pExpr, SQueryInfo* pQuer
tSQLExprItem item = {.pNode = pExpr, .aliasName = NULL};
// sql function in selection clause, append sql function info in pSqlCmd structure sequentially
// sql function list in selection clause.
// Append the sqlExpr into exprList of pQueryInfo structure sequentially
if (addExprAndResultField(pCmd, pQueryInfo, outputIndex, &item, false) != TSDB_CODE_SUCCESS) {
return TSDB_CODE_TSC_INVALID_SQL;
}
// It is invalid in case of more than one sqlExpr, such as first(ts, k) - last(ts, k)
int32_t inc = (int32_t) tscSqlExprNumOfExprs(pQueryInfo) - outputIndex;
if (inc > 1) {
return TSDB_CODE_TSC_INVALID_SQL;
}
// Not supported data type in arithmetic expression
for(int32_t i = 0; i < inc; ++i) {
SSqlExpr* p1 = tscSqlExprGet(pQueryInfo, i + outputIndex);
int16_t t = p1->resType;
if (t == TSDB_DATA_TYPE_BINARY || t == TSDB_DATA_TYPE_NCHAR || t == TSDB_DATA_TYPE_BOOL || t == TSDB_DATA_TYPE_TIMESTAMP) {
return TSDB_CODE_TSC_INVALID_SQL;
}
}
}
return TSDB_CODE_SUCCESS;
......@@ -4062,7 +4079,7 @@ static int32_t getTagQueryCondExpr(SSqlCmd* pCmd, SQueryInfo* pQueryInfo, SCondE
tExprNode* p = NULL;
SArray* colList = taosArrayInit(10, sizeof(SColIndex));
ret = exprTreeFromSqlExpr(pCmd, &p, p1, NULL, pQueryInfo, colList);
ret = exprTreeFromSqlExpr(pCmd, &p, p1, pQueryInfo, colList, NULL);
SBufferWriter bw = tbufInitWriter(NULL, false);
TRY(0) {
......@@ -4733,7 +4750,7 @@ int32_t setAlterTableInfo(SSqlObj* pSql, struct SSqlInfo* pInfo) {
// validate the length of binary
if ((pTagsSchema->type == TSDB_DATA_TYPE_BINARY || pTagsSchema->type == TSDB_DATA_TYPE_NCHAR) &&
(pVarList->a[1].pVar.nLen + VARSTR_HEADER_SIZE) > pTagsSchema->bytes) {
varDataTLen(pAlterSQL->tagData.data) > pTagsSchema->bytes) {
return invalidSqlErrMsg(tscGetErrorMsgPayload(pCmd), msg14);
}
......@@ -5259,26 +5276,6 @@ int32_t parseCreateDBOptions(SSqlCmd* pCmd, SCreateDBInfo* pCreateDbSql) {
return TSDB_CODE_SUCCESS;
}
//void tscAddTimestampColumn(SQueryInfo* pQueryInfo, int16_t functionId, int16_t tableIndex) {
// // the first column not timestamp column, add it
// SSqlExpr* pExpr = NULL;
// if (tscSqlExprNumOfExprs(pQueryInfo) > 0) {
// pExpr = tscSqlExprGet(pQueryInfo, 0);
// }
//
// if (pExpr == NULL || pExpr->colInfo.colId != PRIMARYKEY_TIMESTAMP_COL_INDEX || pExpr->functionId != functionId) {
// SColumnIndex index = {tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX};
//
// pExpr = tscSqlExprInsert(pQueryInfo, 0, functionId, &index, TSDB_DATA_TYPE_TIMESTAMP, TSDB_KEYSIZE, TSDB_KEYSIZE, false);
// pExpr->colInfo.flag = TSDB_COL_NORMAL;
//
// // NOTE: tag column does not add to source column list
// SColumnList ids = getColumnList(1, tableIndex, PRIMARYKEY_TIMESTAMP_COL_INDEX);
//
// insertResultField(pQueryInfo, 0, &ids, TSDB_KEYSIZE, TSDB_DATA_TYPE_TIMESTAMP, "ts", pExpr);
// }
//}
void addGroupInfoForSubquery(SSqlObj* pParentObj, SSqlObj* pSql, int32_t subClauseIndex, int32_t tableIndex) {
SQueryInfo* pParentQueryInfo = tscGetQueryInfoDetail(&pParentObj->cmd, subClauseIndex);
......@@ -5335,7 +5332,7 @@ void doAddGroupColumnForSubquery(SQueryInfo* pQueryInfo, int32_t tagIndex) {
tscAddSpecialColumnForSelect(pQueryInfo, (int32_t)size, TSDB_FUNC_PRJ, &colIndex, pSchema, TSDB_COL_NORMAL);
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, (int32_t)size);
SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, (int32_t)size);
doLimitOutputNormalColOfGroupby(pInfo->pSqlExpr);
pInfo->visible = false;
}
......@@ -6380,19 +6377,19 @@ int32_t doCheckForQuery(SSqlObj* pSql, SQuerySQL* pQuerySql, int32_t index) {
return TSDB_CODE_SUCCESS; // Does not build query message here
}
int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSQLExpr* pSqlExpr, SArray* pExprInfo, SQueryInfo* pQueryInfo, SArray* pCols) {
int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSQLExpr* pSqlExpr, SQueryInfo* pQueryInfo, SArray* pCols, int64_t *uid) {
tExprNode* pLeft = NULL;
tExprNode* pRight= NULL;
if (pSqlExpr->pLeft != NULL) {
int32_t ret = exprTreeFromSqlExpr(pCmd, &pLeft, pSqlExpr->pLeft, pExprInfo, pQueryInfo, pCols);
int32_t ret = exprTreeFromSqlExpr(pCmd, &pLeft, pSqlExpr->pLeft, pQueryInfo, pCols, uid);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
}
if (pSqlExpr->pRight != NULL) {
int32_t ret = exprTreeFromSqlExpr(pCmd, &pRight, pSqlExpr->pRight, pExprInfo, pQueryInfo, pCols);
int32_t ret = exprTreeFromSqlExpr(pCmd, &pRight, pSqlExpr->pRight, pQueryInfo, pCols, uid);
if (ret != TSDB_CODE_SUCCESS) {
return ret;
}
......@@ -6419,14 +6416,19 @@ int32_t exprTreeFromSqlExpr(SSqlCmd* pCmd, tExprNode **pExpr, const tSQLExpr* pS
strncpy((*pExpr)->pSchema->name, pSqlExpr->operand.z, pSqlExpr->operand.n);
// set the input column data byte and type.
size_t size = taosArrayGetSize(pExprInfo);
size_t size = taosArrayGetSize(pQueryInfo->exprList);
for (int32_t i = 0; i < size; ++i) {
SSqlExpr* p1 = taosArrayGetP(pExprInfo, i);
SSqlExpr* p1 = taosArrayGetP(pQueryInfo->exprList, i);
if (strcmp((*pExpr)->pSchema->name, p1->aliasName) == 0) {
(*pExpr)->pSchema->type = (uint8_t)p1->resType;
(*pExpr)->pSchema->type = (uint8_t)p1->resType;
(*pExpr)->pSchema->bytes = p1->resBytes;
if (uid != NULL) {
*uid = p1->uid;
}
break;
}
}
......
src/client/src/tscSchemaUtil.c
浏览文件 @ d74400f3
......@@ -145,10 +145,11 @@ static void tscInitCorVgroupInfo(SCMCorVgroupInfo *corVgroupInfo, SCMVgroupInfo
corVgroupInfo->inUse = 0;
corVgroupInfo->numOfEps = vgroupInfo->numOfEps;
for (int32_t i = 0; i < corVgroupInfo->numOfEps; i++) {
strncpy(corVgroupInfo->epAddr[i].fqdn, vgroupInfo->epAddr[i].fqdn, TSDB_FQDN_LEN);
corVgroupInfo->epAddr[i].fqdn = strdup(vgroupInfo->epAddr[i].fqdn);
corVgroupInfo->epAddr[i].port = vgroupInfo->epAddr[i].port;
}
}
STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size) {
assert(pTableMetaMsg != NULL);
......@@ -162,9 +163,19 @@ STableMeta* tscCreateTableMetaFromMsg(STableMetaMsg* pTableMetaMsg, size_t* size
.numOfColumns = pTableMetaMsg->numOfColumns,
};
pTableMeta->id.tid = pTableMetaMsg->sid;
pTableMeta->id.tid = pTableMetaMsg->tid;
pTableMeta->id.uid = pTableMetaMsg->uid;
pTableMeta->vgroupInfo = pTableMetaMsg->vgroup;
SCMVgroupInfo* pVgroupInfo = &pTableMeta->vgroupInfo;
pVgroupInfo->numOfEps = pTableMetaMsg->vgroup.numOfEps;
pVgroupInfo->vgId = pTableMetaMsg->vgroup.vgId;
for(int32_t i = 0; i < pVgroupInfo->numOfEps; ++i) {
SEpAddrMsg* pEpMsg = &pTableMetaMsg->vgroup.epAddr[i];
pVgroupInfo->epAddr[i].fqdn = strndup(pEpMsg->fqdn, tListLen(pEpMsg->fqdn));
pVgroupInfo->epAddr[i].port = pEpMsg->port;
}
tscInitCorVgroupInfo(&pTableMeta->corVgroupInfo, &pTableMeta->vgroupInfo);
......
src/client/src/tscServer.c
浏览文件 @ d74400f3
......@@ -124,9 +124,11 @@ static void tscUpdateVgroupInfo(SSqlObj *pObj, SRpcEpSet *pEpSet) {
pVgroupInfo->inUse = pEpSet->inUse;
pVgroupInfo->numOfEps = pEpSet->numOfEps;
for (int32_t i = 0; i < pVgroupInfo->numOfEps; i++) {
tstrncpy(pVgroupInfo->epAddr[i].fqdn, pEpSet->fqdn[i], TSDB_FQDN_LEN);
taosTFree(pVgroupInfo->epAddr[i].fqdn);
pVgroupInfo->epAddr[i].fqdn = strndup(pEpSet->fqdn[i], tListLen(pEpSet->fqdn[i]));
pVgroupInfo->epAddr[i].port = pEpSet->port[i];
}
tscDebug("after: EndPoint in use: %d", pVgroupInfo->inUse);
taosCorEndWrite(&pVgroupInfo->version);
}
......@@ -1366,7 +1368,7 @@ static int tscSetResultPointer(SQueryInfo *pQueryInfo, SSqlRes *pRes) {
for (int i = 0; i < pQueryInfo->fieldsInfo.numOfOutput; ++i) {
int16_t offset = tscFieldInfoGetOffset(pQueryInfo, i);
pRes->tsrow[i] = ((char*) pRes->data + offset * pRes->numOfRows);
pRes->tsrow[i] = (unsigned char*)((char*) pRes->data + offset * pRes->numOfRows);
}
return 0;
......@@ -1648,7 +1650,7 @@ int tscBuildHeartBeatMsg(SSqlObj *pSql, SSqlInfo *pInfo) {
int tscProcessTableMetaRsp(SSqlObj *pSql) {
STableMetaMsg *pMetaMsg = (STableMetaMsg *)pSql->res.pRsp;
pMetaMsg->sid = htonl(pMetaMsg->sid);
pMetaMsg->tid = htonl(pMetaMsg->tid);
pMetaMsg->sversion = htons(pMetaMsg->sversion);
pMetaMsg->tversion = htons(pMetaMsg->tversion);
pMetaMsg->vgroup.vgId = htonl(pMetaMsg->vgroup.vgId);
......@@ -1658,9 +1660,9 @@ int tscProcessTableMetaRsp(SSqlObj *pSql) {
pMetaMsg->numOfColumns = htons(pMetaMsg->numOfColumns);
if ((pMetaMsg->tableType != TSDB_SUPER_TABLE) &&
(pMetaMsg->sid <= 0 || pMetaMsg->vgroup.vgId < 2 || pMetaMsg->vgroup.numOfEps <= 0)) {
(pMetaMsg->tid <= 0 || pMetaMsg->vgroup.vgId < 2 || pMetaMsg->vgroup.numOfEps <= 0)) {
tscError("invalid value in table numOfEps:%d, vgId:%d tid:%d, name:%s", pMetaMsg->vgroup.numOfEps, pMetaMsg->vgroup.vgId,
pMetaMsg->sid, pMetaMsg->tableId);
pMetaMsg->tid, pMetaMsg->tableId);
return TSDB_CODE_TSC_INVALID_VALUE;
}
......@@ -1839,22 +1841,30 @@ int tscProcessSTableVgroupRsp(SSqlObj *pSql) {
SSqlCmd* pCmd = &parent->cmd;
for(int32_t i = 0; i < pStableVgroup->numOfTables; ++i) {
STableMetaInfo *pInfo = tscGetTableMetaInfoFromCmd(pCmd, pCmd->clauseIndex, i);
SVgroupsInfo * pVgroupInfo = (SVgroupsInfo *)pMsg;
pVgroupInfo->numOfVgroups = htonl(pVgroupInfo->numOfVgroups);
size_t size = sizeof(SCMVgroupInfo) * pVgroupInfo->numOfVgroups + sizeof(SVgroupsInfo);
pInfo->vgroupList = calloc(1, size);
SVgroupsMsg * pVgroupMsg = (SVgroupsMsg *) pMsg;
pVgroupMsg->numOfVgroups = htonl(pVgroupMsg->numOfVgroups);
size_t size = sizeof(SCMVgroupMsg) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsMsg);
size_t vgroupsz = sizeof(SCMVgroupInfo) * pVgroupMsg->numOfVgroups + sizeof(SVgroupsInfo);
pInfo->vgroupList = calloc(1, vgroupsz);
assert(pInfo->vgroupList != NULL);
memcpy(pInfo->vgroupList, pVgroupInfo, size);
pInfo->vgroupList->numOfVgroups = pVgroupMsg->numOfVgroups;
for (int32_t j = 0; j < pInfo->vgroupList->numOfVgroups; ++j) {
//just init, no need to lock
SCMVgroupInfo *pVgroups = &pInfo->vgroupList->vgroups[j];
pVgroups->vgId = htonl(pVgroups->vgId);
assert(pVgroups->numOfEps >= 1);
SCMVgroupMsg *vmsg = &pVgroupMsg->vgroups[j];
pVgroups->vgId = htonl(vmsg->vgId);
pVgroups->numOfEps = vmsg->numOfEps;
assert(pVgroups->numOfEps >= 1 && pVgroups->vgId >= 1);
for (int32_t k = 0; k < pVgroups->numOfEps; ++k) {
pVgroups->epAddr[k].port = htons(pVgroups->epAddr[k].port);
pVgroups->epAddr[k].port = htons(vmsg->epAddr[k].port);
pVgroups->epAddr[k].fqdn = strndup(vmsg->epAddr[k].fqdn, tListLen(vmsg->epAddr[k].fqdn));
}
}
......@@ -1890,7 +1900,7 @@ int tscProcessShowRsp(SSqlObj *pSql) {
pMetaMsg->numOfColumns = ntohs(pMetaMsg->numOfColumns);
pSchema = pMetaMsg->schema;
pMetaMsg->sid = ntohs(pMetaMsg->sid);
pMetaMsg->tid = ntohs(pMetaMsg->tid);
for (int i = 0; i < pMetaMsg->numOfColumns; ++i) {
pSchema->bytes = htons(pSchema->bytes);
pSchema++;
......@@ -1924,7 +1934,7 @@ int tscProcessShowRsp(SSqlObj *pSql) {
tscColumnListInsert(pQueryInfo->colList, &index);
TAOS_FIELD f = tscCreateField(pSchema->type, pSchema->name, pSchema->bytes);
SFieldSupInfo* pInfo = tscFieldInfoAppend(pFieldInfo, &f);
SInternalField* pInfo = tscFieldInfoAppend(pFieldInfo, &f);
pInfo->pSqlExpr = tscSqlExprAppend(pQueryInfo, TSDB_FUNC_TS_DUMMY, &index,
pTableSchema[i].type, pTableSchema[i].bytes, pTableSchema[i].bytes, false);
......
src/client/src/tscSql.c
浏览文件 @ d74400f3
......@@ -51,8 +51,8 @@ static bool validPassword(const char* passwd) {
return validImpl(passwd, TSDB_PASSWORD_LEN - 1);
}
SSqlObj *taosConnectImpl(const char *ip, const char *user, const char *pass, const char *auth, const char *db,
uint16_t port, void (*fp)(void *, TAOS_RES *, int), void *param, void **taos) {
static SSqlObj *taosConnectImpl(const char *ip, const char *user, const char *pass, const char *auth, const char *db,
uint16_t port, void (*fp)(void *, TAOS_RES *, int), void *param, TAOS **taos) {
taos_init();
if (!validUserName(user)) {
......@@ -243,16 +243,19 @@ static void asyncConnCallback(void *param, TAOS_RES *tres, int code) {
}
TAOS *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int),
void *param, void **taos) {
SSqlObj* pSql = taosConnectImpl(ip, user, pass, NULL, db, port, asyncConnCallback, param, taos);
void *param, TAOS **taos) {
STscObj *pObj = NULL;
SSqlObj *pSql = taosConnectImpl(ip, user, pass, NULL, db, port, asyncConnCallback, param, (void **)&pObj);
if (pSql == NULL) {
return NULL;
}
if (taos) *taos = pObj;
pSql->fetchFp = fp;
pSql->res.code = tscProcessSql(pSql);
tscDebug("%p DB async connection is opening", taos);
return taos;
return pObj;
}
void taos_close(TAOS *taos) {
......@@ -370,7 +373,7 @@ int taos_num_fields(TAOS_RES *res) {
size_t numOfCols = tscNumOfFields(pQueryInfo);
for(int32_t i = 0; i < numOfCols; ++i) {
SFieldSupInfo* pInfo = taosArrayGet(pQueryInfo->fieldsInfo.pSupportInfo, i);
SInternalField* pInfo = taosArrayGet(pQueryInfo->fieldsInfo.internalField, i);
if (pInfo->visible) {
num++;
}
......@@ -406,8 +409,24 @@ TAOS_FIELD *taos_fetch_fields(TAOS_RES *res) {
if (numOfCols == 0) {
return NULL;
}
return pQueryInfo->fieldsInfo.pFields->pData;
SFieldInfo *pFieldInfo = &pQueryInfo->fieldsInfo;
if (pFieldInfo->final == NULL) {
TAOS_FIELD* f = calloc(pFieldInfo->numOfOutput, sizeof(TAOS_FIELD));
int32_t j = 0;
for(int32_t i = 0; i < pFieldInfo->numOfOutput; ++i) {
SInternalField* pField = tscFieldInfoGetInternalField(pFieldInfo, i);
if (pField->visible) {
f[j++] = pField->field;
}
}
pFieldInfo->final = f;
}
return pFieldInfo->final;
}
int taos_retrieve(TAOS_RES *res) {
......@@ -588,7 +607,8 @@ static UNUSED_FUNC bool tscKillQueryInDnode(SSqlObj* pSql) {
return true;
}
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, 0);
if ((pQueryInfo == NULL) || tscIsTwoStageSTableQuery(pQueryInfo, 0)) {
return true;
}
......@@ -702,6 +722,7 @@ static void tscKillSTableQuery(SSqlObj *pSql) {
SSqlCmd* pCmd = &pSql->cmd;
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
if (!tscIsTwoStageSTableQuery(pQueryInfo, 0)) {
return;
}
......@@ -750,6 +771,7 @@ void taos_stop_query(TAOS_RES *res) {
pSql->res.code = TSDB_CODE_TSC_QUERY_CANCELLED;
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
if (tscIsTwoStageSTableQuery(pQueryInfo, 0)) {
assert(pSql->pRpcCtx == NULL);
tscKillSTableQuery(pSql);
......@@ -878,7 +900,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
if (pSql->sqlstr == NULL) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
tscError("%p failed to malloc sql string buffer", pSql);
tscDebug("%p Valid SQL result:%d, %s pObj:%p", pSql, pRes->code, taos_errstr(taos), pObj);
tscDebug("%p Valid SQL result:%d, %s pObj:%p", pSql, pRes->code, taos_errstr(pSql), pObj);
taosTFree(pSql);
return pRes->code;
}
......@@ -903,7 +925,7 @@ int taos_validate_sql(TAOS *taos, const char *sql) {
}
if (code != TSDB_CODE_SUCCESS) {
tscDebug("%p Valid SQL result:%d, %s pObj:%p", pSql, code, taos_errstr(taos), pObj);
tscDebug("%p Valid SQL result:%d, %s pObj:%p", pSql, code, taos_errstr(pSql), pObj);
}
taos_free_result(pSql);
......@@ -1047,7 +1069,7 @@ int taos_load_table_info(TAOS *taos, const char *tableNameList) {
tscDoQuery(pSql);
tscDebug("%p load multi table meta result:%d %s pObj:%p", pSql, pRes->code, taos_errstr(taos), pObj);
tscDebug("%p load multi table meta result:%d %s pObj:%p", pSql, pRes->code, taos_errstr(pSql), pObj);
if ((code = pRes->code) != TSDB_CODE_SUCCESS) {
tscFreeSqlObj(pSql);
}
......
src/client/src/tscStream.c
浏览文件 @ d74400f3
......@@ -168,8 +168,8 @@ static void tscProcessStreamQueryCallback(void *param, TAOS_RES *tres, int numOf
STableMetaInfo* pTableMetaInfo = tscGetTableMetaInfoFromCmd(&pStream->pSql->cmd, 0, 0);
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), true);
taosTFree(pTableMetaInfo->vgroupList);
pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList);
tscSetRetryTimer(pStream, pStream->pSql, retryDelay);
return;
}
......@@ -275,7 +275,7 @@ static void tscProcessStreamRetrieveResult(void *param, TAOS_RES *res, int numOf
tscFreeSqlResult(pSql);
taosTFree(pSql->pSubs);
pSql->subState.numOfSub = 0;
taosTFree(pTableMetaInfo->vgroupList);
pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList);
tscSetNextLaunchTimer(pStream, pSql);
}
}
......
src/client/src/tscSubquery.c
浏览文件 @ d74400f3
......@@ -449,7 +449,7 @@ void tscBuildVgroupTableInfo(SSqlObj* pSql, STableMetaInfo* pTableMetaInfo, SArr
SVgroupTableInfo info = {{0}};
for (int32_t m = 0; m < pvg->numOfVgroups; ++m) {
if (tt->vgId == pvg->vgroups[m].vgId) {
info.vgInfo = pvg->vgroups[m];
tscSCMVgroupInfoCopy(&info.vgInfo, &pvg->vgroups[m]);
break;
}
}
......@@ -1142,7 +1142,6 @@ static void tscRetrieveDataRes(void *param, TAOS_RES *tres, int code);
static SSqlObj *tscCreateSTableSubquery(SSqlObj *pSql, SRetrieveSupport *trsupport, SSqlObj *prevSqlObj);
// TODO
int32_t tscCreateJoinSubquery(SSqlObj *pSql, int16_t tableIndex, SJoinSupporter *pSupporter) {
SSqlCmd * pCmd = &pSql->cmd;
SQueryInfo *pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
......@@ -1298,14 +1297,6 @@ void tscHandleMasterJoinQuery(SSqlObj* pSql) {
assert((pQueryInfo->type & TSDB_QUERY_TYPE_SUBQUERY) == 0);
int32_t code = TSDB_CODE_SUCCESS;
// todo add test
// SSubqueryState *pState = calloc(1, sizeof(SSubqueryState));
// if (pState == NULL) {
// code = TSDB_CODE_TSC_OUT_OF_MEMORY;
// goto _error;
// }
pSql->subState.numOfSub = pQueryInfo->numOfTables;
bool hasEmptySub = false;
......@@ -1645,9 +1636,9 @@ static void tscAllDataRetrievedFromDnode(SRetrieveSupport *trsupport, SSqlObj* p
// data in from current vnode is stored in cache and disk
uint32_t numOfRowsFromSubquery = (uint32_t)(trsupport->pExtMemBuffer[idx]->numOfTotalElems + trsupport->localBuffer->num);
tscDebug("%p sub:%p all data retrieved from ep:%s, vgId:%d, numOfRows:%d, orderOfSub:%d", pParentSql, pSql,
pTableMetaInfo->vgroupList->vgroups[0].epAddr[0].fqdn, pTableMetaInfo->vgroupList->vgroups[0].vgId,
numOfRowsFromSubquery, idx);
SVgroupsInfo* vgroupsInfo = pTableMetaInfo->vgroupList;
tscDebug("%p sub:%p all data retrieved from ep:%s, vgId:%d, numOfRows:%d, orderOfSub:%d", pParentSql, pSql,
vgroupsInfo->vgroups[0].epAddr[0].fqdn, vgroupsInfo->vgroups[0].vgId, numOfRowsFromSubquery, idx);
tColModelCompact(pDesc->pColumnModel, trsupport->localBuffer, pDesc->pColumnModel->capacity);
......@@ -2022,7 +2013,7 @@ int32_t tscHandleMultivnodeInsert(SSqlObj *pSql) {
static char* getResultBlockPosition(SSqlCmd* pCmd, SSqlRes* pRes, int32_t columnIndex, int16_t* bytes) {
SQueryInfo* pQueryInfo = tscGetQueryInfoDetail(pCmd, pCmd->clauseIndex);
SFieldSupInfo* pInfo = (SFieldSupInfo*) TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.pSupportInfo, columnIndex);
SInternalField* pInfo = (SInternalField*) TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.internalField, columnIndex);
assert(pInfo->pSqlExpr != NULL);
*bytes = pInfo->pSqlExpr->resBytes;
......@@ -2130,7 +2121,7 @@ static void transferNcharData(SSqlObj *pSql, int32_t columnIndex, TAOS_FIELD *pF
int32_t length = taosUcs4ToMbs(pRes->tsrow[columnIndex], pRes->length[columnIndex], pRes->buffer[columnIndex]);
if ( length >= 0 ) {
pRes->tsrow[columnIndex] = pRes->buffer[columnIndex];
pRes->tsrow[columnIndex] = (unsigned char*)pRes->buffer[columnIndex];
pRes->length[columnIndex] = length;
} else {
tscError("%p charset:%s to %s. val:%s convert failed.", pSql, DEFAULT_UNICODE_ENCODEC, tsCharset, (char*)pRes->tsrow[columnIndex]);
......@@ -2158,7 +2149,7 @@ static char *getArithemicInputSrc(void *param, const char *name, int32_t colId)
return pSupport->data[index] + pSupport->offset * pExpr->resBytes;
}
void **doSetResultRowData(SSqlObj *pSql, bool finalResult) {
TAOS_ROW doSetResultRowData(SSqlObj *pSql, bool finalResult) {
SSqlCmd *pCmd = &pSql->cmd;
SSqlRes *pRes = &pSql->res;
......@@ -2172,7 +2163,7 @@ void **doSetResultRowData(SSqlObj *pSql, bool finalResult) {
size_t size = tscNumOfFields(pQueryInfo);
for (int i = 0; i < size; ++i) {
SFieldSupInfo* pSup = TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.pSupportInfo, i);
SInternalField* pSup = TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.internalField, i);
if (pSup->pSqlExpr != NULL) {
tscGetResultColumnChr(pRes, &pQueryInfo->fieldsInfo, i);
}
......@@ -2182,7 +2173,7 @@ void **doSetResultRowData(SSqlObj *pSql, bool finalResult) {
continue;
}
TAOS_FIELD *pField = TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.pFields, i);
TAOS_FIELD *pField = TARRAY_GET_ELEM(pQueryInfo->fieldsInfo.internalField, i);
if (pRes->tsrow[i] != NULL && pField->type == TSDB_DATA_TYPE_NCHAR) {
transferNcharData(pSql, i, pField);
}
......@@ -2211,7 +2202,7 @@ void **doSetResultRowData(SSqlObj *pSql, bool finalResult) {
tExprTreeCalcTraverse(pRes->pArithSup->pArithExpr->pExpr, 1, pRes->buffer[i], pRes->pArithSup,
TSDB_ORDER_ASC, getArithemicInputSrc);
pRes->tsrow[i] = pRes->buffer[i];
pRes->tsrow[i] = (unsigned char*)pRes->buffer[i];
}
}
......
src/client/src/tscSystem.c
浏览文件 @ d74400f3
......@@ -77,6 +77,7 @@ int32_t tscInitRpc(const char *user, const char *secretEncrypt, void **pDnodeCon
return 0;
}
void taos_init_imp(void) {
char temp[128];
......@@ -124,8 +125,9 @@ void taos_init_imp(void) {
double factor = (tscEmbedded == 0)? 2.0:4.0;
tscNumOfThreads = (int)(tsNumOfCores * tsNumOfThreadsPerCore / factor);
if (tscNumOfThreads < 2) tscNumOfThreads = 2;
if (tscNumOfThreads < 2) {
tscNumOfThreads = 2;
}
tscQhandle = taosInitScheduler(queueSize, tscNumOfThreads, "tsc");
if (NULL == tscQhandle) {
......@@ -140,7 +142,7 @@ void taos_init_imp(void) {
int64_t refreshTime = 10; // 10 seconds by default
if (tscMetaCache == NULL) {
tscMetaCache = taosCacheInit(TSDB_DATA_TYPE_BINARY, refreshTime, false, NULL, "tableMeta");
tscMetaCache = taosCacheInit(TSDB_DATA_TYPE_BINARY, refreshTime, false, tscFreeTableMetaHelper, "tableMeta");
tscObjCache = taosCacheInit(TSDB_CACHE_PTR_KEY, refreshTime / 2, false, tscFreeRegisteredSqlObj, "sqlObj");
}
......
src/client/src/tscUtil.c
浏览文件 @ d74400f3
......@@ -114,9 +114,9 @@ bool tscIsTwoStageSTableQuery(SQueryInfo* pQueryInfo, int32_t tableIndex) {
}
// for select query super table, the super table vgroup list can not be null in any cases.
if (pQueryInfo->command == TSDB_SQL_SELECT && UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
assert(pTableMetaInfo->vgroupList != NULL);
}
// if (pQueryInfo->command == TSDB_SQL_SELECT && UTIL_TABLE_IS_SUPER_TABLE(pTableMetaInfo)) {
// assert(pTableMetaInfo->vgroupList != NULL);
// }
if ((pQueryInfo->type & TSDB_QUERY_TYPE_FREE_RESOURCE) == TSDB_QUERY_TYPE_FREE_RESOURCE) {
return false;
......@@ -408,6 +408,24 @@ void tscFreeRegisteredSqlObj(void *pSql) {
}
}
void tscFreeTableMetaHelper(void *pTableMeta) {
STableMeta* p = (STableMeta*) pTableMeta;
int32_t numOfEps = p->vgroupInfo.numOfEps;
assert(numOfEps >= 0 && numOfEps <= TSDB_MAX_REPLICA);
for(int32_t i = 0; i < numOfEps; ++i) {
taosTFree(p->vgroupInfo.epAddr[i].fqdn);
}
int32_t numOfEps1 = p->corVgroupInfo.numOfEps;
assert(numOfEps1 >= 0 && numOfEps1 <= TSDB_MAX_REPLICA);
for(int32_t i = 0; i < numOfEps1; ++i) {
taosTFree(p->corVgroupInfo.epAddr[i].fqdn);
}
}
void tscFreeSqlObj(SSqlObj* pSql) {
if (pSql == NULL || pSql->signature != pSql) {
return;
......@@ -686,16 +704,14 @@ static int32_t getRowExpandSize(STableMeta* pTableMeta) {
int32_t tscMergeTableDataBlocks(SSqlObj* pSql, SArray* pTableDataBlockList) {
SSqlCmd* pCmd = &pSql->cmd;
// the maximum expanded size in byte when a row-wise data is converted to SDataRow format
STableDataBlocks* pOneTableBlock = taosArrayGetP(pTableDataBlockList, 0);
int32_t expandSize = getRowExpandSize(pOneTableBlock->pTableMeta);
void* pVnodeDataBlockHashList = taosHashInit(128, taosGetDefaultHashFunction(TSDB_DATA_TYPE_BIGINT), true, false);
SArray* pVnodeDataBlockList = taosArrayInit(8, POINTER_BYTES);
size_t total = taosArrayGetSize(pTableDataBlockList);
for (int32_t i = 0; i < total; ++i) {
pOneTableBlock = taosArrayGetP(pTableDataBlockList, i);
// the maximum expanded size in byte when a row-wise data is converted to SDataRow format
STableDataBlocks* pOneTableBlock = taosArrayGetP(pTableDataBlockList, i);
int32_t expandSize = getRowExpandSize(pOneTableBlock->pTableMeta);
STableDataBlocks* dataBuf = NULL;
int32_t ret =
......@@ -829,35 +845,30 @@ TAOS_FIELD tscCreateField(int8_t type, const char* name, int16_t bytes) {
return f;
}
SFieldSupInfo* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField) {
SInternalField* tscFieldInfoAppend(SFieldInfo* pFieldInfo, TAOS_FIELD* pField) {
assert(pFieldInfo != NULL);
taosArrayPush(pFieldInfo->pFields, pField);
pFieldInfo->numOfOutput++;
struct SFieldSupInfo info = {
struct SInternalField info = {
.pSqlExpr = NULL,
.pArithExprInfo = NULL,
.visible = true,
};
return taosArrayPush(pFieldInfo->pSupportInfo, &info);
}
SFieldSupInfo* tscFieldInfoGetSupp(SFieldInfo* pFieldInfo, int32_t index) {
return TARRAY_GET_ELEM(pFieldInfo->pSupportInfo, index);
info.field = *pField;
return taosArrayPush(pFieldInfo->internalField, &info);
}
SFieldSupInfo* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_FIELD* field) {
taosArrayInsert(pFieldInfo->pFields, index, field);
SInternalField* tscFieldInfoInsert(SFieldInfo* pFieldInfo, int32_t index, TAOS_FIELD* field) {
pFieldInfo->numOfOutput++;
struct SFieldSupInfo info = {
struct SInternalField info = {
.pSqlExpr = NULL,
.pArithExprInfo = NULL,
.visible = true,
};
return taosArrayInsert(pFieldInfo->pSupportInfo, index, &info);
info.field = *field;
return taosArrayInsert(pFieldInfo->internalField, index, &info);
}
void tscFieldInfoUpdateOffset(SQueryInfo* pQueryInfo) {
......@@ -891,29 +902,18 @@ void tscFieldInfoUpdateOffsetForInterResult(SQueryInfo* pQueryInfo) {
}
}
void tscFieldInfoCopy(SFieldInfo* dst, const SFieldInfo* src) {
dst->numOfOutput = src->numOfOutput;
if (dst->pFields == NULL) {
dst->pFields = taosArrayClone(src->pFields);
} else {
taosArrayCopy(dst->pFields, src->pFields);
}
if (dst->pSupportInfo == NULL) {
dst->pSupportInfo = taosArrayClone(src->pSupportInfo);
} else {
taosArrayCopy(dst->pSupportInfo, src->pSupportInfo);
}
SInternalField* tscFieldInfoGetInternalField(SFieldInfo* pFieldInfo, int32_t index) {
assert(index < pFieldInfo->numOfOutput);
return TARRAY_GET_ELEM(pFieldInfo->internalField, index);
}
TAOS_FIELD* tscFieldInfoGetField(SFieldInfo* pFieldInfo, int32_t index) {
assert(index < pFieldInfo->numOfOutput);
return TARRAY_GET_ELEM(pFieldInfo->pFields, index);
return &((SInternalField*)TARRAY_GET_ELEM(pFieldInfo->internalField, index))->field;
}
int16_t tscFieldInfoGetOffset(SQueryInfo* pQueryInfo, int32_t index) {
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pQueryInfo->fieldsInfo, index);
SInternalField* pInfo = tscFieldInfoGetInternalField(&pQueryInfo->fieldsInfo, index);
assert(pInfo != NULL && pInfo->pSqlExpr != NULL);
return pInfo->pSqlExpr->offset;
......@@ -960,10 +960,8 @@ void tscFieldInfoClear(SFieldInfo* pFieldInfo) {
return;
}
taosArrayDestroy(pFieldInfo->pFields);
for(int32_t i = 0; i < pFieldInfo->numOfOutput; ++i) {
SFieldSupInfo* pInfo = taosArrayGet(pFieldInfo->pSupportInfo, i);
SInternalField* pInfo = taosArrayGet(pFieldInfo->internalField, i);
if (pInfo->pArithExprInfo != NULL) {
tExprTreeDestroy(&pInfo->pArithExprInfo->pExpr, NULL);
......@@ -971,7 +969,9 @@ void tscFieldInfoClear(SFieldInfo* pFieldInfo) {
}
}
taosArrayDestroy(pFieldInfo->pSupportInfo);
taosArrayDestroy(pFieldInfo->internalField);
taosTFree(pFieldInfo->final);
memset(pFieldInfo, 0, sizeof(SFieldInfo));
}
......@@ -1615,11 +1615,8 @@ STableMetaInfo* tscGetTableMetaInfoByUid(SQueryInfo* pQueryInfo, uint64_t uid, i
}
void tscInitQueryInfo(SQueryInfo* pQueryInfo) {
assert(pQueryInfo->fieldsInfo.pFields == NULL);
pQueryInfo->fieldsInfo.pFields = taosArrayInit(4, sizeof(TAOS_FIELD));
assert(pQueryInfo->fieldsInfo.pSupportInfo == NULL);
pQueryInfo->fieldsInfo.pSupportInfo = taosArrayInit(4, sizeof(SFieldSupInfo));
assert(pQueryInfo->fieldsInfo.internalField == NULL);
pQueryInfo->fieldsInfo.internalField = taosArrayInit(4, sizeof(SInternalField));
assert(pQueryInfo->exprList == NULL);
pQueryInfo->exprList = taosArrayInit(4, POINTER_BYTES);
......@@ -1682,8 +1679,14 @@ void tscClearSubqueryInfo(SSqlCmd* pCmd) {
void tscFreeVgroupTableInfo(SArray* pVgroupTables) {
if (pVgroupTables != NULL) {
for (size_t i = 0; i < taosArrayGetSize(pVgroupTables); i++) {
size_t num = taosArrayGetSize(pVgroupTables);
for (size_t i = 0; i < num; i++) {
SVgroupTableInfo* pInfo = taosArrayGet(pVgroupTables, i);
for(int32_t j = 0; j < pInfo->vgInfo.numOfEps; ++j) {
taosTFree(pInfo->vgInfo.epAddr[j].fqdn);
}
taosArrayDestroy(pInfo->itemList);
}
taosArrayDestroy(pVgroupTables);
......@@ -1695,6 +1698,7 @@ void clearAllTableMetaInfo(SQueryInfo* pQueryInfo, const char* address, bool rem
for(int32_t i = 0; i < pQueryInfo->numOfTables; ++i) {
STableMetaInfo* pTableMetaInfo = tscGetMetaInfo(pQueryInfo, i);
tscFreeVgroupTableInfo(pTableMetaInfo->pVgroupTables);
tscClearTableMetaInfo(pTableMetaInfo, removeFromCache);
free(pTableMetaInfo);
......@@ -1727,13 +1731,7 @@ STableMetaInfo* tscAddTableMetaInfo(SQueryInfo* pQueryInfo, const char* name, ST
pTableMetaInfo->pTableMeta = pTableMeta;
if (vgroupList != NULL) {
size_t size = sizeof(SVgroupsInfo) + sizeof(SCMVgroupInfo) * vgroupList->numOfVgroups;
pTableMetaInfo->vgroupList = malloc(size);
if (pTableMetaInfo->vgroupList == NULL) {
return NULL;
}
memcpy(pTableMetaInfo->vgroupList, vgroupList, size);
pTableMetaInfo->vgroupList = tscVgroupInfoClone(vgroupList);
}
pTableMetaInfo->tagColList = taosArrayInit(4, POINTER_BYTES);
......@@ -1762,8 +1760,7 @@ void tscClearTableMetaInfo(STableMetaInfo* pTableMetaInfo, bool removeFromCache)
taosCacheRelease(tscMetaCache, (void**)&(pTableMetaInfo->pTableMeta), removeFromCache);
}
taosTFree(pTableMetaInfo->vgroupList);
pTableMetaInfo->vgroupList = tscVgroupInfoClear(pTableMetaInfo->vgroupList);
tscColumnListDestroy(pTableMetaInfo->tagColList);
pTableMetaInfo->tagColList = NULL;
}
......@@ -1831,54 +1828,23 @@ SSqlObj* createSimpleSubObj(SSqlObj* pSql, void (*fp)(), void* param, int32_t cm
return pNew;
}
// current sql function is not direct output result, so create a dummy output field
static void doSetNewFieldInfo(SQueryInfo* pNewQueryInfo, SSqlExpr* pExpr) {
TAOS_FIELD f = {.type = (uint8_t)pExpr->resType, .bytes = pExpr->resBytes};
tstrncpy(f.name, pExpr->aliasName, sizeof(f.name));
SFieldSupInfo* pInfo1 = tscFieldInfoAppend(&pNewQueryInfo->fieldsInfo, &f);
pInfo1->pSqlExpr = pExpr;
pInfo1->visible = false;
}
static void doSetSqlExprAndResultFieldInfo(SQueryInfo* pQueryInfo, SQueryInfo* pNewQueryInfo, int64_t uid) {
int32_t numOfOutput = (int32_t)tscSqlExprNumOfExprs(pNewQueryInfo);
if (numOfOutput == 0) {
return;
}
size_t numOfExprs = tscSqlExprNumOfExprs(pQueryInfo);
SFieldInfo* pFieldInfo = &pQueryInfo->fieldsInfo;
// set the field info in pNewQueryInfo object
// set the field info in pNewQueryInfo object according to sqlExpr information
size_t numOfExprs = tscSqlExprNumOfExprs(pNewQueryInfo);
for (int32_t i = 0; i < numOfExprs; ++i) {
SSqlExpr* pExpr = tscSqlExprGet(pQueryInfo, i);
SSqlExpr* pExpr = tscSqlExprGet(pNewQueryInfo, i);
if (pExpr->uid == uid) {
if (i < pFieldInfo->numOfOutput) {
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(pFieldInfo, i);
if (pInfo->pSqlExpr != NULL) {
TAOS_FIELD* p = tscFieldInfoGetField(pFieldInfo, i);
assert(strcmp(p->name, pExpr->aliasName) == 0);
SFieldSupInfo* pInfo1 = tscFieldInfoAppend(&pNewQueryInfo->fieldsInfo, p);
*pInfo1 = *pInfo;
} else {
assert(pInfo->pArithExprInfo != NULL);
doSetNewFieldInfo(pNewQueryInfo, pExpr);
}
} else { // it is a arithmetic column, does not have actual field for sqlExpr, so build it
doSetNewFieldInfo(pNewQueryInfo, pExpr);
}
}
TAOS_FIELD f = tscCreateField((int8_t) pExpr->resType, pExpr->aliasName, pExpr->resBytes);
SInternalField* pInfo1 = tscFieldInfoAppend(&pNewQueryInfo->fieldsInfo, &f);
pInfo1->pSqlExpr = pExpr;
}
// make sure the the sqlExpr for each fields is correct
numOfExprs = tscSqlExprNumOfExprs(pNewQueryInfo);
// update the pSqlExpr pointer in SFieldSupInfo according the field name
// update the pSqlExpr pointer in SInternalField according the field name
// make sure the pSqlExpr point to the correct SqlExpr in pNewQueryInfo, not SqlExpr in pQueryInfo
for (int32_t f = 0; f < pNewQueryInfo->fieldsInfo.numOfOutput; ++f) {
TAOS_FIELD* field = tscFieldInfoGetField(&pNewQueryInfo->fieldsInfo, f);
......@@ -1888,7 +1854,7 @@ static void doSetSqlExprAndResultFieldInfo(SQueryInfo* pQueryInfo, SQueryInfo* p
SSqlExpr* pExpr1 = tscSqlExprGet(pNewQueryInfo, k1);
if (strcmp(field->name, pExpr1->aliasName) == 0) { // establish link according to the result field name
SFieldSupInfo* pInfo = tscFieldInfoGetSupp(&pNewQueryInfo->fieldsInfo, f);
SInternalField* pInfo = tscFieldInfoGetInternalField(&pNewQueryInfo->fieldsInfo, f);
pInfo->pSqlExpr = pExpr1;
matched = true;
......@@ -2403,3 +2369,58 @@ void tscClearSqlOwner(SSqlObj* pSql) {
assert(taosCheckPthreadValid(pSql->owner));
atomic_store_64(&pSql->owner, 0);
}
SVgroupsInfo* tscVgroupInfoClone(SVgroupsInfo *vgroupList) {
if (vgroupList == NULL) {
return NULL;
}
size_t size = sizeof(SVgroupsInfo) + sizeof(SCMVgroupInfo) * vgroupList->numOfVgroups;
SVgroupsInfo* pNew = calloc(1, size);
if (pNew == NULL) {
return NULL;
}
pNew->numOfVgroups = vgroupList->numOfVgroups;
for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) {
SCMVgroupInfo* pNewVInfo = &pNew->vgroups[i];
SCMVgroupInfo* pvInfo = &vgroupList->vgroups[i];
pNewVInfo->vgId = pvInfo->vgId;
pNewVInfo->numOfEps = pvInfo->numOfEps;
for(int32_t j = 0; j < pvInfo->numOfEps; ++j) {
pNewVInfo->epAddr[j].fqdn = strdup(pvInfo->epAddr[j].fqdn);
pNewVInfo->epAddr[j].port = pvInfo->epAddr[j].port;
}
}
return pNew;
}
void* tscVgroupInfoClear(SVgroupsInfo *vgroupList) {
if (vgroupList == NULL) {
return NULL;
}
for(int32_t i = 0; i < vgroupList->numOfVgroups; ++i) {
SCMVgroupInfo* pVgroupInfo = &vgroupList->vgroups[i];
for(int32_t j = 0; j < pVgroupInfo->numOfEps; ++j) {
taosTFree(pVgroupInfo->epAddr[j].fqdn);
}
}
taosTFree(vgroupList);
return NULL;
}
void tscSCMVgroupInfoCopy(SCMVgroupInfo* dst, const SCMVgroupInfo* src) {
dst->vgId = src->vgId;
dst->numOfEps = src->numOfEps;
for(int32_t i = 0; i < dst->numOfEps; ++i) {
dst->epAddr[i].port = src->epAddr[i].port;
dst->epAddr[i].fqdn = strdup(src->epAddr[i].fqdn);
}
}
src/connector/jdbc/CMakeLists.txt
浏览文件 @ d74400f3
......@@ -8,7 +8,7 @@ IF (TD_MVN_INSTALLED)
ADD_CUSTOM_COMMAND(OUTPUT ${JDBC_CMD_NAME}
POST_BUILD
COMMAND mvn -Dmaven.test.skip=true install -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.0-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND ${CMAKE_COMMAND} -E copy ${CMAKE_CURRENT_SOURCE_DIR}/target/taos-jdbcdriver-2.0.8-dist.jar ${LIBRARY_OUTPUT_PATH}
COMMAND mvn -Dmaven.test.skip=true clean -f ${CMAKE_CURRENT_SOURCE_DIR}/pom.xml
COMMENT "build jdbc driver")
ADD_CUSTOM_TARGET(${JDBC_TARGET_NAME} ALL WORKING_DIRECTORY ${EXECUTABLE_OUTPUT_PATH} DEPENDS ${JDBC_CMD_NAME})
......
src/connector/jdbc/pom.xml
浏览文件 @ d74400f3
......@@ -3,7 +3,7 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.0</version>
<version>2.0.8</version>
<packaging>jar</packaging>
<name>JDBCDriver</name>
<url>https://github.com/taosdata/TDengine/tree/master/src/connector/jdbc</url>
......
src/connector/jdbc/src/main/java/com/taosdata/jdbc/TSDBDatabaseMetaData.java
浏览文件 @ d74400f3
......@@ -587,7 +587,6 @@ public class TSDBDatabaseMetaData implements java.sql.DatabaseMetaData {
public ResultSet getColumns(String catalog, String schemaPattern, String tableNamePattern, String columnNamePattern)
throws SQLException {
/** add by zyyang **********/
Statement stmt = null;
if (null != conn && !conn.isClosed()) {
......
src/connector/odbc/CMakeLists.txt 0 → 100644
浏览文件 @ d74400f3
CMAKE_MINIMUM_REQUIRED(VERSION 2.8)
PROJECT(TDengine)
IF (TD_LINUX_64)
find_program(HAVE_ODBCINST NAMES odbcinst)
IF (HAVE_ODBCINST)
include(CheckSymbolExists)
# shall we revert CMAKE_REQUIRED_LIBRARIES and how?
set(CMAKE_REQUIRED_LIBRARIES odbc)
check_symbol_exists(SQLExecute "sql.h" HAVE_ODBC_DEV)
if(NOT (HAVE_ODBC_DEV))
unset(HAVE_ODBC_DEV CACHE)
message(WARNING "unixodbc-dev is not installed yet, you may install it under ubuntu by typing: sudo apt install unixodbc-dev")
else ()
message(STATUS "unixodbc/unixodbc-dev are installed, and odbc connector will be built")
AUX_SOURCE_DIRECTORY(src SRC)
# generate dynamic library (*.so)
ADD_LIBRARY(todbc SHARED ${SRC})
SET_TARGET_PROPERTIES(todbc PROPERTIES CLEAN_DIRECT_OUTPUT 1)
SET_TARGET_PROPERTIES(todbc PROPERTIES VERSION ${TD_VER_NUMBER} SOVERSION 1)
TARGET_LINK_LIBRARIES(todbc taos)
install(CODE "execute_process(COMMAND ${CMAKE_CURRENT_SOURCE_DIR}/src/install.sh ${CMAKE_BINARY_DIR})")
ADD_SUBDIRECTORY(tests)
endif()
ELSE ()
message(WARNING "unixodbc is not installed yet, you may install it under ubuntu by typing: sudo apt install unixodbc")
ENDIF ()
ENDIF ()
src/connector/odbc/src/install.sh 0 → 100755
浏览文件 @ d74400f3
#!/bin/bash
set -u
BLD_DIR="$1"
rm -f "${BLD_DIR}/template.ini"
rm -f "${BLD_DIR}/template.dsn"
cat > "${BLD_DIR}/template.ini" <<EOF
[TAOS]
Description = taos odbc driver
Driver = ${BLD_DIR}/build/lib/libtodbc.so
EOF
cat > "${BLD_DIR}/template.dsn" <<EOF
[TAOS_DSN]
Description=Connection to TAOS
Driver=TAOS
EOF
sudo odbcinst -i -d -f "${BLD_DIR}/template.ini" &&
odbcinst -i -s -f "${BLD_DIR}/template.dsn" &&
echo "odbc install done"
src/connector/odbc/src/todbc.c 0 → 100644
浏览文件 @ d74400f3
此差异已折叠。
src/connector/odbc/src/todbc_util.c 0 → 100644
浏览文件 @ d74400f3
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#include "todbc_util.h"
#include "iconv.h"
#include <sql.h>
#include <sqltypes.h>
#include <sqlext.h>
#include <stdlib.h>
#include <string.h>
const char* sql_sql_type(int type) {
switch (type) {
case SQL_BIT: return "SQL_BIT";
case SQL_TINYINT: return "SQL_TINYINT";
case SQL_SMALLINT: return "SQL_SMALLINT";
case SQL_INTEGER: return "SQL_INTEGER";
case SQL_BIGINT: return "SQL_BIGINT";
case SQL_FLOAT: return "SQL_FLOAT";
case SQL_DOUBLE: return "SQL_DOUBLE";
case SQL_DECIMAL: return "SQL_DECIMAL";
case SQL_NUMERIC: return "SQL_NUMERIC";
case SQL_REAL: return "SQL_REAL";
case SQL_CHAR: return "SQL_CHAR";
case SQL_VARCHAR: return "SQL_VARCHAR";
case SQL_LONGVARCHAR: return "SQL_LONGVARCHAR";
case SQL_WCHAR: return "SQL_WCHAR";
case SQL_WVARCHAR: return "SQL_WVARCHAR";
case SQL_WLONGVARCHAR: return "SQL_WLONGVARCHAR";
case SQL_BINARY: return "SQL_BINARY";
case SQL_VARBINARY: return "SQL_VARBINARY";
case SQL_LONGVARBINARY: return "SQL_LONGVARBINARY";
case SQL_DATE: return "SQL_DATE";
case SQL_TIME: return "SQL_TIME";
case SQL_TIMESTAMP: return "SQL_TIMESTAMP";
case SQL_TYPE_DATE: return "SQL_TYPE_DATE";
case SQL_TYPE_TIME: return "SQL_TYPE_TIME";
case SQL_TYPE_TIMESTAMP: return "SQL_TYPE_TIMESTAMP";
case SQL_INTERVAL_MONTH: return "SQL_INTERVAL_MONTH";
case SQL_INTERVAL_YEAR: return "SQL_INTERVAL_YEAR";
case SQL_INTERVAL_YEAR_TO_MONTH: return "SQL_INTERVAL_YEAR_TO_MONTH";
case SQL_INTERVAL_DAY: return "SQL_INTERVAL_DAY";
case SQL_INTERVAL_HOUR: return "SQL_INTERVAL_HOUR";
case SQL_INTERVAL_MINUTE: return "SQL_INTERVAL_MINUTE";
case SQL_INTERVAL_SECOND: return "SQL_INTERVAL_SECOND";
case SQL_INTERVAL_DAY_TO_HOUR: return "SQL_INTERVAL_DAY_TO_HOUR";
case SQL_INTERVAL_DAY_TO_MINUTE: return "SQL_INTERVAL_DAY_TO_MINUTE";
case SQL_INTERVAL_DAY_TO_SECOND: return "SQL_INTERVAL_DAY_TO_SECOND";
case SQL_INTERVAL_HOUR_TO_MINUTE: return "SQL_INTERVAL_HOUR_TO_MINUTE";
case SQL_INTERVAL_HOUR_TO_SECOND: return "SQL_INTERVAL_HOUR_TO_SECOND";
case SQL_INTERVAL_MINUTE_TO_SECOND: return "SQL_INTERVAL_MINUTE_TO_SECOND";
case SQL_GUID: return "SQL_GUID";
default: return "UNKNOWN";
}
}
const char* sql_c_type(int type) {
switch (type) {
case SQL_C_CHAR: return "SQL_C_CHAR";
case SQL_C_WCHAR: return "SQL_C_WCHAR";
case SQL_C_SHORT: return "SQL_C_SHORT";
case SQL_C_SSHORT: return "SQL_C_SSHORT";
case SQL_C_USHORT: return "SQL_C_USHORT";
case SQL_C_LONG: return "SQL_C_LONG";
case SQL_C_SLONG: return "SQL_C_SLONG";
case SQL_C_ULONG: return "SQL_C_ULONG";
case SQL_C_FLOAT: return "SQL_C_FLOAT";
case SQL_C_DOUBLE: return "SQL_C_DOUBLE";
case SQL_C_BIT: return "SQL_C_BIT";
case SQL_C_TINYINT: return "SQL_C_TINYINT";
case SQL_C_STINYINT: return "SQL_C_STINYINT";
case SQL_C_UTINYINT: return "SQL_C_UTINYINT";
case SQL_C_SBIGINT: return "SQL_C_SBIGINT";
case SQL_C_UBIGINT: return "SQL_C_UBIGINT";
case SQL_C_BINARY: return "SQL_C_BINARY";
case SQL_C_DATE: return "SQL_C_DATE";
case SQL_C_TIME: return "SQL_C_TIME";
case SQL_C_TIMESTAMP: return "SQL_C_TIMESTAMP";
case SQL_C_TYPE_DATE: return "SQL_C_TYPE_DATE";
case SQL_C_TYPE_TIME: return "SQL_C_TYPE_TIME";
case SQL_C_TYPE_TIMESTAMP: return "SQL_C_TYPE_TIMESTAMP";
case SQL_C_NUMERIC: return "SQL_C_NUMERIC";
case SQL_C_GUID: return "SQL_C_GUID";
default: return "UNKNOWN";
}
}
int is_valid_sql_c_type(int type) {
const char *ctype = sql_c_type(type);
if (strcmp(ctype, "UNKNOWN")==0) return 0;
return 1;
}
int is_valid_sql_sql_type(int type) {
const char *sqltype = sql_sql_type(type);
if (strcmp(sqltype, "UNKNOWN")==0) return 0;
return 1;
}
int string_conv(const char *fromcode, const char *tocode,
const unsigned char *src, size_t sbytes,
unsigned char *dst, size_t dbytes,
size_t *consumed, size_t *generated)
{
if (consumed) *consumed = 0;
if (generated) *generated = 0;
if (dbytes <= 0) return -1;
dst[0] = '\0';
iconv_t conv = iconv_open(tocode, fromcode);
if (!conv) return -1;
int r = 0;
do {
char *s = (char*)src;
char *d = (char*)dst;
size_t sl = sbytes;
size_t dl = dbytes;
r = iconv(conv, &s, &sl, &d, &dl);
*d = '\0';
if (consumed) *consumed = sbytes - sl;
if (generated) *generated = dbytes - dl;
} while (0);
iconv_close(conv);
return r;
}
int utf8_chars(const char *src)
{
const char *fromcode = "UTF-8";
const char *tocode = "UCS-2LE";
iconv_t conv = iconv_open(tocode, fromcode);
if (!conv) return -1;
size_t slen = strlen(src);
char buf[4096];
size_t dlen = sizeof(buf);
char *ps = (char*)src;
char *pd = buf;
iconv(conv, &ps, &slen, &pd, &dlen);
DASSERT(slen==0);
size_t chars = (sizeof(buf) - dlen) / 2;
iconv_close(conv);
return chars;
}
unsigned char* utf8_to_ucs4le(const char *utf8, size_t *chars)
{
const char *tocode = "UCS-4LE";
const char *fromcode = "UTF-8";
iconv_t conv = iconv_open(tocode, fromcode);
if (!conv) return NULL;
unsigned char *ucs4le = NULL;
do {
size_t slen = strlen(utf8);
size_t dlen = slen * 4;
ucs4le = (unsigned char*)malloc(dlen+1);
if (!ucs4le) break;
char *src = (char*)utf8;
char *dst = (char*)ucs4le;
size_t s = slen;
size_t d = dlen;
iconv(conv, &src, &s, &dst, &d);
dst[0] = '\0';
if (chars) *chars = (dlen - d) / 4;
} while (0);
iconv_close(conv);
return ucs4le;
}
char* ucs4le_to_utf8(const unsigned char *ucs4le, size_t slen, size_t *chars)
{
const char *fromcode = "UCS-4LE";
const char *tocode = "UTF-8";
iconv_t conv = iconv_open(tocode, fromcode);
if (!conv) return NULL;
char *utf8 = NULL;
do {
size_t dlen = slen;
utf8 = (char*)malloc(dlen+1);
if (!utf8) break;
char *dst = utf8;
char *src = (char*)ucs4le;
size_t s = slen;
size_t d = dlen;
iconv(conv, &src, &s, &dst, &d);
dst[0] = '\0';
if (chars) *chars = (slen - s) / 4;
} while (0);
iconv_close(conv);
return utf8;
}
SQLCHAR* wchars_to_chars(const SQLWCHAR *wchars, size_t chs, size_t *bytes)
{
size_t dlen = chs * 4;
SQLCHAR *dst = (SQLCHAR*)malloc(dlen + 1);
if (!dst) return NULL;
string_conv("UCS-2LE", "UTF-8", (const unsigned char*)wchars, chs * sizeof(*wchars), dst, dlen + 1, NULL, bytes);
return dst;
}
src/connector/odbc/src/todbc_util.h 0 → 100644
浏览文件 @ d74400f3
/*
* Copyright (c) 2019 TAOS Data, Inc. <jhtao@taosdata.com>
*
* This program is free software: you can use, redistribute, and/or modify
* it under the terms of the GNU Affero General Public License, version 3
* or later ("AGPL"), as published by the Free Software Foundation.
*
* This program is distributed in the hope that it will be useful, but WITHOUT
* ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
* FITNESS FOR A PARTICULAR PURPOSE.
*
* You should have received a copy of the GNU Affero General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef _TODBC_UTIL_H_
#define _TODBC_UTIL_H_
#include <libgen.h>
#include <stddef.h>
#include <stdio.h>
#include <stdlib.h>
#include <sql.h>
#define D(fmt, ...) \
fprintf(stderr, \
"%s[%d]:%s() " fmt "\n", \
basename((char*)__FILE__), __LINE__, __func__, \
##__VA_ARGS__)
#define DASSERT(statement) \
do { \
if (statement) break; \
D("Assertion failure: %s", #statement); \
abort(); \
} while (0)
#define DASSERTX(statement, fmt, ...) \
do { \
if (statement) break; \
D("Assertion failure: %s, " fmt "", #statement, ##__VA_ARGS__); \
abort(); \
} while (0)
const char* sql_sql_type(int type);
const char* sql_c_type(int type);
int is_valid_sql_c_type(int type);
int is_valid_sql_sql_type(int type);
int string_conv(const char *fromcode, const char *tocode,
const unsigned char *src, size_t sbytes,
unsigned char *dst, size_t dbytes,
size_t *consumed, size_t *generated);
int utf8_chars(const char *src);
unsigned char* utf8_to_ucs4le(const char *utf8, size_t *chars);
char* ucs4le_to_utf8(const unsigned char *ucs4le, size_t slen, size_t *chars);
SQLCHAR* wchars_to_chars(const SQLWCHAR *wchars, size_t chs, size_t *bytes);
#endif // _TODBC_UTIL_H_
src/connector/odbc/tests/CMakeLists.txt 0 → 100644
浏览文件 @ d74400f3
PROJECT(TDengine)
IF (TD_LINUX)
AUX_SOURCE_DIRECTORY(. SRC)
ADD_EXECUTABLE(tcodbc main.c)
TARGET_LINK_LIBRARIES(tcodbc odbc)
ENDIF ()
src/connector/odbc/tests/main.c 0 → 100644
浏览文件 @ d74400f3
#include <sql.h>
#include <sqlext.h>
#include <stdio.h>
#include <string.h>
#include "os.h"
// static const char *dsn = "TAOS_DSN";
// static const char *uid = "root";
// static const char *pwd = "taosdata";
typedef struct data_s data_t;
struct data_s {
int64_t ts;
int8_t b;
int8_t v1;
int16_t v2;
int32_t v4;
int64_t v8;
float f4;
double f8;
char bin[40+1];
char blob[40+1]; // why 80? ref: tests/examples/c/apitest.c
};
static const char *pre_stmts[] = {
"create database db",
"use db",
"create table t (ts timestamp, b bool, v1 tinyint, v2 smallint, v4 int, v8 bigint, f4 float, f8 double, bin binary(40), blob nchar(10))"
};
static const char *pro_stmts[] = {
// "insert into t values ('2019-07-15 00:00:00', 1)",
// "insert into t values ('2019-07-15 01:00:00', 2)",
"select * from t"
// "drop database db"
};
#define CHK_RESULT(r, ht, h) \
do { \
if (r==0) break; \
SQLCHAR ss[10]; \
SQLINTEGER ne = 0; \
SQLCHAR es[4096]; \
SQLSMALLINT n = 0; \
ss[0] = '\0'; \
es[0] = '\0'; \
SQLRETURN ret = SQLGetDiagRec(ht, h, 1, ss, &ne, es, sizeof(es), &n); \
if (ret) break; \
fprintf(stderr, "%s%s\n", ss, es); \
} while (0)
static int do_statement(SQLHSTMT stmt, const char *statement) {
SQLRETURN r = 0;
do {
fprintf(stderr, "prepare [%s]\n", statement);
r = SQLPrepare(stmt, (SQLCHAR*)statement, strlen(statement));
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "execute [%s]\n", statement);
r = SQLExecute(stmt);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "done\n");
} while (0);
fprintf(stderr, "r: [%x][%d]\n", r, r);
return r;
}
static int do_insert(SQLHSTMT stmt, data_t data) {
SQLRETURN r = 0;
SQLLEN lbin;
SQLLEN lblob;
const char *statement = "insert into t values (?, ?, ?, ?, ?, ?, ?, ?, ?,?)";
int ignored = 0;
do {
fprintf(stderr, "prepare [%s]\n", statement);
r = SQLPrepare(stmt, (SQLCHAR*)statement, strlen(statement));
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 1 [%s]\n", statement);
r = SQLBindParameter(stmt, 1, SQL_PARAM_INPUT, SQL_C_SBIGINT, SQL_TIMESTAMP, ignored, ignored, &data.ts, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 2 [%s]\n", statement);
r = SQLBindParameter(stmt, 2, SQL_PARAM_INPUT, SQL_C_BIT, SQL_BIT, ignored, ignored, &data.b, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 3 [%s]\n", statement);
r = SQLBindParameter(stmt, 3, SQL_PARAM_INPUT, SQL_C_TINYINT, SQL_TINYINT, ignored, ignored, &data.v1, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 4 [%s]\n", statement);
r = SQLBindParameter(stmt, 4, SQL_PARAM_INPUT, SQL_C_SHORT, SQL_SMALLINT, ignored, ignored, &data.v2, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 5 [%s]\n", statement);
r = SQLBindParameter(stmt, 5, SQL_PARAM_INPUT, SQL_C_LONG, SQL_INTEGER, ignored, ignored, &data.v4, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 6 [%s]\n", statement);
r = SQLBindParameter(stmt, 6, SQL_PARAM_INPUT, SQL_C_SBIGINT, SQL_BIGINT, ignored, ignored, &data.v8, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 7 [%s]\n", statement);
r = SQLBindParameter(stmt, 7, SQL_PARAM_INPUT, SQL_C_FLOAT, SQL_FLOAT, ignored, ignored, &data.f4, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 8 [%s]\n", statement);
r = SQLBindParameter(stmt, 8, SQL_PARAM_INPUT, SQL_C_DOUBLE, SQL_DOUBLE, ignored, ignored, &data.f8, ignored, NULL);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 9 [%s]\n", statement);
lbin = SQL_NTS;
r = SQLBindParameter(stmt, 9, SQL_PARAM_INPUT, SQL_C_BINARY, SQL_VARBINARY, sizeof(data.bin)-1, ignored, &data.bin, ignored, &lbin);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "bind 10 [%s]\n", statement);
lblob = SQL_NTS;
r = SQLBindParameter(stmt, 10, SQL_PARAM_INPUT, SQL_C_CHAR, SQL_VARCHAR, sizeof(data.blob)-1, ignored, &data.blob, ignored, &lblob);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
fprintf(stderr, "execute [%s]\n", statement);
r = SQLExecute(stmt);
CHK_RESULT(r, SQL_HANDLE_STMT, stmt);
if (r) break;
// ts += 1;
// v = 2;
// fprintf(stderr, "execute [%s]\n", statement);
// r = SQLExecute(stmt);
// if (r) break;
fprintf(stderr, "done\n");
} while (0);
fprintf(stderr, "r: [%x][%d]\n", r, r);
return r;
}
int main(int argc, char *argv[]) {
if (argc < 4) return 1;
const char *dsn = argv[1];
const char *uid = argv[2];
const char *pwd = argv[3];
SQLRETURN r;
SQLHENV env = {0};
SQLHDBC conn = {0};
r = SQLAllocEnv(&env);
if (r!=SQL_SUCCESS) return 1;
do {
r = SQLAllocConnect(env, &conn);
CHK_RESULT(r, SQL_HANDLE_ENV, env);
if (r!=SQL_SUCCESS) break;
do {
r = SQLConnect(conn, (SQLCHAR*)dsn, strlen(dsn),
(SQLCHAR*)uid, strlen(uid),
(SQLCHAR*)pwd, strlen(pwd));
CHK_RESULT(r, SQL_HANDLE_DBC, conn);
if (r!=SQL_SUCCESS) break;
do {
SQLHSTMT stmt = {0};
r = SQLAllocHandle(SQL_HANDLE_STMT, conn, &stmt);
if (r!=SQL_SUCCESS) break;
do {
do_statement(stmt, "drop database db");
for (size_t i=0; i<sizeof(pre_stmts)/sizeof(pre_stmts[0]); ++i) {
r = do_statement(stmt, pre_stmts[i]);
if (r!=SQL_SUCCESS) break;
}
do {
data_t data = {0};
data.ts = 1591060628001;
data.b = 1;
data.v1 = 127;
data.v2 = 32767;
data.v4 = 2147483647;
data.v8 = 9223372036854775807;
data.f4 = 123.456;
data.f8 = 9999999.999999;
memset(data.bin, 0, sizeof(data.bin));
memset(data.blob, 0, sizeof(data.blob));
snprintf(data.bin, sizeof(data.bin), "hel我lo");
snprintf(data.blob, sizeof(data.blob), "world");
snprintf(data.blob, sizeof(data.blob), "wo人rld");
SQLHSTMT stmt = {0};
r = SQLAllocHandle(SQL_HANDLE_STMT, conn, &stmt);
if (r!=SQL_SUCCESS) break;
do {
r = do_insert(stmt, data);
if (r!=SQL_SUCCESS) break;
} while (0);
SQLFreeHandle(SQL_HANDLE_STMT, stmt);
// r = SQLAllocHandle(SQL_HANDLE_STMT, conn, &stmt);
// if (r!=SQL_SUCCESS) break;
// do {
// r = do_insert(stmt, ts++, v++);
// if (r!=SQL_SUCCESS) break;
// } while (0);
// SQLFreeHandle(SQL_HANDLE_STMT, stmt);
} while (0);
if (r!=SQL_SUCCESS) break;
for (size_t i=0; i<sizeof(pro_stmts)/sizeof(pro_stmts[0]); ++i) {
r = do_statement(stmt, pro_stmts[i]);
if (r!=SQL_SUCCESS) break;
}
} while (0);
SQLFreeHandle(SQL_HANDLE_STMT, stmt);
} while (0);
SQLDisconnect(conn);
} while (0);
SQLFreeConnect(conn);
} while (0);
SQLFreeEnv(env);
return r ? 1 : 0;
}
src/connector/odbc/tests/odbc.py 0 → 100644
浏览文件 @ d74400f3
import pyodbc
cnxn = pyodbc.connect('DSN=TAOS_DSN;UID=root;PWD=taosdata', autocommit=True)
cnxn.setdecoding(pyodbc.SQL_CHAR, encoding='utf-8')
#cnxn.setdecoding(pyodbc.SQL_WCHAR, encoding='utf-8')
#cnxn.setencoding(encoding='utf-8')
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.t")
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
#cursor = cnxn.cursor()
#cursor.execute("""
#INSERT INTO db.t values (?,?,?,?,?,?,?,?,?,?)
#""",
#"2020-12-12 00:00:00",
#1,
#27,
#32767,
#147483647,
#223372036854775807,
#23.456,
#899.999999,
#"foo",
#"bar")
cursor = cnxn.cursor()
cursor.execute("drop database if exists db");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("create database db");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("create table db.t (ts timestamp, b bool, v1 tinyint, v2 smallint, v4 int, v8 bigint, f4 float, f8 double, bin binary(40), blob nchar(10))");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("insert into db.t values('2020-10-13 06:44:00', 1, 127, 32767, 32768, 32769, 123.456, 789.987, 'hello', 'world')")
cursor.close()
cursor = cnxn.cursor()
cursor.execute("insert into db.t values(?,?,?,?,?,?,?,?,?,?)", "2020-10-13 07:06:00", 0, 127, 32767, 32768, 32769, 123.456, 789.987, "hel后lo", "wo哈rld");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.t")
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
cursor = cnxn.cursor()
cursor.execute("drop database if exists db");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("create database db");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("create table db.t (ts timestamp, b bool, v1 tinyint, v2 smallint, v4 int, v8 bigint, f4 float, f8 double, bin binary(4), blob nchar(4))");
cursor.close()
cursor = cnxn.cursor()
cursor.execute("insert into db.t values('2020-10-13 06:44:00', 1, 127, 32767, 32768, 32769, 123.456, 789.987, 'hell', 'worl')")
cursor.close()
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.t")
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
cursor = cnxn.cursor()
cursor.execute("create table db.v (ts timestamp, v1 tinyint)")
cursor.close()
params = [ ('A', 1), ('B', 2), ('C', 3) ]
params = [ ('A', 1), ('B', 2), ('C', 3) ]
params = [ ('2020-10-16 00:00:00', 1),
('2020-10-16 00:00:01', 4),
('2020-10-16 00:00:02', 5),
('2020-10-16 00:00:03.009', 6) ]
cursor = cnxn.cursor()
cursor.fast_executemany = True
cursor.executemany("insert into db.v values (?, ?)", params)
cursor.close()
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.v")
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.v where v1 > ?", 4)
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
cursor = cnxn.cursor()
cursor.execute("SELECT * from db.v where v1 > ?", '5')
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
cursor.close()
src/connector/python/linux/python2/taos/cursor.py
浏览文件 @ d74400f3
......@@ -192,8 +192,10 @@ class TDengineCursor(object):
buffer = [[] for i in range(len(self._fields))]
self._rowcount = 0
while True:
block, num_of_fields = CTaosInterface.fetchBlock(
self._result, self._fields)
block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields)
errno = CTaosInterface.libtaos.taos_errno(self._result)
if errno != 0:
raise ProgrammingError(CTaosInterface.errStr(self._result), errno)
if num_of_fields == 0:
break
self._rowcount += num_of_fields
......
src/connector/python/linux/python3/taos/cursor.py
浏览文件 @ d74400f3
......@@ -207,8 +207,10 @@ class TDengineCursor(object):
buffer = [[] for i in range(len(self._fields))]
self._rowcount = 0
while True:
block, num_of_fields = CTaosInterface.fetchBlock(
self._result, self._fields)
block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields)
errno = CTaosInterface.libtaos.taos_errno(self._result)
if errno != 0:
raise ProgrammingError(CTaosInterface.errStr(self._result), errno)
if num_of_fields == 0:
break
self._rowcount += num_of_fields
......
src/connector/python/windows/python2/taos/cursor.py
浏览文件 @ d74400f3
......@@ -142,6 +142,9 @@ class TDengineCursor(object):
self._rowcount = 0
while True:
block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields)
errno = CTaosInterface.libtaos.taos_errno(self._result)
if errno != 0:
raise ProgrammingError(CTaosInterface.errStr(self._result), errno)
if num_of_fields == 0: break
self._rowcount += num_of_fields
for i in range(len(self._fields)):
......
src/connector/python/windows/python3/taos/cursor.py
浏览文件 @ d74400f3
......@@ -142,6 +142,9 @@ class TDengineCursor(object):
self._rowcount = 0
while True:
block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields)
errno = CTaosInterface.libtaos.taos_errno(self._result)
if errno != 0:
raise ProgrammingError(CTaosInterface.errStr(self._result), errno)
if num_of_fields == 0: break
self._rowcount += num_of_fields
for i in range(len(self._fields)):
......
src/dnode/src/dnodeShell.c
浏览文件 @ d74400f3
......@@ -173,15 +173,15 @@ static int dnodeRetrieveUserAuthInfo(char *user, char *spi, char *encrypt, char
return rpcRsp.code;
}
void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t sid) {
dDebug("vgId:%d, sid:%d send config table msg to mnode", vgId, sid);
void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t tid) {
dDebug("vgId:%d, tid:%d send config table msg to mnode", vgId, tid);
int32_t contLen = sizeof(SDMConfigTableMsg);
SDMConfigTableMsg *pMsg = rpcMallocCont(contLen);
pMsg->dnodeId = htonl(dnodeGetDnodeId());
pMsg->vgId = htonl(vgId);
pMsg->sid = htonl(sid);
pMsg->tid = htonl(tid);
SRpcMsg rpcMsg = {0};
rpcMsg.pCont = pMsg;
......@@ -194,18 +194,18 @@ void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t sid) {
if (rpcRsp.code != 0) {
rpcFreeCont(rpcRsp.pCont);
dError("vgId:%d, sid:%d failed to config table from mnode", vgId, sid);
dError("vgId:%d, tid:%d failed to config table from mnode", vgId, tid);
return NULL;
} else {
dInfo("vgId:%d, sid:%d config table msg is received", vgId, sid);
dInfo("vgId:%d, tid:%d config table msg is received", vgId, tid);
// delete this after debug finished
SMDCreateTableMsg *pTable = rpcRsp.pCont;
int16_t numOfColumns = htons(pTable->numOfColumns);
int16_t numOfTags = htons(pTable->numOfTags);
int32_t sid = htonl(pTable->sid);
int32_t tableId = htonl(pTable->tid);
uint64_t uid = htobe64(pTable->uid);
dInfo("table:%s, numOfColumns:%d numOfTags:%d sid:%d uid:%" PRIu64, pTable->tableId, numOfColumns, numOfTags, sid, uid);
dInfo("table:%s, numOfColumns:%d numOfTags:%d tid:%d uid:%" PRIu64, pTable->tableId, numOfColumns, numOfTags, tableId, uid);
return rpcRsp.pCont;
}
......
src/dnode/src/dnodeVRead.c
浏览文件 @ d74400f3
......@@ -189,7 +189,6 @@ void dnodeSendRpcReadRsp(void *pVnode, SReadMsg *pRead, int32_t code) {
void dnodeDispatchNonRspMsg(void *pVnode, SReadMsg *pRead, int32_t code) {
rpcFreeCont(pRead->rpcMsg.pCont);
vnodeRelease(pVnode);
return;
}
static void *dnodeProcessReadQueue(void *param) {
......@@ -213,7 +212,8 @@ static void *dnodeProcessReadQueue(void *param) {
} else {
if (code == TSDB_CODE_QRY_HAS_RSP) {
dnodeSendRpcReadRsp(pVnode, pReadMsg, pReadMsg->rpcMsg.code);
} else { // code == TSDB_CODE_NOT_READY, do not return msg to client
} else { // code == TSDB_CODE_QRY_NOT_READY, do not return msg to client
assert(pReadMsg->rpcMsg.handle == NULL || (pReadMsg->rpcMsg.handle != NULL && pReadMsg->rpcMsg.msgType == 5));
dnodeDispatchNonRspMsg(pVnode, pReadMsg, code);
}
}
......
src/inc/dnode.h
浏览文件 @ d74400f3
......@@ -49,7 +49,7 @@ void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg));
void dnodeSendMsgToDnode(SRpcEpSet *epSet, SRpcMsg *rpcMsg);
void dnodeSendMsgToMnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp);
void dnodeSendMsgToDnodeRecv(SRpcMsg *rpcMsg, SRpcMsg *rpcRsp, SRpcEpSet *epSet);
void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t sid);
void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t tid);
void *dnodeAllocateVnodeWqueue(void *pVnode);
void dnodeFreeVnodeWqueue(void *queue);
......
src/inc/taos.h
浏览文件 @ d74400f3
......@@ -22,12 +22,12 @@
extern "C" {
#endif
typedef void TAOS;
typedef void** TAOS_ROW;
typedef void TAOS_RES;
typedef void TAOS_SUB;
typedef void TAOS_STREAM;
typedef void TAOS_STMT;
typedef void TAOS;
typedef void TAOS_STMT;
typedef void TAOS_RES;
typedef void TAOS_STREAM;
typedef void TAOS_SUB;
typedef void **TAOS_ROW;
// Data type definition
#define TSDB_DATA_TYPE_NULL 0 // 1 bytes
......@@ -69,6 +69,8 @@ DLL_EXPORT int taos_options(TSDB_OPTION option, const void *arg, ...);
DLL_EXPORT TAOS *taos_connect(const char *ip, const char *user, const char *pass, const char *db, uint16_t port);
DLL_EXPORT void taos_close(TAOS *taos);
const char *taos_data_type(int type);
typedef struct TAOS_BIND {
int buffer_type;
void * buffer;
......@@ -77,10 +79,26 @@ typedef struct TAOS_BIND {
int * is_null;
int is_unsigned; // unused
int * error; // unused
union {
int64_t ts;
int8_t b;
int8_t v1;
int16_t v2;
int32_t v4;
int64_t v8;
float f4;
double f8;
unsigned char *bin;
char *nchar;
} u;
unsigned int allocated;
} TAOS_BIND;
TAOS_STMT *taos_stmt_init(TAOS *taos);
int taos_stmt_prepare(TAOS_STMT *stmt, const char *sql, unsigned long length);
int taos_stmt_is_insert(TAOS_STMT *stmt, int *insert);
int taos_stmt_num_params(TAOS_STMT *stmt, int *nums);
int taos_stmt_get_param(TAOS_STMT *stmt, int idx, int *type, int *bytes);
int taos_stmt_bind_param(TAOS_STMT *stmt, TAOS_BIND *bind);
int taos_stmt_add_batch(TAOS_STMT *stmt);
int taos_stmt_execute(TAOS_STMT *stmt);
......
src/inc/taoserror.h
浏览文件 @ d74400f3
......@@ -28,7 +28,7 @@ extern "C" {
#else
#define TAOS_DEFINE_ERROR(name, mod, code, msg) static const int32_t name = (0x80000000 | ((mod)<<16) | (code));
#endif
#define TAOS_SYSTEM_ERROR(code) (0x80ff0000 | (code))
#define TAOS_SUCCEEDED(err) ((err) >= 0)
#define TAOS_FAILED(err) ((err) < 0)
......@@ -37,7 +37,7 @@ const char* tstrerror(int32_t err);
int32_t* taosGetErrno();
#define terrno (*taosGetErrno())
#define TSDB_CODE_SUCCESS 0
#ifdef TAOS_ERROR_C
......@@ -182,7 +182,7 @@ TAOS_DEFINE_ERROR(TSDB_CODE_DND_OUT_OF_MEMORY, 0, 0x0401, "Dnode out
TAOS_DEFINE_ERROR(TSDB_CODE_DND_NO_WRITE_ACCESS, 0, 0x0402, "No permission for disk files in dnode")
TAOS_DEFINE_ERROR(TSDB_CODE_DND_INVALID_MSG_LEN, 0, 0x0403, "Invalid message length")
// vnode
// vnode
TAOS_DEFINE_ERROR(TSDB_CODE_VND_ACTION_IN_PROGRESS, 0, 0x0500, "Action in progress")
TAOS_DEFINE_ERROR(TSDB_CODE_VND_MSG_NOT_PROCESSED, 0, 0x0501, "Message not processed")
TAOS_DEFINE_ERROR(TSDB_CODE_VND_ACTION_NEED_REPROCESSED, 0, 0x0502, "Action need to be reprocessed")
......@@ -354,6 +354,23 @@ TAOS_DEFINE_ERROR(TSDB_CODE_HTTP_OP_TAG_VALUE_TOO_LONG, 0, 0x11A4, "tag value
TAOS_DEFINE_ERROR(TSDB_CODE_HTTP_OP_VALUE_NULL, 0, 0x11A5, "value not find")
TAOS_DEFINE_ERROR(TSDB_CODE_HTTP_OP_VALUE_TYPE, 0, 0x11A6, "value type should be boolean, number or string")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_OOM, 0, 0x2101, "out of memory")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_CONV_UNDEF, 0, 0x2102, "convertion undefined")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_CONV_TRUNC, 0, 0x2103, "convertion truncated")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_CONV_NOT_SUPPORT, 0, 0x2104, "convertion not supported")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_OUT_OF_RANGE, 0, 0x2105, "out of range")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_NOT_SUPPORT, 0, 0x2106, "not supported yet")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_INVALID_HANDLE, 0, 0x2107, "invalid handle")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_NO_RESULT, 0, 0x2108, "no result set")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_NO_FIELDS, 0, 0x2109, "no fields returned")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_INVALID_CURSOR, 0, 0x2110, "invalid cursor")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_STATEMENT_NOT_READY, 0, 0x2111, "statement not ready")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_CONNECTION_BUSY, 0, 0x2112, "connection still busy")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_BAD_CONNSTR, 0, 0x2113, "bad connection string")
TAOS_DEFINE_ERROR(TSDB_CODE_ODBC_BAD_ARG, 0, 0x2114, "bad argument")
#ifdef TAOS_ERROR_C
};
#endif
......
src/inc/taosmsg.h
浏览文件 @ d74400f3
......@@ -182,10 +182,16 @@ extern char *taosMsg[];
#pragma pack(push, 1)
// null-terminated string instead of char array to avoid too many memory consumption in case of more than 1M tableMeta
typedef struct {
char fqdn[TSDB_FQDN_LEN];
uint16_t port;
} SEpAddr;
} SEpAddrMsg;
typedef struct {
char* fqdn;
uint16_t port;
} SEpAddr1;
typedef struct {
int32_t numOfVnodes;
......@@ -245,7 +251,7 @@ typedef struct {
int8_t tableType;
int16_t numOfColumns;
int16_t numOfTags;
int32_t sid;
int32_t tid;
int32_t sversion;
int32_t tversion;
int32_t tagDataLen;
......@@ -354,7 +360,7 @@ typedef struct {
typedef struct {
int32_t contLen;
int32_t vgId;
int32_t sid;
int32_t tid;
uint64_t uid;
char tableId[TSDB_TABLE_FNAME_LEN];
} SMDDropTableMsg;
......@@ -401,6 +407,7 @@ typedef struct SExprInfo {
int16_t bytes;
int16_t type;
int32_t interBytes;
int64_t uid;
} SExprInfo;
typedef struct SColumnFilterInfo {
......@@ -662,16 +669,27 @@ typedef struct SCMSTableVgroupMsg {
} SCMSTableVgroupMsg, SCMSTableVgroupRspMsg;
typedef struct {
int32_t vgId;
int8_t numOfEps;
SEpAddr epAddr[TSDB_MAX_REPLICA];
int32_t vgId;
int8_t numOfEps;
SEpAddr1 epAddr[TSDB_MAX_REPLICA];
} SCMVgroupInfo;
typedef struct {
int32_t vgId;
int8_t numOfEps;
SEpAddrMsg epAddr[TSDB_MAX_REPLICA];
} SCMVgroupMsg;
typedef struct {
int32_t numOfVgroups;
SCMVgroupInfo vgroups[];
} SVgroupsInfo;
typedef struct {
int32_t numOfVgroups;
SCMVgroupMsg vgroups[];
} SVgroupsMsg;
typedef struct STableMetaMsg {
int32_t contLen;
char tableId[TSDB_TABLE_FNAME_LEN]; // table id
......@@ -682,9 +700,9 @@ typedef struct STableMetaMsg {
int16_t numOfColumns;
int16_t sversion;
int16_t tversion;
int32_t sid;
int32_t tid;
uint64_t uid;
SCMVgroupInfo vgroup;
SCMVgroupMsg vgroup;
SSchema schema[];
} STableMetaMsg;
......@@ -730,7 +748,7 @@ typedef struct {
typedef struct {
int32_t dnodeId;
int32_t vgId;
int32_t sid;
int32_t tid;
} SDMConfigTableMsg;
typedef struct {
......
src/inc/vnode.h
浏览文件 @ d74400f3
......@@ -41,6 +41,8 @@ typedef struct {
SRpcMsg rpcMsg;
} SReadMsg;
extern char *vnodeStatus[];
int32_t vnodeCreate(SMDCreateVnodeMsg *pVnodeCfg);
int32_t vnodeDrop(int32_t vgId);
int32_t vnodeOpen(int32_t vgId, char *rootDir);
......@@ -54,6 +56,7 @@ void vnodeRelease(void *pVnode); // dec refCount
void* vnodeGetWal(void *pVnode);
int32_t vnodeProcessWrite(void *pVnode, int qtype, void *pHead, void *item);
int32_t vnodeCheckWrite(void *pVnode);
int32_t vnodeGetVnodeList(int32_t vnodeList[], int32_t *numOfVnodes);
void vnodeBuildStatusMsg(void *param);
void vnodeConfirmForward(void *param, uint64_t version, int32_t code);
......@@ -63,6 +66,7 @@ int32_t vnodeInitResources();
void vnodeCleanupResources();
int32_t vnodeProcessRead(void *pVnode, SReadMsg *pReadMsg);
int32_t vnodeCheckRead(void *pVnode);
#ifdef __cplusplus
}
......
src/kit/shell/inc/shell.h
浏览文件 @ d74400f3
......@@ -60,7 +60,7 @@ typedef struct SShellArguments {
extern void shellParseArgument(int argc, char* argv[], SShellArguments* arguments);
extern TAOS* shellInit(SShellArguments* args);
extern void* shellLoopQuery(void* arg);
extern void taos_error(TAOS* con);
extern void taos_error(TAOS_RES* tres);
extern int regex_match(const char* s, const char* reg, int cflags);
void shellReadCommand(TAOS* con, char command[]);
int32_t shellRunCommand(TAOS* con, char* command);
......@@ -72,7 +72,7 @@ void source_dir(TAOS* con, SShellArguments* args);
void get_history_path(char* history);
void cleanup_handler(void* arg);
void exitShell();
int shellDumpResult(TAOS* con, char* fname, int* error_no, bool printMode);
int shellDumpResult(TAOS_RES* con, char* fname, int* error_no, bool printMode);
void shellGetGrantInfo(void *con);
int isCommentLine(char *line);
......
src/kit/shell/src/shellEngine.c
浏览文件 @ d74400f3
......@@ -493,7 +493,7 @@ static int dumpResultToFile(const char* fname, TAOS_RES* tres) {
if (i > 0) {
fputc(',', fp);
}
dumpFieldToFile(fp, row[i], fields +i, length[i], precision);
dumpFieldToFile(fp, (const char*)row[i], fields +i, length[i], precision);
}
fputc('\n', fp);
......@@ -619,7 +619,7 @@ static int verticalPrintResult(TAOS_RES* tres) {
int padding = (int)(maxColNameLen - strlen(field->name));
printf("%*.s%s: ", padding, " ", field->name);
printField(row[i], field, 0, length[i], precision);
printField((const char*)row[i], field, 0, length[i], precision);
putchar('\n');
}
......@@ -720,7 +720,7 @@ static int horizontalPrintResult(TAOS_RES* tres) {
int32_t* length = taos_fetch_lengths(tres);
for (int i = 0; i < num_fields; i++) {
putchar(' ');
printField(row[i], fields + i, width[i], length[i], precision);
printField((const char*)row[i], fields + i, width[i], length[i], precision);
putchar(' ');
putchar('|');
}
......
src/kit/shell/src/shellImport.c
浏览文件 @ d74400f3
......@@ -204,7 +204,7 @@ static void shellSourceFile(TAOS *con, char *fptr) {
int32_t code = taos_errno(pSql);
if (code != 0) {
fprintf(stderr, "DB error: %s: %s (%d)\n", taos_errstr(con), fname, lineNo);
fprintf(stderr, "DB error: %s: %s (%d)\n", taos_errstr(pSql), fname, lineNo);
}
/* free local resouce: allocated memory/metric-meta refcnt */
......@@ -243,7 +243,7 @@ static void shellRunImportThreads(SShellArguments* args)
pThread->totalThreads = args->threadNum;
pThread->taos = taos_connect(args->host, args->user, args->password, args->database, tsDnodeShellPort);
if (pThread->taos == NULL) {
fprintf(stderr, "ERROR: thread:%d failed connect to TDengine, error:%s\n", pThread->threadIndex, taos_errstr(pThread->taos));
fprintf(stderr, "ERROR: thread:%d failed connect to TDengine, error:%s\n", pThread->threadIndex, "null taos"/*taos_errstr(pThread->taos)*/);
exit(0);
}
......
src/mnode/inc/mnodeDef.h
浏览文件 @ d74400f3
......@@ -115,7 +115,7 @@ typedef struct {
uint64_t suid;
int64_t createdTime;
int32_t numOfColumns; //used by normal table
int32_t sid;
int32_t tid;
int32_t vgId;
int32_t sqlLen;
int8_t updateEnd[4];
......
src/mnode/src/mnodeProfile.c
浏览文件 @ d74400f3
......@@ -111,7 +111,6 @@ void mnodeReleaseConn(SConnObj *pConn) {
}
SConnObj *mnodeAccquireConn(int32_t connId, char *user, uint32_t ip, uint16_t port) {
uint64_t expireTime = CONN_KEEP_TIME * 1000 + (uint64_t)taosGetTimestampMs();
SConnObj *pConn = taosCacheAcquireByKey(tsMnodeConnCache, &connId, sizeof(int32_t));
if (pConn == NULL) {
mDebug("connId:%d, is already destroyed, user:%s ip:%s:%u", connId, user, taosIpStr(ip), port);
......@@ -126,7 +125,7 @@ SConnObj *mnodeAccquireConn(int32_t connId, char *user, uint32_t ip, uint16_t po
}
// mDebug("connId:%d, is incoming, user:%s ip:%s:%u", connId, pConn->user, taosIpStr(pConn->ip), pConn->port);
pConn->lastAccess = expireTime;
pConn->lastAccess = CONN_KEEP_TIME * 1000 + (uint64_t)taosGetTimestampMs();
return pConn;
}
......@@ -626,7 +625,7 @@ static int32_t mnodeProcessKillConnectionMsg(SMnodeMsg *pMsg) {
SCMKillConnMsg *pKill = pMsg->rpcMsg.pCont;
int32_t connId = atoi(pKill->queryId);
SConnObj * pConn = taosCacheAcquireByKey(tsMnodeConnCache, &connId, sizeof(int32_t));
SConnObj *pConn = taosCacheAcquireByKey(tsMnodeConnCache, &connId, sizeof(int32_t));
if (pConn == NULL) {
mError("connId:%s, failed to kill, conn not exist", pKill->queryId);
return TSDB_CODE_MND_INVALID_CONN_ID;
......
src/mnode/src/mnodeTable.c
浏览文件 @ d74400f3
......@@ -303,7 +303,7 @@ static int32_t mnodeChildTableActionRestored() {
SVgObj *pVgroup = mnodeGetVgroup(pTable->vgId);
if (pVgroup == NULL) {
mError("ctable:%s, failed to get vgId:%d sid:%d, discard it", pTable->info.tableId, pTable->vgId, pTable->sid);
mError("ctable:%s, failed to get vgId:%d tid:%d, discard it", pTable->info.tableId, pTable->vgId, pTable->tid);
pTable->vgId = 0;
SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb};
sdbDeleteRow(&desc);
......@@ -314,7 +314,7 @@ static int32_t mnodeChildTableActionRestored() {
if (strcmp(pVgroup->dbName, pDb->name) != 0) {
mError("ctable:%s, db:%s not match with vgId:%d db:%s sid:%d, discard it",
pTable->info.tableId, pDb->name, pTable->vgId, pVgroup->dbName, pTable->sid);
pTable->info.tableId, pDb->name, pTable->vgId, pVgroup->dbName, pTable->tid);
pTable->vgId = 0;
SSdbOper desc = {.type = SDB_OPER_LOCAL, .pObj = pTable, .table = tsChildTableSdb};
sdbDeleteRow(&desc);
......@@ -771,8 +771,8 @@ static int32_t mnodeProcessDropTableMsg(SMnodeMsg *pMsg) {
return mnodeProcessDropSuperTableMsg(pMsg);
} else {
SChildTableObj *pCTable = (SChildTableObj *)pMsg->pTable;
mInfo("app:%p:%p, table:%s, start to drop ctable, vgId:%d sid:%d uid:%" PRIu64, pMsg->rpcMsg.ahandle, pMsg,
pDrop->tableId, pCTable->vgId, pCTable->sid, pCTable->uid);
mInfo("app:%p:%p, table:%s, start to drop ctable, vgId:%d tid:%d uid:%" PRIu64, pMsg->rpcMsg.ahandle, pMsg,
pDrop->tableId, pCTable->vgId, pCTable->tid, pCTable->uid);
return mnodeProcessDropChildTableMsg(pMsg);
}
}
......@@ -1476,13 +1476,14 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) {
int32_t numOfTable = htonl(pInfo->numOfTables);
// reserve space
int32_t contLen = sizeof(SCMSTableVgroupRspMsg) + 32 * sizeof(SCMVgroupInfo) + sizeof(SVgroupsInfo);
int32_t contLen = sizeof(SCMSTableVgroupRspMsg) + 32 * sizeof(SCMVgroupMsg) + sizeof(SVgroupsMsg);
for (int32_t i = 0; i < numOfTable; ++i) {
char *stableName = (char*)pInfo + sizeof(SCMSTableVgroupMsg) + (TSDB_TABLE_FNAME_LEN) * i;
SSuperTableObj *pTable = mnodeGetSuperTable(stableName);
if (pTable != NULL && pTable->vgHash != NULL) {
contLen += (taosHashGetSize(pTable->vgHash) * sizeof(SCMVgroupInfo) + sizeof(SVgroupsInfo));
contLen += (taosHashGetSize(pTable->vgHash) * sizeof(SCMVgroupMsg) + sizeof(SVgroupsMsg));
}
mnodeDecTableRef(pTable);
}
......@@ -1510,12 +1511,12 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) {
// even this super table has no corresponding table, still return
pRsp->numOfTables++;
SVgroupsInfo *pVgroupInfo = (SVgroupsInfo *)msg;
pVgroupInfo->numOfVgroups = 0;
SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg;
pVgroupMsg->numOfVgroups = 0;
msg += sizeof(SVgroupsInfo);
msg += sizeof(SVgroupsMsg);
} else {
SVgroupsInfo *pVgroupInfo = (SVgroupsInfo *)msg;
SVgroupsMsg *pVgroupMsg = (SVgroupsMsg *)msg;
SHashMutableIterator *pIter = taosHashCreateIter(pTable->vgHash);
int32_t vgSize = 0;
......@@ -1524,15 +1525,17 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) {
SVgObj * pVgroup = mnodeGetVgroup(*pVgId);
if (pVgroup == NULL) continue;
pVgroupInfo->vgroups[vgSize].vgId = htonl(pVgroup->vgId);
pVgroupMsg->vgroups[vgSize].vgId = htonl(pVgroup->vgId);
pVgroupMsg->vgroups[vgSize].numOfEps = 0;
for (int32_t vn = 0; vn < pVgroup->numOfVnodes; ++vn) {
SDnodeObj *pDnode = pVgroup->vnodeGid[vn].pDnode;
if (pDnode == NULL) break;
tstrncpy(pVgroupInfo->vgroups[vgSize].epAddr[vn].fqdn, pDnode->dnodeFqdn, TSDB_FQDN_LEN);
pVgroupInfo->vgroups[vgSize].epAddr[vn].port = htons(pDnode->dnodePort);
tstrncpy(pVgroupMsg->vgroups[vgSize].epAddr[vn].fqdn, pDnode->dnodeFqdn, TSDB_FQDN_LEN);
pVgroupMsg->vgroups[vgSize].epAddr[vn].port = htons(pDnode->dnodePort);
pVgroupInfo->vgroups[vgSize].numOfEps++;
pVgroupMsg->vgroups[vgSize].numOfEps++;
}
vgSize++;
......@@ -1542,10 +1545,10 @@ static int32_t mnodeProcessSuperTableVgroupMsg(SMnodeMsg *pMsg) {
taosHashDestroyIter(pIter);
mnodeDecTableRef(pTable);
pVgroupInfo->numOfVgroups = htonl(vgSize);
pVgroupMsg->numOfVgroups = htonl(vgSize);
// one table is done, try the next table
msg += sizeof(SVgroupsInfo) + vgSize * sizeof(SCMVgroupInfo);
msg += sizeof(SVgroupsMsg) + vgSize * sizeof(SCMVgroupMsg);
pRsp->numOfTables++;
}
}
......@@ -1595,7 +1598,7 @@ static void *mnodeBuildCreateChildTableMsg(SCMCreateTableMsg *pMsg, SChildTableO
pCreate->vgId = htonl(pTable->vgId);
pCreate->tableType = pTable->info.type;
pCreate->createdTime = htobe64(pTable->createdTime);
pCreate->sid = htonl(pTable->sid);
pCreate->tid = htonl(pTable->tid);
pCreate->sqlDataLen = htonl(pTable->sqlLen);
pCreate->uid = htobe64(pTable->uid);
......@@ -1644,7 +1647,7 @@ static int32_t mnodeDoCreateChildTableFp(SMnodeMsg *pMsg) {
assert(pTable);
mDebug("app:%p:%p, table:%s, created in mnode, vgId:%d sid:%d, uid:%" PRIu64, pMsg->rpcMsg.ahandle, pMsg,
pTable->info.tableId, pTable->vgId, pTable->sid, pTable->uid);
pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid);
SCMCreateTableMsg *pCreate = pMsg->rpcMsg.pCont;
SMDCreateTableMsg *pMDCreate = mnodeBuildCreateChildTableMsg(pCreate, pTable);
......@@ -1686,7 +1689,7 @@ static int32_t mnodeDoCreateChildTableCb(SMnodeMsg *pMsg, int32_t code) {
return TSDB_CODE_MND_ACTION_IN_PROGRESS;
} else {
mError("app:%p:%p, table:%s, failed to create table sid:%d, uid:%" PRIu64 ", reason:%s", pMsg->rpcMsg.ahandle, pMsg,
pTable->info.tableId, pTable->sid, pTable->uid, tstrerror(code));
pTable->info.tableId, pTable->tid, pTable->uid, tstrerror(code));
SSdbOper desc = {.type = SDB_OPER_GLOBAL, .pObj = pTable, .table = tsChildTableSdb};
sdbDeleteRow(&desc);
return code;
......@@ -1710,7 +1713,7 @@ static int32_t mnodeDoCreateChildTable(SMnodeMsg *pMsg, int32_t tid) {
pTable->info.tableId = strdup(pCreate->tableId);
pTable->createdTime = taosGetTimestampMs();
pTable->sid = tid;
pTable->tid = tid;
pTable->vgId = pVgroup->vgId;
if (pTable->info.type == TSDB_CHILD_TABLE) {
......@@ -1724,7 +1727,7 @@ static int32_t mnodeDoCreateChildTable(SMnodeMsg *pMsg, int32_t tid) {
}
pTable->suid = pMsg->pSTable->uid;
pTable->uid = (((uint64_t)pTable->vgId) << 48) + ((((uint64_t)pTable->sid) & ((1ul << 24) - 1ul)) << 24) +
pTable->uid = (((uint64_t)pTable->vgId) << 48) + ((((uint64_t)pTable->tid) & ((1ul << 24) - 1ul)) << 24) +
((sdbGetVersion() & ((1ul << 16) - 1ul)) << 8) + (taosRand() & ((1ul << 8) - 1ul));
pTable->superTable = pMsg->pSTable;
} else {
......@@ -1732,7 +1735,7 @@ static int32_t mnodeDoCreateChildTable(SMnodeMsg *pMsg, int32_t tid) {
int64_t us = taosGetTimestampUs();
pTable->uid = (us << 24) + ((sdbGetVersion() & ((1ul << 16) - 1ul)) << 8) + (taosRand() & ((1ul << 8) - 1ul));
} else {
pTable->uid = (((uint64_t)pTable->vgId) << 48) + ((((uint64_t)pTable->sid) & ((1ul << 24) - 1ul)) << 24) +
pTable->uid = (((uint64_t)pTable->vgId) << 48) + ((((uint64_t)pTable->tid) & ((1ul << 24) - 1ul)) << 24) +
((sdbGetVersion() & ((1ul << 16) - 1ul)) << 8) + (taosRand() & ((1ul << 8) - 1ul));
}
......@@ -1789,7 +1792,7 @@ static int32_t mnodeDoCreateChildTable(SMnodeMsg *pMsg, int32_t tid) {
tstrerror(code));
} else {
mDebug("app:%p:%p, table:%s, allocated in vgroup, vgId:%d sid:%d uid:%" PRIu64, pMsg->rpcMsg.ahandle, pMsg,
pTable->info.tableId, pVgroup->vgId, pTable->sid, pTable->uid);
pTable->info.tableId, pVgroup->vgId, pTable->tid, pTable->uid);
}
return code;
......@@ -1807,8 +1810,8 @@ static int32_t mnodeProcessCreateChildTableMsg(SMnodeMsg *pMsg) {
if (pMsg->retry == 0) {
if (pMsg->pTable == NULL) {
SVgObj *pVgroup = NULL;
int32_t sid = 0;
code = mnodeGetAvailableVgroup(pMsg, &pVgroup, &sid);
int32_t tid = 0;
code = mnodeGetAvailableVgroup(pMsg, &pVgroup, &tid);
if (code != TSDB_CODE_SUCCESS) {
mDebug("app:%p:%p, table:%s, failed to get available vgroup, reason:%s", pMsg->rpcMsg.ahandle, pMsg,
pCreate->tableId, tstrerror(code));
......@@ -1822,7 +1825,7 @@ static int32_t mnodeProcessCreateChildTableMsg(SMnodeMsg *pMsg) {
pMsg->pVgroup = pVgroup;
mnodeIncVgroupRef(pVgroup);
return mnodeDoCreateChildTable(pMsg, sid);
return mnodeDoCreateChildTable(pMsg, tid);
}
} else {
if (pMsg->pTable == NULL) pMsg->pTable = mnodeGetTable(pCreate->tableId);
......@@ -1852,13 +1855,13 @@ static int32_t mnodeSendDropChildTableMsg(SMnodeMsg *pMsg, bool needReturn) {
tstrncpy(pDrop->tableId, pTable->info.tableId, TSDB_TABLE_FNAME_LEN);
pDrop->vgId = htonl(pTable->vgId);
pDrop->contLen = htonl(sizeof(SMDDropTableMsg));
pDrop->sid = htonl(pTable->sid);
pDrop->tid = htonl(pTable->tid);
pDrop->uid = htobe64(pTable->uid);
SRpcEpSet epSet = mnodeGetEpSetFromVgroup(pMsg->pVgroup);
mInfo("app:%p:%p, ctable:%s, send drop ctable msg, vgId:%d sid:%d uid:%" PRIu64, pMsg->rpcMsg.ahandle, pMsg,
pDrop->tableId, pTable->vgId, pTable->sid, pTable->uid);
pDrop->tableId, pTable->vgId, pTable->tid, pTable->uid);
SRpcMsg rpcMsg = {
.ahandle = pMsg,
......@@ -2097,7 +2100,7 @@ static int32_t mnodeDoGetChildTableMeta(SMnodeMsg *pMsg, STableMetaMsg *pMeta) {
SChildTableObj *pTable = (SChildTableObj *)pMsg->pTable;
pMeta->uid = htobe64(pTable->uid);
pMeta->sid = htonl(pTable->sid);
pMeta->tid = htonl(pTable->tid);
pMeta->precision = pDb->cfg.precision;
pMeta->tableType = pTable->info.type;
tstrncpy(pMeta->tableId, pTable->info.tableId, TSDB_TABLE_FNAME_LEN);
......@@ -2137,7 +2140,7 @@ static int32_t mnodeDoGetChildTableMeta(SMnodeMsg *pMsg, STableMetaMsg *pMeta) {
pMeta->vgroup.vgId = htonl(pMsg->pVgroup->vgId);
mDebug("app:%p:%p, table:%s, uid:%" PRIu64 " table meta is retrieved, vgId:%d sid:%d", pMsg->rpcMsg.ahandle, pMsg,
pTable->info.tableId, pTable->uid, pTable->vgId, pTable->sid);
pTable->info.tableId, pTable->uid, pTable->vgId, pTable->tid);
return TSDB_CODE_SUCCESS;
}
......@@ -2289,11 +2292,11 @@ static void mnodeDropAllChildTablesInStable(SSuperTableObj *pStable) {
}
#if 0
static SChildTableObj* mnodeGetTableByPos(int32_t vnode, int32_t sid) {
static SChildTableObj* mnodeGetTableByPos(int32_t vnode, int32_t tid) {
SVgObj *pVgroup = mnodeGetVgroup(vnode);
if (pVgroup == NULL) return NULL;
SChildTableObj *pTable = pVgroup->tableList[sid - 1];
SChildTableObj *pTable = pVgroup->tableList[tid - 1];
mnodeIncTableRef((STableObj *)pTable);
mnodeDecVgroupRef(pVgroup);
......@@ -2341,12 +2344,12 @@ static void mnodeProcessDropChildTableRsp(SRpcMsg *rpcMsg) {
assert(pTable);
mInfo("app:%p:%p, table:%s, drop table rsp received, vgId:%d sid:%d uid:%" PRIu64 ", thandle:%p result:%s",
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->sid, pTable->uid,
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid,
mnodeMsg->rpcMsg.handle, tstrerror(rpcMsg->code));
if (rpcMsg->code != TSDB_CODE_SUCCESS) {
mError("app:%p:%p, table:%s, failed to drop in dnode, vgId:%d sid:%d uid:%" PRIu64 ", reason:%s",
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->sid, pTable->uid,
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid,
tstrerror(rpcMsg->code));
dnodeSendRpcMnodeWriteRsp(mnodeMsg, rpcMsg->code);
return;
......@@ -2384,7 +2387,7 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) {
// If the table is deleted by another thread during creation, stop creating and send drop msg to vnode
if (sdbCheckRowDeleted(tsChildTableSdb, pTable)) {
mDebug("app:%p:%p, table:%s, create table rsp received, but a deleting opertion incoming, vgId:%d sid:%d uid:%" PRIu64,
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->sid, pTable->uid);
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid);
// if the vgroup is already dropped from hash, it can't be accquired by pTable->vgId
// so the refCount of vgroup can not be decreased
......@@ -2419,13 +2422,13 @@ static void mnodeProcessCreateChildTableRsp(SRpcMsg *rpcMsg) {
if (mnodeMsg->retry++ < 10) {
mDebug("app:%p:%p, table:%s, create table rsp received, need retry, times:%d vgId:%d sid:%d uid:%" PRIu64
" result:%s thandle:%p",
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, mnodeMsg->retry, pTable->vgId, pTable->sid,
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, mnodeMsg->retry, pTable->vgId, pTable->tid,
pTable->uid, tstrerror(rpcMsg->code), mnodeMsg->rpcMsg.handle);
dnodeDelayReprocessMnodeWriteMsg(mnodeMsg);
} else {
mError("app:%p:%p, table:%s, failed to create in dnode, vgId:%d sid:%d uid:%" PRIu64 ", result:%s thandle:%p",
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->sid, pTable->uid,
mnodeMsg->rpcMsg.ahandle, mnodeMsg, pTable->info.tableId, pTable->vgId, pTable->tid, pTable->uid,
tstrerror(rpcMsg->code), mnodeMsg->rpcMsg.handle);
SSdbOper oper = {.type = SDB_OPER_GLOBAL, .table = tsChildTableSdb, .pObj = pTable};
......@@ -2680,7 +2683,7 @@ static int32_t mnodeRetrieveShowTables(SShowObj *pShow, char *data, int32_t rows
// tid
pWrite = data + pShow->offset[cols] * rows + pShow->bytes[cols] * numOfRows;
*(int32_t*) pWrite = pTable->sid;
*(int32_t*) pWrite = pTable->tid;
cols++;
//vgid
......
src/mnode/src/mnodeVgroup.c
浏览文件 @ d74400f3
......@@ -793,12 +793,12 @@ static int32_t mnodeRetrieveVgroups(SShowObj *pShow, char *data, int32_t rows, v
void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
int32_t idPoolSize = taosIdPoolMaxSize(pVgroup->idPool);
if (pTable->sid > idPoolSize) {
if (pTable->tid > idPoolSize) {
mnodeAllocVgroupIdPool(pVgroup);
}
if (pTable->sid >= 1) {
taosIdPoolMarkStatus(pVgroup->idPool, pTable->sid);
if (pTable->tid >= 1) {
taosIdPoolMarkStatus(pVgroup->idPool, pTable->tid);
pVgroup->numOfTables++;
// The create vgroup message may be received later than the create table message
// and the writing order in sdb is therefore uncertain
......@@ -808,8 +808,8 @@ void mnodeAddTableIntoVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
}
void mnodeRemoveTableFromVgroup(SVgObj *pVgroup, SChildTableObj *pTable) {
if (pTable->sid >= 1) {
taosFreeId(pVgroup->idPool, pTable->sid);
if (pTable->tid >= 1) {
taosFreeId(pVgroup->idPool, pTable->tid);
pVgroup->numOfTables--;
// The create vgroup message may be received later than the create table message
// and the writing order in sdb is therefore uncertain
......
src/os/inc/osDef.h
浏览文件 @ d74400f3
......@@ -27,7 +27,7 @@ extern "C" {
#define FD_VALID(x) ((x) > STDERR_FILENO)
#define FD_INITIALIZER ((int32_t)-1)
#define WCHAR wchar_t
// #define WCHAR wchar_t
#define POINTER_SHIFT(p, b) ((void *)((char *)(p) + (b)))
#define POINTER_DISTANCE(p1, p2) ((char *)(p1) - (char *)(p2))
......
src/plugins/http/inc/httpInt.h
浏览文件 @ d74400f3
......@@ -122,9 +122,9 @@ typedef struct {
} HttpDecodeMethod;
typedef struct {
void (*startJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd, void *result);
void (*startJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result);
void (*stopJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd);
bool (*buildQueryJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd, void *result, int numOfRows);
bool (*buildQueryJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result, int numOfRows);
void (*buildAffectRowJsonFp)(struct HttpContext *pContext, HttpSqlCmd *cmd, int affectRows);
void (*initJsonFp)(struct HttpContext *pContext);
void (*cleanJsonFp)(struct HttpContext *pContext);
......@@ -148,7 +148,7 @@ typedef struct HttpContext {
char ipstr[22];
char user[TSDB_USER_LEN]; // parsed from auth token or login message
char pass[TSDB_PASSWORD_LEN];
void * taos;
TAOS * taos;
void * ppContext;
HttpSession *session;
z_stream gzipStream;
......
src/plugins/http/src/httpGcJson.c
浏览文件 @ d74400f3
......@@ -217,7 +217,7 @@ bool gcBuildQueryJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result,
break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR:
httpJsonStringForTransMean(jsonBuf, row[i], fields[i].bytes);
httpJsonStringForTransMean(jsonBuf, (char*)row[i], fields[i].bytes);
break;
case TSDB_DATA_TYPE_TIMESTAMP:
if (precision == TSDB_TIME_PRECISION_MILLI) { //ms
......
src/plugins/http/src/httpRestJson.c
浏览文件 @ d74400f3
......@@ -131,7 +131,7 @@ bool restBuildSqlJson(HttpContext *pContext, HttpSqlCmd *cmd, TAOS_RES *result,
break;
case TSDB_DATA_TYPE_BINARY:
case TSDB_DATA_TYPE_NCHAR:
httpJsonStringForTransMean(jsonBuf, row[i], length[i]);
httpJsonStringForTransMean(jsonBuf, (char*)row[i], length[i]);
break;
case TSDB_DATA_TYPE_TIMESTAMP:
if (timestampFormat == REST_TIMESTAMP_FMT_LOCAL_STRING) {
......@@ -195,4 +195,4 @@ void restStopSqlJson(HttpContext *pContext, HttpSqlCmd *cmd) {
httpJsonToken(jsonBuf, JsonObjEnd);
httpWriteJsonBufEnd(jsonBuf);
}
\ No newline at end of file
}
src/plugins/http/src/httpSql.c
浏览文件 @ d74400f3
......@@ -27,8 +27,6 @@
#include "httpSession.h"
#include "httpQueue.h"
void *taos_connect_a(char *ip, char *user, char *pass, char *db, uint16_t port, void (*fp)(void *, TAOS_RES *, int),
void *param, void **taos);
void httpProcessMultiSql(HttpContext *pContext);
void httpProcessMultiSqlRetrieveCallBack(void *param, TAOS_RES *result, int numOfRows);
......
src/query/inc/qExecutor.h
浏览文件 @ d74400f3
......@@ -213,6 +213,7 @@ typedef struct SQInfo {
void* pBuf; // allocated buffer for STableQueryInfo, sizeof(STableQueryInfo)*numOfTables;
pthread_mutex_t lock; // used to synchronize the rsp/query threads
tsem_t ready;
int32_t dataReady; // denote if query result is ready or not
void* rspContext; // response context
} SQInfo;
......
src/query/inc/qSqlparser.h
浏览文件 @ d74400f3
......@@ -276,8 +276,6 @@ tSQLExpr *tSQLExprNodeClone(tSQLExpr *pExpr);
SAlterTableSQL *tAlterTableSQLElems(SStrToken *pMeterName, tFieldList *pCols, tVariantList *pVals, int32_t type);
tSQLExprListList *tSQLListListAppend(tSQLExprListList *pList, tSQLExprList *pExprList);
void destroyAllSelectClause(SSubclauseInfo *pSql);
void doDestroyQuerySql(SQuerySQL *pSql);
......
src/query/src/qAst.c
浏览文件 @ d74400f3
......@@ -646,9 +646,7 @@ static bool filterItem(tExprNode *pExpr, const void *pItem, SExprTraverseSupp *p
}
// handle the leaf node
assert(pLeft->nodeType == TSQL_NODE_COL && pRight->nodeType == TSQL_NODE_VALUE);
param->setupInfoFn(pExpr, param->pExtInfo);
return param->nodeFilterFn(pItem, pExpr->_node.info);
}
......@@ -769,6 +767,7 @@ void tExprTreeTraverse(tExprNode *pExpr, SSkipList *pSkipList, SArray *result, S
assert(taosArrayGetSize(result) == 0);
tSQLBinaryTraverseOnSkipList(pExpr, result, pSkipList, param);
}
return;
}
......
src/query/src/qExecutor.c
浏览文件 @ d74400f3
......@@ -6227,7 +6227,9 @@ static SQInfo *createQInfoImpl(SQueryTableMsg *pQueryMsg, SSqlGroupbyExpr *pGrou
// NOTE: pTableCheckInfo need to update the query time range and the lastKey info
pQInfo->arrTableIdInfo = taosArrayInit(tableIndex, sizeof(STableIdInfo));
pQInfo->dataReady = QUERY_RESULT_NOT_READY;
pQInfo->rspContext = NULL;
pthread_mutex_init(&pQInfo->lock, NULL);
tsem_init(&pQInfo->ready, 0, 0);
pQuery->pos = -1;
pQuery->window = pQueryMsg->window;
......@@ -6692,12 +6694,14 @@ static bool doBuildResCheck(SQInfo* pQInfo) {
pQInfo->dataReady = QUERY_RESULT_READY;
buildRes = (pQInfo->rspContext != NULL);
pthread_mutex_unlock(&pQInfo->lock);
// clear qhandle owner
// clear qhandle owner, it must be in the secure area. other thread may run ahead before current, after it is
// put into task to be executed.
assert(pQInfo->owner == taosGetPthreadId());
pQInfo->owner = 0;
pthread_mutex_unlock(&pQInfo->lock);
tsem_post(&pQInfo->ready);
return buildRes;
}
......@@ -6761,18 +6765,24 @@ int32_t qRetrieveQueryResultInfo(qinfo_t qinfo, bool* buildRes, void* pRspContex
SQInfo *pQInfo = (SQInfo *)qinfo;
if (pQInfo == NULL || !isValidQInfo(pQInfo)) {
qError("QInfo:%p invalid qhandle", pQInfo);
return TSDB_CODE_QRY_INVALID_QHANDLE;
}
*buildRes = false;
SQuery *pQuery = pQInfo->runtimeEnv.pQuery;
if (IS_QUERY_KILLED(pQInfo)) {
qDebug("QInfo:%p query is killed, code:%d", pQInfo, pQInfo->code);
return pQInfo->code;
}
int32_t code = TSDB_CODE_SUCCESS;
#if 0
SQuery *pQuery = pQInfo->runtimeEnv.pQuery;
pthread_mutex_lock(&pQInfo->lock);
assert(pQInfo->rspContext == NULL);
if (pQInfo->dataReady == QUERY_RESULT_READY) {
*buildRes = true;
qDebug("QInfo:%p retrieve result info, rowsize:%d, rows:%"PRId64", code:%d", pQInfo, pQuery->rowSize, pQuery->rec.rows,
......@@ -6781,10 +6791,17 @@ int32_t qRetrieveQueryResultInfo(qinfo_t qinfo, bool* buildRes, void* pRspContex
*buildRes = false;
qDebug("QInfo:%p retrieve req set query return result after paused", pQInfo);
pQInfo->rspContext = pRspContext;
assert(pQInfo->rspContext != NULL);
}
code = pQInfo->code;
pthread_mutex_unlock(&pQInfo->lock);
#else
tsem_wait(&pQInfo->ready);
*buildRes = true;
code = pQInfo->code;
#endif
return code;
}
......@@ -7101,6 +7118,7 @@ void qCleanupQueryMgmt(void* pQMgmt) {
void** qRegisterQInfo(void* pMgmt, uint64_t qInfo) {
if (pMgmt == NULL) {
terrno = TSDB_CODE_VND_INVALID_VGROUP_ID;
return NULL;
}
......@@ -7109,6 +7127,7 @@ void** qRegisterQInfo(void* pMgmt, uint64_t qInfo) {
SQueryMgmt *pQueryMgmt = pMgmt;
if (pQueryMgmt->qinfoPool == NULL) {
qError("QInfo:%p failed to add qhandle into qMgmt, since qMgmt is closed", (void *)qInfo);
terrno = TSDB_CODE_VND_INVALID_VGROUP_ID;
return NULL;
}
......@@ -7116,6 +7135,7 @@ void** qRegisterQInfo(void* pMgmt, uint64_t qInfo) {
if (pQueryMgmt->closed) {
// pthread_mutex_unlock(&pQueryMgmt->lock);
qError("QInfo:%p failed to add qhandle into cache, since qMgmt is colsing", (void *)qInfo);
terrno = TSDB_CODE_VND_INVALID_VGROUP_ID;
return NULL;
} else {
TSDB_CACHE_PTR_TYPE handleVal = (TSDB_CACHE_PTR_TYPE) qInfo;
......
src/query/src/qParserImpl.c
浏览文件 @ d74400f3
......@@ -130,13 +130,15 @@ tSQLExpr *tSQLExprIdValueCreate(SStrToken *pToken, int32_t optrType) {
tVariantCreate(&pSQLExpr->val, pToken);
pSQLExpr->nSQLOptr = optrType;
} else if (optrType == TK_NOW) {
// default use microsecond
// use microsecond by default
pSQLExpr->val.i64Key = taosGetTimestamp(TSDB_TIME_PRECISION_MICRO);
pSQLExpr->val.nType = TSDB_DATA_TYPE_BIGINT;
pSQLExpr->nSQLOptr = TK_TIMESTAMP; // TK_TIMESTAMP used to denote the time value is in microsecond
} else if (optrType == TK_VARIABLE) {
int32_t ret = parseAbsoluteDuration(pToken->z, pToken->n, &pSQLExpr->val.i64Key);
UNUSED(ret);
if (ret != TSDB_CODE_SUCCESS) {
terrno = TSDB_CODE_TSC_SQL_SYNTAX_ERROR;
}
pSQLExpr->val.nType = TSDB_DATA_TYPE_BIGINT;
pSQLExpr->nSQLOptr = TK_TIMESTAMP;
......@@ -148,6 +150,7 @@ tSQLExpr *tSQLExprIdValueCreate(SStrToken *pToken, int32_t optrType) {
pSQLExpr->nSQLOptr = optrType;
}
return pSQLExpr;
}
......@@ -532,26 +535,6 @@ SQuerySQL *tSetQuerySQLElems(SStrToken *pSelectToken, tSQLExprList *pSelection,
return pQuery;
}
tSQLExprListList *tSQLListListAppend(tSQLExprListList *pList, tSQLExprList *pExprList) {
if (pList == NULL) pList = calloc(1, sizeof(tSQLExprListList));
if (pList->nAlloc <= pList->nList) { //
pList->nAlloc = (pList->nAlloc << 1) + 4;
pList->a = realloc(pList->a, pList->nAlloc * sizeof(pList->a[0]));
if (pList->a == 0) {
pList->nList = pList->nAlloc = 0;
return pList;
}
}
assert(pList->a != 0);
if (pExprList) {
pList->a[pList->nList++] = pExprList;
}
return pList;
}
void doDestroyQuerySql(SQuerySQL *pQuerySql) {
if (pQuerySql == NULL) {
return;
......
src/query/src/qPercentile.c
浏览文件 @ d74400f3
......@@ -365,7 +365,7 @@ void tMemBucketDestroy(tMemBucket *pBucket) {
taosTFree(pBucket);
}
void tMemBucketUpdateBoundingBox(MinMaxEntry *r, char *data, int32_t dataType) {
void tMemBucketUpdateBoundingBox(MinMaxEntry *r, const char *data, int32_t dataType) {
switch (dataType) {
case TSDB_DATA_TYPE_INT: {
int32_t val = *(int32_t *)data;
......
src/query/src/qSyntaxtreefunction.c
浏览文件 @ d74400f3
......@@ -1247,7 +1247,10 @@ _bi_consumer_fn_t tGetBiConsumerFn(int32_t leftType, int32_t rightType, int32_t
case TSDB_BINARY_OP_REMAINDER:
return rem_function_arraylist[leftType][rightType];
default:
assert(0);
return NULL;
}
assert(0);
return NULL;
}
src/rpc/src/rpcMain.c
浏览文件 @ d74400f3
......@@ -412,7 +412,7 @@ void rpcSendResponse(const SRpcMsg *pRsp) {
rpcLockConn(pConn);
if ( pConn->inType == 0 || pConn->user[0] == 0 ) {
tDebug("%s, connection is already released, rsp wont be sent", pConn->info);
tError("%s, connection is already released, rsp wont be sent", pConn->info);
rpcUnlockConn(pConn);
rpcFreeCont(pMsg->pCont);
rpcDecRef(pRpc);
......
src/sync/CMakeLists.txt
浏览文件 @ d74400f3
......@@ -5,14 +5,14 @@ INCLUDE_DIRECTORIES(inc)
AUX_SOURCE_DIRECTORY(src SRC)
IF (TD_LINUX)
LIST(REMOVE_ITEM SRC ./src/tarbitrator.c)
LIST(REMOVE_ITEM SRC src/tarbitrator.c)
ADD_LIBRARY(sync ${SRC})
TARGET_LINK_LIBRARIES(sync tutil pthread common)
LIST(APPEND BIN_SRC ./src/tarbitrator.c)
LIST(APPEND BIN_SRC ./src/taosTcpPool.c)
LIST(APPEND BIN_SRC src/tarbitrator.c)
LIST(APPEND BIN_SRC src/taosTcpPool.c)
ADD_EXECUTABLE(tarbitrator ${BIN_SRC})
TARGET_LINK_LIBRARIES(tarbitrator sync common osdetail tutil)
ADD_SUBDIRECTORY(test)
#ADD_SUBDIRECTORY(test)
ENDIF ()
src/tsdb/src/tsdbMeta.c
浏览文件 @ d74400f3
......@@ -239,7 +239,7 @@ STableCfg *tsdbCreateTableCfgFromMsg(SMDCreateTableMsg *pMsg) {
return NULL;
}
if (tsdbInitTableCfg(pCfg, pMsg->tableType, htobe64(pMsg->uid), htonl(pMsg->sid)) < 0) goto _err;
if (tsdbInitTableCfg(pCfg, pMsg->tableType, htobe64(pMsg->uid), htonl(pMsg->tid)) < 0) goto _err;
if (tdInitTSchemaBuilder(&schemaBuilder, htonl(pMsg->sversion)) < 0) {
terrno = TSDB_CODE_TDB_OUT_OF_MEMORY;
goto _err;
......
src/vnode/src/vnodeMain.c
浏览文件 @ d74400f3
......@@ -56,6 +56,14 @@ int syncGetNodesRole(tsync_h shandle, SNodesRole * cfg) { return 0; }
void syncConfirmForward(tsync_h shandle, uint64_t version, int32_t code) {}
#endif
char* vnodeStatus[] = {
"init",
"ready",
"closing",
"updating",
"reset"
};
int32_t vnodeInitResources() {
int code = syncInit();
if (code != 0) return code;
......@@ -74,6 +82,7 @@ int32_t vnodeInitResources() {
void vnodeCleanupResources() {
if (tsDnodeVnodesHash != NULL) {
vDebug("vnode list is cleanup");
taosHashCleanup(tsDnodeVnodesHash);
tsDnodeVnodesHash = NULL;
}
......@@ -84,9 +93,10 @@ void vnodeCleanupResources() {
int32_t vnodeCreate(SMDCreateVnodeMsg *pVnodeCfg) {
int32_t code;
SVnodeObj *pTemp = (SVnodeObj *)taosHashGet(tsDnodeVnodesHash, (const char *)&pVnodeCfg->cfg.vgId, sizeof(int32_t));
if (pTemp != NULL) {
vInfo("vgId:%d, vnode already exist, pVnode:%p", pVnodeCfg->cfg.vgId, pTemp);
SVnodeObj *pVnode = vnodeAcquire(pVnodeCfg->cfg.vgId);
if (pVnode != NULL) {
vDebug("vgId:%d, vnode already exist, refCount:%d pVnode:%p", pVnodeCfg->cfg.vgId, pVnode->refCount, pVnode);
vnodeRelease(pVnode);
return TSDB_CODE_SUCCESS;
}
......@@ -143,22 +153,24 @@ int32_t vnodeCreate(SMDCreateVnodeMsg *pVnodeCfg) {
return TSDB_CODE_VND_INIT_FAILED;
}
vInfo("vgId:%d, vnode is created, walLevel:%d fsyncPeriod:%d", pVnodeCfg->cfg.vgId, pVnodeCfg->cfg.walLevel, pVnodeCfg->cfg.fsyncPeriod);
vInfo("vgId:%d, vnode dir is created, walLevel:%d fsyncPeriod:%d", pVnodeCfg->cfg.vgId, pVnodeCfg->cfg.walLevel,
pVnodeCfg->cfg.fsyncPeriod);
code = vnodeOpen(pVnodeCfg->cfg.vgId, rootDir);
return code;
}
int32_t vnodeDrop(int32_t vgId) {
SVnodeObj **ppVnode = (SVnodeObj **)taosHashGet(tsDnodeVnodesHash, (const char *)&vgId, sizeof(int32_t));
if (ppVnode == NULL || *ppVnode == NULL) {
vDebug("vgId:%d, failed to drop, vgId not find", vgId);
SVnodeObj *pVnode = vnodeAcquire(vgId);
if (pVnode == NULL) {
vDebug("vgId:%d, failed to drop, vnode not find", vgId);
return TSDB_CODE_VND_INVALID_VGROUP_ID;
}
SVnodeObj *pVnode = *ppVnode;
vTrace("vgId:%d, vnode will be dropped, refCount:%d", pVnode->vgId, pVnode->refCount);
vInfo("vgId:%d, vnode will be dropped, refCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
pVnode->dropped = 1;
vnodeRelease(pVnode);
vnodeCleanUp(pVnode);
return TSDB_CODE_SUCCESS;
......@@ -340,11 +352,11 @@ int32_t vnodeOpen(int32_t vnode, char *rootDir) {
}
int32_t vnodeClose(int32_t vgId) {
SVnodeObj **ppVnode = (SVnodeObj **)taosHashGet(tsDnodeVnodesHash, (const char *)&vgId, sizeof(int32_t));
if (ppVnode == NULL || *ppVnode == NULL) return 0;
SVnodeObj *pVnode = vnodeAcquire(vgId);
if (pVnode == NULL) return 0;
SVnodeObj *pVnode = *ppVnode;
vDebug("vgId:%d, vnode will be closed", pVnode->vgId);
vDebug("vgId:%d, vnode will be closed, pVnode:%p", pVnode->vgId, pVnode);
vnodeRelease(pVnode);
vnodeCleanUp(pVnode);
return 0;
......@@ -355,21 +367,27 @@ void vnodeRelease(void *pVnodeRaw) {
int32_t vgId = pVnode->vgId;
int32_t refCount = atomic_sub_fetch_32(&pVnode->refCount, 1);
vTrace("vgId:%d, release vnode, refCount:%d pVnode:%p", vgId, refCount, pVnode);
assert(refCount >= 0);
if (refCount > 0) {
vDebug("vgId:%d, release vnode, refCount:%d", vgId, refCount);
if (pVnode->status == TAOS_VN_STATUS_RESET && refCount == 2)
if (pVnode->status == TAOS_VN_STATUS_RESET && refCount == 2) {
tsem_post(&pVnode->sem);
}
return;
}
qCleanupQueryMgmt(pVnode->qMgmt);
pVnode->qMgmt = NULL;
vDebug("vgId:%d, vnode will be destroyed, refCount:%d pVnode:%p", vgId, refCount, pVnode);
if (pVnode->tsdb)
if (pVnode->qMgmt) {
qCleanupQueryMgmt(pVnode->qMgmt);
pVnode->qMgmt = NULL;
}
if (pVnode->tsdb) {
tsdbCloseRepo(pVnode->tsdb, 1);
pVnode->tsdb = NULL;
pVnode->tsdb = NULL;
}
// stop continuous query
if (pVnode->cq) {
......@@ -378,18 +396,21 @@ void vnodeRelease(void *pVnodeRaw) {
cqClose(cq);
}
if (pVnode->wal)
if (pVnode->wal) {
walClose(pVnode->wal);
pVnode->wal = NULL;
pVnode->wal = NULL;
}
if (pVnode->wqueue)
if (pVnode->wqueue) {
dnodeFreeVnodeWqueue(pVnode->wqueue);
pVnode->wqueue = NULL;
pVnode->wqueue = NULL;
}
if (pVnode->rqueue)
if (pVnode->rqueue) {
dnodeFreeVnodeRqueue(pVnode->rqueue);
pVnode->rqueue = NULL;
pVnode->rqueue = NULL;
}
taosTFree(pVnode->rootDir);
if (pVnode->dropped) {
......@@ -413,31 +434,41 @@ void vnodeRelease(void *pVnodeRaw) {
free(pVnode);
int32_t count = taosHashGetSize(tsDnodeVnodesHash);
vDebug("vgId:%d, vnode is released, vnodes:%d", vgId, count);
vDebug("vgId:%d, vnode is destroyed, vnodes:%d", vgId, count);
}
static void vnodeIncRef(void *ptNode) {
assert(ptNode != NULL);
SVnodeObj **ppVnode = (SVnodeObj **)ptNode;
assert(ppVnode);
assert(*ppVnode);
SVnodeObj *pVnode = *ppVnode;
atomic_add_fetch_32(&pVnode->refCount, 1);
vTrace("vgId:%d, get vnode, refCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
}
void *vnodeAcquire(int32_t vgId) {
SVnodeObj **ppVnode = (SVnodeObj **)taosHashGet(tsDnodeVnodesHash, (const char *)&vgId, sizeof(int32_t));
SVnodeObj **ppVnode = taosHashGetCB(tsDnodeVnodesHash, &vgId, sizeof(int32_t), vnodeIncRef, NULL, sizeof(void *));
if (ppVnode == NULL || *ppVnode == NULL) {
terrno = TSDB_CODE_VND_INVALID_VGROUP_ID;
vInfo("vgId:%d, not exist", vgId);
vDebug("vgId:%d, not exist", vgId);
return NULL;
}
SVnodeObj *pVnode = *ppVnode;
atomic_add_fetch_32(&pVnode->refCount, 1);
vDebug("vgId:%d, get vnode, refCount:%d", pVnode->vgId, pVnode->refCount);
return pVnode;
return *ppVnode;
}
void *vnodeAcquireRqueue(int32_t vgId) {
SVnodeObj *pVnode = vnodeAcquire(vgId);
if (pVnode == NULL) return NULL;
if (pVnode->status == TAOS_VN_STATUS_RESET) {
terrno = TSDB_CODE_APP_NOT_READY;
vInfo("vgId:%d, status is in reset", vgId);
int32_t code = vnodeCheckRead(pVnode);
if (code != TSDB_CODE_SUCCESS) {
terrno = code;
vInfo("vgId:%d, can not provide read service, status is %s", vgId, vnodeStatus[pVnode->status]);
vnodeRelease(pVnode);
return NULL;
}
......@@ -449,13 +480,14 @@ void *vnodeAcquireWqueue(int32_t vgId) {
SVnodeObj *pVnode = vnodeAcquire(vgId);
if (pVnode == NULL) return NULL;
if (pVnode->status == TAOS_VN_STATUS_RESET) {
terrno = TSDB_CODE_APP_NOT_READY;
vInfo("vgId:%d, status is in reset", vgId);
int32_t code = vnodeCheckWrite(pVnode);
if (code != TSDB_CODE_SUCCESS) {
terrno = code;
vInfo("vgId:%d, can not provide write service, status is %s", vgId, vnodeStatus[pVnode->status]);
vnodeRelease(pVnode);
return NULL;
}
return pVnode->wqueue;
}
......@@ -528,7 +560,7 @@ void vnodeSetAccess(SDMVgroupAccess *pAccess, int32_t numOfVnodes) {
if (pVnode != NULL) {
pVnode->accessState = pAccess[i].accessState;
if (pVnode->accessState != TSDB_VN_ALL_ACCCESS) {
vDebug("vgId:%d, access state is set to %d", pAccess[i].vgId, pVnode->accessState)
vDebug("vgId:%d, access state is set to %d", pAccess[i].vgId, pVnode->accessState);
}
vnodeRelease(pVnode);
}
......@@ -538,11 +570,12 @@ void vnodeSetAccess(SDMVgroupAccess *pAccess, int32_t numOfVnodes) {
static void vnodeCleanUp(SVnodeObj *pVnode) {
// remove from hash, so new messages wont be consumed
taosHashRemove(tsDnodeVnodesHash, (const char *)&pVnode->vgId, sizeof(int32_t));
int i = 0;
if (pVnode->status != TAOS_VN_STATUS_INIT) {
// it may be in updateing or reset state, then it shall wait
while (atomic_val_compare_exchange_8(&pVnode->status, TAOS_VN_STATUS_READY, TAOS_VN_STATUS_CLOSING) != TAOS_VN_STATUS_READY) {
int i = 0;
while (atomic_val_compare_exchange_8(&pVnode->status, TAOS_VN_STATUS_READY, TAOS_VN_STATUS_CLOSING) !=
TAOS_VN_STATUS_READY) {
if (++i % 1000 == 0) {
sched_yield();
}
......@@ -556,7 +589,7 @@ static void vnodeCleanUp(SVnodeObj *pVnode) {
syncStop(sync);
}
vTrace("vgId:%d, vnode will cleanup, refCount:%d", pVnode->vgId, pVnode->refCount);
vDebug("vgId:%d, vnode will cleanup, refCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
// release local resources only after cutting off outside connections
qQueryMgmtNotifyClosed(pVnode->qMgmt);
......@@ -613,17 +646,19 @@ static int vnodeResetTsdb(SVnodeObj *pVnode)
char rootDir[128] = "\0";
sprintf(rootDir, "%s/tsdb", pVnode->rootDir);
if (atomic_val_compare_exchange_8(&pVnode->status, TAOS_VN_STATUS_READY, TAOS_VN_STATUS_RESET) != TAOS_VN_STATUS_READY)
if (atomic_val_compare_exchange_8(&pVnode->status, TAOS_VN_STATUS_READY, TAOS_VN_STATUS_RESET) != TAOS_VN_STATUS_READY) {
return -1;
}
void *tsdb = pVnode->tsdb;
pVnode->tsdb = NULL;
// acquire vnode
int32_t refCount = atomic_add_fetch_32(&pVnode->refCount, 1);
int32_t refCount = atomic_add_fetch_32(&pVnode->refCount, 1);
if (refCount > 2)
if (refCount > 2) {
tsem_wait(&pVnode->sem);
}
// close tsdb, then open tsdb
tsdbCloseRepo(tsdb, 0);
......
src/vnode/src/vnodeRead.c
浏览文件 @ d74400f3
......@@ -14,7 +14,8 @@
*/
#define _DEFAULT_SOURCE
//#include <dnode.h>
#define _NON_BLOCKING_RETRIEVE 0
#include "os.h"
#include "tglobal.h"
......@@ -38,6 +39,11 @@ void vnodeInitReadFp(void) {
vnodeProcessReadMsgFp[TSDB_MSG_TYPE_FETCH] = vnodeProcessFetchMsg;
}
//
// After the fetch request enters the vnode queue, if the vnode cannot provide services, the process function are
// still required, or there will be a deadlock, so we don’t do any check here, but put the check codes before the
// request enters the queue
//
int32_t vnodeProcessRead(void *param, SReadMsg *pReadMsg) {
SVnodeObj *pVnode = (SVnodeObj *)param;
int msgType = pReadMsg->rpcMsg.msgType;
......@@ -47,48 +53,72 @@ int32_t vnodeProcessRead(void *param, SReadMsg *pReadMsg) {
return TSDB_CODE_VND_MSG_NOT_PROCESSED;
}
return (*vnodeProcessReadMsgFp[msgType])(pVnode, pReadMsg);
}
int32_t vnodeCheckRead(void *param) {
SVnodeObj *pVnode = param;
if (pVnode->status != TAOS_VN_STATUS_READY) {
vDebug("vgId:%d, msgType:%s not processed, vnode status is %d", pVnode->vgId, taosMsg[msgType], pVnode->status);
vDebug("vgId:%d, vnode status is %s, recCount:%d pVnode:%p", pVnode->vgId, vnodeStatus[pVnode->status],
pVnode->refCount, pVnode);
return TSDB_CODE_APP_NOT_READY;
}
// tsdb may be in reset state
if (pVnode->tsdb == NULL) return TSDB_CODE_APP_NOT_READY;
if (pVnode->status == TAOS_VN_STATUS_CLOSING) return TSDB_CODE_APP_NOT_READY;
if (pVnode->tsdb == NULL) {
vDebug("vgId:%d, tsdb is null, recCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
return TSDB_CODE_APP_NOT_READY;
}
// TODO: Later, let slave to support query
// if (pVnode->syncCfg.replica > 1 && pVnode->role != TAOS_SYNC_ROLE_MASTER) {
if (pVnode->role != TAOS_SYNC_ROLE_SLAVE && pVnode->role != TAOS_SYNC_ROLE_MASTER) {
vDebug("vgId:%d, msgType:%s not processed, replica:%d role:%s", pVnode->vgId, taosMsg[msgType],
pVnode->syncCfg.replica, syncRole[pVnode->role]);
vDebug("vgId:%d, replica:%d role:%s, recCount:%d pVnode:%p", pVnode->vgId, pVnode->syncCfg.replica,
syncRole[pVnode->role], pVnode->refCount, pVnode);
return TSDB_CODE_APP_NOT_READY;
}
return (*vnodeProcessReadMsgFp[msgType])(pVnode, pReadMsg);
return TSDB_CODE_SUCCESS;
}
static int32_t vnodePutItemIntoReadQueue(SVnodeObj *pVnode, void **qhandle, void *ahandle) {
int32_t code = vnodeCheckRead(pVnode);
if (code != TSDB_CODE_SUCCESS) return code;
static void vnodePutItemIntoReadQueue(SVnodeObj *pVnode, void **qhandle) {
SReadMsg *pRead = (SReadMsg *)taosAllocateQitem(sizeof(SReadMsg));
pRead->rpcMsg.msgType = TSDB_MSG_TYPE_QUERY;
pRead->pCont = qhandle;
pRead->contLen = 0;
pRead->rpcMsg.handle = NULL;
pRead->rpcMsg.ahandle = ahandle;
atomic_add_fetch_32(&pVnode->refCount, 1);
vDebug("QInfo:%p add to vread queue for exec query, msg:%p", *qhandle, pRead);
taosWriteQitem(pVnode->rqueue, TAOS_QTYPE_QUERY, pRead);
return TSDB_CODE_SUCCESS;
}
static int32_t vnodeDumpQueryResult(SRspRet *pRet, void *pVnode, void **handle, bool *freeHandle) {
/**
*
* @param pRet response message object
* @param pVnode the vnode object
* @param handle qhandle for executing query
* @param freeHandle free qhandle or not
* @param ahandle sqlObj address at client side
* @return
*/
static int32_t vnodeDumpQueryResult(SRspRet *pRet, void *pVnode, void **handle, bool *freeHandle, void *ahandle) {
bool continueExec = false;
int32_t code = TSDB_CODE_SUCCESS;
if ((code = qDumpRetrieveResult(*handle, (SRetrieveTableRsp **)&pRet->rsp, &pRet->len, &continueExec)) == TSDB_CODE_SUCCESS) {
if (continueExec) {
*freeHandle = false;
vnodePutItemIntoReadQueue(pVnode, handle);
pRet->qhandle = *handle;
code = vnodePutItemIntoReadQueue(pVnode, handle, ahandle);
if (code != TSDB_CODE_SUCCESS) {
*freeHandle = true;
return code;
} else {
pRet->qhandle = *handle;
}
} else {
*freeHandle = true;
vDebug("QInfo:%p exec completed, free handle:%d", *handle, *freeHandle);
......@@ -165,11 +195,13 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
// current connect is broken
if (code == TSDB_CODE_SUCCESS) {
handle = qRegisterQInfo(pVnode->qMgmt, (uint64_t)pQInfo);
if (handle == NULL) { // failed to register qhandle, todo add error test case
if (handle == NULL) { // failed to register qhandle
pRsp->code = terrno;
terrno = 0;
vError("vgId:%d QInfo:%p register qhandle failed, return to app, code:%s", pVnode->vgId, (void *)pQInfo,
tstrerror(pRsp->code));
pRsp->code = TSDB_CODE_QRY_INVALID_QHANDLE;
qDestroyQueryInfo(pQInfo); // destroy it directly
return pRsp->code;
} else {
assert(*handle == pQInfo);
pRsp->qhandle = htobe64((uint64_t)pQInfo);
......@@ -189,7 +221,12 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
if (handle != NULL) {
vDebug("vgId:%d, QInfo:%p, dnode query msg disposed, create qhandle and returns to app", vgId, *handle);
vnodePutItemIntoReadQueue(pVnode, handle);
code = vnodePutItemIntoReadQueue(pVnode, handle, pReadMsg->rpcMsg.ahandle);
if (code != TSDB_CODE_SUCCESS) {
pRsp->code = code;
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, true);
return pRsp->code;
}
}
} else {
assert(pCont != NULL);
......@@ -197,6 +234,8 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
vDebug("vgId:%d, QInfo:%p, dnode continues to exec query", pVnode->vgId, *qhandle);
#if _NON_BLOCKING_RETRIEVE
bool freehandle = false;
bool buildRes = qTableQuery(*qhandle); // do execute query
......@@ -210,11 +249,14 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
pReadMsg->rpcMsg.handle);
// set the real rsp error code
pReadMsg->rpcMsg.code = vnodeDumpQueryResult(&pReadMsg->rspRet, pVnode, qhandle, &freehandle);
pReadMsg->rpcMsg.code = vnodeDumpQueryResult(&pReadMsg->rspRet, pVnode, qhandle, &freehandle, pReadMsg->rpcMsg.ahandle);
// NOTE: set return code to be TSDB_CODE_QRY_HAS_RSP to notify dnode to return msg to client
code = TSDB_CODE_QRY_HAS_RSP;
} else {
void* h1 = qGetResultRetrieveMsg(*qhandle);
assert(h1 == NULL);
freehandle = qQueryCompleted(*qhandle);
}
......@@ -223,6 +265,10 @@ static int32_t vnodeProcessQueryMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
if (freehandle || (!buildRes)) {
qReleaseQInfo(pVnode->qMgmt, (void **)&qhandle, freehandle);
}
#else
qTableQuery(*qhandle); // do execute query
qReleaseQInfo(pVnode->qMgmt, (void **)&qhandle, false);
#endif
}
return code;
......@@ -282,12 +328,18 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
memset(pRet->rsp, 0, sizeof(SRetrieveTableRsp));
freeHandle = true;
} else { // result is not ready, return immediately
assert(buildRes == true);
#if _NON_BLOCKING_RETRIEVE
if (!buildRes) {
assert(pReadMsg->rpcMsg.handle != NULL);
qReleaseQInfo(pVnode->qMgmt, (void **)&handle, false);
return TSDB_CODE_QRY_NOT_READY;
}
#endif
code = vnodeDumpQueryResult(pRet, pVnode, handle, &freeHandle);
// ahandle is the sqlObj pointer
code = vnodeDumpQueryResult(pRet, pVnode, handle, &freeHandle, pReadMsg->rpcMsg.ahandle);
}
// If qhandle is not added into vread queue, the query should be completed already or paused with error.
......
src/vnode/src/vnodeWrite.c
浏览文件 @ d74400f3
......@@ -56,15 +56,6 @@ int32_t vnodeProcessWrite(void *param1, int qtype, void *param2, void *item) {
return TSDB_CODE_VND_MSG_NOT_PROCESSED;
}
if (!(pVnode->accessState & TSDB_VN_WRITE_ACCCESS)) {
vDebug("vgId:%d, msgType:%s not processed, no write auth", pVnode->vgId, taosMsg[pHead->msgType]);
return TSDB_CODE_VND_NO_WRITE_AUTH;
}
// tsdb may be in reset state
if (pVnode->tsdb == NULL) return TSDB_CODE_APP_NOT_READY;
if (pVnode->status == TAOS_VN_STATUS_CLOSING) return TSDB_CODE_APP_NOT_READY;
if (pHead->version == 0) { // from client or CQ
if (pVnode->status != TAOS_VN_STATUS_READY) {
vDebug("vgId:%d, msgType:%s not processed, vnode status is %d", pVnode->vgId, taosMsg[pHead->msgType],
......@@ -105,6 +96,28 @@ int32_t vnodeProcessWrite(void *param1, int qtype, void *param2, void *item) {
return syncCode;
}
int32_t vnodeCheckWrite(void *param) {
SVnodeObj *pVnode = param;
if (!(pVnode->accessState & TSDB_VN_WRITE_ACCCESS)) {
vDebug("vgId:%d, no write auth, recCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
return TSDB_CODE_VND_NO_WRITE_AUTH;
}
// tsdb may be in reset state
if (pVnode->tsdb == NULL) {
vDebug("vgId:%d, tsdb is null, recCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
return TSDB_CODE_APP_NOT_READY;
}
if (pVnode->status == TAOS_VN_STATUS_CLOSING) {
vDebug("vgId:%d, vnode status is %s, recCount:%d pVnode:%p", pVnode->vgId, vnodeStatus[pVnode->status],
pVnode->refCount, pVnode);
return TSDB_CODE_APP_NOT_READY;
}
return TSDB_CODE_SUCCESS;
}
void vnodeConfirmForward(void *param, uint64_t version, int32_t code) {
SVnodeObj *pVnode = (SVnodeObj *)param;
syncConfirmForward(pVnode->sync, version, code);
......@@ -151,7 +164,7 @@ static int32_t vnodeProcessDropTableMsg(SVnodeObj *pVnode, void *pCont, SRspRet
int32_t code = TSDB_CODE_SUCCESS;
vDebug("vgId:%d, table:%s, start to drop", pVnode->vgId, pTable->tableId);
STableId tableId = {.uid = htobe64(pTable->uid), .tid = htonl(pTable->sid)};
STableId tableId = {.uid = htobe64(pTable->uid), .tid = htonl(pTable->tid)};
if (tsdbDropTable(pVnode->tsdb, tableId) < 0) code = terrno;
......@@ -202,7 +215,7 @@ int vnodeWriteCqMsgToQueue(void *param, void *data, int type) {
memcpy(pWal, pHead, size);
atomic_add_fetch_32(&pVnode->refCount, 1);
vDebug("CQ: vgId:%d, get vnode wqueue, refCount:%d", pVnode->vgId, pVnode->refCount);
vTrace("CQ: vgId:%d, get vnode wqueue, refCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
taosWriteQitem(pVnode->wqueue, type, pSync);
......@@ -219,7 +232,7 @@ int vnodeWriteToQueue(void *param, void *data, int type) {
memcpy(pWal, pHead, size);
atomic_add_fetch_32(&pVnode->refCount, 1);
vDebug("vgId:%d, get vnode wqueue, refCount:%d", pVnode->vgId, pVnode->refCount);
vTrace("vgId:%d, get vnode wqueue, refCount:%d pVnode:%p", pVnode->vgId, pVnode->refCount, pVnode);
taosWriteQitem(pVnode->wqueue, type, pWal);
......
tests/comparisonTest/tdengine/tdengineTest.c
浏览文件 @ d74400f3
......@@ -108,9 +108,9 @@ void parseArg(int argc, char *argv[]) {
}
}
void taos_error(TAOS *con) {
printf("TDengine error: %s\n", taos_errstr(con));
taos_close(con);
static void taos_error(TAOS_RES *tres, TAOS *conn) {
printf("TDengine error: %s\n", tres?taos_errstr(tres):"null result");
taos_close(conn);
exit(1);
}
......@@ -125,13 +125,17 @@ void writeDataImp(void *param) {
printf("Thread %d, writing sID %d, eID %d\n", pThread->threadId, pThread->sID, pThread->eID);
void *taos = taos_connect("127.0.0.1", "root", "taosdata", NULL, 0);
if (taos == NULL)
taos_error(taos);
if (taos == NULL) {
// where to find errstr?
// taos_error(NULL, taos);
printf("TDengine error: %s\n", "failed to connect");
exit(1);
}
TAOS_RES* result = taos_query(taos, "use db");
int32_t code = taos_errno(result);
if (code != 0) {
taos_error(taos);
taos_error(result, taos);
}
taos_free_result(result);
......@@ -227,12 +231,17 @@ void writeData() {
taos_init();
void *taos = taos_connect("127.0.0.1", "root", "taosdata", NULL, 0);
if (taos == NULL) taos_error(taos);
if (taos == NULL) {
// where to find errstr?
// taos_error(NULL, taos);
printf("TDengine error: %s\n", "failed to connect");
exit(1);
}
TAOS_RES *result = taos_query(taos, "create database if not exists db");
int32_t code = taos_errno(result);
if (code != 0) {
taos_error(taos);
taos_error(result, taos);
}
taos_free_result(result);
......@@ -241,7 +250,7 @@ void writeData() {
"tags(devid int, devname binary(16), devgroup int)");
code = taos_errno(result);
if (code != 0) {
taos_error(taos);
taos_error(result, taos);
}
taos_free_result(result);
......@@ -293,8 +302,12 @@ void readDataImp(void *param)
printf("open file %s success\n", arguments.sql);
void *taos = taos_connect("127.0.0.1", "root", "taosdata", NULL, 0);
if (taos == NULL)
taos_error(taos);
if (taos == NULL) {
// where to find errstr?
// taos_error(NULL, taos);
printf("TDengine error: %s\n", "failed to connect");
exit(1);
}
char *line = NULL;
size_t len = 0;
......@@ -313,7 +326,7 @@ void readDataImp(void *param)
TAOS_RES *result = taos_query(taos, line);
int32_t code = taos_errno(result);
if (code != 0) {
taos_error(taos);
taos_error(result, taos);
}
TAOS_ROW row;
......@@ -343,8 +356,12 @@ void readData() {
printf("---- clients: %d\n", arguments.clients);
void *taos = taos_connect("127.0.0.1", "root", "taosdata", NULL, 0);
if (taos == NULL)
taos_error(taos);
if (taos == NULL) {
// where to find errstr?
// taos_error(NULL, taos);
printf("TDengine error: %s\n", "failed to connect");
exit(1);
}
ThreadObj *threads = calloc((size_t)arguments.clients, sizeof(ThreadObj));
......
tests/examples/JDBC/JDBCDemo/pom.xml
浏览文件 @ d74400f3
......@@ -5,18 +5,11 @@
<modelVersion>4.0.0</modelVersion>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>jdbcdemo</artifactId>
<version>1.0-SNAPSHOT</version>
<artifactId>jdbcChecker</artifactId>
<version>SNAPSHOT</version>
<packaging>jar</packaging>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-plugins</artifactId>
<version>30</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
......@@ -30,7 +23,7 @@
<configuration>
<archive>
<manifest>
<mainClass>TSDBSyncSample</mainClass>
<mainClass>com.taosdata.example.JdbcChecker</mainClass>
</manifest>
</archive>
<descriptorRefs>
......@@ -63,12 +56,18 @@
<dependency>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.4</version>
<version>2.0.8</version>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.13.1</version>
<scope>test</scope>
</dependency>
</dependencies>
</project>
tests/examples/JDBC/JDBCDemo/readme.md
浏览文件 @ d74400f3
......@@ -2,12 +2,14 @@
TDengine's JDBC demo project is organized in a Maven way so that users can easily compile, package and run the project. If you don't have Maven on your server, you may install it using
<pre>sudo apt-get install maven</pre>
## Compile and Install JDBC Driver
TDengine's JDBC driver jar is not yet published to maven center repo, so we need to manually compile it and install it to the local Maven repository. This can be easily done with Maven. Go to source directory of the JDBC driver ``TDengine/src/connector/jdbc`` and execute
<pre>mvn clean package install</pre>
## Install TDengine Client
Make sure you have already installed a tdengine client on your current develop environment.
Download the tdengine package on our website: ``https://www.taosdata.com/cn/all-downloads/`` and install the client.
## Compile the Demo Code and Run It
To compile the demo project, go to the source directory ``TDengine/tests/examples/JDBC/JDBCDemo`` and execute
<pre>mvn clean assembly:single package</pre>
The ``pom.xml`` is configured to package all the dependencies into one executable jar file. To run it, go to ``examples/JDBC/JDBCDemo/target`` and execute
<pre>java -jar jdbcdemo-1.0-SNAPSHOT-jar-with-dependencies.jar</pre>
<pre>mvn clean package assembly:single</pre>
The ``pom.xml`` is configured to package all the dependencies into one executable jar file.
To run it, go to ``examples/JDBC/JDBCDemo/target`` and execute
<pre>java -jar jdbcChecker-SNAPSHOT-jar-with-dependencies.jar -host localhost</pre>
\ No newline at end of file
tests/examples/JDBC/JDBCDemo/src/main/java/TSDBSyncSample.java 已删除 100644 → 0
浏览文件 @ 22ec4f4a
import java.sql.*;
public class TSDBSyncSample {
private static final String JDBC_PROTOCAL = "jdbc:TAOS://";
private static final String TSDB_DRIVER = "com.taosdata.jdbc.TSDBDriver";
private String host = "127.0.0.1";
private String user = "root";
private String password = "taosdata";
private int port = 0;
private String jdbcUrl = "";
private String databaseName = "db";
private String metricsName = "mt";
private String tablePrefix = "t";
private int tablesCount = 1;
private int loopCount = 2;
private int batchSize = 10;
private long beginTimestamp = 1519833600000L;
private long rowsInserted = 0;
static {
try {
Class.forName(TSDB_DRIVER);
} catch (Exception e) {
e.printStackTrace();
}
}
/**
* @param args
*/
public static void main(String[] args) {
TSDBSyncSample tester = new TSDBSyncSample();
tester.doReadArgument(args);
System.out.println("---------------------------------------------------------------");
System.out.println("Start testing...");
System.out.println("---------------------------------------------------------------");
tester.doMakeJdbcUrl();
tester.doCreateDbAndTable();
tester.doExecuteInsert();
tester.doExecuteQuery();
System.out.println("\n---------------------------------------------------------------");
System.out.println("Stop testing...");
System.out.println("---------------------------------------------------------------");
}
private void doReadArgument(String[] args) {
System.out.println("Arguments format: host tables loop batchs");
if (args.length >= 1) {
this.host = args[0];
}
if (args.length >= 2) {
this.tablesCount = Integer.parseInt(args[1]);
}
if (args.length >= 3) {
this.loopCount = Integer.parseInt(args[2]);
}
if (args.length >= 4) {
this.batchSize = Integer.parseInt(args[3]);
}
}
private void doMakeJdbcUrl() {
// jdbc:TSDB://127.0.0.1:0/dbname?user=root&password=taosdata
System.out.println("\nJDBC URL to use:");
this.jdbcUrl = String.format("%s%s:%d/%s?user=%s&password=%s", JDBC_PROTOCAL, this.host, this.port, "",
this.user, this.password);
System.out.println(this.jdbcUrl);
}
private void doCreateDbAndTable() {
System.out.println("\n---------------------------------------------------------------");
System.out.println("Start creating databases and tables...");
String sql = "";
try (Connection conn = DriverManager.getConnection(jdbcUrl);
Statement stmt = conn.createStatement()){
sql = "create database if not exists " + this.databaseName;
stmt.executeUpdate(sql);
System.out.printf("Successfully executed: %s\n", sql);
sql = "use " + this.databaseName;
stmt.executeUpdate(sql);
System.out.printf("Successfully executed: %s\n", sql);
sql = "create table if not exists " + this.metricsName + " (ts timestamp, v1 int) tags(t1 int)";
stmt.executeUpdate(sql);
System.out.printf("Successfully executed: %s\n", sql);
for (int i = 0; i < this.tablesCount; i++) {
sql = String.format("create table if not exists %s%d using %s tags(%d)", this.tablePrefix, i,
this.metricsName, i);
stmt.executeUpdate(sql);
System.out.printf("Successfully executed: %s\n", sql);
}
} catch (SQLException e) {
e.printStackTrace();
System.out.printf("Failed to execute SQL: %s\n", sql);
System.exit(4);
} catch (Exception e) {
e.printStackTrace();
System.exit(4);
}
System.out.println("Successfully created databases and tables");
}
public void doExecuteInsert() {
System.out.println("\n---------------------------------------------------------------");
System.out.println("Start inserting data...");
int start = (int) System.currentTimeMillis();
StringBuilder sql = new StringBuilder("");
try (Connection conn = DriverManager.getConnection(jdbcUrl);
Statement stmt = conn.createStatement()){
stmt.executeUpdate("use " + databaseName);
for (int loop = 0; loop < this.loopCount; loop++) {
for (int table = 0; table < this.tablesCount; ++table) {
sql = new StringBuilder("insert into ");
sql.append(this.tablePrefix).append(table).append(" values");
for (int batch = 0; batch < this.batchSize; ++batch) {
int rows = loop * this.batchSize + batch;
sql.append("(").append(this.beginTimestamp + rows).append(",").append(rows).append(")");
}
int affectRows = stmt.executeUpdate(sql.toString());
this.rowsInserted += affectRows;
}
}
} catch (SQLException e) {
e.printStackTrace();
System.out.printf("Failed to execute SQL: %s\n", sql.toString());
System.exit(4);
} catch (Exception e) {
e.printStackTrace();
System.exit(4);
}
int end = (int) System.currentTimeMillis();
System.out.println("Inserting completed!");
System.out.printf("Total %d rows inserted, %d rows failed, time spend %d seconds.\n", this.rowsInserted,
this.loopCount * this.batchSize - this.rowsInserted, (end - start) / 1000);
}
public void doExecuteQuery() {
System.out.println("\n---------------------------------------------------------------");
System.out.println("Starting querying data...");
ResultSet resSet = null;
StringBuilder sql = new StringBuilder("");
StringBuilder resRow = new StringBuilder("");
try (Connection conn = DriverManager.getConnection(jdbcUrl);
Statement stmt = conn.createStatement()){
stmt.executeUpdate("use " + databaseName);
for (int i = 0; i < this.tablesCount; ++i) {
sql = new StringBuilder("select * from ").append(this.tablePrefix).append(i);
resSet = stmt.executeQuery(sql.toString());
if (resSet == null) {
System.out.println(sql + " failed");
System.exit(4);
}
ResultSetMetaData metaData = resSet.getMetaData();
System.out.println("Retrieve metadata of " + tablePrefix + i);
for (int column = 1; column <= metaData.getColumnCount(); ++column) {
System.out.printf("Column%d: name = %s, type = %d, type name = %s, display size = %d\n", column, metaData.getColumnName(column), metaData.getColumnType(column),
metaData.getColumnTypeName(column), metaData.getColumnDisplaySize(column));
}
int rows = 0;
System.out.println("Retrieve data of " + tablePrefix + i);
while (resSet.next()) {
resRow = new StringBuilder();
for (int col = 1; col <= metaData.getColumnCount(); col++) {
resRow.append(metaData.getColumnName(col)).append("=").append(resSet.getObject(col))
.append(" ");
}
System.out.println(resRow.toString());
rows++;
}
try {
if (resSet != null)
resSet.close();
} catch (SQLException e) {
e.printStackTrace();
}
System.out.printf("Successfully executed query: %s;\nTotal rows returned: %d\n", sql.toString(), rows);
}
} catch (SQLException e) {
e.printStackTrace();
System.out.printf("Failed to execute query: %s\n", sql.toString());
System.exit(4);
} catch (Exception e) {
e.printStackTrace();
System.exit(4);
}
System.out.println("Query completed!");
}
}
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JDBCConnectorChecker.java → tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/JdbcChecker.java
浏览文件 @ d74400f3
......@@ -5,7 +5,7 @@ import com.taosdata.jdbc.TSDBDriver;
import java.sql.*;
import java.util.Properties;
public class JDBCConnectorChecker {
public class JdbcChecker {
private static String host;
private static String dbName = "test";
private static String tbName = "weather";
......@@ -158,7 +158,7 @@ public class JDBCConnectorChecker {
return;
}
JDBCConnectorChecker checker = new JDBCConnectorChecker();
JdbcChecker checker = new JdbcChecker();
checker.init();
checker.createDatabase();
checker.useDatabase();
......
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/jdbcTaosdemo/JdbcTaosdemo.java
浏览文件 @ d74400f3
......@@ -25,6 +25,7 @@ public class JdbcTaosdemo {
}
public static void main(String[] args) {
// parse config from args
JdbcTaosdemoConfig config = new JdbcTaosdemoConfig(args);
boolean isHelp = Arrays.asList(args).contains("--help");
......@@ -38,27 +39,51 @@ public class JdbcTaosdemo {
}
JdbcTaosdemo taosdemo = new JdbcTaosdemo(config);
// establish connection
taosdemo.init();
// drop database
taosdemo.dropDatabase();
// create database
taosdemo.createDatabase();
// use db
taosdemo.useDatabase();
// create super table
taosdemo.createSuperTable();
// create sub tables
taosdemo.createTableMultiThreads();
boolean infinite = Arrays.asList(args).contains("--infinite");
if (infinite) {
logger.info("!!! Infinite Insert Mode Started. !!!!");
logger.info("!!! Infinite Insert Mode Started. !!!");
taosdemo.insertInfinite();
} else {
// insert into table
taosdemo.insertMultiThreads();
// single table select
// select from sub table
taosdemo.selectFromTableLimit();
taosdemo.selectCountFromTable();
taosdemo.selectAvgMinMaxFromTable();
// super table select
// select last from
taosdemo.selectLastFromTable();
// select from super table
taosdemo.selectFromSuperTableLimit();
taosdemo.selectCountFromSuperTable();
taosdemo.selectAvgMinMaxFromSuperTable();
//select avg ,max from stb where tag
taosdemo.selectAvgMinMaxFromSuperTableWhereTag();
//select last from stb where location = ''
taosdemo.selectLastFromSuperTableWhere();
// select group by
taosdemo.selectGroupBy();
// select like
taosdemo.selectLike();
// select where ts >= ts<=
taosdemo.selectLastOneHour();
taosdemo.selectLastOneDay();
taosdemo.selectLastOneWeek();
taosdemo.selectLastOneMonth();
taosdemo.selectLastOneYear();
// drop super table
if (config.isDeleteTable())
taosdemo.dropSuperTable();
......@@ -196,6 +221,11 @@ public class JdbcTaosdemo {
executeQuery(sql);
}
private void selectLastFromTable() {
String sql = SqlSpeller.selectLastFromTableSQL(config.getDbName(), config.getTbPrefix(), 1);
executeQuery(sql);
}
private void selectFromSuperTableLimit() {
String sql = SqlSpeller.selectFromSuperTableLimitSQL(config.getDbName(), config.getStbName(), 10, 0);
executeQuery(sql);
......@@ -211,6 +241,52 @@ public class JdbcTaosdemo {
executeQuery(sql);
}
private void selectAvgMinMaxFromSuperTableWhereTag() {
String sql = SqlSpeller.selectAvgMinMaxFromSuperTableWhere("current", config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastFromSuperTableWhere() {
String sql = SqlSpeller.selectLastFromSuperTableWhere("current", config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectGroupBy() {
String sql = SqlSpeller.selectGroupBy("current", config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLike() {
String sql = SqlSpeller.selectLike(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastOneHour() {
String sql = SqlSpeller.selectLastOneHour(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastOneDay() {
String sql = SqlSpeller.selectLastOneDay(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastOneWeek() {
String sql = SqlSpeller.selectLastOneWeek(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastOneMonth() {
String sql = SqlSpeller.selectLastOneMonth(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void selectLastOneYear() {
String sql = SqlSpeller.selectLastOneYear(config.getDbName(), config.getStbName());
executeQuery(sql);
}
private void close() {
try {
if (connection != null) {
......@@ -241,6 +317,7 @@ public class JdbcTaosdemo {
long end = System.currentTimeMillis();
printSql(sql, execute, (end - start));
} catch (SQLException e) {
logger.error("ERROR execute SQL ===> " + sql);
logger.error(e.getMessage());
e.printStackTrace();
}
......@@ -258,6 +335,7 @@ public class JdbcTaosdemo {
printSql(sql, true, (end - start));
printResult(resultSet);
} catch (SQLException e) {
logger.error("ERROR execute SQL ===> " + sql);
logger.error(e.getMessage());
e.printStackTrace();
}
......
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/jdbcTaosdemo/domain/JdbcTaosdemoConfig.java
浏览文件 @ d74400f3
......@@ -14,9 +14,9 @@ public final class JdbcTaosdemoConfig {
//Destination database. Default is 'test'
private String dbName = "test";
//keep
private int keep = 365 * 20;
private int keep = 3650;
//days
private int days = 30;
private int days = 10;
//Super table Name. Default is 'meters'
private String stbName = "meters";
......@@ -35,7 +35,7 @@ public final class JdbcTaosdemoConfig {
private boolean deleteTable = false;
public static void printHelp() {
System.out.println("Usage: java -jar JDBCConnectorChecker.jar [OPTION...]");
System.out.println("Usage: java -jar JdbcTaosDemo.jar [OPTION...]");
System.out.println("-h host The host to connect to TDengine. you must input one");
System.out.println("-p port The TCP/IP port number to use for the connection. Default is 6030");
System.out.println("-u user The TDengine user name to use when connecting to the server. Default is 'root'");
......
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/jdbcTaosdemo/task/InsertTableTask.java
浏览文件 @ d74400f3
......@@ -3,44 +3,49 @@ package com.taosdata.example.jdbcTaosdemo.task;
import com.taosdata.example.jdbcTaosdemo.domain.JdbcTaosdemoConfig;
import com.taosdata.example.jdbcTaosdemo.utils.ConnectionFactory;
import com.taosdata.example.jdbcTaosdemo.utils.SqlSpeller;
import com.taosdata.example.jdbcTaosdemo.utils.TimeStampUtil;
import org.apache.log4j.Logger;
import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.concurrent.atomic.AtomicLong;
import java.time.Duration;
import java.time.Instant;
import java.time.temporal.ChronoUnit;
public class InsertTableTask implements Runnable {
private static final Logger logger = Logger.getLogger(InsertTableTask.class);
private static AtomicLong beginTimestamp = new AtomicLong(TimeStampUtil.datetimeToLong("2005-01-01 00:00:00.000"));
private final JdbcTaosdemoConfig config;
private final int startIndex;
private final int startTbIndex;
private final int tableNumber;
private final int recordsNumber;
private final int recordsNumberPerTable;
public InsertTableTask(JdbcTaosdemoConfig config, int startIndex, int tableNumber, int recordsNumber) {
public InsertTableTask(JdbcTaosdemoConfig config, int startTbIndex, int tableNumber, int recordsNumberPerTable) {
this.config = config;
this.startIndex = startIndex;
this.startTbIndex = startTbIndex;
this.tableNumber = tableNumber;
this.recordsNumber = recordsNumber;
this.recordsNumberPerTable = recordsNumberPerTable;
}
@Override
public void run() {
try {
Connection connection = ConnectionFactory.build(config);
int keep = config.getKeep();
Instant end = Instant.now();
Instant start = end.minus(Duration.ofDays(keep - 1));
long timeGap = ChronoUnit.MILLIS.between(start, end) / (recordsNumberPerTable - 1);
// iterate insert
for (int j = 0; j < recordsNumber; j++) {
long ts = beginTimestamp.getAndIncrement();
for (int j = 0; j < recordsNumberPerTable; j++) {
long ts = start.toEpochMilli() + (j * timeGap);
// insert data into echo table
for (int i = startIndex; i < startIndex + tableNumber; i++) {
for (int i = startTbIndex; i < startTbIndex + tableNumber; i++) {
String sql = SqlSpeller.insertOneRowSQL(config.getDbName(), config.getTbPrefix(), i + 1, ts);
logger.info(Thread.currentThread().getName() + ">>> " + sql);
Statement statement = connection.createStatement();
statement.execute(sql);
statement.close();
logger.info(Thread.currentThread().getName() + ">>> " + sql);
}
}
connection.close();
......
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/jdbcTaosdemo/utils/SqlSpeller.java
浏览文件 @ d74400f3
......@@ -78,5 +78,49 @@ public class SqlSpeller {
return "select avg(" + field + "),min(" + field + "),max(" + field + ") from " + dbName + "." + stbName + "";
}
public static String selectLastFromTableSQL(String dbName, String tbPrefix, int tbIndex) {
return "select last(*) from " + dbName + "." + tbPrefix + "" + tbIndex;
}
//select avg ,max from stb where tag
public static String selectAvgMinMaxFromSuperTableWhere(String field, String dbName, String stbName) {
return "select avg(" + field + "),min(" + field + "),max(" + field + ") from " + dbName + "." + stbName + " where location = '" + locations[random.nextInt(locations.length)] + "'";
}
//select last from stb where
public static String selectLastFromSuperTableWhere(String field, String dbName, String stbName) {
return "select last(" + field + ") from " + dbName + "." + stbName + " where location = '" + locations[random.nextInt(locations.length)] + "'";
}
public static String selectGroupBy(String field, String dbName, String stbName) {
return "select avg(" + field + ") from " + dbName + "." + stbName + " group by location";
}
public static String selectLike(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where location like 'S%'";
}
public static String selectLastOneHour(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where ts >= now - 1h";
}
public static String selectLastOneDay(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where ts >= now - 1d";
}
public static String selectLastOneWeek(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where ts >= now - 1w";
}
public static String selectLastOneMonth(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where ts >= now - 1n";
}
public static String selectLastOneYear(String dbName, String stbName) {
return "select * from " + dbName + "." + stbName + " where ts >= now - 1y";
}
// select group by
// select like
// select ts >= ts<=
}
\ No newline at end of file
tests/examples/JDBC/JDBCDemo/src/main/java/com/taosdata/example/jdbcTaosdemo/utils/TimeStampUtil.java
浏览文件 @ d74400f3
package com.taosdata.example.jdbcTaosdemo.utils;
import java.sql.Date;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.Duration;
import java.time.Instant;
import java.util.Date;
public class TimeStampUtil {
private static final String datetimeFormat = "yyyy-MM-dd HH:mm:ss.SSS";
......@@ -21,14 +23,14 @@ public class TimeStampUtil {
return sdf.format(new Date(time));
}
public static void main(String[] args) {
final String startTime = "2005-01-01 00:00:00.000";
public static void main(String[] args) throws ParseException {
// Instant now = Instant.now();
// System.out.println(now);
// Instant years20Ago = now.minus(Duration.ofDays(365));
// System.out.println(years20Ago);
long start = TimeStampUtil.datetimeToLong(startTime);
System.out.println(start);
String datetime = TimeStampUtil.longToDatetime(1519833600000L);
System.out.println(datetime);
}
......
tests/examples/JDBC/JDBCDemo/src/test/java/com/taosdata/example/jdbcTaosdemo/utils/TimeStampUtilTest.java 0 → 100644
浏览文件 @ d74400f3
package com.taosdata.example.jdbcTaosdemo.utils;
import org.junit.Test;
import java.text.ParseException;
import java.text.SimpleDateFormat;
import java.time.Duration;
import java.time.Instant;
import java.time.format.DateTimeFormatter;
import java.time.temporal.ChronoUnit;
import java.util.Date;
import static org.junit.Assert.*;
public class TimeStampUtilTest {
@Test
public void datetimeToLong() {
final String startTime = "2005-01-01 00:00:00.000";
long start = TimeStampUtil.datetimeToLong(startTime);
assertEquals(1104508800000l, start);
}
@Test
public void longToDatetime() {
String datetime = TimeStampUtil.longToDatetime(1510000000000L);
assertEquals("2017-11-07 04:26:40.000", datetime);
}
@Test
public void getStartDateTime() {
int keep = 365;
Instant end = Instant.now();
System.out.println(end.toString());
System.out.println(end.toEpochMilli());
Instant start = end.minus(Duration.ofDays(keep));
System.out.println(start.toString());
System.out.println(start.toEpochMilli());
int numberOfRecordsPerTable = 10;
long timeGap = ChronoUnit.MILLIS.between(start, end) / (numberOfRecordsPerTable - 1);
System.out.println(timeGap);
System.out.println("===========================");
for (int i = 0; i < numberOfRecordsPerTable; i++) {
long ts = start.toEpochMilli() + (i * timeGap);
System.out.println(i + " : " + ts);
}
}
}
\ No newline at end of file
tests/examples/c/demo.c
浏览文件 @ d74400f3
......@@ -38,7 +38,7 @@ int main(int argc, char *argv[]) {
taos = taos_connect(argv[1], "root", "taosdata", NULL, 0);
if (taos == NULL) {
printf("failed to connect to server, reason:%s\n", taos_errstr(taos));
printf("failed to connect to server, reason:%s\n", "null taos"/*taos_errstr(taos)*/);
exit(1);
}
printf("success to connect to server\n");
......@@ -48,7 +48,7 @@ int main(int argc, char *argv[]) {
result = taos_query(taos, "create database demo");
if (result == NULL) {
printf("failed to create database, reason:%s\n", taos_errstr(taos));
printf("failed to create database, reason:%s\n", "null result"/*taos_errstr(taos)*/);
exit(1);
}
printf("success to create database\n");
......@@ -57,7 +57,7 @@ int main(int argc, char *argv[]) {
// create table
if (taos_query(taos, "create table m1 (ts timestamp, ti tinyint, si smallint, i int, bi bigint, f float, d double, b binary(10))") == 0) {
printf("failed to create table, reason:%s\n", taos_errstr(taos));
printf("failed to create table, reason:%s\n", taos_errstr(result));
exit(1);
}
printf("success to create table\n");
......@@ -70,9 +70,19 @@ int main(int argc, char *argv[]) {
for (i = 0; i < 10; ++i) {
sprintf(qstr, "insert into m1 values (%" PRId64 ", %d, %d, %d, %d, %f, %lf, '%s')", 1546300800000 + i * 1000, i, i, i, i*10000000, i*1.0, i*2.0, "hello");
printf("qstr: %s\n", qstr);
if (taos_query(taos, qstr)) {
printf("insert row: %i, reason:%s\n", i, taos_errstr(taos));
// note: how do you wanna do if taos_query returns non-NULL
// if (taos_query(taos, qstr)) {
// printf("insert row: %i, reason:%s\n", i, taos_errstr(taos));
// }
TAOS_RES *result = taos_query(taos, qstr);
if (result) {
printf("insert row: %i\n", i);
} else {
printf("failed to insert row: %i, reason:%s\n", i, "null result"/*taos_errstr(result)*/);
exit(1);
}
//sleep(1);
}
printf("success to insert rows, total %d rows\n", i);
......
tests/examples/nodejs/nodejsChecker.js 0 → 100644
浏览文件 @ d74400f3
const taos = require('td2.0-connector');
var host = null;
var port = 6030;
for(var i = 2; i < global.process.argv.length; i++){
var key = global.process.argv[i].split("=")[0];
var value = global.process.argv[i].split("=")[1];
if("host" == key){
host = value;
}
if("port" == key){
port = value;
}
}
if(host == null){
console.log("Usage: node nodejsChecker.js host=<hostname> port=<port>");
process.exit(0);
}
// establish connection
var conn = taos.connect({host:host, user:"root", password:"taosdata",port:port});
var cursor = conn.cursor();
// create database
executeSql("create database if not exists test", 0);
// use db
executeSql("use test", 0);
// drop table
executeSql("drop table if exists test.weather", 0);
// create table
executeSql("create table if not exists test.weather(ts timestamp, temperature float, humidity int)", 0);
// insert
executeSql("insert into test.weather (ts, temperature, humidity) values(now, 20.5, 34)", 1);
// select
executeQuery("select * from test.weather");
// close connection
conn.close();
function executeQuery(sql){
var start = new Date().getTime();
var promise = cursor.query(sql, true);
var end = new Date().getTime();
printSql(sql, promise != null,(end - start));
promise.then(function(result){
result.pretty();
});
}
function executeSql(sql, affectRows){
var start = new Date().getTime();
var promise = cursor.execute(sql);
var end = new Date().getTime();
printSql(sql, promise == affectRows, (end - start));
}
function printSql(sql, succeed, cost){
console.log("[ "+(succeed ? "OK" : "ERROR!")+" ] time cost: " + cost + " ms, execute statement ====> " + sql);
}
tests/examples/rust/src/bindings.rs
浏览文件 @ d74400f3
......@@ -308,12 +308,6 @@ extern "C" {
extern "C" {
pub fn taos_unsubscribe(tsub: *mut ::std::os::raw::c_void);
}
extern "C" {
pub fn taos_subfields_count(tsub: *mut ::std::os::raw::c_void) -> ::std::os::raw::c_int;
}
extern "C" {
pub fn taos_fetch_subfields(tsub: *mut ::std::os::raw::c_void) -> *mut TAOS_FIELD;
}
extern "C" {
pub fn taos_open_stream(
taos: *mut ::std::os::raw::c_void,
......
tests/examples/rust/src/subscriber.rs
浏览文件 @ d74400f3
......@@ -37,16 +37,16 @@ impl Subscriber {
println!("subscribed to {} user:{}, db:{}, tb:{}, time:{}, mseconds:{}",
host, username, db, table, time, mseconds);
let mut fields = taos_fetch_subfields(tsub);
let mut fields = taos_fetch_fields(tsub);
if fields.is_null() {
taos_unsubscribe(tsub);
return Err("fetch subfields error")
return Err("fetch fields error")
}
let fcount = taos_subfields_count(tsub);
let fcount = taos_field_count(tsub);
if fcount == 0 {
taos_unsubscribe(tsub);
return Err("subfields count is 0")
return Err("fields count is 0")
}
Ok(Subscriber{tsub, fields, fcount})
......@@ -74,4 +74,4 @@ impl Drop for Subscriber {
fn drop(&mut self) {
unsafe {taos_unsubscribe(self.tsub);}
}
}
\ No newline at end of file
}
tests/pytest/fulltest.sh
浏览文件 @ d74400f3
......@@ -18,7 +18,9 @@ python3 ./test.py -f insert/multi.py
python3 ./test.py -f insert/randomNullCommit.py
python3 insert/retentionpolicy.py
python3 ./test.py -f insert/alterTableAndInsert.py
python3 ./test.py -f insert/insertIntoTwoTables.py
python3 ./test.py -f table/alter_wal0.py
python3 ./test.py -f table/column_name.py
python3 ./test.py -f table/column_num.py
python3 ./test.py -f table/db_table.py
......@@ -149,6 +151,7 @@ python3 ./test.py -f query/queryNullValueTest.py
python3 ./test.py -f query/queryInsertValue.py
python3 ./test.py -f query/queryConnection.py
python3 ./test.py -f query/natualInterval.py
python3 ./test.py -f query/bug1471.py
#stream
python3 ./test.py -f stream/metric_1.py
......
tests/pytest/insert/before_1970.py 0 → 100644
浏览文件 @ d74400f3
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import taos
from util.log import tdLog
from util.cases import tdCases
from util.sql import tdSql
from util.dnodes import tdDnodes
class TDTestCase:
"""
add test data before 1970s
"""
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def run(self):
tdSql.prepare()
print("==============step1")
tdSql.execute("create database if not exists demo keep 36500;");
print("==============create db demo keep 365000 days")
tdSql.execute("use demo;")
tdSql.execute("CREATE table if not exists test (ts timestamp, f1 int);")
print("==============create table test")
print("==============step2")
#TODO : should add more testcases
tdSql.execute("insert into test values('1930-12-12 01:19:20.345', 1);")
tdSql.execute("insert into test values('1969-12-30 23:59:59.999', 2);")
tdSql.execute("insert into test values(-3600, 3);")
tdSql.execute("insert into test values('2020-10-20 14:02:53.770', 4);")
print("==============insert data")
# tdSql.query("select * from test;")
#
# tdSql.checkRows(3)
#
# tdSql.checkData(0,0,'1969-12-12 01:19:20.345000')
# tdSql.checkData(1,0,'1970-01-01 07:00:00.000000')
# tdSql.checkData(2,0,'2020-10-20 14:02:53.770000')
print("==============step3")
tdDnodes.stopAll()
tdDnodes.start(1)
print("==============restart taosd")
print("==============step4")
tdSql.execute("use demo;")
tdSql.query("select * from test;")
# print(tdSql.queryResult)
tdSql.checkRows(4)
tdSql.checkData(0,0,'1930-12-12 01:19:20.345000')
tdSql.checkData(1,0,'1969-12-30 23:59:59.999000')
tdSql.checkData(2,0,'1970-01-01 07:00:00.000000')
tdSql.checkData(3,0,'2020-10-20 14:02:53.770000')
print("==============check data")
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
tests/pytest/insert/insertIntoTwoTables.py 0 → 100644
浏览文件 @ d74400f3
此差异已折叠。
tests/pytest/query/bug1471.py 0 → 100644
浏览文件 @ d74400f3
此差异已折叠。
tests/pytest/query/removeDBAndSTable.py 0 → 100644
浏览文件 @ d74400f3
此差异已折叠。
tests/pytest/table/alter_wal0.py 0 → 100644
浏览文件 @ d74400f3
此差异已折叠。
tests/script/general/parser/tags_dynamically_specifiy.sim
浏览文件 @ d74400f3
......@@ -96,5 +96,10 @@ if $rows != 14 then
return -1
endi
print ===============================> td-1765
sql create table m1(ts timestamp, k int) tags(a binary(4), b nchar(4));
sql create table tm0 using m1 tags('abcd', 'abcd');
sql_error alter table tm0 set tag b = 'abcd1';
sql_error alter table tm0 set tag a = 'abcd1';
system sh/exec.sh -n dnode1 -s stop -x SIGINT
\ No newline at end of file
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或