Skip to content

T
TDengine

taosdata / TDengine
大约 2 年 前同步成功

通知 1192
Star 22018
Fork 4786
T
TDengine
提交 6b7e4a85 编写于 作者: C Cary Xu
浏览文件
操作

Merge branch 'develop' into enhance/TS-385

上级 a2459fae b78fb511
显示空白变更内容
内联 并排
Showing with 811 addition and 437 deletion
README.md
浏览文件 @ 6b7e4a85
...@@ -202,6 +202,19 @@ taos ...@@ -202,6 +202,19 @@ taos
If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown. If TDengine shell connects the server successfully, welcome messages and version info are printed. Otherwise, an error message is shown.
## Install TDengine by apt-get
If you use Debian or Ubuntu system, you can use 'apt-get' command to intall TDengine from official repository. Please use following commands to setup:
```
wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add -
echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | sudo tee /etc/apt/sources.list.d/tdengine-stable.list
[Optional] echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list
sudo apt-get update
apt-get policy tdengine
sudo apt-get install tdengine
```
## Quick Run ## Quick Run
If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`) If you don't want to run TDengine as a service, you can run it in current shell. For example, to quickly start a TDengine server after building, run the command below in terminal: (We take Linux as an example, command on Windows will be `taosd.exe`)
......
cmake/define.inc
浏览文件 @ 6b7e4a85
...@@ -124,6 +124,10 @@ IF (TD_APLHINE) ...@@ -124,6 +124,10 @@ IF (TD_APLHINE)
MESSAGE(STATUS "aplhine is defined") MESSAGE(STATUS "aplhine is defined")
ENDIF () ENDIF ()
IF (TD_BUILD_HTTP)
ADD_DEFINITIONS(-DHTTP_EMBEDDED)
ENDIF ()
IF (TD_LINUX) IF (TD_LINUX)
ADD_DEFINITIONS(-DLINUX) ADD_DEFINITIONS(-DLINUX)
ADD_DEFINITIONS(-D_LINUX) ADD_DEFINITIONS(-D_LINUX)
......
cmake/input.inc
浏览文件 @ 6b7e4a85
...@@ -90,6 +90,12 @@ IF (${BUILD_JDBC} MATCHES "false") ...@@ -90,6 +90,12 @@ IF (${BUILD_JDBC} MATCHES "false")
SET(TD_BUILD_JDBC FALSE) SET(TD_BUILD_JDBC FALSE)
ENDIF () ENDIF ()
SET(TD_BUILD_HTTP TRUE)
IF (${BUILD_HTTP} MATCHES "false")
SET(TD_BUILD_HTTP FALSE)
ENDIF ()
SET(TD_MEMORY_SANITIZER FALSE) SET(TD_MEMORY_SANITIZER FALSE)
IF (${MEMORY_SANITIZER} MATCHES "true") IF (${MEMORY_SANITIZER} MATCHES "true")
SET(TD_MEMORY_SANITIZER TRUE) SET(TD_MEMORY_SANITIZER TRUE)
......
documentation20/cn/02.getting-started/docs.md
浏览文件 @ 6b7e4a85
...@@ -22,6 +22,18 @@ TDengine 的安装非常简单，从下载到安装成功仅仅只要几秒钟 ...@@ -22,6 +22,18 @@ TDengine 的安装非常简单，从下载到安装成功仅仅只要几秒钟
具体的安装过程，请参见 [TDengine 多种安装包的安装和卸载](https://www.taosdata.com/blog/2019/08/09/566.html) 以及 [视频教程](https://www.taosdata.com/blog/2020/11/11/1941.html)。 具体的安装过程，请参见 [TDengine 多种安装包的安装和卸载](https://www.taosdata.com/blog/2019/08/09/566.html) 以及 [视频教程](https://www.taosdata.com/blog/2020/11/11/1941.html)。
### 使用 apt-get 安装
如果使用 Debian 或 Ubuntu 系统，也可以使用 apt-get 从官方仓库安装，设置方法为：
```
wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add -
echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | sudo tee /etc/apt/sources.list.d/tdengine-stable.list
[ beta 版安装包仓库为可选安装项 ] echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list
sudo apt-get update
apt-get policy tdengine
sudo apt-get install tdengine
```
<a class="anchor" id="start"></a> <a class="anchor" id="start"></a>
## 轻松启动 ## 轻松启动
......
documentation20/en/02.getting-started/docs.md
浏览文件 @ 6b7e4a85
...@@ -20,6 +20,19 @@ Three different packages for TDengine server are provided, please pick up the on ...@@ -20,6 +20,19 @@ Three different packages for TDengine server are provided, please pick up the on
Click [here](https://www.taosdata.com/en/getting-started/#Install-from-Package) to download the install package. Click [here](https://www.taosdata.com/en/getting-started/#Install-from-Package) to download the install package.
### Install TDengine by apt-get
If you use Debian or Ubuntu system you can use 'apt-get' command to install TDengine from official repository. Please use following commands to setup:
```
wget -qO - http://repos.taosdata.com/tdengine.key | sudo apt-key add -
echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-stable stable main" | sudo tee /etc/apt/sources.list.d/tdengine-stable.list
[Optional] echo "deb [arch=amd64] http://repos.taosdata.com/tdengine-beta beta main" | sudo tee /etc/apt/sources.list.d/tdengine-beta.list
sudo apt-get update
apt-get policy tdengine
sudo apt-get install tdengine
```
## <a class="anchor" id="start"></a>Quick Launch ## <a class="anchor" id="start"></a>Quick Launch
After installation, you can start the TDengine service by the `systemctl` command. After installation, you can start the TDengine service by the `systemctl` command.
......
src/client/CMakeLists.txt
浏览文件 @ 6b7e4a85
...@@ -5,7 +5,11 @@ INCLUDE_DIRECTORIES(inc) ...@@ -5,7 +5,11 @@ INCLUDE_DIRECTORIES(inc)
INCLUDE_DIRECTORIES(jni) INCLUDE_DIRECTORIES(jni)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/query/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/query/inc)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/zlib-1.2.11/inc) INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/deps/zlib-1.2.11/inc)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/plugins/http/inc)
IF (TD_BUILD_HTTP)
INCLUDE_DIRECTORIES(${TD_COMMUNITY_DIR}/src/plugins/http/inc)
ENDIF ()
AUX_SOURCE_DIRECTORY(src SRC) AUX_SOURCE_DIRECTORY(src SRC)
IF (TD_LINUX) IF (TD_LINUX)
......
src/client/src/tscUtil.c
浏览文件 @ 6b7e4a85
...@@ -29,7 +29,10 @@ ...@@ -29,7 +29,10 @@
#include "tsclient.h" #include "tsclient.h"
#include "ttimer.h" #include "ttimer.h"
#include "ttokendef.h" #include "ttokendef.h"
#ifdef HTTP_EMBEDDED
#include "httpInt.h" #include "httpInt.h"
#endif
static void freeQueryInfoImpl(SQueryInfo* pQueryInfo); static void freeQueryInfoImpl(SQueryInfo* pQueryInfo);
...@@ -5118,11 +5121,14 @@ void tscRemoveCachedTableMeta(STableMetaInfo* pTableMetaInfo, uint64_t id) { ...@@ -5118,11 +5121,14 @@ void tscRemoveCachedTableMeta(STableMetaInfo* pTableMetaInfo, uint64_t id) {
char* cloneCurrentDBName(SSqlObj* pSql) { char* cloneCurrentDBName(SSqlObj* pSql) {
char *p = NULL; char *p = NULL;
#ifdef HTTP_EMBEDDED
HttpContext *pCtx = NULL; HttpContext *pCtx = NULL;
#endif
pthread_mutex_lock(&pSql->pTscObj->mutex); pthread_mutex_lock(&pSql->pTscObj->mutex);
STscObj *pTscObj = pSql->pTscObj; STscObj *pTscObj = pSql->pTscObj;
switch (pTscObj->from) { switch (pTscObj->from) {
#ifdef HTTP_EMBEDDED
case TAOS_REQ_FROM_HTTP: case TAOS_REQ_FROM_HTTP:
pCtx = pSql->param; pCtx = pSql->param;
if (pCtx && pCtx->db[0] != '\0') { if (pCtx && pCtx->db[0] != '\0') {
...@@ -5133,6 +5139,7 @@ char* cloneCurrentDBName(SSqlObj* pSql) { ...@@ -5133,6 +5139,7 @@ char* cloneCurrentDBName(SSqlObj* pSql) {
p = strdup(db); p = strdup(db);
} }
break; break;
#endif
default: default:
break; break;
} }
......
src/dnode/CMakeLists.txt
浏览文件 @ 6b7e4a85
...@@ -18,7 +18,12 @@ ELSE () ...@@ -18,7 +18,12 @@ ELSE ()
ENDIF () ENDIF ()
ADD_EXECUTABLE(taosd ${SRC}) ADD_EXECUTABLE(taosd ${SRC})
IF (TD_BUILD_HTTP)
TARGET_LINK_LIBRARIES(taosd mnode monitor http tsdb twal vnode cJson lua lz4 balance sync ${LINK_JEMALLOC}) TARGET_LINK_LIBRARIES(taosd mnode monitor http tsdb twal vnode cJson lua lz4 balance sync ${LINK_JEMALLOC})
ELSE ()
TARGET_LINK_LIBRARIES(taosd mnode monitor tsdb twal vnode cJson lua lz4 balance sync ${LINK_JEMALLOC})
ENDIF ()
IF (TD_SOMODE_STATIC) IF (TD_SOMODE_STATIC)
TARGET_LINK_LIBRARIES(taosd taos_static) TARGET_LINK_LIBRARIES(taosd taos_static)
......
src/dnode/src/dnodeModule.c
浏览文件 @ 6b7e4a85
...@@ -49,6 +49,7 @@ static void dnodeAllocModules() { ...@@ -49,6 +49,7 @@ static void dnodeAllocModules() {
tsModule[TSDB_MOD_MNODE].startFp = mnodeStartSystem; tsModule[TSDB_MOD_MNODE].startFp = mnodeStartSystem;
tsModule[TSDB_MOD_MNODE].stopFp = mnodeStopSystem; tsModule[TSDB_MOD_MNODE].stopFp = mnodeStopSystem;
#ifdef HTTP_EMBEDDED
tsModule[TSDB_MOD_HTTP].enable = (tsEnableHttpModule == 1); tsModule[TSDB_MOD_HTTP].enable = (tsEnableHttpModule == 1);
tsModule[TSDB_MOD_HTTP].name = "http"; tsModule[TSDB_MOD_HTTP].name = "http";
tsModule[TSDB_MOD_HTTP].initFp = httpInitSystem; tsModule[TSDB_MOD_HTTP].initFp = httpInitSystem;
...@@ -58,6 +59,7 @@ static void dnodeAllocModules() { ...@@ -58,6 +59,7 @@ static void dnodeAllocModules() {
if (tsEnableHttpModule) { if (tsEnableHttpModule) {
dnodeSetModuleStatus(TSDB_MOD_HTTP); dnodeSetModuleStatus(TSDB_MOD_HTTP);
} }
#endif
#ifdef _MQTT #ifdef _MQTT
tsModule[TSDB_MOD_MQTT].enable = (tsEnableMqttModule == 1); tsModule[TSDB_MOD_MQTT].enable = (tsEnableMqttModule == 1);
......
src/dnode/src/dnodeShell.c
浏览文件 @ 6b7e4a85
...@@ -240,7 +240,9 @@ void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t tid) { ...@@ -240,7 +240,9 @@ void *dnodeSendCfgTableToRecv(int32_t vgId, int32_t tid) {
SStatisInfo dnodeGetStatisInfo() { SStatisInfo dnodeGetStatisInfo() {
SStatisInfo info = {0}; SStatisInfo info = {0};
if (dnodeGetRunStatus() == TSDB_RUN_STATUS_RUNING) { if (dnodeGetRunStatus() == TSDB_RUN_STATUS_RUNING) {
#ifdef HTTP_EMBEDDED
info.httpReqNum = httpGetReqCount(); info.httpReqNum = httpGetReqCount();
#endif
info.queryReqNum = atomic_exchange_32(&tsQueryReqNum, 0); info.queryReqNum = atomic_exchange_32(&tsQueryReqNum, 0);
info.submitReqNum = atomic_exchange_32(&tsSubmitReqNum, 0); info.submitReqNum = atomic_exchange_32(&tsSubmitReqNum, 0);
} }
......
src/kit/taosdemo/taosdemo.c
浏览文件 @ 6b7e4a85
...@@ -986,36 +986,55 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { ...@@ -986,36 +986,55 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) {
arguments->performance_print = true; arguments->performance_print = true;
} else if ((0 == strncmp(argv[i], "-P", strlen("-P"))) } else if ((0 == strncmp(argv[i], "-P", strlen("-P")))
|| (0 == strncmp(argv[i], "--port", strlen("--port")))) { || (0 == strncmp(argv[i], "--port", strlen("--port")))) {
uint64_t port;
char strPort[BIGINT_BUFF_LEN];
if (2 == strlen(argv[i])) { if (2 == strlen(argv[i])) {
if (argc == i+1) { if (argc == i+1) {
errorPrintReqArg(argv[0], "P"); errorPrintReqArg(argv[0], "P");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} else if (!isStringNumber(argv[i+1])) { } else if (isStringNumber(argv[i+1])) {
tstrncpy(strPort, argv[++i], BIGINT_BUFF_LEN);
} else {
errorPrintReqArg2(argv[0], "P"); errorPrintReqArg2(argv[0], "P");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
arguments->port = atoi(argv[++i]);
} else if (0 == strncmp(argv[i], "--port=", strlen("--port="))) { } else if (0 == strncmp(argv[i], "--port=", strlen("--port="))) {
if (isStringNumber((char *)(argv[i] + strlen("--port=")))) { if (isStringNumber((char *)(argv[i] + strlen("--port=")))) {
arguments->port = atoi((char *)(argv[i]+strlen("--port="))); tstrncpy(strPort, (char *)(argv[i]+strlen("--port=")), BIGINT_BUFF_LEN);
} else {
errorPrintReqArg2(argv[0], "--port");
exit(EXIT_FAILURE);
} }
} else if (0 == strncmp(argv[i], "-P", strlen("-P"))) { } else if (0 == strncmp(argv[i], "-P", strlen("-P"))) {
if (isStringNumber((char *)(argv[i] + strlen("-P")))) { if (isStringNumber((char *)(argv[i] + strlen("-P")))) {
arguments->port = atoi((char *)(argv[i]+strlen("-P"))); tstrncpy(strPort, (char *)(argv[i]+strlen("-P")), BIGINT_BUFF_LEN);
} else {
errorPrintReqArg2(argv[0], "--port");
exit(EXIT_FAILURE);
} }
} else if (strlen("--port") == strlen(argv[i])) { } else if (strlen("--port") == strlen(argv[i])) {
if (argc == i+1) { if (argc == i+1) {
errorPrintReqArg3(argv[0], "--port"); errorPrintReqArg3(argv[0], "--port");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} else if (!isStringNumber(argv[i+1])) { } else if (isStringNumber(argv[i+1])) {
tstrncpy(strPort, argv[++i], BIGINT_BUFF_LEN);
} else {
errorPrintReqArg2(argv[0], "--port"); errorPrintReqArg2(argv[0], "--port");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
arguments->port = atoi(argv[++i]);
} else { } else {
errorUnrecognized(argv[0], argv[i]); errorUnrecognized(argv[0], argv[i]);
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
port = atoi(strPort);
if (port > 65535) {
errorWrongValue("taosdump", "-P or --port", strPort);
exit(EXIT_FAILURE);
}
arguments->port = (uint16_t)port;
} else if ((0 == strncmp(argv[i], "-I", strlen("-I"))) } else if ((0 == strncmp(argv[i], "-I", strlen("-I")))
|| (0 == strncmp(argv[i], "--interface", strlen("--interface")))) { || (0 == strncmp(argv[i], "--interface", strlen("--interface")))) {
if (2 == strlen(argv[i])) { if (2 == strlen(argv[i])) {
...@@ -1581,7 +1600,9 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { ...@@ -1581,7 +1600,9 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) {
&& strcasecmp(dataType, "BIGINT") && strcasecmp(dataType, "BIGINT")
&& strcasecmp(dataType, "DOUBLE") && strcasecmp(dataType, "DOUBLE")
&& strcasecmp(dataType, "TIMESTAMP") && strcasecmp(dataType, "TIMESTAMP")
&& !regexMatch(dataType, "^(NCHAR|BINARY)(\\([1-9][0-9]*\\))?$", REG_ICASE | REG_EXTENDED) && !regexMatch(dataType,
"^(NCHAR|BINARY)(\\([1-9][0-9]*\\))?$",
REG_ICASE | REG_EXTENDED)
&& strcasecmp(dataType, "UTINYINT") && strcasecmp(dataType, "UTINYINT")
&& strcasecmp(dataType, "USMALLINT") && strcasecmp(dataType, "USMALLINT")
&& strcasecmp(dataType, "UINT") && strcasecmp(dataType, "UINT")
...@@ -1603,11 +1624,13 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) { ...@@ -1603,11 +1624,13 @@ static void parse_args(int argc, char *argv[], SArguments *arguments) {
arguments->data_type[0] = TSDB_DATA_TYPE_FLOAT; arguments->data_type[0] = TSDB_DATA_TYPE_FLOAT;
} else if (0 == strcasecmp(dataType, "DOUBLE")) { } else if (0 == strcasecmp(dataType, "DOUBLE")) {
arguments->data_type[0] = TSDB_DATA_TYPE_DOUBLE; arguments->data_type[0] = TSDB_DATA_TYPE_DOUBLE;
} else if (1 == regexMatch(dataType, "^BINARY(\\([1-9][0-9]*\\))?$", REG_ICASE | } else if (1 == regexMatch(dataType,
REG_EXTENDED)) { "^BINARY(\\([1-9][0-9]*\\))?$",
REG_ICASE | REG_EXTENDED)) {
arguments->data_type[0] = TSDB_DATA_TYPE_BINARY; arguments->data_type[0] = TSDB_DATA_TYPE_BINARY;
} else if (1 == regexMatch(dataType, "^NCHAR(\\([1-9][0-9]*\\))?$", REG_ICASE | } else if (1 == regexMatch(dataType,
REG_EXTENDED)) { "^NCHAR(\\([1-9][0-9]*\\))?$",
REG_ICASE | REG_EXTENDED)) {
arguments->data_type[0] = TSDB_DATA_TYPE_NCHAR; arguments->data_type[0] = TSDB_DATA_TYPE_NCHAR;
} else if (0 == strcasecmp(dataType, "BOOL")) { } else if (0 == strcasecmp(dataType, "BOOL")) {
arguments->data_type[0] = TSDB_DATA_TYPE_BOOL; arguments->data_type[0] = TSDB_DATA_TYPE_BOOL;
...@@ -2663,6 +2686,8 @@ static int printfInsertMeta() { ...@@ -2663,6 +2686,8 @@ static int printfInsertMeta() {
} }
} }
if (g_args.use_metric) {
printf(" super table count: \033[33m%"PRIu64"\033[0m\n", printf(" super table count: \033[33m%"PRIu64"\033[0m\n",
g_Dbs.db[i].superTblCount); g_Dbs.db[i].superTblCount);
for (uint64_t j = 0; j < g_Dbs.db[i].superTblCount; j++) { for (uint64_t j = 0; j < g_Dbs.db[i].superTblCount; j++) {
...@@ -2774,6 +2799,14 @@ static int printfInsertMeta() { ...@@ -2774,6 +2799,14 @@ static int printfInsertMeta() {
} }
printf("\n"); printf("\n");
} }
} else {
printf(" childTblCount: \033[33m%"PRId64"\033[0m\n",
g_args.ntables);
printf(" insertRows: \033[33m%"PRId64"\033[0m\n",
g_args.insertRows);
}
printf("\n"); printf("\n");
} }
...@@ -10270,7 +10303,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, ...@@ -10270,7 +10303,7 @@ static void startMultiThreadInsertData(int threads, char* db_name,
&stbInfo->childTblName, &childTblCount, &stbInfo->childTblName, &childTblCount,
limit, limit,
offset); offset);
ntables = childTblCount; // CBD ntables = childTblCount;
} else { } else {
ntables = g_args.ntables; ntables = g_args.ntables;
tableFrom = 0; tableFrom = 0;
...@@ -10289,8 +10322,7 @@ static void startMultiThreadInsertData(int threads, char* db_name, ...@@ -10289,8 +10322,7 @@ static void startMultiThreadInsertData(int threads, char* db_name,
b = ntables % threads; b = ntables % threads;
} }
if ((stbInfo) if (g_args.iface == REST_IFACE || ((stbInfo) && (stbInfo->iface == REST_IFACE))) {
&& (stbInfo->iface == REST_IFACE)) {
if (convertHostToServAddr( if (convertHostToServAddr(
g_Dbs.host, g_Dbs.port, &(g_Dbs.serv_addr)) != 0) { g_Dbs.host, g_Dbs.port, &(g_Dbs.serv_addr)) != 0) {
ERROR_EXIT("convert host to server address"); ERROR_EXIT("convert host to server address");
......
src/kit/taosdump/taosdump.c
浏览文件 @ 6b7e4a85
...@@ -13,6 +13,8 @@ ...@@ -13,6 +13,8 @@
* along with this program. If not, see <http://www.gnu.org/licenses/>. * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/ */
#include <stdio.h>
#include <pthread.h>
#include <iconv.h> #include <iconv.h>
#include <sys/stat.h> #include <sys/stat.h>
#include <sys/syscall.h> #include <sys/syscall.h>
...@@ -26,6 +28,12 @@ ...@@ -26,6 +28,12 @@
#include "tsdb.h" #include "tsdb.h"
#include "tutil.h" #include "tutil.h"
#define AVRO_SUPPORT 0
#if AVRO_SUPPORT == 1
#include <avro.h>
#endif
#define TSDB_SUPPORT_NANOSECOND 1 #define TSDB_SUPPORT_NANOSECOND 1
#define MAX_FILE_NAME_LEN 256 // max file name length on linux is 255 #define MAX_FILE_NAME_LEN 256 // max file name length on linux is 255
...@@ -38,8 +46,8 @@ ...@@ -38,8 +46,8 @@
static int converStringToReadable(char *str, int size, char *buf, int bufsize); static int converStringToReadable(char *str, int size, char *buf, int bufsize);
static int convertNCharToReadable(char *str, int size, char *buf, int bufsize); static int convertNCharToReadable(char *str, int size, char *buf, int bufsize);
static void taosDumpCharset(FILE *fp); static void dumpCharset(FILE *fp);
static void taosLoadFileCharset(FILE *fp, char *fcharset); static void loadFileCharset(FILE *fp, char *fcharset);
typedef struct { typedef struct {
short bytes; short bytes;
...@@ -128,19 +136,23 @@ enum _describe_table_index { ...@@ -128,19 +136,23 @@ enum _describe_table_index {
TSDB_MAX_DESCRIBE_METRIC TSDB_MAX_DESCRIBE_METRIC
}; };
#define COL_NOTE_LEN 128 #define COL_NOTE_LEN 4
#define COL_TYPEBUF_LEN 16
#define COL_VALUEBUF_LEN 32
typedef struct { typedef struct {
char field[TSDB_COL_NAME_LEN + 1]; char field[TSDB_COL_NAME_LEN];
char type[16]; char type[COL_TYPEBUF_LEN];
int length; int length;
char note[COL_NOTE_LEN]; char note[COL_NOTE_LEN];
} SColDes; char value[COL_VALUEBUF_LEN];
char *var_value;
} ColDes;
typedef struct { typedef struct {
char name[TSDB_TABLE_NAME_LEN]; char name[TSDB_TABLE_NAME_LEN];
SColDes cols[]; ColDes cols[];
} STableDef; } TableDef;
extern char version[]; extern char version[];
...@@ -159,7 +171,8 @@ typedef struct { ...@@ -159,7 +171,8 @@ typedef struct {
} TableRecord; } TableRecord;
typedef struct { typedef struct {
bool isStable; bool isStb;
bool belongStb;
int64_t dumpNtbCount; int64_t dumpNtbCount;
TableRecord **dumpNtbInfos; TableRecord **dumpNtbInfos;
TableRecord tableRecord; TableRecord tableRecord;
...@@ -329,24 +342,21 @@ static resultStatistics g_resultStatistics = {0}; ...@@ -329,24 +342,21 @@ static resultStatistics g_resultStatistics = {0};
static FILE *g_fpOfResult = NULL; static FILE *g_fpOfResult = NULL;
static int g_numOfCores = 1; static int g_numOfCores = 1;
static int taosDumpOut(); static int dumpOut();
static int taosDumpIn(); static int dumpIn();
static void taosDumpCreateDbClause(SDbInfo *dbInfo, bool isDumpProperty, static void dumpCreateDbClause(SDbInfo *dbInfo, bool isDumpProperty,
FILE *fp); FILE *fp);
//static int taosDumpDb(SDbInfo *dbInfo, FILE *fp, TAOS *taos); static int dumpCreateTableClause(TableDef *tableDes, int numOfCols,
static int dumpStable(char *table, FILE *fp, SDbInfo *dbInfo);
static int dumpCreateTableClause(STableDef *tableDes, int numOfCols,
FILE *fp, char* dbName); FILE *fp, char* dbName);
static void taosDumpCreateMTableClause(STableDef *tableDes, char *stable, static int getTableDes(
int numOfCols, FILE *fp, char* dbName); char* dbName, char *table,
static int64_t taosDumpTable(char *tbName, char *stable, TableDef *stableDes, bool isSuperTable);
FILE *fp, char* dbName, int precision);
static int64_t dumpTableData(FILE *fp, char *tbName, static int64_t dumpTableData(FILE *fp, char *tbName,
char* dbName, char* dbName,
int precision, int precision,
char *jsonAvroSchema); char *jsonAvroSchema);
static int checkParam(); static int checkParam();
static void taosFreeDbInfos(); static void freeDbInfos();
struct arguments g_args = { struct arguments g_args = {
// connection option // connection option
...@@ -420,16 +430,16 @@ static void printVersion() { ...@@ -420,16 +430,16 @@ static void printVersion() {
} }
} }
UNUSED_FUNC void errorWrongValue(char *program, char *wrong_arg, char *wrong_value) void errorWrongValue(char *program, char *wrong_arg, char *wrong_value)
{ {
fprintf(stderr, "%s %s: %s is an invalid value\n", program, wrong_arg, wrong_value); fprintf(stderr, "%s %s: %s is an invalid value\n", program, wrong_arg, wrong_value);
fprintf(stderr, "Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); fprintf(stderr, "Try `taosdump --help' or `taosdump --usage' for more information.\n");
} }
static void errorUnrecognized(char *program, char *wrong_arg) static void errorUnrecognized(char *program, char *wrong_arg)
{ {
fprintf(stderr, "%s: unrecognized options '%s'\n", program, wrong_arg); fprintf(stderr, "%s: unrecognized options '%s'\n", program, wrong_arg);
fprintf(stderr, "Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); fprintf(stderr, "Try `taosdump --help' or `taosdump --usage' for more information.\n");
} }
static void errorPrintReqArg(char *program, char *wrong_arg) static void errorPrintReqArg(char *program, char *wrong_arg)
...@@ -438,7 +448,7 @@ static void errorPrintReqArg(char *program, char *wrong_arg) ...@@ -438,7 +448,7 @@ static void errorPrintReqArg(char *program, char *wrong_arg)
"%s: option requires an argument -- '%s'\n", "%s: option requires an argument -- '%s'\n",
program, wrong_arg); program, wrong_arg);
fprintf(stderr, fprintf(stderr,
"Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); "Try `taosdump --help' or `taosdump --usage' for more information.\n");
} }
static void errorPrintReqArg2(char *program, char *wrong_arg) static void errorPrintReqArg2(char *program, char *wrong_arg)
...@@ -447,7 +457,7 @@ static void errorPrintReqArg2(char *program, char *wrong_arg) ...@@ -447,7 +457,7 @@ static void errorPrintReqArg2(char *program, char *wrong_arg)
"%s: option requires a number argument '-%s'\n", "%s: option requires a number argument '-%s'\n",
program, wrong_arg); program, wrong_arg);
fprintf(stderr, fprintf(stderr,
"Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); "Try `taosdump --help' or `taosdump --usage' for more information.\n");
} }
static void errorPrintReqArg3(char *program, char *wrong_arg) static void errorPrintReqArg3(char *program, char *wrong_arg)
...@@ -456,7 +466,7 @@ static void errorPrintReqArg3(char *program, char *wrong_arg) ...@@ -456,7 +466,7 @@ static void errorPrintReqArg3(char *program, char *wrong_arg)
"%s: option '%s' requires an argument\n", "%s: option '%s' requires an argument\n",
program, wrong_arg); program, wrong_arg);
fprintf(stderr, fprintf(stderr,
"Try `taosdemo --help' or `taosdemo --usage' for more information.\n"); "Try `taosdump --help' or `taosdump --usage' for more information.\n");
} }
/* Parse a single option. */ /* Parse a single option. */
...@@ -483,7 +493,14 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { ...@@ -483,7 +493,14 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
errorPrintReqArg2("taosdump", "P"); errorPrintReqArg2("taosdump", "P");
exit(EXIT_FAILURE); exit(EXIT_FAILURE);
} }
g_args.port = atoi(arg);
uint64_t port = atoi(arg);
if (port > 65535) {
errorWrongValue("taosdump", "-P or --port", arg);
exit(EXIT_FAILURE);
}
g_args.port = (uint16_t)port;
break; break;
case 'q': case 'q':
g_args.mysqlFlag = atoi(arg); g_args.mysqlFlag = atoi(arg);
...@@ -606,6 +623,17 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) { ...@@ -606,6 +623,17 @@ static error_t parse_opt(int key, char *arg, struct argp_state *state) {
return 0; return 0;
} }
static void freeTbDes(TableDef *tableDes)
{
for (int i = 0; i < TSDB_MAX_COLUMNS; i ++) {
if (tableDes->cols[i].var_value) {
free(tableDes->cols[i].var_value);
}
}
free(tableDes);
}
static int queryDbImpl(TAOS *taos, char *command) { static int queryDbImpl(TAOS *taos, char *command) {
TAOS_RES *res = NULL; TAOS_RES *res = NULL;
int32_t code = -1; int32_t code = -1;
...@@ -764,7 +792,7 @@ static int getPrecisionByString(char *precision) ...@@ -764,7 +792,7 @@ static int getPrecisionByString(char *precision)
return -1; return -1;
} }
static void taosFreeDbInfos() { static void freeDbInfos() {
if (g_dbInfos == NULL) return; if (g_dbInfos == NULL) return;
for (int i = 0; i < g_args.dumpDbCount; i++) for (int i = 0; i < g_args.dumpDbCount; i++)
tfree(g_dbInfos[i]); tfree(g_dbInfos[i]);
...@@ -815,15 +843,20 @@ static int getTableRecordInfo( ...@@ -815,15 +843,20 @@ static int getTableRecordInfo(
while ((row = taos_fetch_row(result)) != NULL) { while ((row = taos_fetch_row(result)) != NULL) {
isSet = true; isSet = true;
pTableRecordInfo->isStable = false; pTableRecordInfo->isStb = false;
tstrncpy(pTableRecordInfo->tableRecord.name, tstrncpy(pTableRecordInfo->tableRecord.name,
(char *)row[TSDB_SHOW_TABLES_NAME_INDEX], (char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes + 1)); fields[TSDB_SHOW_TABLES_NAME_INDEX].bytes + 1));
if (strlen((char *)row[TSDB_SHOW_TABLES_METRIC_INDEX]) > 0) {
pTableRecordInfo->belongStb = true;
tstrncpy(pTableRecordInfo->tableRecord.stable, tstrncpy(pTableRecordInfo->tableRecord.stable,
(char *)row[TSDB_SHOW_TABLES_METRIC_INDEX], (char *)row[TSDB_SHOW_TABLES_METRIC_INDEX],
min(TSDB_TABLE_NAME_LEN, min(TSDB_TABLE_NAME_LEN,
fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes + 1)); fields[TSDB_SHOW_TABLES_METRIC_INDEX].bytes + 1));
} else {
pTableRecordInfo->belongStb = false;
}
break; break;
} }
...@@ -848,7 +881,7 @@ static int getTableRecordInfo( ...@@ -848,7 +881,7 @@ static int getTableRecordInfo(
while ((row = taos_fetch_row(result)) != NULL) { while ((row = taos_fetch_row(result)) != NULL) {
isSet = true; isSet = true;
pTableRecordInfo->isStable = true; pTableRecordInfo->isStb = true;
tstrncpy(pTableRecordInfo->tableRecord.stable, table, tstrncpy(pTableRecordInfo->tableRecord.stable, table,
TSDB_TABLE_NAME_LEN); TSDB_TABLE_NAME_LEN);
break; break;
...@@ -885,7 +918,6 @@ static int inDatabasesSeq( ...@@ -885,7 +918,6 @@ static int inDatabasesSeq(
dbname = strsep(&running, ","); dbname = strsep(&running, ",");
} }
} }
return -1; return -1;
...@@ -951,7 +983,213 @@ static int getDumpDbCount() ...@@ -951,7 +983,213 @@ static int getDumpDbCount()
return count; return count;
} }
static int64_t dumpNormalTableWithoutStb(SDbInfo *dbInfo, char *ntbName) static void dumpCreateMTableClause(
char* dbName,
char *stable,
TableDef *tableDes,
int numOfCols,
FILE *fp
) {
int counter = 0;
int count_temp = 0;
char* tmpBuf = (char *)malloc(COMMAND_SIZE);
if (tmpBuf == NULL) {
errorPrint("%s() LN%d, failed to allocate %d memory\n",
__func__, __LINE__, COMMAND_SIZE);
return;
}
char *pstr = NULL;
pstr = tmpBuf;
pstr += sprintf(tmpBuf,
"CREATE TABLE IF NOT EXISTS %s.%s USING %s.%s TAGS (",
dbName, tableDes->name, dbName, stable);
for (; counter < numOfCols; counter++) {
if (tableDes->cols[counter].note[0] != '\0') break;
}
assert(counter < numOfCols);
count_temp = counter;
for (; counter < numOfCols; counter++) {
if (counter != count_temp) {
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, ", \'%s\'", tableDes->cols[counter].note);
if (tableDes->cols[counter].var_value) {
pstr += sprintf(pstr, ", %s",
tableDes->cols[counter].var_value);
} else {
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].value);
}
} else {
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].value);
}
} else {
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].note);
if (tableDes->cols[counter].var_value) {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].var_value);
} else {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].value);
}
} else {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].value);
}
/* pstr += sprintf(pstr, "%s", tableDes->cols[counter].note); */
}
/* if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 || strcasecmp(tableDes->cols[counter].type, "nchar")
* == 0) { */
/* pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length); */
/* } */
}
pstr += sprintf(pstr, ");");
fprintf(fp, "%s\n", tmpBuf);
free(tmpBuf);
}
static int convertTbDesToAvroSchema(
char *dbName, char *tbName, TableDef *tableDes, int colCount,
char **avroSchema)
{
errorPrint("%s() LN%d TODO: covert table schema to avro schema\n",
__func__, __LINE__);
// {
// "namesapce": "database name",
// "type": "record",
// "name": "table name",
// "fields": [
// {
// "name": "col0 name",
// "type": "long"
// },
// {
// "name": "col1 name",
// "type": ["int", "null"]
// },
// {
// "name": "col2 name",
// "type": ["float", "null"]
// },
// ...
// {
// "name": "coln name",
// "type": ["string", "null"]
// }
// ]
// }
*avroSchema = (char *)calloc(1,
17 + TSDB_DB_NAME_LEN /* dbname section */
+ 17 /* type: record */
+ 11 + TSDB_TABLE_NAME_LEN /* tbname section */
+ 10 /* fields section */
+ (TSDB_COL_NAME_LEN + 11 + 16) * colCount + 4); /* fields section */
if (*avroSchema == NULL) {
errorPrint("%s() LN%d, memory allocation failed!\n", __func__, __LINE__);
return -1;
}
char *pstr = *avroSchema;
pstr += sprintf(pstr,
"{\"namespace\": \"%s\", \"type\": \"record\", \"name\": \"%s\", \"fields\": [",
dbName, tbName);
for (int i = 0; i < colCount; i ++) {
if (0 == i) {
pstr += sprintf(pstr,
"{\"name\": \"%s\", \"type\": \"%s\"",
tableDes->cols[i].field, "long");
} else {
if (strcasecmp(tableDes->cols[i].type, "binary") == 0 ||
strcasecmp(tableDes->cols[i].type, "nchar") == 0) {
pstr += sprintf(pstr,
"{\"name\": \"%s\", \"type\": [\"%s\", \"null\"]",
tableDes->cols[i].field, "string");
} else {
pstr += sprintf(pstr,
"{\"name\": \"%s\", \"type\": [\"%s\", \"null\"]",
tableDes->cols[i].field, tableDes->cols[i].type);
}
}
if ((i != (colCount -1))
&& (strcmp(tableDes->cols[i + 1].note, "TAG") != 0)) {
pstr += sprintf(pstr, "},");
} else {
pstr += sprintf(pstr, "}");
break;
}
}
pstr += sprintf(pstr, "]}");
debugPrint("%s() LN%d, avroSchema: %s\n", __func__, __LINE__, *avroSchema);
return 0;
}
static int64_t dumpNormalTable(
char *dbName,
char *stable,
char *tbName,
int precision,
FILE *fp
) {
int colCount = 0;
TableDef *tableDes = (TableDef *)calloc(1, sizeof(TableDef)
+ sizeof(ColDes) * TSDB_MAX_COLUMNS);
if (stable != NULL && stable[0] != '\0') { // dump table schema which is created by using super table
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
free(tableDes);
return -1;
}
// create child-table using super-table
dumpCreateMTableClause(dbName, stable, tableDes, colCount, fp);
} else { // dump table definition
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
free(tableDes);
return -1;
}
// create normal-table or super-table
dumpCreateTableClause(tableDes, colCount, fp, dbName);
}
char *jsonAvroSchema = NULL;
if (g_args.avro) {
if (0 != convertTbDesToAvroSchema(
dbName, tbName, tableDes, colCount, &jsonAvroSchema)) {
freeTbDes(tableDes);
return -1;
}
}
free(tableDes);
int64_t ret = 0;
if (!g_args.schemaonly) {
ret = dumpTableData(fp, tbName, dbName, precision,
jsonAvroSchema);
}
return ret;
}
static int64_t dumpNormalTableBelongStb(
SDbInfo *dbInfo, char *stbName, char *ntbName)
{ {
int64_t count = 0; int64_t count = 0;
...@@ -973,21 +1211,47 @@ static int64_t dumpNormalTableWithoutStb(SDbInfo *dbInfo, char *ntbName) ...@@ -973,21 +1211,47 @@ static int64_t dumpNormalTableWithoutStb(SDbInfo *dbInfo, char *ntbName)
return -1; return -1;
} }
count = taosDumpTable(ntbName, NULL, count = dumpNormalTable(
fp, dbInfo->name, getPrecisionByString(dbInfo->precision)); dbInfo->name,
stbName,
ntbName,
getPrecisionByString(dbInfo->precision),
fp);
fclose(fp); fclose(fp);
return count; return count;
} }
static int64_t dumpNormalTable(FILE *fp, TAOS *taos, char *dbName, char *tbName, static int64_t dumpNormalTableWithoutStb(SDbInfo *dbInfo, char *ntbName)
char *stbName,
int precision)
{ {
int64_t count = 0; int64_t count = 0;
count = taosDumpTable(tbName, stbName,
fp, dbName, precision);
char tmpBuf[4096] = {0};
FILE *fp = NULL;
if (g_args.outpath[0] != 0) {
sprintf(tmpBuf, "%s/%s.%s.sql",
g_args.outpath, dbInfo->name, ntbName);
} else {
sprintf(tmpBuf, "%s.%s.sql",
dbInfo->name, ntbName);
}
fp = fopen(tmpBuf, "w");
if (fp == NULL) {
errorPrint("%s() LN%d, failed to open file %s\n",
__func__, __LINE__, tmpBuf);
return -1;
}
count = dumpNormalTable(
dbInfo->name,
NULL,
ntbName,
getPrecisionByString(dbInfo->precision),
fp);
fclose(fp);
return count; return count;
} }
...@@ -1021,12 +1285,12 @@ static void *dumpNtbOfDb(void *arg) { ...@@ -1021,12 +1285,12 @@ static void *dumpNtbOfDb(void *arg) {
debugPrint("[%d] No.\t%"PRId64" table name: %s\n", debugPrint("[%d] No.\t%"PRId64" table name: %s\n",
pThreadInfo->threadIndex, i, pThreadInfo->threadIndex, i,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name); ((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name);
dumpNormalTable(fp, dumpNormalTable(
pThreadInfo->taos,
pThreadInfo->dbName, pThreadInfo->dbName,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name,
((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->stable, ((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->stable,
pThreadInfo->precision); ((TableInfo *)(g_tablesList + pThreadInfo->tableFrom+i))->name,
pThreadInfo->precision,
fp);
} }
fclose(fp); fclose(fp);
...@@ -1080,12 +1344,12 @@ static void *dumpNormalTablesOfStb(void *arg) { ...@@ -1080,12 +1344,12 @@ static void *dumpNormalTablesOfStb(void *arg) {
debugPrint("[%d] sub table %"PRId64": name: %s\n", debugPrint("[%d] sub table %"PRId64": name: %s\n",
pThreadInfo->threadIndex, i++, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX]); pThreadInfo->threadIndex, i++, (char *)row[TSDB_SHOW_TABLES_NAME_INDEX]);
dumpNormalTable(fp, dumpNormalTable(
pThreadInfo->taos,
pThreadInfo->dbName, pThreadInfo->dbName,
(char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
pThreadInfo->stbName, pThreadInfo->stbName,
pThreadInfo->precision); (char *)row[TSDB_SHOW_TABLES_NAME_INDEX],
pThreadInfo->precision,
fp);
} }
fclose(fp); fclose(fp);
...@@ -1271,6 +1535,34 @@ static int64_t dumpNtbOfStbByThreads( ...@@ -1271,6 +1535,34 @@ static int64_t dumpNtbOfStbByThreads(
return records; return records;
} }
static int dumpStableClasuse(SDbInfo *dbInfo, char *stbName, FILE *fp)
{
uint64_t sizeOfTableDes =
(uint64_t)(sizeof(TableDef) + sizeof(ColDes) * TSDB_MAX_COLUMNS);
TableDef *tableDes = (TableDef *)calloc(1, sizeOfTableDes);
if (NULL == tableDes) {
errorPrint("%s() LN%d, failed to allocate %"PRIu64" memory\n",
__func__, __LINE__, sizeOfTableDes);
exit(-1);
}
int colCount = getTableDes(dbInfo->name,
stbName, tableDes, true);
if (colCount < 0) {
free(tableDes);
errorPrint("%s() LN%d, failed to get stable[%s] schema\n",
__func__, __LINE__, stbName);
exit(-1);
}
dumpCreateTableClause(tableDes, colCount, fp, dbInfo->name);
free(tableDes);
return 0;
}
static int64_t dumpCreateSTableClauseOfDb( static int64_t dumpCreateSTableClauseOfDb(
SDbInfo *dbInfo, FILE *fp) SDbInfo *dbInfo, FILE *fp)
{ {
...@@ -1300,7 +1592,7 @@ static int64_t dumpCreateSTableClauseOfDb( ...@@ -1300,7 +1592,7 @@ static int64_t dumpCreateSTableClauseOfDb(
int64_t superTblCnt = 0; int64_t superTblCnt = 0;
while ((row = taos_fetch_row(res)) != NULL) { while ((row = taos_fetch_row(res)) != NULL) {
if (0 == dumpStable(row[TSDB_SHOW_TABLES_NAME_INDEX], fp, dbInfo)) { if (0 == dumpStableClasuse(dbInfo, row[TSDB_SHOW_TABLES_NAME_INDEX], fp)) {
superTblCnt ++; superTblCnt ++;
} }
} }
...@@ -1382,7 +1674,7 @@ static int64_t dumpNTablesOfDb(SDbInfo *dbInfo) ...@@ -1382,7 +1674,7 @@ static int64_t dumpNTablesOfDb(SDbInfo *dbInfo)
static int64_t dumpWholeDatabase(SDbInfo *dbInfo, FILE *fp) static int64_t dumpWholeDatabase(SDbInfo *dbInfo, FILE *fp)
{ {
taosDumpCreateDbClause(dbInfo, g_args.with_property, fp); dumpCreateDbClause(dbInfo, g_args.with_property, fp);
fprintf(g_fpOfResult, "\n#### database: %s\n", fprintf(g_fpOfResult, "\n#### database: %s\n",
dbInfo->name); dbInfo->name);
...@@ -1393,14 +1685,13 @@ static int64_t dumpWholeDatabase(SDbInfo *dbInfo, FILE *fp) ...@@ -1393,14 +1685,13 @@ static int64_t dumpWholeDatabase(SDbInfo *dbInfo, FILE *fp)
return dumpNTablesOfDb(dbInfo); return dumpNTablesOfDb(dbInfo);
} }
static int taosDumpOut() { static int dumpOut() {
TAOS *taos = NULL; TAOS *taos = NULL;
TAOS_RES *result = NULL; TAOS_RES *result = NULL;
TAOS_ROW row; TAOS_ROW row;
FILE *fp = NULL; FILE *fp = NULL;
int32_t count = 0; int32_t count = 0;
TableRecordInfo tableRecordInfo;
char tmpBuf[4096] = {0}; char tmpBuf[4096] = {0};
if (g_args.outpath[0] != 0) { if (g_args.outpath[0] != 0) {
...@@ -1446,7 +1737,7 @@ static int taosDumpOut() { ...@@ -1446,7 +1737,7 @@ static int taosDumpOut() {
/* --------------------------------- Main Code -------------------------------- */ /* --------------------------------- Main Code -------------------------------- */
/* if (g_args.databases || g_args.all_databases) { // dump part of databases or all databases */ /* if (g_args.databases || g_args.all_databases) { // dump part of databases or all databases */
/* */ /* */
taosDumpCharset(fp); dumpCharset(fp);
sprintf(command, "show databases"); sprintf(command, "show databases");
result = taos_query(taos, command); result = taos_query(taos, command);
...@@ -1568,11 +1859,13 @@ static int taosDumpOut() { ...@@ -1568,11 +1859,13 @@ static int taosDumpOut() {
g_totalDumpOutRows += records; g_totalDumpOutRows += records;
} }
} else { } else {
taosDumpCreateDbClause(g_dbInfos[0], g_args.with_property, fp); dumpCreateDbClause(g_dbInfos[0], g_args.with_property, fp);
} }
int superTblCnt = 0 ; int superTblCnt = 0 ;
for (int i = 1; g_args.arg_list[i]; i++) { for (int i = 1; g_args.arg_list[i]; i++) {
TableRecordInfo tableRecordInfo;
if (getTableRecordInfo(g_dbInfos[0]->name, if (getTableRecordInfo(g_dbInfos[0]->name,
g_args.arg_list[i], g_args.arg_list[i],
&tableRecordInfo) < 0) { &tableRecordInfo) < 0) {
...@@ -1582,14 +1875,24 @@ static int taosDumpOut() { ...@@ -1582,14 +1875,24 @@ static int taosDumpOut() {
} }
int64_t records = 0; int64_t records = 0;
if (tableRecordInfo.isStable) { // dump all table of this stable if (tableRecordInfo.isStb) { // dump all table of this stable
int ret = dumpStable( int ret = dumpStableClasuse(
g_dbInfos[0],
tableRecordInfo.tableRecord.stable, tableRecordInfo.tableRecord.stable,
fp, g_dbInfos[0]); fp);
if (ret >= 0) { if (ret >= 0) {
superTblCnt++; superTblCnt++;
records = dumpNtbOfStbByThreads(g_dbInfos[0], g_args.arg_list[i]); records = dumpNtbOfStbByThreads(g_dbInfos[0], g_args.arg_list[i]);
} }
} else if (tableRecordInfo.belongStb){
dumpStableClasuse(
g_dbInfos[0],
tableRecordInfo.tableRecord.stable,
fp);
records = dumpNormalTableBelongStb(
g_dbInfos[0],
tableRecordInfo.tableRecord.stable,
g_args.arg_list[i]);
} else { } else {
records = dumpNormalTableWithoutStb(g_dbInfos[0], g_args.arg_list[i]); records = dumpNormalTableWithoutStb(g_dbInfos[0], g_args.arg_list[i]);
} }
...@@ -1604,7 +1907,7 @@ static int taosDumpOut() { ...@@ -1604,7 +1907,7 @@ static int taosDumpOut() {
/* Close the handle and return */ /* Close the handle and return */
fclose(fp); fclose(fp);
taos_free_result(result); taos_free_result(result);
taosFreeDbInfos(); freeDbInfos();
fprintf(stderr, "dump out rows: %" PRId64 "\n", g_totalDumpOutRows); fprintf(stderr, "dump out rows: %" PRId64 "\n", g_totalDumpOutRows);
return 0; return 0;
...@@ -1612,14 +1915,14 @@ _exit_failure: ...@@ -1612,14 +1915,14 @@ _exit_failure:
fclose(fp); fclose(fp);
taos_close(taos); taos_close(taos);
taos_free_result(result); taos_free_result(result);
taosFreeDbInfos(); freeDbInfos();
errorPrint("dump out rows: %" PRId64 "\n", g_totalDumpOutRows); errorPrint("dump out rows: %" PRId64 "\n", g_totalDumpOutRows);
return -1; return -1;
} }
static int getTableDes( static int getTableDes(
char* dbName, char *table, char* dbName, char *table,
STableDef *stableDes, bool isSuperTable) { TableDef *tableDes, bool isSuperTable) {
TAOS_ROW row = NULL; TAOS_ROW row = NULL;
TAOS_RES* res = NULL; TAOS_RES* res = NULL;
int colCount = 0; int colCount = 0;
...@@ -1648,22 +1951,21 @@ static int getTableDes( ...@@ -1648,22 +1951,21 @@ static int getTableDes(
TAOS_FIELD *fields = taos_fetch_fields(res); TAOS_FIELD *fields = taos_fetch_fields(res);
tstrncpy(stableDes->name, table, TSDB_TABLE_NAME_LEN); tstrncpy(tableDes->name, table, TSDB_TABLE_NAME_LEN);
while ((row = taos_fetch_row(res)) != NULL) { while ((row = taos_fetch_row(res)) != NULL) {
tstrncpy(stableDes->cols[colCount].field, tstrncpy(tableDes->cols[colCount].field,
(char *)row[TSDB_DESCRIBE_METRIC_FIELD_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_FIELD_INDEX],
min(TSDB_COL_NAME_LEN + 1, min(TSDB_COL_NAME_LEN + 1,
fields[TSDB_DESCRIBE_METRIC_FIELD_INDEX].bytes + 1)); fields[TSDB_DESCRIBE_METRIC_FIELD_INDEX].bytes + 1));
tstrncpy(stableDes->cols[colCount].type, tstrncpy(tableDes->cols[colCount].type,
(char *)row[TSDB_DESCRIBE_METRIC_TYPE_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_TYPE_INDEX],
min(16, fields[TSDB_DESCRIBE_METRIC_TYPE_INDEX].bytes + 1)); min(16, fields[TSDB_DESCRIBE_METRIC_TYPE_INDEX].bytes + 1));
stableDes->cols[colCount].length = tableDes->cols[colCount].length =
*((int *)row[TSDB_DESCRIBE_METRIC_LENGTH_INDEX]); *((int *)row[TSDB_DESCRIBE_METRIC_LENGTH_INDEX]);
tstrncpy(stableDes->cols[colCount].note, tstrncpy(tableDes->cols[colCount].note,
(char *)row[TSDB_DESCRIBE_METRIC_NOTE_INDEX], (char *)row[TSDB_DESCRIBE_METRIC_NOTE_INDEX],
min(COL_NOTE_LEN, min(COL_NOTE_LEN,
fields[TSDB_DESCRIBE_METRIC_NOTE_INDEX].bytes + 1)); fields[TSDB_DESCRIBE_METRIC_NOTE_INDEX].bytes + 1));
colCount++; colCount++;
} }
...@@ -1676,10 +1978,10 @@ static int getTableDes( ...@@ -1676,10 +1978,10 @@ static int getTableDes(
// if child-table have tag, using select tagName from table to get tagValue // if child-table have tag, using select tagName from table to get tagValue
for (int i = 0 ; i < colCount; i++) { for (int i = 0 ; i < colCount; i++) {
if (strcmp(stableDes->cols[i].note, "TAG") != 0) continue; if (strcmp(tableDes->cols[i].note, "TAG") != 0) continue;
sprintf(sqlstr, "select %s from %s.%s", sprintf(sqlstr, "select %s from %s.%s",
stableDes->cols[i].field, dbName, table); tableDes->cols[i].field, dbName, table);
res = taos_query(taos, sqlstr); res = taos_query(taos, sqlstr);
code = taos_errno(res); code = taos_errno(res);
...@@ -1703,7 +2005,7 @@ static int getTableDes( ...@@ -1703,7 +2005,7 @@ static int getTableDes(
} }
if (row[TSDB_SHOW_TABLES_NAME_INDEX] == NULL) { if (row[TSDB_SHOW_TABLES_NAME_INDEX] == NULL) {
sprintf(stableDes->cols[i].note, "%s", "NULL"); sprintf(tableDes->cols[i].note, "%s", "NUL");
taos_free_result(res); taos_free_result(res);
res = NULL; res = NULL;
taos_close(taos); taos_close(taos);
...@@ -1715,58 +2017,82 @@ static int getTableDes( ...@@ -1715,58 +2017,82 @@ static int getTableDes(
//int32_t* length = taos_fetch_lengths(tmpResult); //int32_t* length = taos_fetch_lengths(tmpResult);
switch (fields[0].type) { switch (fields[0].type) {
case TSDB_DATA_TYPE_BOOL: case TSDB_DATA_TYPE_BOOL:
sprintf(stableDes->cols[i].note, "%d", sprintf(tableDes->cols[i].value, "%d",
((((int32_t)(*((char *)row[TSDB_SHOW_TABLES_NAME_INDEX]))) == 1) ? 1 : 0)); ((((int32_t)(*((char *)row[TSDB_SHOW_TABLES_NAME_INDEX]))) == 1) ? 1 : 0));
break; break;
case TSDB_DATA_TYPE_TINYINT: case TSDB_DATA_TYPE_TINYINT:
sprintf(stableDes->cols[i].note, "%d", sprintf(tableDes->cols[i].value, "%d",
*((int8_t *)row[TSDB_SHOW_TABLES_NAME_INDEX])); *((int8_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_SMALLINT: case TSDB_DATA_TYPE_SMALLINT:
sprintf(stableDes->cols[i].note, "%d", *((int16_t *)row[TSDB_SHOW_TABLES_NAME_INDEX])); sprintf(tableDes->cols[i].value, "%d",
*((int16_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_INT: case TSDB_DATA_TYPE_INT:
sprintf(stableDes->cols[i].note, "%d", *((int32_t *)row[TSDB_SHOW_TABLES_NAME_INDEX])); sprintf(tableDes->cols[i].value, "%d",
*((int32_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_BIGINT: case TSDB_DATA_TYPE_BIGINT:
sprintf(stableDes->cols[i].note, "%" PRId64 "", *((int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX])); sprintf(tableDes->cols[i].value, "%" PRId64 "",
*((int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_FLOAT: case TSDB_DATA_TYPE_FLOAT:
sprintf(stableDes->cols[i].note, "%f", GET_FLOAT_VAL(row[TSDB_SHOW_TABLES_NAME_INDEX])); sprintf(tableDes->cols[i].value, "%f",
GET_FLOAT_VAL(row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_DOUBLE: case TSDB_DATA_TYPE_DOUBLE:
sprintf(stableDes->cols[i].note, "%f", GET_DOUBLE_VAL(row[TSDB_SHOW_TABLES_NAME_INDEX])); sprintf(tableDes->cols[i].value, "%f",
GET_DOUBLE_VAL(row[TSDB_SHOW_TABLES_NAME_INDEX]));
break; break;
case TSDB_DATA_TYPE_BINARY: case TSDB_DATA_TYPE_BINARY:
{ memset(tableDes->cols[i].value, 0,
memset(stableDes->cols[i].note, 0, sizeof(stableDes->cols[i].note)); sizeof(tableDes->cols[i].value));
stableDes->cols[i].note[0] = '\''; int len = strlen((char *)row[0]);
char tbuf[COL_NOTE_LEN]; // FIXME for long value
converStringToReadable((char *)row[TSDB_SHOW_TABLES_NAME_INDEX], length[0], tbuf, COL_NOTE_LEN); if (len < (COL_VALUEBUF_LEN - 2)) {
char* pstr = stpcpy(&(stableDes->cols[i].note[1]), tbuf); tableDes->cols[i].value[0] = '\'';
*(pstr++) = '\''; converStringToReadable(
break; (char *)row[0],
length[0],
tableDes->cols[i].value + 1,
len);
tableDes->cols[i].value[len+1] = '\'';
} else {
tableDes->cols[i].var_value = calloc(1, len + 2);
if (tableDes->cols[i].var_value == NULL) {
errorPrint("%s() LN%d, memory alalocation failed!\n",
__func__, __LINE__);
taos_free_result(res);
return -1;
}
tableDes->cols[i].var_value[0] = '\'';
converStringToReadable((char *)row[0],
length[0],
(char *)(tableDes->cols[i].var_value + 1), len);
tableDes->cols[i].var_value[len+1] = '\'';
} }
break;
case TSDB_DATA_TYPE_NCHAR: case TSDB_DATA_TYPE_NCHAR:
{ {
memset(stableDes->cols[i].note, 0, sizeof(stableDes->cols[i].note)); memset(tableDes->cols[i].value, 0, sizeof(tableDes->cols[i].note));
char tbuf[COL_NOTE_LEN-2]; // need reserve 2 bytes for ' ' char tbuf[COL_NOTE_LEN-2]; // need reserve 2 bytes for ' '
convertNCharToReadable((char *)row[TSDB_SHOW_TABLES_NAME_INDEX], length[0], tbuf, COL_NOTE_LEN); convertNCharToReadable((char *)row[TSDB_SHOW_TABLES_NAME_INDEX], length[0], tbuf, COL_NOTE_LEN);
sprintf(stableDes->cols[i].note, "\'%s\'", tbuf); sprintf(tableDes->cols[i].value, "\'%s\'", tbuf);
break; break;
} }
case TSDB_DATA_TYPE_TIMESTAMP: case TSDB_DATA_TYPE_TIMESTAMP:
sprintf(stableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]); sprintf(tableDes->cols[i].value, "%" PRId64 "", *(int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]);
#if 0 #if 0
if (!g_args.mysqlFlag) { if (!g_args.mysqlFlag) {
sprintf(tableDes->cols[i].note, "%" PRId64 "", *(int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]); sprintf(tableDes->cols[i].value, "%" PRId64 "", *(int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]);
} else { } else {
char buf[64] = "\0"; char buf[64] = "\0";
int64_t ts = *((int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]); int64_t ts = *((int64_t *)row[TSDB_SHOW_TABLES_NAME_INDEX]);
time_t tt = (time_t)(ts / 1000); time_t tt = (time_t)(ts / 1000);
struct tm *ptm = localtime(&tt); struct tm *ptm = localtime(&tt);
strftime(buf, 64, "%y-%m-%d %H:%M:%S", ptm); strftime(buf, 64, "%y-%m-%d %H:%M:%S", ptm);
sprintf(tableDes->cols[i].note, "\'%s.%03d\'", buf, (int)(ts % 1000)); sprintf(tableDes->cols[i].value, "\'%s.%03d\'", buf, (int)(ts % 1000));
} }
#endif #endif
break; break;
...@@ -1781,74 +2107,7 @@ static int getTableDes( ...@@ -1781,74 +2107,7 @@ static int getTableDes(
return colCount; return colCount;
} }
static int convertSchemaToAvroSchema(STableDef *stableDes, char **avroSchema) static void dumpCreateDbClause(
{
errorPrint("%s() LN%d TODO: covert table schema to avro schema\n",
__func__, __LINE__);
return 0;
}
static int64_t taosDumpTable(
char *tbName, char *stable,
FILE *fp, char* dbName, int precision) {
int colCount = 0;
STableDef *tableDes = (STableDef *)calloc(1, sizeof(STableDef)
+ sizeof(SColDes) * TSDB_MAX_COLUMNS);
if (stable != NULL && stable[0] != '\0') { // dump table schema which is created by using super table
/*
colCount = getTableDes(stable, tableDes, taos);
if (count < 0) {
free(tableDes);
return -1;
}
dumpCreateTableClause(tableDes, count, fp);
memset(tableDes, 0, sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS);
*/
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
free(tableDes);
return -1;
}
// create child-table using super-table
taosDumpCreateMTableClause(tableDes, stable, colCount, fp, dbName);
} else { // dump table definition
colCount = getTableDes(dbName, tbName, tableDes, false);
if (colCount < 0) {
free(tableDes);
return -1;
}
// create normal-table or super-table
dumpCreateTableClause(tableDes, colCount, fp, dbName);
}
char *jsonAvroSchema = NULL;
if (g_args.avro) {
convertSchemaToAvroSchema(tableDes, &jsonAvroSchema);
}
free(tableDes);
int64_t ret = 0;
if (!g_args.schemaonly) {
ret = dumpTableData(fp, tbName, dbName, precision,
jsonAvroSchema);
}
return ret;
}
static void taosDumpCreateDbClause(
SDbInfo *dbInfo, bool isDumpProperty, FILE *fp) { SDbInfo *dbInfo, bool isDumpProperty, FILE *fp) {
char sqlstr[TSDB_MAX_SQL_LEN] = {0}; char sqlstr[TSDB_MAX_SQL_LEN] = {0};
...@@ -1870,35 +2129,7 @@ static void taosDumpCreateDbClause( ...@@ -1870,35 +2129,7 @@ static void taosDumpCreateDbClause(
fprintf(fp, "%s\n\n", sqlstr); fprintf(fp, "%s\n\n", sqlstr);
} }
static int dumpStable(char *stbName, FILE *fp, SDbInfo *dbInfo) static int dumpCreateTableClause(TableDef *tableDes, int numOfCols,
{
uint64_t sizeOfTableDes =
(uint64_t)(sizeof(STableDef) + sizeof(SColDes) * TSDB_MAX_COLUMNS);
STableDef *stableDes = (STableDef *)calloc(1, sizeOfTableDes);
if (NULL == stableDes) {
errorPrint("%s() LN%d, failed to allocate %"PRIu64" memory\n",
__func__, __LINE__, sizeOfTableDes);
exit(-1);
}
int colCount = getTableDes(dbInfo->name,
stbName, stableDes, true);
if (colCount < 0) {
free(stableDes);
errorPrint("%s() LN%d, failed to get stable[%s] schema\n",
__func__, __LINE__, stbName);
exit(-1);
}
dumpCreateTableClause(stableDes, colCount, fp, dbInfo->name);
free(stableDes);
return 0;
}
static int dumpCreateTableClause(STableDef *tableDes, int numOfCols,
FILE *fp, char* dbName) { FILE *fp, char* dbName) {
int counter = 0; int counter = 0;
int count_temp = 0; int count_temp = 0;
...@@ -1949,64 +2180,6 @@ static int dumpCreateTableClause(STableDef *tableDes, int numOfCols, ...@@ -1949,64 +2180,6 @@ static int dumpCreateTableClause(STableDef *tableDes, int numOfCols,
return fprintf(fp, "%s\n\n", sqlstr); return fprintf(fp, "%s\n\n", sqlstr);
} }
static void taosDumpCreateMTableClause(STableDef *tableDes, char *stable,
int numOfCols, FILE *fp, char* dbName) {
int counter = 0;
int count_temp = 0;
char* tmpBuf = (char *)malloc(COMMAND_SIZE);
if (tmpBuf == NULL) {
errorPrint("%s() LN%d, failed to allocate %d memory\n",
__func__, __LINE__, COMMAND_SIZE);
return;
}
char *pstr = NULL;
pstr = tmpBuf;
pstr += sprintf(tmpBuf,
"CREATE TABLE IF NOT EXISTS %s.%s USING %s.%s TAGS (",
dbName, tableDes->name, dbName, stable);
for (; counter < numOfCols; counter++) {
if (tableDes->cols[counter].note[0] != '\0') break;
}
assert(counter < numOfCols);
count_temp = counter;
for (; counter < numOfCols; counter++) {
if (counter != count_temp) {
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, ", \'%s\'", tableDes->cols[counter].note);
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].note);
} else {
pstr += sprintf(pstr, ", %s", tableDes->cols[counter].note);
}
} else {
if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 ||
strcasecmp(tableDes->cols[counter].type, "nchar") == 0) {
//pstr += sprintf(pstr, "\'%s\'", tableDes->cols[counter].note);
pstr += sprintf(pstr, "%s", tableDes->cols[counter].note);
} else {
pstr += sprintf(pstr, "%s", tableDes->cols[counter].note);
}
/* pstr += sprintf(pstr, "%s", tableDes->cols[counter].note); */
}
/* if (strcasecmp(tableDes->cols[counter].type, "binary") == 0 || strcasecmp(tableDes->cols[counter].type, "nchar")
* == 0) { */
/* pstr += sprintf(pstr, "(%d)", tableDes->cols[counter].length); */
/* } */
}
pstr += sprintf(pstr, ");");
fprintf(fp, "%s\n", tmpBuf);
free(tmpBuf);
}
static int writeSchemaToAvro(char *jsonAvroSchema) static int writeSchemaToAvro(char *jsonAvroSchema)
{ {
errorPrint("%s() LN%d, TODO: implement write schema to avro", errorPrint("%s() LN%d, TODO: implement write schema to avro",
...@@ -2103,10 +2276,7 @@ static int64_t writeResultToSql(TAOS_RES *res, FILE *fp, char *dbName, char *tbN ...@@ -2103,10 +2276,7 @@ static int64_t writeResultToSql(TAOS_RES *res, FILE *fp, char *dbName, char *tbN
case TSDB_DATA_TYPE_BINARY: case TSDB_DATA_TYPE_BINARY:
{ {
char tbuf[COMMAND_SIZE] = {0}; char tbuf[COMMAND_SIZE] = {0};
//*(pstr++) = '\'';
converStringToReadable((char *)row[col], length[col], tbuf, COMMAND_SIZE); converStringToReadable((char *)row[col], length[col], tbuf, COMMAND_SIZE);
//pstr = stpcpy(pstr, tbuf);
//*(pstr++) = '\'';
curr_sqlstr_len += sprintf(pstr + curr_sqlstr_len, "\'%s\'", tbuf); curr_sqlstr_len += sprintf(pstr + curr_sqlstr_len, "\'%s\'", tbuf);
break; break;
} }
...@@ -2362,7 +2532,6 @@ static int converStringToReadable(char *str, int size, char *buf, int bufsize) { ...@@ -2362,7 +2532,6 @@ static int converStringToReadable(char *str, int size, char *buf, int bufsize) {
static int convertNCharToReadable(char *str, int size, char *buf, int bufsize) { static int convertNCharToReadable(char *str, int size, char *buf, int bufsize) {
char *pstr = str; char *pstr = str;
char *pbuf = buf; char *pbuf = buf;
// TODO
wchar_t wc; wchar_t wc;
while (size > 0) { while (size > 0) {
if (*pstr == '\0') break; if (*pstr == '\0') break;
...@@ -2386,7 +2555,7 @@ static int convertNCharToReadable(char *str, int size, char *buf, int bufsize) { ...@@ -2386,7 +2555,7 @@ static int convertNCharToReadable(char *str, int size, char *buf, int bufsize) {
return 0; return 0;
} }
static void taosDumpCharset(FILE *fp) { static void dumpCharset(FILE *fp) {
char charsetline[256]; char charsetline[256];
(void)fseek(fp, 0, SEEK_SET); (void)fseek(fp, 0, SEEK_SET);
...@@ -2394,7 +2563,7 @@ static void taosDumpCharset(FILE *fp) { ...@@ -2394,7 +2563,7 @@ static void taosDumpCharset(FILE *fp) {
(void)fwrite(charsetline, strlen(charsetline), 1, fp); (void)fwrite(charsetline, strlen(charsetline), 1, fp);
} }
static void taosLoadFileCharset(FILE *fp, char *fcharset) { static void loadFileCharset(FILE *fp, char *fcharset) {
char * line = NULL; char * line = NULL;
size_t line_size = 0; size_t line_size = 0;
...@@ -2526,7 +2695,7 @@ static void taosMallocDumpFiles() ...@@ -2526,7 +2695,7 @@ static void taosMallocDumpFiles()
} }
} }
static void taosFreeDumpFiles() static void freeDumpFiles()
{ {
for (int i = 0; i < g_tsSqlFileNum; i++) { for (int i = 0; i < g_tsSqlFileNum; i++) {
tfree(g_tsDumpInSqlFiles[i]); tfree(g_tsDumpInSqlFiles[i]);
...@@ -2594,7 +2763,7 @@ static FILE* taosOpenDumpInFile(char *fptr) { ...@@ -2594,7 +2763,7 @@ static FILE* taosOpenDumpInFile(char *fptr) {
return f; return f;
} }
static int taosDumpInOneFile(TAOS* taos, FILE* fp, char* fcharset, static int dumpInOneFile(TAOS* taos, FILE* fp, char* fcharset,
char* encode, char* fileName) { char* encode, char* fileName) {
int read_len = 0; int read_len = 0;
char * cmd = NULL; char * cmd = NULL;
...@@ -2651,7 +2820,7 @@ static int taosDumpInOneFile(TAOS* taos, FILE* fp, char* fcharset, ...@@ -2651,7 +2820,7 @@ static int taosDumpInOneFile(TAOS* taos, FILE* fp, char* fcharset,
return 0; return 0;
} }
static void* taosDumpInWorkThreadFp(void *arg) static void* dumpInWorkThreadFp(void *arg)
{ {
threadInfo *pThread = (threadInfo*)arg; threadInfo *pThread = (threadInfo*)arg;
setThreadName("dumpInWorkThrd"); setThreadName("dumpInWorkThrd");
...@@ -2665,14 +2834,14 @@ static void* taosDumpInWorkThreadFp(void *arg) ...@@ -2665,14 +2834,14 @@ static void* taosDumpInWorkThreadFp(void *arg)
} }
fprintf(stderr, ", Success Open input file: %s\n", fprintf(stderr, ", Success Open input file: %s\n",
SQLFileName); SQLFileName);
taosDumpInOneFile(pThread->taos, fp, g_tsCharset, g_args.encode, SQLFileName); dumpInOneFile(pThread->taos, fp, g_tsCharset, g_args.encode, SQLFileName);
} }
} }
return NULL; return NULL;
} }
static void taosStartDumpInWorkThreads() static void startDumpInWorkThreads()
{ {
pthread_attr_t thattr; pthread_attr_t thattr;
threadInfo *pThread; threadInfo *pThread;
...@@ -2704,7 +2873,7 @@ static void taosStartDumpInWorkThreads() ...@@ -2704,7 +2873,7 @@ static void taosStartDumpInWorkThreads()
pthread_attr_setdetachstate(&thattr, PTHREAD_CREATE_JOINABLE); pthread_attr_setdetachstate(&thattr, PTHREAD_CREATE_JOINABLE);
if (pthread_create(&(pThread->threadID), &thattr, if (pthread_create(&(pThread->threadID), &thattr,
taosDumpInWorkThreadFp, (void*)pThread) != 0) { dumpInWorkThreadFp, (void*)pThread) != 0) {
errorPrint("%s() LN%d, thread:%d failed to start\n", errorPrint("%s() LN%d, thread:%d failed to start\n",
__func__, __LINE__, pThread->threadIndex); __func__, __LINE__, pThread->threadIndex);
exit(0); exit(0);
...@@ -2721,7 +2890,7 @@ static void taosStartDumpInWorkThreads() ...@@ -2721,7 +2890,7 @@ static void taosStartDumpInWorkThreads()
free(threadObj); free(threadObj);
} }
static int taosDumpIn() { static int dumpIn() {
assert(g_args.isDumpIn); assert(g_args.isDumpIn);
TAOS *taos = NULL; TAOS *taos = NULL;
...@@ -2750,19 +2919,19 @@ static int taosDumpIn() { ...@@ -2750,19 +2919,19 @@ static int taosDumpIn() {
} }
fprintf(stderr, "Success Open input file: %s\n", g_tsDbSqlFile); fprintf(stderr, "Success Open input file: %s\n", g_tsDbSqlFile);
taosLoadFileCharset(fp, g_tsCharset); loadFileCharset(fp, g_tsCharset);
taosDumpInOneFile(taos, fp, g_tsCharset, g_args.encode, dumpInOneFile(taos, fp, g_tsCharset, g_args.encode,
g_tsDbSqlFile); g_tsDbSqlFile);
} }
taos_close(taos); taos_close(taos);
if (0 != tsSqlFileNumOfTbls) { if (0 != tsSqlFileNumOfTbls) {
taosStartDumpInWorkThreads(); startDumpInWorkThreads();
} }
taosFreeDumpFiles(); freeDumpFiles();
return 0; return 0;
} }
...@@ -2883,7 +3052,7 @@ int main(int argc, char *argv[]) { ...@@ -2883,7 +3052,7 @@ int main(int argc, char *argv[]) {
fprintf(g_fpOfResult, "# DumpIn start time: %d-%02d-%02d %02d:%02d:%02d\n", fprintf(g_fpOfResult, "# DumpIn start time: %d-%02d-%02d %02d:%02d:%02d\n",
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_year + 1900, tm.tm_mon + 1,
tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec); tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
if (taosDumpIn() < 0) { if (dumpIn() < 0) {
ret = -1; ret = -1;
} }
} else { } else {
...@@ -2891,7 +3060,7 @@ int main(int argc, char *argv[]) { ...@@ -2891,7 +3060,7 @@ int main(int argc, char *argv[]) {
fprintf(g_fpOfResult, "# DumpOut start time: %d-%02d-%02d %02d:%02d:%02d\n", fprintf(g_fpOfResult, "# DumpOut start time: %d-%02d-%02d %02d:%02d:%02d\n",
tm.tm_year + 1900, tm.tm_mon + 1, tm.tm_year + 1900, tm.tm_mon + 1,
tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec); tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
if (taosDumpOut() < 0) { if (dumpOut() < 0) {
ret = -1; ret = -1;
} else { } else {
fprintf(g_fpOfResult, "\n============================== TOTAL STATISTICS ============================== \n"); fprintf(g_fpOfResult, "\n============================== TOTAL STATISTICS ============================== \n");
......
src/plugins/CMakeLists.txt
浏览文件 @ 6b7e4a85
...@@ -2,7 +2,11 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.8...3.20) ...@@ -2,7 +2,11 @@ CMAKE_MINIMUM_REQUIRED(VERSION 2.8...3.20)
PROJECT(TDengine) PROJECT(TDengine)
ADD_SUBDIRECTORY(monitor) ADD_SUBDIRECTORY(monitor)
ADD_SUBDIRECTORY(http)
IF (TD_BUILD_HTTP)
ADD_SUBDIRECTORY(http)
ENDIF ()
IF (TD_LINUX AND TD_MQTT) IF (TD_LINUX AND TD_MQTT)
ADD_SUBDIRECTORY(mqtt) ADD_SUBDIRECTORY(mqtt)
ENDIF () ENDIF ()
tests/connectorTest/C#Test/nanosupport/nanotest.cs
浏览文件 @ 6b7e4a85
...@@ -29,21 +29,7 @@ namespace TDengineDriver ...@@ -29,21 +29,7 @@ namespace TDengineDriver
private string password="taosdata"; private string password="taosdata";
private short port = 0; private short port = 0;
//sql parameters
private string dbName;
private string tbName;
private string precision;
private bool isInsertData;
private bool isQueryData;
private long tableCount;
private long totalRows;
private long batchRows;
private long beginTimestamp = 1551369600000L;
private IntPtr conn = IntPtr.Zero; private IntPtr conn = IntPtr.Zero;
private long rowsInserted = 0;
static void Main(string[] args) static void Main(string[] args)
{ {
...@@ -73,15 +59,6 @@ namespace TDengineDriver ...@@ -73,15 +59,6 @@ namespace TDengineDriver
tester.executeQuery("select * from tb;"); tester.executeQuery("select * from tb;");
// Console.WriteLine("expected is : {0}", width);
// tdSql.checkData(0,0,"2021-06-10 0:00:00.100000001");
// tdSql.checkData(1,0,"2021-06-10 0:00:00.150000000");
// tdSql.checkData(2,0,"2021-06-10 0:00:00.299999999");
// tdSql.checkData(3,1,3);
// tdSql.checkData(4,1,5);
// tdSql.checkData(5,1,7);
// tdSql.checkRows(6);
tester.executeQuery("select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400100000002;"); tester.executeQuery("select count(*) from tb where ts > 1623254400100000000 and ts < 1623254400100000002;");
Console.WriteLine("expected is : 1 " ); Console.WriteLine("expected is : 1 " );
tester.executeQuery("select count(*) from tb where ts > '2021-06-10 0:00:00.100000001' and ts < '2021-06-10 0:00:00.160000000';"); tester.executeQuery("select count(*) from tb where ts > '2021-06-10 0:00:00.100000001' and ts < '2021-06-10 0:00:00.160000000';");
...@@ -271,8 +248,8 @@ namespace TDengineDriver ...@@ -271,8 +248,8 @@ namespace TDengineDriver
// tdSql.checkData(0,0,1623258000123456789); // tdSql.checkData(0,0,1623258000123456789);
Console.WriteLine("usdb" ); Console.WriteLine("usdb" );
tester.execute("drop database if exists usdb;");
tester.execute("create database usdb precision 'us';"); tester.execute("create database usdb precision 'us';");
tester.execute("use usdb;"); tester.execute("use usdb;");
tester.execute("create stable st (ts timestamp ,speed float ) tags(time timestamp ,id int);"); tester.execute("create stable st (ts timestamp ,speed float ) tags(time timestamp ,id int);");
...@@ -290,12 +267,8 @@ namespace TDengineDriver ...@@ -290,12 +267,8 @@ namespace TDengineDriver
tester.executeQuery("select first(*) from tb1;"); tester.executeQuery("select first(*) from tb1;");
Console.WriteLine("expected is : 1623258000123 " ); Console.WriteLine("expected is : 1623258000123 " );
tester.CloseConnection(); tester.CloseConnection();
tester.cleanup(); tester.cleanup();
} }
public void InitTDengine() public void InitTDengine()
...@@ -321,13 +294,13 @@ namespace TDengineDriver ...@@ -321,13 +294,13 @@ namespace TDengineDriver
Console.WriteLine("[ OK ] Connection established."); Console.WriteLine("[ OK ] Connection established.");
} }
} }
//EXECUTE SQL //EXECUTE SQL
public void execute(string sql) public void execute(string sql)
{ {
DateTime dt1 = DateTime.Now; DateTime dt1 = DateTime.Now;
IntPtr res = TDengine.Query(this.conn, sql.ToString()); IntPtr res = TDengine.Query(this.conn, sql.ToString());
DateTime dt2 = DateTime.Now; DateTime dt2 = DateTime.Now;
TimeSpan span = dt2 - dt1;
if ((res == IntPtr.Zero) || (TDengine.ErrorNo(res) != 0)) if ((res == IntPtr.Zero) || (TDengine.ErrorNo(res) != 0))
{ {
...@@ -481,22 +454,16 @@ namespace TDengineDriver ...@@ -481,22 +454,16 @@ namespace TDengineDriver
switch(psc) switch(psc)
{ {
case 0: case 0:
Console.WriteLine("db：[{0:G}]'s precision is {1:G}",this.dbName,"millisecond"); Console.WriteLine("db：[{0:G}]'s precision is {1:G} millisecond");
break; break;
case 1: case 1:
Console.WriteLine("db：[{0:G}]'s precision is {1:G}",this.dbName,"microsecond"); Console.WriteLine("db：[{0:G}]'s precision is {1:G} microsecond");
break; break;
case 2: case 2:
Console.WriteLine("db：[{0:G}]'s precision is {1:G}",this.dbName,"nanosecond"); Console.WriteLine("db：[{0:G}]'s precision is {1:G} nanosecond");
break; break;
} }
} }
// public void checkData(int x ,int y , long ts ){
// }
} }
} }
tests/gotest/batchtest.bat
浏览文件 @ 6b7e4a85
...@@ -9,7 +9,7 @@ if "%severIp%"=="" (set severIp=127.0.0.1) ...@@ -9,7 +9,7 @@ if "%severIp%"=="" (set severIp=127.0.0.1)
if "%serverPort%"=="" (set serverPort=6030) if "%serverPort%"=="" (set serverPort=6030)
go env -w GO111MODULE=on go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.io,direct go env -w GOPROXY=https://goproxy.cn,direct
cd case001 cd case001
case001.bat %severIp% %serverPort% case001.bat %severIp% %serverPort%
......
tests/gotest/batchtest.sh
浏览文件 @ 6b7e4a85
...@@ -14,7 +14,7 @@ if [ ! -n "$serverPort" ]; then ...@@ -14,7 +14,7 @@ if [ ! -n "$serverPort" ]; then
fi fi
go env -w GO111MODULE=on go env -w GO111MODULE=on
go env -w GOPROXY=https://goproxy.io,direct go env -w GOPROXY=https://goproxy.cn,direct
bash ./case001/case001.sh $severIp $serverPort bash ./case001/case001.sh $severIp $serverPort
bash ./case002/case002.sh $severIp $serverPort bash ./case002/case002.sh $severIp $serverPort
......
tests/pytest/client/nettest.py 0 → 100644
浏览文件 @ 6b7e4a85
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import taos
import subprocess
from util.log import tdLog
from util.cases import tdCases
from util.sql import tdSql
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
def run(self):
try:
str1 = 'taos -n speed -P 6030 -N 1000 -l 100000 -S tcp'
result1 = subprocess.call(str1)
except Exception as result1:
if result1 == 1:
tdLog.exit("the shell 'taos -n speed -P 6030 -N 1000 -l 100000 -S tcp' is wrong")
try:
str2 = 'taos -n speed -P 6030 -N 1000 -l 100000 -S udp'
result2 = subprocess.call(str2)
except Exception as result2:
if result2 == 1:
tdLog.exit("the shell 'taos -n speed -P 6030 -N 1000 -l 100000 -S udp' is wrong")
try:
str3 = 'taos -n fqdn'
result3 = subprocess.call(str3)
except Exception as result3:
if result3 ==1:
tdLog.exit('the shell"taos -n fqdn" is wrong')
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
tests/pytest/fulltest.sh
浏览文件 @ 6b7e4a85
...@@ -398,7 +398,12 @@ python3 ./test.py -f query/queryWildcardLength.py ...@@ -398,7 +398,12 @@ python3 ./test.py -f query/queryWildcardLength.py
python3 ./test.py -f query/queryTbnameUpperLower.py python3 ./test.py -f query/queryTbnameUpperLower.py
python3 ./test.py -f query/query.py python3 ./test.py -f query/query.py
python3 ./test.py -f query/queryDiffColsOr.py python3 ./test.py -f query/queryDiffColsOr.py
python3 ./test.py -f client/nettest.py
python3 ./test.py -f query/queryRegex.py python3 ./test.py -f query/queryRegex.py
#======================p4-end=============== #======================p4-end===============
......
tests/pytest/insert/schemaChangeTest.py 0 → 100644
浏览文件 @ 6b7e4a85
###################################################################
# Copyright (c) 2016 by TAOS Technologies, Inc.
# All rights reserved.
#
# This file is proprietary and confidential to TAOS Technologies.
# No part of this file may be reproduced, stored, transmitted,
# disclosed or used in any form or by any means other than as
# expressly provided by the written permission from Jianhui Tao
#
###################################################################
# -*- coding: utf-8 -*-
import sys
import taos
from util.log import *
from util.cases import *
from util.sql import *
from util.dnodes import *
import multiprocessing as mp
class TDTestCase:
def init(self, conn, logSql):
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor())
self.ts = 1609430400000
def alterTableSchema(self):
conn1 = taos.connect(host="127.0.0.1", user="root", password="taosdata", config=tdDnodes.getSimCfgPath())
c1 = conn1.cursor()
c1.execute("use db")
c1.execute("alter table st drop column c2")
c1.execute("alter table st add column c2 double")
tdLog.sleep(1)
c1.execute("select * from st")
for data in c1:
print("Process 1: c2 = %s" % data[2])
def insertData(self):
conn2 = taos.connect(host="127.0.0.1", user="root", password="taosdata", config=tdDnodes.getSimCfgPath())
c2 = conn2.cursor()
tdLog.sleep(1)
c2.execute("use db")
c2.execute("insert into t1 values(%d, 2, 2.22)" % (self.ts + 1))
c2.execute("select * from st")
for data in c2:
print("Process 2: c2 = %f" % data[2])
def run(self):
tdSql.prepare()
tdSql.execute("create table st(ts timestamp, c1 int, c2 float) tags(t1 int)")
tdSql.execute("create table t1 using st tags(1)")
tdSql.execute("insert into t1 values(%d, 1, 1.11)" % self.ts)
p1 = mp.Process(target=self.alterTableSchema, args=())
p2 = mp.Process(target=self.insertData, args=())
p1.start()
p2.start()
p1.join()
p2.join()
def stop(self):
tdSql.close()
tdLog.success("%s successfully executed" % __file__)
tdCases.addWindows(__file__, TDTestCase())
tdCases.addLinux(__file__, TDTestCase())
\ No newline at end of file
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或