提交 bf9d36e5 编写于 作者: P Ping Xiao

Merge branch 'develop' into coverity_scan

......@@ -119,7 +119,7 @@ WantedBy=multi-user.target
return nil
}
const version = "TDengine alert v2.0.0.0"
const version = "TDengine alert v2.0.0.1"
func main() {
var (
......
......@@ -131,23 +131,83 @@ TDengine集群中加入一个新的dnode时,涉及集群相关的一些参数
- maxTablesPerVnode: 每个vnode中能够创建的最大表个数。默认值:1000000。
- maxVgroupsPerDb: 每个数据库中能够使用的最大vnode个数。
- arbitrator: 系统中裁决器的end point,缺省为空
- timezone:时区。从系统中动态获取当前的时区设置。
- locale:系统区位信息及编码格式。系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置。
- charset:字符集编码。系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置。
- timezone、locale、charset 的配置见客户端配置。
## 客户端配置
TDengine系统的前台交互客户端应用程序为taos,它与taosd共享同一个配置文件taos.cfg。运行taos时,使用参数-c指定配置文件目录,如taos -c /home/cfg,表示使用/home/cfg/目录下的taos.cfg配置文件中的参数,缺省目录是/etc/taos。更多taos的使用方法请见[Shell命令行程序](#_TDengine_Shell命令行程序)。本节主要讲解taos客户端应用在配置文件taos.cfg文件中使用到的参数。
TDengine系统的前台交互客户端应用程序为taos,它与taosd共享同一个配置文件taos.cfg。运行taos时,使用参数-c指定配置文件目录,如taos -c /home/cfg,表示使用/home/cfg/目录下的taos.cfg配置文件中的参数,缺省目录是/etc/taos。本节主要说明 taos 客户端应用在配置文件 taos.cfg 文件中使用到的参数。
客户端配置参数列表及解释
客户端配置参数
- firstEp: taos启动时,主动连接的集群中第一个taosd实例的end point, 缺省值为 localhost:6030。
- secondEp: taos启动时,如果first连接不上,尝试连接集群中第二个taosd实例的end point, 缺省值为空。
- charset:字符集编码。系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置。
- locale:系统区位信息及编码格式。系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置。
- maxBinaryDisplayWidth:Shell中binary 和 nchar字段的显示宽度上限,超过此限制的部分将被隐藏。默认值:30。可在 shell 中通过命令 set max_binary_display_width *nn* 动态修改此选项。
- locale
日志的配置参数,与server的配置参数完全一样。
> 默认值:系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置
TDengine为存储中文、日文、韩文等非ASCII编码的宽字符,提供一种专门的字段类型nchar。写入nchar字段的数据将统一采用UCS4-LE格式进行编码并发送到服务器。需要注意的是,编码正确性是客户端来保证。因此,如果用户想要正常使用nchar字段来存储诸如中文、日文、韩文等非ASCII字符,需要正确设置客户端的编码格式。
客户端的输入的字符均采用操作系统当前默认的编码格式,在Linux系统上多为UTF-8,部分中文系统编码则可能是GB18030或GBK等。在docker环境中默认的编码是POSIX。在中文版Windows系统中,编码则是CP936。客户端需要确保正确设置自己所使用的字符集,即客户端运行的操作系统当前编码字符集,才能保证nchar中的数据正确转换为UCS4-LE编码格式。
在 Linux 中 locale 的命名规则为: <语言>_<地区>.<字符集编码> 如:zh_CN.UTF-8,zh代表中文,CN代表大陆地区,UTF-8表示字符集。字符集编码为客户端正确解析本地字符串提供编码转换的说明。Linux系统与 Mac OSX 系统可以通过设置locale来确定系统的字符编码,由于Windows使用的locale中不是POSIX标准的locale格式,因此在Windows下需要采用另一个配置参数charset来指定字符编码。在Linux 系统中也可以使用charset来指定字符编码。
- charset
> 默认值:系统中动态获取,如果自动获取失败,需要用户在配置文件设置或通过API设置
如果配置文件中不设置charset,在Linux系统中,taos在启动时候,自动读取系统当前的locale信息,并从locale信息中解析提取charset编码格式。如果自动读取locale信息失败,则尝试读取charset配置,如果读取charset配置也失败,则中断启动过程。
在Linux系统中,locale信息包含了字符编码信息,因此正确设置了Linux系统locale以后可以不用再单独设置charset。例如:
```
locale zh_CN.UTF-8
```
在Windows系统中,无法从locale获取系统当前编码。如果无法从配置文件中读取字符串编码信息,taos默认设置为字符编码为CP936。其等效在配置文件中添加如下配置:
```
charset CP936
```
如果需要调整字符编码,请查阅当前操作系统使用的编码,并在配置文件中正确设置。
在Linux系统中,如果用户同时设置了locale和字符集编码charset,并且locale和charset的不一致,后设置的值将覆盖前面设置的值。
```
locale zh_CN.UTF-8
charset GBK
```
则charset的有效值是GBK。
```
charset GBK
locale zh_CN.UTF-8
```
charset的有效值是UTF-8。
日志的配置参数,与server 的配置参数完全一样。
- timezone
默认值:从系统中动态获取当前的时区设置
客户端运行系统所在的时区。为应对多时区的数据写入和查询问题,TDengine 采用 Unix 时间戳(Unix Timestamp)来记录和存储时间戳。Unix 时间戳的特点决定了任一时刻不论在任何时区,产生的时间戳均一致。需要注意的是,Unix时间戳是在客户端完成转换和记录。为了确保客户端其他形式的时间转换为正确的 Unix 时间戳,需要设置正确的时区。
在Linux系统中,客户端会自动读取系统设置的时区信息。用户也可以采用多种方式在配置文件设置时区。例如:
```
timezone UTC-8
timezone GMT-8
timezone Asia/Shanghai
```
均是合法的设置东八区时区的格式。
时区的设置对于查询和写入SQL语句中非Unix时间戳的内容(时间戳字符串、关键词now的解析)产生影响。例如:
```
SELECT count(*) FROM table_name WHERE TS<'2019-04-11 12:01:08';
```
在东八区,SQL语句等效于
```
SELECT count(*) FROM table_name WHERE TS<1554955268000;
```
在UTC时区,SQL语句等效于
```
SELECT count(*) FROM table_name WHERE TS<1554984068000;
```
为了避免使用字符串时间格式带来的不确定性,也可以直接使用Unix时间戳。此外,还可以在SQL语句中使用带有时区的时间戳字符串,例如:RFC3339格式的时间戳字符串,2013-04-12T15:52:01.123+08:00或者ISO-8601格式时间戳字符串2013-04-12T15:52:01.123+0800。上述两个字符串转化为Unix时间戳不受系统所在时区的影响。
启动taos时,也可以从命令行指定一个taosd实例的end point,否则就从taos.cfg读取。
......
......@@ -2,13 +2,15 @@
多个taosd的运行实例可以组成一个集群,以保证TDengine的高可靠运行,并提供水平扩展能力。要了解TDengine 2.0的集群管理,需要对集群的基本概念有所了解,请看TDengine 2.0整体架构一章。而且在安装集群之前,请按照[《立即开始》](https://www.taosdata.com/cn/getting-started20/)一章安装并体验过单节点功能。
集群的每个节点是由End Point来唯一标识的,End Point是由FQDN(Fully Qualified Domain Name)外加Port组成,比如 h1.taosdata.com:6030。一般FQDN就是服务器的hostname,可通过Linux命令`hostname -f`获取。端口是这个节点对外服务的端口号,缺省是6030,但可以通过taos.cfg里配置参数serverPort进行修改。一个节点可能配置了多个hostname, TDengine会自动获取第一个,但也可以通过taos.cfg里配置参数fqdn进行指定。如果习惯IP地址直接访问,可以将参数fqdn设置为本节点的IP地址。
集群的每个节点是由End Point来唯一标识的,End Point是由FQDN(Fully Qualified Domain Name)外加Port组成,比如 h1.taosdata.com:6030。一般FQDN就是服务器的hostname,可通过Linux命令`hostname -f`获取,FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)。端口是这个节点对外服务的端口号,缺省是6030,但可以通过taos.cfg里配置参数serverPort进行修改。一个节点可能配置了多个hostname, TDengine会自动获取第一个,但也可以通过taos.cfg里配置参数fqdn进行指定。如果习惯IP地址直接访问,可以将参数fqdn设置为本节点的IP地址。
TDengine的集群管理极其简单,除添加和删除节点需要人工干预之外,其他全部是自动完成,最大程度的降低了运维的工作量。本章对集群管理的操作做详细的描述。
## 准备工作
**第一步**:如果搭建集群的节点中,存有之前的测试数据、装过1.X的版本,或者装过其他版本的TDengine,请先将其删除,并清空所有数据,具体步骤请参考博客[《TDengine多种安装包的安装和卸载》](https://www.taosdata.com/blog/2019/08/09/566.html )
**注意1:**因为FQDN的信息会写进文件,如果之前没有配置或者更改FQDN,且启动了TDengine。请一定在确保数据无用或者备份的前提下,清理一下之前的数据(rm -rf /var/lib/taos/);
**注意2:**客户端也需要配置,确保它可以正确解析每个节点的FQDN配置,不管是通过DNS服务,还是 Host 文件。
**第二步**:建议关闭防火墙,至少保证端口:6030 - 6042的TCP和UDP端口都是开放的。**强烈建议**先关闭防火墙,集群搭建完毕之后,再来配置端口;
......@@ -23,7 +25,7 @@ TDengine的集群管理极其简单,除添加和删除节点需要人工干预
**第五步**:修改TDengine的配置文件(所有节点的文件/etc/taos/taos.cfg都需要修改)。假设准备启动的第一个节点End Point为 h1.taosdata.com:6030, 那么以下几个参数与集群相关:
```
// firstEp 是每个节点启动后连接的第一个节点
// firstEp 集群中所有节点的配置都是一致的,对其第一次访问后,就获得了整个集群的信息
firstEp h1.taosdata.com:6030
// 配置本节点的FQDN,如果本机只有一个hostname, 无需配置
......@@ -32,7 +34,7 @@ fqdn h1.taosdata.com
// 配置本节点的端口号,缺省是6030
serverPort 6030
// 副本数为偶数的时候,需要配置,请参考《Arbitrator的使用》的部分
// 服务端节点数为偶数的时候,需要配置,请参考《Arbitrator的使用》的部分
arbitrator ha.taosdata.com:6042
```
......
......@@ -32,7 +32,7 @@
3. 在服务器,执行 `systemctl status taosd` 检查*taosd*运行状态。如果没有运行,启动*taosd*
4. 确认客户端连接时指定了正确的服务器FQDN (Fully Qualified Domain Name(可在服务器上执行Linux命令hostname -f获得
4. 确认客户端连接时指定了正确的服务器FQDN (Fully Qualified Domain Name(可在服务器上执行Linux命令hostname -f获得)),FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)
5. ping服务器FQDN,如果没有反应,请检查你的网络,DNS设置,或客户端所在计算机的系统hosts文件
......@@ -54,7 +54,7 @@
## 6. 遇到错误“Unexpected generic error in RPC”或者"TDengine Error: Unable to resolve FQDN", 我怎么办?
产生这个错误,是由于客户端或数据节点无法解析FQDN(Fully Qualified Domain Name)导致。对于TAOS Shell或客户端应用,请做如下检查:
1. 请检查连接的服务器的FQDN是否正确
1. 请检查连接的服务器的FQDN是否正确,FQDN配置参考:[一篇文章说清楚TDengine的FQDN](https://www.taosdata.com/blog/2020/09/11/1824.html)
2. 如果网络配置有DNS server, 请检查是否正常工作
3. 如果网络没有配置DNS server, 请检查客户端所在机器的hosts文件,查看该FQDN是否配置,并是否有正确的IP地址。
4. 如果网络配置OK,从客户端所在机器,你需要能Ping该连接的FQDN,否则客户端是无法链接服务器的
......
......@@ -2090,6 +2090,10 @@ int tscProcessRetrieveRspFromNode(SSqlObj *pSql) {
SSqlCmd *pCmd = &pSql->cmd;
SRetrieveTableRsp *pRetrieve = (SRetrieveTableRsp *)pRes->pRsp;
if (pRetrieve == NULL) {
pRes->code = TSDB_CODE_TSC_OUT_OF_MEMORY;
return pRes->code;
}
pRes->numOfRows = htonl(pRetrieve->numOfRows);
pRes->precision = htons(pRetrieve->precision);
......
......@@ -93,14 +93,13 @@
<version>3.6.1</version>
<configuration>
<encoding>UTF-8</encoding>
<source>11</source>
<target>11</target>
<source>8</source>
<target>8</target>
<debug>true</debug>
<showDeprecation>true</showDeprecation>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-source-plugin</artifactId>
......
......@@ -57,9 +57,9 @@ public class TSDBConnection implements Connection {
File cfgDir = loadConfigDir(info.getProperty(TSDBDriver.PROPERTY_KEY_CONFIG_DIR));
File cfgFile = cfgDir.listFiles((dir, name) -> "taos.cfg".equalsIgnoreCase(name))[0];
List<String> endpoints = loadConfigEndpoints(cfgFile);
if (!endpoints.isEmpty()){
info.setProperty(TSDBDriver.PROPERTY_KEY_HOST,endpoints.get(0).split(":")[0]);
info.setProperty(TSDBDriver.PROPERTY_KEY_PORT,endpoints.get(0).split(":")[1]);
if (!endpoints.isEmpty()) {
info.setProperty(TSDBDriver.PROPERTY_KEY_HOST, endpoints.get(0).split(":")[0]);
info.setProperty(TSDBDriver.PROPERTY_KEY_PORT, endpoints.get(0).split(":")[1]);
}
//load taos.cfg end
......@@ -69,15 +69,15 @@ public class TSDBConnection implements Connection {
info.getProperty(TSDBDriver.PROPERTY_KEY_PASSWORD));
}
private List<String> loadConfigEndpoints(File cfgFile){
private List<String> loadConfigEndpoints(File cfgFile) {
List<String> endpoints = new ArrayList<>();
try(BufferedReader reader = new BufferedReader(new FileReader(cfgFile))) {
try (BufferedReader reader = new BufferedReader(new FileReader(cfgFile))) {
String line = null;
while ((line = reader.readLine())!=null){
if (line.trim().startsWith("firstEp") || line.trim().startsWith("secondEp")){
endpoints.add(line.substring(line.indexOf('p')+1).trim());
while ((line = reader.readLine()) != null) {
if (line.trim().startsWith("firstEp") || line.trim().startsWith("secondEp")) {
endpoints.add(line.substring(line.indexOf('p') + 1).trim());
}
if (endpoints.size()>1)
if (endpoints.size() > 1)
break;
}
} catch (FileNotFoundException e) {
......@@ -91,7 +91,7 @@ public class TSDBConnection implements Connection {
/**
* @param cfgDirPath
* @return return the config dir
* **/
**/
private File loadConfigDir(String cfgDirPath) {
if (cfgDirPath == null)
return loadDefaultConfigDir();
......@@ -103,8 +103,8 @@ public class TSDBConnection implements Connection {
/**
* @return search the default config dir, if the config dir is not exist will return null
* */
private File loadDefaultConfigDir(){
*/
private File loadDefaultConfigDir() {
File cfgDir;
File cfgDir_linux = new File("/etc/taos");
cfgDir = cfgDir_linux.exists() ? cfgDir_linux : null;
......@@ -132,7 +132,9 @@ public class TSDBConnection implements Connection {
public Statement createStatement() throws SQLException {
if (!this.connector.isClosed()) {
return new TSDBStatement(this.connector);
TSDBStatement statement = new TSDBStatement(this, this.connector);
statement.setConnection(this);
return statement;
} else {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
}
......@@ -153,7 +155,7 @@ public class TSDBConnection implements Connection {
public PreparedStatement prepareStatement(String sql) throws SQLException {
if (!this.connector.isClosed()) {
return new TSDBPreparedStatement(this.connector, sql);
return new TSDBPreparedStatement(this, this.connector, sql);
} else {
throw new SQLException(TSDBConstants.FixErrMsg(TSDBConstants.JNI_CONNECTION_NULL));
}
......
......@@ -42,8 +42,8 @@ public class TSDBPreparedStatement extends TSDBStatement implements PreparedStat
private SavedPreparedStatement savedPreparedStatement;
TSDBPreparedStatement(TSDBJNIConnector connecter, String sql) {
super(connecter);
TSDBPreparedStatement(TSDBConnection connection, TSDBJNIConnector connecter, String sql) {
super(connection, connecter);
init(sql);
}
......
......@@ -21,10 +21,14 @@ import java.util.List;
public class TSDBStatement implements Statement {
private TSDBJNIConnector connecter = null;
/** To store batched commands */
/**
* To store batched commands
*/
protected List<String> batchedArgs;
/** Timeout for a query */
/**
* Timeout for a query
*/
protected int queryTimeout = 0;
private Long pSql = 0l;
......@@ -35,7 +39,14 @@ public class TSDBStatement implements Statement {
private boolean isClosed = true;
private int affectedRows = 0;
TSDBStatement(TSDBJNIConnector connecter) {
private TSDBConnection connection;
public void setConnection(TSDBConnection connection) {
this.connection = connection;
}
TSDBStatement(TSDBConnection connection, TSDBJNIConnector connecter) {
this.connection = connection;
this.connecter = connecter;
this.isClosed = false;
}
......@@ -256,6 +267,8 @@ public class TSDBStatement implements Statement {
}
public Connection getConnection() throws SQLException {
if (this.connecter != null)
return this.connection;
throw new SQLException(TSDBConstants.UNSUPPORT_METHOD_EXCEPTIONZ_MSG);
}
......
......@@ -23,6 +23,7 @@ import java.sql.SQLException;
public class SqlSyntaxValidator {
private TSDBConnection tsdbConnection;
public SqlSyntaxValidator(Connection connection) {
this.tsdbConnection = (TSDBConnection) connection;
}
......
......@@ -10,7 +10,6 @@ public class BaseTest {
private static boolean testCluster = false;
private static TDNodes nodes = new TDNodes();
@BeforeClass
public static void setupEnv() {
try{
......@@ -19,11 +18,9 @@ public class BaseTest {
nodes.getTDNode(1).setRunning(1);
nodes.stop(1);
}
nodes.setTestCluster(testCluster);
nodes.deploy(1);
nodes.start(1);
} catch (Exception e) {
e.printStackTrace();
}
......
......@@ -7,13 +7,11 @@ import org.junit.Test;
import java.sql.*;
import java.util.Properties;
import java.util.Random;
import static org.junit.Assert.assertEquals;
import java.util.Properties;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.*;
import java.util.concurrent.TimeUnit;
import static org.junit.Assert.assertTrue;
import static org.junit.Assert.assertEquals;
public class BatchInsertTest extends BaseTest {
......
......@@ -34,7 +34,6 @@ public class SelectTest extends BaseTest {
statement.executeUpdate("drop database if exists " + dbName);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("create table if not exists " + dbName + "." + tName + " (ts timestamp, k int, v int)");
}
@Test
......@@ -66,6 +65,5 @@ public class SelectTest extends BaseTest {
statement.close();
connection.close();
Thread.sleep(10);
}
}
......@@ -10,8 +10,6 @@ import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
import static org.junit.Assert.assertTrue;
public class SubscribeTest extends BaseTest {
Connection connection = null;
Statement statement = null;
......
package com.taosdata.jdbc.cases;
import com.taosdata.jdbc.lib.TSDBCommon;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import java.sql.Connection;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import static org.junit.Assert.assertEquals;
public class BatchInsertTest {
static String host = "localhost";
static String dbName = "test";
static String stbName = "meters";
static int numOfTables = 30;
final static int numOfRecordsPerTable = 1000;
static long ts = 1496732686000l;
final static String tablePrefix = "t";
private Connection connection;
@Before
public void before() {
try {
connection = TSDBCommon.getConn(host);
TSDBCommon.createDatabase(connection, dbName);
TSDBCommon.createStable(connection, stbName);
TSDBCommon.createTables(connection, numOfTables, stbName, tablePrefix);
} catch (Exception e) {
e.printStackTrace();
}
}
@Test
public void testBatchInsert(){
ExecutorService executorService = Executors.newFixedThreadPool(numOfTables);
for (int i = 0; i < numOfTables; i++) {
final int index = i;
executorService.execute(new Runnable() {
@Override
public void run() {
try {
long startTime = System.currentTimeMillis();
Statement statement = connection.createStatement(); // get statement
StringBuilder sb = new StringBuilder();
sb.append("INSERT INTO " + tablePrefix + index + " VALUES");
Random rand = new Random();
for (int j = 1; j <= numOfRecordsPerTable; j++) {
sb.append("(" + (ts + j) + ", ");
sb.append(rand.nextInt(100) + ", ");
sb.append(rand.nextInt(100) + ", ");
sb.append(rand.nextInt(100) + ")");
}
statement.addBatch(sb.toString());
statement.executeBatch();
long endTime = System.currentTimeMillis();
System.out.println("Thread " + index + " takes " + (endTime - startTime) + " microseconds");
connection.commit();
statement.close();
} catch (Exception e) {
e.printStackTrace();
}
}
});
}
executorService.shutdown();
try {
executorService.awaitTermination(Long.MAX_VALUE, TimeUnit.NANOSECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
try{
Statement statement = connection.createStatement();
ResultSet rs = statement.executeQuery("select * from meters");
int num = 0;
while (rs.next()) {
num++;
}
assertEquals(num, numOfTables * numOfRecordsPerTable);
rs.close();
}catch (Exception e){
e.printStackTrace();
}
}
@After
public void after() {
try {
if (connection != null)
connection.close();
} catch (SQLException e) {
e.printStackTrace();
}
}
}
package com.taosdata.jdbc.lib;
import com.taosdata.jdbc.TSDBDriver;
import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.SQLException;
import java.sql.Statement;
import java.util.Properties;
public class TSDBCommon {
public static Connection getConn(String host) throws SQLException, ClassNotFoundException {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
return DriverManager.getConnection("jdbc:TAOS://" + host + ":0/", properties);
}
public static void createDatabase(Connection connection, String dbName) throws SQLException {
Statement statement = connection.createStatement();
statement.executeUpdate("drop database if exists " + dbName);
statement.executeUpdate("create database if not exists " + dbName);
statement.executeUpdate("use " + dbName);
statement.close();
}
public static void createStable(Connection connection, String stbName) throws SQLException {
Statement statement = connection.createStatement();
String createTableSql = "create table " + stbName + "(ts timestamp, f1 int, f2 int, f3 int) tags(areaid int, loc binary(20))";
statement.executeUpdate(createTableSql);
statement.close();
}
public static void createTables(Connection connection, int numOfTables, String stbName,String tablePrefix) throws SQLException {
Statement statement = connection.createStatement();
for(int i = 0; i < numOfTables; i++) {
String loc = i % 2 == 0 ? "beijing" : "shanghai";
String createSubTalbesSql = "create table " + tablePrefix + i + " using " + stbName + " tags(" + i + ", '" + loc + "')";
statement.executeUpdate(createSubTalbesSql);
}
statement.close();
}
}
......@@ -16,11 +16,13 @@
#define _DEFAULT_SOURCE
#include "os.h"
#include "taosdef.h"
#include "taosmsg.h"
#include "tglobal.h"
#include "mnode.h"
#include "http.h"
#include "tmqtt.h"
#include "monitor.h"
#include "dnode.h"
#include "dnodeInt.h"
#include "dnodeModule.h"
......@@ -129,17 +131,34 @@ void dnodeProcessModuleStatus(uint32_t moduleStatus) {
for (int32_t module = TSDB_MOD_MNODE; module < TSDB_MOD_HTTP; ++module) {
bool enableModule = moduleStatus & (1 << module);
if (!tsModule[module].enable && enableModule) {
dInfo("module status:%u is received, start %s module", tsModuleStatus, tsModule[module].name);
dInfo("module status:%u is set, start %s module", moduleStatus, tsModule[module].name);
tsModule[module].enable = true;
dnodeSetModuleStatus(module);
(*tsModule[module].startFp)();
}
if (tsModule[module].enable && !enableModule) {
dInfo("module status:%u is received, stop %s module", tsModuleStatus, tsModule[module].name);
dInfo("module status:%u is set, stop %s module", moduleStatus, tsModule[module].name);
tsModule[module].enable = false;
dnodeUnSetModuleStatus(module);
(*tsModule[module].stopFp)();
}
}
}
bool dnodeCheckMnodeStarting() {
if (tsModuleStatus & TSDB_MOD_MNODE) return false;
SDMMnodeInfos *mnodes = dnodeGetMnodeInfos();
for (int32_t i = 0; i < mnodes->nodeNum; ++i) {
SDMMnodeInfo *node = &mnodes->nodeInfos[i];
if (node->nodeId == dnodeGetDnodeId()) {
uint32_t moduleStatus = tsModuleStatus | (1 << TSDB_MOD_MNODE);;
dInfo("start mnode module, module status:%d, new status:%d", tsModuleStatus, moduleStatus);
dnodeProcessModuleStatus(moduleStatus);
return true;
}
}
return false;
}
......@@ -43,6 +43,7 @@ void dnodeGetMnodeEpSetForPeer(void *epSet);
void dnodeGetMnodeEpSetForShell(void *epSet);
void * dnodeGetMnodeInfos();
int32_t dnodeGetDnodeId();
bool dnodeCheckMnodeStarting();
void dnodeAddClientRspHandle(uint8_t msgType, void (*fp)(SRpcMsg *rpcMsg));
void dnodeSendMsgToDnode(SRpcEpSet *epSet, SRpcMsg *rpcMsg);
......
......@@ -103,6 +103,9 @@ typedef struct {
typedef void* tsync_h;
int32_t syncInit();
void syncCleanUp();
tsync_h syncStart(const SSyncInfo *);
void syncStop(tsync_h shandle);
int32_t syncReconfig(tsync_h shandle, const SSyncCfg *);
......
......@@ -22,6 +22,7 @@
#include "tqueue.h"
#include "twal.h"
#include "tsync.h"
#include "ttimer.h"
#include "tglobal.h"
#include "dnode.h"
#include "mnode.h"
......@@ -64,6 +65,7 @@ typedef struct _SSdbTable {
int32_t (*encodeFp)(SSdbOper *pOper);
int32_t (*destroyFp)(SSdbOper *pOper);
int32_t (*restoredFp)();
pthread_mutex_t mutex;
} SSdbTable;
typedef struct {
......@@ -88,6 +90,8 @@ typedef struct {
SSdbWriteWorker *writeWorker;
} SSdbWriteWorkerPool;
extern void * tsMnodeTmr;
static void * tsUpdateSyncTmr;
static SSdbObject tsSdbObj = {0};
static taos_qset tsSdbWriteQset;
static taos_qall tsSdbWriteQall;
......@@ -290,11 +294,17 @@ static void sdbConfirmForward(void *ahandle, void *param, int32_t code) {
taosFreeQitem(pOper);
}
static void sdbUpdateSyncTmrFp(void *param, void *tmrId) { sdbUpdateSync(); }
void sdbUpdateSync() {
if (!mnodeIsRunning()) {
mDebug("mnode not start yet, update sync info later");
if (dnodeCheckMnodeStarting()) {
taosTmrReset(sdbUpdateSyncTmrFp, 1000, NULL, tsMnodeTmr, &tsUpdateSyncTmr);
}
return;
}
mDebug("update sync info in sdb");
SSyncCfg syncCfg = {0};
int32_t index = 0;
......@@ -387,8 +397,6 @@ int32_t sdbInit() {
tsSdbObj.role = TAOS_SYNC_ROLE_MASTER;
}
sdbUpdateSync();
tsSdbObj.status = SDB_STATUS_SERVING;
return TSDB_CODE_SUCCESS;
}
......@@ -448,8 +456,9 @@ static void *sdbGetRowMeta(SSdbTable *pTable, void *key) {
}
void **ppRow = (void **)taosHashGet(pTable->iHandle, key, keySize);
if (ppRow == NULL) return NULL;
return *ppRow;
if (ppRow != NULL) return *ppRow;
return NULL;
}
static void *sdbGetRowMetaFromObj(SSdbTable *pTable, void *key) {
......@@ -457,13 +466,14 @@ static void *sdbGetRowMetaFromObj(SSdbTable *pTable, void *key) {
}
void *sdbGetRow(void *handle, void *key) {
SSdbTable *pTable = handle;
pthread_mutex_lock(&pTable->mutex);
void *pRow = sdbGetRowMeta(handle, key);
if (pRow) {
sdbIncRef(handle, pRow);
if (pRow) sdbIncRef(handle, pRow);
pthread_mutex_unlock(&pTable->mutex);
return pRow;
} else {
return NULL;
}
}
static void *sdbGetRowFromObj(SSdbTable *pTable, void *key) {
......@@ -478,7 +488,9 @@ static int32_t sdbInsertHash(SSdbTable *pTable, SSdbOper *pOper) {
keySize = strlen((char *)key);
}
pthread_mutex_lock(&pTable->mutex);
taosHashPut(pTable->iHandle, key, keySize, &pOper->pObj, sizeof(int64_t));
pthread_mutex_unlock(&pTable->mutex);
sdbIncRef(pTable, pOper->pObj);
atomic_add_fetch_32(&pTable->numOfRows, 1);
......@@ -519,7 +531,10 @@ static int32_t sdbDeleteHash(SSdbTable *pTable, SSdbOper *pOper) {
keySize = strlen((char *)key);
}
pthread_mutex_lock(&pTable->mutex);
taosHashRemove(pTable->iHandle, key, keySize);
pthread_mutex_unlock(&pTable->mutex);
atomic_sub_fetch_32(&pTable->numOfRows, 1);
sdbDebug("table:%s, delete record:%s from hash, numOfRows:%" PRId64 ", msg:%p", pTable->tableName,
......@@ -861,6 +876,7 @@ void *sdbOpenTable(SSdbTableDesc *pDesc) {
if (pTable == NULL) return NULL;
pthread_mutex_init(&pTable->mutex, NULL);
tstrncpy(pTable->tableName, pDesc->tableName, SDB_TABLE_LEN);
pTable->keyType = pDesc->keyType;
pTable->tableId = pDesc->tableId;
......@@ -908,6 +924,7 @@ void sdbCloseTable(void *handle) {
taosHashDestroyIter(pIter);
taosHashCleanup(pTable->iHandle);
pthread_mutex_destroy(&pTable->mutex);
sdbDebug("table:%s, is closed, numOfTables:%d", pTable->tableName, tsSdbObj.numOfTables);
free(pTable);
......
此差异已折叠。
......@@ -57,6 +57,9 @@ void syncConfirmForward(tsync_h shandle, uint64_t version, int32_t code) {}
#endif
int32_t vnodeInitResources() {
int code = syncInit();
if (code != 0) return code;
vnodeInitWriteFp();
vnodeInitReadFp();
......@@ -70,11 +73,12 @@ int32_t vnodeInitResources() {
}
void vnodeCleanupResources() {
if (tsDnodeVnodesHash != NULL) {
taosHashCleanup(tsDnodeVnodesHash);
tsDnodeVnodesHash = NULL;
}
syncCleanUp();
}
int32_t vnodeCreate(SMDCreateVnodeMsg *pVnodeCfg) {
......
......@@ -93,11 +93,12 @@ static int32_t vnodeDumpQueryResult(SRspRet *pRet, void* pVnode, void** handle,
vDebug("QInfo:%p exec completed, free handle:%d", *handle, *freeHandle);
}
} else {
SRetrieveTableRsp* pRsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp));
SRetrieveTableRsp *pRsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp));
memset(pRsp, 0, sizeof(SRetrieveTableRsp));
pRsp->completed = true;
pRet->rsp = pRsp;
pRet->len = sizeof(SRetrieveTableRsp);
*freeHandle = true;
}
......@@ -270,6 +271,7 @@ static int32_t vnodeProcessFetchMsg(SVnodeObj *pVnode, SReadMsg *pReadMsg) {
if (code != TSDB_CODE_SUCCESS) {
//TODO handle malloc failure
pRet->rsp = (SRetrieveTableRsp *)rpcMallocCont(sizeof(SRetrieveTableRsp));
pRet->len = sizeof(SRetrieveTableRsp);
memset(pRet->rsp, 0, sizeof(SRetrieveTableRsp));
freeHandle = true;
} else { // result is not ready, return immediately
......
......@@ -9,21 +9,20 @@
<version>1.0-SNAPSHOT</version>
<packaging>jar</packaging>
<build>
<pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-plugins</artifactId>
<version>30</version>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
<version>3.0.0</version>
</plugin>
</plugins>
</pluginManagement>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-assembly-plugin</artifactId>
......@@ -48,6 +47,7 @@
</execution>
</executions>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
......
package com.taosdata.example;
import com.taosdata.jdbc.TSDBDriver;
import java.sql.*;
import java.util.Properties;
public class JDBCConnectorChecker {
private static String host;
private static String dbName = "test";
private static String tbName = "weather";
private Connection connection;
/**
* get connection
**/
private void init() {
try {
Class.forName("com.taosdata.jdbc.TSDBDriver");
Properties properties = new Properties();
properties.setProperty(TSDBDriver.PROPERTY_KEY_HOST, host);
properties.setProperty(TSDBDriver.PROPERTY_KEY_CHARSET, "UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_LOCALE, "en_US.UTF-8");
properties.setProperty(TSDBDriver.PROPERTY_KEY_TIME_ZONE, "UTC-8");
System.out.println("get connection starting...");
connection = DriverManager.getConnection("jdbc:TAOS://" + host + ":0/", properties);
if (connection != null)
System.out.println("[ OK ] Connection established.");
} catch (ClassNotFoundException | SQLException e) {
throw new RuntimeException("connection failed: " + host);
}
}
/**
* create database
*/
private void createDatabase() {
String sql = "create database if not exists " + dbName;
exuete(sql);
}
/**
* use database
*/
private void useDatabase() {
String sql = "use " + dbName;
exuete(sql);
}
/**
* select
*/
private void checkSelect() {
final String sql = "select * from test.weather";
executeQuery(sql);
}
private void executeQuery(String sql) {
try (Statement statement = connection.createStatement()) {
long start = System.currentTimeMillis();
ResultSet resultSet = statement.executeQuery(sql);
long end = System.currentTimeMillis();
printSql(sql, true, (end - start));
printResult(resultSet);
} catch (SQLException e) {
e.printStackTrace();
}
}
private void printResult(ResultSet resultSet) throws SQLException {
ResultSetMetaData metaData = resultSet.getMetaData();
while (resultSet.next()) {
for (int i = 1; i <= metaData.getColumnCount(); i++) {
String columnLabel = metaData.getColumnLabel(i);
String value = resultSet.getString(i);
System.out.printf("%s: %s\t", columnLabel, value);
}
System.out.println();
}
}
private String formatString(String str) {
StringBuilder sb = new StringBuilder();
int blankCnt = (26 - str.length()) / 2;
for (int j = 0; j < blankCnt; j++)
sb.append(" ");
sb.append(str);
for (int j = 0; j < blankCnt; j++)
sb.append(" ");
sb.append("|");
return sb.toString();
}
/**
* insert
*/
private void checkInsert() {
final String sql = "insert into test.weather (ts, temperature, humidity) values(now, 20.5, 34)";
exuete(sql);
}
/**
* create table
*/
private void createTable() {
final String sql = "create table if not exists " + dbName + "." + tbName + " (ts timestamp, temperature float, humidity int)";
exuete(sql);
}
private final void printSql(String sql, boolean succeed, long cost) {
System.out.println("[ " + (succeed ? "OK" : "ERROR!") + " ] time cost: " + cost + " ms, execute statement ====> " + sql);
}
private final void exuete(String sql) {
try (Statement statement = connection.createStatement()) {
long start = System.currentTimeMillis();
boolean execute = statement.execute(sql);
long end = System.currentTimeMillis();
printSql(sql, execute, (end - start));
} catch (SQLException e) {
e.printStackTrace();
}
}
private void close() {
try {
if (connection != null) {
this.connection.close();
System.out.println("connection closed.");
}
} catch (SQLException e) {
e.printStackTrace();
}
}
private void checkDropTable() {
final String sql = "drop table if exists " + dbName + "." + tbName + "";
exuete(sql);
}
public static void main(String[] args) {
for (int i = 0; i < args.length; i++) {
if ("-host".equalsIgnoreCase(args[i]) && i < args.length - 1) {
host = args[++i];
}
if ("-db".equalsIgnoreCase(args[i]) && i < args.length - 1) {
dbName = args[++i];
}
if ("-t".equalsIgnoreCase(args[i]) && i < args.length - 1) {
tbName = args[++i];
}
}
if (host == null) {
System.out.println("Usage: java -jar JDBCConnectorChecker.jar -host <hostname>");
return;
}
JDBCConnectorChecker checker = new JDBCConnectorChecker();
checker.init();
checker.createDatabase();
checker.useDatabase();
checker.checkDropTable();
checker.createTable();
checker.checkInsert();
checker.checkSelect();
checker.checkDropTable();
checker.close();
}
}
......@@ -22,26 +22,32 @@
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
<version>4.3.2.RELEASE</version>
<version>5.2.8.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-jdbc</artifactId>
<version>4.3.2.RELEASE</version>
<version>5.1.9.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-test</artifactId>
<version>5.1.9.RELEASE</version>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.11</version>
<version>4.13</version>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.taosdata.jdbc</groupId>
<artifactId>taos-jdbcdriver</artifactId>
<version>2.0.2</version>
<version>2.0.4</version>
</dependency>
</dependencies>
......@@ -63,7 +69,7 @@
<configuration>
<archive>
<manifest>
<mainClass>com.taosdata.jdbc.App</mainClass>
<mainClass>com.taosdata.jdbc.example.jdbcTemplate.App</mainClass>
</manifest>
</archive>
<descriptorRefs>
......
package com.taosdata.jdbc;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.util.CollectionUtils;
import java.util.List;
import java.util.Map;
public class App {
public static void main( String[] args ) {
ApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
JdbcTemplate jdbcTemplate = (JdbcTemplate) ctx.getBean("jdbcTemplate");
// create database
jdbcTemplate.execute("create database if not exists db ");
// create table
jdbcTemplate.execute("create table if not exists db.tb (ts timestamp, temperature int, humidity float)");
String insertSql = "insert into db.tb values(now, 23, 10.3) (now + 1s, 20, 9.3)";
// insert rows
int affectedRows = jdbcTemplate.update(insertSql);
System.out.println("insert success " + affectedRows + " rows.");
// query for list
List<Map<String, Object>> resultList = jdbcTemplate.queryForList("select * from db.tb");
if(!CollectionUtils.isEmpty(resultList)){
for (Map<String, Object> row : resultList){
System.out.printf("%s, %d, %s\n", row.get("ts"), row.get("temperature"), row.get("humidity"));
}
}
}
}
package com.taosdata.jdbc.example.jdbcTemplate;
import com.taosdata.jdbc.example.jdbcTemplate.dao.ExecuteAsStatement;
import com.taosdata.jdbc.example.jdbcTemplate.dao.WeatherDao;
import com.taosdata.jdbc.example.jdbcTemplate.domain.Weather;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;
import java.sql.Timestamp;
import java.util.Date;
import java.util.List;
import java.util.Random;
public class App {
private static Random random = new Random(System.currentTimeMillis());
public static void main(String[] args) {
ApplicationContext ctx = new ClassPathXmlApplicationContext("applicationContext.xml");
ExecuteAsStatement executor = ctx.getBean(ExecuteAsStatement.class);
// drop database
executor.doExecute("drop database if exists test");
// create database
executor.doExecute("create database if not exists test");
//use database
executor.doExecute("use test");
// create table
executor.doExecute("create table if not exists test.weather (ts timestamp, temperature int, humidity float)");
WeatherDao weatherDao = ctx.getBean(WeatherDao.class);
Weather weather = new Weather(new Timestamp(new Date().getTime()), random.nextFloat() * 50.0f, random.nextInt(100));
// insert rows
int affectedRows = weatherDao.add(weather);
System.out.println("insert success " + affectedRows + " rows.");
// query for list
int limit = 10, offset = 0;
List<Weather> weatherList = weatherDao.queryForList(limit, offset);
for (Weather w : weatherList) {
System.out.println(w);
}
}
}
package com.taosdata.jdbc.example.jdbcTemplate.dao;
public interface ExecuteAsStatement{
void doExecute(String sql);
}
package com.taosdata.jdbc.example.jdbcTemplate.dao;
import com.taosdata.jdbc.example.jdbcTemplate.domain.Weather;
import java.util.List;
public interface WeatherDao {
int add(Weather weather);
int[] batchInsert(List<Weather> weatherList);
List<Weather> queryForList(int limit, int offset);
int count();
}
package com.taosdata.jdbc.example.jdbcTemplate.dao.impl;
import com.taosdata.jdbc.example.jdbcTemplate.dao.ExecuteAsStatement;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.stereotype.Repository;
@Repository
public class ExecuteAsStatementImpl implements ExecuteAsStatement {
@Autowired
private JdbcTemplate jdbcTemplate;
@Override
public void doExecute(String sql) {
jdbcTemplate.execute(sql);
}
}
package com.taosdata.jdbc.example.jdbcTemplate.dao.impl;
import com.taosdata.jdbc.example.jdbcTemplate.dao.WeatherDao;
import com.taosdata.jdbc.example.jdbcTemplate.domain.Weather;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.jdbc.core.BatchPreparedStatementSetter;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.namedparam.SqlParameterSourceUtils;
import org.springframework.jdbc.core.simple.SimpleJdbcInsert;
import org.springframework.stereotype.Repository;
import java.sql.PreparedStatement;
import java.sql.SQLException;
import java.sql.Timestamp;
import java.util.HashMap;
import java.util.List;
import java.util.Map;
@Repository
public class WeatherDaoImpl implements WeatherDao {
@Autowired
private JdbcTemplate jdbcTemplate;
@Override
public int add(Weather weather) {
return jdbcTemplate.update(
"insert into test.weather(ts, temperature, humidity) VALUES(?,?,?)",
weather.getTs(), weather.getTemperature(), weather.getHumidity()
);
}
@Override
public int[] batchInsert(List<Weather> weatherList) {
return jdbcTemplate.batchUpdate("insert into test.weather(ts, temperature, humidity) values( ?, ?, ?)", new BatchPreparedStatementSetter() {
@Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
ps.setTimestamp(1, weatherList.get(i).getTs());
ps.setFloat(2, weatherList.get(i).getTemperature());
ps.setInt(3, weatherList.get(i).getHumidity());
}
@Override
public int getBatchSize() {
return weatherList.size();
}
});
}
@Override
public List<Weather> queryForList(int limit, int offset) {
return jdbcTemplate.query("select * from test.weather limit ? offset ?", (rs, rowNum) -> {
Timestamp ts = rs.getTimestamp("ts");
float temperature = rs.getFloat("temperature");
int humidity = rs.getInt("humidity");
return new Weather(ts, temperature, humidity);
}, limit, offset);
}
@Override
public int count() {
return jdbcTemplate.queryForObject("select count(*) from test.weather", Integer.class);
}
}
package com.taosdata.jdbc.example.jdbcTemplate.domain;
import java.sql.Timestamp;
public class Weather {
private Timestamp ts;
private float temperature;
private int humidity;
public Weather() {
}
public Weather(Timestamp ts, float temperature, int humidity) {
this.ts = ts;
this.temperature = temperature;
this.humidity = humidity;
}
@Override
public String toString() {
return "Weather{" +
"ts=" + ts +
", temperature=" + temperature +
", humidity=" + humidity +
'}';
}
public Timestamp getTs() {
return ts;
}
public void setTs(Timestamp ts) {
this.ts = ts;
}
public float getTemperature() {
return temperature;
}
public void setTemperature(float temperature) {
this.temperature = temperature;
}
public int getHumidity() {
return humidity;
}
public void setHumidity(int humidity) {
this.humidity = humidity;
}
}
......@@ -5,20 +5,21 @@
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd
"
http://www.springframework.org/schema/context http://www.springframework.org/schema/context/spring-context.xsd"
default-autowire="byName">
<bean id="dataSource" class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.taosdata.jdbc.TSDBDriver"></property>
<property name="url" value="jdbc:TAOS://127.0.0.1:6030/log"></property>
<property name="url" value="jdbc:TAOS://192.168.236.137:6030/"></property>
<property name="username" value="root"></property>
<property name="password" value="taosdata"></property>
</bean>
<bean id = "jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate" >
<property name="dataSource" ref = "dataSource" ></property>
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource" ref="dataSource"></property>
</bean>
<context:component-scan base-package="com.taosdata.jdbc.example.jdbcTemplate"/>
</beans>
......@@ -7,14 +7,12 @@ import org.junit.Test;
/**
* Unit test for simple App.
*/
public class AppTest
{
public class AppTest {
/**
* Rigorous Test :-)
*/
@Test
public void shouldAnswerWithTrue()
{
assertTrue( true );
public void shouldAnswerWithTrue() {
assertTrue(true);
}
}
package com.taosdata.jdbc.example.jdbcTemplate;
import com.taosdata.jdbc.example.jdbcTemplate.dao.ExecuteAsStatement;
import com.taosdata.jdbc.example.jdbcTemplate.dao.WeatherDao;
import com.taosdata.jdbc.example.jdbcTemplate.domain.Weather;
import org.junit.Before;
import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.test.context.ContextConfiguration;
import org.springframework.test.context.junit4.SpringJUnit4ClassRunner;
import java.sql.Timestamp;
import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import static org.junit.Assert.assertEquals;
@RunWith(SpringJUnit4ClassRunner.class)
@ContextConfiguration({"classpath:applicationContext.xml"})
public class BatcherInsertTest {
@Autowired
private WeatherDao weatherDao;
@Autowired
private ExecuteAsStatement executor;
private static final int numOfRecordsPerTable = 1000;
private static long ts = 1496732686000l;
private static Random random = new Random(System.currentTimeMillis());
@Before
public void before() {
// drop database
executor.doExecute("drop database if exists test");
// create database
executor.doExecute("create database if not exists test");
//use database
executor.doExecute("use test");
// create table
executor.doExecute("create table if not exists test.weather (ts timestamp, temperature int, humidity float)");
}
@Test
public void batchInsert() {
List<Weather> weatherList = new ArrayList<>();
for (int i = 0; i < numOfRecordsPerTable; i++) {
ts += 1000;
Weather weather = new Weather(new Timestamp(ts), random.nextFloat() * 50.0f, random.nextInt(100));
weatherList.add(weather);
}
long start = System.currentTimeMillis();
weatherDao.batchInsert(weatherList);
long end = System.currentTimeMillis();
System.out.println("batch insert(" + numOfRecordsPerTable + " rows) time cost ==========> " + (end - start) + " ms");
int count = weatherDao.count();
assertEquals(count, numOfRecordsPerTable);
}
}
......@@ -25,7 +25,7 @@ class TDTestCase:
tdLog.debug("start to execute %s" % __file__)
tdSql.init(conn.cursor(), logSql)
self.numOfRecords = 10
self.types = ["tinyint", "smallint", "int", "bigint", "float", "double", "bool", "binary(10)", "nchar(10)"]
self.ts = 1537146000000
def checkNullValue(self, result):
......@@ -38,139 +38,41 @@ class TDTestCase:
return False
return True
def restartTaosd(self):
tdDnodes.stop(1)
tdDnodes.start(1)
tdSql.execute("use db")
def run(self):
tdSql.prepare()
print("==============step1")
tdSql.execute(
"create table meters (ts timestamp, col1 int) tags(tgcol1 int)")
tdSql.execute("create table t0 using meters tags(NULL)")
for i in range (self.numOfRecords):
tdSql.execute("insert into t0 values (%d, %d)" % (self.ts + i, i));
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col2 tinyint")
tdSql.execute("alter table meters drop column col1")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col2 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col1 int")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col1 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col3 smallint")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col3 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col4 bigint")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col4 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col5 float")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col5 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col6 double")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col6 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col7 bool")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col7 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col8 binary(20)")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col8 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add column col9 nchar(20)")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select col9 from meters")
tdSql.checkRows(10)
tdSql.execute("alter table meters add tag tgcol2 tinyint")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol2 from meters")
tdSql.checkRows(1)
for i in range(len(self.types)):
print("======== checking type %s ==========" % self.types[i])
tdSql.execute("create table t0 (ts timestamp, col %s)" % self.types[i])
tdSql.execute("insert into t0 values (%d, NULL)" % (self.ts))
tdSql.execute("alter table meters add tag tgcol3 smallint")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol3 from meters")
tdSql.checkRows(1)
tdSql.execute("alter table meters add tag tgcol4 bigint")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol4 from meters")
tdSql.checkRows(1)
tdSql.execute("alter table meters add tag tgcol5 float")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol5 from meters")
tdDnodes.stop(1)
tdLog.sleep(10)
tdDnodes.start(1)
tdSql.execute("use db")
tdSql.query("select * from t0")
tdSql.checkRows(1)
tdSql.execute("alter table meters add tag tgcol6 double")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol6 from meters")
tdSql.checkRows(1)
if self.checkNullValue(tdSql.queryResult) is False:
tdLog.exit("no None value is detected")
tdSql.execute("alter table meters add tag tgcol7 bool")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol7 from meters")
tdSql.checkRows(1)
tdSql.execute("create table t1 (ts timestamp, col %s)" % self.types[i])
tdSql.execute("insert into t1 values (%d, NULL)" % (self.ts))
tdDnodes.stop(1)
tdLog.sleep(10)
tdDnodes.start(1)
tdSql.execute("use db")
tdSql.execute("alter table meters add tag tgcol8 binary(20)")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol8 from meters")
tdSql.checkRows(1)
for j in range(150):
tdSql.execute("insert into t1 values (%d, NULL)" % (self.ts + j + 1));
tdSql.execute("alter table meters add tag tgcol9 nchar(20)")
tdSql.query("select * from meters")
tdSql.checkRows(10)
tdSql.query("select tgcol9 from meters")
tdSql.checkRows(1)
tdSql.query("select * from t1")
tdSql.checkRows(151)
self.restartTaosd()
tdSql.query("select * from meters")
tdSql.checkRows(10)
if self.checkNullValue(tdSql.queryResult) is False:
tdLog.exit("non None value is detected")
tdLog.exit("no None value is detected")
print("======== None value check for type %s is OK ==========" % self.types[i])
def stop(self):
tdSql.close()
......
......@@ -53,7 +53,7 @@ system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
sql create dnode $hostname2
sql create dnode $hostname3
sleep 3000
sleep 5000
$sleepTimer = 3000
......@@ -225,6 +225,7 @@ if $data00 != $totalRows then
endi
print ============== step5: stop dnode2, and remove its vnode
sleep 5000
system sh/exec.sh -n dnode2 -s stop -x SIGINT
sleep $sleepTimer
......
......@@ -193,6 +193,7 @@ if $data00 != $totalRows then
endi
print ============== step5: stop dnode1
sleep 5000
system sh/exec.sh -n dnode1 -s stop
sleep 3000
......
......@@ -105,6 +105,15 @@ if $dnode4Vnodes != null then
goto show1
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step2
print ========= start dnode4
sql create dnode $hostname4
......@@ -132,6 +141,15 @@ if $dnode4Vnodes != 2 then
goto show2
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step3
print ========= drop dnode2
sql drop dnode $hostname2
......@@ -167,6 +185,15 @@ if $dnode4Vnodes != 3 then
goto show3
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
system sh/exec.sh -n dnode2 -s stop -x SIGINT
print ============================== step4
......@@ -195,6 +222,15 @@ if $dnode5Vnodes != 2 then
goto show4
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step5
print ========= drop dnode3
sql drop dnode $hostname3
......@@ -232,6 +268,15 @@ endi
system sh/exec.sh -n dnode3 -s stop -x SIGINT
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step6
sql create dnode $hostname6
system sh/exec.sh -n dnode6 -s start
......@@ -258,6 +303,15 @@ if $dnode6Vnodes != 2 then
goto show6
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step7
print ========= drop dnode4
sql drop dnode $hostname4
......@@ -294,6 +348,14 @@ if $dnode4Vnodes != null then
endi
system sh/exec.sh -n dnode4 -s stop -x SIGINT
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step8
sql create dnode $hostname7
......@@ -321,6 +383,15 @@ if $dnode7Vnodes != 2 then
goto show8
endi
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step9
print ========= drop dnode1
system sh/exec.sh -n dnode1 -s stop -x SIGINT
......@@ -335,15 +406,20 @@ sql show mnodes
$dnode1Role = $data2_1
$dnode4Role = $data2_4
$dnode5Role = $data2_5
print dnode1 ==> $dnode1Role
print dnode4 ==> $dnode4Role
print dnode5 ==> $dnode5Role
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
if $dnode1Role != offline then
return -1
endi
print ============================== step9.1
sleep 2000
system sh/exec.sh -n dnode1 -s start
$x = 0
......@@ -353,6 +429,19 @@ show9:
if $x == 20 then
return -1
endi
sql show mnodes
$dnode1Role = $data2_1
$dnode4Role = $data2_4
$dnode5Role = $data2_5
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
sql show dnodes -x show9
$dnode5Vnodes = $data2_5
print dnode5 $dnode5Vnodes
......@@ -374,6 +463,15 @@ endi
system sh/exec.sh -n dnode1 -s stop -x SIGINT
sleep 5000
sql show mnodes
print dnode1 ==> $data2_1
print dnode2 ==> $data2_2
print dnode3 ==> $data2_3
print dnode4 ==> $data2_4
print dnode5 ==> $data2_5
print dnode6 ==> $data2_6
print dnode7 ==> $data2_7
print ============================== step11
print ========= add db4
......
......@@ -21,9 +21,10 @@ system sh/exec.sh -n dnode1 -s start
sleep 3000
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
sleep 3000
print ======== step1
sql create database db replica 3 blocks 3
......
......@@ -18,13 +18,14 @@ system sh/cfg.sh -n dnode3 -c numOfMnodes -v 3
print ========== step1
system sh/exec.sh -n dnode1 -s start
sleep 3000
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sleep 3000
sql create dnode $hostname3
system sh/exec.sh -n dnode3 -s start
sleep 3000
sleep 5000
sql show dnodes
print dnode1 $data5_1
......
......@@ -31,8 +31,8 @@ sleep 3000
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
......
......@@ -13,8 +13,8 @@ system sh/exec.sh -n dnode1 -s start
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
$x = 0
......
......@@ -17,8 +17,8 @@ system sh/exec.sh -n dnode1 -s start
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
$x = 0
createDnode:
......
......@@ -21,10 +21,10 @@ system sh/exec.sh -n dnode1 -s start
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode3 -s start
sql create dnode $hostname4
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
system sh/exec.sh -n dnode4 -s start
$x = 0
......
......@@ -18,8 +18,8 @@ system sh/exec.sh -n dnode1 -s start
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
$x = 0
createDnode:
......
......@@ -21,10 +21,10 @@ system sh/exec.sh -n dnode1 -s start
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode3 -s start
sql create dnode $hostname4
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
system sh/exec.sh -n dnode4 -s start
$x = 0
createDnode:
......
......@@ -25,8 +25,8 @@ system sh/exec.sh -n dnode1 -s start
sleep 3000
sql connect
sql create dnode $hostname2
system sh/exec.sh -n dnode2 -s start
sql create dnode $hostname3
system sh/exec.sh -n dnode2 -s start
system sh/exec.sh -n dnode3 -s start
sleep 3000
......
......@@ -667,7 +667,7 @@ bool simExecuteNativeSqlCommand(SScript *script, char *rest, bool isSlow) {
TAOS_RES* pSql = NULL;
for (int attempt = 0; attempt < 3; ++attempt) {
for (int attempt = 0; attempt < 10; ++attempt) {
simLogSql(rest, false);
pSql = taos_query(script->taos, rest);
ret = taos_errno(pSql);
......
Markdown is supported
0% .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑!
想要评论请 注册