Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
taosdata
TDengine
提交
14de9676
TDengine
项目概览
taosdata
/
TDengine
大约 2 年 前同步成功
通知
1192
Star
22018
Fork
4786
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
TDengine
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
14de9676
编写于
7月 15, 2022
作者:
D
dingbo
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
docs: high-volume.md
上级
cd0ed59a
变更
13
展开全部
隐藏空白更改
内联
并排
Showing
13 changed file
with
544 addition
and
114 deletion
+544
-114
docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java
...ain/java/com/taos/example/highvolume/DataBaseMonitor.java
+47
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java
...in/java/com/taos/example/highvolume/FastWriteExample.java
+1
-46
docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java
...main/java/com/taos/example/highvolume/MockDataSource.java
+53
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java
...a/src/main/java/com/taos/example/highvolume/ReadTask.java
+1
-54
docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java
.../src/main/java/com/taos/example/highvolume/SQLWriter.java
+1
-4
docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java
...src/main/java/com/taos/example/highvolume/StmtWriter.java
+4
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java
.../src/main/java/com/taos/example/highvolume/WriteTask.java
+1
-3
docs/examples/java/src/main/resources/highvolume2.drawio
docs/examples/java/src/main/resources/highvolume2.drawio
+6
-6
docs/examples/python/highvolume_faster_queue.py
docs/examples/python/highvolume_faster_queue.py
+1
-1
docs/examples/python/mockdatasource.py
docs/examples/python/mockdatasource.py
+0
-0
docs/zh/07-develop/03-insert-data/05-high-volume.md
docs/zh/07-develop/03-insert-data/05-high-volume.md
+429
-0
docs/zh/07-develop/03-insert-data/high.png
docs/zh/07-develop/03-insert-data/high.png
+0
-0
docs/zh/07-develop/03-insert-data/highvolume.webp
docs/zh/07-develop/03-insert-data/highvolume.webp
+0
-0
未找到文件。
docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
import
java.sql.*
;
/**
* Prepare target database.
* Count total records in database periodically so that we can estimate the writing speed.
*/
public
class
DataBaseMonitor
{
private
Connection
conn
;
private
Statement
stmt
;
public
DataBaseMonitor
init
()
throws
SQLException
{
if
(
conn
==
null
)
{
String
jdbcURL
=
System
.
getenv
(
"TDENGINE_JDBC_URL"
);
conn
=
DriverManager
.
getConnection
(
jdbcURL
);
stmt
=
conn
.
createStatement
();
}
return
this
;
}
public
void
close
()
{
try
{
stmt
.
close
();
}
catch
(
SQLException
e
)
{
}
try
{
conn
.
close
();
}
catch
(
SQLException
e
)
{
}
}
public
void
prepareDatabase
()
throws
SQLException
{
stmt
.
execute
(
"DROP DATABASE IF EXISTS test"
);
stmt
.
execute
(
"CREATE DATABASE test"
);
stmt
.
execute
(
"CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"
);
}
public
Long
count
()
throws
SQLException
{
if
(!
stmt
.
isClosed
())
{
ResultSet
result
=
stmt
.
executeQuery
(
"SELECT count(*) from test.meters"
);
result
.
next
();
return
result
.
getLong
(
1
);
}
return
null
;
}
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java
浏览文件 @
14de9676
...
...
@@ -9,51 +9,7 @@ import java.util.List;
import
java.util.concurrent.ArrayBlockingQueue
;
import
java.util.concurrent.BlockingQueue
;
/**
* Prepare target database.
* Count total records in database periodically so that we can estimate the writing speed.
*/
class
DataBaseMonitor
{
private
Connection
conn
;
private
Statement
stmt
;
public
DataBaseMonitor
init
()
throws
SQLException
{
if
(
conn
==
null
)
{
String
jdbcURL
=
System
.
getenv
(
"TDENGINE_JDBC_URL"
);
conn
=
DriverManager
.
getConnection
(
jdbcURL
);
stmt
=
conn
.
createStatement
();
}
return
this
;
}
public
void
close
()
{
try
{
stmt
.
close
();
}
catch
(
SQLException
e
)
{
}
try
{
conn
.
close
();
}
catch
(
SQLException
e
)
{
}
}
public
void
prepareDatabase
()
throws
SQLException
{
stmt
.
execute
(
"DROP DATABASE IF EXISTS test"
);
stmt
.
execute
(
"CREATE DATABASE test"
);
stmt
.
execute
(
"CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"
);
}
public
Long
count
()
throws
SQLException
{
if
(!
stmt
.
isClosed
())
{
ResultSet
result
=
stmt
.
executeQuery
(
"SELECT count(*) from test.meters"
);
result
.
next
();
return
result
.
getLong
(
1
);
}
return
null
;
}
}
// ANCHOR: main
public
class
FastWriteExample
{
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
FastWriteExample
.
class
);
...
...
@@ -110,5 +66,4 @@ public class FastWriteExample {
lastCount
=
count
;
}
}
}
// ANCHOR_END: main
\ No newline at end of file
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
import
java.util.Iterator
;
/**
* Generate test data
*/
class
MockDataSource
implements
Iterator
{
private
String
tbNamePrefix
;
private
int
tableCount
;
private
long
maxRowsPerTable
=
1000000000L
;
// 100 milliseconds between two neighbouring rows.
long
startMs
=
System
.
currentTimeMillis
()
-
maxRowsPerTable
*
100
;
private
int
currentRow
=
0
;
private
int
currentTbId
=
-
1
;
// mock values
String
[]
location
=
{
"LosAngeles"
,
"SanDiego"
,
"Hollywood"
,
"Compton"
,
"San Francisco"
};
float
[]
current
=
{
8.8f
,
10.7f
,
9.9f
,
8.9f
,
9.4f
};
int
[]
voltage
=
{
119
,
116
,
111
,
113
,
118
};
float
[]
phase
=
{
0.32f
,
0.34f
,
0.33f
,
0.329f
,
0.141f
};
public
MockDataSource
(
String
tbNamePrefix
,
int
tableCount
)
{
this
.
tbNamePrefix
=
tbNamePrefix
;
this
.
tableCount
=
tableCount
;
}
@Override
public
boolean
hasNext
()
{
currentTbId
+=
1
;
if
(
currentTbId
==
tableCount
)
{
currentTbId
=
0
;
currentRow
+=
1
;
}
return
currentRow
<
maxRowsPerTable
;
}
@Override
public
String
next
()
{
long
ts
=
startMs
+
100
*
currentRow
;
int
groupId
=
currentTbId
%
5
==
0
?
currentTbId
/
5
:
currentTbId
/
5
+
1
;
StringBuilder
sb
=
new
StringBuilder
(
tbNamePrefix
+
"_"
+
currentTbId
+
","
);
// tbName
sb
.
append
(
ts
).
append
(
','
);
// ts
sb
.
append
(
current
[
currentRow
%
5
]).
append
(
','
);
// current
sb
.
append
(
voltage
[
currentRow
%
5
]).
append
(
','
);
// voltage
sb
.
append
(
phase
[
currentRow
%
5
]).
append
(
','
);
// phase
sb
.
append
(
location
[
currentRow
%
5
]).
append
(
','
);
// location
sb
.
append
(
groupId
);
// groupID
return
sb
.
toString
();
}
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java
浏览文件 @
14de9676
...
...
@@ -7,57 +7,6 @@ import java.util.Iterator;
import
java.util.List
;
import
java.util.concurrent.BlockingQueue
;
/**
* Generate test data
*/
class
MockDataSource
implements
Iterator
{
private
String
tbNamePrefix
;
private
int
tableCount
;
private
long
maxRowsPerTable
=
1000000000L
;
// 100 milliseconds between two neighbouring rows.
long
startMs
=
System
.
currentTimeMillis
()
-
maxRowsPerTable
*
100
;
private
int
currentRow
=
0
;
private
int
currentTbId
=
-
1
;
// mock values
String
[]
location
=
{
"LosAngeles"
,
"SanDiego"
,
"Hollywood"
,
"Compton"
,
"San Francisco"
};
float
[]
current
=
{
8.8f
,
10.7f
,
9.9f
,
8.9f
,
9.4f
};
int
[]
voltage
=
{
119
,
116
,
111
,
113
,
118
};
float
[]
phase
=
{
0.32f
,
0.34f
,
0.33f
,
0.329f
,
0.141f
};
public
MockDataSource
(
String
tbNamePrefix
,
int
tableCount
)
{
this
.
tbNamePrefix
=
tbNamePrefix
;
this
.
tableCount
=
tableCount
;
}
@Override
public
boolean
hasNext
()
{
currentTbId
+=
1
;
if
(
currentTbId
==
tableCount
)
{
currentTbId
=
0
;
currentRow
+=
1
;
}
return
currentRow
<
maxRowsPerTable
;
}
@Override
public
String
next
()
{
long
ts
=
startMs
+
100
*
currentRow
;
int
groupId
=
currentTbId
%
5
==
0
?
currentTbId
/
5
:
currentTbId
/
5
+
1
;
StringBuilder
sb
=
new
StringBuilder
(
tbNamePrefix
+
"_"
+
currentTbId
+
","
);
// tbName
sb
.
append
(
ts
).
append
(
','
);
// ts
sb
.
append
(
current
[
currentRow
%
5
]).
append
(
','
);
// current
sb
.
append
(
voltage
[
currentRow
%
5
]).
append
(
','
);
// voltage
sb
.
append
(
phase
[
currentRow
%
5
]).
append
(
','
);
// phase
sb
.
append
(
location
[
currentRow
%
5
]).
append
(
','
);
// location
sb
.
append
(
groupId
);
// groupID
return
sb
.
toString
();
}
}
// ANCHOR: ReadTask
class
ReadTask
implements
Runnable
{
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
ReadTask
.
class
);
private
final
int
taskId
;
...
...
@@ -106,6 +55,4 @@ class ReadTask implements Runnable {
logger
.
info
(
"stop"
);
this
.
active
=
false
;
}
}
// ANCHOR_END: ReadTask
\ No newline at end of file
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java
浏览文件 @
14de9676
...
...
@@ -7,8 +7,6 @@ import java.sql.*;
import
java.util.HashMap
;
import
java.util.Map
;
// ANCHOR: SQLWriter
/**
* A helper class encapsulate the logic of writing using SQL.
* <p>
...
...
@@ -204,5 +202,4 @@ public class SQLWriter {
}
catch
(
SQLException
e
)
{
}
}
}
// ANCHOR_END: SQLWriter
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
public
class
StmtWriter
{
}
docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java
浏览文件 @
14de9676
...
...
@@ -5,7 +5,6 @@ import org.slf4j.LoggerFactory;
import
java.util.concurrent.BlockingQueue
;
// ANCHOR: WriteTask
class
WriteTask
implements
Runnable
{
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
WriteTask
.
class
);
private
final
int
maxBatchSize
;
...
...
@@ -56,5 +55,4 @@ class WriteTask implements Runnable {
logger
.
info
(
"stop"
);
this
.
active
=
false
;
}
}
// ANCHOR_END: WriteTask
\ No newline at end of file
}
\ No newline at end of file
docs/examples/java/src/main/resources/highvolume2.drawio
浏览文件 @
14de9676
<mxfile host="65bd71144e">
<diagram id="_BjMg4p5x31hL4Gv-87s" name="第 1 页">
<mxGraphModel dx="
728" dy="412
" grid="0" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="900" background="none" math="0" shadow="0">
<mxGraphModel dx="
1259" dy="615
" grid="0" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="900" background="none" math="0" shadow="0">
<root>
<mxCell id="0"/>
<mxCell id="1" parent="0"/>
...
...
@@ -52,22 +52,22 @@
<mxPoint x="275" y="162.5" as="targetPoint"/>
</mxGeometry>
</mxCell>
<mxCell id="20" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
2
</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxCell id="20" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
1
</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxGeometry x="588" y="129" width="100" height="31" as="geometry"/>
</mxCell>
<mxCell id="21" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 2</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxGeometry x="588" y="175" width="100" height="33" as="geometry"/>
</mxCell>
<mxCell id="30" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;"
edge="1" parent="1" source="26" target="29
">
<mxCell id="30" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;"
parent="1" source="26" target="29" edge="1
">
<mxGeometry relative="1" as="geometry"/>
</mxCell>
<mxCell id="26" value="Queue
3" style="rounded=0;whiteSpace=wrap;html=1;fontSize=17;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent
="1">
<mxCell id="26" value="Queue
4" style="rounded=0;whiteSpace=wrap;html=1;fontSize=17;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex
="1">
<mxGeometry x="265" y="271" width="245" height="29" as="geometry"/>
</mxCell>
<mxCell id="28" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" vertex="1" parent
="1">
<mxCell id="28" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
3</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex
="1">
<mxGeometry x="588" y="221.5" width="100" height="33" as="geometry"/>
</mxCell>
<mxCell id="29" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1"
vertex="1" parent
="1">
<mxCell id="29" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1"
parent="1" vertex
="1">
<mxGeometry x="588" y="271" width="100" height="33" as="geometry"/>
</mxCell>
</root>
...
...
docs/examples/python/highvolume_faster_queue.py
浏览文件 @
14de9676
...
...
@@ -10,7 +10,7 @@ import time
import
os
from
multiprocessing
import
Process
from
faster_fifo
import
Queue
from
mockdataso
ru
ce
import
MockDataSource
from
mockdataso
ur
ce
import
MockDataSource
from
queue
import
Empty
from
typing
import
List
...
...
docs/examples/python/mockdataso
ru
ce.py
→
docs/examples/python/mockdataso
ur
ce.py
浏览文件 @
14de9676
文件已移动
docs/zh/07-develop/03-insert-data/05-high-volume.md
x
→
docs/zh/07-develop/03-insert-data/05-high-volume.md
浏览文件 @
14de9676
此差异已折叠。
点击以展开。
docs/zh/07-develop/03-insert-data/high.png
0 → 100644
浏览文件 @
14de9676
25.8 KB
docs/zh/07-develop/03-insert-data/highvolume.webp
浏览文件 @
14de9676
无法预览此类型文件
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录