Skip to content
体验新版
项目
组织
正在加载...
登录
切换导航
打开侧边栏
taosdata
TDengine
提交
14de9676
T
TDengine
项目概览
taosdata
/
TDengine
大约 2 年 前同步成功
通知
1192
Star
22018
Fork
4786
代码
文件
提交
分支
Tags
贡献者
分支图
Diff
Issue
1
列表
看板
标记
里程碑
合并请求
0
Wiki
0
Wiki
分析
仓库
DevOps
项目成员
Pages
T
TDengine
项目概览
项目概览
详情
发布
仓库
仓库
文件
提交
分支
标签
贡献者
分支图
比较
Issue
1
Issue
1
列表
看板
标记
里程碑
合并请求
0
合并请求
0
Pages
分析
分析
仓库分析
DevOps
Wiki
0
Wiki
成员
成员
收起侧边栏
关闭侧边栏
动态
分支图
创建新Issue
提交
Issue看板
提交
14de9676
编写于
7月 15, 2022
作者:
D
dingbo
浏览文件
操作
浏览文件
下载
电子邮件补丁
差异文件
docs: high-volume.md
上级
cd0ed59a
变更
13
展开全部
显示空白变更内容
内联
并排
Showing
13 changed file
with
544 addition
and
114 deletion
+544
-114
docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java
...ain/java/com/taos/example/highvolume/DataBaseMonitor.java
+47
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java
...in/java/com/taos/example/highvolume/FastWriteExample.java
+1
-46
docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java
...main/java/com/taos/example/highvolume/MockDataSource.java
+53
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java
...a/src/main/java/com/taos/example/highvolume/ReadTask.java
+1
-54
docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java
.../src/main/java/com/taos/example/highvolume/SQLWriter.java
+1
-4
docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java
...src/main/java/com/taos/example/highvolume/StmtWriter.java
+4
-0
docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java
.../src/main/java/com/taos/example/highvolume/WriteTask.java
+1
-3
docs/examples/java/src/main/resources/highvolume2.drawio
docs/examples/java/src/main/resources/highvolume2.drawio
+6
-6
docs/examples/python/highvolume_faster_queue.py
docs/examples/python/highvolume_faster_queue.py
+1
-1
docs/examples/python/mockdatasource.py
docs/examples/python/mockdatasource.py
+0
-0
docs/zh/07-develop/03-insert-data/05-high-volume.md
docs/zh/07-develop/03-insert-data/05-high-volume.md
+429
-0
docs/zh/07-develop/03-insert-data/high.png
docs/zh/07-develop/03-insert-data/high.png
+0
-0
docs/zh/07-develop/03-insert-data/highvolume.webp
docs/zh/07-develop/03-insert-data/highvolume.webp
+0
-0
未找到文件。
docs/examples/java/src/main/java/com/taos/example/highvolume/DataBaseMonitor.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
import
java.sql.*
;
/**
* Prepare target database.
* Count total records in database periodically so that we can estimate the writing speed.
*/
public
class
DataBaseMonitor
{
private
Connection
conn
;
private
Statement
stmt
;
public
DataBaseMonitor
init
()
throws
SQLException
{
if
(
conn
==
null
)
{
String
jdbcURL
=
System
.
getenv
(
"TDENGINE_JDBC_URL"
);
conn
=
DriverManager
.
getConnection
(
jdbcURL
);
stmt
=
conn
.
createStatement
();
}
return
this
;
}
public
void
close
()
{
try
{
stmt
.
close
();
}
catch
(
SQLException
e
)
{
}
try
{
conn
.
close
();
}
catch
(
SQLException
e
)
{
}
}
public
void
prepareDatabase
()
throws
SQLException
{
stmt
.
execute
(
"DROP DATABASE IF EXISTS test"
);
stmt
.
execute
(
"CREATE DATABASE test"
);
stmt
.
execute
(
"CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"
);
}
public
Long
count
()
throws
SQLException
{
if
(!
stmt
.
isClosed
())
{
ResultSet
result
=
stmt
.
executeQuery
(
"SELECT count(*) from test.meters"
);
result
.
next
();
return
result
.
getLong
(
1
);
}
return
null
;
}
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/FastWriteExample.java
浏览文件 @
14de9676
...
@@ -9,51 +9,7 @@ import java.util.List;
...
@@ -9,51 +9,7 @@ import java.util.List;
import
java.util.concurrent.ArrayBlockingQueue
;
import
java.util.concurrent.ArrayBlockingQueue
;
import
java.util.concurrent.BlockingQueue
;
import
java.util.concurrent.BlockingQueue
;
/**
* Prepare target database.
* Count total records in database periodically so that we can estimate the writing speed.
*/
class
DataBaseMonitor
{
private
Connection
conn
;
private
Statement
stmt
;
public
DataBaseMonitor
init
()
throws
SQLException
{
if
(
conn
==
null
)
{
String
jdbcURL
=
System
.
getenv
(
"TDENGINE_JDBC_URL"
);
conn
=
DriverManager
.
getConnection
(
jdbcURL
);
stmt
=
conn
.
createStatement
();
}
return
this
;
}
public
void
close
()
{
try
{
stmt
.
close
();
}
catch
(
SQLException
e
)
{
}
try
{
conn
.
close
();
}
catch
(
SQLException
e
)
{
}
}
public
void
prepareDatabase
()
throws
SQLException
{
stmt
.
execute
(
"DROP DATABASE IF EXISTS test"
);
stmt
.
execute
(
"CREATE DATABASE test"
);
stmt
.
execute
(
"CREATE STABLE test.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"
);
}
public
Long
count
()
throws
SQLException
{
if
(!
stmt
.
isClosed
())
{
ResultSet
result
=
stmt
.
executeQuery
(
"SELECT count(*) from test.meters"
);
result
.
next
();
return
result
.
getLong
(
1
);
}
return
null
;
}
}
// ANCHOR: main
public
class
FastWriteExample
{
public
class
FastWriteExample
{
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
FastWriteExample
.
class
);
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
FastWriteExample
.
class
);
...
@@ -111,4 +67,3 @@ public class FastWriteExample {
...
@@ -111,4 +67,3 @@ public class FastWriteExample {
}
}
}
}
}
}
\ No newline at end of file
// ANCHOR_END: main
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/MockDataSource.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
import
java.util.Iterator
;
/**
* Generate test data
*/
class
MockDataSource
implements
Iterator
{
private
String
tbNamePrefix
;
private
int
tableCount
;
private
long
maxRowsPerTable
=
1000000000L
;
// 100 milliseconds between two neighbouring rows.
long
startMs
=
System
.
currentTimeMillis
()
-
maxRowsPerTable
*
100
;
private
int
currentRow
=
0
;
private
int
currentTbId
=
-
1
;
// mock values
String
[]
location
=
{
"LosAngeles"
,
"SanDiego"
,
"Hollywood"
,
"Compton"
,
"San Francisco"
};
float
[]
current
=
{
8.8f
,
10.7f
,
9.9f
,
8.9f
,
9.4f
};
int
[]
voltage
=
{
119
,
116
,
111
,
113
,
118
};
float
[]
phase
=
{
0.32f
,
0.34f
,
0.33f
,
0.329f
,
0.141f
};
public
MockDataSource
(
String
tbNamePrefix
,
int
tableCount
)
{
this
.
tbNamePrefix
=
tbNamePrefix
;
this
.
tableCount
=
tableCount
;
}
@Override
public
boolean
hasNext
()
{
currentTbId
+=
1
;
if
(
currentTbId
==
tableCount
)
{
currentTbId
=
0
;
currentRow
+=
1
;
}
return
currentRow
<
maxRowsPerTable
;
}
@Override
public
String
next
()
{
long
ts
=
startMs
+
100
*
currentRow
;
int
groupId
=
currentTbId
%
5
==
0
?
currentTbId
/
5
:
currentTbId
/
5
+
1
;
StringBuilder
sb
=
new
StringBuilder
(
tbNamePrefix
+
"_"
+
currentTbId
+
","
);
// tbName
sb
.
append
(
ts
).
append
(
','
);
// ts
sb
.
append
(
current
[
currentRow
%
5
]).
append
(
','
);
// current
sb
.
append
(
voltage
[
currentRow
%
5
]).
append
(
','
);
// voltage
sb
.
append
(
phase
[
currentRow
%
5
]).
append
(
','
);
// phase
sb
.
append
(
location
[
currentRow
%
5
]).
append
(
','
);
// location
sb
.
append
(
groupId
);
// groupID
return
sb
.
toString
();
}
}
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/ReadTask.java
浏览文件 @
14de9676
...
@@ -7,57 +7,6 @@ import java.util.Iterator;
...
@@ -7,57 +7,6 @@ import java.util.Iterator;
import
java.util.List
;
import
java.util.List
;
import
java.util.concurrent.BlockingQueue
;
import
java.util.concurrent.BlockingQueue
;
/**
* Generate test data
*/
class
MockDataSource
implements
Iterator
{
private
String
tbNamePrefix
;
private
int
tableCount
;
private
long
maxRowsPerTable
=
1000000000L
;
// 100 milliseconds between two neighbouring rows.
long
startMs
=
System
.
currentTimeMillis
()
-
maxRowsPerTable
*
100
;
private
int
currentRow
=
0
;
private
int
currentTbId
=
-
1
;
// mock values
String
[]
location
=
{
"LosAngeles"
,
"SanDiego"
,
"Hollywood"
,
"Compton"
,
"San Francisco"
};
float
[]
current
=
{
8.8f
,
10.7f
,
9.9f
,
8.9f
,
9.4f
};
int
[]
voltage
=
{
119
,
116
,
111
,
113
,
118
};
float
[]
phase
=
{
0.32f
,
0.34f
,
0.33f
,
0.329f
,
0.141f
};
public
MockDataSource
(
String
tbNamePrefix
,
int
tableCount
)
{
this
.
tbNamePrefix
=
tbNamePrefix
;
this
.
tableCount
=
tableCount
;
}
@Override
public
boolean
hasNext
()
{
currentTbId
+=
1
;
if
(
currentTbId
==
tableCount
)
{
currentTbId
=
0
;
currentRow
+=
1
;
}
return
currentRow
<
maxRowsPerTable
;
}
@Override
public
String
next
()
{
long
ts
=
startMs
+
100
*
currentRow
;
int
groupId
=
currentTbId
%
5
==
0
?
currentTbId
/
5
:
currentTbId
/
5
+
1
;
StringBuilder
sb
=
new
StringBuilder
(
tbNamePrefix
+
"_"
+
currentTbId
+
","
);
// tbName
sb
.
append
(
ts
).
append
(
','
);
// ts
sb
.
append
(
current
[
currentRow
%
5
]).
append
(
','
);
// current
sb
.
append
(
voltage
[
currentRow
%
5
]).
append
(
','
);
// voltage
sb
.
append
(
phase
[
currentRow
%
5
]).
append
(
','
);
// phase
sb
.
append
(
location
[
currentRow
%
5
]).
append
(
','
);
// location
sb
.
append
(
groupId
);
// groupID
return
sb
.
toString
();
}
}
// ANCHOR: ReadTask
class
ReadTask
implements
Runnable
{
class
ReadTask
implements
Runnable
{
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
ReadTask
.
class
);
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
ReadTask
.
class
);
private
final
int
taskId
;
private
final
int
taskId
;
...
@@ -107,5 +56,3 @@ class ReadTask implements Runnable {
...
@@ -107,5 +56,3 @@ class ReadTask implements Runnable {
this
.
active
=
false
;
this
.
active
=
false
;
}
}
}
}
\ No newline at end of file
// ANCHOR_END: ReadTask
\ No newline at end of file
docs/examples/java/src/main/java/com/taos/example/highvolume/SQLWriter.java
浏览文件 @
14de9676
...
@@ -7,8 +7,6 @@ import java.sql.*;
...
@@ -7,8 +7,6 @@ import java.sql.*;
import
java.util.HashMap
;
import
java.util.HashMap
;
import
java.util.Map
;
import
java.util.Map
;
// ANCHOR: SQLWriter
/**
/**
* A helper class encapsulate the logic of writing using SQL.
* A helper class encapsulate the logic of writing using SQL.
* <p>
* <p>
...
@@ -205,4 +203,3 @@ public class SQLWriter {
...
@@ -205,4 +203,3 @@ public class SQLWriter {
}
}
}
}
}
}
\ No newline at end of file
// ANCHOR_END: SQLWriter
docs/examples/java/src/main/java/com/taos/example/highvolume/StmtWriter.java
0 → 100644
浏览文件 @
14de9676
package
com.taos.example.highvolume
;
public
class
StmtWriter
{
}
docs/examples/java/src/main/java/com/taos/example/highvolume/WriteTask.java
浏览文件 @
14de9676
...
@@ -5,7 +5,6 @@ import org.slf4j.LoggerFactory;
...
@@ -5,7 +5,6 @@ import org.slf4j.LoggerFactory;
import
java.util.concurrent.BlockingQueue
;
import
java.util.concurrent.BlockingQueue
;
// ANCHOR: WriteTask
class
WriteTask
implements
Runnable
{
class
WriteTask
implements
Runnable
{
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
WriteTask
.
class
);
private
final
static
Logger
logger
=
LoggerFactory
.
getLogger
(
WriteTask
.
class
);
private
final
int
maxBatchSize
;
private
final
int
maxBatchSize
;
...
@@ -57,4 +56,3 @@ class WriteTask implements Runnable {
...
@@ -57,4 +56,3 @@ class WriteTask implements Runnable {
this
.
active
=
false
;
this
.
active
=
false
;
}
}
}
}
\ No newline at end of file
// ANCHOR_END: WriteTask
\ No newline at end of file
docs/examples/java/src/main/resources/highvolume2.drawio
浏览文件 @
14de9676
<mxfile host="65bd71144e">
<mxfile host="65bd71144e">
<diagram id="_BjMg4p5x31hL4Gv-87s" name="第 1 页">
<diagram id="_BjMg4p5x31hL4Gv-87s" name="第 1 页">
<mxGraphModel dx="
728" dy="412
" grid="0" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="900" background="none" math="0" shadow="0">
<mxGraphModel dx="
1259" dy="615
" grid="0" gridSize="10" guides="1" tooltips="1" connect="1" arrows="1" fold="1" page="1" pageScale="1" pageWidth="900" pageHeight="900" background="none" math="0" shadow="0">
<root>
<root>
<mxCell id="0"/>
<mxCell id="0"/>
<mxCell id="1" parent="0"/>
<mxCell id="1" parent="0"/>
...
@@ -52,22 +52,22 @@
...
@@ -52,22 +52,22 @@
<mxPoint x="275" y="162.5" as="targetPoint"/>
<mxPoint x="275" y="162.5" as="targetPoint"/>
</mxGeometry>
</mxGeometry>
</mxCell>
</mxCell>
<mxCell id="20" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
2
</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxCell id="20" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
1
</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxGeometry x="588" y="129" width="100" height="31" as="geometry"/>
<mxGeometry x="588" y="129" width="100" height="31" as="geometry"/>
</mxCell>
</mxCell>
<mxCell id="21" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 2</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxCell id="21" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 2</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex="1">
<mxGeometry x="588" y="175" width="100" height="33" as="geometry"/>
<mxGeometry x="588" y="175" width="100" height="33" as="geometry"/>
</mxCell>
</mxCell>
<mxCell id="30" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;"
edge="1" parent="1" source="26" target="29
">
<mxCell id="30" style="edgeStyle=none;html=1;exitX=1;exitY=0.5;exitDx=0;exitDy=0;entryX=0;entryY=0.5;entryDx=0;entryDy=0;"
parent="1" source="26" target="29" edge="1
">
<mxGeometry relative="1" as="geometry"/>
<mxGeometry relative="1" as="geometry"/>
</mxCell>
</mxCell>
<mxCell id="26" value="Queue
3" style="rounded=0;whiteSpace=wrap;html=1;fontSize=17;fillColor=#f8cecc;strokeColor=#b85450;" vertex="1" parent
="1">
<mxCell id="26" value="Queue
4" style="rounded=0;whiteSpace=wrap;html=1;fontSize=17;fillColor=#f8cecc;strokeColor=#b85450;" parent="1" vertex
="1">
<mxGeometry x="265" y="271" width="245" height="29" as="geometry"/>
<mxGeometry x="265" y="271" width="245" height="29" as="geometry"/>
</mxCell>
</mxCell>
<mxCell id="28" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" vertex="1" parent
="1">
<mxCell id="28" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k
3</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1" parent="1" vertex
="1">
<mxGeometry x="588" y="221.5" width="100" height="33" as="geometry"/>
<mxGeometry x="588" y="221.5" width="100" height="33" as="geometry"/>
</mxCell>
</mxCell>
<mxCell id="29" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1"
vertex="1" parent
="1">
<mxCell id="29" value="<font style="font-size: 14px"><font style="font-size: 14px">Write Tas</font><font style="font-size: 14px">k 4</font></font>" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#dae8fc;strokeColor=#6c8ebf;fontStyle=1"
parent="1" vertex
="1">
<mxGeometry x="588" y="271" width="100" height="33" as="geometry"/>
<mxGeometry x="588" y="271" width="100" height="33" as="geometry"/>
</mxCell>
</mxCell>
</root>
</root>
...
...
docs/examples/python/highvolume_faster_queue.py
浏览文件 @
14de9676
...
@@ -10,7 +10,7 @@ import time
...
@@ -10,7 +10,7 @@ import time
import
os
import
os
from
multiprocessing
import
Process
from
multiprocessing
import
Process
from
faster_fifo
import
Queue
from
faster_fifo
import
Queue
from
mockdataso
ru
ce
import
MockDataSource
from
mockdataso
ur
ce
import
MockDataSource
from
queue
import
Empty
from
queue
import
Empty
from
typing
import
List
from
typing
import
List
...
...
docs/examples/python/mockdataso
ru
ce.py
→
docs/examples/python/mockdataso
ur
ce.py
浏览文件 @
14de9676
文件已移动
docs/zh/07-develop/03-insert-data/05-high-volume.md
x
→
docs/zh/07-develop/03-insert-data/05-high-volume.md
浏览文件 @
14de9676
此差异已折叠。
点击以展开。
docs/zh/07-develop/03-insert-data/high.png
0 → 100644
浏览文件 @
14de9676
25.8 KB
docs/zh/07-develop/03-insert-data/highvolume.webp
浏览文件 @
14de9676
无法预览此类型文件
编辑
预览
Markdown
is supported
0%
请重试
或
添加新附件
.
添加附件
取消
You are about to add
0
people
to the discussion. Proceed with caution.
先完成此消息的编辑!
取消
想要评论请
注册
或
登录