Skip to content

T
TDengine

taosdata / TDengine
1 年多 前同步成功

通知 1185
Star 22017
Fork 4786
T
TDengine
提交 f336272e 编写于 作者: J jiajingbin
浏览文件
操作

[TD-11129]<test>: Finish taosadapter performance scripts

上级 55655b22
隐藏空白更改
内联 并排
Showing with 2267 addition and 224 deletion
tests/perftest-scripts/taosadapter_perftest/config/env_init.py 0 → 100644
浏览文件 @ f336272e
# -*- coding: utf-8 -*-
import yaml
import os
import time
from loguru import logger
current_time = time.strftime("%Y-%m-%d-%H:%M:%S", time.localtime(time.time()))
current_dir = os.path.dirname(os.path.realpath(__file__))
config_file = os.path.join(current_dir, '../config/perf_test.yaml')
f = open(config_file)
config = yaml.load(f)
log_file = os.path.join(current_dir, f'../log/performance_{current_time}.log')
logger.add(log_file)
logger.info(f'init env success, log will be export to {log_file}')
tests/perftest-scripts/taosadapter_perftest/config/perf_test.yaml 0 → 100644
浏览文件 @ f336272e
deploy_mode: no
taosd_autodeploy: False
install_package: /home/ubuntu/TDengine/release/TDengine-server-2.3.4.0-beta-Linux-x64.tar.gz
clean_env: True
hostname_prefix: vm
timeout: 10
taosd_autodeploy: False
taosd_dnode1:
ip: 192.168.1.85
port: 22
restful_port: 6041
telnet_port: 6051
username: root
password: tbase125!
modify_cfg: False
cfg:
dataDir: /data/lib/taos
logDir: /data/log/taos
taosd_cluster: False
taosadapter_separate_deploy: True
taosd_dnode2:
ip: 192.168.1.83
port: 22
restful_port: 6041
telnet_port: 6046
username: root
password: tbase125!
modify_cfg: False
cfg:
dataDir: /data/lib/taos
logDir: /data/log/taos
taosd_dnode3:
ip: 192.168.1.84
port: 22
restful_port: 6041
telnet_port: 6046
username: root
password: tbase125!
modify_cfg: False
cfg:
dataDir: /data/lib/taos
logDir: /data/log/taos
taosd_dnode4:
ip: 192.168.1.86
port: 22
restful_port: 6041
telnet_port: 6046
username: root
password: tbase125!
modify_cfg: False
cfg:
dataDir: /data/lib/taos
logDir: /data/log/taos
prometheus:
autodeploy: True
ip: 192.168.1.101
port: 22
username: root
password: cpwu101
scrape_interval: 3s
evaluation_interval: 1s
scrape_timeout: 3s
prometheus_addr: http://39.105.163.10:9000/prometheus-2.31.1.linux-amd64.tar.gz
node_exporter_addr: https://github.com/prometheus/node_exporter/releases/download/v1.3.1/node_exporter-1.3.1.linux-amd64.tar.gz
process_exporter_addr: https://github.com/ncabatoff/process-exporter/releases/download/v0.7.10/process-exporter-0.7.10.linux-amd64.tar.gz
jmeter:
autodeploy: False
aggregate_report: True
ip: 127.0.0.1
port: 22
username: root
password: tbase125!
jmeter_addr: https://dlcdn.apache.org//jmeter/binaries/apache-jmeter-5.4.1.tgz
testcases:
testcase1:
threads: 24
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase2:
threads: 32
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase3:
threads: 64
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase4:
threads: 100
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase5:
threads: 100
protocol: telnet-restful
taosadapter_count: 2
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase6:
threads: 100
protocol: telnet-restful
taosadapter_count: 3
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase7:
threads: 100
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1000000
tb_count: 1
row_count: 1
sleep_time: 60s
tag_count: 10
col_count: 1
testcase8:
threads: 100
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 5000000
row_count: 1
sleep_time: 60s
tag_count: 10
col_count: 1
testcase9:
threads: 100
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 100000
row_count: 1000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase10:
threads: 100
protocol: telnet-restful
taosadapter_count: 1
stb_count: 1
tb_count: 10
row_count: 10000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase11:
threads: 24
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase12:
threads: 32
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase13:
threads: 64
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 3000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase14:
threads: 100
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase15:
threads: 100
protocol: json
taosadapter_count: 2
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase16:
threads: 100
protocol: json
taosadapter_count: 3
stb_count: 1
tb_count: 1
row_count: 5000000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase17:
threads: 100
protocol: json
taosadapter_count: 1
stb_count: 1000000
tb_count: 1
row_count: 1
sleep_time: 60s
tag_count: 10
col_count: 1
testcase18:
threads: 100
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 5000000
row_count: 1
sleep_time: 60s
tag_count: 10
col_count: 1
testcase19:
threads: 100
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 100000
row_count: 1000
sleep_time: 60s
tag_count: 10
col_count: 1
testcase20:
threads: 100
protocol: json
taosadapter_count: 1
stb_count: 1
tb_count: 10
row_count: 10000000
sleep_time: 60s
tag_count: 10
col_count: 1
tests/perftest-scripts/taosadapter_perftest/config/taosadapter_performance_test.jmx 0 → 100644
浏览文件 @ f336272e
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">true</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="drop db" enabled="drop_db_status">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="drop db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">drop database if exists db_name</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://restful_ip:restful_port/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="create db" enabled="create_db_status">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="create db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">create database if not exists db_name precision &apos;ms&apos;</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://restful_ip:restful_port/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="taosadapter performance test" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">loop_count</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">perf_threads</stringProp>
<stringProp name="ThreadGroup.ramp_time"></stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="telnet-restful" enabled="telnet_restful_status">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">input_line</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://restful_ip:restful_port/opentsdb/v1/put/line_protocol/db_name</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<TCPSampler guiclass="TCPSamplerGui" testclass="TCPSampler" testname="telnet-tcp" enabled="telnet_tcp_status">
<stringProp name="TCPSampler.classname">org.apache.jmeter.protocol.tcp.sampler.TCPClientImpl</stringProp>
<stringProp name="TCPSampler.server">telnet_ip</stringProp>
<boolProp name="TCPSampler.reUseConnection">true</boolProp>
<stringProp name="TCPSampler.port">telnet_port</stringProp>
<boolProp name="TCPSampler.nodelay">false</boolProp>
<stringProp name="TCPSampler.timeout">1</stringProp>
<stringProp name="TCPSampler.ctimeout">500</stringProp>
<stringProp name="TCPSampler.request">input_line</stringProp>
<boolProp name="TCPSampler.closeConnection">false</boolProp>
<stringProp name="TCPSampler.EolByte">10</stringProp>
<stringProp name="ConfigTestElement.username"></stringProp>
<stringProp name="ConfigTestElement.password"></stringProp>
</TCPSampler>
<hashTree/>
</hashTree>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<ResultCollector guiclass="ViewResultsFullVisualizer" testclass="ResultCollector" testname="View Results Tree" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<CSVDataSet guiclass="TestBeanGUI" testclass="CSVDataSet" testname="CSV Data Set Config" enabled="true">
<stringProp name="delimiter">,</stringProp>
<stringProp name="fileEncoding">UTF-8</stringProp>
<stringProp name="filename">import_file</stringProp>
<boolProp name="ignoreFirstLine">false</boolProp>
<boolProp name="quotedData">false</boolProp>
<boolProp name="recycle">true</boolProp>
<stringProp name="shareMode">shareMode.all</stringProp>
<boolProp name="stopThread">false</boolProp>
<stringProp name="variableNames">row_csv_count,tb_csv_count,stb_csv_count</stringProp>
</CSVDataSet>
<hashTree/>
</hashTree>
</hashTree>
</jmeterTestPlan>
tests/perftest-scripts/taosadapter_perftest/old/README.CN.MD 0 → 100644
浏览文件 @ f336272e
# 注: 目前测试脚本需按测试报告中的环境说明手动部署taosd及taosadapter
- 测试报告: https://jira.taosdata.com:18090/pages/viewpage.action?pageId=127470422
# 1. 单实例集中部署
## 1.1 环境部署: 在192.168.1.85上直接编译develop分支启动taosd即可
## 1.2 脚本运行方法: 在192.168.1.83环境直接运行python3 /home/ubuntu/TDengine/tests/perftest-scripts/taosadapter_perftest/taosadapter_perftest.py即可, 可输出测试报告中3.4.1和3.5.1中的测试结果, 结果会实时打印到同目录下的taosadapter_perftest.log中
# 2. 多实例分离部署
## 2.1 环境部署: 在192.168.1.98上部署taosd, 在192.168.1.83、84、85、86环境将部署taosadapter, 配置对应的/etc/hosts, 并将/etc/taos/taos.cfg中的firstEp指向taosd的地址(vm98:6030)
## 2.2 脚本运行方法:
- 在192.168.1.93环境根据节点数量需求运行:
- jmeter -n -t opentsdb_insertRows83.jmx
- jmeter -n -t opentsdb_insertRows84.jmx
- jmeter -n -t opentsdb_insertRows85.jmx
- ...
- 需实时监控各节点cpu占用情况, jmeter结果会直接打印到控制台
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/old/README.EN.MD 0 → 100644
浏览文件 @ f336272e
# Note: The current test script needs to manually deploy taosd and taosadapter according to the environment description in the test report
- Test Report: https://jira.taosdata.com:18090/pages/viewpage.action?pageId=127470422
# 1. Single instance centralized deployment
## 1.1 Environment deployment: directly compile the develop branch on 192.168.1.85 and start taosd
## 1.2 Script running method: Run python3 /home/ubuntu/TDengine/tests/perftest-scripts/taosadapter_perftest/taosadapter_perftest.py directly in the 192.168.1.83 environment, which can output the test results in 3.4.1 and 3.5.1 in the test report. The results will be printed in real time to taosadapter_perftest.log in the same directory
# 2. Separate deployment of multiple instances
## 2.1 Environment deployment: Deploy taosd on 192.168.1.98, deploy taosadapter in 192.168.1.83, 84, 85, 86 environments, configure the corresponding /etc/hosts, and point the firstEp in /etc/taos/taos.cfg to taosd Address (vm98:6030)
## 2.2 Script running method:
- Run according to the count of nodes in the 192.168.1.93 environment:
- jmeter -n -t opentsdb_insertRows83.jmx
- jmeter -n -t opentsdb_insertRows84.jmx
- jmeter -n -t opentsdb_insertRows85.jmx
- ...
- You need to monitor the CPU occupancy of each node in real time, and the jmeter results will be printed directly to the console.
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/old/opentsdb_insertRows83.jmx 0 → 100644
浏览文件 @ f336272e
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<SetupThreadGroup guiclass="SetupThreadGroupGui" testclass="SetupThreadGroup" testname="setUp create db" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</SetupThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="create db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">create database if not exists test83 precision &apos;ms&apos;</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.83:6041/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="opentsdb_insertRows" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">10000000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">24</stringProp>
<stringProp name="ThreadGroup.ramp_time"></stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="insert rows" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">cpu.usage_user.rows ${ts_counter} 22.345567 arch=x64 datacenter=us-west-1b hostname=host_5 os=Ubuntu16 rack=13 region=us-west-1 service=10 service_environment=staging service_version=0 team=NYC</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.83:6041/opentsdb/v1/put/telnet/test83</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="ts_counter" enabled="true">
<stringProp name="CounterConfig.start">1614530008000</stringProp>
<stringProp name="CounterConfig.end"></stringProp>
<stringProp name="CounterConfig.incr">1</stringProp>
<stringProp name="CounterConfig.name">ts_counter</stringProp>
<stringProp name="CounterConfig.format"></stringProp>
<boolProp name="CounterConfig.per_user">false</boolProp>
</CounterConfig>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>
tests/perftest-scripts/taosadapter_perftest/old/opentsdb_insertRows84.jmx 0 → 100644
浏览文件 @ f336272e
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<SetupThreadGroup guiclass="SetupThreadGroupGui" testclass="SetupThreadGroup" testname="setUp create db" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</SetupThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="create db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">create database if not exists test84 precision &apos;ms&apos;</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.84:6041/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="opentsdb_insertRows" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1000000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">24</stringProp>
<stringProp name="ThreadGroup.ramp_time"></stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="insert rows" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">cpu.usage_user.rows ${ts_counter} 22.345567 arch=x64 datacenter=us-west-1b hostname=host_5 os=Ubuntu16 rack=13 region=us-west-1 service=10 service_environment=staging service_version=0 team=NYC</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.84:6041/opentsdb/v1/put/telnet/test84</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="ts_counter" enabled="true">
<stringProp name="CounterConfig.start">1614530008000</stringProp>
<stringProp name="CounterConfig.end"></stringProp>
<stringProp name="CounterConfig.incr">1</stringProp>
<stringProp name="CounterConfig.name">ts_counter</stringProp>
<stringProp name="CounterConfig.format"></stringProp>
<boolProp name="CounterConfig.per_user">false</boolProp>
</CounterConfig>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>
tests/perftest-scripts/taosadapter_perftest/old/opentsdb_insertRows85.jmx 0 → 100644
浏览文件 @ f336272e
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<SetupThreadGroup guiclass="SetupThreadGroupGui" testclass="SetupThreadGroup" testname="setUp create db" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</SetupThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="create db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">create database if not exists test85 precision &apos;ms&apos;</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.85:6041/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="opentsdb_insertRows" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1000000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">24</stringProp>
<stringProp name="ThreadGroup.ramp_time"></stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="insert rows" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">cpu.usage_user.rows ${ts_counter} 22.345567 arch=x64 datacenter=us-west-1b hostname=host_5 os=Ubuntu16 rack=13 region=us-west-1 service=10 service_environment=staging service_version=0 team=NYC</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.85:6041/opentsdb/v1/put/telnet/test85</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="ts_counter" enabled="true">
<stringProp name="CounterConfig.start">1614530008000</stringProp>
<stringProp name="CounterConfig.end"></stringProp>
<stringProp name="CounterConfig.incr">1</stringProp>
<stringProp name="CounterConfig.name">ts_counter</stringProp>
<stringProp name="CounterConfig.format"></stringProp>
<boolProp name="CounterConfig.per_user">false</boolProp>
</CounterConfig>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>
tests/perftest-scripts/taosadapter_perftest/old/opentsdb_insertRows86.jmx 0 → 100644
浏览文件 @ f336272e
<?xml version="1.0" encoding="UTF-8"?>
<jmeterTestPlan version="1.2" properties="5.0" jmeter="5.4.1">
<hashTree>
<TestPlan guiclass="TestPlanGui" testclass="TestPlan" testname="Test Plan" enabled="true">
<stringProp name="TestPlan.comments"></stringProp>
<boolProp name="TestPlan.functional_mode">false</boolProp>
<boolProp name="TestPlan.tearDown_on_shutdown">true</boolProp>
<boolProp name="TestPlan.serialize_threadgroups">false</boolProp>
<elementProp name="TestPlan.user_defined_variables" elementType="Arguments" guiclass="ArgumentsPanel" testclass="Arguments" testname="User Defined Variables" enabled="true">
<collectionProp name="Arguments.arguments"/>
</elementProp>
<stringProp name="TestPlan.user_define_classpath"></stringProp>
</TestPlan>
<hashTree>
<SetupThreadGroup guiclass="SetupThreadGroupGui" testclass="SetupThreadGroup" testname="setUp create db" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">1</stringProp>
<stringProp name="ThreadGroup.ramp_time">1</stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</SetupThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="create db" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">create database if not exists test86 precision &apos;ms&apos;</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.86:6041/rest/sql</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
</hashTree>
<ThreadGroup guiclass="ThreadGroupGui" testclass="ThreadGroup" testname="opentsdb_insertRows" enabled="true">
<stringProp name="ThreadGroup.on_sample_error">continue</stringProp>
<elementProp name="ThreadGroup.main_controller" elementType="LoopController" guiclass="LoopControlPanel" testclass="LoopController" testname="Loop Controller" enabled="true">
<boolProp name="LoopController.continue_forever">false</boolProp>
<stringProp name="LoopController.loops">1000000</stringProp>
</elementProp>
<stringProp name="ThreadGroup.num_threads">24</stringProp>
<stringProp name="ThreadGroup.ramp_time"></stringProp>
<boolProp name="ThreadGroup.scheduler">false</boolProp>
<stringProp name="ThreadGroup.duration"></stringProp>
<stringProp name="ThreadGroup.delay"></stringProp>
<boolProp name="ThreadGroup.same_user_on_next_iteration">true</boolProp>
</ThreadGroup>
<hashTree>
<HTTPSamplerProxy guiclass="HttpTestSampleGui" testclass="HTTPSamplerProxy" testname="insert rows" enabled="true">
<boolProp name="HTTPSampler.postBodyRaw">true</boolProp>
<elementProp name="HTTPsampler.Arguments" elementType="Arguments">
<collectionProp name="Arguments.arguments">
<elementProp name="" elementType="HTTPArgument">
<boolProp name="HTTPArgument.always_encode">false</boolProp>
<stringProp name="Argument.value">cpu.usage_user.rows ${ts_counter} 22.345567 arch=x64 datacenter=us-west-1b hostname=host_5 os=Ubuntu16 rack=13 region=us-west-1 service=10 service_environment=staging service_version=0 team=NYC</stringProp>
<stringProp name="Argument.metadata">=</stringProp>
</elementProp>
</collectionProp>
</elementProp>
<stringProp name="HTTPSampler.domain"></stringProp>
<stringProp name="HTTPSampler.port"></stringProp>
<stringProp name="HTTPSampler.protocol"></stringProp>
<stringProp name="HTTPSampler.contentEncoding"></stringProp>
<stringProp name="HTTPSampler.path">http://192.168.1.86:6041/opentsdb/v1/put/telnet/test86</stringProp>
<stringProp name="HTTPSampler.method">POST</stringProp>
<boolProp name="HTTPSampler.follow_redirects">true</boolProp>
<boolProp name="HTTPSampler.auto_redirects">false</boolProp>
<boolProp name="HTTPSampler.use_keepalive">true</boolProp>
<boolProp name="HTTPSampler.DO_MULTIPART_POST">false</boolProp>
<stringProp name="HTTPSampler.embedded_url_re"></stringProp>
<stringProp name="HTTPSampler.connect_timeout"></stringProp>
<stringProp name="HTTPSampler.response_timeout"></stringProp>
</HTTPSamplerProxy>
<hashTree/>
<ResultCollector guiclass="StatVisualizer" testclass="ResultCollector" testname="Aggregate Report" enabled="true">
<boolProp name="ResultCollector.error_logging">false</boolProp>
<objProp>
<name>saveConfig</name>
<value class="SampleSaveConfiguration">
<time>true</time>
<latency>true</latency>
<timestamp>true</timestamp>
<success>true</success>
<label>true</label>
<code>true</code>
<message>true</message>
<threadName>true</threadName>
<dataType>true</dataType>
<encoding>false</encoding>
<assertions>true</assertions>
<subresults>true</subresults>
<responseData>false</responseData>
<samplerData>false</samplerData>
<xml>false</xml>
<fieldNames>true</fieldNames>
<responseHeaders>false</responseHeaders>
<requestHeaders>false</requestHeaders>
<responseDataOnError>false</responseDataOnError>
<saveAssertionResultsFailureMessage>true</saveAssertionResultsFailureMessage>
<assertionsResultsToSave>0</assertionsResultsToSave>
<bytes>true</bytes>
<sentBytes>true</sentBytes>
<url>true</url>
<threadCounts>true</threadCounts>
<idleTime>true</idleTime>
<connectTime>true</connectTime>
</value>
</objProp>
<stringProp name="filename"></stringProp>
</ResultCollector>
<hashTree/>
<HeaderManager guiclass="HeaderPanel" testclass="HeaderManager" testname="HTTP Header Manager" enabled="true">
<collectionProp name="HeaderManager.headers">
<elementProp name="" elementType="Header">
<stringProp name="Header.name">Authorization</stringProp>
<stringProp name="Header.value">Basic cm9vdDp0YW9zZGF0YQ==</stringProp>
</elementProp>
</collectionProp>
</HeaderManager>
<hashTree/>
<CounterConfig guiclass="CounterConfigGui" testclass="CounterConfig" testname="ts_counter" enabled="true">
<stringProp name="CounterConfig.start">1614530008000</stringProp>
<stringProp name="CounterConfig.end"></stringProp>
<stringProp name="CounterConfig.incr">1</stringProp>
<stringProp name="CounterConfig.name">ts_counter</stringProp>
<stringProp name="CounterConfig.format"></stringProp>
<boolProp name="CounterConfig.per_user">false</boolProp>
</CounterConfig>
<hashTree/>
</hashTree>
</hashTree>
</hashTree>
</jmeterTestPlan>
tests/perftest-scripts/taosadapter_perftest/run_performance.py 0 → 100644
浏览文件 @ f336272e
from config.env_init import *
from src.common.common import Common
from src.common.dnodes import Dnodes
from src.common.monitor import Monitor
from src.util.jmeter import Jmeter
class RunPerformance:
def __init__(self):
self.COM = Common()
self.current_dir = os.path.dirname(os.path.realpath(__file__))
def runJmeter(self):
for key, value in config['testcases'].items():
jmx_file_list = list()
logger.info(f'executing {key}')
for jmx_file in self.COM.genJmxFile(key)[:value["taosadapter_count"]]:
jmx_filename = jmx_file.split('/')[-1]
import_file_name = jmx_filename.replace('jmx', 'txt')
import_file = os.path.join(self.current_dir, f'./config/{import_file_name}')
loop_count = self.COM.getLoopCount(value["stb_count"], value["tb_count"], value["row_count"], value["threads"])
self.COM.genMixStbTbRows(import_file, value["stb_count"], value["tb_count"], value["row_count"])
input_line = self.COM.genProtocolLine(value["protocol"], value["tag_count"])
with open(jmx_file, 'r', encoding='utf-8') as f:
file_data = ""
for line in f:
if value['protocol'] == 'telnet-tcp':
if "telnet_tcp_status" in line:
line = line.replace("telnet_tcp_status", "true")
if value['protocol'] == 'telnet-restful' or value['protocol'] == 'json':
if "drop_db_status" in line:
line = line.replace("drop_db_status", "true")
if "create_db_status" in line:
line = line.replace("create_db_status", "true")
if "telnet_restful_status" in line:
line = line.replace("telnet_restful_status", "true")
if "line_protocol" in line:
if value['protocol'] == 'telnet-restful':
line = line.replace("line_protocol", 'telnet')
elif value['protocol'] == 'json':
line = line.replace("line_protocol", 'json')
else:
pass
if "db_name" in line:
db_name = jmx_filename.split('.')[0]
line = line.replace("db_name", db_name)
if "import_file" in line:
line = line.replace("import_file", import_file)
if "input_line" in line:
line = line.replace("input_line", input_line)
if "perf_threads" in line:
line = line.replace("perf_threads", str(value['threads']))
if "loop_count" in line:
line = line.replace("loop_count", str(loop_count))
file_data += line
with open(jmx_file, "w", encoding="utf-8") as f:
f.write(file_data)
jmx_file_list.append(jmx_file)
jmeter_cmd_list = self.COM.genJmeterCmd(jmx_file_list)
self.COM.multiThreadRun(self.COM.genJmeterThreads(jmeter_cmd_list))
time.sleep(int(''.join(list(filter(str.isdigit, str(value["sleep_time"]))))))
if __name__ == '__main__':
current_dir = os.path.dirname(os.path.realpath(__file__))
log_dir = os.path.join(current_dir, 'log')
Performance = RunPerformance()
DNODES = Dnodes()
MONITOR = Monitor()
JMETER = Jmeter()
if config['deploy_mode'] == "auto":
if config['taosd_autodeploy']:
DNODES.deployNodes()
if config["prometheus"]["autodeploy"]:
MONITOR.deployAllNodeExporters()
MONITOR.deployAllProcessExporters()
MONITOR.deployPrometheus()
MONITOR.deployGrafana()
if config["jmeter"]["autodeploy"]:
JMETER.deployJmeter()
Performance.runJmeter()
tests/perftest-scripts/taosadapter_perftest/src/common/common.py 0 → 100644
浏览文件 @ f336272e
import sys
sys.path.append("../../")
from config.env_init import *
import shutil
import threading
import time
import json
class Common:
def __init__(self):
self.ip_list = list()
self.current_dir = os.path.dirname(os.path.realpath(__file__))
self.base_jmx_file = os.path.join(self.current_dir, '../../config/taosadapter_performance_test.jmx')
self.log_dir = os.path.join(self.current_dir, '../../log')
def exec_local_cmd(self,shell_cmd):
logger.info(f'executing cmd: {shell_cmd}')
result = os.popen(shell_cmd).read().strip()
logger.info(result)
return result
def genTelnetMulTagStr(self, count):
tag_str = ""
for i in range(1, count):
if i < (count-1):
tag_str += f't{i}={i} '
else:
tag_str += f't{i}={i}'
return tag_str
def genJsonMulTagDict(self, count):
tag_dict = dict()
for i in range(1, count):
tag_dict[f"t{i}"] = f"{i}"
return tag_dict
def genProtocolLine(self, protocol, tag_count, col_count=None):
if protocol == "telnet-restful":
base_str = 'stb_${stb_csv_count} ${row_csv_count} 32.261068286779754 t0=${tb_csv_count} '
tag_str = self.genTelnetMulTagStr(tag_count)
telnet_line = base_str + tag_str
return telnet_line
elif protocol == "telnet-tcp":
base_str = 'tstb_${stb_csv_count} ${row_csv_count} 32.261068286779754 t0=${tb_csv_count} '
tag_str = self.genTelnetMulTagStr(tag_count)
telnet_line = base_str + tag_str + '${__unescape(\r\n)}'
return telnet_line
elif protocol == "json":
base_tag_dict = {"t0":"${tb_csv_count}"}
dict_merged = base_tag_dict.copy()
dict_merged.update(self.genJsonMulTagDict(tag_count))
json_line = '{"metric": "stb_${stb_csv_count}", "timestamp":${row_csv_count}, "value":32.261068286779754, ' + f'"tags": {dict_merged}' + '}'
return json_line.replace('\'','"')
elif protocol == "influxdb":
# TODO
pass
else:
pass
def genMixStbTbRows(self, filename, stb_count, tb_count, row_count):
if stb_count == 0:
stb_count = 1
if tb_count == 0:
tb_count = 1
if row_count == 0:
row_count = 1
logger.info(f'generating import data file: {filename}')
ts_start = 1614530008000
with open(filename, "w", encoding="utf-8") as f_w:
for k in range(stb_count):
for i in range(tb_count):
for j in range(row_count):
input_line = str(ts_start) + "," + str(i) + "," + str(k) + '\n'
ts_start += 1
f_w.write(input_line)
def genJmxFile(self, testcase):
des_jmx_file_list = list()
base_jmx_file = os.path.join(self.current_dir, '../../config/taosadapter_performance_test.jmx')
count_flag = 0
if config["taosadapter_separate_deploy"]:
for key in config:
if "taosd_dnode" in str(key) and "taosd_dnode1" not in str(key):
if count_flag < int(config['testcases'][testcase]['taosadapter_count']):
count_flag += 1
else:
break
des_jmx_file = os.path.join(self.current_dir, f'../../config/{testcase}_{key}.jmx')
shutil.copyfile(base_jmx_file, des_jmx_file)
with open(des_jmx_file, 'r', encoding='utf-8') as f:
file_data = ""
for line in f:
if "restful_ip" in line:
line = line.replace("restful_ip", config[key]['ip'])
if "restful_port" in line:
line = line.replace("restful_port", str(config[key]['restful_port']))
if "telnet_ip" in line:
line = line.replace("telnet_ip", config[key]['ip'])
if "telnet_port" in line:
line = line.replace("telnet_port", str(config[key]['telnet_port']))
# if "db_name" in line:
# line = line.replace("db_name", key)
file_data += line
with open(des_jmx_file, "w", encoding="utf-8") as f:
f.write(file_data)
des_jmx_file_list.append(des_jmx_file)
else:
des_jmx_file = os.path.join(self.current_dir, f'../../config/taosd_dnode1.jmx')
shutil.copyfile(base_jmx_file, des_jmx_file)
with open(des_jmx_file, 'r', encoding='utf-8') as f:
file_data = ""
for line in f:
if "restful_ip" in line:
line = line.replace("restful_ip", config['taosd_dnode1']['ip'])
if "restful_port" in line:
line = line.replace("restful_port", str(config['taosd_dnode1']['restful_port']))
if "telnet_ip" in line:
line = line.replace("telnet_ip", config['taosd_dnode1']['ip'])
if "telnet_port" in line:
line = line.replace("telnet_port", str(config['taosd_dnode1']['telnet_port']))
# if "db_name" in line:
# line = line.replace("db_name", "taosd_dnode1")
file_data += line
with open(des_jmx_file, "w", encoding="utf-8") as f:
f.write(file_data)
des_jmx_file_list.append(des_jmx_file)
return des_jmx_file_list
def getLoopCount(self, stb_count, tb_count, row_count, threads):
if (stb_count * tb_count * row_count) % threads == 0:
loop_count = int((stb_count * tb_count * row_count) / threads)
else:
loop_count = int((stb_count * tb_count * row_count) / threads) + 1
return loop_count
def recreateReportDir(self, path):
'''
recreate jmeter report path
'''
if os.path.exists(path):
self.exec_local_cmd(f'ls {path} | grep -v performance_ | xargs rm -rf')
else:
os.makedirs(path)
def genJmeterCmd(self, jmx_file_list):
jmeter_cmd_list = list()
for jmx_file in jmx_file_list:
jmeter_cmd = f'jmeter -n -t {jmx_file}'
if config['jmeter']['aggregate_report']:
current_time = time.strftime("%Y_%m_%d_%H_%M_%S", time.localtime(time.time()))
jmx_filename = jmx_file.split('/')[-1].replace('.jmx', '')
jmx_filelog = f'{jmx_filename}_{current_time}'
jmeter_report_dir = f'{self.log_dir}/{jmx_filelog}'
self.recreateReportDir(jmeter_report_dir)
jmeter_cmd += f' -l {jmeter_report_dir}/{jmx_filelog}.log -e -o {jmeter_report_dir}'
jmeter_cmd_list.append(jmeter_cmd)
return jmeter_cmd_list
def genJmeterThreads(self, jmeter_cmd_list):
tlist = list()
for jmeter_cmd in jmeter_cmd_list:
t = threading.Thread(target=self.exec_local_cmd, args=(jmeter_cmd,))
tlist.append(t)
return tlist
def multiThreadRun(self, tlist):
for t in tlist:
t.start()
for t in tlist:
t.join()
if __name__ == '__main__':
com = Common()
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/src/common/dnodes.py 0 → 100644
浏览文件 @ f336272e
import sys
import json
sys.path.append("../../")
from config.env_init import *
from src.util.RemoteModule import RemoteModule
class Dnode:
def __init__(self, index, dnode_ip, dnode_port, dnode_username, dnode_password):
self.install_package = config["install_package"]
self.hostname_prefix = config["hostname_prefix"]
self.ip_suffix = dnode_ip.split('.')[-1]
self.dnode_name = self.hostname_prefix + self.ip_suffix
self.index = index
self.dnode_dict = config[f'taosd_dnode{self.index}']
self.dnode_ip = dnode_ip
self.dnode_port = dnode_port
self.dnode_username = dnode_username
self.dnode_password = dnode_password
self.dnode_conn = RemoteModule(self.dnode_ip, self.dnode_port, self.dnode_username, self.dnode_password)
if self.dnode_username == "root":
self.home_dir = "/root"
else:
self.home_dir = f"/home/{self.dnode_username}"
def installPackage(self):
if bool(int(self.dnode_conn.exec_cmd(f'cat /etc/os-release | grep ubuntu >> /dev/null && echo 1 || echo 0'))):
package_list = ["wget", "screen"]
for package in package_list:
if not bool(int(self.dnode_conn.exec_cmd(f'sudo dpkg -s {package} >> /dev/null && echo 1 || echo 0'))):
self.dnode_conn.exec_cmd(f'apt update -y && apt install -y {package}')
elif bool(int(self.dnode_conn.exec_cmd(f'cat /etc/os-release | grep centos >> /dev/null && echo 1 || echo 0'))):
package_list = ["wget", "screen"]
for package in package_list:
if not bool(int(self.dnode_conn.exec_cmd(f'sudo rpm -qa | grep {package} >> /dev/null && echo 1 || echo 0'))):
self.dnode_conn.exec_cmd(f'yum update -y && yum install -y {package}')
else:
pass
def startTaosd(self):
logger.info(f'{self.dnode_ip}: starting taosd')
self.dnode_conn.exec_cmd("sudo systemctl start taosd")
def stopTaosd(self):
logger.info(f'{self.dnode_ip}: stopping taosd')
self.dnode_conn.exec_cmd("sudo systemctl stop taosd")
def killTaosd(self):
logger.info(f'{self.dnode_ip}: killing taosd')
self.dnode_conn.exec_cmd("ps -ef | grep -w taosd | grep -v grep | awk \'{print $2}\' | sudo xargs kill -9")
def restartTaosd(self):
logger.info(f'{self.dnode_ip}: restarting taosd')
self.dnode_conn.exec_cmd("sudo systemctl restart taosd")
def startTaosadapter(self):
logger.info(f'{self.dnode_ip}: starting taosadapter')
self.dnode_conn.exec_cmd("sudo systemctl start taosadapter")
def stopTaosadapter(self):
logger.info(f'{self.dnode_ip}: stopping taosadapter')
self.dnode_conn.exec_cmd("sudo systemctl stop taosadapter")
def killTaosadapter(self):
logger.info(f'{self.dnode_ip}: killing taosadapter')
self.dnode_conn.exec_cmd("ps -ef | grep -w taosadapter | grep -v grep | awk \'{print $2}\' | sudo xargs kill -9")
def restartTaosadapter(self):
logger.info(f'{self.dnode_ip}: restarting taosadapter')
self.dnode_conn.exec_cmd("sudo systemctl restart taosadapter")
def rmTaosd(self):
logger.info(f'{self.dnode_ip}: removing taosd')
self.dnode_conn.exec_cmd("rmtaos")
def rmTaosdLog(self):
logger.info(f'{self.dnode_ip}: removing taosd log')
if self.dnode_dict["modify_cfg"]:
self.dnode_conn.exec_cmd(f'sudo rm -rf {self.dnode_dict["cfg"]["logDir"]}/*')
else:
self.dnode_conn.exec_cmd("sudo rm -rf /var/log/taos/*")
def rmTaosdData(self):
logger.info(f'{self.dnode_ip}: removing taosd data')
if self.dnode_dict["modify_cfg"]:
self.dnode_conn.exec_cmd(f'sudo rm -rf {self.dnode_dict["cfg"]["dataDir"]}/*')
else:
self.dnode_conn.exec_cmd("sudo rm -rf /var/lib/taos/*")
def rmTaosCfg(self):
logger.info(f'{self.dnode_ip}: removing taos.cfg')
self.dnode_conn.exec_cmd("sudo rm -rf /etc/taos/taos.cfg")
def modifyTaosCfg(self, firstEp=None):
hostname = self.configHostname()
if self.dnode_dict["modify_cfg"]:
logger.info(f'{self.dnode_ip}: modify /etc/taos/taos.cfg')
for key, value in self.dnode_dict['cfg'].items():
self.createRemoteDir(value)
self.dnode_conn.exec_cmd(f'echo {key} {value} >> /etc/taos/taos.cfg')
if firstEp is not None:
self.dnode_conn.exec_cmd(f'echo "firstEp {firstEp}" >> /etc/taos/taos.cfg')
self.dnode_conn.exec_cmd(f'echo "fqdn {hostname}" >> /etc/taos/taos.cfg')
def createRemoteDir(self, dir):
'''
if exist: echo 1
else: echo 0
'''
res = bool(int(self.dnode_conn.exec_cmd(f'[ -e {dir} ] && echo 1 || echo 0')))
if not res:
self.dnode_conn.exec_cmd(f'sudo mkdir -p {dir}')
def getHostname(self, ip=None):
if ip == self.dnode_ip:
return self.dnode_conn.exec_cmd('hostname').strip()
else:
return False
def configHostname(self):
logger.info(f'{self.dnode_ip}: config dnode hostname')
ori_hostname = self.dnode_conn.exec_cmd('hostname').strip()
if "localhost" in str(ori_hostname).lower():
self.dnode_conn.exec_cmd(f"sudo hostnamectl set-hostname {self.dnode_name}")
return self.dnode_name
return ori_hostname
def hostsIsExist(self, ip, hostname):
host_count = int(self.dnode_conn.exec_cmd(f'grep "^{ip}.*.{hostname}" /etc/hosts | wc -l'))
if host_count > 0:
logger.info(f'{self.dnode_ip}: check /etc/hosts: {ip} {hostname} existed')
return True
else:
logger.info(f'{self.dnode_ip}: check /etc/hosts: {ip} {hostname} not exist')
return False
def configHosts(self, ip, hostname):
if not self.hostsIsExist(ip, hostname):
logger.info(f'{self.dnode_ip}: config dnode /etc/hosts: {ip} {hostname}')
self.dnode_conn.exec_cmd(f'sudo echo "{ip} {hostname}" >> /etc/hosts')
def checkStatus(self, process):
process_count = self.dnode_conn.exec_cmd(f'ps -ef | grep -w {process} | grep -v grep | wc -l')
if int(process_count.strip()) > 0:
logger.info(f'check {self.dnode_ip} {process} existed')
return True
else:
logger.info(f'check {self.dnode_ip} {process} not exist')
return False
def taoscCreateDnodes(self):
firstEp = f'{self.configHostname()}:6030'
self.dnode_conn.exec_cmd(f'sudo taos -s "create dnode \'{firstEp}\'"')
ready_count = self.dnode_conn.exec_cmd(f'taos -s "show dnodes" | grep {firstEp} | grep ready | wc -l')
ready_flag = 0
if int(ready_count) == 1:
logger.success(f'deploy dnode {firstEp} success')
while int(ready_count) != 1:
if ready_flag < config["timeout"]:
ready_flag += 1
else:
logger.error(f'deploy cluster {firstEp} failed, please check by manual')
time.sleep(1)
ready_count = self.dnode_conn.exec_cmd(f'taos -s "show dnodes" | grep {firstEp} | grep ready | wc -l')
if int(ready_count) == 1:
logger.success(f'deploy dnode {firstEp} success')
def downloadNodeExporter(self):
logger.info(f'{self.dnode_ip}: downloading node_exporter from {config["prometheus"]["node_exporter_addr"]}')
tar_file_name = config["prometheus"]["node_exporter_addr"].split("/")[-1]
if not bool(int(self.dnode_conn.exec_cmd(f'[ -e ~/{tar_file_name} ] && echo 1 || echo 0'))):
self.dnode_conn.exec_cmd(f'wget -P ~ {config["prometheus"]["node_exporter_addr"]}')
def configNodeExporterService(self):
logger.info(f'{self.dnode_ip}: configing /lib/systemd/system/node_exporter.service')
if not bool(int(self.dnode_conn.exec_cmd(f'[ -e /lib/systemd/system/node_exporter.service ] && echo 1 || echo 0'))):
self.dnode_conn.exec_cmd(f'sudo echo -e [Service]\n\
User=prometheus\n\
Group=prometheus\n\
ExecStart=/usr/local/bin/node_exporter\n\
[Install]\n\
WantedBy=multi-user.target\n\
[Unit]\n\
Description=node_exporter\n\
After=network.target \
>> /lib/systemd/system/node_exporter.service')
def killNodeExporter(self):
logger.info(f'{self.dnode_ip}: killing node_exporter')
self.dnode_conn.exec_cmd("ps -ef | grep -w node_exporter | grep -v grep | awk \'{print $2}\' | sudo xargs kill -9")
def deployNodeExporter(self):
logger.info(f'{self.dnode_ip}: deploying node_exporter')
self.killNodeExporter()
self.downloadNodeExporter()
tar_file_name = config["prometheus"]["node_exporter_addr"].split("/")[-1]
tar_file_dir = tar_file_name.replace(".tar.gz", "")
self.dnode_conn.exec_cmd(f'cd ~ && tar -xvf {tar_file_name} && cd {tar_file_dir} && cp -rf node_exporter /usr/local/bin')
self.configNodeExporterService()
self.dnode_conn.exec_cmd('sudo groupadd -r prometheus')
self.dnode_conn.exec_cmd('sudo useradd -r -g prometheus -s /sbin/nologin -M -c "prometheus Daemons" prometheus')
self.dnode_conn.exec_cmd('systemctl start node_exporter && systemctl enable node_exporter && systemctl status node_exporter')
def downloadProcessExporter(self):
tar_file_name = config["prometheus"]["process_exporter_addr"].split("/")[-1]
logger.info(f'{self.dnode_ip}: downloading process_exporter from {config["prometheus"]["process_exporter_addr"]}')
if not bool(int(self.dnode_conn.exec_cmd(f'[ -e ~/{tar_file_name} ] && echo 1 || echo 0'))):
self.dnode_conn.exec_cmd(f'wget -P ~ {config["prometheus"]["process_exporter_addr"]}')
def killProcessExporter(self):
logger.info(f'{self.dnode_ip}: killing process_exporter')
self.dnode_conn.exec_cmd("ps -ef | grep -w process_exporter | grep -v grep | awk \'{print $2}\' | sudo xargs kill -9")
def uploadProcessExporterYml(self, process_list):
logger.info(f'{self.dnode_ip}: generating process_exporter yml')
sub_list = list()
for process in process_list:
sub_list.append({'name':'{{.Comm}}', 'cmdline': [process]})
djson = {'process_names': sub_list}
dstr=json.dumps(djson)
dyml=yaml.load(dstr)
stream = open('process_name.yml', 'w')
yaml.safe_dump(dyml, stream, default_flow_style=False)
self.dnode_conn.upload_file(self.home_dir, 'process_name.yml')
def deployProcessExporter(self, process_list):
logger.info(f'{self.dnode_ip}: deploying process_exporter')
self.killProcessExporter()
self.downloadProcessExporter()
self.uploadProcessExporterYml(process_list)
tar_file_name = config["prometheus"]["process_exporter_addr"].split("/")[-1]
tar_file_dir = tar_file_name.replace(".tar.gz", "")
self.dnode_conn.exec_cmd(f'cd ~ && tar -xvf {tar_file_name} && mv -f ~/process_name.yml ~/{tar_file_dir}')
self.dnode_conn.exec_cmd(f'screen -d -m ~/{tar_file_dir}/process-exporter --config.path ~/{tar_file_dir}/process_name.yml')
def deployTaosd(self, firstEp=None, deploy_type="taosd"):
'''
deploy_type = taosd/taosadapter
'''
self.dnode_conn.upload_file(self.home_dir, self.install_package)
if config["clean_env"]:
self.rmTaosCfg()
self.rmTaosdLog()
self.rmTaosdData()
package_name = self.install_package.split("/")[-1]
package_dir = '-'.join(package_name.split("-", 3)[0:3])
self.stopTaosd()
self.killTaosd()
logger.info(f'{self.dnode_ip}: installing taosd')
logger.info(self.dnode_conn.exec_cmd(f'cd {self.home_dir} && tar -xvf {self.home_dir}/{package_name} && cd {package_dir} && yes|./install.sh'))
self.modifyTaosCfg(firstEp)
if deploy_type == "taosd":
self.startTaosd()
elif deploy_type == "taosadapter":
self.startTaosadapter()
if self.checkStatus(deploy_type):
logger.success(f'{self.dnode_ip}: {deploy_type} deploy success')
else:
logger.error(f'{self.dnode_ip}: {deploy_type} deploy failed, please check by manual')
sys.exit(1)
class Dnodes:
def __init__(self):
self.dnodes = list()
self.ip_list = list()
index = 1
for key in config:
if "taosd_dnode" in str(key):
self.dnodes.append(Dnode(index, config[key]["ip"], config[key]["port"], config[key]["username"], config[key]["password"]))
self.ip_list.append(config[key]["ip"])
index += 1
def installDnodesPackage(self):
for index in range(len(self.dnodes)):
self.dnodes[index].installPackage()
def rmDnodeTaosd(self, index):
self.dnodes[index - 1].rmTaosd()
def rmDnodeTaosdLog(self, index):
self.dnodes[index - 1].rmTaosdLog()
def rmDnodeTaosdData(self, index):
self.dnodes[index - 1].rmTaosdData()
def rmDnodeTaosCfg(self, index):
self.dnodes[index - 1].rmTaosCfg()
def modifyDnodeTaosCfg(self, index, taosCfgKey=None, taosCfgValue=None):
self.dnodes[index - 1].modifyTaosCfg(taosCfgKey, taosCfgValue)
def configDnodesHostname(self):
for index in range(len(self.dnodes)):
self.dnodes[index].configHostname()
def configDnodesHosts(self):
for index in range(len(self.dnodes)):
for ip in self.ip_list:
self.dnodes[index].configHosts(ip)
def startDnodeTaosd(self, index):
self.dnodes[index - 1].startTaosd()
def stopDnodeTaosd(self, index):
self.dnodes[index - 1].stopTaosd()
def killDnodeTaosd(self, index):
self.dnodes[index - 1].killTaosd()
def restartDnodeTaosd(self, index):
self.dnodes[index - 1].restartTaosd()
def startAllTaosd(self):
for index in range(len(self.dnodes)):
self.dnodes[index].startTaosd()
def stopAllTaosd(self):
for index in range(len(self.dnodes)):
self.dnodes[index].stopTaosd()
def killAllTaosd(self):
for index in range(len(self.dnodes)):
self.dnodes[index].stopTaosd()
def restartAllTaosd(self):
for index in range(len(self.dnodes)):
self.dnodes[index].restartTaosd()
def startNodeTaosadapter(self, index):
self.dnodes[index - 1].startTaosadapter()
def stopNodeTaosadapter(self, index):
self.dnodes[index - 1].stopTaosadapter()
def killNodeTaosadapter(self, index):
self.dnodes[index - 1].killTaosadapter()
def restartNodeTaosadapter(self, index):
self.dnodes[index - 1].restartTaosadapter()
def startAllTaosadapters(self):
for index in range(len(self.dnodes)):
self.dnodes[index].startTaosadapter()
def stopAllTaosadapters(self):
for index in range(len(self.dnodes)):
self.dnodes[index].stopTaosadapter()
def killAllTaosadapters(self):
for index in range(len(self.dnodes)):
self.dnodes[index].killTaosadapter()
def restartAllTaosadapters(self):
for index in range(len(self.dnodes)):
self.dnodes[index].restartTaosadapter()
def configDnodesHostname(self):
for index in range(len(self.dnodes)):
self.dnodes[index].configHostname()
def configDnodesHosts(self):
ip_hostname_dict = dict()
for index in range(len(self.dnodes)):
for ip in self.ip_list:
hostname = self.dnodes[index].getHostname(ip)
if hostname is not False:
ip_hostname_dict[ip] = hostname
for index in range(len(self.dnodes)):
for ip, hostname in ip_hostname_dict.items():
self.dnodes[index].configHosts(ip, hostname)
def deployNodes(self):
self.configDnodesHostname()
self.configDnodesHosts()
firstEp = f'{self.dnodes[0].configHostname()}:6030'
if not config["taosadapter_separate_deploy"] and not config["taosd_cluster"]:
self.dnodes[0].deployTaosd()
elif config["taosadapter_separate_deploy"] and not config["taosd_cluster"]:
for index in range(len(self.dnodes)):
if index == 0:
self.dnodes[index].deployTaosd(firstEp, "taosd")
else:
self.dnodes[index].deployTaosd(firstEp, "taosadapter")
else:
for index in range(len(self.dnodes)):
self.dnodes[index].deployTaosd(firstEp)
for index in range(len(self.dnodes)):
if index != 0:
self.dnodes[index].taoscCreateDnodes()
if __name__ == '__main__':
deploy = Dnodes()
deploy.deployNodes()
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/src/common/monitor.py 0 → 100644
浏览文件 @ f336272e
import sys
import json
sys.path.append("../../")
from config.env_init import *
from src.util.RemoteModule import RemoteModule
from src.common.dnodes import Dnodes, Dnode
class Monitor:
def __init__(self):
self.monitor_ip = config["prometheus"]["ip"]
self.monitor_port = config["prometheus"]["port"]
self.monitor_username = config["prometheus"]["username"]
self.monitor_password = config["prometheus"]["password"]
self.monitor_conn = RemoteModule(self.monitor_ip, self.monitor_port, self.monitor_username, self.monitor_password)
self.dnodes = list()
index = 1
for key in config:
if "taosd_dnode" in str(key):
self.dnodes.append(Dnode(index, config[key]["ip"], config[key]["port"], config[key]["username"], config[key]["password"]))
index += 1
if self.monitor_username == "root":
self.home_dir = "/root"
else:
self.home_dir = f"/home/{self.monitor_username}"
def installDnodesPackage(self):
for index in range(len(self.dnodes)):
self.dnodes[index].installPackage()
def deployAllNodeExporters(self):
for index in range(len(self.dnodes)):
self.dnodes[index].deployNodeExporter()
def deployAllProcessExporters(self):
for index in range(len(self.dnodes)):
if index == 0:
self.dnodes[index].deployProcessExporter(['taosd', 'taosadapter'])
else:
if config['taosd_cluster'] and config['taosadapter_separate_deploy']:
self.dnodes[index].deployProcessExporter(['taosd', 'taosadapter'])
elif config['taosd_cluster'] and not config['taosadapter_separate_deploy']:
self.dnodes[index].deployProcessExporter(['taosd'])
elif not config['taosd_cluster'] and config['taosadapter_separate_deploy']:
self.dnodes[index].deployProcessExporter(['taosadapter'])
else:
pass
def downloadPrometheus(self):
logger.info(f'{self.monitor_ip}: downloading prometheus from {config["prometheus"]["prometheus_addr"]}')
tar_file_name = config["prometheus"]["prometheus_addr"].split("/")[-1]
if not bool(int(self.monitor_conn.exec_cmd(f'[ -e ~/{tar_file_name} ] && echo 1 || echo 0'))):
self.monitor_conn.exec_cmd(f'wget -P ~ {config["prometheus"]["prometheus_addr"]}')
def killPrometheus(self):
logger.info(f'{self.monitor_ip}: killing prometheus')
self.monitor_conn.exec_cmd("ps -ef | grep -w prometheus | grep -v grep | awk \'{print $2}\' | sudo xargs kill -9")
def uploadPrometheusYml(self):
logger.info('generating prometheus yml')
scrape_configs = [{'job_name': 'prometheus', 'static_configs': [{'targets': ['localhost:9090']}]}]
for index in range(len(self.dnodes)):
if not config['taosd_cluster'] and not config['taosadapter_separate_deploy']:
pass
else:
scrape_configs.append({'job_name': f'{self.dnodes[index].dnode_ip}_sys', 'static_configs': [{'targets': [f'{self.dnodes[index].dnode_ip}:9100'], 'labels': {'instance': f'{self.dnodes[index].dnode_ip}_sys'}}]})
scrape_configs.append({'job_name': f'{self.dnodes[index].dnode_ip}', 'static_configs': [{'targets': [f'{self.dnodes[index].dnode_ip}:9256'], 'labels': {'instance': f'{self.dnodes[index].dnode_ip}'}}]})
djson = {'global': {'scrape_interval': config["prometheus"]["scrape_interval"], 'evaluation_interval': config["prometheus"]["evaluation_interval"], 'scrape_timeout': config["prometheus"]["scrape_timeout"]}, 'alerting': {'alertmanagers': [{'static_configs': [{'targets': None}]}]}, 'rule_files': None, 'scrape_configs': scrape_configs}
dstr=json.dumps(djson)
dyml=yaml.load(dstr)
stream = open('prometheus.yml', 'w')
yaml.safe_dump(dyml, stream, default_flow_style=False)
self.monitor_conn.upload_file(self.home_dir, 'prometheus.yml')
def deployPrometheus(self):
logger.info(f'{self.monitor_ip}: deploying prometheus')
self.installDnodesPackage()
self.killPrometheus()
self.downloadPrometheus()
self.uploadPrometheusYml()
tar_file_name = config["prometheus"]["prometheus_addr"].split("/")[-1]
tar_file_dir = tar_file_name.replace(".tar.gz", "")
self.monitor_conn.exec_cmd(f'cd ~ && tar -xvf {tar_file_name} && mv ~/prometheus.yml ~/{tar_file_dir}')
self.monitor_conn.exec_cmd(f'screen -d -m ~/{tar_file_dir}/prometheus --config.file={self.home_dir}/{tar_file_dir}/prometheus.yml')
def installGrafana(self):
logger.info(f'{self.monitor_ip}: installing grafana')
if bool(int(self.monitor_conn.exec_cmd(f'cat /etc/os-release | grep ubuntu >> /dev/null && echo 1 || echo 0'))):
if not bool(int(self.monitor_conn.exec_cmd(f'sudo dpkg -s grafana >> /dev/null && echo 1 || echo 0'))):
self.monitor_conn.exec_cmd('sudo apt-get install -y apt-transport-https')
self.monitor_conn.exec_cmd('sudo apt-get install -y software-properties-common wget')
self.monitor_conn.exec_cmd('wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -')
self.monitor_conn.exec_cmd('echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list')
self.monitor_conn.exec_cmd('apt-get update')
self.monitor_conn.exec_cmd('sudo apt-get -y install grafana')
elif bool(int(self.monitor_conn.exec_cmd(f'cat /etc/os-release | grep centos >> /dev/null && echo 1 || echo 0'))):
if not bool(int(self.monitor_conn.exec_cmd(f'sudo rpm -qa | grep grafana >> /dev/null && echo 1 || echo 0'))):
self.monitor_conn.exec_cmd('rm -rf /etc/yum.repos.d/grafana.repo')
self.monitor_conn.exec_cmd('sudo echo -e "[grafana]\nname=grafana\nbaseurl=https://packages.grafana.com/oss/rpm\nrepo_gpgcheck=1\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.grafana.com/gpg.key\nsslverify=1\nsslcacert=/etc/pki/tls/certs/ca-bundle.crt" \
>> /etc/yum.repos.d/grafana.repo')
self.monitor_conn.exec_cmd('yum install -y grafana')
else:
pass
def deployGrafana(self):
self.installGrafana()
self.monitor_conn.exec_cmd('systemctl daemon-reload')
self.monitor_conn.exec_cmd('systemctl start grafana-server')
self.monitor_conn.exec_cmd('systemctl enable grafana-server.service')
self.monitor_conn.exec_cmd('systemctl status grafana-server')
if __name__ == '__main__':
deploy = Dnodes()
deploy.deployNodes()
monitor = Monitor()
monitor.deployAllNodeExporters()
monitor.deployAllProcessExporters()
monitor.deployPrometheus()
monitor.deployGrafana()
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/src/util/RemoteModule.py 0 → 100644
浏览文件 @ f336272e
# -*-coding: utf-8-*-
from fabric import Connection
from config.env_init import *
class RemoteModule():
def __init__(self, ip, port, user, passwd):
self.ip = ip
self.port = port
self.user = user
self.passwd = passwd
def upload_file(self, remote_dir, upload_file):
"""
remote_dir: remote upload dir
upload_file: local file with path
"""
try:
logger.info(f'{self.ip}: uploading {upload_file} to {remote_dir}')
c = Connection(self.ip, user=self.user, port=self.port, connect_timeout=120, connect_kwargs={"password": self.passwd})
c.put(upload_file, remote_dir)
c.close()
except Exception as e:
logger.error(f"{upload_file} send failed----{e}, please check config/perf_test.yaml")
def download_file(self, remote_file_with_path, local_path):
"""
remote_file_with_path:: file with Absolute Path eg:/root/maple/bin/maple
local_path:: remote path eg:/root
"""
try:
c = Connection(self.ip, user=self.user, port=self.port, connect_timeout=120, connect_kwargs={"password": self.passwd})
c.get(remote_file_with_path, local_path)
c.close()
except Exception as e:
logger.error(f"download file {remote_file_with_path} failed：{e}");
def exec_cmd(self, cmd):
"""
cmd:: remote exec cmd
"""
try:
logger.info(f'{self.ip}: executing cmd: {cmd}')
c = Connection(self.ip, user=self.user, port=self.port, connect_timeout=120, connect_kwargs={"password": self.passwd})
result = c.run(cmd, pty=False, warn=True, hide=False)
c.close()
return result.stdout
except Exception as e:
logger.error(f"exec cmd {cmd} failed：{e}");
if __name__ == '__main__':
pass
\ No newline at end of file
tests/perftest-scripts/taosadapter_perftest/src/util/jmeter.py 0 → 100644
浏览文件 @ f336272e
import sys
sys.path.append("../../")
from config.env_init import *
from src.util.RemoteModule import RemoteModule
from src.common.common import Common
class Jmeter:
def __init__(self):
self.Com = Common()
self.jmeter_ip = config["jmeter"]["ip"]
self.jmeter_port = config["jmeter"]["port"]
self.jmeter_username = config["jmeter"]["username"]
self.jmeter_password = config["jmeter"]["password"]
self.jmeter_conn = RemoteModule(self.jmeter_ip, self.jmeter_port, self.jmeter_username, self.jmeter_password)
self.tar_file_name = config["jmeter"]["jmeter_addr"].split("/")[-1]
self.tar_file_dir = self.tar_file_name.replace(".tgz", "")
if self.jmeter_username == "root":
self.home_dir = "/root"
else:
self.home_dir = f"/home/{self.jmeter_username}"
def installPkg(self, pkg_name):
if bool(int(self.jmeter_conn.exec_cmd('cat /etc/os-release | grep ubuntu >> /dev/null && echo 1 || echo 0'))):
if not bool(int(self.jmeter_conn.exec_cmd(f'sudo dpkg -s {pkg_name} >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'sudo apt-get install -y {pkg_name}')
elif bool(int(self.jmeter_conn.exec_cmd(f'cat /etc/os-release | grep centos >> /dev/null && echo 1 || echo 0'))):
if not bool(int(self.jmeter_conn.exec_cmd(f'sudo rpm -qa | grep {pkg_name} >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'sudo yum install -y {pkg_name}')
else:
pass
def installJava(self):
self.installPkg("openjdk-8-jdk")
def downloadJmeter(self):
logger.info(f'{self.jmeter_ip}: downloading jmeter from {config["jmeter"]["jmeter_addr"]}')
if not bool(int(self.jmeter_conn.exec_cmd(f'[ -e ~/{self.tar_file_name} ] && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'wget -P ~ {config["jmeter"]["jmeter_addr"]}')
def deployJmeter(self):
logger.info(f'{self.jmeter_ip}: deploying jmeter')
self.downloadJmeter()
self.installJava()
if not bool(int(self.jmeter_conn.exec_cmd(f'ls ~/{self.tar_file_dir} >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'cd ~ && tar -xvf {self.tar_file_name}')
if not bool(int(self.jmeter_conn.exec_cmd(f'grep "^jmeter.reportgenerator.overall_granularity" ~/{self.tar_file_dir}/bin/user.properties >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'echo "jmeter.reportgenerator.overall_granularity=300000" >> ~/{self.tar_file_dir}/bin/user.properties')
if not bool(int(self.jmeter_conn.exec_cmd(f'ls /usr/local/{self.tar_file_dir} >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'mv ~/{self.tar_file_dir} /usr/local')
if not bool(int(self.jmeter_conn.exec_cmd(f'grep "jmeter" ~/.bashrc >> /dev/null && echo 1 || echo 0'))):
self.jmeter_conn.exec_cmd(f'echo "export PATH=$PATH:/usr/local/{self.tar_file_dir}/bin" >> ~/.bashrc')
# if bool(int(self.jmeter_conn.exec_cmd(f'jmeter -v >> /dev/null && echo 1 || echo 0'))):
# logger.success('deploy jmeter successful')
# else:
# logger.error('deploy jmeter failed')
# sys.exit(1)
return f"/usr/local/{self.tar_file_dir}/bin/jmeter"
if __name__ == '__main__':
deploy = Jmeter()
deploy.deployJmeter()
tests/perftest-scripts/taosadapter_perftest/taosadapter_perftest.py 已删除 100644 → 0
浏览文件 @ 55655b22
from fabric import Connection
from loguru import logger
import shutil
import os
import time
class TaosadapterPerftest():
def __init__(self):
self.ip = "192.168.1.85"
self.port = "22"
self.user = "root"
self.passwd = "tbase125!"
self.telnetCreateStbJmxFile = "opentsdb_telnet_createStb.jmx"
self.telnetCreateTbJmxFile = "opentsdb_telnet_createTb.jmx"
self.telnetInsertRowsFile = "opentsdb_telnet_insertRows.jmx"
# self.telnetMixJmxFile = "opentsdb_telnet_MixTbRows.jmx"
self.telnetMixJmxFile = "opentsdb_telnet_jmeter_csv_import.jmx"
self.jsonCreateStbJmxFile = "opentsdb_json_createStb.jmx"
self.jsonCreateTbJmxFile = "opentsdb_json_createTb.jmx"
self.jsonInsertRowsFile = "opentsdb_json_insertRows.jmx"
# self.jsonMixJmxFile = "opentsdb_json_MixTbRows.jmx"
self.jsonMixJmxFile = "opentsdb_json_jmeter_csv_import.jmx"
self.logfile = "taosadapter_perftest.log"
self.createStbThreads = 100
self.createTbThreads = 100
self.insertRowsThreads = 24
logger.add(self.logfile)
def exec_remote_cmd(self, cmd):
"""
remote exec shell cmd
"""
try:
c = Connection(self.ip, user=self.user, port=self.port, connect_timeout=120, connect_kwargs={"password": self.passwd})
result = c.run(cmd, pty=False, warn=True, hide=True).stdout
c.close()
return result
except Exception as e:
logger.error(f"exec cmd {cmd} failed：{e}");
def exec_local_cmd(self, shell_cmd):
'''
exec local shell cmd
'''
result = os.popen(shell_cmd).read().strip()
return result
def modifyJxmLooptimes(self, filename, looptimes, row_count=None, import_file_name=None):
'''
modify looptimes
'''
with open(filename, "r", encoding="utf-8") as f:
lines = f.readlines()
with open(filename, "w", encoding="utf-8") as f_w:
for line in lines:
if "looptimes" in line:
line = line.replace("looptimes", looptimes)
if row_count is not None:
if "row_count" in line:
line = line.replace("row_count", row_count)
if import_file_name is not None:
if "import_file_name" in line:
line = line.replace("import_file_name", import_file_name)
f_w.write(line)
def cleanAndRestartTaosd(self):
'''
restart taosd and clean env
'''
logger.info("---- restarting taosd and taosadapter ----")
self.exec_remote_cmd("systemctl stop taosd")
self.exec_remote_cmd("rm -rf /var/lib/taos/* /var/log/taos/*")
self.exec_remote_cmd("systemctl start taosd")
logger.info("---- finish restart ----")
time.sleep(60)
def recreateReportDir(self, path):
'''
recreate jmeter report path
'''
if os.path.exists(path):
self.exec_local_cmd(f'rm -rf {path}/*')
else:
os.makedirs(path)
def cleanLog(self):
'''
clean log
'''
with open(self.logfile, 'w') as f:
f.seek(0)
f.truncate()
def genMixTbRows(self, filename, table_count, row_count):
logger.info('generating import data file')
ts_start = 1614530008000
with open(filename, "w", encoding="utf-8") as f_w:
for i in range(table_count):
for j in range(row_count):
input_line = str(ts_start) + "," + str(i) + '\n'
ts_start += 1
f_w.write(input_line)
def outputParams(self, protocol, create_type):
'''
procotol is "telnet" or "json"
create_type is "stb" or "tb" or "rows"
'''
if protocol == "telnet":
if create_type == "stb":
return self.telnetCreateStbJmxFile, self.createStbThreads
elif create_type == "tb":
return self.telnetCreateTbJmxFile, self.createTbThreads
elif create_type == "rows":
return self.telnetInsertRowsFile, self.insertRowsThreads
else:
logger.error("create type error!")
else:
if create_type == "stb":
return self.jsonCreateStbJmxFile, self.createStbThreads
elif create_type == "tb":
return self.jsonCreateTbJmxFile, self.createTbThreads
elif create_type == "rows":
return self.jsonInsertRowsFile, self.insertRowsThreads
else:
logger.error("create type error!")
def insertTDengine(self, procotol, create_type, count):
'''
create stb/tb or insert rows
'''
self.cleanAndRestartTaosd()
jmxfile, threads = self.outputParams(procotol, create_type)
handle_file = str(count) + jmxfile
report_dir = f'testreport/{handle_file}'
self.recreateReportDir(report_dir)
shutil.copyfile(jmxfile, handle_file)
replace_count = int(count/threads)
self.modifyJxmLooptimes(handle_file, str(replace_count))
logger.info(f'jmeter running ----- jmeter -n -t {handle_file} -l {report_dir}/{handle_file}.txt -e -o {report_dir}')
result = self.exec_local_cmd(f"jmeter -n -t {handle_file} -l {report_dir}/{handle_file}.txt -e -o {report_dir}")
logger.info(result)
logger.info("----- sleep 120s and please record data -----")
time.sleep(120)
def insertMixTbRows(self, procotol, table_count, row_count):
self.cleanAndRestartTaosd()
local_path = os.getcwd()
jmxfile = f"opentsdb_{procotol}_{table_count}Tb{row_count}Rows.jmx"
import_file_name = f"import_opentsdb_{procotol}_{table_count}Tb{row_count}Rows.txt"
import_file_path = local_path + '/' + import_file_name
self.genMixTbRows(import_file_name, table_count, row_count)
report_dir = f'testreport/{jmxfile}'
self.recreateReportDir(report_dir)
if procotol == "telnet":
shutil.copyfile(self.telnetMixJmxFile, jmxfile)
else:
shutil.copyfile(self.jsonMixJmxFile, jmxfile)
self.modifyJxmLooptimes(jmxfile, str(int(table_count*row_count/100)), import_file_name=import_file_path)
logger.info(f'jmeter running ----- jmeter -n -t {jmxfile} -l {report_dir}/{jmxfile}.txt -e -o {report_dir}')
result = self.exec_local_cmd(f"jmeter -n -t {jmxfile} -l {report_dir}/{jmxfile}.txt -e -o {report_dir}")
logger.info(result)
logger.info("----- sleep 120s and please record data -----")
time.sleep(120)
# def insertMixTbRows(self, procotol, looptimes, row_count):
# self.cleanAndRestartTaosd()
# jmxfile = f"opentsdb_{procotol}_{looptimes}Tb100Rows.jmx"
# report_dir = f'testreport/{jmxfile}'
# self.recreateReportDir(report_dir)
# if procotol == "telnet":
# shutil.copyfile(self.telnetMixJmxFile, jmxfile)
# else:
# shutil.copyfile(self.jsonMixJmxFile, jmxfile)
# self.modifyJxmLooptimes(jmxfile, str(looptimes), str(row_count))
# result = self.exec_local_cmd(f"jmeter -n -t {jmxfile} -l {report_dir}/{jmxfile}.txt -e -o {report_dir}")
# logger.info(result)
# logger.info("----- sleep 120s and please record data -----")
# time.sleep(120)
if __name__ == '__main__':
taosadapterPerftest = TaosadapterPerftest()
taosadapterPerftest.cleanLog()
logger.info('------------ Start testing the scenarios in the report chapter 3.4.1 ------------')
for procotol in ["telnet", "json"]:
logger.info(f'----- {procotol} protocol ------- Creating 30W stable ------------')
taosadapterPerftest.insertTDengine(procotol, "stb", 300000)
logger.info(f'----- {procotol} protocol ------- Creating 100W table with stb "cpu.usage_user" ------------')
taosadapterPerftest.insertTDengine(procotol, "tb", 1000000)
logger.info(f'----- {procotol} protocol ------- inserting 100W rows ------------')
taosadapterPerftest.insertTDengine(procotol, "rows", 1000000)
logger.info(f'----- {procotol} protocol ------- Creating 50W stable ------------')
taosadapterPerftest.insertTDengine(procotol, "stb", 500000)
logger.info(f'----- {procotol} protocol ------- Creating 500W table with stb "cpu.usage_user" ------------')
taosadapterPerftest.insertTDengine(procotol, "tb", 5000000)
logger.info(f'----- {procotol} protocol ------- inserting 500W rows ------------')
taosadapterPerftest.insertTDengine(procotol, "rows", 5000000)
logger.info(f'----- {procotol} protocol ------- Creating 100W stable ------------')
taosadapterPerftest.insertTDengine(procotol, "stb", 1000000)
logger.info(f'----- {procotol} protocol ------- Creating 1000W table with stb "cpu.usage_user" ------------')
taosadapterPerftest.insertTDengine(procotol, "tb", 10000000)
logger.info(f'----- {procotol} protocol ------- inserting 1000W rows ------------')
taosadapterPerftest.insertTDengine(procotol, "rows", 10000000)
logger.info(f'----- {procotol} protocol ------- Creating 10W stable 1000Rows ------------')
taosadapterPerftest.insertMixTbRows(procotol, 100000, 1000)
logger.info(f'----- {procotol} protocol ------- Creating 100W stable 100Rows ------------')
taosadapterPerftest.insertMixTbRows(procotol, 1000000, 100)
logger.info(f'----- {procotol} protocol ------- Creating 500W stable 20Rows ------------')
taosadapterPerftest.insertMixTbRows(procotol, 5000000, 20)
logger.info(f'----- {procotol} protocol ------- Creating 1000W stable 10Rows ------------')
taosadapterPerftest.insertMixTbRows(procotol, 10000000, 10)
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或