Skip to content

T
TDengine

慢慢CG / TDengine
与 Fork 源项目一致

Fork自 taosdata / TDengine

通知 1
Star 0
Fork 0
T
TDengine
提交 3b334b0a 编写于 作者: H hjxilinx
浏览文件
操作

Merge branch 'develop' into feature/liaohj

上级 55173222 e7bc44ba
隐藏空白更改
内联 并排
Showing with 7098 addition and 20 deletion
README.md
浏览文件 @ 3b334b0a
......@@ -45,10 +45,14 @@ mkdir build && cd build
cmake .. && cmake --build .
```
if compiling on an aarch64 processor, you need add one parameter:
if compiling on an ARM processor(aarch64 or aarch32), you need add one parameter:
```cmd
aarch64:
cmake .. -DCPUTYPE=aarch64 && cmake --build .
aarch32:
cmake .. -DCPUTYPE=aarch32 && cmake --build .
```
# Quick Run
......
minidevops/README.MD 0 → 100644
浏览文件 @ 3b334b0a
# 一分钟快速搭建一个DevOps监控系统
为了让更多的Devops领域的开发者快速体验TDengine的优秀特性，本文介绍了一种快速搭建Devops领域性能监控的demo，方便大家更方便的了解TDengine，并基于此文拓展Devops领域的应用。
为了快速上手，本文用到的软件全部采用Docker容器方式部署，大家只需要安装Docker软件，就可以直接通过脚本运行所有软件，无需安装。这个Demo用到了以下Docker容器，都可以从Dockerhub上拉取相关镜像
- tdengine/tdengine:1.6.4.5 TDengine开源版1.6.4.5.的镜像
- tdengine/blm_telegraf:latest 用于telegraf写入TDengine的API，可以schemaless的将telegraf的数据写入TDengine
- tdengine/blm_prometheus:latest 用于Prometheus写入TDengine的API，可以schemaless的将Prometheus的数据写入TDengine
- grafana/grafana Grafana的镜像，一个广泛应用的开源可视化监控软件
- telegraf:latest 一个广泛应用的开源数据采集程序
- prom/prometheus:latest 一个广泛应用的k8s领域的开源数据采集程序
## 说明
本文中的图片链接在Github上显示不出来，建议将MD文件下载后用vscode或其他md文件浏览工具进行查看
## 前提条件
1. 一台linux服务器或运行linux操作系统的虚拟机或者运行MacOS的计算机
2. 安装了Docker软件。Docker软件的安装方法请参考linux下安装Docker
3. sudo权限
4. 下载本文用到的配置文件和脚本压缩包:[下载地址](http://www.taosdata.com/download/minidevops.tar.gz)
压缩包下载下来后解压生成一个minidevops的文件夹，其结构如下
```sh
minidevops$ tree
.
├── demodashboard.json
├── grafana
│   └── tdengine
│   ├── README.md
│   ├── css
│   │   └── query-editor.css
│   ├── datasource.js
│   ├── img
│   │   └── taosdata_logo.png
│   ├── module.js
│   ├── partials
│   │   ├── config.html
│   │   └── query.editor.html
│   ├── plugin.json
│   └── query_ctrl.js
├── prometheus
│   └── prometheus.yml
├── run.sh
└── telegraf
└── telegraf.conf
```
`grafana`子文件夹里是TDengine的插件，用于在grafana中导入TDengine的数据源。
`prometheus`子文件夹里是prometheus需要的配置文件。
`run.sh`是运行脚本。
`telegraf`子文件夹里是telegraf的配置文件。
## 启动Docker镜像
启动前，请确保系统里没有运行TDengine和Grafana，以及Telegraf和Prometheus，因为这些程序会占用docker所需的端口，造成脚本运行失败，建议先关闭这些程序。
然后，只用在minidevops路径下执行
```sh
sudo run.sh
```
我们来看看`run.sh`里干了些什么：
```sh
#!/bin/bash
LP=`pwd`
#为了让脚本能够顺利执行，避免重复执行时出现错误， 首先将系统里所有docker容器停止了。请注意，如果该linux上已经运行了其他docker容器，也会被停止掉。
docker rm -f `docker ps -a -q`
#专门创建一个叫minidevops的虚拟网络，并指定了172.15.1.1~255这个地址段。
docker network create --ip-range 172.15.1.255/24 --subnet 172.15.1.1/16 minidevops
#启动grafana程序，并将tdengine插件文件所在路径绑定到容器中
docker run -d --net minidevops --ip 172.15.1.11 -v $LP/grafana:/var/lib/grafana/plugins -p 3000:3000 grafana/grafana
#启动tdengine的docker容器，并指定IP地址为172.15.1.6，绑定需要的端口
docker run -d --net minidevops --ip 172.15.1.6 -p 6030:6030 -p 6020:6020 -p 6031:6031 -p 6032:6032 -p 6033:6033 -p 6034:6034 -p 6035:6035 -p 6036:6036 -p 6037:6037 -p 6038:6038 -p 6039:6039 tdengine/tdengine:1.6.4.5
#启动prometheus的写入代理程序，这个程序可以将prometheus发来的数据直接写入TDengine中，无需提前建立相关超级表和表，实现schemaless写入功能
docker run -d --net minidevops --ip 172.15.1.7 -p 10203:10203 tdengine/blm_prometheus 172.15.1.6
#启动telegraf的写入代理程序，这个程序可以将telegraf发来的数据直接写入TDengine中，无需提前建立相关超级表和表，实现schemaless写入功能
docker run -d --net minidevops --ip 172.15.1.8 -p 10202:10202 tdengine/blm_telegraf 172.15.1.6
#启动prometheus程序，并将配置文件所在路径绑定到容器中
docker run -d --net minidevops --ip 172.15.1.9 -v $LP/prometheus:/etc/prometheus -p 9090:9090 prom/prometheus
#启动telegraf程序，并将配置文件所在路径绑定到容器中
docker run -d --net minidevops --ip 172.15.1.10 -v $LP/telegraf:/etc/telegraf -p 8092:8092 -p 8094:8094 -p 8125:8125 telegraf
#通过Grafana的API，将TDengine配置成Grafana的datasources
curl -X POST http://localhost:3000/api/datasources --header "Content-Type:application/json" -u admin:admin -d '{"Name": "TDengine","Type": "tdengine","TypeLogoUrl": "public/plugins/tdengine/img/taosdata_logo.png","Access": "proxy","Url": "http://172.15.1.6:6020","BasicAuth": false,"isDefault": true,"jsonData": {},"readOnly": false}'
#通过Grafana的API，配置一个示范的监控面板
curl -X POST http://localhost:3000/api/dashboards/db --header "Content-Type:application/json" -u admin:admin -d '{"dashboard":{"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":true,"gnetId":null,"graphTooltip":0,"id":1,"links":[],"panels":[{"datasource":null,"gridPos":{"h":8,"w":6,"x":0,"y":0},"id":6,"options":{"fieldOptions":{"calcs":["mean"],"defaults":{"color":{"mode":"thresholds"},"links":[{"title":"","url":""}],"mappings":[],"max":100,"min":0,"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null},{"color":"red","value":80}]},"unit":"percent"},"overrides":[],"values":false},"orientation":"auto","showThresholdLabels":false,"showThresholdMarkers":true},"pluginVersion":"6.6.0","targets":[{"refId":"A","sql":"select last_row(value) from telegraf.mem where field=\"used_percent\""}],"timeFrom":null,"timeShift":null,"title":"Memory used percent","type":"gauge"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":null,"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":6,"y":0},"hiddenSeries":false,"id":8,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"MEMUSED-PERCENT","refId":"A","sql":"select avg(value) from telegraf.mem where field=\"used_percent\" interval(1m)"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"Panel Title","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"datasource":null,"gridPos":{"h":9,"w":6,"x":0,"y":8},"id":10,"options":{"fieldOptions":{"calcs":["mean"],"defaults":{"mappings":[],"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null}]},"unit":"percent"},"overrides":[],"values":false},"orientation":"auto","showThresholdLabels":false,"showThresholdMarkers":true},"pluginVersion":"6.6.0","targets":[{"alias":"CPU-SYS","refId":"A","sql":"select last_row(value) from telegraf.cpu where field=\"usage_system\""},{"alias":"CPU-IDLE","refId":"B","sql":"select last_row(value) from telegraf.cpu where field=\"usage_idle\""},{"alias":"CPU-USER","refId":"C","sql":"select last_row(value) from telegraf.cpu where field=\"usage_user\""}],"timeFrom":null,"timeShift":null,"title":"Panel Title","type":"gauge"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"TDengine","description":"General CPU monitor","fill":1,"fillGradient":0,"gridPos":{"h":9,"w":12,"x":6,"y":8},"hiddenSeries":false,"id":2,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"CPU-USER","refId":"A","sql":"select avg(value) from telegraf.cpu where field=\"usage_user\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-SYS","refId":"B","sql":"select avg(value) from telegraf.cpu where field=\"usage_system\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-IDLE","refId":"C","sql":"select avg(value) from telegraf.cpu where field=\"usage_idle\" and cpu=\"cpu-total\" interval(1m)"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"CPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"10s","schemaVersion":22,"style":"dark","tags":["demo"],"templating":{"list":[]},"time":{"from":"now-3h","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"]},"timezone":"","title":"TDengineDashboardDemo","id":null,"uid":null,"version":0}}'
```
执行以上脚本后，可以通过docker container ls命令来确认容器运行的状态：
```sh
$docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
f875bd7d90d1 telegraf "/entrypoint.sh tele…" 6 hours ago Up 6 hours 0.0.0.0:8092->8092/tcp, 8092/udp, 0.0.0.0:8094->8094/tcp, 8125/udp, 0.0.0.0:8125->8125/tcp wonderful_antonelli
38ee2d5c3cb3 prom/prometheus "/bin/prometheus --c…" 6 hours ago Up 6 hours 0.0.0.0:9090->9090/tcp infallible_mestorf
1a1939386c07 tdengine/blm_telegraf "/root/blm_telegraf …" 6 hours ago Up 6 hours 0.0.0.0:10202->10202/tcp stupefied_hypatia
7063eb05caa4 tdengine/blm_prometheus "/root/blm_prometheu…" 6 hours ago Up 6 hours 0.0.0.0:10203->10203/tcp jovial_feynman
4a7b27931d21 tdengine/tdengine:1.6.4.5 "taosd" 6 hours ago Up 6 hours 0.0.0.0:6020->6020/tcp, 0.0.0.0:6030-6039->6030-6039/tcp, 6040-6050/tcp eager_kowalevski
ad2895760bc0 grafana/grafana "/run.sh" 6 hours ago Up 6 hours 0.0.0.0:3000->3000/tcp romantic_mccarthy
```
当以上几个容器都已正常运行后，则我们的demo小系统已经开始工作了。
## Grafana中进行配置
打开浏览器，在地址栏输入服务器所在的IP地址
`http://localhost:3000`
就可以访问到grafana的页面，如果不在本机打开浏览器，则将localhost改成server的ip地址即可。
进入登录页面，用户名和密码都是缺省的admin，输入后，即可进入grafana的控制台输入用户名/密码后，会进入修改密码页面，选择skip，跳过这一步。进入Grafana后，可以在页面的左下角看到TDengineDashboardDemo已经创建好了，![](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-2-1_22-50-58-1024x465.png)对于有些浏览器打开时，可能会在home页面中没有TDengineDashboardDemo的选项，可以通过在Dashboard->Manage中选择![](https://www.taosdata.com/blog/wp-content/uploads/2020/02/2-1024x553.png)TDengineDashboardDemo。点击TDengineDashboardDemo进入示例监控面板。刚点进去页面时，监控曲线是空白的，因为监控数据还不够多，需要等待一段时间，让数据采集程序采集更多的数据。![](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image-5-1024x853.png)
如上两个监控面板分别监控了CPU和内存占用率。点击面板上的标题可以选择Edit进入编辑界面，新增监控数据。关于Grafana的监控面板设置，可以详细参考Grafana官网文档[Getting Started](https://grafana.com/docs/grafana/latest/guides/getting_started/)。
## 原理介绍
按上面的操作，我们已经将监控系统搭建起来了，目前可以监控系统的CPU占有率了。下面介绍下这个Demo系统的工作原理。
如下图所示，这个系统由数据采集功能（prometheus，telegraf），时序数据库功能（TDengine和适配程序），可视化功能(Grafana)组成。下面虚线框里的TDengine，blm_prometheus, blm_telegraf三个容器组成了一个schemaless写入的时序数据库，对于采用telegraf和prometheus作为采集程序的监控对象，可以直接将数据写入TDengine，并通过grafana进行可视化呈现。
![architecture](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-1-29_21-22-6.png)
### 数据采集
数据采集由Telegraf和Prometheus完成。Telegraf根据配置，从操作系统层面采集系统的相关统计值，并按配置上报给指定的URL，上报的数据json格式为
```json
{
"fields":{
"usage_guest":0,
"usage_guest_nice":0,
"usage_idle":87.73726273726274,
"usage_iowait":0,
"usage_irq":0,
"usage_nice":0,
"usage_softirq":0,
"usage_steal":0,
"usage_system":2.6973026973026974,
"usage_user":9.565434565434565
},
"name":"cpu",
"tags":{
"cpu":"cpu-total",
"host":"liutaodeMacBook-Pro.local"
},
"timestamp":1571665100
}
```
其中name将被作为超级表的表名，tags作为普通表的tags，fields的名称也会作为一个tag用来描述普通表的标签。举个例子，一个普通表的结构如下，这是一个存储usage_softirq数据的普通表。
![表结构](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-1-29_21-38-24.png)
### Telegraf的配置
对于使用telegraf作为数据采集程序的监控对象，可以在telegraf的配置文件telegraf.conf中将outputs.http部分的配置按以下配置修改，就可以直接将数据写入TDengine中了
```toml
[[outputs.http]]
# ## URL is the address to send metrics to
url = "http://172.15.1.8:10202/telegraf"
#
# ## HTTP Basic Auth credentials
# # username = "username"
# # password = "pa$$word"
#
data_format = "json"
json_timestamp_units = "1ms"
```
可以打开HTTP basic Auth验证机制，本Demo为了简化没有打开验证功能。
对于多个被监控对象，只需要在telegraf.conf文件中都写上以上的配置内容，就可以将数据写入TDengine中了。
### Telegraf数据在TDengine中的存储结构
Telegraf的数据在TDengine中的存储，是以数据name为超级表名，以tags值加上监控对象的ip地址，以及field的属性名作为tag值，存入TDengine中的。
以name为cpu的数据为例，telegraf产生的数据为:
```json
{
"fields":{
"usage_guest":0,
"usage_guest_nice":0,
"usage_idle":87.73726273726274,
"usage_iowait":0,
"usage_irq":0,
"usage_nice":0,
"usage_softirq":0,
"usage_steal":0,
"usage_system":2.6973026973026974,
"usage_user":9.565434565434565
},
"name":"cpu",
"tags":{
"cpu":"cpu-total",
"host":"liutaodeMacBook-Pro.local"
},
"timestamp":1571665100
}
```
则写入TDengine时会自动存入一个名为cpu的超级表中，这个表的结构如下
![telegraf表结构](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-2-2_0-37-49.png)
这个超级表的tag字段有cpu,host,srcip,field；其中cpu，host是原始数据携带的tag，而srcip是监控对象的IP地址，field是监控对象cpu类型数据中的fields属性，取值空间为[usage_guest,usage_guest_nice,usage_idle,usage_iowait,usage_irq,usage_nice,usage_softirq,usage_steal,usage_system,usage_user]，每个field值对应着一个具体含义的数据。
因此，在查询的时候，可以用这些tag来过滤数据，也可以用超级表来聚合数据。
### Prometheus的配置
对于使用Prometheus作为数据采集程序的监控对象，可以在Prometheus的配置文件prometheus.yaml文件中，将remote write部分的配置按以下配置修改，就可以直接将数据写入TDengine中了。
```yaml
remote_write:
- url: "http://172.15.1.7:10203/receive"
```
对于多个被监控对象，只需要在每个被监控对象的prometheus配置中增加以上配置内容，就可以将数据写入TDengine中了。
### Prometheus数据在TDengine中的存储结构
Prometheus的数据在TDengine中的存储，与telegraf类似，也是以数据的name字段为超级表名，以数据的label作为tag值，存入TDengine中
以prometheus_engine_queries这个数据为例[prom表结构](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-2-2_0-51-4.png)
在TDengine中会自动创建一个prometheus_engine_queries的超级表，tag字段为t_instance，t_job，t_monitor。
查询时，可以用这些tag来过滤数据，也可以用超级表来聚合数据。
## 数据查询
我们可以登陆到TDengine的客户端命令，通过命令行看看TDengine里面都存储了些什么数据，顺便也能体验一下TDengine的高性能查询。如何才能登陆到TDengine的客户端，我们可以通过以下几步来完成。
首先通过下面的命令查询一下tdengine的Docker ID
```sh
docker container ls
```
然后再执行
```sh
docker exec -it tdengine的containerID bash
```
就可以进入TDengine容器的命令行，执行taos，就进入以下界面![](https://www.taosdata.com/blog/wp-content/uploads/2020/02/image2020-1-29_21-55-53.png)
Telegraf的数据写入时，自动创建了一个名为telegraf的database，可以通过
```
use telegraf；
```
使用telegraf这个数据库。然后执行show tables，describe table等命令详细查询下telegraf这个库里保存了些什么数据。
具体TDengine的查询语句可以参考[TDengine官方文档](https://www.taosdata.com/cn/documentation/taos-sql/)
## 接入多个监控对象
就像前面原理介绍的，这个miniDevops的小系统，已经提供了一个时序数据库和可视化系统，对于多台机器的监控，只需要将每台机器的telegraf或prometheus配置按上面所述修改，就可以完成监控数据采集和可视化呈现了。
minidevops/demodashboard.json 0 → 100644
浏览文件 @ 3b334b0a
{
"annotations": {
"list": [
{
"builtIn": 1,
"datasource": "-- Grafana --",
"enable": true,
"hide": true,
"iconColor": "rgba(0, 211, 255, 1)",
"name": "Annotations & Alerts",
"type": "dashboard"
}
]
},
"editable": true,
"gnetId": null,
"graphTooltip": 0,
"id": 1,
"links": [],
"panels": [
{
"datasource": null,
"gridPos": {
"h": 8,
"w": 6,
"x": 0,
"y": 0
},
"id": 6,
"options": {
"fieldOptions": {
"calcs": [
"mean"
],
"defaults": {
"color": {
"mode": "thresholds"
},
"links": [
{
"title": "",
"url": ""
}
],
"mappings": [],
"max": 100,
"min": 0,
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
},
{
"color": "red",
"value": 80
}
]
},
"unit": "percent"
},
"overrides": [],
"values": false
},
"orientation": "auto",
"showThresholdLabels": false,
"showThresholdMarkers": true
},
"pluginVersion": "6.6.0",
"targets": [
{
"refId": "A",
"sql": "select last_row(value) from telegraf.mem where field=\"used_percent\""
}
],
"timeFrom": null,
"timeShift": null,
"title": "Memory used percent",
"type": "gauge"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": null,
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 8,
"w": 12,
"x": 6,
"y": 0
},
"hiddenSeries": false,
"id": 8,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"dataLinks": []
},
"percentage": false,
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"alias": "MEMUSED-PERCENT",
"refId": "A",
"sql": "select avg(value) from telegraf.mem where field=\"used_percent\" interval(1m)"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "MEM",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
},
{
"datasource": null,
"gridPos": {
"h": 3,
"w": 18,
"x": 0,
"y": 8
},
"id": 10,
"options": {
"displayMode": "lcd",
"fieldOptions": {
"calcs": [
"mean"
],
"defaults": {
"mappings": [],
"thresholds": {
"mode": "absolute",
"steps": [
{
"color": "green",
"value": null
}
]
},
"unit": "percent"
},
"overrides": [],
"values": false
},
"orientation": "auto",
"showUnfilled": true
},
"pluginVersion": "6.6.0",
"targets": [
{
"alias": "CPU-SYS",
"refId": "A",
"sql": "select last_row(value) from telegraf.cpu where field=\"usage_system\""
},
{
"alias": "CPU-IDLE",
"refId": "B",
"sql": "select last_row(value) from telegraf.cpu where field=\"usage_idle\""
},
{
"alias": "CPU-USER",
"refId": "C",
"sql": "select last_row(value) from telegraf.cpu where field=\"usage_user\""
}
],
"timeFrom": null,
"timeShift": null,
"title": "CPU-USED",
"type": "bargauge"
},
{
"aliasColors": {},
"bars": false,
"dashLength": 10,
"dashes": false,
"datasource": "TDengine",
"description": "General CPU monitor",
"fill": 1,
"fillGradient": 0,
"gridPos": {
"h": 9,
"w": 18,
"x": 0,
"y": 11
},
"hiddenSeries": false,
"id": 2,
"legend": {
"avg": false,
"current": false,
"max": false,
"min": false,
"show": true,
"total": false,
"values": false
},
"lines": true,
"linewidth": 1,
"nullPointMode": "null",
"options": {
"dataLinks": []
},
"percentage": false,
"pointradius": 2,
"points": false,
"renderer": "flot",
"seriesOverrides": [],
"spaceLength": 10,
"stack": false,
"steppedLine": false,
"targets": [
{
"alias": "CPU-USER",
"refId": "A",
"sql": "select avg(value) from telegraf.cpu where field=\"usage_user\" and cpu=\"cpu-total\" interval(1m)"
},
{
"alias": "CPU-SYS",
"refId": "B",
"sql": "select avg(value) from telegraf.cpu where field=\"usage_system\" and cpu=\"cpu-total\" interval(1m)"
},
{
"alias": "CPU-IDLE",
"refId": "C",
"sql": "select avg(value) from telegraf.cpu where field=\"usage_idle\" and cpu=\"cpu-total\" interval(1m)"
}
],
"thresholds": [],
"timeFrom": null,
"timeRegions": [],
"timeShift": null,
"title": "CPU",
"tooltip": {
"shared": true,
"sort": 0,
"value_type": "individual"
},
"type": "graph",
"xaxis": {
"buckets": null,
"mode": "time",
"name": null,
"show": true,
"values": []
},
"yaxes": [
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
},
{
"format": "short",
"label": null,
"logBase": 1,
"max": null,
"min": null,
"show": true
}
],
"yaxis": {
"align": false,
"alignLevel": null
}
}
],
"refresh": "10s",
"schemaVersion": 22,
"style": "dark",
"tags": [
"demo"
],
"templating": {
"list": []
},
"time": {
"from": "now-3h",
"to": "now"
},
"timepicker": {
"refresh_intervals": [
"5s",
"10s",
"30s",
"1m",
"5m",
"15m",
"30m",
"1h",
"2h",
"1d"
]
},
"timezone": "",
"title": "TDengineDashboardDemo",
"uid": "2lF1wNUWz",
"version": 4
}
\ No newline at end of file
minidevops/grafana/tdengine/README.md 0 → 100644
浏览文件 @ 3b334b0a
TDengine Datasource - build by Taosdata Inc. www.taosdata.com
TDengine backend server implement 2 urls:
* `/heartbeat` return 200 ok. Used for "Test connection" on the datasource config page.
* `/query` return data based on input sqls.
## Installation
To install this plugin:
Copy the data source you want to /var/lib/grafana/plugins/. Then restart grafana-server. The new data source should now be available in the data source type dropdown in the Add Data Source View.
```
cp -r <tdengine-extrach-dir>/connector/grafana/tdengine /var/lib/grafana/plugins/
sudo service grafana-server restart
```
### Query API
Example request
``` javascript
[{
"refId": "A",
"alias": "taosd-memory",
"sql": "select avg(mem_taosd) from sys.dn where ts > now-5m and ts < now interval(500a)"
},
{
"refId": "B",
"alias": "system-memory",
"sql": "select avg(mem_system) from sys.dn where ts > now-5m and ts < now interval(500a)"
}]
```
Example response
``` javascript
[{
"datapoints": [
[206.488281, 1538137825000],
[206.488281, 1538137855000],
[206.488281, 1538137885500],
[210.609375, 1538137915500],
[210.867188, 1538137945500]
],
"refId": "A",
"target": "taosd-memory"
},
{
"datapoints": [
[2910.218750, 1538137825000],
[2912.265625, 1538137855000],
[2912.437500, 1538137885500],
[2916.644531, 1538137915500],
[2917.066406, 1538137945500]
],
"refId": "B",
"target": "system-memory"
}]
```
### Heartbeat API
Example request
``` javascript
<null> get request
```
Example response
``` javascript
{
"message": "Grafana server receive a quest from you!"
}
```
minidevops/grafana/tdengine/css/query-editor.css 0 → 100644
浏览文件 @ 3b334b0a
.generic-datasource-query-row .query-keyword {
width: 75px;
}
\ No newline at end of file
minidevops/grafana/tdengine/datasource.js 0 → 100644
浏览文件 @ 3b334b0a
'use strict';
System.register(['lodash'], function (_export, _context) {
"use strict";
var _, _createClass, GenericDatasource;
function strTrim(str) {
return str.replace(/^\s+|\s+$/gm,'');
}
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_lodash) {
_ = _lodash.default;
}],
execute: function () {
_createClass = function () {
function defineProperties(target, props) {
for (var i = 0; i < props.length; i++) {
var descriptor = props[i];
descriptor.enumerable = descriptor.enumerable || false;
descriptor.configurable = true;
if ("value" in descriptor) descriptor.writable = true;
Object.defineProperty(target, descriptor.key, descriptor);
}
}
return function (Constructor, protoProps, staticProps) {
if (protoProps) defineProperties(Constructor.prototype, protoProps);
if (staticProps) defineProperties(Constructor, staticProps);
return Constructor;
};
}();
_export('GenericDatasource', GenericDatasource = function () {
function GenericDatasource(instanceSettings, $q, backendSrv, templateSrv) {
_classCallCheck(this, GenericDatasource);
this.type = instanceSettings.type;
this.url = instanceSettings.url;
this.name = instanceSettings.name;
this.q = $q;
this.backendSrv = backendSrv;
this.templateSrv = templateSrv;
//this.withCredentials = instanceSettings.withCredentials;
this.headers = { 'Content-Type': 'application/json' };
var taosuser = instanceSettings.jsonData.user;
var taospwd = instanceSettings.jsonData.password;
if (taosuser == null || taosuser == undefined || taosuser == "") {
taosuser = "root";
}
if (taospwd == null || taospwd == undefined || taospwd == "") {
taospwd = "taosdata";
}
this.headers.Authorization = "Basic " + this.encode(taosuser + ":" + taospwd);
}
_createClass(GenericDatasource, [{
key: 'encode',
value: function encode(input) {
var _keyStr = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=";
var output = "";
var chr1, chr2, chr3, enc1, enc2, enc3, enc4;
var i = 0;
while (i < input.length) {
chr1 = input.charCodeAt(i++);
chr2 = input.charCodeAt(i++);
chr3 = input.charCodeAt(i++);
enc1 = chr1 >> 2;
enc2 = ((chr1 & 3) << 4) | (chr2 >> 4);
enc3 = ((chr2 & 15) << 2) | (chr3 >> 6);
enc4 = chr3 & 63;
if (isNaN(chr2)) {
enc3 = enc4 = 64;
} else if (isNaN(chr3)) {
enc4 = 64;
}
output = output + _keyStr.charAt(enc1) + _keyStr.charAt(enc2) + _keyStr.charAt(enc3) + _keyStr.charAt(enc4);
}
return output;
}
}, {
key: 'generateSql',
value: function generateSql(sql, queryStart, queryEnd, intervalMs) {
if (queryStart == undefined || queryStart == null) {
queryStart = "now-1h";
}
if (queryEnd == undefined || queryEnd == null) {
queryEnd = "now";
}
if (intervalMs == undefined || intervalMs == null) {
intervalMs = "20000";
}
intervalMs += "a";
sql = sql.replace(/^\s+|\s+$/gm, '');
sql = sql.replace("$from", "'" + queryStart + "'");
sql = sql.replace("$begin", "'" + queryStart + "'");
sql = sql.replace("$to", "'" + queryEnd + "'");
sql = sql.replace("$end", "'" + queryEnd + "'");
sql = sql.replace("$interval", intervalMs);
return sql;
}
}, {
key: 'query',
value: function query(options) {
var querys = new Array;
for (var i = 0; i < options.targets.length; ++i) {
var query = new Object;
query.refId = options.targets[i].refId;
query.alias = options.targets[i].alias;
if (query.alias == null || query.alias == undefined) {
query.alias = "";
}
//query.sql = this.generateSql(options.targets[i].sql, options.range.raw.from, options.range.raw.to, options.intervalMs);
query.sql = this.generateSql(options.targets[i].sql, options.range.from.toISOString(), options.range.to.toISOString(), options.intervalMs);
console.log(query.sql);
querys.push(query);
}
if (querys.length <= 0) {
return this.q.when({ data: [] });
}
return this.doRequest({
url: this.url + '/grafana/query',
data: querys,
method: 'POST'
});
}
}, {
key: 'testDatasource',
value: function testDatasource() {
return this.doRequest({
url: this.url + '/grafana/heartbeat',
method: 'GET'
}).then(function (response) {
if (response.status === 200) {
return { status: "success", message: "TDengine Data source is working", title: "Success" };
}
});
}
}, {
key: 'doRequest',
value: function doRequest(options) {
options.headers = this.headers;
//console.log(options);
return this.backendSrv.datasourceRequest(options);
}
}]);
return GenericDatasource;
}());
_export('GenericDatasource', GenericDatasource);
}
};
});
//# sourceMappingURL=datasource.js.map
minidevops/grafana/tdengine/module.js 0 → 100644
浏览文件 @ 3b334b0a
'use strict';
System.register(['./datasource', './query_ctrl'], function (_export, _context) {
"use strict";
var GenericDatasource, GenericDatasourceQueryCtrl, GenericConfigCtrl, GenericQueryOptionsCtrl, GenericAnnotationsQueryCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
return {
setters: [function (_datasource) {
GenericDatasource = _datasource.GenericDatasource;
}, function (_query_ctrl) {
GenericDatasourceQueryCtrl = _query_ctrl.GenericDatasourceQueryCtrl;
}],
execute: function () {
_export('ConfigCtrl', GenericConfigCtrl = function GenericConfigCtrl() {
_classCallCheck(this, GenericConfigCtrl);
});
GenericConfigCtrl.templateUrl = 'partials/config.html';
_export('QueryOptionsCtrl', GenericQueryOptionsCtrl = function GenericQueryOptionsCtrl() {
_classCallCheck(this, GenericQueryOptionsCtrl);
});
GenericQueryOptionsCtrl.templateUrl = 'partials/query.options.html';
_export('AnnotationsQueryCtrl', GenericAnnotationsQueryCtrl = function GenericAnnotationsQueryCtrl() {
_classCallCheck(this, GenericAnnotationsQueryCtrl);
});
GenericAnnotationsQueryCtrl.templateUrl = 'partials/annotations.editor.html';
_export('Datasource', GenericDatasource);
_export('QueryCtrl', GenericDatasourceQueryCtrl);
_export('ConfigCtrl', GenericConfigCtrl);
_export('QueryOptionsCtrl', GenericQueryOptionsCtrl);
_export('AnnotationsQueryCtrl', GenericAnnotationsQueryCtrl);
}
};
});
//# sourceMappingURL=module.js.map
minidevops/grafana/tdengine/partials/config.html 0 → 100644
浏览文件 @ 3b334b0a
<h3 class="page-heading">TDengine Connection</h3>
<div class="gf-form-group">
<div class="gf-form max-width-30">
<span class="gf-form-label width-7">Host</span>
<input type="text" class="gf-form-input" ng-model='ctrl.current.url' placeholder="http://localhost:6020" bs-typeahead="{{['http://localhost:6020']}}" required></input>
</div>
<div class="gf-form-inline">
<div class="gf-form max-width-15">
<span class="gf-form-label width-7">User</span>
<input type="text" class="gf-form-input" ng-model='ctrl.current.jsonData.user' placeholder="root"></input>
</div>
<div class="gf-form max-width-15">
<span class="gf-form-label width-7">Password</span>
<input type="password" class="gf-form-input" ng-model='ctrl.current.jsonData.password' placeholder="taosdata"></input>
</div>
</div>
</div>
\ No newline at end of file
minidevops/grafana/tdengine/partials/query.editor.html 0 → 100644
浏览文件 @ 3b334b0a
<query-editor-row query-ctrl="ctrl" can-collapse="true" >
<div class="gf-form-inline">
<div class="gf-form gf-form--grow">
<label class="gf-form-label query-keyword width-7">INPUT SQL</label>
<input type="text" class="gf-form-input" ng-model="ctrl.target.sql" spellcheck='false' placeholder="select count(*) from sys.cpu where ts >= $from and ts < $to interval($interval)" ng-blur="ctrl.panelCtrl.refresh()" data-mode="sql"></input>
</div>
</div>
<div class="gf-form-inline">
<div class="gf-form-inline" ng-hide="ctrl.target.resultFormat === 'table'">
<div class="gf-form max-width-30">
<label class="gf-form-label query-keyword width-7">ALIAS BY</label>
<input type="text" class="gf-form-input" ng-model="ctrl.target.alias" spellcheck='false' placeholder="Naming pattern" ng-blur="ctrl.panelCtrl.refresh()">
</div>
</div>
<div class="gf-form">
<label class="gf-form-label query-keyword" ng-click="ctrl.generateSQL()">
GENERATE SQL
<i class="fa fa-caret-down" ng-show="ctrl.showGenerateSQL"></i>
<i class="fa fa-caret-right" ng-hide="ctrl.showGenerateSQL"></i>
</label>
</div>
<div class="gf-form">
<label class="gf-form-label query-keyword" ng-click="ctrl.showHelp = !ctrl.showHelp">
SHOW HELP
<i class="fa fa-caret-down" ng-show="ctrl.showHelp"></i>
<i class="fa fa-caret-right" ng-hide="ctrl.showHelp"></i>
</label>
</div>
</div>
<div class="gf-form" ng-show="ctrl.showGenerateSQL">
<pre class="gf-form-pre">{{ctrl.lastGenerateSQL}}</pre>
</div>
<div class="gf-form" ng-show="ctrl.showHelp">
<pre class="gf-form-pre alert alert-info">Use any SQL that can return Resultset such as:
- [[timestamp1, value1], [timestamp2, value2], ... ]
Macros:
- $from -&gt; start timestamp of panel
- $to -&gt; stop timestamp of panel
- $interval -&gt; interval of panel
Example of SQL:
&nbsp;&nbsp;SELECT count(*)
&nbsp;&nbsp;FROM db.table
&nbsp;&nbsp;WHERE ts > $from and ts < $to
&nbsp;&nbsp;INTERVAL ($interval)
</pre>
</div>
<div class="gf-form" ng-show="ctrl.lastQueryError">
<pre class="gf-form-pre alert alert-error">{{ctrl.lastQueryError}}</pre>
</div>
</query-editor-row>
minidevops/grafana/tdengine/plugin.json 0 → 100644
浏览文件 @ 3b334b0a
{
"name": "TDengine",
"id": "tdengine",
"type": "datasource",
"partials": {
"config": "partials/config.html"
},
"metrics": true,
"annotations": false,
"alerting": true,
"info": {
"description": "TDengine datasource",
"author": {
"name": "Taosdata Inc.",
"url": "https://www.taosdata.com"
},
"logos": {
"small": "img/taosdata_logo.png",
"large": "img/taosdata_logo.png"
},
"version": "1.6.0",
"updated": "2019-07-01"
},
"dependencies": {
"grafanaVersion": "5.2.4",
"plugins": [ ]
}
}
minidevops/grafana/tdengine/query_ctrl.js 0 → 100644
浏览文件 @ 3b334b0a
'use strict';
System.register(['app/plugins/sdk'], function (_export, _context) {
"use strict";
var QueryCtrl, _createClass, GenericDatasourceQueryCtrl;
function _classCallCheck(instance, Constructor) {
if (!(instance instanceof Constructor)) {
throw new TypeError("Cannot call a class as a function");
}
}
function _possibleConstructorReturn(self, call) {
if (!self) {
throw new ReferenceError("this hasn't been initialised - super() hasn't been called");
}
return call && (typeof call === "object" || typeof call === "function") ? call : self;
}
function _inherits(subClass, superClass) {
if (typeof superClass !== "function" && superClass !== null) {
throw new TypeError("Super expression must either be null or a function, not " + typeof superClass);
}
subClass.prototype = Object.create(superClass && superClass.prototype, {
constructor: {
value: subClass,
enumerable: false,
writable: true,
configurable: true
}
});
if (superClass) Object.setPrototypeOf ? Object.setPrototypeOf(subClass, superClass) : subClass.__proto__ = superClass;
}
return {
setters: [function (_appPluginsSdk) {
QueryCtrl = _appPluginsSdk.QueryCtrl;
}, function (_cssQueryEditorCss) {}],
execute: function () {
_createClass = function () {
function defineProperties(target, props) {
for (var i = 0; i < props.length; i++) {
var descriptor = props[i];
descriptor.enumerable = descriptor.enumerable || false;
descriptor.configurable = true;
if ("value" in descriptor) descriptor.writable = true;
Object.defineProperty(target, descriptor.key, descriptor);
}
}
return function (Constructor, protoProps, staticProps) {
if (protoProps) defineProperties(Constructor.prototype, protoProps);
if (staticProps) defineProperties(Constructor, staticProps);
return Constructor;
};
}();
_export('GenericDatasourceQueryCtrl', GenericDatasourceQueryCtrl = function (_QueryCtrl) {
_inherits(GenericDatasourceQueryCtrl, _QueryCtrl);
function GenericDatasourceQueryCtrl($scope, $injector) {
_classCallCheck(this, GenericDatasourceQueryCtrl);
var _this = _possibleConstructorReturn(this, (GenericDatasourceQueryCtrl.__proto__ || Object.getPrototypeOf(GenericDatasourceQueryCtrl)).call(this, $scope, $injector));
_this.scope = $scope;
return _this;
}
_createClass(GenericDatasourceQueryCtrl, [{
key: 'generateSQL',
value: function generateSQL(query) {
//this.lastGenerateSQL = this.datasource.generateSql(this.target.sql, this.panelCtrl.range.raw.from, this.panelCtrl.range.raw.to, this.panelCtrl.intervalMs);
this.lastGenerateSQL = this.datasource.generateSql(this.target.sql, this.panelCtrl.range.from.toISOString(), this.panelCtrl.range.to.toISOString(), this.panelCtrl.intervalMs);
this.showGenerateSQL = !this.showGenerateSQL;
}
}]);
return GenericDatasourceQueryCtrl;
}(QueryCtrl));
_export('GenericDatasourceQueryCtrl', GenericDatasourceQueryCtrl);
GenericDatasourceQueryCtrl.templateUrl = 'partials/query.editor.html';
}
};
});
//# sourceMappingURL=query_ctrl.js.map
minidevops/prometheus/prometheus.yml 0 → 100644
浏览文件 @ 3b334b0a
global:
scrape_interval: 15s # By default, scrape targets every 15 seconds.
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'codelab-monitor'
remote_write:
- url: "http://172.15.1.7:10203/receive"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'prometheus'
# Override the global default and scrape targets from this job every 5 seconds.
scrape_interval: 5s
static_configs:
- targets: ['localhost:9090']
# - job_name: 'example-random'
# # Override the global default and scrape targets from this job every 5 seconds.
# scrape_interval: 5s
# static_configs:
# - targets: ['172.17.0.6:8080', '172.17.0.6:8081']
# labels:
# group: 'production'
# - targets: ['172.17.0.6:8082']
# labels:
# group: 'canary'
minidevops/run.sh 0 → 100755
浏览文件 @ 3b334b0a
#!/bin/bash
#set -x
LP=`pwd`
#echo $LP
docker rm -f `docker ps -a -q`
docker network rm minidevops
docker network create --ip-range 172.15.1.255/24 --subnet 172.15.1.1/16 minidevops
#docker run -d --net="host" --pid="host" -v "/:/host:ro" quay.io/prometheus/node-exporter --path.rootfs=/host
docker run -d --net minidevops --ip 172.15.1.11 -v $LP/grafana:/var/lib/grafana/plugins -p 3000:3000 grafana/grafana
#docker run -d --net minidevops --ip 172.15.1.11 -v /Users/tom/Documents/minidevops/grafana:/var/lib/grafana/plugins -p 3000:3000 grafana/grafana
TDENGINE=`docker run -d --net minidevops --ip 172.15.1.6 -p 6030:6030 -p 6020:6020 -p 6031:6031 -p 6032:6032 -p 6033:6033 -p 6034:6034 -p 6035:6035 -p 6036:6036 -p 6037:6037 -p 6038:6038 -p 6039:6039 tdengine/tdengine:1.6.4.5`
docker cp /etc/localtime $TDENGINE:/etc/localtime
BLMPROMETHEUS=`docker run -d --net minidevops --ip 172.15.1.7 -p 10203:10203 tdengine/blm_prometheus 172.15.1.6`
BLMPTELEGRAF=`docker run -d --net minidevops --ip 172.15.1.8 -p 10202:10202 tdengine/blm_telegraf 172.15.1.6`
docker run -d --net minidevops --ip 172.15.1.9 -v $LP/prometheus:/etc/prometheus -p 9090:9090 prom/prometheus
#docker run -d --net minidevops --ip 172.15.1.9 -v /Users/tom/Documents/minidevops/prometheus:/etc/prometheus -p 9090:9090 prom/prometheus
docker run -d --net minidevops --ip 172.15.1.10 -v $LP/telegraf:/etc/telegraf -p 8092:8092 -p 8094:8094 -p 8125:8125 telegraf
#docker run -d --net minidevops --ip 172.15.1.10 -v /Users/tom/Documents/minidevops/telegraf:/etc/telegraf -p 8092:8092 -p 8094:8094 -p 8125:8125 telegraf
sleep 10
curl -X POST http://localhost:3000/api/datasources --header "Content-Type:application/json" -u admin:admin -d '{"Name": "TDengine","Type": "tdengine","TypeLogoUrl": "public/plugins/tdengine/img/taosdata_logo.png","Access": "proxy","Url": "http://172.15.1.6:6020","BasicAuth": false,"isDefault": true,"jsonData": {},"readOnly": false}'
curl -X POST http://localhost:3000/api/dashboards/db --header "Content-Type:application/json" -u admin:admin -d '{"dashboard":{"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":true,"gnetId":null,"graphTooltip":0,"id":1,"links":[],"panels":[{"datasource":null,"gridPos":{"h":8,"w":6,"x":0,"y":0},"id":6,"options":{"fieldOptions":{"calcs":["mean"],"defaults":{"color":{"mode":"thresholds"},"links":[{"title":"","url":""}],"mappings":[],"max":100,"min":0,"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null},{"color":"red","value":80}]},"unit":"percent"},"overrides":[],"values":false},"orientation":"auto","showThresholdLabels":false,"showThresholdMarkers":true},"pluginVersion":"6.6.0","targets":[{"refId":"A","sql":"select last_row(value) from telegraf.mem where field=\"used_percent\""}],"timeFrom":null,"timeShift":null,"title":"Memory used percent","type":"gauge"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":null,"fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":6,"y":0},"hiddenSeries":false,"id":8,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"MEMUSED-PERCENT","refId":"A","sql":"select avg(value) from telegraf.mem where field=\"used_percent\" interval(1m)"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"MEM","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"datasource":null,"gridPos":{"h":3,"w":18,"x":0,"y":8},"id":10,"options":{"displayMode":"lcd","fieldOptions":{"calcs":["mean"],"defaults":{"mappings":[],"thresholds":{"mode":"absolute","steps":[{"color":"green","value":null}]},"unit":"percent"},"overrides":[],"values":false},"orientation":"auto","showUnfilled":true},"pluginVersion":"6.6.0","targets":[{"alias":"CPU-SYS","refId":"A","sql":"select last_row(value) from telegraf.cpu where field=\"usage_system\""},{"alias":"CPU-IDLE","refId":"B","sql":"select last_row(value) from telegraf.cpu where field=\"usage_idle\""},{"alias":"CPU-USER","refId":"C","sql":"select last_row(value) from telegraf.cpu where field=\"usage_user\""}],"timeFrom":null,"timeShift":null,"title":"CPU-USED","type":"bargauge"},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"TDengine","description":"General CPU monitor","fill":1,"fillGradient":0,"gridPos":{"h":9,"w":18,"x":0,"y":11},"hiddenSeries":false,"id":2,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"CPU-USER","refId":"A","sql":"select avg(value) from telegraf.cpu where field=\"usage_user\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-SYS","refId":"B","sql":"select avg(value) from telegraf.cpu where field=\"usage_system\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-IDLE","refId":"C","sql":"select avg(value) from telegraf.cpu where field=\"usage_idle\" and cpu=\"cpu-total\" interval(1m)"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"CPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"10s","schemaVersion":22,"style":"dark","tags":["demo"],"templating":{"list":[]},"time":{"from":"now-3h","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"]},"timezone":"","title":"TDengineDashboardDemo","id":null,"uid":null,"version":0}}'
#curl -X POST http://localhost:3000/api/dashboards/db --header "Content-Type:application/json" -u admin:admin -d '{"dashboard":{"annotations":{"list":[{"builtIn":1,"datasource":"-- Grafana --","enable":true,"hide":true,"iconColor":"rgba(0, 211, 255, 1)","name":"Annotations & Alerts","type":"dashboard"}]},"editable":true,"gnetId":null,"graphTooltip":0,"id":3,"links":[],"panels":[{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"TDengine","description":"memory used percent","fill":1,"fillGradient":0,"gridPos":{"h":8,"w":12,"x":0,"y":0},"hiddenSeries":false,"id":4,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"memused-percent","refId":"A","sql":"select avg(value) from telegraf.mem where field=\"used_percent\" interval(1m)"},{"refId":"B","sql":""}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"MEM","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}},{"aliasColors":{},"bars":false,"dashLength":10,"dashes":false,"datasource":"TDengine","description":"General CPU monitor","fill":1,"fillGradient":0,"gridPos":{"h":9,"w":12,"x":0,"y":8},"hiddenSeries":false,"id":2,"legend":{"avg":false,"current":false,"max":false,"min":false,"show":true,"total":false,"values":false},"lines":true,"linewidth":1,"nullPointMode":"null","options":{"dataLinks":[]},"percentage":false,"pointradius":2,"points":false,"renderer":"flot","seriesOverrides":[],"spaceLength":10,"stack":false,"steppedLine":false,"targets":[{"alias":"CPU-USER","refId":"A","sql":"select avg(value) from telegraf.cpu where field=\"usage_user\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-SYS","refId":"B","sql":"select avg(value) from telegraf.cpu where field=\"usage_system\" and cpu=\"cpu-total\" interval(1m)"},{"alias":"CPU-IDLE","refId":"C","sql":"select avg(value) from telegraf.cpu where field=\"usage_idle\" and cpu=\"cpu-total\" interval(1m)"}],"thresholds":[],"timeFrom":null,"timeRegions":[],"timeShift":null,"title":"CPU","tooltip":{"shared":true,"sort":0,"value_type":"individual"},"type":"graph","xaxis":{"buckets":null,"mode":"time","name":null,"show":true,"values":[]},"yaxes":[{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true},{"format":"short","label":null,"logBase":1,"max":null,"min":null,"show":true}],"yaxis":{"align":false,"alignLevel":null}}],"refresh":"10s","schemaVersion":21,"style":"dark","tags":["demo"],"templating":{"list":[]},"time":{"from":"now-6h","to":"now"},"timepicker":{"refresh_intervals":["5s","10s","30s","1m","5m","15m","30m","1h","2h","1d"]},"timezone":"","title":"TDengineDashboardDemo","id":null,"uid":null,"version":0}}'
minidevops/telegraf/telegraf.conf 0 → 100644
浏览文件 @ 3b334b0a
因为 它太大了无法显示 source diff 。你可以改为 查看blob。
packaging/deb/makedeb.sh
浏览文件 @ 3b334b0a
......@@ -73,7 +73,7 @@ sed -i "2c$debver" ${pkg_dir}/DEBIAN/control
if [ "$verMode" == "cluster" ]; then
debname="TDengine-server-"${tdengine_ver}-${osType}-${cpuType}
elif [ "$verMode" == "lite" ]; then
debname="TDengine-server-edge"-${tdengine_ver}-${osType}-${cpuType}
debname="TDengine-server"-${tdengine_ver}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or lite"
exit 1
......
packaging/release.sh
浏览文件 @ 3b334b0a
......@@ -212,7 +212,7 @@ cd ${curr_dir}
#echo "osinfo: ${osinfo}"
if [ "$osType" != "Darwin" ]; then
if [ "$verMode" != "cluster" ]; then
if [[ "$verMode" != "cluster" ]] && [[ "$cpuType" == "x64" ]]; then
echo "====do deb package for the ubuntu system===="
output_dir="${top_dir}/debs"
if [ -d ${output_dir} ]; then
......
packaging/rpm/makerpm.sh
浏览文件 @ 3b334b0a
......@@ -66,7 +66,7 @@ cp_rpm_package ${pkg_dir}/RPMS
if [ "$verMode" == "cluster" ]; then
rpmname="TDengine-server-"${tdengine_ver}-${osType}-${cpuType}
elif [ "$verMode" == "lite" ]; then
rpmname="TDengine-server-edge"-${tdengine_ver}-${osType}-${cpuType}
rpmname="TDengine-server"-${tdengine_ver}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or lite"
exit 1
......
packaging/tools/makeclient.sh
浏览文件 @ 3b334b0a
......@@ -111,7 +111,7 @@ cd ${release_dir}
if [ "$verMode" == "cluster" ]; then
pkg_name=${install_dir}-${version}-${osType}-${cpuType}
elif [ "$verMode" == "lite" ]; then
pkg_name=${install_dir}-edge-${version}-${osType}-${cpuType}
pkg_name=${install_dir}-${version}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or lite"
exit 1
......
packaging/tools/makepkg.sh
浏览文件 @ 3b334b0a
......@@ -111,7 +111,7 @@ cd ${release_dir}
if [ "$verMode" == "cluster" ]; then
pkg_name=${install_dir}-${version}-${osType}-${cpuType}
elif [ "$verMode" == "lite" ]; then
pkg_name=${install_dir}-edge-${version}-${osType}-${cpuType}
pkg_name=${install_dir}-${version}-${osType}-${cpuType}
else
echo "unknow verMode, nor cluster or lite"
exit 1
......@@ -128,4 +128,4 @@ fi
tar -zcv -f "$(basename ${pkg_name}).tar.gz" $(basename ${install_dir}) --remove-files || :
cd ${curr_dir}
\ No newline at end of file
cd ${curr_dir}
src/client/src/tscFunctionImpl.c
浏览文件 @ 3b334b0a
......@@ -3195,7 +3195,7 @@ static void diff_function(SQLFunctionCtx *pCtx) {
GET_RES_INFO(pCtx)->numOfRes += forwardStep;
pCtx->aOutputBuf += forwardStep * pCtx->outputBytes;
pCtx->ptsOutputBuf += forwardStep * TSDB_KEYSIZE;
pCtx->ptsOutputBuf = (char*)pCtx->ptsOutputBuf + forwardStep * TSDB_KEYSIZE;
}
}
......
src/inc/tglobalcfg.h
浏览文件 @ 3b334b0a
......@@ -80,6 +80,7 @@ extern short tsNumOfVnodesPerCore;
extern short tsNumOfTotalVnodes;
extern short tsCheckHeaderFile;
extern uint32_t tsPublicIpInt;
extern short tsAffectedRowsMod;
extern int tsSessionsPerVnode;
extern int tsAverageCacheBlocks;
......
src/modules/http/src/restHandle.c
浏览文件 @ 3b334b0a
......@@ -67,7 +67,10 @@ bool restProcessSqlRequest(HttpContext* pContext, int timestampFmt) {
return false;
}
// for async test
/*
* for async test
* /
/*
if (httpCheckUsedbSql(sql)) {
httpSendErrorResp(pContext, HTTP_NO_EXEC_USEDB);
......
src/system/detail/src/mgmtProfile.c
浏览文件 @ 3b334b0a
......@@ -499,10 +499,9 @@ int mgmtKillConnection(char *qidstr, SConnObj *pConn) {
uint32_t ip = inet_addr(temp);
temp = chr + 1;
short port = htons(atoi(temp));
uint16_t port = htons(atoi(temp));
SAcctObj *pAcct = pConn->pAcct;
pthread_mutex_lock(&pAcct->mutex);
pConn = pAcct->pConn;
......
src/system/detail/src/vnodeImport.c
浏览文件 @ 3b334b0a
......@@ -911,6 +911,7 @@ static int vnodeMergeDataIntoFile(SImportInfo *pImport, const char *payload, int
blockIter.nextKey = maxFileKey + 1;
} else { // Case 3. need to search the block for slot and pos
if (key == minKey || key == maxKey) {
if (tsAffectedRowsMod) pointsImported++;
payloadIter++;
continue;
}
......@@ -939,6 +940,7 @@ static int vnodeMergeDataIntoFile(SImportInfo *pImport, const char *payload, int
} while (left < right);
if (key == blockMinKey || key == blockMaxKey) { // duplicate key
if (tsAffectedRowsMod) pointsImported++;
payloadIter++;
continue;
}
......@@ -955,6 +957,7 @@ static int vnodeMergeDataIntoFile(SImportInfo *pImport, const char *payload, int
if (key == importHandle.pBlocks[blockIter.slot].keyFirst ||
key == importHandle.pBlocks[blockIter.slot].keyLast) {
if (tsAffectedRowsMod) pointsImported++;
payloadIter++;
continue;
}
......@@ -976,6 +979,7 @@ static int vnodeMergeDataIntoFile(SImportInfo *pImport, const char *payload, int
importHandle.data[PRIMARYKEY_TIMESTAMP_COL_INDEX]->data, pBlock->numOfPoints, key, TSQL_SO_ASC);
assert(pos != 0);
if (KEY_AT_INDEX(importHandle.data[PRIMARYKEY_TIMESTAMP_COL_INDEX]->data, sizeof(TSKEY), pos) == key) {
if (tsAffectedRowsMod) pointsImported++;
payloadIter++;
continue;
}
......@@ -1106,6 +1110,7 @@ static int vnodeMergeDataIntoFile(SImportInfo *pImport, const char *payload, int
if (KEY_AT_INDEX(payload, pObj->bytesPerPoint, payloadIter) ==
KEY_AT_INDEX(importHandle.data[PRIMARYKEY_TIMESTAMP_COL_INDEX]->data, sizeof(TSKEY),
blockIter.pos)) { // duplicate key
if (tsAffectedRowsMod) pointsImported++;
payloadIter++;
continue;
} else if (KEY_AT_INDEX(payload, pObj->bytesPerPoint, payloadIter) <
......@@ -1320,7 +1325,10 @@ int vnodeImportDataToCache(SImportInfo *pImport, const char *payload, const int
pImport->lastKey = lastKey;
for (payloadIter = 0; payloadIter < rows; payloadIter++) {
TSKEY key = KEY_AT_INDEX(payload, pObj->bytesPerPoint, payloadIter);
if (key == pObj->lastKey) continue;
if (key == pObj->lastKey) {
if (tsAffectedRowsMod) rowsImported++;
continue;
}
if (key > pObj->lastKey) { // Just as insert
pImport->slot = pInfo->currentSlot;
pImport->pos = pInfo->cacheBlocks[pImport->slot]->numOfPoints;
......@@ -1333,11 +1341,12 @@ int vnodeImportDataToCache(SImportInfo *pImport, const char *payload, const int
}
if (pImport->firstKey != pImport->key) break;
if (tsAffectedRowsMod) rowsImported++;
}
}
if (payloadIter == rows) {
pImport->importedRows = 0;
pImport->importedRows += rowsImported;
code = 0;
goto _exit;
}
......@@ -1470,6 +1479,7 @@ int vnodeImportDataToCache(SImportInfo *pImport, const char *payload, const int
payloadIter++;
} else {
if (tsAffectedRowsMod) rowsImported++;
payloadIter++;
continue;
}
......
src/util/CMakeLists.txt
浏览文件 @ 3b334b0a
......@@ -98,6 +98,7 @@ ELSEIF(TD_DARWIN_64)
LIST(APPEND SRC ./src/ttypes.c)
LIST(APPEND SRC ./src/tutil.c)
LIST(APPEND SRC ./src/version.c)
LIST(APPEND SRC ./src/hash.c)
ADD_LIBRARY(tutil ${SRC})
TARGET_LINK_LIBRARIES(tutil iconv pthread os)
ENDIF()
......
src/util/src/tglobalcfg.c
浏览文件 @ 3b334b0a
......@@ -83,6 +83,12 @@ short tsCheckHeaderFile = 0;
int tsSessionsPerVnode = 1000;
int tsCacheBlockSize = 16384; // 256 columns
int tsAverageCacheBlocks = TSDB_DEFAULT_AVG_BLOCKS;
/**
* Change the meaning of affected rows:
* 0: affected rows not include those duplicate records
* 1: affected rows include those duplicate records
*/
short tsAffectedRowsMod = 0;
int tsRowsInFileBlock = 4096;
float tsFileBlockMinPercent = 0.05;
......@@ -550,6 +556,9 @@ static void doInitGlobalConfig() {
tsInitConfigOption(cfg++, "alternativeRole", &tsAlternativeRole, TSDB_CFG_VTYPE_INT,
TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_CLUSTER,
0, 2, 0, TSDB_CFG_UTYPE_NONE);
tsInitConfigOption(cfg++, "affectedRowsMod", &tsAffectedRowsMod, TSDB_CFG_VTYPE_SHORT,
TSDB_CFG_CTYPE_B_CONFIG | TSDB_CFG_CTYPE_B_LOG | TSDB_CFG_CTYPE_B_CLIENT,
0, 1, 0, TSDB_CFG_UTYPE_NONE);
// 0-any, 1-mgmt, 2-dnode
// timer
......
src/util/src/ttime.c
浏览文件 @ 3b334b0a
......@@ -24,7 +24,43 @@
#include "ttime.h"
#include "tutil.h"
/*
* mktime64 - Converts date to seconds.
* Converts Gregorian date to seconds since 1970-01-01 00:00:00.
* Assumes input in normal date format, i.e. 1980-12-31 23:59:59
* => year=1980, mon=12, day=31, hour=23, min=59, sec=59.
*
* [For the Julian calendar (which was used in Russia before 1917,
* Britain & colonies before 1752, anywhere else before 1582,
* and is still in use by some communities) leave out the
* -year/100+year/400 terms, and add 10.]
*
* This algorithm was first published by Gauss (I think).
*
* A leap second can be indicated by calling this function with sec as
* 60 (allowable under ISO 8601). The leap second is treated the same
* as the following second since they don't exist in UNIX time.
*
* An encoding of midnight at the end of the day as 24:00:00 - ie. midnight
* tomorrow - (allowable under ISO 8601) is supported.
*/
int64_t user_mktime64(const unsigned int year0, const unsigned int mon0,
const unsigned int day, const unsigned int hour,
const unsigned int min, const unsigned int sec)
{
unsigned int mon = mon0, year = year0;
/* 1..12 -> 11,12,1..10 */
if (0 >= (int) (mon -= 2)) {
mon += 12; /* Puts Feb last since it has leap day */
year -= 1;
}
int64_t res = (((((int64_t) (year/4 - year/100 + year/400 + 367*mon/12 + day) +
year*365 - 719499)*24 + hour)*60 + min)*60 + sec);
return (res + timezone);
}
// ==== mktime() kernel code =================//
static int64_t m_deltaUtc = 0;
void deltaToUtcInitOnce() {
......@@ -293,7 +329,8 @@ int32_t parseLocaltime(char* timestr, int64_t* time, int32_t timePrec) {
/* mktime will be affected by TZ, set by using taos_options */
//int64_t seconds = mktime(&tm);
int64_t seconds = (int64_t)user_mktime(&tm);
//int64_t seconds = (int64_t)user_mktime(&tm);
int64_t seconds = user_mktime64(tm.tm_year+1900, tm.tm_mon+1, tm.tm_mday, tm.tm_hour, tm.tm_min, tm.tm_sec);
int64_t fraction = 0;
......
src/util/src/ttypes.c
浏览文件 @ 3b334b0a
......@@ -976,11 +976,21 @@ void assignVal(char *val, const char *src, int32_t len, int32_t type) {
break;
}
case TSDB_DATA_TYPE_FLOAT: {
#ifdef _TD_ARM_32_
float fv = GET_FLOAT_VAL(src);
SET_FLOAT_VAL_ALIGN(val, &fv);
#else
*((float *)val) = GET_FLOAT_VAL(src);
#endif
break;
};
case TSDB_DATA_TYPE_DOUBLE: {
#ifdef _TD_ARM_32_
double dv = GET_DOUBLE_VAL(src);
SET_DOUBLE_VAL_ALIGN(val, &dv);
#else
*((double *)val) = GET_DOUBLE_VAL(src);
#endif
break;
};
case TSDB_DATA_TYPE_TIMESTAMP:
......
src/util/src/version.c
浏览文件 @ 3b334b0a
char version[64] = "1.6.5.3";
char version[64] = "1.6.5.4";
char compatible_version[64] = "1.6.1.0";
char gitinfo[128] = "700305490a82228ec1b0244afb838bdbb9de9793";
char gitinfoOfInternal[128] = "";
char buildinfo[512] = "Built by at 2020-01-17 13:22";
char gitinfo[128] = "3264067e97300c84caa61ac909d548c9ca56de6b";
char gitinfoOfInternal[128] = "da88f4a2474737d1f9c76adcf0ff7fd0975e7342";
char buildinfo[512] = "Built by root at 2020-02-05 14:38";
void libtaos_edge_1_6_5_1_Linux_x64() {};
void libtaos_1_6_5_4_Linux_x64() {};
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或