@@ -101,9 +101,9 @@ You can go to jira to pick up the existing issue or create your own issue and ge
Recommended Use Intellij idea. ```mvn clean package -DskipTests``` After putting ```antlr/target/generated-sources/antlr4``` and ```thrift/target/generated-sources/thrift``` marked as ```Source Root```。
* Server main function：```server/src/main/java/org/apache/iotdb/db/service/IoTDB```，Can be started in debug mode
* Client：```cli/src/main/java/org/apache/iotdb/cli/```，Use Client for linux and WinClint for windows, you can start directly, need the parameter "-h 127.0.0.1 -p 6667 -u root -pw root"
* Server rpc implementation (mainly used for client and server communication, generally start interruption point here):```server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl```
* Server main function：```server/src/main/java/org/apache/iotdb/db/service/IoTDB``, Can be started in debug mode
* Cli：```cli/src/main/java/org/apache/iotdb/cli/```，Use Cli for linux and WinCli for windows, you can start directly with the parameter "-h 127.0.0.1 -p 6667 -u root -pw root"
* Server rpc implementation (mainly used for cli and server communication, generally start interruption point here):```server/src/main/java/org/apache/iotdb/db/service/TSServiceImpl```
* all jdbc statements：executeStatement(TSExecuteStatementReq req)
@@ -426,15 +426,6 @@ The permission definitions are in ${IOTDB\_CONF}/conf/jmx.access.
|Default| false |
|Effective|After restart system|
* back\_loop\_period_in_second
|Name| back\_loop\_period\_in\_second |
|:---:|:---|
|Description| The frequency at which the system statistic module triggers(in seconds). |
|Type|Int32|
|Default| 5 |
|Effective|After restart system|
* concurrent\_flush\_thread
|Name| concurrent\_flush\_thread |
...
...
@@ -444,24 +435,6 @@ The permission definitions are in ${IOTDB\_CONF}/conf/jmx.access.
|Default| 0 |
|Effective|After restart system|
* stat\_monitor\_detect\_freq\_in\_second
|Name| stat\_monitor\_detect\_freq\_in\_second |
|:---:|:---|
|Description| The time interval which the system check whether the current record statistic time range exceeds stat_monitor_retain_interval every time (in seconds) and perform regular cleaning|
@@ -31,332 +31,113 @@ After starting JConsole tool and connecting to IoTDB server, a basic look at IoT
#### JMX MBean Monitoring
By using JConsole tool and connecting with JMX you are provided with some system statistics and parameters.
This section describes how to use the JConsole ```Mbean``` tab to monitor the number of files opened by the IoTDB service process, the size of the data file, and so on. Once connected to JMX, you can find the ```MBean``` named ```org.apache.iotdb.service``` through the ```MBeans``` tab, as shown in the following Figure.
This section describes how to use the JConsole ```Mbean```tab of jconsole to monitor some system configurations of IoTDB, the statistics of writing, and so on. After connecting to JMX, you can find the "MBean" of "org.apache.iotdb.service", as shown in the figure below.
There are several attributes under Monitor, including the numbers of files opened in different folders, the data file size statistics and the values of some system parameters. By double-clicking the value corresponding to an attribute it also displays a line chart of that attribute. Currently, all the opened file count statistics are only supported on ```MacOS``` and most ```Linux``` distro except ```CentOS```. For the OS not supported these statistics returns ```-2```. See the following section for specific introduction of the Monitor attributes.
There are several attributes under monitor, including data file directory, the statistics of writing and the values of some system parameters. It can also display a line chart of the property by double clicking the value corresponding to the property. For a detailed description of monitor attributes, see the following sections.
|Name| DataSizeInByte |
|:---:|:---|
|Description| The total size of data file.|
|Unit| Byte |
|Type| Long |
* FileNodeNum
|Name| FileNodeNum |
|:---:|:---|
|Description| The count number of FileNode. (Currently not supported)|
|Type| Long |
##### MBean Monitor Attributes List
*OverflowCacheSize
*SystemDirectory
|Name| OverflowCacheSize |
|Name| SystemDirectory |
|:---:|:---|
|Description| The size of out-of-order data cache. (Currently not supported)|
|Unit| Byte |
|Type| Long |
|Description| The absolute directory of data system file. |
|Type| String |
*BufferWriteCacheSize
*DataSizeInByte
|Name| BufferWriteCacheSize |
|Name| DataSizeInByte |
|:---:|:---|
|Description| The size of BufferWriter cache. (Currently not supported)|
|Description| The total size of data file.|
|Unit| Byte |
|Type| Long |
*BaseDirectory
*EnableStatMonitor
|Name| BaseDirectory |
|Name| EnableStatMonitor |
|:---:|:---|
|Description| The absolute directory of data file. |
|Type| String |
* WriteAheadLogStatus
|Name| WriteAheadLogStatus |
|:---:|:---|
|Description| The status of write-ahead-log (WAL). ```True``` means WAL is enabled. |
|Description| If the monitor module is open |
|Type| Boolean |
* TotalOpenFileNum
### Data Monitoring
|Name| TotalOpenFileNum |
|:---:|:---|
|Description| All the opened file number of IoTDB server process. |
|Type| Int |
This module is for providing some statistics info about the writing operations:
* DeltaOpenFileNum
- the data size (in bytes) in IoTDB, the number of data points in IoTDB;
- how many operations are successful or failed executed.
|Name| DeltaOpenFileNum |
|:---:|:---|
|Description| The opened TsFile file number of IoTDB server process. |
|Default Directory| /data/data/settled |
|Type| Int |
#### Enable/disable the module
* WalOpenFileNum
Users can choose to enable or disable the feature of data statistics monitoring (set the `enable_stat_monitor` item in the configuration file).
|Name| WalOpenFileNum |
|:---:|:---|
|Description| The opened write-ahead-log file number of IoTDB server process. |
|Default Directory| /data/wal |
|Type| Int |
* MetadataOpenFileNum
|Name| MetadataOpenFileNum |
|:---:|:---|
|Description| The opened meta-data file number of IoTDB server process. |
|Default Directory| /data/system/schema |
|Type| Int |
* DigestOpenFileNum
|Name| DigestOpenFileNum |
|:---:|:---|
|Description| The opened info file number of IoTDB server process. |
|Default Directory| /data/system/info |
|Type| Int |
* SocketOpenFileNum
|Name| SocketOpenFileNum |
|:---:|:---|
|Description| The Socket link (TCP or UDP) number of the operation system. |
|Type| Int |
* MergePeriodInSecond
|Name| MergePeriodInSecond |
|:---:|:---|
|Description| The interval at which the IoTDB service process periodically triggers the merge process. |
|Unit| Second |
|Type| Long |
* ClosePeriodInSecond
|Name| ClosePeriodInSecond |
|:---:|:---|
|Description| The interval at which the IoTDB service process periodically flushes memory data to disk. |
|Unit| Second |
|Type| Long |
#### Statistics Data Storing
### Data Status Monitoring
By default, the statistics data is only saved in memory and can be accessed using Jconsole.
This module is the statistical monitoring method provided by IoTDB for users to store data information. The statistical data are recorded in the system and stored in the database. The current 0.8.0 version of IoTDB provides statistics for writing data.
The data can also be written as some time series on disk. To enable it, set `enable_monitor_series_write=true` in the configuration file. If so, using `select` statement in IoTDB-cli can query these time series.
The user can choose to enable or disable the data statistics monitoring function (set the `enable_stat_monitor` item in the configuration file).
> Note:
> if `enable_monitor_series_write=true`, when IoTDB is restarted, the previous statistics data will be recovered into memory.
> if `enable_monitor_series_write=false`, IoTDB will forget all statistics data after the instance is restarted.
#### Writing Data Monitor
The current statistics of writing data by the system can be divided into two major modules: **Global Writing Data Statistics** and **Storage Group Writing Data Statistics**. **Global Writing Data Statistics** records the point number written by the user and the number of requests. **Storage Group Writing Data Statistics** records data of a certain storage group.
At present, the monitor system can be divided into two modules: global writing statistics and storage group writing statistics. The global statistics records the number of total points and requests, and the storage group statistics counts the write data of each storage group.
The system defaults to collect data every 5 seconds, and writes the statistics to the IoTDB and stores them in a system-specified locate. (If you need to change the statistic frequency, you can set The `back_loop_period_in_second entry` in the configuration file, see Section [Engine Layer](../Server/Single%20Node%20Setup.md) for details). After the system is refreshed or restarted, IoTDB does not recover the statistics, and the statistics data will restart from zero.
The system sets the collection granularity of the monitoring module to **update the statistical information once one data file is flushed into the disk **, so the data accuracy may be different from the actual situation. To obtain accurate information, **Please call the `flush` method before querying **.
To avoid the excessive use of statistical information, a mechanism is set to periodically clear invalid data for statistical information. The system deletes invalid data at regular intervals. The user set the trigger frequency (`stat_monitor_retain_interval_in_second`, default is 600s, see section [Engine Layer](../Server/Single%20Node%20Setup.md) for details) to set the frequency of deleting data. By setting the valid data duration (`stat_monitor_detect_freq_in_second entry`, the default is 600s, see section [Engine Layer](../Server/Single%20Node%20Setup.md) for details) to set the time period of valid data, that is, the data within the time of the clear operation trigger time is stat_monitor_detect_freq_in_second is valid data. In order to ensure the stability of the system, it is not allowed to delete the statistics frequently. Therefore, if the configuration parameter time is less than the default value (600s), the system will abort the configuration parameter and use the default parameter.
Here are the writing data statistics (the range supported is shown in brackets):
It's convenient for you to use `select` clause to get the writing data statistics the same as other timeseries.
Here are the writing data statistics:
* TOTAL_POINTS (GLOABAL)
* TOTAL_POINTS (GLOBAL)
|Name| TOTAL\_POINTS |
|:---:|:---|
|Description| Calculate the global writing points number.|
The above attributes also support visualization in JConsole. For the statistical information of each storage group, in order to avoid the display confusion caused by too many storage groups, the user can input the storage group name in the operation method under monitor MBean to query the corresponding statistical information.
|Example| select TOTAL\_POINTS\_FAIL from root.stats.write.\<storage\_group\_name\>|
> Note:
>
> \<storage\_group\_name\> should be replaced by real storage group name, and the '.' in storage group need to be replaced by '_'. For example, the storage group name is 'root.a.b', when using in the statistics, it will change to 'root\_a\_b'
Here we give some example of using writing data statistics.
Here we give some examples of using writing data statistics.
To know the global successful writing points number, use `select` clause to query it's value. The query statement is:
To know the total number of global writing points, use `select` clause to query it's value. The query statement is:
```
select TOTAL_POINTS_SUCCESS from root.stats.write.global
```sql
selectTOTAL_POINTSfromroot.stats."global"
```
To know the successfule writing points number of root.ln (storage group), the query statement is:
To know the total number of global writing points of root.ln (storage group), the query statement is:
```
select TOTAL_POINTS_SUCCESS from root.stats.write.root_ln
```sql
selectTOTAL_POINTSfromroot.stats."root.ln"
```
To know the current timeseries point in the system, use `MAX_VALUE` function to query. Here is the query statement:
To know the latest statistics of the current system, you can use the latest data query. Here is the query statement:
```sql
flush
selectlastTOTAL_POINTSfromroot.stats."global"
```
select MAX_VALUE(TOTAL_POINTS_SUCCESS) from root.stats.write.root_ln
```
#### File Size Monitor
Sometimes we are concerned about how the data file size of IoTDB changes, maybe to help calculate how much disk space is left or the data ingestion speed. The File Size Monitor provides several statistics to show how different types of file-sizes change.
The file size monitor defaults to collect file size data every 5 seconds using the same shared parameter ```back_loop_period_in_second```,
Unlike Writing Data Monitor, currently File Size Monitor does not delete statistic data at regular intervals.
You can also use `select` clause to get the file size statistics like other time series.
Here are the file size statistics:
* DATA
|Name| DATA |
|:---:|:---|
|Description| Calculate the sum of all the files's sizes under the data directory (```data/data``` by default) in byte.|
|Type| File size statistics |
|Timeseries Name| root.stats.file\_size.DATA |
|Reset After Restarting System| No |
|Example| select DATA from root.stats.file\_size.DATA|
* SETTLED
|Name| SETTLED |
|:---:|:---|
|Description| Calculate the sum of all the ```TsFile``` size (under ```data/data/settled``` by default) in byte. If there are multiple ```TsFile``` directories like ```{data/data/settled1, data/data/settled2}```, this statistic is the sum of their size.|
|Type| File size statistics |
|Timeseries Name| root.stats.file\_size.SETTLED |
|Reset After Restarting System| No |
|Example| select SETTLED from root.stats.file\_size.SETTLED|
* OVERFLOW
|Name| OVERFLOW |
|:---:|:---|
|Description| Calculate the sum of all the ```out-of-order data file``` size (under ```data/data/unsequence``` by default) in byte.|
