“d8aebaf50c38c88a05728f3bb915da7e767ff496”上不存在“paddle/fluid/platform/dynload/nccl.h”
07-python.mdx 30.2 KB
Newer Older
1 2
---
title: TDengine Python Connector
D
danielclow 已提交
3 4
sidebar_label: Python
description: This document describes taospy, the TDengine Python connector.
5 6 7 8 9
---

import Tabs from "@theme/Tabs";
import TabItem from "@theme/TabItem";

10
`taospy` is the official Python connector for TDengine. taospy provides a rich API that makes it easy for Python applications to use TDengine. `taospy` wraps both the [native interface](/reference/connector/cpp) and [REST interface](/reference/rest-api) of TDengine, which correspond to the `taos` and `taosrest` modules of the `taospy` package, respectively.
11 12
In addition to wrapping the native and REST interfaces, `taospy` also provides a set of programming interfaces that conforms to the [Python Data Access Specification (PEP 249)](https://peps.python.org/pep-0249/). It is easy to integrate `taospy` with many third-party tools, such as [SQLAlchemy](https://www.sqlalchemy.org/) and [pandas](https://pandas.pydata.org/).

13
`taos-ws-py` is an optional package to enable using WebSocket to connect TDengine.
14

15
The direct connection to the server using the native interface provided by the client driver is referred to hereinafter as a "native connection"; the connection to the server using the REST or WebSocket interface provided by taosAdapter is referred to hereinafter as a "REST connection" or "WebSocket connection".
16

17
The source code for the Python connector is hosted on [GitHub](https://github.com/taosdata/taos-connector-python).
D
danielclow 已提交
18
## Supported platforms
19

20
- The [supported platforms](/reference/connector/#supported-platforms) for the native connection are the same as the ones supported by the TDengine client.
21 22
- REST connections are supported on all platforms that can run Python.

S
sunpeng 已提交
23 24 25 26 27
### Supported features

- Native connections support all the core features of TDengine, including connection management, SQL execution, bind interface, subscriptions, and schemaless writing.
- REST connections support features such as connection management and SQL execution. (SQL execution allows you to: manage databases, tables, and supertables, write data, query data, create continuous queries, etc.).

28 29
## Version selection

30
We recommend using the latest version of `taospy`, regardless of the version of TDengine.
31

32 33 34 35 36 37 38 39 40 41
|Python Connector Version|major changes|
|:-------------------:|:----:|
|2.7.9|support for getting assignment and seek function on subscription|
|2.7.8|add `execute_many` method|

|Python Websocket Connector Version|major changes|
|:----------------------------:|:-----:|
|0.2.5|1. support for getting assignment and seek function on subscription <br/> 2. support schemaless <br/> 3. support STMT|
|0.2.4|support `unsubscribe` on subscription|

S
sunpeng 已提交
42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71
## Handling Exceptions

There are 4 types of exception in python connector.

- The exception of Python Connector itself.
- The exception of native library.
- The exception of websocket
- The exception of subscription.
- The exception of other TDengine function modules.

|Error Type|Description|Suggested Actions|
|:--------:|:---------:|:---------------:|
|InterfaceError|the native library is too old that it cannot support the function|please check the TDengine client version|
|ConnectionError|connection error|please check TDengine's status and the connection params|
|DatabaseError|database error|please upgrade Python connector to latest|
|OperationalError|operation error||
|ProgrammingError|||
|StatementError|the exception of stmt||
|ResultError|||
|SchemalessError|the exception of stmt schemaless||
|TmqError|the exception of stmt tmq||

It usually uses try-expect to handle exceptions in python. For exception handling, please refer to [Python Errors and Exceptions Documentation](https://docs.python.org/3/tutorial/errors.html).

All exceptions from the Python Connector are thrown directly. Applications should handle these exceptions. For example:

```python
{{#include docs/examples/python/handle_exception.py}}
```

S
sunpeng 已提交
72
## TDengine DataType vs. Python DataType
73

S
sunpeng 已提交
74 75 76 77 78 79 80 81 82 83 84 85 86 87 88
TDengine currently supports timestamp, number, character, Boolean type, and the corresponding type conversion with Python is as follows:

|TDengine DataType|Python DataType|
|:---------------:|:-------------:|
|TIMESTAMP|datetime|
|INT|int|
|BIGINT|int|
|FLOAT|float|
|DOUBLE|int|
|SMALLINT|int|
|TINYINT|int|
|BOOL|bool|
|BINARY|str|
|NCHAR|str|
|JSON|str|
89 90 91 92 93

## Installation

### Preparation

94
1. Install Python. The recent taospy package requires Python 3.6.2+. The earlier versions of taospy require Python 3.7+.  The taos-ws-py package requires Python 3.7+. If Python is not available on your system, refer to the [Python BeginnersGuide](https://wiki.python.org/moin/BeginnersGuide/Download) to install it.
95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112
2. Install [pip](https://pypi.org/project/pip/). In most cases, the Python installer comes with the pip utility. If not, please refer to [pip documentation](https://pip.pypa.io/en/stable/installation/) to install it.
If you use a native connection, you will also need to [Install Client Driver](/reference/connector#Install-Client-Driver). The client install package includes the TDengine client dynamic link library (`libtaos.so` or `taos.dll`) and the TDengine CLI.

### Install via pip

#### Uninstalling an older version

If you have installed an older version of the Python Connector, please uninstall it beforehand.

```
pip3 uninstall taos taospy
```

:::note
Earlier TDengine client software includes the Python connector. If the Python connector is installed from the client package's installation directory, the corresponding Python package name is `taos`. So the above uninstall command includes `taos`, and it doesn't matter if it doesn't exist.

:::

113
#### To install `taospy`
114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139

<Tabs>
<TabItem label="Install from PyPI" value="pypi">

Install the latest version of:

```
pip3 install taospy
```

You can also specify a specific version to install:

```
pip3 install taospy==2.3.0
```

</TabItem>
<TabItem label="Install from GitHub" value="github">

```
pip3 install git+https://github.com/taosdata/taos-connector-python.git
```

</TabItem>
</Tabs>

140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155
#### Install `taos-ws-py` (Optional)

The taos-ws-py package provides the way to access TDengine via WebSocket.

##### Install taos-ws-py with taospy

```bash
pip3 install taospy[ws]
```

##### Install taos-ws-py only

```bash
pip3 install taos-ws-py
```

D
danielclow 已提交
156
### Verify
157

G
gccgdb1234 已提交
158
<Tabs defaultValue="rest">
G
gccgdb1234 已提交
159
<TabItem value="native" label="native connection">
160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176

For native connection, you need to verify that both the client driver and the Python connector itself are installed correctly. The client driver and Python connector have been installed properly if you can successfully import the `taos` module. In the Python Interactive Shell, you can type.

```python
import taos
```

</TabItem>
<TabItem value="rest" label="REST connection">

For REST connections, verifying that the `taosrest` module can be imported successfully can be done in the Python Interactive Shell by typing.

```python
import taosrest
```

</TabItem>
177 178 179 180 181 182 183 184
<TabItem  value="ws" label="WebSocket connection">

For WebSocket connection, verifying that the `taosws` module can be imported successfully can be done in the Python Interactive Shell by typing.

```python
import taosws
```

185
</TabItem>
186 187 188 189 190 191 192 193 194
</Tabs>

:::tip
If you have multiple versions of Python on your system, you may have various `pip` commands. Be sure to use the correct path for the `pip` command. Above, we installed the `pip3` command, which rules out the possibility of using the `pip` corresponding to Python 2.x versions. However, if you have more than one version of Python 3.x on your system, you still need to check that the installation path is correct. The easiest way to verify this is to type `pip3 install taospy` again in the command, and it will print out the exact location of `taospy`, for example, on Windows.

```
C:\> pip3 install taospy
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Requirement already satisfied: taospy in c:\users\username\appdata\local\programs\python\python310\lib\site-packages (2.3.0)
D
danielclow 已提交
195
```
196 197 198

:::

D
danielclow 已提交
199
## Establishing a connection
200 201 202 203 204

### Connectivity testing

Before establishing a connection with the connector, we recommend testing the connectivity of the local TDengine CLI to the TDengine cluster.

G
gccgdb1234 已提交
205
<Tabs defaultValue="rest">
206 207
<TabItem value="native" label="native connection">

208
Ensure that the TDengine instance is up and that the FQDN of the machines in the cluster (the FQDN defaults to hostname if you are starting a stand-alone version) can be resolved locally, by testing with the `ping` command.
209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224

```
ping <FQDN>
```

Then test if the cluster can be appropriately connected with TDengine CLI:

```
taos -h <FQDN> -p <PORT>
```

The FQDN above can be the FQDN of any dnode in the cluster, and the PORT is the serverPort corresponding to this dnode.

</TabItem>
<TabItem value="rest" label="REST connection" groupId="connect">

225
For REST connections, make sure the cluster and taosAdapter component, are running. This can be tested using the following `curl ` command.
226 227 228 229 230 231 232 233 234 235

```
curl -u root:taosdata http://<FQDN>:<PORT>/rest/sql -d "select server_version()"
```

The FQDN above is the FQDN of the machine running taosAdapter, PORT is the port taosAdapter listening, default is `6041`.
If the test is successful, it will output the server version information, e.g.

```json
{
236 237 238 239 240 241 242 243 244 245 246 247 248
  "code": 0,
  "column_meta": [
    [
      "server_version()",
      "VARCHAR",
      7
    ]
  ],
  "data": [
    [
      "3.0.0.0"
    ]
  ],
249 250 251 252
  "rows": 1
}
```

253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274
</TabItem>
<TabItem value="ws" label="WebSocket connection" groupId="connect">

For WebSocket connection, make sure the cluster and taosAdapter component, are running. This can be testetd using the following `curl` command.

```
curl -i -N -d "show databases" -H "Authorization: Basic cm9vdDp0YW9zZGF0YQ==" -H "Connection: Upgrade" -H "Upgrade: websocket" -H "Host: <FQDN>:<PORT>" -H "Origin: http://<FQDN>:<PORT>" http://<FQDN>:<PORT>/rest/sql
```

The FQDN above is the FQDN of the machine running taosAdapter, PORT is the port taosAdapter listening, default is `6041`.

If the test is successful, it will output the server version information, e.g.

```json
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Date: Tue, 21 Mar 2023 09:29:17 GMT
Transfer-Encoding: chunked

{"status":"succ","head":["server_version()"],"column_meta":[["server_version()",8,8]],"data":[["2.6.0.27"]],"rows":1}
```

275 276 277 278 279 280 281
</TabItem>
</Tabs>

### Using connectors to establish connections

The following example code assumes that TDengine is installed locally and that the default configuration is used for both FQDN and serverPort.

G
gccgdb1234 已提交
282
<Tabs defaultValue="rest">
283 284 285
<TabItem value="native" label="native connection" groupId="connect">

```python
D
dingbo 已提交
286
{{#include docs/examples/python/connect_native_reference.py}}
287 288 289 290 291 292 293 294
```

All arguments of the `connect()` function are optional keyword arguments. The following are the connection parameters specified.

- `host` : The FQDN of the node to connect to. There is no default value. If this parameter is not provided, the firstEP in the client configuration file will be connected.
- `user` : The TDengine user name. The default value is `root`.
- `password` : TDengine user password. The default value is `taosdata`.
- `port` : The starting port of the data node to connect to, i.e., the serverPort configuration. The default value is 6030, which will only take effect if the host parameter is provided.
wafwerar's avatar
wafwerar 已提交
295
- `config` : The path to the client configuration file. On Windows systems, the default is `C:\TDengine\cfg`. The default is `/etc/taos/` on Linux/macOS.
296 297 298 299 300 301 302 303 304 305 306 307 308 309 310
- `timezone` : The timezone used to convert the TIMESTAMP data in the query results to python `datetime` objects. The default is the local timezone.

:::warning
`config` and `timezone` are both process-level configurations. We recommend that all connections made by a process use the same parameter values. Otherwise, unpredictable errors may occur.
:::

:::tip
The `connect()` function returns a `taos.TaosConnection` instance. In client-side multi-threaded scenarios, we recommend that each thread request a separate connection instance rather than sharing a connection between multiple threads.

:::

</TabItem>
<TabItem value="rest" label="REST connection">

```python
D
dingbo 已提交
311
{{#include docs/examples/python/connect_rest_examples.py:connect}}
312 313 314 315
```

All arguments to the `connect()` function are optional keyword arguments. The following are the connection parameters specified.

316
- `url`: The URL of taosAdapter REST service. The default is <http://localhost:6041>.
317 318
- `user`: TDengine user name. The default is `root`.
- `password`: TDengine user password. The default is `taosdata`.
D
danielclow 已提交
319
- `timeout`: HTTP request timeout. Enter a value in seconds. The default is `socket._GLOBAL_DEFAULT_TIMEOUT`. Usually, no configuration is needed.
320

321 322 323 324 325 326 327 328 329 330
</TabItem>

<TabItem value="websocket" label="WebSocket connection">

```python
{{#include docs/examples/python/connect_websocket_examples.py:connect}}
```

The parameter of `connect()` is the url of TDengine, and the protocol is `taosws` or `ws`.

331 332 333
</TabItem>
</Tabs>

D
danielclow 已提交
334
## Example program
335 336 337

### Basic Usage

G
gccgdb1234 已提交
338
<Tabs defaultValue="rest">
339 340 341 342 343 344 345
<TabItem value="native" label="native connection">

##### TaosConnection class

The `TaosConnection` class contains both an implementation of the PEP249 Connection interface (e.g., the `cursor()` method and the `close()` method) and many extensions (e.g., the `execute()`, `query()`, `schemaless_insert()`, and `subscribe()` methods).

```python title="execute method"
D
dingbo 已提交
346
{{#include docs/examples/python/connection_usage_native_reference.py:insert}}
347 348 349
```

```python title="query method"
D
dingbo 已提交
350
{{#include docs/examples/python/connection_usage_native_reference.py:query}}
351 352 353 354 355 356 357 358 359 360 361
```

:::tip
The queried results can only be fetched once. For example, only one of `fetch_all()` and `fetch_all_into_dict()` can be used in the example above. Repeated fetches will result in an empty list.
:::

##### Use of TaosResult class

In the above example of using the `TaosConnection` class, we have shown two ways to get the result of a query: `fetch_all()` and `fetch_all_into_dict()`. In addition, `TaosResult` also provides methods to iterate through the result set by rows (`rows_iter`) or by data blocks (`blocks_iter`). Using these two methods will be more efficient in scenarios where the query has a large amount of data.

```python title="blocks_iter method"
D
dingbo 已提交
362
{{#include docs/examples/python/result_set_examples.py}}
363 364 365 366 367 368
```
##### Use of the TaosCursor class

The `TaosConnection` class and the `TaosResult` class already implement all the functionality of the native interface. If you are familiar with the interfaces in the PEP249 specification, you can also use the methods provided by the `TaosCursor` class.

```python title="Use of TaosCursor"
D
dingbo 已提交
369
{{#include docs/examples/python/cursor_usage_native_reference.py}}
370 371 372 373 374 375 376 377 378 379 380 381
```

:::note
The TaosCursor class uses native connections for write and query operations. In a client-side multi-threaded scenario, this cursor instance must remain thread exclusive and cannot be shared across threads for use, otherwise, it will result in errors in the returned results.

:::

</TabItem>
<TabItem value="rest" label="REST connection">

##### Use of TaosRestCursor class

D
danielclow 已提交
382
The `TaosRestCursor` class is an implementation of the PEP249 Cursor interface.
383 384

```python title="Use of TaosRestCursor"
D
dingbo 已提交
385
{{#include docs/examples/python/connect_rest_examples.py:basic}}
386
```
D
danielclow 已提交
387
- `cursor.execute`: Used to execute arbitrary SQL statements.
388 389 390 391 392 393 394 395
- `cursor.rowcount` : For write operations, returns the number of successful rows written. For query operations, returns the number of rows in the result set.
- `cursor.description` : Returns the description of the field. Please refer to [TaosRestCursor](https://docs.taosdata.com/api/taospy/taosrest/cursor.html) for the specific format of the description information.

##### Use of the RestClient class

The `RestClient` class is a direct wrapper for the [REST API](/reference/rest-api). It contains only a `sql()` method for executing arbitrary SQL statements and returning the result.

```python title="Use of RestClient"
D
dingbo 已提交
396
{{#include docs/examples/python/rest_client_example.py}}
397 398 399 400
```

For a more detailed description of the `sql()` method, please refer to [RestClient](https://docs.taosdata.com/api/taospy/taosrest/restclient.html).

401 402 403
</TabItem>
<TabItem value="websocket" label="WebSocket connection">

S
sunpeng 已提交
404 405
The `Connection` class contains both an implementation of the PEP249 Connection interface (e.g., the `cursor()` method and the `close()` method) and many extensions (e.g., the `execute()`, `query()`, `schemaless_insert()`, and `subscribe()` methods).

406 407 408
```python
{{#include docs/examples/python/connect_websocket_examples.py:basic}}
```
D
danielclow 已提交
409

410 411
- `conn.execute`: can use to execute arbitrary SQL statements, and return the number of rows affected.
- `conn.query`: can use to execute query SQL statements, and return the query results.
D
danielclow 已提交
412

413 414 415
</TabItem>
</Tabs>

S
sunpeng 已提交
416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455
### Querying Data

<Tabs defaultValue="rest">
<TabItem value="native" label="native connection">

The `query` method of the `TaosConnection` class can be used to query data and return the result data of type `TaosResult`.

```python
{{#include docs/examples/python/connection_usage_native_reference.py:query}}
```

:::tip
The queried results can only be fetched once. For example, only one of `fetch_all()` and `fetch_all_into_dict()` can be used in the example above. Repeated fetches will result in an empty list.
:::

</TabItem>

<TabItem value="rest" label="REST connection">

The `RestClient` class is a direct wrapper for the [REST API](/reference/rest-api). It contains only a `sql()` method for executing arbitrary SQL statements and returning the result.

```python
{{#include docs/examples/python/rest_client_example.py}}
```

For a more detailed description of the `sql()` method, please refer to [RestClient](https://docs.taosdata.com/api/taospy/taosrest/restclient.html).

</TabItem>

<TabItem value="websocket" label="WebSocket connection">

The `query` method of the `TaosConnection` class can be used to query data and return the result data of type `TaosResult`.

```python
{{#include docs/examples/python/connect_websocket_examples.py:basic}}
```

</TabItem>
</Tabs>

456 457 458 459 460 461 462 463 464
### Usage with req_id

By using the optional req_id parameter, you can specify a request ID that can be used for tracing.

<Tabs defaultValue="rest">
<TabItem value="native" label="native connection">

##### TaosConnection class

465
As the way to connect introduced above but add `req_id` argument.
466 467 468 469 470 471 472 473 474 475 476

```python title="execute method"
{{#include docs/examples/python/connection_usage_native_reference_with_req_id.py:insert}}
```

```python title="query method"
{{#include docs/examples/python/connection_usage_native_reference_with_req_id.py:query}}
```

##### Use of TaosResult class

477
As the way to fetch data introduced above but add `req_id` argument.
478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494

```python title="blocks_iter method"
{{#include docs/examples/python/result_set_with_req_id_examples.py}}
```
##### Use of the TaosCursor class

The `TaosConnection` class and the `TaosResult` class already implement all the functionality of the native interface. If you are familiar with the interfaces in the PEP249 specification, you can also use the methods provided by the `TaosCursor` class.

```python title="Use of TaosCursor"
{{#include docs/examples/python/cursor_usage_native_reference_with_req_id.py}}
```

</TabItem>
<TabItem value="rest" label="REST connection">

##### Use of TaosRestCursor class

495
As the way to connect introduced above but add `req_id` argument.
496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514

```python title="Use of TaosRestCursor"
{{#include docs/examples/python/connect_rest_with_req_id_examples.py:basic}}
```
- `cursor.execute`: Used to execute arbitrary SQL statements.
- `cursor.rowcount` : For write operations, returns the number of successful rows written. For query operations, returns the number of rows in the result set.
- `cursor.description` : Returns the description of the field. Please refer to [TaosRestCursor](https://docs.taosdata.com/api/taospy/taosrest/cursor.html) for the specific format of the description information.

##### Use of the RestClient class

The `RestClient` class is a direct wrapper for the [REST API](/reference/rest-api). It contains only a `sql()` method for executing arbitrary SQL statements and returning the result.

```python title="Use of RestClient"
{{#include docs/examples/python/rest_client_with_req_id_example.py}}
```

For a more detailed description of the `sql()` method, please refer to [RestClient](https://docs.taosdata.com/api/taospy/taosrest/restclient.html).

</TabItem>
515

516 517
<TabItem value="websocket" label="WebSocket connection">

518 519
As the way to connect introduced above but add `req_id` argument.

520 521 522 523 524 525 526 527 528 529
```python
{{#include docs/examples/python/connect_websocket_with_req_id_examples.py:basic}}
```

- `conn.execute`: can use to execute arbitrary SQL statements, and return the number of rows affected.
- `conn.query`: can use to execute query SQL statements, and return the query results.

</TabItem>
</Tabs>

530 531
### Used with pandas

G
gccgdb1234 已提交
532
<Tabs defaultValue="rest">
533 534 535
<TabItem value="native" label="native connection">

```python
D
dingbo 已提交
536
{{#include docs/examples/python/conn_native_pandas.py}}
537 538 539 540 541 542
```

</TabItem>
<TabItem value="rest" label="REST connection">

```python
D
dingbo 已提交
543
{{#include docs/examples/python/conn_rest_pandas.py}}
544 545
```

546 547 548 549 550 551 552
</TabItem>
<TabItem value="websocket" label="WebSocket connection">

```python
{{#include docs/examples/python/conn_websocket_pandas.py}}
```

553 554 555
</TabItem>
</Tabs>

S
sunpeng 已提交
556 557 558 559 560 561 562 563 564
### Subscription

Connector support data subscription. For more information about subscroption, please refer to [Data Subscription](../../../develop/tmq/).

<Tabs defaultValue="native">
<TabItem value="native" label="native connection">

The `consumer` in the connector contains the subscription api. 

S
sunpeng 已提交
565
##### Create Consumer
S
sunpeng 已提交
566 567 568 569 570 571 572 573 574

The syntax for creating a consumer is `consumer = Consumer(configs)`. For more subscription api parameters, please refer to [Data Subscription](../../../develop/tmq/).

```python
from taos.tmq import Consumer

consumer = Consumer({"group.id": "local", "td.connect.ip": "127.0.0.1"})
```

S
sunpeng 已提交
575
##### Subscribe topics
S
sunpeng 已提交
576 577 578 579 580 581 582

The `subscribe` function is used to subscribe to a list of topics.

```python
consumer.subscribe(['topic1', 'topic2'])
```

S
sunpeng 已提交
583
##### Consume
S
sunpeng 已提交
584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600

The `poll` function is used to consume data in tmq. The parameter of the `poll` function is a value of type float representing the timeout in seconds. It returns a `Message` before timing out, or `None` on timing out. You have to handle error messages in response data.

```python
while True:
    res = consumer.poll(1)
    if not res:
        continue
    err = res.error()
    if err is not None:
        raise err
    val = res.value()

    for block in val:
        print(block.fetchall())
```

S
sunpeng 已提交
601
##### assignment
S
sunpeng 已提交
602 603 604 605 606 607 608

The `assignment` function is used to get the assignment of the topic. 

```python
assignments = consumer.assignment()
```

S
sunpeng 已提交
609
##### Seek
S
sunpeng 已提交
610 611 612 613 614 615 616 617

The `seek` function is used to reset the assignment of the topic.

```python
tp = TopicPartition(topic='topic1', partition=0, offset=0)
consumer.seek(tp)
```

S
sunpeng 已提交
618
##### After consuming data
S
sunpeng 已提交
619 620 621 622 623 624 625 626

You should unsubscribe to the topics and close the consumer after consuming.

```python
consumer.unsubscribe()
consumer.close()
```

S
sunpeng 已提交
627
##### Tmq subscription example
S
sunpeng 已提交
628 629 630 631 632

```python
{{#include docs/examples/python/tmq_example.py}}
```

S
sunpeng 已提交
633
##### assignment and seek example
S
sunpeng 已提交
634 635 636 637 638 639 640 641 642 643 644

```python
{{#include docs/examples/python/tmq_assignment_example.py:taos_get_assignment_and_seek_demo}}
```

</TabItem>

<TabItem value="websocket" label="WebSocket connection">

In addition to native connections, the connector also supports subscriptions via websockets.

S
sunpeng 已提交
645
##### Create Consumer
S
sunpeng 已提交
646 647 648 649 650 651 652 653 654

The syntax for creating a consumer is "consumer = consumer = Consumer(conf=configs)". You need to specify that the `td.connect.websocket.scheme` parameter is set to "ws" in the configuration. For more subscription api parameters, please refer to [Data Subscription](../../../develop/tmq/#create-a-consumer).

```python
import taosws

consumer = taosws.(conf={"group.id": "local", "td.connect.websocket.scheme": "ws"})
```

S
sunpeng 已提交
655
##### subscribe topics
S
sunpeng 已提交
656 657 658 659 660 661 662

The `subscribe` function is used to subscribe to a list of topics.

```python
consumer.subscribe(['topic1', 'topic2'])
```

S
sunpeng 已提交
663
##### Consume
S
sunpeng 已提交
664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679

The `poll` function is used to consume data in tmq. The parameter of the `poll` function is a value of type float representing the timeout in seconds. It returns a `Message` before timing out, or `None` on timing out. You have to handle error messages in response data.

```python
while True:
    res = consumer.poll(timeout=1.0)
    if not res:
        continue
    err = res.error()
    if err is not None:
        raise err
    for block in message:
        for row in block:
            print(row)
```

S
sunpeng 已提交
680
##### assignment
S
sunpeng 已提交
681 682 683 684 685 686 687

The `assignment` function is used to get the assignment of the topic. 

```python
assignments = consumer.assignment()
```

S
sunpeng 已提交
688
##### Seek
S
sunpeng 已提交
689 690 691 692 693 694 695

The `seek` function is used to reset the assignment of the topic.

```python
consumer.seek(topic='topic1', partition=0, offset=0)
```

S
sunpeng 已提交
696
##### After consuming data
S
sunpeng 已提交
697 698 699 700 701 702 703 704

You should unsubscribe to the topics and close the consumer after consuming.

```python
consumer.unsubscribe()
consumer.close()
```

S
sunpeng 已提交
705
##### Subscription example
S
sunpeng 已提交
706 707 708 709 710

```python
{{#include docs/examples/python/tmq_websocket_example.py}}
```

S
sunpeng 已提交
711
##### Assignment and seek example
S
sunpeng 已提交
712 713 714 715 716 717 718 719

```python
{{#include docs/examples/python/tmq_websocket_assgnment_example.py:taosws_get_assignment_and_seek_demo}}
```

</TabItem>
</Tabs>

720 721 722 723 724 725 726
### Schemaless Insert

Connector support schemaless insert.

<Tabs defaultValue="list">
<TabItem value="list" label="List Insert">

S
sunpeng 已提交
727
##### Simple insert
728 729 730 731 732

```python
{{#include docs/examples/python/schemaless_insert.py}}
```

S
sunpeng 已提交
733
##### Insert with ttl argument
734 735 736 737 738

```python
{{#include docs/examples/python/schemaless_insert_ttl.py}}
```

S
sunpeng 已提交
739
##### Insert with req_id argument
740 741 742 743 744 745 746 747 748

```python
{{#include docs/examples/python/schemaless_insert_req_id.py}}
```

</TabItem>

<TabItem value="raw" label="Raw Insert">

S
sunpeng 已提交
749
##### Simple insert
750 751 752 753 754

```python
{{#include docs/examples/python/schemaless_insert_raw.py}}
```

S
sunpeng 已提交
755
##### Insert with ttl argument
756 757 758 759 760

```python
{{#include docs/examples/python/schemaless_insert_raw_ttl.py}}
```

S
sunpeng 已提交
761
##### Insert with req_id argument
762 763 764 765 766 767 768 769

```python
{{#include docs/examples/python/schemaless_insert_raw_req_id.py}}
```

</TabItem>
</Tabs>

S
sunpeng 已提交
770 771 772 773 774 775 776
### Parameter Binding

The Python connector provides a parameter binding api for inserting data. Similar to most databases, TDengine currently only supports the question mark `?` to indicate the parameters to be bound.

<Tabs>
<TabItem value="native" label="native connection">

S
sunpeng 已提交
777
##### Create Stmt
S
sunpeng 已提交
778 779 780 781 782 783 784 785 786 787

Call the `statement` method in `Connection` to create the `stmt` for parameter binding.

```
import taos

conn = taos.connect()
stmt = conn.statement("insert into log values(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)")
```

S
sunpeng 已提交
788
##### parameter binding
S
sunpeng 已提交
789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817

Call the `new_multi_binds` function to create the parameter list for parameter bindings.

```
params = new_multi_binds(16)
params[0].timestamp((1626861392589, 1626861392590, 1626861392591))
params[1].bool((True, None, False))
params[2].tinyint([-128, -128, None])  # -128 is tinyint null
params[3].tinyint([0, 127, None])
params[4].smallint([3, None, 2])
params[5].int([3, 4, None])
params[6].bigint([3, 4, None])
params[7].tinyint_unsigned([3, 4, None])
params[8].smallint_unsigned([3, 4, None])
params[9].int_unsigned([3, 4, None])
params[10].bigint_unsigned([3, 4, None])
params[11].float([3, None, 1])
params[12].double([3, None, 1.2])
params[13].binary(["abc", "dddafadfadfadfadfa", None])
params[14].nchar(["涛思数据", None, "a long string with 中文字符"])
params[15].timestamp([None, None, 1626861392591])
```

Call the `bind_param` (for a single row) method or the `bind_param_batch` (for multiple rows) method to set the values.

```
stmt.bind_param_batch(params)
```

S
sunpeng 已提交
818
##### execute sql
S
sunpeng 已提交
819 820 821 822 823 824 825

Call `execute` method to execute sql.

```
stmt.execute()
```

S
sunpeng 已提交
826
##### Close Stmt
S
sunpeng 已提交
827 828 829 830 831

```
stmt.close()
```

S
sunpeng 已提交
832
##### Example
S
sunpeng 已提交
833 834 835 836 837 838 839 840

```python
{{#include docs/examples/python/stmt_example.py}}
```
</TabItem>

<TabItem value="websocket" label="WebSocket connection">

S
sunpeng 已提交
841
##### Create Stmt
S
sunpeng 已提交
842 843 844 845 846 847 848 849 850 851

Call the `statement` method in `Connection` to create the `stmt` for parameter binding.

```
import taosws

conn = taosws.connect('taosws://localhost:6041/test')
stmt = conn.statement()
```

S
sunpeng 已提交
852
##### Prepare sql
S
sunpeng 已提交
853 854 855 856 857 858 859

Call `prepare` method in stmt to prepare sql.

```
stmt.prepare("insert into t1 values (?, ?, ?, ?)")
```

S
sunpeng 已提交
860
##### parameter binding
S
sunpeng 已提交
861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878

Call the `bind_param` method to bind parameters.

```
stmt.bind_param([
    taosws.millis_timestamps_to_column([1686844800000, 1686844801000, 1686844802000, 1686844803000]),
    taosws.ints_to_column([1, 2, 3, 4]),
    taosws.floats_to_column([1.1, 2.2, 3.3, 4.4]),
    taosws.varchar_to_column(['a', 'b', 'c', 'd']),
])
```

Call the `add_batch` method to add parameters to the batch.

```
stmt.add_batch()
```

S
sunpeng 已提交
879
##### execute sql
S
sunpeng 已提交
880 881 882 883 884 885 886

Call `execute` method to execute sql.

```
stmt.execute()
```

S
sunpeng 已提交
887
##### Close Stmt
S
sunpeng 已提交
888 889 890 891 892

```
stmt.close()
```

S
sunpeng 已提交
893
##### Example
S
sunpeng 已提交
894 895 896 897 898 899 900

```python
{{#include docs/examples/python/stmt_websocket_example.py}}
```
</TabItem>
</Tabs>

901 902 903 904
### Other sample programs

| Example program links | Example program content |
| ------------------------------------------------------------------------------------------------------------- | ------------------- ---- |
S
sunpeng 已提交
905 906
| [bind_multi.py](https://github.com/taosdata/taos-connector-python/blob/main/examples/bind-multi.py) | parameter binding, 
bind multiple rows at once |
907 908 909
| [bind_row.py](https://github.com/taosdata/taos-connector-python/blob/main/examples/bind-row.py) | bind_row.py
| [insert_lines.py](https://github.com/taosdata/taos-connector-python/blob/main/examples/insert-lines.py) | InfluxDB line protocol writing |
| [json_tag.py](https://github.com/taosdata/taos-connector-python/blob/main/examples/json-tag.py) | Use JSON type tags |
D
danielclow 已提交
910
| [tmq.py](https://github.com/taosdata/taos-connector-python/blob/main/examples/tmq.py)                         | TMQ subscription              |
911 912 913 914 915

## Other notes 

### About nanoseconds

916
Due to the current imperfection of Python's nanosecond support (see link below), the current implementation returns integers at nanosecond precision instead of the `datetime` type produced by `ms` and `us`, which application developers will need to handle on their own. And it is recommended to use pandas' to_datetime(). The Python Connector may modify the interface in the future if Python officially supports nanoseconds in full.
917 918 919 920 921 922 923 924 925 926 927 928

1. https://stackoverflow.com/questions/10611328/parsing-datetime-strings-containing-nanoseconds
2. https://www.python.org/dev/peps/pep-0564/

## Important Update

[**Release Notes**] (https://github.com/taosdata/taos-connector-python/releases)

## API Reference

- [taos](https://docs.taosdata.com/api/taospy/taos/)
- [taosrest](https://docs.taosdata.com/api/taospy/taosrest)
D
danielclow 已提交
929 930 931 932
  
## Frequently Asked Questions

Welcome to [ask questions or report questions](https://github.com/taosdata/taos-connector-python/issues).