diff --git a/documentation20/cn/04.model/docs.md b/documentation20/cn/04.model/docs.md index 7ecfa6128f1e70d2c8bf33417e8f10b22f9737bd..1a25e4407d0ed77c71040f676656fdc1451e2f81 100644 --- a/documentation20/cn/04.model/docs.md +++ b/documentation20/cn/04.model/docs.md @@ -43,7 +43,7 @@ CREATE STABLE meters (ts timestamp, current float, voltage int, phase float) TAG ## 创建表 TDengine对每个数据采集点需要独立建表。与标准的关系型数据一样，一张表有表名，Schema，但除此之外，还可以带有一到多个标签。创建时，需要使用超级表做模板，同时指定标签的具体值。以表一中的智能电表为例，可以使用如下的SQL命令建表： -```cmd +```mysql CREATE TABLE d1001 USING meters TAGS ("Beijing.Chaoyang", 2); ``` 其中d1001是表名，meters是超级表的表名，后面紧跟标签Location的具体标签值”Beijing.Chaoyang"，标签groupId的具体标签值2。虽然在创建表时，需要指定标签值，但可以事后修改。详细细则请见 [TAOS SQL 的表管理](https://www.taosdata.com/cn/documentation/taos-sql#table) 章节。 @@ -54,10 +54,12 @@ TDengine建议将数据采集点的全局唯一ID作为表名(比如设备序列 **自动建表**：在某些特殊场景中，用户在写数据时并不确定某个数据采集点的表是否存在，此时可在写入数据时使用自动建表语法来创建不存在的表，若该表已存在则不会建立新表。比如： -```cmd +```mysql INSERT INTO d1001 USING METERS TAGS ("Beijng.Chaoyang", 2) VALUES (now, 10.2, 219, 0.32); ``` -上述SQL语句将记录(now, 10.2, 219, 0.32) 插入进表d1001。如果表d1001还未创建，则使用超级表meters做模板自动创建，同时打上标签值“Beijing.Chaoyang", 2。 +上述SQL语句将记录 (now, 10.2, 219, 0.32) 插入表d1001。如果表d1001还未创建，则使用超级表meters做模板自动创建，同时打上标签值“Beijing.Chaoyang", 2。 + +关于自动建表的详细语法请参见 [插入记录时自动建表](https://www.taosdata.com/cn/documentation/taos-sql#auto_create_table) 章节。 ## 多列模型 vs 单列模型 diff --git a/documentation20/cn/12.taos-sql/docs.md b/documentation20/cn/12.taos-sql/docs.md index 7ef6fe20605eaf562c85c9778c8ac9fd40754010..3aee7da7dbdf3120afc2b15da3376cab94c7657d 100644 --- a/documentation20/cn/12.taos-sql/docs.md +++ b/documentation20/cn/12.taos-sql/docs.md @@ -152,6 +152,14 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic ``` 以指定的超级表为模板，指定 tags 的值来创建数据表。 +- **以超级表为模板创建数据表，并指定具体的 tags 列** + + ```mysql + CREATE TABLE [IF NOT EXISTS] tb_name USING stb_name (tag_name1, ...) TAGS (tag_value1, ...); + ``` + 以指定的超级表为模板，指定一部分 tags 列的值来创建数据表。（没被指定的 tags 列会设为空值。） + 说明：从 2.0.17 版本开始支持这种方式。在之前的版本中，不允许指定 tags 列，而必须显式给出所有 tags 列的取值。 + - **批量创建数据表** ```mysql @@ -306,7 +314,7 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic - **插入一条记录，数据对应到指定的列** ```mysql - INSERT INTO tb_name (field1_name, ...) VALUES (field1_value, ...) + INSERT INTO tb_name (field1_name, ...) VALUES (field1_value1, ...); ``` 向表tb_name中插入一条记录，数据对应到指定的列。SQL语句中没有出现的列，数据库将自动填充为NULL。主键（时间戳）不能为NULL。 @@ -340,25 +348,19 @@ TDengine缺省的时间戳是毫秒精度，但通过修改配置参数enableMic 1) 如果时间戳为0，系统将自动使用服务器当前时间作为该记录的时间戳； 2) 允许插入的最老记录的时间戳，是相对于当前服务器时间，减去配置的keep值（数据保留的天数），允许插入的最新记录的时间戳，是相对于当前服务器时间，加上配置的days值（数据文件存储数据的时间跨度，单位为天）。keep和days都是可以在创建数据库时指定的，缺省值分别是3650天和10天。 -**历史记录写入**：可使用IMPORT或者INSERT命令，IMPORT的语法，功能与INSERT完全一样。 - -## 数据查询 +- **插入记录时自动建表** + ```mysql + INSERT INTO tb_name USING stb_name TAGS (tag_value1, ...) VALUES (field_value1, ...); + ``` + 如果用户在写数据时并不确定某个表是否存在，此时可以在写入数据时使用自动建表语法来创建不存在的表，若该表已存在则不会建立新表。自动建表时，要求必须以超级表为模板，并写明数据表的 tags 取值。 -### 查询语法： +- **插入记录时自动建表，并指定具体的 tags 列** + ```mysql + INSERT INTO tb_name USING stb_name (tag_name1, ...) TAGS (tag_value1, ...) VALUES (field_value1, ...); + ``` + 在自动建表时，可以只是指定部分 tags 列的取值，未被指定的 tags 列将取为空值。 -```mysql -SELECT select_expr [, select_expr ...] - FROM {tb_name_list} - [WHERE where_condition] - [INTERVAL (interval_val [, interval_offset])] - [FILL fill_val] - [SLIDING fill_val] - [GROUP BY col_list] - [ORDER BY col_list { DESC | ASC }] - [SLIMIT limit_val [, SOFFSET offset_val]] - [LIMIT limit_val [, OFFSET offset_val]] - [>> export_file] -``` +**历史记录写入**：可使用IMPORT或者INSERT命令，IMPORT的语法，功能与INSERT完全一样。 说明：针对 insert 类型的 SQL 语句，我们采用的流式解析策略，在发现后面的错误之前，前面正确的部分SQL仍会执行。下面的sql中，insert语句是无效的，但是d1001仍会被创建。 @@ -386,6 +388,24 @@ taos> SHOW TABLES; Query OK, 1 row(s) in set (0.001091s) ``` +## 数据查询 + +### 查询语法： + +```mysql +SELECT select_expr [, select_expr ...] + FROM {tb_name_list} + [WHERE where_condition] + [INTERVAL (interval_val [, interval_offset])] + [FILL fill_val] + [SLIDING fill_val] + [GROUP BY col_list] + [ORDER BY col_list { DESC | ASC }] + [SLIMIT limit_val [, SOFFSET offset_val]] + [LIMIT limit_val [, OFFSET offset_val]] + [>> export_file]; +``` + #### SELECT子句 一个选择子句可以是联合查询（UNION）和另一个查询的子查询（SUBQUERY）。 diff --git a/src/connector/python/linux/python2/taos/cinterface.py b/src/connector/python/linux/python2/taos/cinterface.py index c29045c855fc070a17c9dc177628c7075515497f..555cc3435bcbea302b34cbde09772ac5f6fe32b2 100644 --- a/src/connector/python/linux/python2/taos/cinterface.py +++ b/src/connector/python/linux/python2/taos/cinterface.py @@ -67,13 +67,13 @@ def _crow_tinyint_unsigned_to_python( return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] @@ -102,13 +102,13 @@ def _crow_smallint_unsigned_to_python( return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] @@ -130,13 +130,13 @@ def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] @@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] diff --git a/src/connector/python/linux/python2/taos/constants.py b/src/connector/python/linux/python2/taos/constants.py index 3566eb437f7ba45988f37519d2b07af46deb2867..93466f5184a6bf37c2e1c915a00aa5c5e91d1801 100644 --- a/src/connector/python/linux/python2/taos/constants.py +++ b/src/connector/python/linux/python2/taos/constants.py @@ -19,10 +19,10 @@ class FieldType(object): C_BINARY = 8 C_TIMESTAMP = 9 C_NCHAR = 10 - C_TINYINT_UNSIGNED = 12 - C_SMALLINT_UNSIGNED = 13 - C_INT_UNSIGNED = 14 - C_BIGINT_UNSIGNED = 15 + C_TINYINT_UNSIGNED = 11 + C_SMALLINT_UNSIGNED = 12 + C_INT_UNSIGNED = 13 + C_BIGINT_UNSIGNED = 14 # NULL value definition # NOTE: These values should change according to C definition in tsdb.h C_BOOL_NULL = 0x02 diff --git a/src/connector/python/linux/python3/taos/cinterface.py b/src/connector/python/linux/python3/taos/cinterface.py index c29045c855fc070a17c9dc177628c7075515497f..555cc3435bcbea302b34cbde09772ac5f6fe32b2 100644 --- a/src/connector/python/linux/python3/taos/cinterface.py +++ b/src/connector/python/linux/python3/taos/cinterface.py @@ -67,13 +67,13 @@ def _crow_tinyint_unsigned_to_python( return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] @@ -102,13 +102,13 @@ def _crow_smallint_unsigned_to_python( return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] @@ -130,13 +130,13 @@ def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] @@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] diff --git a/src/connector/python/linux/python3/taos/constants.py b/src/connector/python/linux/python3/taos/constants.py index 3566eb437f7ba45988f37519d2b07af46deb2867..93466f5184a6bf37c2e1c915a00aa5c5e91d1801 100644 --- a/src/connector/python/linux/python3/taos/constants.py +++ b/src/connector/python/linux/python3/taos/constants.py @@ -19,10 +19,10 @@ class FieldType(object): C_BINARY = 8 C_TIMESTAMP = 9 C_NCHAR = 10 - C_TINYINT_UNSIGNED = 12 - C_SMALLINT_UNSIGNED = 13 - C_INT_UNSIGNED = 14 - C_BIGINT_UNSIGNED = 15 + C_TINYINT_UNSIGNED = 11 + C_SMALLINT_UNSIGNED = 12 + C_INT_UNSIGNED = 13 + C_BIGINT_UNSIGNED = 14 # NULL value definition # NOTE: These values should change according to C definition in tsdb.h C_BOOL_NULL = 0x02 diff --git a/src/connector/python/osx/python3/taos/cinterface.py b/src/connector/python/osx/python3/taos/cinterface.py index 4db927919b8a372aec231d544f6d2acfb7a0444b..6f56cf0c5e09c14fdc9d1296c80e434ab672ef44 100644 --- a/src/connector/python/osx/python3/taos/cinterface.py +++ b/src/connector/python/osx/python3/taos/cinterface.py @@ -67,13 +67,13 @@ def _crow_tinyint_unsigned_to_python( return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_byte))[ + ctypes.c_ubyte))[ :abs(num_of_rows)]] @@ -102,13 +102,13 @@ def _crow_smallint_unsigned_to_python( return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_short))[ + ctypes.c_ushort))[ :abs(num_of_rows)]] @@ -130,13 +130,13 @@ def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_int))[ + ctypes.c_uint))[ :abs(num_of_rows)]] @@ -162,13 +162,13 @@ def _crow_bigint_unsigned_to_python( return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] else: return [ None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( data, ctypes.POINTER( - ctypes.c_long))[ + ctypes.c_ulong))[ :abs(num_of_rows)]] diff --git a/src/connector/python/osx/python3/taos/constants.py b/src/connector/python/osx/python3/taos/constants.py index 3566eb437f7ba45988f37519d2b07af46deb2867..93466f5184a6bf37c2e1c915a00aa5c5e91d1801 100644 --- a/src/connector/python/osx/python3/taos/constants.py +++ b/src/connector/python/osx/python3/taos/constants.py @@ -19,10 +19,10 @@ class FieldType(object): C_BINARY = 8 C_TIMESTAMP = 9 C_NCHAR = 10 - C_TINYINT_UNSIGNED = 12 - C_SMALLINT_UNSIGNED = 13 - C_INT_UNSIGNED = 14 - C_BIGINT_UNSIGNED = 15 + C_TINYINT_UNSIGNED = 11 + C_SMALLINT_UNSIGNED = 12 + C_INT_UNSIGNED = 13 + C_BIGINT_UNSIGNED = 14 # NULL value definition # NOTE: These values should change according to C definition in tsdb.h C_BOOL_NULL = 0x02 diff --git a/src/connector/python/windows/python2/taos/__init__.py b/src/connector/python/windows/python2/taos/__init__.py index d41216a2dd50929b6f9d35460c74f2d5ae4c8b01..973263573808232e4e71dc0158585624a8e7d2ab 100644 --- a/src/connector/python/windows/python2/taos/__init__.py +++ b/src/connector/python/windows/python2/taos/__init__.py @@ -8,6 +8,7 @@ paramstyle = 'pyformat' __all__ = ['connection', 'cursor'] + def connect(*args, **kwargs): """ Function to return a TDengine connector object diff --git a/src/connector/python/windows/python2/taos/cinterface.py b/src/connector/python/windows/python2/taos/cinterface.py index 14f4f49be8ae719da16dbed7e79631db7ef9689e..d8cdce2ad138c34db5193e3972ba51d46c693254 100644 --- a/src/connector/python/windows/python2/taos/cinterface.py +++ b/src/connector/python/windows/python2/taos/cinterface.py @@ -4,11 +4,14 @@ from .error import * import math import datetime + def _convert_millisecond_to_datetime(milli): - return datetime.datetime.fromtimestamp(milli/1000.0) + return datetime.datetime.fromtimestamp(milli / 1000.0) + def _convert_microsecond_to_datetime(micro): - return datetime.datetime.fromtimestamp(micro/1000000.0) + return datetime.datetime.fromtimestamp(micro / 1000000.0) + def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bool row to python row @@ -18,168 +21,309 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): _timestamp_converter = _convert_microsecond_to_datetime if num_of_rows > 0: - return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)])) + return list(map(_timestamp_converter, ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) else: - return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)])) + return list(map(_timestamp_converter, ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) + def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bool row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_byte))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_bool))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_bool))[ + :abs(num_of_rows)]] + def _crow_tinyint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C tinyint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)]] + else: + return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)]] + + +def _crow_tinyint_unsigned_to_python( + data, + num_of_rows, + nbytes=None, + micro=False): + """Function to convert C tinyint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ubyte))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] - + return [ + None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ubyte))[ + :abs(num_of_rows)]] + + def _crow_smallint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C smallint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_short))[:abs(num_of_rows)]] + return [ + None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_short))[ + :abs(num_of_rows)]] + else: + return [ + None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_short))[ + :abs(num_of_rows)]] + + +def _crow_smallint_unsigned_to_python( + data, num_of_rows, nbytes=None, micro=False): + """Function to convert C smallint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ushort))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_short))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ushort))[ + :abs(num_of_rows)]] + def _crow_int_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C int row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]] + + +def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): + """Function to convert C int row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_uint))[ + :abs(num_of_rows)]] + else: + return [ + None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_uint))[ + :abs(num_of_rows)]] + def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bigint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] + else: + return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] + + +def _crow_bigint_unsigned_to_python( + data, + num_of_rows, + nbytes=None, + micro=False): + """Function to convert C bigint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ulong))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ulong))[ + :abs(num_of_rows)]] + def _crow_float_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C float row to python row """ if num_of_rows > 0: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]] else: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]] + def _crow_double_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C double row to python row """ if num_of_rows > 0: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]] else: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]] + def _crow_binary_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C binary row to python row """ assert(nbytes is not None) if num_of_rows > 0: - return [ None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode('utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + return [None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode( + 'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] else: - return [ None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode('utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + return [None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode( + 'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + def _crow_nchar_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C nchar row to python row """ assert(nbytes is not None) - res=[] + res = [] for i in range(abs(num_of_rows)): try: if num_of_rows >= 0: tmpstr = ctypes.c_char_p(data) - res.append( tmpstr.value.decode() ) + res.append(tmpstr.value.decode()) else: - res.append( (ctypes.cast(data+nbytes*i, ctypes.POINTER(ctypes.c_wchar * (nbytes//4))))[0].value ) + res.append((ctypes.cast(data + nbytes * i, + ctypes.POINTER(ctypes.c_wchar * (nbytes // 4))))[0].value) except ValueError: res.append(None) - return res + return res + def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, micro=False): """Function to convert C binary row to python row """ assert(nbytes is not None) - res=[] + res = [] if num_of_rows > 0: for i in range(abs(num_of_rows)): try: - rbyte=ctypes.cast(data+nbytes*i,ctypes.POINTER(ctypes.c_short))[:1].pop() - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode()[0:rbyte] ) + rbyte = ctypes.cast( + data + nbytes * i, + ctypes.POINTER( + ctypes.c_short))[ + :1].pop() + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()[0:rbyte]) except ValueError: res.append(None) else: for i in range(abs(num_of_rows)): try: - rbyte=ctypes.cast(data+nbytes*i,ctypes.POINTER(ctypes.c_short))[:1].pop() - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode()[0:rbyte] ) + rbyte = ctypes.cast( + data + nbytes * i, + ctypes.POINTER( + ctypes.c_short))[ + :1].pop() + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()[0:rbyte]) except ValueError: res.append(None) return res + def _crow_nchar_to_python_block(data, num_of_rows, nbytes=None, micro=False): """Function to convert C nchar row to python row """ assert(nbytes is not None) - res=[] + res = [] if num_of_rows >= 0: for i in range(abs(num_of_rows)): try: - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode() ) + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()) except ValueError: res.append(None) else: for i in range(abs(num_of_rows)): try: - res.append( (ctypes.cast(data+nbytes*i+2, ctypes.POINTER(ctypes.c_wchar * (nbytes//4))))[0].value ) + res.append((ctypes.cast(data + nbytes * i + 2, + ctypes.POINTER(ctypes.c_wchar * (nbytes // 4))))[0].value) except ValueError: res.append(None) return res + _CONVERT_FUNC = { FieldType.C_BOOL: _crow_bool_to_python, - FieldType.C_TINYINT : _crow_tinyint_to_python, - FieldType.C_SMALLINT : _crow_smallint_to_python, - FieldType.C_INT : _crow_int_to_python, - FieldType.C_BIGINT : _crow_bigint_to_python, - FieldType.C_FLOAT : _crow_float_to_python, - FieldType.C_DOUBLE : _crow_double_to_python, + FieldType.C_TINYINT: _crow_tinyint_to_python, + FieldType.C_SMALLINT: _crow_smallint_to_python, + FieldType.C_INT: _crow_int_to_python, + FieldType.C_BIGINT: _crow_bigint_to_python, + FieldType.C_FLOAT: _crow_float_to_python, + FieldType.C_DOUBLE: _crow_double_to_python, FieldType.C_BINARY: _crow_binary_to_python, - FieldType.C_TIMESTAMP : _crow_timestamp_to_python, - FieldType.C_NCHAR : _crow_nchar_to_python + FieldType.C_TIMESTAMP: _crow_timestamp_to_python, + FieldType.C_NCHAR: _crow_nchar_to_python, + FieldType.C_TINYINT_UNSIGNED: _crow_tinyint_unsigned_to_python, + FieldType.C_SMALLINT_UNSIGNED: _crow_smallint_unsigned_to_python, + FieldType.C_INT_UNSIGNED: _crow_int_unsigned_to_python, + FieldType.C_BIGINT_UNSIGNED: _crow_bigint_unsigned_to_python } _CONVERT_FUNC_BLOCK = { FieldType.C_BOOL: _crow_bool_to_python, - FieldType.C_TINYINT : _crow_tinyint_to_python, - FieldType.C_SMALLINT : _crow_smallint_to_python, - FieldType.C_INT : _crow_int_to_python, - FieldType.C_BIGINT : _crow_bigint_to_python, - FieldType.C_FLOAT : _crow_float_to_python, - FieldType.C_DOUBLE : _crow_double_to_python, + FieldType.C_TINYINT: _crow_tinyint_to_python, + FieldType.C_SMALLINT: _crow_smallint_to_python, + FieldType.C_INT: _crow_int_to_python, + FieldType.C_BIGINT: _crow_bigint_to_python, + FieldType.C_FLOAT: _crow_float_to_python, + FieldType.C_DOUBLE: _crow_double_to_python, FieldType.C_BINARY: _crow_binary_to_python_block, - FieldType.C_TIMESTAMP : _crow_timestamp_to_python, - FieldType.C_NCHAR : _crow_nchar_to_python_block + FieldType.C_TIMESTAMP: _crow_timestamp_to_python, + FieldType.C_NCHAR: _crow_nchar_to_python_block, + FieldType.C_TINYINT_UNSIGNED: _crow_tinyint_unsigned_to_python, + FieldType.C_SMALLINT_UNSIGNED: _crow_smallint_unsigned_to_python, + FieldType.C_INT_UNSIGNED: _crow_int_unsigned_to_python, + FieldType.C_BIGINT_UNSIGNED: _crow_bigint_unsigned_to_python } # Corresponding TAOS_FIELD structure in C + + class TaosField(ctypes.Structure): _fields_ = [('name', ctypes.c_char * 65), ('type', ctypes.c_char), ('bytes', ctypes.c_short)] # C interface class + + class CTaosInterface(object): libtaos = ctypes.windll.LoadLibrary('taos') @@ -216,7 +360,7 @@ class CTaosInterface(object): except AttributeError: raise AttributeError("config is expected as a str") - if config != None: + if config is not None: CTaosInterface.libtaos.taos_options(3, self._config) CTaosInterface.libtaos.taos_init() @@ -227,7 +371,13 @@ class CTaosInterface(object): """ return self._config - def connect(self, host=None, user="root", password="taosdata", db=None, port=0): + def connect( + self, + host=None, + user="root", + password="taosdata", + db=None, + port=0): ''' Function to connect to server @@ -236,7 +386,7 @@ class CTaosInterface(object): # host try: _host = ctypes.c_char_p(host.encode( - "utf-8")) if host != None else ctypes.c_char_p(None) + "utf-8")) if host is not None else ctypes.c_char_p(None) except AttributeError: raise AttributeError("host is expected as a str") @@ -255,7 +405,7 @@ class CTaosInterface(object): # db try: _db = ctypes.c_char_p( - db.encode("utf-8")) if db != None else ctypes.c_char_p(None) + db.encode("utf-8")) if db is not None else ctypes.c_char_p(None) except AttributeError: raise AttributeError("db is expected as a str") @@ -268,11 +418,11 @@ class CTaosInterface(object): connection = ctypes.c_void_p(CTaosInterface.libtaos.taos_connect( _host, _user, _password, _db, _port)) - if connection.value == None: + if connection.value is None: print('connect to TDengine failed') raise ConnectionError("connect to TDengine failed") # sys.exit(1) - #else: + # else: # print('connect to TDengine success') return connection @@ -293,12 +443,13 @@ class CTaosInterface(object): @rtype: 0 on success and -1 on failure ''' try: - return CTaosInterface.libtaos.taos_query(connection, ctypes.c_char_p(sql.encode('utf-8'))) + return CTaosInterface.libtaos.taos_query( + connection, ctypes.c_char_p(sql.encode('utf-8'))) except AttributeError: raise AttributeError("sql is expected as a string") # finally: # CTaosInterface.libtaos.close(connection) - + @staticmethod def affectedRows(result): """The affected rows after runing query @@ -308,7 +459,7 @@ class CTaosInterface(object): @staticmethod def subscribe(connection, restart, topic, sql, interval): """Create a subscription - @restart boolean, + @restart boolean, @sql string, sql statement for data query, must be a 'select' statement. @topic string, name of this subscription """ @@ -360,35 +511,49 @@ class CTaosInterface(object): result, ctypes.byref(pblock)) if num_of_rows == 0: return None, 0 - isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO) + isMicro = (CTaosInterface.libtaos.taos_result_precision( + result) == FieldType.C_TIMESTAMP_MICRO) blocks = [None] * len(fields) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) - fieldLen = [ele for ele in ctypes.cast(fieldL, ctypes.POINTER(ctypes.c_int))[:len(fields)]] + fieldLen = [ + ele for ele in ctypes.cast( + fieldL, ctypes.POINTER( + ctypes.c_int))[ + :len(fields)]] for i in range(len(fields)): data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i] if fields[i]['type'] not in _CONVERT_FUNC_BLOCK: raise DatabaseError("Invalid data type returned from database") - blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']](data, num_of_rows, fieldLen[i], isMicro) + blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']]( + data, num_of_rows, fieldLen[i], isMicro) return blocks, abs(num_of_rows) + @staticmethod def fetchRow(result, fields): pblock = ctypes.c_void_p(0) - pblock = CTaosInterface.libtaos.taos_fetch_row(result) - if pblock : + pblock = CTaosInterface.libtaos.taos_fetch_row(result) + if pblock: num_of_rows = 1 - isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO) + isMicro = (CTaosInterface.libtaos.taos_result_precision( + result) == FieldType.C_TIMESTAMP_MICRO) blocks = [None] * len(fields) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) - fieldLen = [ele for ele in ctypes.cast(fieldL, ctypes.POINTER(ctypes.c_int))[:len(fields)]] + fieldLen = [ + ele for ele in ctypes.cast( + fieldL, ctypes.POINTER( + ctypes.c_int))[ + :len(fields)]] for i in range(len(fields)): data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i] if fields[i]['type'] not in _CONVERT_FUNC: - raise DatabaseError("Invalid data type returned from database") + raise DatabaseError( + "Invalid data type returned from database") if data is None: blocks[i] = [None] else: - blocks[i] = _CONVERT_FUNC[fields[i]['type']](data, num_of_rows, fieldLen[i], isMicro) + blocks[i] = _CONVERT_FUNC[fields[i]['type']]( + data, num_of_rows, fieldLen[i], isMicro) else: return None, 0 return blocks, abs(num_of_rows) diff --git a/src/connector/python/windows/python2/taos/connection.py b/src/connector/python/windows/python2/taos/connection.py index d9576a553b810a975429b2cefc03e5e60f240a88..5729d01c6df8c0e58086726c4001467811e9fee5 100644 --- a/src/connector/python/windows/python2/taos/connection.py +++ b/src/connector/python/windows/python2/taos/connection.py @@ -2,9 +2,11 @@ from .cursor import TDengineCursor from .subscription import TDengineSubscription from .cinterface import CTaosInterface + class TDengineConnection(object): """ TDengine connection object """ + def __init__(self, *args, **kwargs): self._conn = None self._host = None @@ -30,7 +32,7 @@ class TDengineConnection(object): # password if 'password' in kwargs: self._password = kwargs['password'] - + # database if 'database' in kwargs: self._database = kwargs['database'] @@ -44,7 +46,12 @@ class TDengineConnection(object): self._config = kwargs['config'] self._chandle = CTaosInterface(self._config) - self._conn = self._chandle.connect(self._host, self._user, self._password, self._database, self._port) + self._conn = self._chandle.connect( + self._host, + self._user, + self._password, + self._database, + self._port) def close(self): """Close current connection. @@ -56,7 +63,8 @@ class TDengineConnection(object): """ if self._conn is None: return None - sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval) + sub = CTaosInterface.subscribe( + self._conn, restart, topic, sql, interval) return TDengineSubscription(sub) def cursor(self): @@ -81,7 +89,8 @@ class TDengineConnection(object): """ pass + if __name__ == "__main__": conn = TDengineConnection(host='192.168.1.107') conn.close() - print("Hello world") \ No newline at end of file + print("Hello world") diff --git a/src/connector/python/windows/python2/taos/constants.py b/src/connector/python/windows/python2/taos/constants.py index a994bceaf61894ac0bf9a719a574d00a09c584a5..8a8011c3e36c52993e9d03228c2a50e2af6a7c9e 100644 --- a/src/connector/python/windows/python2/taos/constants.py +++ b/src/connector/python/windows/python2/taos/constants.py @@ -3,6 +3,7 @@ from .dbapi import * + class FieldType(object): """TDengine Field Types """ @@ -18,13 +19,21 @@ class FieldType(object): C_BINARY = 8 C_TIMESTAMP = 9 C_NCHAR = 10 + C_TINYINT_UNSIGNED = 11 + C_SMALLINT_UNSIGNED = 12 + C_INT_UNSIGNED = 13 + C_BIGINT_UNSIGNED = 14 # NULL value definition # NOTE: These values should change according to C definition in tsdb.h C_BOOL_NULL = 0x02 C_TINYINT_NULL = -128 + C_TINYINT_UNSIGNED_NULL = 255 C_SMALLINT_NULL = -32768 + C_SMALLINT_UNSIGNED_NULL = 65535 C_INT_NULL = -2147483648 + C_INT_UNSIGNED_NULL = 4294967295 C_BIGINT_NULL = -9223372036854775808 + C_BIGINT_UNSIGNED_NULL = 18446744073709551615 C_FLOAT_NULL = float('nan') C_DOUBLE_NULL = float('nan') C_BINARY_NULL = bytearray([int('0xff', 16)]) diff --git a/src/connector/python/windows/python2/taos/cursor.py b/src/connector/python/windows/python2/taos/cursor.py index 958466985ef050df64ceecdabd994e112716ccf0..0656b6326e173b111eb8293c6e3b76678eccc0e2 100644 --- a/src/connector/python/windows/python2/taos/cursor.py +++ b/src/connector/python/windows/python2/taos/cursor.py @@ -5,6 +5,7 @@ import threading # querySeqNum = 0 + class TDengineCursor(object): """Database cursor which is used to manage the context of a fetch operation. @@ -23,7 +24,7 @@ class TDengineCursor(object): if the cursor has not had an operation invoked via the .execute*() method yet. .rowcount:This read-only attribute specifies the number of rows that the last - .execute*() produced (for DQL statements like SELECT) or affected + .execute*() produced (for DQL statements like SELECT) or affected """ def __init__(self, connection=None): @@ -50,13 +51,14 @@ class TDengineCursor(object): raise OperationalError("Invalid use of fetch iterator") if self._block_rows <= self._block_iter: - block, self._block_rows = CTaosInterface.fetchRow(self._result, self._fields) + block, self._block_rows = CTaosInterface.fetchRow( + self._result, self._fields) if self._block_rows == 0: raise StopIteration self._block = list(map(tuple, zip(*block))) self._block_iter = 0 - data = self._block[self._block_iter] + data = self._block[self._block_iter] self._block_iter += 1 return data @@ -91,7 +93,7 @@ class TDengineCursor(object): """ if self._connection is None: return False - + self._reset_result() self._connection = None @@ -106,19 +108,20 @@ class TDengineCursor(object): if not self._connection: # TODO : change the exception raised here raise ProgrammingError("Cursor is not connected") - + self._reset_result() stmt = operation if params is not None: pass - + self._result = CTaosInterface.query(self._connection._conn, stmt) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno == 0: if CTaosInterface.fieldsCount(self._result) == 0: - self._affected_rows += CTaosInterface.affectedRows(self._result) - return CTaosInterface.affectedRows(self._result ) + self._affected_rows += CTaosInterface.affectedRows( + self._result) + return CTaosInterface.affectedRows(self._result) else: self._fields = CTaosInterface.useResult(self._result) return self._handle_result() @@ -147,17 +150,20 @@ class TDengineCursor(object): buffer = [[] for i in range(len(self._fields))] self._rowcount = 0 while True: - block, num_of_fields = CTaosInterface.fetchRow(self._result, self._fields) + block, num_of_fields = CTaosInterface.fetchRow( + self._result, self._fields) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno != 0: - raise ProgrammingError(CTaosInterface.errStr(self._result), errno) + raise ProgrammingError( + CTaosInterface.errStr( + self._result), errno) if num_of_fields == 0: break self._rowcount += num_of_fields for i in range(len(self._fields)): buffer[i].extend(block[i]) return list(map(tuple, zip(*buffer))) - + def fetchall(self): if self._result is None or self._fields is None: raise OperationalError("Invalid use of fetchall") @@ -165,20 +171,21 @@ class TDengineCursor(object): buffer = [[] for i in range(len(self._fields))] self._rowcount = 0 while True: - block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields) + block, num_of_fields = CTaosInterface.fetchBlock( + self._result, self._fields) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno != 0: - raise ProgrammingError(CTaosInterface.errStr(self._result), errno) - if num_of_fields == 0: break + raise ProgrammingError( + CTaosInterface.errStr( + self._result), errno) + if num_of_fields == 0: + break self._rowcount += num_of_fields for i in range(len(self._fields)): buffer[i].extend(block[i]) - return list(map(tuple, zip(*buffer))) - - def nextset(self): """ """ @@ -209,6 +216,7 @@ class TDengineCursor(object): """ self._description = [] for ele in self._fields: - self._description.append((ele['name'], ele['type'], None, None, None, None, False)) - + self._description.append( + (ele['name'], ele['type'], None, None, None, None, False)) + return self._result diff --git a/src/connector/python/windows/python2/taos/dbapi.py b/src/connector/python/windows/python2/taos/dbapi.py index f1c22bdb512224ac712b78b15ec00207587e65c5..594681ada953abf388e503c23199043cf686e1a3 100644 --- a/src/connector/python/windows/python2/taos/dbapi.py +++ b/src/connector/python/windows/python2/taos/dbapi.py @@ -4,6 +4,7 @@ import time import datetime + class DBAPITypeObject(object): def __init__(self, *values): self.values = values @@ -16,23 +17,28 @@ class DBAPITypeObject(object): else: return -1 + Date = datetime.date Time = datetime.time Timestamp = datetime.datetime + def DataFromTicks(ticks): return Date(*time.localtime(ticks)[:3]) + def TimeFromTicks(ticks): return Time(*time.localtime(ticks)[3:6]) + def TimestampFromTicks(ticks): return Timestamp(*time.localtime(ticks)[:6]) + Binary = bytes # STRING = DBAPITypeObject(*constants.FieldType.get_string_types()) # BINARY = DBAPITypeObject(*constants.FieldType.get_binary_types()) # NUMBER = BAPITypeObject(*constants.FieldType.get_number_types()) # DATETIME = DBAPITypeObject(*constants.FieldType.get_timestamp_types()) -# ROWID = DBAPITypeObject() \ No newline at end of file +# ROWID = DBAPITypeObject() diff --git a/src/connector/python/windows/python2/taos/error.py b/src/connector/python/windows/python2/taos/error.py index 24508a72ed78bb6231187bb6de34d57182e31b22..c584badce8320cd35dc81e8f6b613c56163b1a29 100644 --- a/src/connector/python/windows/python2/taos/error.py +++ b/src/connector/python/windows/python2/taos/error.py @@ -1,35 +1,41 @@ """Python exceptions """ + class Error(Exception): def __init__(self, msg=None, errno=None): self.msg = msg self._full_msg = self.msg self.errno = errno - + def __str__(self): return self._full_msg + class Warning(Exception): """Exception raised for important warnings like data truncations while inserting. """ pass + class InterfaceError(Error): - """Exception raised for errors that are related to the database interface rather than the database itself. + """Exception raised for errors that are related to the database interface rather than the database itself. """ pass + class DatabaseError(Error): - """Exception raised for errors that are related to the database. + """Exception raised for errors that are related to the database. """ pass + class DataError(DatabaseError): """Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range. """ pass + class OperationalError(DatabaseError): """Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer """ @@ -41,17 +47,20 @@ class IntegrityError(DatabaseError): """ pass + class InternalError(DatabaseError): """Exception raised when the database encounters an internal error. """ pass + class ProgrammingError(DatabaseError): """Exception raised for programming errors. """ pass + class NotSupportedError(DatabaseError): """Exception raised in case a method or database API was used which is not supported by the database,. """ - pass \ No newline at end of file + pass diff --git a/src/connector/python/windows/python2/taos/subscription.py b/src/connector/python/windows/python2/taos/subscription.py index d3cf10d5ada578687689b94454378dd543368e3e..270d9de09217fc58a389981a3542698dd1c0428a 100644 --- a/src/connector/python/windows/python2/taos/subscription.py +++ b/src/connector/python/windows/python2/taos/subscription.py @@ -1,32 +1,33 @@ from .cinterface import CTaosInterface from .error import * + class TDengineSubscription(object): """TDengine subscription object """ + def __init__(self, sub): self._sub = sub - def consume(self): """Consume rows of a subscription """ if self._sub is None: raise OperationalError("Invalid use of consume") - + result, fields = CTaosInterface.consume(self._sub) buffer = [[] for i in range(len(fields))] while True: block, num_of_fields = CTaosInterface.fetchBlock(result, fields) - if num_of_fields == 0: break + if num_of_fields == 0: + break for i in range(len(fields)): buffer[i].extend(block[i]) self.fields = fields return list(map(tuple, zip(*buffer))) - - def close(self, keepProgress = True): + def close(self, keepProgress=True): """Close the Subscription. """ if self._sub is None: @@ -38,15 +39,19 @@ class TDengineSubscription(object): if __name__ == '__main__': from .connection import TDengineConnection - conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test") + conn = TDengineConnection( + host="127.0.0.1", + user="root", + password="taosdata", + database="test") # Generate a cursor object to run SQL commands sub = conn.subscribe(True, "test", "select * from meters;", 1000) - for i in range(0,10): + for i in range(0, 10): data = sub.consume() for d in data: print(d) sub.close() - conn.close() \ No newline at end of file + conn.close() diff --git a/src/connector/python/windows/python3/taos/__init__.py b/src/connector/python/windows/python3/taos/__init__.py index a7768a2c6691b506bdaaa9a91a879b6c255ef405..b57e25fd2c320956e46b190d9f0a1139db1cced0 100644 --- a/src/connector/python/windows/python3/taos/__init__.py +++ b/src/connector/python/windows/python3/taos/__init__.py @@ -8,6 +8,7 @@ paramstyle = 'pyformat' __all__ = ['connection', 'cursor'] + def connect(*args, **kwargs): """ Function to return a TDengine connector object diff --git a/src/connector/python/windows/python3/taos/cinterface.py b/src/connector/python/windows/python3/taos/cinterface.py index 42b820ca80feda4a269e5e80f288bd1f5e87adcd..d8cdce2ad138c34db5193e3972ba51d46c693254 100644 --- a/src/connector/python/windows/python3/taos/cinterface.py +++ b/src/connector/python/windows/python3/taos/cinterface.py @@ -4,11 +4,14 @@ from .error import * import math import datetime + def _convert_millisecond_to_datetime(milli): - return datetime.datetime.fromtimestamp(milli/1000.0) + return datetime.datetime.fromtimestamp(milli / 1000.0) + def _convert_microsecond_to_datetime(micro): - return datetime.datetime.fromtimestamp(micro/1000000.0) + return datetime.datetime.fromtimestamp(micro / 1000000.0) + def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bool row to python row @@ -18,170 +21,309 @@ def _crow_timestamp_to_python(data, num_of_rows, nbytes=None, micro=False): _timestamp_converter = _convert_microsecond_to_datetime if num_of_rows > 0: - return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)])) + return list(map(_timestamp_converter, ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) else: - return list(map(_timestamp_converter, ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)])) + return list(map(_timestamp_converter, ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)])) + def _crow_bool_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bool row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_byte))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_bool))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_BOOL_NULL else bool(ele) for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_bool))[ + :abs(num_of_rows)]] + def _crow_tinyint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C tinyint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)] ] - + return [None if ele == FieldType.C_TINYINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_byte))[:abs(num_of_rows)]] + + +def _crow_tinyint_unsigned_to_python( + data, + num_of_rows, + nbytes=None, + micro=False): + """Function to convert C tinyint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ubyte))[ + :abs(num_of_rows)]] + else: + return [ + None if ele == FieldType.C_TINYINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ubyte))[ + :abs(num_of_rows)]] + + def _crow_smallint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C smallint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_short))[:abs(num_of_rows)]] + return [ + None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_short))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_short))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_SMALLINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_short))[ + :abs(num_of_rows)]] + + +def _crow_smallint_unsigned_to_python( + data, num_of_rows, nbytes=None, micro=False): + """Function to convert C smallint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ushort))[ + :abs(num_of_rows)]] + else: + return [ + None if ele == FieldType.C_SMALLINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ushort))[ + :abs(num_of_rows)]] + def _crow_int_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C int row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]] + else: + return [None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)]] + + +def _crow_int_unsigned_to_python(data, num_of_rows, nbytes=None, micro=False): + """Function to convert C int row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_uint))[ + :abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_INT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_int))[:abs(num_of_rows)] ] + return [ + None if ele == FieldType.C_INT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_uint))[ + :abs(num_of_rows)]] + def _crow_bigint_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C bigint row to python row """ if num_of_rows > 0: - return [ None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] else: - return [ None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_longlong))[:abs(num_of_rows)] ] + return [None if ele == FieldType.C_BIGINT_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_long))[:abs(num_of_rows)]] + + +def _crow_bigint_unsigned_to_python( + data, + num_of_rows, + nbytes=None, + micro=False): + """Function to convert C bigint row to python row + """ + if num_of_rows > 0: + return [ + None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ulong))[ + :abs(num_of_rows)]] + else: + return [ + None if ele == FieldType.C_BIGINT_UNSIGNED_NULL else ele for ele in ctypes.cast( + data, ctypes.POINTER( + ctypes.c_ulong))[ + :abs(num_of_rows)]] + def _crow_float_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C float row to python row """ if num_of_rows > 0: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]] else: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_float))[:abs(num_of_rows)]] + def _crow_double_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C double row to python row """ if num_of_rows > 0: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]] else: - return [ None if math.isnan(ele) else ele for ele in ctypes.cast(data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)] ] + return [None if math.isnan(ele) else ele for ele in ctypes.cast( + data, ctypes.POINTER(ctypes.c_double))[:abs(num_of_rows)]] + def _crow_binary_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C binary row to python row """ assert(nbytes is not None) if num_of_rows > 0: - return [ None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode('utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + return [None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode( + 'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] else: - return [ None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode('utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + return [None if ele.value[0:1] == FieldType.C_BINARY_NULL else ele.value.decode( + 'utf-8') for ele in (ctypes.cast(data, ctypes.POINTER(ctypes.c_char * nbytes)))[:abs(num_of_rows)]] + def _crow_nchar_to_python(data, num_of_rows, nbytes=None, micro=False): """Function to convert C nchar row to python row """ assert(nbytes is not None) - res = [] - for i in range(abs(num_of_rows)): try: if num_of_rows >= 0: tmpstr = ctypes.c_char_p(data) - res.append( tmpstr.value.decode() ) + res.append(tmpstr.value.decode()) else: - res.append( (ctypes.cast(data+nbytes*i, ctypes.POINTER(ctypes.c_wchar * (nbytes//4))))[0].value ) + res.append((ctypes.cast(data + nbytes * i, + ctypes.POINTER(ctypes.c_wchar * (nbytes // 4))))[0].value) except ValueError: res.append(None) return res - + + def _crow_binary_to_python_block(data, num_of_rows, nbytes=None, micro=False): """Function to convert C binary row to python row """ assert(nbytes is not None) - res=[] + res = [] if num_of_rows > 0: for i in range(abs(num_of_rows)): try: - rbyte=ctypes.cast(data+nbytes*i,ctypes.POINTER(ctypes.c_short))[:1].pop() - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode()[0:rbyte] ) + rbyte = ctypes.cast( + data + nbytes * i, + ctypes.POINTER( + ctypes.c_short))[ + :1].pop() + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()[0:rbyte]) except ValueError: res.append(None) else: for i in range(abs(num_of_rows)): try: - rbyte=ctypes.cast(data+nbytes*i,ctypes.POINTER(ctypes.c_short))[:1].pop() - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode()[0:rbyte] ) + rbyte = ctypes.cast( + data + nbytes * i, + ctypes.POINTER( + ctypes.c_short))[ + :1].pop() + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()[0:rbyte]) except ValueError: res.append(None) return res + def _crow_nchar_to_python_block(data, num_of_rows, nbytes=None, micro=False): """Function to convert C nchar row to python row """ assert(nbytes is not None) - res=[] + res = [] if num_of_rows >= 0: for i in range(abs(num_of_rows)): try: - tmpstr = ctypes.c_char_p(data+nbytes*i+2) - res.append( tmpstr.value.decode() ) + tmpstr = ctypes.c_char_p(data + nbytes * i + 2) + res.append(tmpstr.value.decode()) except ValueError: res.append(None) else: for i in range(abs(num_of_rows)): try: - res.append( (ctypes.cast(data+nbytes*i+2, ctypes.POINTER(ctypes.c_wchar * (nbytes//4))))[0].value ) + res.append((ctypes.cast(data + nbytes * i + 2, + ctypes.POINTER(ctypes.c_wchar * (nbytes // 4))))[0].value) except ValueError: res.append(None) return res + _CONVERT_FUNC = { FieldType.C_BOOL: _crow_bool_to_python, - FieldType.C_TINYINT : _crow_tinyint_to_python, - FieldType.C_SMALLINT : _crow_smallint_to_python, - FieldType.C_INT : _crow_int_to_python, - FieldType.C_BIGINT : _crow_bigint_to_python, - FieldType.C_FLOAT : _crow_float_to_python, - FieldType.C_DOUBLE : _crow_double_to_python, + FieldType.C_TINYINT: _crow_tinyint_to_python, + FieldType.C_SMALLINT: _crow_smallint_to_python, + FieldType.C_INT: _crow_int_to_python, + FieldType.C_BIGINT: _crow_bigint_to_python, + FieldType.C_FLOAT: _crow_float_to_python, + FieldType.C_DOUBLE: _crow_double_to_python, FieldType.C_BINARY: _crow_binary_to_python, - FieldType.C_TIMESTAMP : _crow_timestamp_to_python, - FieldType.C_NCHAR : _crow_nchar_to_python + FieldType.C_TIMESTAMP: _crow_timestamp_to_python, + FieldType.C_NCHAR: _crow_nchar_to_python, + FieldType.C_TINYINT_UNSIGNED: _crow_tinyint_unsigned_to_python, + FieldType.C_SMALLINT_UNSIGNED: _crow_smallint_unsigned_to_python, + FieldType.C_INT_UNSIGNED: _crow_int_unsigned_to_python, + FieldType.C_BIGINT_UNSIGNED: _crow_bigint_unsigned_to_python } _CONVERT_FUNC_BLOCK = { FieldType.C_BOOL: _crow_bool_to_python, - FieldType.C_TINYINT : _crow_tinyint_to_python, - FieldType.C_SMALLINT : _crow_smallint_to_python, - FieldType.C_INT : _crow_int_to_python, - FieldType.C_BIGINT : _crow_bigint_to_python, - FieldType.C_FLOAT : _crow_float_to_python, - FieldType.C_DOUBLE : _crow_double_to_python, + FieldType.C_TINYINT: _crow_tinyint_to_python, + FieldType.C_SMALLINT: _crow_smallint_to_python, + FieldType.C_INT: _crow_int_to_python, + FieldType.C_BIGINT: _crow_bigint_to_python, + FieldType.C_FLOAT: _crow_float_to_python, + FieldType.C_DOUBLE: _crow_double_to_python, FieldType.C_BINARY: _crow_binary_to_python_block, - FieldType.C_TIMESTAMP : _crow_timestamp_to_python, - FieldType.C_NCHAR : _crow_nchar_to_python_block + FieldType.C_TIMESTAMP: _crow_timestamp_to_python, + FieldType.C_NCHAR: _crow_nchar_to_python_block, + FieldType.C_TINYINT_UNSIGNED: _crow_tinyint_unsigned_to_python, + FieldType.C_SMALLINT_UNSIGNED: _crow_smallint_unsigned_to_python, + FieldType.C_INT_UNSIGNED: _crow_int_unsigned_to_python, + FieldType.C_BIGINT_UNSIGNED: _crow_bigint_unsigned_to_python } # Corresponding TAOS_FIELD structure in C + + class TaosField(ctypes.Structure): _fields_ = [('name', ctypes.c_char * 65), ('type', ctypes.c_char), ('bytes', ctypes.c_short)] # C interface class + + class CTaosInterface(object): libtaos = ctypes.windll.LoadLibrary('taos') @@ -218,7 +360,7 @@ class CTaosInterface(object): except AttributeError: raise AttributeError("config is expected as a str") - if config != None: + if config is not None: CTaosInterface.libtaos.taos_options(3, self._config) CTaosInterface.libtaos.taos_init() @@ -229,7 +371,13 @@ class CTaosInterface(object): """ return self._config - def connect(self, host=None, user="root", password="taosdata", db=None, port=0): + def connect( + self, + host=None, + user="root", + password="taosdata", + db=None, + port=0): ''' Function to connect to server @@ -238,7 +386,7 @@ class CTaosInterface(object): # host try: _host = ctypes.c_char_p(host.encode( - "utf-8")) if host != None else ctypes.c_char_p(None) + "utf-8")) if host is not None else ctypes.c_char_p(None) except AttributeError: raise AttributeError("host is expected as a str") @@ -257,7 +405,7 @@ class CTaosInterface(object): # db try: _db = ctypes.c_char_p( - db.encode("utf-8")) if db != None else ctypes.c_char_p(None) + db.encode("utf-8")) if db is not None else ctypes.c_char_p(None) except AttributeError: raise AttributeError("db is expected as a str") @@ -270,11 +418,11 @@ class CTaosInterface(object): connection = ctypes.c_void_p(CTaosInterface.libtaos.taos_connect( _host, _user, _password, _db, _port)) - if connection.value == None: + if connection.value is None: print('connect to TDengine failed') raise ConnectionError("connect to TDengine failed") # sys.exit(1) - #else: + # else: # print('connect to TDengine success') return connection @@ -295,7 +443,8 @@ class CTaosInterface(object): @rtype: 0 on success and -1 on failure ''' try: - return CTaosInterface.libtaos.taos_query(connection, ctypes.c_char_p(sql.encode('utf-8'))) + return CTaosInterface.libtaos.taos_query( + connection, ctypes.c_char_p(sql.encode('utf-8'))) except AttributeError: raise AttributeError("sql is expected as a string") # finally: @@ -310,7 +459,7 @@ class CTaosInterface(object): @staticmethod def subscribe(connection, restart, topic, sql, interval): """Create a subscription - @restart boolean, + @restart boolean, @sql string, sql statement for data query, must be a 'select' statement. @topic string, name of this subscription """ @@ -362,35 +511,49 @@ class CTaosInterface(object): result, ctypes.byref(pblock)) if num_of_rows == 0: return None, 0 - isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO) + isMicro = (CTaosInterface.libtaos.taos_result_precision( + result) == FieldType.C_TIMESTAMP_MICRO) blocks = [None] * len(fields) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) - fieldLen = [ele for ele in ctypes.cast(fieldL, ctypes.POINTER(ctypes.c_int))[:len(fields)]] + fieldLen = [ + ele for ele in ctypes.cast( + fieldL, ctypes.POINTER( + ctypes.c_int))[ + :len(fields)]] for i in range(len(fields)): data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i] if fields[i]['type'] not in _CONVERT_FUNC_BLOCK: raise DatabaseError("Invalid data type returned from database") - blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']](data, num_of_rows, fieldLen[i], isMicro) + blocks[i] = _CONVERT_FUNC_BLOCK[fields[i]['type']]( + data, num_of_rows, fieldLen[i], isMicro) return blocks, abs(num_of_rows) + @staticmethod def fetchRow(result, fields): pblock = ctypes.c_void_p(0) - pblock = CTaosInterface.libtaos.taos_fetch_row(result) - if pblock : + pblock = CTaosInterface.libtaos.taos_fetch_row(result) + if pblock: num_of_rows = 1 - isMicro = (CTaosInterface.libtaos.taos_result_precision(result) == FieldType.C_TIMESTAMP_MICRO) + isMicro = (CTaosInterface.libtaos.taos_result_precision( + result) == FieldType.C_TIMESTAMP_MICRO) blocks = [None] * len(fields) fieldL = CTaosInterface.libtaos.taos_fetch_lengths(result) - fieldLen = [ele for ele in ctypes.cast(fieldL, ctypes.POINTER(ctypes.c_int))[:len(fields)]] + fieldLen = [ + ele for ele in ctypes.cast( + fieldL, ctypes.POINTER( + ctypes.c_int))[ + :len(fields)]] for i in range(len(fields)): data = ctypes.cast(pblock, ctypes.POINTER(ctypes.c_void_p))[i] if fields[i]['type'] not in _CONVERT_FUNC: - raise DatabaseError("Invalid data type returned from database") + raise DatabaseError( + "Invalid data type returned from database") if data is None: blocks[i] = [None] else: - blocks[i] = _CONVERT_FUNC[fields[i]['type']](data, num_of_rows, fieldLen[i], isMicro) + blocks[i] = _CONVERT_FUNC[fields[i]['type']]( + data, num_of_rows, fieldLen[i], isMicro) else: return None, 0 return blocks, abs(num_of_rows) @@ -476,4 +639,4 @@ if __name__ == '__main__': print(data) cinter.freeResult(result) - cinter.close(conn) \ No newline at end of file + cinter.close(conn) diff --git a/src/connector/python/windows/python3/taos/connection.py b/src/connector/python/windows/python3/taos/connection.py index d9576a553b810a975429b2cefc03e5e60f240a88..5729d01c6df8c0e58086726c4001467811e9fee5 100644 --- a/src/connector/python/windows/python3/taos/connection.py +++ b/src/connector/python/windows/python3/taos/connection.py @@ -2,9 +2,11 @@ from .cursor import TDengineCursor from .subscription import TDengineSubscription from .cinterface import CTaosInterface + class TDengineConnection(object): """ TDengine connection object """ + def __init__(self, *args, **kwargs): self._conn = None self._host = None @@ -30,7 +32,7 @@ class TDengineConnection(object): # password if 'password' in kwargs: self._password = kwargs['password'] - + # database if 'database' in kwargs: self._database = kwargs['database'] @@ -44,7 +46,12 @@ class TDengineConnection(object): self._config = kwargs['config'] self._chandle = CTaosInterface(self._config) - self._conn = self._chandle.connect(self._host, self._user, self._password, self._database, self._port) + self._conn = self._chandle.connect( + self._host, + self._user, + self._password, + self._database, + self._port) def close(self): """Close current connection. @@ -56,7 +63,8 @@ class TDengineConnection(object): """ if self._conn is None: return None - sub = CTaosInterface.subscribe(self._conn, restart, topic, sql, interval) + sub = CTaosInterface.subscribe( + self._conn, restart, topic, sql, interval) return TDengineSubscription(sub) def cursor(self): @@ -81,7 +89,8 @@ class TDengineConnection(object): """ pass + if __name__ == "__main__": conn = TDengineConnection(host='192.168.1.107') conn.close() - print("Hello world") \ No newline at end of file + print("Hello world") diff --git a/src/connector/python/windows/python3/taos/constants.py b/src/connector/python/windows/python3/taos/constants.py index def2bbc0a8c063e85214a634b60c5db7a5fd1259..49fc17b2fb98a6684e74e4a044651fdc6237518e 100644 --- a/src/connector/python/windows/python3/taos/constants.py +++ b/src/connector/python/windows/python3/taos/constants.py @@ -3,6 +3,7 @@ from .dbapi import * + class FieldType(object): """TDengine Field Types """ @@ -18,13 +19,21 @@ class FieldType(object): C_BINARY = 8 C_TIMESTAMP = 9 C_NCHAR = 10 + C_TINYINT_UNSIGNED = 11 + C_SMALLINT_UNSIGNED = 12 + C_INT_UNSIGNED = 13 + C_BIGINT_UNSIGNED = 14 # NULL value definition # NOTE: These values should change according to C definition in tsdb.h C_BOOL_NULL = 0x02 C_TINYINT_NULL = -128 + C_TINYINT_UNSIGNED_NULL = 255 C_SMALLINT_NULL = -32768 + C_SMALLINT_UNSIGNED_NULL = 65535 C_INT_NULL = -2147483648 + C_INT_UNSIGNED_NULL = 4294967295 C_BIGINT_NULL = -9223372036854775808 + C_BIGINT_UNSIGNED_NULL = 18446744073709551615 C_FLOAT_NULL = float('nan') C_DOUBLE_NULL = float('nan') C_BINARY_NULL = bytearray([int('0xff', 16)]) diff --git a/src/connector/python/windows/python3/taos/cursor.py b/src/connector/python/windows/python3/taos/cursor.py index bbac1b1dd5158a5f074325a72f15f6993e97e9da..769cb7cf0f61fe850c16315bf552162f33536502 100644 --- a/src/connector/python/windows/python3/taos/cursor.py +++ b/src/connector/python/windows/python3/taos/cursor.py @@ -24,7 +24,7 @@ class TDengineCursor(object): if the cursor has not had an operation invoked via the .execute*() method yet. .rowcount:This read-only attribute specifies the number of rows that the last - .execute*() produced (for DQL statements like SELECT) or affected + .execute*() produced (for DQL statements like SELECT) or affected """ def __init__(self, connection=None): @@ -51,13 +51,14 @@ class TDengineCursor(object): raise OperationalError("Invalid use of fetch iterator") if self._block_rows <= self._block_iter: - block, self._block_rows = CTaosInterface.fetchRow(self._result, self._fields) + block, self._block_rows = CTaosInterface.fetchRow( + self._result, self._fields) if self._block_rows == 0: raise StopIteration self._block = list(map(tuple, zip(*block))) self._block_iter = 0 - data = self._block[self._block_iter] + data = self._block[self._block_iter] self._block_iter += 1 return data @@ -92,7 +93,7 @@ class TDengineCursor(object): """ if self._connection is None: return False - + self._reset_result() self._connection = None @@ -107,24 +108,25 @@ class TDengineCursor(object): if not self._connection: # TODO : change the exception raised here raise ProgrammingError("Cursor is not connected") - + self._reset_result() stmt = operation if params is not None: pass - + self._result = CTaosInterface.query(self._connection._conn, stmt) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno == 0: if CTaosInterface.fieldsCount(self._result) == 0: - self._affected_rows += CTaosInterface.affectedRows(self._result ) - return CTaosInterface.affectedRows(self._result ) + self._affected_rows += CTaosInterface.affectedRows( + self._result) + return CTaosInterface.affectedRows(self._result) else: - self._fields = CTaosInterface.useResult(self._result ) + self._fields = CTaosInterface.useResult(self._result) return self._handle_result() else: - raise ProgrammingError(CTaosInterface.errStr(self._result ), errno) + raise ProgrammingError(CTaosInterface.errStr(self._result), errno) def executemany(self, operation, seq_of_parameters): """Prepare a database operation (query or command) and then execute it against all parameter sequences or mappings found in the sequence seq_of_parameters. @@ -148,10 +150,13 @@ class TDengineCursor(object): buffer = [[] for i in range(len(self._fields))] self._rowcount = 0 while True: - block, num_of_fields = CTaosInterface.fetchRow(self._result, self._fields) + block, num_of_fields = CTaosInterface.fetchRow( + self._result, self._fields) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno != 0: - raise ProgrammingError(CTaosInterface.errStr(self._result), errno) + raise ProgrammingError( + CTaosInterface.errStr( + self._result), errno) if num_of_fields == 0: break self._rowcount += num_of_fields @@ -166,20 +171,21 @@ class TDengineCursor(object): buffer = [[] for i in range(len(self._fields))] self._rowcount = 0 while True: - block, num_of_fields = CTaosInterface.fetchBlock(self._result, self._fields) + block, num_of_fields = CTaosInterface.fetchBlock( + self._result, self._fields) errno = CTaosInterface.libtaos.taos_errno(self._result) if errno != 0: - raise ProgrammingError(CTaosInterface.errStr(self._result), errno) - if num_of_fields == 0: break + raise ProgrammingError( + CTaosInterface.errStr( + self._result), errno) + if num_of_fields == 0: + break self._rowcount += num_of_fields for i in range(len(self._fields)): buffer[i].extend(block[i]) - return list(map(tuple, zip(*buffer))) - - def nextset(self): """ """ @@ -204,12 +210,13 @@ class TDengineCursor(object): self._block_rows = -1 self._block_iter = 0 self._affected_rows = 0 - + def _handle_result(self): """Handle the return result from query. """ self._description = [] for ele in self._fields: - self._description.append((ele['name'], ele['type'], None, None, None, None, False)) - + self._description.append( + (ele['name'], ele['type'], None, None, None, None, False)) + return self._result diff --git a/src/connector/python/windows/python3/taos/dbapi.py b/src/connector/python/windows/python3/taos/dbapi.py index 9b1cb1321c14619782c801e9381010f1f67fbc2e..a29621f7a3594a618b59b30bdc96197c4222a619 100644 --- a/src/connector/python/windows/python3/taos/dbapi.py +++ b/src/connector/python/windows/python3/taos/dbapi.py @@ -4,6 +4,7 @@ import time import datetime + class DBAPITypeObject(object): def __init__(self, *values): self.values = values @@ -16,23 +17,28 @@ class DBAPITypeObject(object): else: return -1 + Date = datetime.date Time = datetime.time Timestamp = datetime.datetime + def DataFromTicks(ticks): return Date(*time.localtime(ticks)[:3]) + def TimeFromTicks(ticks): return Time(*time.localtime(ticks)[3:6]) + def TimestampFromTicks(ticks): return Timestamp(*time.localtime(ticks)[:6]) + Binary = bytes # STRING = DBAPITypeObject(*constants.FieldType.get_string_types()) # BINARY = DBAPITypeObject(*constants.FieldType.get_binary_types()) # NUMBER = BAPITypeObject(*constants.FieldType.get_number_types()) # DATETIME = DBAPITypeObject(*constants.FieldType.get_timestamp_types()) -# ROWID = DBAPITypeObject() \ No newline at end of file +# ROWID = DBAPITypeObject() diff --git a/src/connector/python/windows/python3/taos/error.py b/src/connector/python/windows/python3/taos/error.py index ccc0e61d84597a7cae9b360b7fa2434b9bc47401..238b293a0b609570e7b5d536648c6ada3ca2f209 100644 --- a/src/connector/python/windows/python3/taos/error.py +++ b/src/connector/python/windows/python3/taos/error.py @@ -1,35 +1,41 @@ """Python exceptions """ + class Error(Exception): def __init__(self, msg=None, errno=None): self.msg = msg self._full_msg = self.msg self.errno = errno - + def __str__(self): return self._full_msg + class Warning(Exception): """Exception raised for important warnings like data truncations while inserting. """ pass + class InterfaceError(Error): - """Exception raised for errors that are related to the database interface rather than the database itself. + """Exception raised for errors that are related to the database interface rather than the database itself. """ pass + class DatabaseError(Error): - """Exception raised for errors that are related to the database. + """Exception raised for errors that are related to the database. """ pass + class DataError(DatabaseError): """Exception raised for errors that are due to problems with the processed data like division by zero, numeric value out of range. """ pass + class OperationalError(DatabaseError): """Exception raised for errors that are related to the database's operation and not necessarily under the control of the programmer """ @@ -41,17 +47,20 @@ class IntegrityError(DatabaseError): """ pass + class InternalError(DatabaseError): """Exception raised when the database encounters an internal error. """ pass + class ProgrammingError(DatabaseError): """Exception raised for programming errors. """ pass + class NotSupportedError(DatabaseError): """Exception raised in case a method or database API was used which is not supported by the database,. """ - pass \ No newline at end of file + pass diff --git a/src/connector/python/windows/python3/taos/subscription.py b/src/connector/python/windows/python3/taos/subscription.py index d3cf10d5ada578687689b94454378dd543368e3e..270d9de09217fc58a389981a3542698dd1c0428a 100644 --- a/src/connector/python/windows/python3/taos/subscription.py +++ b/src/connector/python/windows/python3/taos/subscription.py @@ -1,32 +1,33 @@ from .cinterface import CTaosInterface from .error import * + class TDengineSubscription(object): """TDengine subscription object """ + def __init__(self, sub): self._sub = sub - def consume(self): """Consume rows of a subscription """ if self._sub is None: raise OperationalError("Invalid use of consume") - + result, fields = CTaosInterface.consume(self._sub) buffer = [[] for i in range(len(fields))] while True: block, num_of_fields = CTaosInterface.fetchBlock(result, fields) - if num_of_fields == 0: break + if num_of_fields == 0: + break for i in range(len(fields)): buffer[i].extend(block[i]) self.fields = fields return list(map(tuple, zip(*buffer))) - - def close(self, keepProgress = True): + def close(self, keepProgress=True): """Close the Subscription. """ if self._sub is None: @@ -38,15 +39,19 @@ class TDengineSubscription(object): if __name__ == '__main__': from .connection import TDengineConnection - conn = TDengineConnection(host="127.0.0.1", user="root", password="taosdata", database="test") + conn = TDengineConnection( + host="127.0.0.1", + user="root", + password="taosdata", + database="test") # Generate a cursor object to run SQL commands sub = conn.subscribe(True, "test", "select * from meters;", 1000) - for i in range(0,10): + for i in range(0, 10): data = sub.consume() for d in data: print(d) sub.close() - conn.close() \ No newline at end of file + conn.close() diff --git a/tests/pytest/insert/basic_unsigned.py b/tests/pytest/insert/basic_unsigned.py index 7b974c8d346e13e674f1fbfd1caf533e37b17578..993e58ce877284f73c136bea864c59f5166916d9 100644 --- a/tests/pytest/insert/basic_unsigned.py +++ b/tests/pytest/insert/basic_unsigned.py @@ -26,7 +26,7 @@ class TDTestCase: tdSql.prepare() ret = tdSql.execute( - 'create table tb (ts timestamp, speed int unsigned)') + 'create table tb (ts timestamp, speed tinyint unsigned)') insertRows = 10 tdLog.info("insert %d rows" % (insertRows)) @@ -38,11 +38,11 @@ class TDTestCase: tdLog.info("insert earlier data") tdSql.execute('insert into tb values (now - 5m , 10)') tdSql.execute('insert into tb values (now - 6m , 10)') - tdSql.execute('insert into tb values (now - 7m , 10)') - tdSql.execute('insert into tb values (now - 8m , 4294967294)') + tdSql.execute('insert into tb values (now - 7m , NULL)') + tdSql.execute('insert into tb values (now - 8m , 254)') tdSql.error('insert into tb values (now - 9m, -1)') - tdSql.error('insert into tb values (now - 9m, 4294967295)') + tdSql.error('insert into tb values (now - 9m, 255)') tdSql.query("select * from tb") tdSql.checkRows(insertRows + 4) diff --git a/tests/pytest/pytest_1.sh b/tests/pytest/pytest_1.sh index 729319c23174df888528baa0b8f9f62d3a6f35b7..e6638cbb171ab3d8dc38c13f5767ff2f09c522bb 100755 --- a/tests/pytest/pytest_1.sh +++ b/tests/pytest/pytest_1.sh @@ -16,7 +16,7 @@ python3 ./test.py -f insert/nchar.py python3 ./test.py -f insert/nchar-unicode.py python3 ./test.py -f insert/multi.py python3 ./test.py -f insert/randomNullCommit.py -#python3 insert/retentionpolicy.py +python3 insert/retentionpolicy.py python3 ./test.py -f insert/alterTableAndInsert.py python3 ./test.py -f insert/insertIntoTwoTables.py python3 ./test.py -f insert/before_1970.py diff --git a/tests/pytest/pytest_4.sh b/tests/pytest/pytest_4.sh index 6e201d6885bc481dfd32904c35c0b1ec317cfed6..a68e6a1fefeb09af5e0c7f934de14f0781ae540c 100755 --- a/tests/pytest/pytest_4.sh +++ b/tests/pytest/pytest_4.sh @@ -23,4 +23,11 @@ python3 ./test.py -f functions/function_sum.py -r 1 python3 ./test.py -f functions/function_top.py -r 1 python3 ./test.py -f functions/function_twa.py -r 1 python3 ./test.py -f functions/function_twa_test2.py -python3 ./test.py -f functions/function_stddev_td2555.pyhao \ No newline at end of file +python3 ./test.py -f functions/function_stddev_td2555.py +python3 ./test.py -f insert/metadataUpdate.py +python3 ./test.py -f tools/taosdemoTest2.py +python3 ./test.py -f query/last_cache.py +python3 ./test.py -f query/last_row_cache.py +python3 ./test.py -f account/account_create.py +python3 ./test.py -f alter/alter_table.py +python3 ./test.py -f query/queryGroupbySort.py \ No newline at end of file diff --git a/tests/script/jenkins/basic_4.txt b/tests/script/jenkins/basic_4.txt index 7735c5b3d5e46bd1672576da422a6cd117ebc4fc..5a7d23df719f737d0c5a3c85acfd3875b87872ab 100644 --- a/tests/script/jenkins/basic_4.txt +++ b/tests/script/jenkins/basic_4.txt @@ -37,4 +37,10 @@ ./test.sh -f general/stable/values.sim ./test.sh -f general/stable/vnode3.sim -./test.sh -f unique/column/replica3.sim \ No newline at end of file +./test.sh -f unique/column/replica3.sim +./test.sh -f issue/TD-2713.sim +./test.sh -f general/parser/select_distinct_tag.sim +./test.sh -f unique/mnode/mgmt30.sim +./test.sh -f issue/TD-2677.sim +./test.sh -f issue/TD-2680.sim +./test.sh -f unique/dnode/lossdata.sim \ No newline at end of file