* Using column instead of AST to store scalar subquery results for better performance. Setting `enable_scalar_subquery_optimization` was added in 19.17 and it was enabled by default. It leads to errors like [this](https://github.com/ClickHouse/ClickHouse/issues/7851) during upgrade to 19.17.2 or 19.17.3 from previous versions. This setting was disabled by default in 19.17.4, to make possible upgrading from 19.16 and older versions without errors. [#7392](https://github.com/ClickHouse/ClickHouse/pull/7392)([Amos Bird](https://github.com/amosbird))
### New Feature
* Add the ability to create dictionaries with DDL queries. [#7360](https://github.com/ClickHouse/ClickHouse/pull/7360)([alesapin](https://github.com/alesapin))
* Authentication in S3 table function and storage. Now we have complete support for S3 import/export. [#7623](https://github.com/ClickHouse/ClickHouse/pull/7623)([Vladimir Chebotarev](https://github.com/excitoon))
* Make `bloom_filter` type of index supporting `LowCardinality` and `Nullable`[#7363](https://github.com/ClickHouse/ClickHouse/issues/7363)[#7561](https://github.com/ClickHouse/ClickHouse/pull/7561) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
* Add function `isValidJSON` to check that passed string is a valid json. [#5910](https://github.com/ClickHouse/ClickHouse/issues/5910)[#7293](https://github.com/ClickHouse/ClickHouse/pull/7293) ([Vdimir](https://github.com/Vdimir))
* Implement `arrayCompact` function [#7328](https://github.com/ClickHouse/ClickHouse/pull/7328)([Memo](https://github.com/Joeywzr))
* Created function `hex` for Decimal numbers. It works like `hex(reinterpretAsString())`, but doesn't delete last zero bytes. [#7355](https://github.com/ClickHouse/ClickHouse/pull/7355)([Mikhail Korotov](https://github.com/millb))
* Add the ability to create dictionaries with DDL queries. [#7360](https://github.com/ClickHouse/ClickHouse/pull/7360)([alesapin](https://github.com/alesapin))
* Add `arrayFill` and `arrayReverseFill` functions, which replace elements by other elements in front/back of them in the array. [#7380](https://github.com/ClickHouse/ClickHouse/pull/7380)([hcz](https://github.com/hczhcz))
* Up precision of `avg` aggregate function result to max of type [#7446](https://github.com/ClickHouse/ClickHouse/pull/7446)([Andrey Konyaev](https://github.com/akonyaev90))
* Up precision of `avg` aggregate function result to max of `Decimal`type [#7446](https://github.com/ClickHouse/ClickHouse/pull/7446)([Andrey Konyaev](https://github.com/akonyaev90))
* Add `CRC32IEEE()`/`CRC64()` support [#7480](https://github.com/ClickHouse/ClickHouse/pull/7480)([Azat Khuzhin](https://github.com/azat))
* Implement `char` function similar to one in [mysql](https://dev.mysql.com/doc/refman/8.0/en/string-functions.html#function_char)[#7486](https://github.com/ClickHouse/ClickHouse/pull/7486) ([sundyli](https://github.com/sundy-li))
* Add `bitmapTransform` function. It transforms an array of values in a bitmap to another array of values, the result is a new bitmap [#7598](https://github.com/ClickHouse/ClickHouse/pull/7598)([Zhichang Yu](https://github.com/yuzhichang))
* Add `_shard_num` virtual column for the Distributed engine [#7624](https://github.com/ClickHouse/ClickHouse/pull/7624)([Azat Khuzhin](https://github.com/azat))
* Implemented `javaHashUTF16LE()` function [#7651](https://github.com/ClickHouse/ClickHouse/pull/7651)([achimbab](https://github.com/achimbab))
* Authentication in S3 table function and storage [#7623](https://github.com/ClickHouse/ClickHouse/pull/7623)([Vladimir Chebotarev](https://github.com/excitoon))
* Make `bloom_filter` type of index supporting `LowCardinality` and `Nullable`[#7363](https://github.com/ClickHouse/ClickHouse/issues/7363)[#7561](https://github.com/ClickHouse/ClickHouse/pull/7561) ([Nikolai Kochetov](https://github.com/KochetovNicolai))
* Add `_shard_num` virtual column for the Distributed engine [#7624](https://github.com/ClickHouse/ClickHouse/pull/7624)([Azat Khuzhin](https://github.com/azat))
### Experimental Feature
* Support for processors (new query execution pipeline) in `MergeTree`. [#7181](https://github.com/ClickHouse/ClickHouse/pull/7181)([Nikolai Kochetov](https://github.com/KochetovNicolai))
### Bug Fix
* Fix incorrect float parsing in `Values`[#7817](https://github.com/ClickHouse/ClickHouse/issues/7817)[#7870](https://github.com/ClickHouse/ClickHouse/pull/7870) ([tavplubix](https://github.com/tavplubix))
* Fix rare deadlock which can happen when trace_log is enabled. [#7838](https://github.com/ClickHouse/ClickHouse/pull/7838)([filimonov](https://github.com/filimonov))
* Prevent message duplication when producing Kafka table has any MVs selecting from it [#7265](https://github.com/ClickHouse/ClickHouse/pull/7265)([Ivan](https://github.com/abyss7))
* Support for `Array(LowCardinality(Nullable(String)))` in `IN`. Resolves [#7364](https://github.com/ClickHouse/ClickHouse/issues/7364) ... [#7366](https://github.com/ClickHouse/ClickHouse/pull/7366)([achimbab](https://github.com/achimbab))
* Fix `INSERT` into Distributed with `MATERIALIZED` columns [#7377](https://github.com/ClickHouse/ClickHouse/pull/7377)([Azat Khuzhin](https://github.com/azat))
*`Nullable` columns recognized and NULL-values handled correctly by ODBC-bridge [#7402](https://github.com/ClickHouse/ClickHouse/pull/7402)([Vasily Nemkov](https://github.com/Enmk))
* Support for `Array(LowCardinality(Nullable(String)))` in `IN`. Resolves [#7364](https://github.com/ClickHouse/ClickHouse/issues/7364)[#7366](https://github.com/ClickHouse/ClickHouse/pull/7366) ([achimbab](https://github.com/achimbab))
* Add handling of `SQL_TINYINT` and `SQL_BIGINT`, and fix handling of `SQL_FLOAT` data source types in ODBC Bridge. [#7491](https://github.com/ClickHouse/ClickHouse/pull/7491)([Denis Glazachev](https://github.com/traceon))
* Fix aggregation (`avg` and quantiles) over empty decimal columns [#7431](https://github.com/ClickHouse/ClickHouse/pull/7431)([Andrey Konyaev](https://github.com/akonyaev90))
* Fix `INSERT` into Distributed with `MATERIALIZED` columns [#7377](https://github.com/ClickHouse/ClickHouse/pull/7377)([Azat Khuzhin](https://github.com/azat))
* Make `MOVE PARTITION` work if some parts of partition are already on destination disk or volume [#7434](https://github.com/ClickHouse/ClickHouse/pull/7434)([Vladimir Chebotarev](https://github.com/excitoon))
* Add handling of `SQL_TINYINT` and `SQL_BIGINT`, and fix handling of `SQL_FLOAT` data source types in ODBC Bridge. [#7491](https://github.com/ClickHouse/ClickHouse/pull/7491)([Denis Glazachev](https://github.com/traceon))
* Fix parsing of queries like `CREATE TABLE ... AS (SELECT (1, 2))`[#7542](https://github.com/ClickHouse/ClickHouse/pull/7542)([hcz](https://github.com/hczhcz))
* Fixed bug with hardlinks failing to be created during mutations in `ReplicatedMergeTree` in multi-disk configurations. [#7558](https://github.com/ClickHouse/ClickHouse/pull/7558)([Vladimir Chebotarev](https://github.com/excitoon))
* Fixed a bug with a mutation on a MergeTree when whole part remains unchanged and best space is being found on another disk [#7602](https://github.com/ClickHouse/ClickHouse/pull/7602)([Vladimir Chebotarev](https://github.com/excitoon))
* Fixed bug with `keep_free_space_ratio` not being read from disks configuration [#7645](https://github.com/ClickHouse/ClickHouse/pull/7645)([Vladimir Chebotarev](https://github.com/excitoon))
* Fix bug with table contains only `Tuple` columns or columns with complex paths. Fixes [7541](https://github.com/ClickHouse/ClickHouse/issues/7541). [#7545](https://github.com/ClickHouse/ClickHouse/pull/7545)([alesapin](https://github.com/alesapin))
* Do not account memory for Buffer engine in max_memory_usage limit [#7552](https://github.com/ClickHouse/ClickHouse/pull/7552)([Azat Khuzhin](https://github.com/azat))
* Fixed bug with hardlinks failing to be created during mutations in `ReplicatedMergeTree` in multi-disk configurations. [#7558](https://github.com/ClickHouse/ClickHouse/pull/7558)([Vladimir Chebotarev](https://github.com/excitoon))
* Fix final mark usage in `MergeTree` tables ordered by `tuple()`. In rare cases it could lead to `Can't adjust last granule` error while select. [#7639](https://github.com/ClickHouse/ClickHouse/pull/7639)([Anton Popov](https://github.com/CurtizJ))
* Fix bug in mutations that have predicate with actions that require context (for example functions for json), which may lead to crashes or strange exceptions. [#7664](https://github.com/ClickHouse/ClickHouse/pull/7664)([alesapin](https://github.com/alesapin))
* Fix mismatch of database and table names escaping in `data/` and `shadow/` directories [#7575](https://github.com/ClickHouse/ClickHouse/pull/7575)([Alexander Burmak](https://github.com/Alex-Burmak))
* Support duplicated keys in RIGHT|FULL JOINs, e.g. ```ON t.x = u.x AND t.x = u.y```. Fix crash in this case. [#7586](https://github.com/ClickHouse/ClickHouse/pull/7586)([Artem Zuikov](https://github.com/4ertus2))
* Fix `Not found column <expression> in block` when joining on expression with RIGHT or FULL JOIN. [#7641](https://github.com/ClickHouse/ClickHouse/pull/7641)([Artem Zuikov](https://github.com/4ertus2))
* One more attempt to fix infinite loop in `PrettySpace` format [#7591](https://github.com/ClickHouse/ClickHouse/pull/7591)([Olga Khvostikova](https://github.com/stavrolia))
* Write current batch for distributed send atomically [#7600](https://github.com/ClickHouse/ClickHouse/pull/7600)([Azat Khuzhin](https://github.com/azat))
* Fixed a bug with a mutation on a MergeTree when whole part remains unchanged and best space is being found on another disk [#7602](https://github.com/ClickHouse/ClickHouse/pull/7602)([Vladimir Chebotarev](https://github.com/excitoon))
* Fix bug in `concat` function when all arguments were `FixedString` of the same size. [#7635](https://github.com/ClickHouse/ClickHouse/pull/7635)([alesapin](https://github.com/alesapin))
* Fix final mark usage in `MergeTree` tables ordered by `tuple()`. In rare cases it could lead to `Can't adjust last granule` error while select. [#7639](https://github.com/ClickHouse/ClickHouse/pull/7639)([Anton Popov](https://github.com/CurtizJ))
* Fix `Not found column <expression> in block` when joining on expression with RIGHT or FULL JOIN. [#7641](https://github.com/ClickHouse/ClickHouse/pull/7641)([Artem Zuikov](https://github.com/4ertus2))
* Fixed bug with `keep_free_space_ratio` not being read from disks configuration [#7645](https://github.com/ClickHouse/ClickHouse/pull/7645)([Vladimir Chebotarev](https://github.com/excitoon))
* Fix bug in mutations that have predicate with actions that require context (for example functions for json), which may lead to crashes or strange exceptions. [#7664](https://github.com/ClickHouse/ClickHouse/pull/7664)([alesapin](https://github.com/alesapin))
* Fixed exception in case of using 1 argument while defining S3, URL and HDFS storages. [#7618](https://github.com/ClickHouse/ClickHouse/pull/7618)([Vladimir Chebotarev](https://github.com/excitoon))
* Fix rare deadlock which can happen when trace_log is in enabled. [#7838](https://github.com/ClickHouse/ClickHouse/pull/7838)([filimonov](https://github.com/filimonov))
* Fix incorrect float parsing in `Values`[#7817](https://github.com/ClickHouse/ClickHouse/issues/7817)[#7870](https://github.com/ClickHouse/ClickHouse/pull/7870) ([tavplubix](https://github.com/tavplubix))
* Fix scope of the InterpreterSelectQuery for views with query [#7601](https://github.com/ClickHouse/ClickHouse/pull/7601)([Azat Khuzhin](https://github.com/azat))
### Improvement
* Support for processors (new query execution pipeline) in `MergeTree`. [#7181](https://github.com/ClickHouse/ClickHouse/pull/7181)([Nikolai Kochetov](https://github.com/KochetovNicolai))
*`Nullable` columns recognized and NULL-values handled correctly by ODBC-bridge [#7402](https://github.com/ClickHouse/ClickHouse/pull/7402)([Vasily Nemkov](https://github.com/Enmk))
* Write current batch for distributed send atomically [#7600](https://github.com/ClickHouse/ClickHouse/pull/7600)([Azat Khuzhin](https://github.com/azat))
* Throw an exception if we cannot detect table for column name in query. [#7358](https://github.com/ClickHouse/ClickHouse/pull/7358)([Artem Zuikov](https://github.com/4ertus2))
* Add `merge_max_block_size` setting to `MergeTreeSettings`[#7412](https://github.com/ClickHouse/ClickHouse/pull/7412)([Artem Zuikov](https://github.com/4ertus2))
* Queries with `HAVING` and without `GROUP BY` assume group by constant. So, `SELECT 1 HAVING 1` now returns a result. [#7496](https://github.com/ClickHouse/ClickHouse/pull/7496)([Amos Bird](https://github.com/amosbird))
...
...
@@ -53,26 +55,25 @@
* Make `range` function behaviors almost like pythonic one. [#7518](https://github.com/ClickHouse/ClickHouse/pull/7518)([sundyli](https://github.com/sundy-li))
* Add `constraints` columns to table `system.settings`[#7553](https://github.com/ClickHouse/ClickHouse/pull/7553)([Vitaly Baranov](https://github.com/vitlibar))
* Better Null format for tcp handler, so that it's possible to use `select ignore(<expression>) from table format Null` for perf measure via clickhouse-client [#7606](https://github.com/ClickHouse/ClickHouse/pull/7606)([Amos Bird](https://github.com/amosbird))
* Queries like `CREATE TABLE ... AS (SELECT (1, 2))` are parsed correctly [#7542](https://github.com/ClickHouse/ClickHouse/pull/7542)([hcz](https://github.com/hczhcz))
### Performance Improvement
* The performance of aggregation over short string keys is improved. [#6243](https://github.com/ClickHouse/ClickHouse/pull/6243)([Alexander Kuzmenkov](https://github.com/akuzm))
* The performance of aggregation over short string keys is improved. [#6243](https://github.com/ClickHouse/ClickHouse/pull/6243)([Alexander Kuzmenkov](https://github.com/akuzm), [Amos Bird](https://github.com/amosbird))
* Run another pass of syntax/expression analysis to get potential optimizations after constant predicates are folded. [#7497](https://github.com/ClickHouse/ClickHouse/pull/7497)([Amos Bird](https://github.com/amosbird))
* Use storage meta info to evaluate trivial `SELECT count() FROM table;`[#7510](https://github.com/ClickHouse/ClickHouse/pull/7510)([Amos Bird](https://github.com/amosbird))
* Use storage meta info to evaluate trivial `SELECT count() FROM table;`[#7510](https://github.com/ClickHouse/ClickHouse/pull/7510)([Amos Bird](https://github.com/amosbird), [alexey-milovidov](https://github.com/alexey-milovidov))
* Vectorize processing `arrayReduce` similar to Aggregator `addBatch`. [#7608](https://github.com/ClickHouse/ClickHouse/pull/7608)([Amos Bird](https://github.com/amosbird))
* Minor improvements in performance of `Kafka` consumption [#7475](https://github.com/ClickHouse/ClickHouse/pull/7475)([Ivan](https://github.com/abyss7))
### Build/Testing/Packaging Improvement
* Add support for cross-compiling to the CPU architecture AARCH64. Refactor packager script. [#7370](https://github.com/ClickHouse/ClickHouse/pull/7370)[#7539](https://github.com/ClickHouse/ClickHouse/pull/7539) ([Ivan](https://github.com/abyss7))
* Unpack darwin-x86_64 and linux-aarch64 toolchains into mounted Docker volume when building packages [#7534](https://github.com/ClickHouse/ClickHouse/pull/7534)([Ivan](https://github.com/abyss7))
* Update Docker Image for Binary Packager [#7474](https://github.com/ClickHouse/ClickHouse/pull/7474)([Ivan](https://github.com/abyss7))
* Fixed compile errors on MacOS Catalina [#7585](https://github.com/ClickHouse/ClickHouse/pull/7585)([Ernest Poletaev](https://github.com/ernestp))
* Some refactoring in query analysis logic: split complex class into several simple ones. [#7454](https://github.com/ClickHouse/ClickHouse/pull/7454)([Artem Zuikov](https://github.com/4ertus2))
* Fix scope of the InterpreterSelectQuery for views with query [#7601](https://github.com/ClickHouse/ClickHouse/pull/7601)([Azat Khuzhin](https://github.com/azat))
* Fix build without submodules [#7295](https://github.com/ClickHouse/ClickHouse/pull/7295)([proller](https://github.com/proller))
* Add support for cross-compiling to the CPU architecture AARCH64 [#7370](https://github.com/ClickHouse/ClickHouse/pull/7370)([Ivan](https://github.com/abyss7))
* Better `add_globs` in CMake files [#7418](https://github.com/ClickHouse/ClickHouse/pull/7418)([Amos Bird](https://github.com/amosbird))
* Remove hardcoded paths in `unwind` target [#7460](https://github.com/ClickHouse/ClickHouse/pull/7460)([Konstantin Podshumok](https://github.com/podshumok))
* Update Docker Image for Binary Packager [#7474](https://github.com/ClickHouse/ClickHouse/pull/7474)([Ivan](https://github.com/abyss7))
* Allow to use mysql format without ssl [#7524](https://github.com/ClickHouse/ClickHouse/pull/7524)([proller](https://github.com/proller))
* Unpack toolchains into mounted volume [#7534](https://github.com/ClickHouse/ClickHouse/pull/7534)([Ivan](https://github.com/abyss7))
