Issue0402

1. Support for multiple-cluster Proxyro passwords
2. Support for rich client capabilities
3. Support for specifying Server IP routing
4. Support for the Latin1 character set
5. Support for transparently passing connection attributes with special prefixes
6. Support for distributed transaction routing
7. Support for MySQL binlog
8. Support for read-write separation and transaction splitting
9. Support for mapping VIPs to Cluster/Tenant through means other than MetaDB
10. Support for global-level weak-read and read-only configuration parameters
11. Support for GIS, JSON data types
12. Support for IPv6
13. Support for accurately routing access to the sys tenant OceanBase library in QA Mode
14. Support for ROWID routing under PS
15. Support for multi-column partition key routing
16. Support for obtaining column default value routing
17. Support for calculating expressions such as nvl, sysdate, mod,  to_char, and substr
18. Support for is null and is not null routing
19. Support for XA Start statement hold
20. 【OB Sharding】Support for rolling upgrades
21. 【OB Sharding】Support for SHOW TOPOLOGY syntax
22. 【OB Sharding】Added hint support for session stickiness: route to the previous non-scan-all statement shard
23. 【OB Sharding】Support for information_schema.partitions and information_schema.coulmns
24. 【OB Sharding】Support for SQL-level timeout control
25. 【OB Sharding】Single-shard support for SQL syntax: batch update, batch delete, batch insert

Improvements
1. RTO optimization, less than 8S
2. OB 2.0 protocol optimization, reducing synchronous statements
3. OBProxy actively disconnects and sends Error packets to the client
4. Supports global-level PS cache
5. client_sock_option_flag_out default value is changed from 2 to 3
6. obproxy_error.log will print OBServer TraceId
7. Memory optimization, including connection memory and PS memory
8. RTO probe optimization, considering it normal when receiving Handshake packets
9. The default character encoding in HandShake packets is changed to 46
10. New enable_binlog_service configuration item, default false, indicating whether to enable binlog function
11. 【OB Sharding】Log optimization
12. 【OB Sharding】insert/update/delete statements support multi-table syntax
13. 【OB Sharding】scan_all optimization
14. 【OB Sharding】Black screen DDL optimization

Bug fixes
1. Fixed Core issue when accessing partitioned tables with RowID
2. Fixed issue where deleting text ps under Client 2.0 protocol does not return anything
3. Fixed issue where executing text ps under Client 2.0 protocol, then executing sql normally causes "requestId mismatch" error
4. Fixed issue where erroneous format of err packet in internal SQL causes client to hang
5. Fixed issue where enabling v2 protocol causes "4016" error and disconnection
6. Fixed issue where enabling "enable_ob_protocol_v2_with_client=true" causes disconnection during secondary sql routing
7. Fixed issue where connection id mismatch causes proxy disconnection
8. Fixed issue where enabling compression protocol causes disconnection
9. Fixed issue where using COM_CHANGE_USER command with username not in user_name@tenant_name format causes inability to use
10. Fixed issue where date type is inaccurately routed in certain scenarios
11. Fixed Core issue caused by abnormal to_timestamp parameter
12. Fixed Core issue caused by executing anonymous blocks in ODC
13. Fixed remote routing issue with to_date function
14. Fixed routing inaccuracy issue with timestamp with local time zone type list partitioning under Oracle mode
15. Fixed Core issue caused by some keywords not having pointers set
16. Fixed issue where detection optimization causes inability to whiteness
17. Fixed issue where SSL configuration retrieval fails
18. Fixed issue where enabling CPU resource isolation on public cloud causes inability to connect to cluster
19. Fixed issue where using int form for vid configuration does not take effect
20. Fixed issue with substr() causing index out of bounds
21. Fixed partition calculation error with range partitioning under "where c1 >20" condition
22. Fixed issue where type conversion fails and causes inability to route when there are NULL values in list and list column partitioning
23. Fixed Core issue introduced by global PS
24. Fixed issue where autocommit flag in OK packet returned by set autocommit=0 is still 1
25. Fixed issue where some fields in RespResult are not initialized in constructor, causing abnormal disconnection when used
26. Fixed issue where XA_START hold&sync PS 2-in-1 protocol supports complex parameter types
27. Fixed issue where XA_START hold&sync synchronous process cannot accept multiple OB20 packets
28. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObProxyExprCalculator::calc_partition_id_with_rowid_str
29. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObProto20Utils::analyze_compressed_packet_header
30. Fixed obproxy Core issue where old database saved in proxy session is used, causing obproxy core at oceanbase::common::ObLogger::log_message_kv
31. Fixed obproxy Core dump at oceanbase::obproxy::proxy::ObMysqlSM::setup_error_transfer
32. Fixed obproxy binary Core dump at obproxy/obutils/ob_proxy_sql_parser.cpp:890
33. Fixed obproxy Core dump at oceanbase::obproxy::proxy::ObMysqlSM::tunnel_handler_response_transfered
34. Fixed Core issue with oceanbase::obproxy::proxy::ObMysqlTransact::is_internal_request
35. Fixed odp Core issue with expression mod() calculation at oceanbase::obproxy::opsql::ObProxyExprMod::calc
36. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObMysqlVCTable::cleanup_entry
37. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObMysqlTransact::is_internal_request
38. Fixed Core issue at oceanbase::obproxy::proxy::ObProxyPartMgr::get_first_part at obproxy/proxy/route/obproxy_part_mgr.cpp
39. Fixed obproxy core issue at ObCongestionEntry::to_string in obproxy/obutils/ob_congestion_entry.cpp
40. Fixed a disconnect issue when executing insert statement with "binary" keyword in the "values" clause in OB Sharding
41. Fixed an issue in OB Sharding where the "case when" condition in the "where" clause is incorrectly identified as a sharding parameter
42. Fixed a core issue in OB Sharding when using unsupported expressions in the "order by" or "group by" clause

WIP:drop useless code of bt

fix annotation error 1019

1. Support session information synchronization mechanism
2. Support public cloud shared proxy CPU resource tenant isolation
3. Support random routing
4. Support specifying ip routing
5. Support to configure tenant isolation
6. Support OB_MYSQL_COM_STMT_RESET
7. Support Primary Zone routing
8. Fast fail-fast for request probes is supported. POC use only
9. Support text PS
10. Support OB_MYSQL_COM_RESET_CONNECTION
11. Support connection string ClusterId custom separator
12. Support non-Alibaba cloud environment, use remote ip as virtual ip
13. 【OB Sharding】Support SHOW INDEX FROM statement
14. 【OB Sharding】Support truncate syntax
15. 【OB Sharding】Support SHOW FULL TABLES statement
16. 【OB Sharding】SHOW KEYS FROM statement
17. 【OB Sharding】Support SHOW FULL COLUMNS statement
18. 【OB Sharding】When the scan_all switch is false, union and union
     all are supported;
19. 【OB Sharding】When the scan_all switch is true, union all is
     supported
20. 【OB Sharding】show table, show index, etc. support like syntax
21. 【OB Sharding】Limited support for subqueries
22. 【OB Sharding】Support to adjust the memory limit according to the
     container memory size
23. Turn off the default parameter for fast-fail request probes. Off by
    default
24. Adjust the whitelist policy, no longer use the client_ip sent by the
    client, and directly use the tcp peer IP

Improvements
1. Add max_used_connections to reduce log printing when client
   connection is disconnected
2. locality changes support all_dummy refresh
3. Intercept configuration table multi-row insertion
4. Compatible with MySQL 8.0, HandleShake Capability adds multi related
   value
5. Optimize partially duplicated logs

Bug fixes
1. Fix the memory problem caused by accessing internal tables and
   abnormally obtaining partition information
2. Fixed OB_MYSQL_COM_STMT_EXECUTE protocol, ref_cursor reported error
   fetch out of sequence under function
3. Fix the issue that the enable_client_ip_checkout parameter does not
   take effect when there is a connect attr parameter
4. Fix the Sharding scenario, the select table name of the sub-database
   and sub-table is case-sensitive, resulting in an error report
5. Fix the Sharding scenario, the select of sub-database and sub-table
   will be intercepted and unsupported syntax problem
6. Fixed an issue where a single SQL was sent multiple times, causing
   the SQL to take more than 40ms
7. Fix performance issues introduced by random partitions
8. Fix the problem of disconnection of changUser command under normal
   protocol
9. Fixed the problem that file systems such as xfs cannot accurately
   determine whether it is a directory using the readdir system call
10. Fix the problem that rslist startup fails more than 3 times and does
    not fall back to the original rslist
11. Fixed inaccurate sql routing caused by text PS escape symbols
12. Fix name super long text PS, prepare stage odp core
13. Fix text PS case insensitivity
14. Fix the error Unknown prepared statement handle when executing the
    text PS large request
15. Fix text PS date type route as partition key, text ps parameter
    routing is not allowed
16. Fix text PS trigger pl routing is not allowed
17. Fix text PS name contains illegal characters, execute error 1243
18. Fix the problem that user, value, etc. are used as keywords in
    lexical analysis
19. Fixed the problem that SSL cannot be used for the link between
    Client and Proxy
20. 【OB Sharding】Fixed that the hint with partition key cannot
    calculate the partition key when inserting
21. 【OB Sharding】Fixed that if the partition key is after the 64th
    column in the insert statement, it will report that it cannot be
    calculated
22. 【OB Sharding】Fix the problem of disconnection of sequence query
23. 【OB Sharding】Fix the core caused by not supporting sum(case when)
     in the function
24. When hot restart, if the old connection receives the COM_PING
    command, the old Proxy will actively disconnect the connection
25. 【OBSharding】 Fix the syntax parsing problem of delete and update

fix issue 29

1. Added the parameter location_expire_period_time to set the active
    expiration time of the location cache. The value range is [0s, 30d],
    the default is 0
 2. show proxyroute supports printing location cache expiration time
 3. Distinguish between business requests and internal, full-time
    requests, and business request parameter control
 4. When the business request is enabled_cached_server = true, the
    Partition is not random

Bugfix:
 1. Fix core issue when obproxy::omt::ObResourceUnitTableProcessor::inc_conn
 2. Fix the problem that the route cannot be routed when establishing
    a connection with a cluster id
 3. Fix the timestamp accuracy modification caused the old version of
    the observer to be inaccurate routing
 4. Fix the Sharding scenario, the select table name of the sub-database
    and sub-table is case-sensitive, resulting in an error report
 5. Fix the Sharding scenario, the select of sub-database and sub-table
    will be intercepted and unsupported syntax problem
 6. Fixed an issue where a single SQL was sent multiple times, causing
    the SQL to take more than 40ms
 7. Fix the rslist method, specify 127.0.0.1 to start obproxy, in the
    server failure scenario, more than 3 times, the connection fails to be
    established
 8. Fixed the issue that if the cluster scheduled task fails more than
    10 times, the blacklist will be invalid
 9. Fixed the problem that OBProxy still thinks the connection is
    successful when the server hangs up
 10. Fixed the problem that the cluster scheduled task failed more than
     10 times, causing it to remain in the blacklist
 11. Fix the problem that COM_CHANGE_USER will be disconnected

 1. Added the parameter location_expire_period_time to set the active
    expiration time of the location cache. The value range is [0s, 30d],
    the default is 0
 2. show proxyroute supports printing location cache expiration time
 3. Distinguish between business requests and internal, full-time
    requests, and business request parameter control
 4. When the business request is enabled_cached_server = true, the
    Partition is not random

Bugfix:
 1. Fix core issue when obproxy::omt::ObResourceUnitTableProcessor::inc_conn
 2. Fix the problem that the route cannot be routed when establishing
    a connection with a cluster id
 3. Fix the timestamp accuracy modification caused the old version of
    the observer to be inaccurate routing
 4. Fix the Sharding scenario, the select table name of the sub-database
    and sub-table is case-sensitive, resulting in an error report
 5. Fix the Sharding scenario, the select of sub-database and sub-table
    will be intercepted and unsupported syntax problem
 6. Fixed an issue where a single SQL was sent multiple times, causing
    the SQL to take more than 40ms
 7. Fix the rslist method, specify 127.0.0.1 to start obproxy, in the
    server failure scenario, more than 3 times, the connection fails to be
    established
 8. Fixed the issue that if the cluster scheduled task fails more than
    10 times, the blacklist will be invalid
 9. Fixed the problem that OBProxy still thinks the connection is
    successful when the server hangs up
 10. Fixed the problem that the cluster scheduled task failed more than
     10 times, causing it to remain in the blacklist
 11. Fix the problem that COM_CHANGE_USER will be disconnected

  1. Support public cloud shared proxy connection number resource
     isolation
  2. Support public cloud shared proxy tenant level whitelist
  3. Support shared proxy cluster configuration push
  4. Support hot restart
  5. Supports partition routing for Timestmp data types
  6. Support client-side Execute optimization, allow without Param Type
  7. Proxyd.sh supports port modification, rslist startup, and configurl
  startup
  8. PS memory optimization
  9. 16C model performance optimization
  10. When accessing the cluster for the first time, after OBProxy
  obtains the RSList from the ConfigServer, the All Server List is
  obtained asynchronously and adjusted to obtain it synchronously
  11. Log compression can be turned on with
  enable_syslog_file_compress=True
  12. Full username strict mode can be enabled by
  enable_full_username=True
  13. You can increase the SQL length of obproxy_digest.log by setting
  the digest_sql_length parameter
  14. Added support for partition routing for timestamp type and
  negative numbers
  15. Added support for DDL statements in sharding scenarios
  16. Added tenant-level SSL link encryption
  17. Supports splicing URLs by zone
  18. OB Sharding supports Join

Bugfix:
  1. Fix the problem that the new password under the newly deployed
  cluster etc is inconsistent with the old password
  2. Fix ODP unable to handle configurations larger than 4K, currently
  there is no limit to configuration size
  3. Fix the cluster without LDG. After the enable_ldg switch is turned
  on, the proxy stops updating the location cache, resulting in no sense
  of offline machines.
  4. Fix Under the 2.0 protocol, the data returned to the driver by the ps
  and fetch protocols is incorrect, resulting in the client hang problem
  5. Fix mutli swichover obproxy core at
  databuff_print_key_obj<oceanbase::common::ObLogPrintPointerCntx
  6. Fix obproxy memory leak caused by frequent push of current
  limiting rules
  7. Fix inaccurate routing in col = ("123") mode
  8. Fix too many current-limiting rules causing the push to fail to
  take effect
  9. Fix obproxy memory leak caused by frequent push of current
  limiting rules
  10. Fix After repairing the failover of the standby database into the main
  database, after the original main database is down, the proxy will
  have a short time to route the request to the original main database
  11. Fix Failed to establish connection without specifying cluster id
  within 3 minutes after lossless FO
  12. Fix After repairing the deletion of the standby database, continue to
  send requests to the old clusterId
  13. Fix the problem that the new protocol link proxy cursor scene,
  when the fench reaches the second value, it will be rammed
  14. Fix the problem of ps cursor ramming under non-compression
  protocol
  15. Fix a new connection reporting too many connections error
  16. Fix the case-insensitive name field of the ssl_config
  configuration table, resulting in unreasonable configuration reading
  logic
  17. Fix Compression and 2.0 protocol, obdump import data will be hung
  18. Fix the new protocol, refcursor cannot get the value when there
  are other out parameters
  19. Fix obproxy core at oceanbase::common::ObObjCmpFuncs::compare null
  safe
  20. Fix obproxy core at
  oceanbase::common::ObTimeConverter::str_to_digit_with_date
  21. Fix the error -5560, Unknown prepared statement handle when bmsql
  opens ps when the compression protocol is enabled

Link: https://code.aone.alibaba-inc.com/oceanbase-ce-publish/obproxy/codereview/8205314

  1. Support public cloud shared proxy connection number resource
     isolation
  2. Support public cloud shared proxy tenant level whitelist
  3. Support shared proxy cluster configuration push
  4. Support hot restart
  5. Supports partition routing for Timestmp data types
  6. Support client-side Execute optimization, allow without Param Type
  7. Proxyd.sh supports port modification, rslist startup, and configurl
  startup
  8. PS memory optimization
  9. 16C model performance optimization
  10. When accessing the cluster for the first time, after OBProxy
  obtains the RSList from the ConfigServer, the All Server List is
  obtained asynchronously and adjusted to obtain it synchronously
  11. Log compression can be turned on with
  enable_syslog_file_compress=True
  12. Full username strict mode can be enabled by
  enable_full_username=True
  13. You can increase the SQL length of obproxy_digest.log by setting
  the digest_sql_length parameter
  14. Added support for partition routing for timestamp type and
  negative numbers
  15. Added support for DDL statements in sharding scenarios
  16. Added tenant-level SSL link encryption
  17. Supports splicing URLs by zone
  18. OB Sharding supports Join

Bugfix:
  1. Fix the problem that the new password under the newly deployed
  cluster etc is inconsistent with the old password
  2. Fix ODP unable to handle configurations larger than 4K, currently
  there is no limit to configuration size
  3. Fix the cluster without LDG. After the enable_ldg switch is turned
  on, the proxy stops updating the location cache, resulting in no sense
  of offline machines.
  4. Fix Under the 2.0 protocol, the data returned to the driver by the ps
  and fetch protocols is incorrect, resulting in the client hang problem
  5. Fix mutli swichover obproxy core at
  databuff_print_key_obj<oceanbase::common::ObLogPrintPointerCntx
  6. Fix obproxy memory leak caused by frequent push of current
  limiting rules
  7. Fix inaccurate routing in col = ("123") mode
  8. Fix too many current-limiting rules causing the push to fail to
  take effect
  9. Fix obproxy memory leak caused by frequent push of current
  limiting rules
  10. Fix After repairing the failover of the standby database into the main
  database, after the original main database is down, the proxy will
  have a short time to route the request to the original main database
  11. Fix Failed to establish connection without specifying cluster id
  within 3 minutes after lossless FO
  12. Fix After repairing the deletion of the standby database, continue to
  send requests to the old clusterId
  13. Fix the problem that the new protocol link proxy cursor scene,
  when the fench reaches the second value, it will be rammed
  14. Fix the problem of ps cursor ramming under non-compression
  protocol
  15. Fix a new connection reporting too many connections error
  16. Fix the case-insensitive name field of the ssl_config
  configuration table, resulting in unreasonable configuration reading
  logic
  17. Fix Compression and 2.0 protocol, obdump import data will be hung
  18. Fix the new protocol, refcursor cannot get the value when there
  are other out parameters
  19. Fix obproxy core at oceanbase::common::ObObjCmpFuncs::compare null
  safe
  20. Fix obproxy core at
  oceanbase::common::ObTimeConverter::str_to_digit_with_date
  21. Fix the error -5560, Unknown prepared statement handle when bmsql
  opens ps when the compression protocol is enabled

* Modify the package name to obproxy-ce

1. Use the new prometheus package to fix the core problem

Other:
1. update version to 3.2.0
2. Open source opens the 2.0 protocol by default

1. Optimize the use of epoll_wait, fully event-driven
2. New replication table supports evenly distributed routing
3. Support weak read requests sent to backup copies
4. Supports random selection of the partition route when the partition cannot be calculated
5. Supports routing that contains Chinese full-width punctuation(commas, brackets, spaces)
6. Support COM_CHANGE_USER command

Bugfix
1. Fix the problem of client connection failure when enable_proxy_scramble is configured as true
2. Fix that after starting through rslist, the server list is not
   rebuilt after the server is offline, and the server still accesses the
   offline server, which causes the access to continue to fail
3. Fix the problem that the 127.0.0.1 rslist cannot be matched with the
   server in all_server and cannot be accessed at startup
4. Fix the problem that ODP cannot be accessed normally from the local
   configuration file after OCP hangs
5. Fix the problem of ODP tenant name being injected
6. Fix the problem that thread variables cannot be synchronized when
   switching sessions in ODP 3.x
7. Fix the problem that hint contains special symbols, which causes
   insertion and parsing to fail
8. Fix the disconnection problem when the cursor does not exist
9. Fix the problem that the connection to the main library without
   cluster id can be successfully executed after several queries and the link is broken
11.Fix the problem of continuous access failure if rslist is returned
   to rslist after rslist is started

fix(build.sh): CPU_CORES is null and makes compile oom

fix(ob_proxy_main.cpp): typo fix for obproxy help

Fix typo in README

fix memset to '0' for zerofill

add some features、fix bugs、fix coverity problems