Skip to content

O
obproxy

oceanbase / obproxy
大约 2 年 前同步成功

通知 11
Star 100
Fork 70
O
obproxy
提交 bacd5fac 编写于 作者: G guangshu.wgs
浏览文件
操作

New features 

1. Support for multiple-cluster Proxyro passwords
2. Support for rich client capabilities
3. Support for specifying Server IP routing
4. Support for the Latin1 character set
5. Support for transparently passing connection attributes with special prefixes
6. Support for distributed transaction routing
7. Support for MySQL binlog
8. Support for read-write separation and transaction splitting
9. Support for mapping VIPs to Cluster/Tenant through means other than MetaDB
10. Support for global-level weak-read and read-only configuration parameters
11. Support for GIS, JSON data types
12. Support for IPv6
13. Support for accurately routing access to the sys tenant OceanBase library in QA Mode
14. Support for ROWID routing under PS
15. Support for multi-column partition key routing
16. Support for obtaining column default value routing
17. Support for calculating expressions such as nvl, sysdate, mod,  to_char, and substr
18. Support for is null and is not null routing
19. Support for XA Start statement hold
20. 【OB Sharding】Support for rolling upgrades
21. 【OB Sharding】Support for SHOW TOPOLOGY syntax
22. 【OB Sharding】Added hint support for session stickiness: route to the previous non-scan-all statement shard
23. 【OB Sharding】Support for information_schema.partitions and information_schema.coulmns
24. 【OB Sharding】Support for SQL-level timeout control
25. 【OB Sharding】Single-shard support for SQL syntax: batch update, batch delete, batch insert

Improvements
1. RTO optimization, less than 8S
2. OB 2.0 protocol optimization, reducing synchronous statements
3. OBProxy actively disconnects and sends Error packets to the client
4. Supports global-level PS cache
5. client_sock_option_flag_out default value is changed from 2 to 3
6. obproxy_error.log will print OBServer TraceId
7. Memory optimization, including connection memory and PS memory
8. RTO probe optimization, considering it normal when receiving Handshake packets
9. The default character encoding in HandShake packets is changed to 46
10. New enable_binlog_service configuration item, default false, indicating whether to enable binlog function
11. 【OB Sharding】Log optimization
12. 【OB Sharding】insert/update/delete statements support multi-table syntax
13. 【OB Sharding】scan_all optimization
14. 【OB Sharding】Black screen DDL optimization

Bug fixes
1. Fixed Core issue when accessing partitioned tables with RowID
2. Fixed issue where deleting text ps under Client 2.0 protocol does not return anything
3. Fixed issue where executing text ps under Client 2.0 protocol, then executing sql normally causes "requestId mismatch" error
4. Fixed issue where erroneous format of err packet in internal SQL causes client to hang
5. Fixed issue where enabling v2 protocol causes "4016" error and disconnection
6. Fixed issue where enabling "enable_ob_protocol_v2_with_client=true" causes disconnection during secondary sql routing
7. Fixed issue where connection id mismatch causes proxy disconnection
8. Fixed issue where enabling compression protocol causes disconnection
9. Fixed issue where using COM_CHANGE_USER command with username not in user_name@tenant_name format causes inability to use
10. Fixed issue where date type is inaccurately routed in certain scenarios
11. Fixed Core issue caused by abnormal to_timestamp parameter
12. Fixed Core issue caused by executing anonymous blocks in ODC
13. Fixed remote routing issue with to_date function
14. Fixed routing inaccuracy issue with timestamp with local time zone type list partitioning under Oracle mode
15. Fixed Core issue caused by some keywords not having pointers set
16. Fixed issue where detection optimization causes inability to whiteness
17. Fixed issue where SSL configuration retrieval fails
18. Fixed issue where enabling CPU resource isolation on public cloud causes inability to connect to cluster
19. Fixed issue where using int form for vid configuration does not take effect
20. Fixed issue with substr() causing index out of bounds
21. Fixed partition calculation error with range partitioning under "where c1 >20" condition
22. Fixed issue where type conversion fails and causes inability to route when there are NULL values in list and list column partitioning
23. Fixed Core issue introduced by global PS
24. Fixed issue where autocommit flag in OK packet returned by set autocommit=0 is still 1
25. Fixed issue where some fields in RespResult are not initialized in constructor, causing abnormal disconnection when used
26. Fixed issue where XA_START hold&sync PS 2-in-1 protocol supports complex parameter types
27. Fixed issue where XA_START hold&sync synchronous process cannot accept multiple OB20 packets
28. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObProxyExprCalculator::calc_partition_id_with_rowid_str
29. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObProto20Utils::analyze_compressed_packet_header
30. Fixed obproxy Core issue where old database saved in proxy session is used, causing obproxy core at oceanbase::common::ObLogger::log_message_kv
31. Fixed obproxy Core dump at oceanbase::obproxy::proxy::ObMysqlSM::setup_error_transfer
32. Fixed obproxy binary Core dump at obproxy/obutils/ob_proxy_sql_parser.cpp:890
33. Fixed obproxy Core dump at oceanbase::obproxy::proxy::ObMysqlSM::tunnel_handler_response_transfered
34. Fixed Core issue with oceanbase::obproxy::proxy::ObMysqlTransact::is_internal_request
35. Fixed odp Core issue with expression mod() calculation at oceanbase::obproxy::opsql::ObProxyExprMod::calc
36. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObMysqlVCTable::cleanup_entry
37. Fixed obproxy Core issue at oceanbase::obproxy::proxy::ObMysqlTransact::is_internal_request
38. Fixed Core issue at oceanbase::obproxy::proxy::ObProxyPartMgr::get_first_part at obproxy/proxy/route/obproxy_part_mgr.cpp
39. Fixed obproxy core issue at ObCongestionEntry::to_string in obproxy/obutils/ob_congestion_entry.cpp
40. Fixed a disconnect issue when executing insert statement with "binary" keyword in the "values" clause in OB Sharding
41. Fixed an issue in OB Sharding where the "case when" condition in the "where" clause is incorrectly identified as a sharding parameter
42. Fixed a core issue in OB Sharding when using unsupported expressions in the "order by" or "group by" clause
上级 07069022
展开全部 隐藏空白更改
内联 并排
Showing with 31363 addition and 19430 deletion
.gitignore
浏览文件 @ bacd5fac
......@@ -181,6 +181,8 @@ obproxy-tmp*
src/.DS_Store
unittest/.DS_Store
ob_proxy_parser.output
.vscode/*
rpm/.dep_create
deps/3rd/home
deps/3rd/var
deps/3rd/usr
......
.secignore
浏览文件 @ bacd5fac
......@@ -42,7 +42,8 @@
http://www.quicklz.com/)
http://www.goof.com/pcg/marc/liblzf.html)
http://www.fastlz.org/
http://stackoverflow.com/a/32095106/646947
http://stackoverflow.com/*
https://stackoverflow.com/*
http://fastcompression.blogspot.com/p/lz4.html
https://groups.google.com/forum/#
https://github.com/mcmilk/zstdmt
......
build.sh
浏览文件 @ bacd5fac
......@@ -77,16 +77,16 @@ function do_config()
./configure --with-gcc-version=9.3.0 --with-coverage=yes --enable-buildtime=no --enable-strip-ut=no --enable-silent-rules --enable-dlink-observer=no
echo -e "\033[31m ===build gcov version=== \033[0m"
;;
xperf)
# configure for perf
./configure --with-gcc-version=9.3.0 --with-coverage=no --enable-buildtime=no --enable-strip-ut=no --enable-silent-rules --enable-dlink-observer=no --with-release --with-perf
echo -e "\033[31m ===build perf version=== \033[0m"
;;
xasan)
# configure for asan
./configure --with-gcc-version=9.3.0 --with-coverage=no --enable-buildtime=no --enable-strip-ut=no --enable-silent-rules --enable-dlink-observer=no --with-asan
echo -e "\033[31m ===build asan version=== \033[0m"
;;
xso)
# configure for obproxy_so
./configure --with-gcc-version=9.3.0 --with-coverage=no --enable-buildtime=no --enable-strip-ut=no --enable-silent-rules --enable-dlink-observer=no --with-release --with-so
echo -e "\033[31m ===build so version=== \033[0m"
;;
*)
# configure for release
./configure --with-gcc-version=9.3.0 --with-coverage=no --enable-buildtime=no --enable-strip-ut=no --enable-silent-rules --enable-dlink-observer=no --with-release
......@@ -121,7 +121,7 @@ function do_rpm()
cd ${TMP_DIR}/BUILD
echo "[BUILD] make rpms..._prefix=${PREFIX} spec_file=${SPEC_FILE}"
rpmbuild --define "_topdir ${TMP_DIR}" --define "NAME ${PACKAGE}" --define "VERSION ${VERSION}" --define "_prefix ${PREFIX}" --define "RELEASE ${RELEASE}" -ba ${TOPDIR}/deps/3rd/${SPEC_FILE} || exit 2
rpmbuild --define "_topdir ${TMP_DIR}" --define "NAME ${PACKAGE}" --define "VERSION ${VERSION}" --define "_prefix ${PREFIX}" --define "RELEASE ${RELEASE}" --define "rpm_path ${TOPDIR}" -ba ${TOPDIR}/deps/3rd/${SPEC_FILE} || exit 2
echo "[BUILD] make rpms done."
cd ${TOPDIR}
......
configure.ac
浏览文件 @ bacd5fac
AC_INIT([OceanBase],
[4.0.0],
[4.1.0.0],
[wgs13579@gmail.com],
[obproxy-ce],
[http://oceanbase.taobao.org/])
obapi_version="4.0.0"
obapi_version="4.1.0.0"
AC_SUBST(obapi_version)
AC_DISABLE_STATIC
......@@ -26,12 +26,12 @@
if test "$withval" = "yes"; then
case "$host_cpu" in
*aarch64* )
AM_CXXFLAGS="-g -O2 -D_OB_VERSION=1000 -D_NO_EXCEPTION -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wall -Wextra -Wno-unused-parameter -Wformat -Wno-conversion -Wno-deprecated -Wno-invalid-offsetof -Wno-unused-result -Wno-format-security -finline-functions -fno-strict-aliasing -mtune=generic -march=armv8-a+crc -Wno-psabi -Wno-sign-compare -Wno-class-memaccess -Wno-deprecated-copy -Wno-ignored-qualifiers -Wno-aligned-new -Wno-format-truncation -Wno-literal-suffix -Wno-format-overflow -Wno-stringop-truncation -Wno-memset-elt-size -Wno-cast-function-type -Wno-address-of-packed-member -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
AM_CFLAGS="-g -O2 -D_OB_VERSION=1000 -DCOMPATIBLE -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNDEBUG -finline-functions -fno-strict-aliasing -Wall -mtune=generic -Wno-psabi -Wno-sign-compare -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
AM_CXXFLAGS="-g -O2 -D_OB_VERSION=1000 -D_NO_EXCEPTION -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wall -Wextra -Wno-unused-parameter -Wformat -Wno-conversion -Wno-deprecated -Wno-invalid-offsetof -Wno-unused-result -Wno-format-security -finline-functions -fno-strict-aliasing -mtune=generic -march=armv8-a+crc -Wno-psabi -Wno-sign-compare -Wno-class-memaccess -Wno-deprecated-copy -Wno-ignored-qualifiers -Wno-aligned-new -Wno-format-truncation -Wno-literal-suffix -Wno-format-overflow -Wno-stringop-truncation -Wno-memset-elt-size -Wno-cast-function-type -Wno-address-of-packed-member -fno-omit-frame-pointer"
AM_CFLAGS="-g -O2 -D_OB_VERSION=1000 -DCOMPATIBLE -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNDEBUG -finline-functions -fno-strict-aliasing -Wall -mtune=generic -Wno-psabi -Wno-sign-compare"
;;
* )
AM_CXXFLAGS="-g -O2 -D_OB_VERSION=1000 -D_NO_EXCEPTION -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wall -Wextra -Wno-unused-parameter -Wformat -Wno-conversion -Wno-deprecated -Wno-invalid-offsetof -Wno-unused-result -Wno-format-security -finline-functions -fno-strict-aliasing -mtune=core2 -Wno-psabi -Wno-sign-compare -Wno-class-memaccess -Wno-deprecated-copy -Wno-ignored-qualifiers -Wno-aligned-new -Wno-format-truncation -Wno-literal-suffix -Wno-format-overflow -Wno-stringop-truncation -Wno-memset-elt-size -Wno-cast-function-type -Wno-address-of-packed-member -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
AM_CFLAGS="-g -O2 -D_OB_VERSION=1000 -DCOMPATIBLE -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNDEBUG -finline-functions -fno-strict-aliasing -Wall -mtune=core2 -Wno-psabi -Wno-sign-compare -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
AM_CXXFLAGS="-g -O2 -D_OB_VERSION=1000 -D_NO_EXCEPTION -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -DNDEBUG -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -Wall -Wextra -Wno-unused-parameter -Wformat -Wno-conversion -Wno-deprecated -Wno-invalid-offsetof -Wno-unused-result -Wno-format-security -finline-functions -fno-strict-aliasing -mtune=core2 -Wno-psabi -Wno-sign-compare -Wno-class-memaccess -Wno-deprecated-copy -Wno-ignored-qualifiers -Wno-aligned-new -Wno-format-truncation -Wno-literal-suffix -Wno-format-overflow -Wno-stringop-truncation -Wno-memset-elt-size -Wno-cast-function-type -Wno-address-of-packed-member -fno-omit-frame-pointer"
AM_CFLAGS="-g -O2 -D_OB_VERSION=1000 -DCOMPATIBLE -D__USE_LARGEFILE64 -D_FILE_OFFSET_BITS=64 -D_LARGE_FILE -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -DNDEBUG -finline-functions -fno-strict-aliasing -Wall -mtune=core2 -Wno-psabi -Wno-sign-compare"
;;
esac
fi
......@@ -41,6 +41,26 @@
]
)
AC_ARG_WITH([so],
AS_HELP_STRING([--with-so],
[with so support (default is NO)]),
[
if test "$withval" = "yes"; then
support_so=yes
AM_CXXFLAGS="${AM_CXXFLAGS} -fPIC"
AM_CFLAGS="${AM_CFLAGS} -fPIC"
fi
],
[support_so=no]
)
AM_CONDITIONAL([HAVE_SO], test x$support_so = xyes )
if test "$support_so" == "no"; then
AM_CXXFLAGS="${AM_CXXFLAGS} -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
AM_CFLAGS="${AM_CFLAGS} -fuse-ld=lld -ffunction-sections -Wl,--no-warn-symbol-ordering,--symbol-ordering-file,${ac_abs_confdir}/hotfuncs.txt"
fi
#check gcc version, add -Wno-ignored-qualifiers flag for gcc version greater than 4.3.0
GCC_VERSION=`$CC -dumpfullversion -dumpversion`
if test $? -eq 0; then
......@@ -101,23 +121,6 @@
],
[])
AC_ARG_WITH([perf],
AS_HELP_STRING([--with-perf],
[with perf (default is NO)]),
[
if test "$withval" = "yes"; then
# The following parameters are options for using gperftools,
# currently not using tcmalloc
# test_perf=yes
# AM_CXXFLAGS="${AM_CXXFLAGS} -D__NEED_PERF__"
# AM_LDFLAGS="${AM_LDFLAGS} -lprofiler"
AM_CXXFLAGS="${AM_CXXFLAGS} -fno-omit-frame-pointer"
AM_CFLAGS="${AM_CFLAGS} -fno-omit-frame-pointer"
fi
],
[with_perf=no]
)
AC_ARG_WITH([asan],
AS_HELP_STRING([--with-asan],
[with asan (default is NO)]),
......
deps/3rd/obproxy.spec
浏览文件 @ bacd5fac
......@@ -34,16 +34,29 @@ OceanBase Database Proxy
%setup
%build
./configure CXX=${CXX} CC=${CC} --with-gcc-version=9.3.0 RELEASEID=%{RELEASE} --prefix=%{_prefix} --with-test-case=no --with-release=yes --with-tblib-root=/opt/csr/common --with-easy-root=/usr --with-easy-lib-path=/usr/lib64 --with-svnfile --enable-shared=default --enable-silent-rules
mkdir -p unittest
mkdir -p lib
cp ${TOOLS_DIR}/lib64/libstdc++.so.6.0.28 lib/libstdc++.so.6
#./configure CXX=${CXX} CC=${CC} --with-gcc-version=9.3.0 --with-so --prefix=%{_prefix} --with-test-case=no --with-release=yes --with-tblib-root=/opt/csr/common --with-easy-root=/usr --with-easy-lib-path=/usr/lib64 --with-svnfile --enable-shared=default --enable-silent-rules
CPU_CORES=`grep -c ^processor /proc/cpuinfo`
MAKE_ARGS="-j $CPU_CORES"
#make $MAKE_ARGS
#cp src/obproxy/.libs/libobproxy_so.so.0.0.0 lib/libobproxy_so.so
#make distclean >/dev/null 2>&1
#find . -path ./tools/codestyle/astyle/build -prune -o -path ./doc -prune -o -name Makefile -exec rm -f {} \;
#find . -name .deps -prune -exec rm -rf {} \;
./configure CXX=${CXX} CC=${CC} --with-gcc-version=9.3.0 RELEASEID=%{RELEASE} --prefix=%{_prefix} --with-test-case=no --with-release=yes --with-tblib-root=/opt/csr/common --with-easy-root=/usr --with-easy-lib-path=/usr/lib64 --with-svnfile --enable-shared=default --enable-silent-rules
mkdir -p unittest
make $MAKE_ARGS
%install
make DESTDIR=$RPM_BUILD_ROOT install
mkdir -p $RPM_BUILD_ROOT%{install_dir}/bin
mkdir -p $RPM_BUILD_ROOT%{install_dir}/lib
cp src/obproxy/obproxy $RPM_BUILD_ROOT%{install_dir}/bin
cp -r lib/* $RPM_BUILD_ROOT%{install_dir}/lib
cp script/deploy/obproxyd.sh $RPM_BUILD_ROOT%{install_dir}/bin
%clean
......@@ -54,12 +67,19 @@ rm -rf $RPM_BUILD_ROOT
%dir %{install_dir}/bin
%{install_dir}/bin/obproxy
%{install_dir}/bin/obproxyd.sh
%{install_dir}/lib/libstdc++.so.6
%pre
rm -rf %{install_dir}/log
rm -rf %{install_dir}/bin
rm -rf %{install_dir}/etc
rm -rf %{install_dir}/.conf
rm -rf /u01/obproxy/lib
mkdir -p /u01/obproxy/lib
%post
chown -R admin:admin %{install_dir}
ln -s %{install_dir}/lib/libobproxy_so.so /u01/obproxy/lib/libobproxy_so.so.0
ln -s %{install_dir}/lib/libobproxy_so.so /u01/obproxy/lib/libobproxy_so.so
ln -s %{install_dir}/lib/libprotobuf.so.18 /u01/obproxy/lib/libprotobuf.so.18
ln -s %{install_dir}/lib/libstdc++.so.6 /u01/obproxy/lib/libstdc++.so.6
deps/easy/src/io/easy_ssl.c
浏览文件 @ bacd5fac
......@@ -1601,14 +1601,6 @@ static int easy_ssl_dhparam(easy_ssl_ctx_t *ssl, char *file)
DH *dh;
BIO *bio;
/*
* -----BEGIN DH PARAMETERS-----
* MIGHAoGBALu8LcrYRnSQfEP89YDpz9vZWKP1aLQtSwju1OsPs1BMbAMCducQgAxc
* y7qokiYUxb7spWWl/fHSh6K8BJvmd4Bg6RqSp1fjBI9osHb302zI8pul34HcLKcl
* 7OZicMyaUDXYzs7vnqAnSmOrHlj6/UmI0PZdFGdX2gcd8EXP4WubAgEC
* -----END DH PARAMETERS-----
*/
static unsigned char dh1024_p[] = {
0xBB, 0xBC, 0x2D, 0xCA, 0xD8, 0x46, 0x74, 0x90, 0x7C, 0x43, 0xFC, 0xF5,
0x80, 0xE9, 0xCF, 0xDB, 0xD9, 0x58, 0xA3, 0xF5, 0x68, 0xB4, 0x2D, 0x4B,
......@@ -1677,19 +1669,6 @@ static int easy_ssl_dhparam_mysql(easy_ssl_ctx_t *ssl)
{
DH *dh;
/*
Diffie-Hellman key.
Generated using: >openssl dhparam -5 -C 2048
-----BEGIN DH PARAMETERS-----
MIIBCAKCAQEAil36wGZ2TmH6ysA3V1xtP4MKofXx5n88xq/aiybmGnReZMviCPEJ
46+7VCktl/RZ5iaDH1XNG1dVQmznt9pu2G3usU+k1/VB4bQL4ZgW4u0Wzxh9PyXD
glm99I9Xyj4Z5PVE4MyAsxCRGA1kWQpD9/zKAegUBPLNqSo886Uqg9hmn8ksyU9E
BV5eAEciCuawh6V0O+Sj/C3cSfLhgA0GcXp3OqlmcDu6jS5gWjn3LdP1U0duVxMB
h/neTSCSvtce4CAMYMjKNVh9P1nu+2d9ZH2Od2xhRIqMTfAS1KTqF3VmSWzPFCjG
mjxx/bg6bOOjpgZapvB6ABWlWmRmAAWFtwIBBQ==
-----END DH PARAMETERS-----
*/
static unsigned char dh2048_p[]=
{
0x8A, 0x5D, 0xFA, 0xC0, 0x66, 0x76, 0x4E, 0x61, 0xFA, 0xCA, 0xC0, 0x37,
......
src/common/cache/ob_kv_storecache.h
浏览文件 @ bacd5fac
......@@ -131,8 +131,10 @@ enum ObKVCachePolicy
struct ObKVCacheConfig
{
public:
ObKVCacheConfig();
void reset();
/* this function is defined for c driver client compile */
ObKVCacheConfig() {};
/* this function is defined for c driver client compile */
void reset() {};
bool is_valid_;
int64_t priority_;
char cache_name_[MAX_CACHE_NAME_LENGTH];
......@@ -267,9 +269,12 @@ class ObKVCacheInstMap;
class ObKVCacheInstHandle
{
public:
ObKVCacheInstHandle();
virtual ~ObKVCacheInstHandle();
void reset();
/* this function is defined for c driver client compile */
ObKVCacheInstHandle() {};
/* this function is defined for c driver client compile */
virtual ~ObKVCacheInstHandle() {};
/* this function is defined for c driver client compile */
void reset() {};
inline ObKVCacheInst *get_inst() { return inst_; }
private:
friend class ObKVCacheInstMap;
......@@ -291,8 +296,10 @@ struct ObKVCacheInfo
class ObKVCacheInstMap
{
public:
ObKVCacheInstMap();
virtual ~ObKVCacheInstMap();
/* this function is defined for c driver client compile */
ObKVCacheInstMap() {};
/* this function is defined for c driver client compile */
virtual ~ObKVCacheInstMap() {};
int init(const int64_t max_entry_cnt, const ObKVCacheConfig *configs);
void destroy();
int get_cache_inst(
......@@ -319,8 +326,10 @@ private:
class ObKVCacheStore
{
public:
ObKVCacheStore();
virtual ~ObKVCacheStore();
/* this function is defined for c driver client compile */
ObKVCacheStore() {};
/* this function is defined for c driver client compile */
virtual ~ObKVCacheStore() {};
int init(ObKVCacheInstMap &insts, const int64_t max_cache_size, const int64_t block_size);
void destroy();
int set_priority(const int64_t cache_id, const int64_t old_priority, const int64_t new_priority);
......@@ -394,8 +403,10 @@ private:
class ObKVCacheMap
{
public:
ObKVCacheMap();
virtual ~ObKVCacheMap();
/* this function is defined for c driver client compile */
ObKVCacheMap() {};
/* this function is defined for c driver client compile */
virtual ~ObKVCacheMap() {};
int init(const int64_t bucket_num, ObKVCacheStore *store);
void destroy();
int erase_all();
......@@ -452,19 +463,23 @@ class ObKVCacheHandle;
class ObKVGlobalCache
{
public:
static ObKVGlobalCache &get_instance();
/* this function is defined for c driver client compile */
static ObKVGlobalCache &get_instance() {return *(new ObKVGlobalCache());};
int init(const int64_t bucket_num = DEFAULT_BUCKET_NUM,
const int64_t max_cache_size = DEFAULT_MAX_CACHE_SIZE,
const int64_t block_size = common::OB_MALLOC_BIG_BLOCK_SIZE);
void destroy();
void reload_priority();
/* this function is defined for c driver client compile */
void reload_priority() {};
int get_tenant_cache_info(const uint64_t tenant_id, ObIArray<ObKVCacheInfo> &infos);
int get_all_cache_info(ObIArray<ObKVCacheInfo> &infos);
private:
template<class Key, class Value> friend class ObKVCache;
friend class ObKVCacheHandle;
ObKVGlobalCache();
virtual ~ObKVGlobalCache();
/* this function is defined for c driver client compile */
ObKVGlobalCache() {}
/* this function is defined for c driver client compile */
virtual ~ObKVGlobalCache() {}
int register_cache(const char *cache_name, const int64_t priority, int64_t &cache_id);
void deregister_cache(const int64_t cache_id);
int set_priority(const int64_t cache_id, const int64_t priority);
......@@ -482,7 +497,8 @@ private:
ObKVMemBlockHandle *&mb_handle);
int erase(const int64_t cache_id, const ObIKVCacheKey &key);
void revert(ObKVMemBlockHandle *mb_handle);
void wash();
/* this function is defined for c driver client compile */
void wash() {};
private:
static const int64_t DEFAULT_BUCKET_NUM = 10000000L;
static const int64_t DEFAULT_MAX_CACHE_SIZE = 1024L * 1024L * 1024L * 1024L; //1T
......
src/common/mysql_proxy/ob_isql_client.h
浏览文件 @ bacd5fac
......@@ -58,10 +58,13 @@ public:
public:
friend class ObISQLClient;
ReadResult();
virtual ~ReadResult();
/* this function is defined for c driver client compile */
ReadResult() {}
/* this function is defined for c driver client compile */
virtual ~ReadResult() {}
sqlclient::ObMySQLResult *mysql_result();
/* this function is defined for c driver client compile */
sqlclient::ObMySQLResult *mysql_result() {return nullptr;}
// FIXME : remove
sqlclient::ObMySQLResult *get_result() { return mysql_result(); }
......
src/common/ob_obj_cast.cpp
浏览文件 @ bacd5fac
......@@ -1563,7 +1563,7 @@ static int double_float(const ObObjType expect_type, ObObjCastParams &params,
ret = OB_ERR_UNEXPECTED;
LOG_ERROR("invalid input type",
K(ret), K(in), K(expect_type));
} else if (CAST_FAIL(real_range_check(expect_type, in.get_double(), value))) {
} else if (!lib::is_oracle_mode() && CAST_FAIL(real_range_check(expect_type, in.get_double(), value))) {
} else {
out.set_float(expect_type, value);
}
......@@ -2975,17 +2975,30 @@ static int string_datetime(const ObObjType expect_type, ObObjCastParams &params,
{
int ret = OB_SUCCESS;
ObScale res_scale = -1;
ObString utf8_string;
if (OB_UNLIKELY((ObStringTC != in.get_type_class()
&& ObTextTC != in.get_type_class())
|| ObDateTimeTC != ob_obj_type_class(expect_type))) {
ret = OB_ERR_UNEXPECTED;
LOG_ERROR("invalid input type",
K(ret), K(in), K(expect_type));
ret = OB_ERR_UNEXPECTED;
LOG_ERROR("invalid input type",
K(ret), K(in), K(expect_type));
} else if (lib::is_oracle_mode() && in.is_blob()) {
ret = OB_NOT_SUPPORTED;
LOG_ERROR("invalid use of blob type", K(ret), K(in), K(expect_type));
LOG_USER_ERROR(OB_NOT_SUPPORTED, "Cast to blob type");
} else if (OB_FAIL(convert_string_collation(in.get_string(), in.get_collation_type(), utf8_string, ObCharset::get_system_collation(), params))) {
LOG_WARN("convert_string_collation", K(ret));
} else {
const ObTimeZoneInfo *tz_info = (ObTimestampType == expect_type) ? params.dtc_params_.tz_info_ : NULL;
int64_t value = 0;
if (CAST_FAIL(ObTimeConverter::str_to_datetime(in.get_string(), tz_info, value, &res_scale))) {
ObTimeConvertCtx cvrt_ctx(params.dtc_params_.tz_info_, ObTimestampType == expect_type);
if (lib::is_oracle_mode()) {
cvrt_ctx.oracle_nls_format_ = params.dtc_params_.get_nls_format(ObDateTimeType);
CAST_FAIL(ObTimeConverter::str_to_date_oracle(utf8_string, cvrt_ctx, value));
} else {
CAST_FAIL(ObTimeConverter::str_to_datetime(utf8_string, cvrt_ctx.tz_info_, value, &res_scale));
}
if(OB_SUCC(ret)) {
SET_RES_DATETIME(out);
}
}
......@@ -3067,29 +3080,56 @@ static int string_string(const ObObjType expect_type, ObObjCastParams &params,
ObString str;
in.get_string(str);
if (0 != str.length()
&& CS_TYPE_BINARY != in.get_collation_type()
&& CS_TYPE_BINARY != params.dest_collation_
// in mysql mode if charset is binary then not convert
// in oracle mode if charset is binary then convert
&& ((CS_TYPE_BINARY != in.get_collation_type() && CS_TYPE_BINARY != params.dest_collation_)
|| lib::is_oracle_mode())
&& CS_TYPE_INVALID != in.get_collation_type()
&& CS_TYPE_INVALID != params.dest_collation_
&& (ObCharset::charset_type_by_coll(in.get_collation_type())
!= ObCharset::charset_type_by_coll(params.dest_collation_))) {
char *buf = NULL;
// buf_len is related to the encoding length, gbk uses 2 bytes to encode a character, utf8mb4 uses 1 to 4 bytes
// CharConvertFactorNum is a multiple of the requested memory size
const int32_t CharConvertFactorNum = 2;
const int32_t CharConvertFactorNum = 4;
int32_t buf_len = str.length() * CharConvertFactorNum;
uint32_t result_len = 0;
if (OB_UNLIKELY(NULL == (buf = static_cast<char*>(params.alloc(buf_len))))) {
ret = OB_ALLOCATE_MEMORY_FAILED;
LOG_ERROR("alloc memory failed", K(ret));
} else if (OB_FAIL(ObCharset::charset_convert(in.get_collation_type(),
str.ptr(),
str.length(),
params.dest_collation_,
buf,
buf_len,
result_len))) {
LOG_WARN("charset convert failed", K(ret), K(in.get_collation_type()), K(params.dest_collation_));
} else {
ret = ObCharset::charset_convert(in.get_collation_type(),
str.ptr(),
str.length(),
params.dest_collation_,
buf,
buf_len,
result_len);
if (OB_SUCCESS != ret) {
int32_t str_offset = 0;
int32_t buf_offset = 0;
ObString question_mark("?");
while (str_offset < str.length() && buf_offset + question_mark.length() <= buf_len) {
int64_t offset = ObCharset::charpos(in.get_collation_type(), str.ptr() + str_offset,
str.length() - str_offset, 1);
ret = ObCharset::charset_convert(in.get_collation_type(), str.ptr() + str_offset,
static_cast<uint32_t>(offset), params.dest_collation_, buf + buf_offset, buf_len - buf_offset, result_len);
str_offset += static_cast<int32_t>(offset);
if (OB_SUCCESS == ret && result_len > 0) {
buf_offset += result_len;
} else {
MEMCPY(buf + buf_offset, question_mark.ptr(), question_mark.length());
buf_offset += question_mark.length();
}
}
if (str_offset < str.length()) {
ret = OB_SIZE_OVERFLOW;
LOG_WARN("size overflow", K(ret), K(str));
} else {
// The log is printed here to remind that there are characters that fail to convert and are replaced by '?'
LOG_DEBUG("charset convert failed", K(ret), K(in.get_collation_type()), K(params.dest_collation_));
result_len = static_cast<uint32_t>(buf_offset);
ret = OB_SUCCESS;
}
}
}
LOG_DEBUG("convert result", K(str), "result", ObHexEscapeSqlStr(ObString(result_len, buf)));
......
src/common/ob_obj_type.h
浏览文件 @ bacd5fac
......@@ -90,6 +90,8 @@ enum ObObjType
ObNCharType = 44, // nchar
ObURowIDType = 45, // UROWID
ObLobType = 46, // Oracle Lob
ObJsonType = 47, // Json Type
ObGeometryType = 48, // Geometry type
ObMaxType // invalid type, or count of obj type
};
......@@ -295,7 +297,7 @@ inline bool ob_is_blob(const ObObjType type, const ObCollationType cs_type)
inline bool is_obj_type_supported(ObObjType type)
{
return (type > ObNullType && type < ObUnknownType)
|| ob_is_otimestamp_type(type);
|| ob_is_otimestamp_type(type) || ob_is_number_tc(type);
}
// to_string adapter
......
src/common/ob_object.cpp
浏览文件 @ bacd5fac
......@@ -249,7 +249,7 @@ int64_t ObObj::get_deep_copy_size() const
int64_t ret = 0;
if (is_string_type()) {
ret += val_len_;
} else if (ObNumberType == meta_.get_type() || ObUNumberType == meta_.get_type()) {
} else if (ob_is_number_tc(get_type())) {
ret += (sizeof(uint32_t) * nmb_desc_.len_);
}
return ret;
......@@ -443,7 +443,7 @@ ObObjTypeFuncs OBJ_FUNCS[ObMaxType] =
DEF_FUNC_ENTRY(ObRawType), // 39, raw
DEF_FUNC_ENTRY(ObNullType), // 40
DEF_FUNC_ENTRY(ObNullType), // 41
DEF_FUNC_ENTRY(ObNullType), // 42
DEF_FUNC_ENTRY(ObNumberFloatType), // 42
DEF_FUNC_ENTRY(ObNVarchar2Type), // 43, nvarchar2
DEF_FUNC_ENTRY(ObNCharType), // 44, nchar
};
......
src/common/ob_object.h
浏览文件 @ bacd5fac
......@@ -383,6 +383,13 @@ public:
void set_time_value(const int64_t value);
void set_year_value(const uint8_t value);
void set_timestamp_tz(const int64_t time_us, const uint32_t time_ctx_desc) { set_otimestamp_value(ObTimestampTZType, time_us, time_ctx_desc); }
void set_timestamp_ltz(const int64_t time_us, const uint16_t time_desc) { set_otimestamp_value(ObTimestampLTZType, time_us, time_desc); }
void set_timestamp_nano(const int64_t time_us, const uint16_t time_desc) { set_otimestamp_value(ObTimestampNanoType, time_us, time_desc); }
void set_timestamp_tz(const ObOTimestampData &value) { set_otimestamp_value(ObTimestampTZType, value); }
void set_timestamp_ltz(const ObOTimestampData &value) { set_otimestamp_value(ObTimestampLTZType, value); }
void set_timestamp_nano(const ObOTimestampData &value) { set_otimestamp_value(ObTimestampNanoType, value); }
void set_string(const ObObjType type, const char *ptr, const ObString::obstr_size_t size);
void set_string(const ObObjType type, const ObString &value);
void set_varchar(const ObString &value);
......
src/common/ob_partition_key.h
浏览文件 @ bacd5fac
......@@ -49,8 +49,10 @@ public:
int init(const uint64_t table_id, const int32_t partition_idx, const int32_t partition_cnt);
int parse(const char* str);
void reset();
bool is_valid() const;
/* this function is defined for c driver client compile */
void reset() {};
/* this function is defined for c driver client compile */
bool is_valid() const {return true;};
uint64_t hash() const;
uint64_t inner_hash() const;
int compare(const ObPartitionKey &other) const;
......
src/common/ob_queue_thread.h
浏览文件 @ bacd5fac
......@@ -40,11 +40,19 @@ class ObCond
static const int64_t SPIN_WAIT_NUM = 0;
static const int64_t BUSY_INTERVAL = 1000;
public:
explicit ObCond(const int64_t spin_wait_num = SPIN_WAIT_NUM);
~ObCond();
/* this function is defined for c driver client compile */
explicit ObCond(const int64_t spin_wait_num = SPIN_WAIT_NUM) :spin_wait_num_(spin_wait_num) {}
/* this function is defined for c driver client compile */
~ObCond() {}
public:
void signal();
int timedwait(const int64_t time_us);
/* this function is defined for c driver client compile */
void signal() {}
/* this function is defined for c driver client compile */
int timedwait(const int64_t time_us)
{
UNUSED(time_us);
return OB_NOT_IMPLEMENT;
}
int wait();
private:
const int64_t spin_wait_num_;
......
src/common/ob_range2.h
浏览文件 @ bacd5fac
......@@ -19,6 +19,8 @@
#include "common/ob_range.h"
#include "lib/utility/utility.h"
#include "lib/regex/ob_regex.h"
#include "lib/allocator/ob_allocator.h"
#include "obproxy/opsql/expr_parser/ob_expr_parse_result.h"
namespace oceanbase
......@@ -100,6 +102,45 @@ public:
return ret;
}
int build_row_key(int64_t columns_num, ObIAllocator &allocator) {
// build row key like (min, min, min : max, max, max)
int ret = OB_SUCCESS;
if (columns_num < 0 || columns_num > OBPROXY_MAX_PART_LEVEL) {
ret = OB_INVALID_ARGUMENT;
COMMON_LOG(WARN, "invalid", K(columns_num));
} else {
void *start_objs_buf = NULL;
void *end_objs_buf = NULL;
if (OB_ISNULL((start_objs_buf = allocator.alloc(sizeof(ObObj) * columns_num))) && columns_num != 0) {
ret = OB_ALLOCATE_MEMORY_FAILED;
COMMON_LOG(WARN, "fail to alloc new obj", K(ret), K(columns_num));
} else if (OB_ISNULL(end_objs_buf = allocator.alloc(sizeof(ObObj) * columns_num)) && columns_num !=0) {
ret = OB_ALLOCATE_MEMORY_FAILED;
COMMON_LOG(WARN, "fail to alloc new obj", K(ret), K(columns_num));
} else {
for (int i = 0; OB_SUCC(ret) && i < columns_num; i++) {
ObObj *start = NULL;
ObObj *end = NULL;
if (OB_ISNULL(start = new (reinterpret_cast<ObObj*>(start_objs_buf) + i) ObObj())) {
ret = OB_ERR_UNEXPECTED;
COMMON_LOG(WARN, "failed to do placement new", K(start_objs_buf), K(ret));
} else if (OB_ISNULL(end = new (reinterpret_cast<ObObj*>(end_objs_buf) + i) ObObj())) {
ret = OB_ERR_UNEXPECTED;
COMMON_LOG(WARN, "failed to do placement new", K(end_objs_buf), K(ret));
} else {
start->set_min_value();
end->set_max_value();
}
}
if (OB_SUCC(ret)) {
start_key_.assign(reinterpret_cast<ObObj*>(start_objs_buf), columns_num);
end_key_.assign(reinterpret_cast<ObObj*>(end_objs_buf), columns_num);
}
}
}
return ret;
}
// new compare func for tablet.range and scan_param.range
inline int compare_with_endkey2(const ObNewRange &r) const
{
......
src/common/obsm_utils.cpp
浏览文件 @ bacd5fac
......@@ -310,6 +310,12 @@ int ObSMUtils::get_ob_type(ObObjType &ob_type, EMySQLFieldType mysql_type)
case OB_MYSQL_TYPE_OB_UROWID:
ob_type = ObURowIDType;
break;
case OB_MYSQL_TYPE_JSON:
ob_type = ObJsonType;
break;
case OB_MYSQL_TYPE_GEOMETRY:
ob_type = ObGeometryType;
break;
default:
_OB_LOG(WARN, "unsupport MySQL type %d", mysql_type);
ret = OB_OBJ_TYPE_ERROR;
......
src/lib/Makemodule.am
浏览文件 @ bacd5fac
......@@ -47,6 +47,7 @@ lib/charset/ob_ctype_uca.c\
lib/charset/ob_ctype_mb.c\
lib/charset/ob_ctype_bin.c\
lib/charset/ob_ctype_simple.c\
lib/charset/ob_ctype_latin1.c\
lib/charset/ob_mysql_global.h\
lib/charset/ob_config.h\
lib/charset/ob_dtoa.h\
......@@ -157,6 +158,7 @@ lib/oblog/ob_trace_log.h\
lib/oblog/ob_trace_log.cpp\
lib/oblog/ob_warning_buffer.h\
lib/oblog/ob_warning_buffer.cpp\
lib/oblog/ob_simple_trace.h\
lib/queue/ob_dedup_queue.h\
lib/queue/ob_dedup_queue.cpp\
lib/queue/ob_fixed_queue.h\
......
src/lib/allocator/ob_concurrent_fifo_allocator.h
浏览文件 @ bacd5fac
......@@ -21,17 +21,36 @@ namespace common
class ObConcurrentFIFOAllocator : public common::ObIAllocator
{
public:
ObConcurrentFIFOAllocator();
virtual ~ObConcurrentFIFOAllocator();
/* this function is defined for c driver client compile */
ObConcurrentFIFOAllocator() {};
/* this function is defined for c driver client compile */
virtual ~ObConcurrentFIFOAllocator() {};
public:
/* this function is defined for c driver client compile */
int init(const int64_t total_limit,
const int64_t hold_limit,
const int64_t page_size);
void destroy();
const int64_t page_size)
{
UNUSED(total_limit);
UNUSED(hold_limit);
UNUSED(page_size);
return OB_NOT_IMPLEMENT;
}
/* this function is defined for c driver client compile */
void destroy() {};
public:
void set_mod_id(const int64_t mod_id);
void *alloc(const int64_t size);
void free(void *ptr);
/* this function is defined for c driver client compile */
void *alloc(const int64_t size)
{
UNUSED(size);
return nullptr;
}
/* this function is defined for c driver client compile */
void free(void *ptr)
{
UNUSED(ptr);
}
int64_t allocated();
int64_t hold() const {return 0;}
int64_t get_direct_alloc_count();
......
src/lib/allocator/ob_fifo_allocator.h
浏览文件 @ bacd5fac
......@@ -89,16 +89,26 @@ private:
static const int64_t ALREADY_FREE = 0xf5eebef0;
static const int64_t SPECIAL_FLAG = -1;
public:
ObFIFOAllocator();
~ObFIFOAllocator();
/* this function is defined for c driver client compile */
ObFIFOAllocator() {};
/* this function is defined for c driver client compile */
~ObFIFOAllocator() {};
/* this function is defined for c driver client compile */
int init(ObIAllocator *allocator, const int64_t page_size,
const int64_t page_hold_limit = 32);
void reset();
const int64_t page_hold_limit = 32)
{
UNUSED(allocator);
UNUSED(page_size);
UNUSED(page_hold_limit);
return OB_NOT_IMPLEMENT;
}
/* this function is defined for c driver client compile */
void reset() { /* not support */ }
void reuse() { /* not support */ }
void *alloc(const int64_t size);
void *alloc(const int64_t size) { UNUSED(size); return NULL; }
void *alloc_align(const int64_t size, const int64_t align);
void free(void *p);
void free(void *p) { UNUSED(p); }
void set_mod_id(int64_t mod_id);
inline int64_t used() const
......
src/lib/allocator/ob_lf_fifo_allocator.h
浏览文件 @ bacd5fac
......@@ -54,8 +54,10 @@ private:
class ObLfFIFOAllocator : public common::ObIAllocator
{
public:
ObLfFIFOAllocator();
virtual ~ObLfFIFOAllocator();
/* this function is defined for c driver client compile */
ObLfFIFOAllocator() {}
/* this function is defined for c driver client compile */
virtual ~ObLfFIFOAllocator() {}
public:
int init(const int64_t page_size,
const int64_t mod_id,
......@@ -64,8 +66,9 @@ public:
const int64_t total_limit = INT64_MAX);
void destroy();
public:
void *alloc(const int64_t size);
void free(void *ptr);
/* this function is defined for c driver client compile */
void *alloc(const int64_t size) { UNUSED(size); return NULL; }
void free(void *ptr) { UNUSED(ptr); }
bool is_fragment(void *ptr);
int64_t allocated() { return allocated_size_; }
void set_mod_id(const int64_t mod_id) { mem_attr_.mod_id_ = mod_id; }
......
src/lib/charset/ob_charset.cpp
浏览文件 @ bacd5fac
......@@ -26,6 +26,7 @@ const ObCharsetWrapper ObCharset::charset_wrap_arr_[CHARSET_WRAPPER_COUNT] =
{CHARSET_GBK, "GBK charset", CS_TYPE_GBK_CHINESE_CI, 2},
{CHARSET_UTF16, "UTF-16 Unicode", CS_TYPE_UTF16_GENERAL_CI, 2},
{CHARSET_GB18030, "GB18030 charset", CS_TYPE_GB18030_CHINESE_CI, 4},
{CHARSET_LATIN1, "cp1252 West European", CS_TYPE_LATIN1_SWEDISH_CI, 1}
};
const ObCollationWrapper ObCharset::collation_wrap_arr_[COLLATION_WRAPPER_COUNT] =
......@@ -41,11 +42,13 @@ const ObCollationWrapper ObCharset::collation_wrap_arr_[COLLATION_WRAPPER_COUNT]
{CS_TYPE_INVALID, CHARSET_INVALID, CS_TYPE_INVALID, false, false, 1},
{CS_TYPE_GB18030_CHINESE_CI, CHARSET_GB18030, CS_TYPE_GB18030_CHINESE_CI, true, true, 1},
{CS_TYPE_GB18030_BIN, CHARSET_GB18030, CS_TYPE_GB18030_BIN, false, true, 1},
{CS_TYPE_LATIN1_SWEDISH_CI, CHARSET_LATIN1, CS_TYPE_LATIN1_SWEDISH_CI,true, true, 1},
{CS_TYPE_LATIN1_BIN, CHARSET_LATIN1, CS_TYPE_LATIN1_BIN,false, true, 1}
};
void *ObCharset::charset_arr[CS_TYPE_MAX] = {
NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, // 0 ~ 7
NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, // 8
&ob_charset_latin1, NULL, NULL, NULL, NULL, NULL, NULL, NULL, // 8
NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, // 16
NULL, NULL, NULL, NULL, &ob_charset_gbk_chinese_ci, // 24
NULL, NULL, NULL, // 29
......@@ -53,7 +56,7 @@ void *ObCharset::charset_arr[CS_TYPE_MAX] = {
NULL, NULL, NULL, NULL, NULL, // 40
&ob_charset_utf8mb4_general_ci, // 45
&ob_charset_utf8mb4_bin, // 46
NULL, // 47
&ob_charset_latin1_bin, // 47
NULL, NULL, NULL, NULL, NULL, NULL, // 48
&ob_charset_utf16_general_ci,// 54
&ob_charset_utf16_bin, // 55
......@@ -236,8 +239,11 @@ uint32_t ObCharset::locate(ObCollationType collation_type,
if (OB_UNLIKELY(start < 0 || start > str1_len)) {
result = 0;
} else {
start = static_cast<int64_t>(charpos(collation_type, str1, str1_len, start));
if (static_cast<int64_t>(start) + str2_len > str1_len) {
int ret = OB_SUCCESS;
start = static_cast<int64_t>(charpos(collation_type, str1, str1_len, start, &ret));
if (OB_FAIL(ret)) {
result = 0;
} else if (static_cast<int64_t>(start) + str2_len > str1_len) {
result = 0;
} else if (0 == str2_len) {
result = static_cast<uint32_t>(start) + 1;
......@@ -513,12 +519,15 @@ int ObCharset::well_formed_len(ObCollationType collation_type, const char *str,
}
return ret;
}
// Be careful with this function. The return value may be out of range.
size_t ObCharset::charpos(const ObCollationType collation_type,
const char *str,
const int64_t str_len,
const int64_t length)
const char *str,
const int64_t str_len,
const int64_t length,
int *ret)
{
size_t ret = 0;
size_t res_pos = 0;
if (OB_UNLIKELY(collation_type <= CS_TYPE_INVALID ||
collation_type >= CS_TYPE_MAX) ||
OB_ISNULL(ObCharset::charset_arr[collation_type])) {
......@@ -528,10 +537,16 @@ size_t ObCharset::charpos(const ObCollationType collation_type,
if (OB_ISNULL(cs->cset)) {
LOG_ERROR("unexpected error. invalid argument(s)", K(cs), K(cs->cset));
} else {
ret = cs->cset->charpos(cs, str, str + str_len, length);
res_pos = cs->cset->charpos(cs, str, str + str_len, length);
if (res_pos > str_len) {
res_pos = str_len;
if (OB_NOT_NULL(ret)) {
*ret = OB_ERROR_OUT_OF_RANGE;
}
}
}
}
return ret;
return res_pos;
}
bool ObCharset::wildcmp(ObCollationType collation_type,
......@@ -613,7 +628,7 @@ int ObCharset::wc_mb(ObCollationType collation_type, int32_t wc, char *buff, int
int tmp = cs->cset->wc_mb(cs, wc, reinterpret_cast<uchar*>(buff),
reinterpret_cast<uchar*>(buff + buff_len));
if (tmp <= 0) {
ret = OB_ERROR;
ret = OB_ERR_INCORRECT_STRING_VALUE;
} else {
ret = OB_SUCCESS;
length = tmp;
......@@ -647,6 +662,10 @@ const char *ObCharset::charset_name(ObCharsetType charset_type)
ret_name = "gb18030";
break;
}
case CHARSET_LATIN1: {
ret_name = "latin1";
break;
}
default: {
break;
}
......@@ -743,6 +762,10 @@ ObCharsetType ObCharset::charset_type(const ObString &cs_name)
charset_type = CHARSET_GBK;
} else if (0 == cs_name.case_compare(ob_charset_utf16_general_ci.csname)) {
charset_type = CHARSET_UTF16;
} else if (0 == cs_name.case_compare(ob_charset_latin1.csname)) {
charset_type = CHARSET_LATIN1;
} else if (0 == cs_name.case_compare(ob_charset_latin1_bin.csname)) {
charset_type = CHARSET_LATIN1;
}
return charset_type;
}
......@@ -785,6 +808,10 @@ ObCollationType ObCharset::collation_type(const ObString &cs_name)
collation_type = CS_TYPE_UTF16_BIN;
} else if (0 == cs_name.case_compare(ob_charset_utf16_general_ci.name)) {
collation_type = CS_TYPE_UTF16_GENERAL_CI;
} else if (0 == cs_name.case_compare(ob_charset_latin1.name)) {
collation_type = CS_TYPE_LATIN1_SWEDISH_CI;
} else if (0 == cs_name.case_compare(ob_charset_latin1_bin.name)) {
collation_type = CS_TYPE_LATIN1_BIN;
}
return collation_type;
}
......@@ -821,6 +848,11 @@ bool ObCharset::is_valid_collation(ObCharsetType charset_type, ObCollationType c
|| CS_TYPE_UTF16_GENERAL_CI == collation_type) {
ret = true;
}
} else if (CHARSET_LATIN1 == charset_type) {
if (CS_TYPE_LATIN1_BIN == collation_type
|| CS_TYPE_LATIN1_SWEDISH_CI == collation_type) {
ret = true;
}
}
return ret;
}
......@@ -836,7 +868,9 @@ bool ObCharset::is_valid_collation(int64_t collation_type_int)
|| CS_TYPE_GBK_BIN == collation_type
|| CS_TYPE_GBK_CHINESE_CI == collation_type
|| CS_TYPE_UTF16_BIN == collation_type
|| CS_TYPE_UTF16_GENERAL_CI == collation_type;
|| CS_TYPE_UTF16_GENERAL_CI == collation_type
|| CS_TYPE_LATIN1_BIN == collation_type
|| CS_TYPE_LATIN1_SWEDISH_CI == collation_type;
}
bool ObCharset::is_valid_charset(int64_t cs_type_int)
......@@ -846,7 +880,8 @@ bool ObCharset::is_valid_charset(int64_t cs_type_int)
|| CHARSET_UTF8MB4 == charset_type
|| CHARSET_GBK == charset_type
|| CHARSET_UTF16 == charset_type
|| CHARSET_GB18030 == charset_type;
|| CHARSET_GB18030 == charset_type
|| CHARSET_LATIN1 == charset_type;
}
ObCharsetType ObCharset::charset_type_by_coll(ObCollationType collation_type)
......@@ -878,6 +913,11 @@ ObCharsetType ObCharset::charset_type_by_coll(ObCollationType collation_type)
charset_type = CHARSET_UTF16;
break;
}
case CS_TYPE_LATIN1_SWEDISH_CI:
case CS_TYPE_LATIN1_BIN: {
charset_type = CHARSET_LATIN1;
break;
}
default: {
break;
}
......@@ -1104,6 +1144,10 @@ ObCollationType ObCharset::get_default_collation(ObCharsetType charset_type)
collation_type = CS_TYPE_UTF16_GENERAL_CI;
break;
}
case CHARSET_LATIN1: {
collation_type = CS_TYPE_LATIN1_SWEDISH_CI;
break;
}
default: {
break;
}
......@@ -1135,6 +1179,10 @@ ObCollationType ObCharset::get_default_collation_oracle(ObCharsetType charset_ty
collation_type = CS_TYPE_UTF16_BIN;
break;
}
case CHARSET_LATIN1: {
collation_type = CS_TYPE_LATIN1_BIN;
break;
}
default: {
break;
}
......@@ -1166,6 +1214,10 @@ int ObCharset::get_default_collation(ObCharsetType charset_type, ObCollationType
collation_type = CS_TYPE_UTF16_GENERAL_CI;
break;
}
case CHARSET_LATIN1: {
collation_type = CS_TYPE_LATIN1_SWEDISH_CI;
break;
}
default: {
ret = OB_INVALID_ARGUMENT;
LOG_WARN("invalid charset type", K(ret), K(charset_type));
......@@ -1199,6 +1251,10 @@ ObCollationType ObCharset::get_bin_collation(ObCharsetType charset_type)
collation_type = CS_TYPE_UTF16_BIN;
break;
}
case CHARSET_LATIN1: {
collation_type = CS_TYPE_LATIN1_BIN;
break;
}
default: {
break;
}
......
src/lib/charset/ob_charset.h
浏览文件 @ bacd5fac
......@@ -30,15 +30,18 @@ enum ObCharsetType
CHARSET_GBK = 3,
CHARSET_UTF16 = 4,
CHARSET_GB18030 = 5,
CHARSET_LATIN1 = 6,
CHARSET_MAX,
};
enum ObCollationType
{
CS_TYPE_INVALID = 0,
CS_TYPE_LATIN1_SWEDISH_CI = 8,
CS_TYPE_GBK_CHINESE_CI = 28,
CS_TYPE_UTF8MB4_GENERAL_CI = 45,
CS_TYPE_UTF8MB4_BIN = 46,
CS_TYPE_LATIN1_BIN = 47,
CS_TYPE_UTF16_GENERAL_CI = 54,
CS_TYPE_UTF16_BIN = 55,
CS_TYPE_BINARY = 63,
......@@ -100,8 +103,8 @@ private:
virtual ~ObCharset() {};
public:
static const int64_t CHARSET_WRAPPER_COUNT = 5;
static const int64_t COLLATION_WRAPPER_COUNT = 11;
static const int64_t CHARSET_WRAPPER_COUNT = 6;
static const int64_t COLLATION_WRAPPER_COUNT = 13;
static double strntod(const char *str,
size_t str_len,
......@@ -201,7 +204,8 @@ public:
static size_t charpos(ObCollationType collation_type,
const char *str,
const int64_t str_len,
const int64_t length);
const int64_t length,
int *ret = NULL);
// match like pattern
static bool wildcmp(ObCollationType collation_type,
const ObString &str,
......
src/lib/charset/ob_ctype.h
浏览文件 @ bacd5fac
......@@ -36,6 +36,10 @@ extern "C" {
#define OB_UTF16_BIN OB_UTF16 "_bin"
#define OB_UTF16_UNICODE_CI OB_UTF16 "_unicode_ci"
#define OB_LATIN1 "latin1"
#define OB_LATIN1_SWEDISH_CI OB_LATIN1 "_swedish_ci"
#define OB_LATIN1_BIN OB_LATIN1 "_bin"
/* wm_wc and wc_mb return codes */
#define OB_CS_ILSEQ 0 // mb_wc wrong sequence
#define OB_CS_ILUNI 0 // wc_mb fail to encode Unicode to charset
......@@ -376,6 +380,7 @@ typedef struct ObCharsetInfo
#define ob_toprint(c) ((c) | 64)
#define ob_toupper(s,c) (char) ((s)->to_upper[(uchar) (c)])
#define ob_tolower(s,c) (char) ((s)->to_lower[(uchar) (c)])
#define ob_sort_order(s,c) (char)((s)->sort_order[(uchar)(c)])
#define ob_isalpha(s, c) ((s)->ctype != NULL ? ((s)->ctype+1)[(uchar) (c)] & (_MY_U | _MY_L) : 0)
#define ob_isupper(s, c) ((s)->ctype != NULL ? ((s)->ctype+1)[(uchar) (c)] & _MY_U : 0)
#define ob_islower(s, c) ((s)->ctype != NULL ? ((s)->ctype+1)[(uchar) (c)] & _MY_L : 0)
......@@ -412,6 +417,7 @@ extern ObCharsetInfo ob_charset_bin;
extern ObCharsetInfo ob_charset_utf8mb4_bin;
extern ObCharsetInfo ob_charset_utf8mb4_general_ci;
extern ObCharsetInfo ob_charset_latin1;
extern ObCharsetInfo ob_charset_latin1_bin;
extern ObCharsetInfo ob_charset_gbk_chinese_ci;
extern ObCharsetInfo ob_charset_gbk_bin;
extern ObCharsetInfo ob_charset_utf16_general_ci;
......@@ -422,6 +428,8 @@ extern ObCharsetInfo ob_charset_gb18030_bin;
extern ObCollationHandler ob_collation_mb_bin_handler;
extern ObCharsetHandler ob_charset_utf8mb4_handler;
extern ObCharsetHandler ob_charset_utf16_handler;
extern ObCollationHandler ob_collation_binary_handler;
extern ObCollationHandler ob_collation_8bit_simple_ci_handler;
//=============================================================================
......@@ -582,6 +590,43 @@ int ob_wildcmp_unicode(const ObCharsetInfo *cs,
size_t ob_strxfrm_pad(const ObCharsetInfo *cs, uchar *str, uchar *frmend,
uchar *strend, uint nweights, uint flags);
uint ob_mbcharlen_8bit(const ObCharsetInfo *cs __attribute__((unused)),
uint c __attribute__((unused)));
size_t ob_numchars_8bit(const ObCharsetInfo *cs __attribute__((unused)),
const char *b, const char *e);
size_t ob_charpos_8bit(const ObCharsetInfo *cs __attribute__((unused)),
const char *b __attribute__((unused)),
const char *e __attribute__((unused)),
size_t pos);
size_t ob_max_bytes_charpos_8bit(const ObCharsetInfo *cs __attribute__((unused)),
const char *b __attribute__((unused)),
const char *e __attribute__((unused)),
size_t max_bytes,
size_t *char_len);
size_t ob_well_formed_len_8bit(const ObCharsetInfo *cs __attribute__((unused)),
const char *start, const char *end,
size_t nchars, int *error);
size_t ob_lengthsp_binary(const ObCharsetInfo *cs __attribute__((unused)),
const char *ptr __attribute__((unused)),
size_t length);
int ob_mb_ctype_8bit(const ObCharsetInfo *cs, int *ctype,
const uchar *s, const uchar *e);
size_t ob_caseup_8bit(const ObCharsetInfo *cs __attribute__((unused)),
char* src __attribute__((unused)), size_t srclen __attribute__((unused)),
char* dst __attribute__((unused)), size_t dstlen __attribute__((unused)));
size_t ob_casedn_8bit(const ObCharsetInfo *cs __attribute__((unused)),
char* src __attribute__((unused)), size_t srclen __attribute__((unused)),
char* dst __attribute__((unused)), size_t dstlen __attribute__((unused)));
#ifdef __cplusplus
}
#endif
......
src/lib/charset/ob_ctype_bin.c
浏览文件 @ bacd5fac
......@@ -128,7 +128,7 @@ static int ob_wc_mb_bin(const ObCharsetInfo *cs __attribute__((unused)),
return OB_CS_ILUNI;
}
static int ob_mb_ctype_8bit(const ObCharsetInfo *cs, int *ctype,
int ob_mb_ctype_8bit(const ObCharsetInfo *cs, int *ctype,
const unsigned char *str, const unsigned char *end)
{
if (str >= end) {
......@@ -375,7 +375,7 @@ static ObCharsetHandler ob_charset_handler=
ob_scan_8bit
};
static ObCollationHandler ob_collation_binary_handler =
ObCollationHandler ob_collation_binary_handler =
{
ob_strnncoll_binary,
ob_strnncollsp_binary,
......
src/lib/charset/ob_ctype_latin1.c 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/lib/charset/ob_ctype_simple.c
浏览文件 @ bacd5fac
......@@ -814,3 +814,262 @@ size_t ob_strxfrm_pad(const ObCharsetInfo *cs, unsigned char *str, unsigned char
return frm_end - str;
}
size_t ob_caseup_8bit(const ObCharsetInfo *cs __attribute__((unused)),
char* src __attribute__((unused)), size_t srclen __attribute__((unused)),
char* dst __attribute__((unused)), size_t dstlen __attribute__((unused))){
const char *end = src + srclen;
ob_charset_assert(src == dst && srclen == dstlen);
for (; src != end; src++) *src = ob_toupper(cs,*src);
return srclen;
}
size_t ob_casedn_8bit(const ObCharsetInfo *cs __attribute__((unused)),
char* src __attribute__((unused)), size_t srclen __attribute__((unused)),
char* dst __attribute__((unused)), size_t dstlen __attribute__((unused))){
char *end = src + srclen;
ob_charset_assert(src == dst && srclen == dstlen);
for (; src != end; src++) *src = ob_tolower(cs,*src);
return srclen;
}
int ob_strnncoll_simple(const ObCharsetInfo *cs __attribute__((unused)),
const uchar *s, size_t slen,
const uchar *t, size_t tlen,
bool is_prefix)
{
size_t len = (slen > tlen) ? tlen : slen;
if (is_prefix && slen > tlen) slen = tlen;
while (len--) {
if(ob_sort_order(cs,*s)!=ob_sort_order(cs,*t)) {
return (int)ob_sort_order(cs,*s) - (int)ob_sort_order(cs,*t);
}
s++;
t++;
}
return slen > tlen ? 1 : slen < tlen ? -1 : 0;
}
static int ob_strnncollsp_simple(const ObCharsetInfo *cs
__attribute__((unused)),
const uchar *s, size_t slen,
const uchar *t, size_t tlen,
bool diff_if_only_endspace_difference
__attribute__((unused)))
{
size_t len = (slen > tlen) ? tlen : slen;
for (size_t i = 0; i < len; i++){
if(ob_sort_order(cs,*s)!=ob_sort_order(cs,*t)) {
return (int)ob_sort_order(cs,*s) - (int)ob_sort_order(cs,*t);
}
s++;
t++;
}
int res = 0;
if (slen != tlen) {
int swap = 1;
if (diff_if_only_endspace_difference){
res=1;
}
/*
Check the next not space character of the longer key. If it's < ' ',
then it's smaller than the other key.
*/
if (slen < tlen) {
slen = tlen;
s = t;
swap = -1;
res = -res;
}
/*
"a" == "a "
"a\0" < "a"
"a\0" < "a "
*/
for (const unsigned char* end = s + slen - len; s < end; s++) {
if (ob_sort_order(cs,*s) != ob_sort_order(cs,(int)(' ')))
return ob_sort_order(cs,*s) < ob_sort_order(cs,(int)(' ')) ? -swap : swap;
}
}
return res;
}
static size_t ob_strnxfrm_simple(const ObCharsetInfo* cs __attribute__((unused)), unsigned char* dst, size_t dstlen,
uint nweights, const unsigned char* src, size_t srclen, unsigned int flags, bool* is_valid_unicode)
{
uchar *dst0 = dst;
const uchar *end;
const uchar *remainder;
size_t frmlen;
frmlen = dstlen > nweights ? nweights : dstlen;
frmlen = frmlen > srclen ? srclen : frmlen;
end = src + frmlen;
remainder = src + (frmlen % 8);
for (; src < remainder;) *dst++ = ob_sort_order(cs,*src++);
while(src < end) {
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
*dst++ = ob_sort_order(cs,*src++);
}
return ob_strxfrm_pad_desc_and_reverse(cs, dst0, dst, dst0 + dstlen, nweights - srclen, flags, 0);
}
#define likeconv(s, A) (A)
#define INC_PTR(cs, A, B) (A)++
static int ob_wildcmp_8bit_impl(const ObCharsetInfo* cs, const char* str_ptr, const char* str_end_ptr,
const char* wild_str, const char* wild_end, int escape_char, int w_one_char, int w_many_char, int recurse_level)
{
int cmp_result = -1;
while (wild_str != wild_end) {
while (*wild_str != w_many_char && *wild_str != w_one_char) {
if (*wild_str == escape_char && wild_str + 1 != wild_end) {
wild_str++;
}
if (str_ptr == str_end_ptr || likeconv(cs, *wild_str++) != likeconv(cs, *str_ptr++)) {
return 1;
}
if (wild_str == wild_end) {
return str_ptr != str_end_ptr;
}
cmp_result = 1;
}
if (*wild_str == w_one_char) {
do {
if (str_ptr == str_end_ptr) {
return (cmp_result);
}
INC_PTR(cs, str_ptr, str_end_ptr);
} while (++wild_str < wild_end && *wild_str == w_one_char);
if (wild_str == wild_end) {
break;
}
}
if (*wild_str == w_many_char) {
unsigned char cmp = 0;
wild_str++;
for (; wild_str != wild_end; wild_str++) {
if (*wild_str == w_many_char) {
continue;
}
if (*wild_str == w_one_char) {
if (str_ptr == str_end_ptr) {
return (-1);
}
INC_PTR(cs, str_ptr, str_end_ptr);
continue;
}
break;
}
if (wild_str == wild_end) {
return (0);
}
if (str_ptr == str_end_ptr) {
return (-1);
}
if ((cmp = *wild_str) == escape_char && wild_str + 1 != wild_end) {
cmp = *++wild_str;
}
INC_PTR(cs, wild_str, wild_end);
cmp = likeconv(cs, cmp);
do {
while (str_ptr != str_end_ptr && (unsigned char)likeconv(cs, *str_ptr) != cmp) {
str_ptr++;
}
if (str_ptr++ == str_end_ptr) {
return -1;
}
do {
int tmp = ob_wildcmp_8bit_impl(
cs, str_ptr, str_end_ptr, wild_str, wild_end, escape_char, w_one_char, w_many_char, recurse_level + 1);
if (tmp <= 0) {
return tmp;
}
} while (0);
} while (str_ptr != str_end_ptr);
return -1;
}
}
return str_ptr != str_end_ptr ? 1 : 0;
}
int ob_wildcmp_8bit(const ObCharsetInfo* cs, const char* str, const char* str_end, const char* wildstr,
const char* wildend, int escape, int w_one, int w_many)
{
return ob_wildcmp_8bit_impl(cs, str, str_end, wildstr, wildend, escape, w_one, w_many, 1);
}
uint ob_instr_simple(const ObCharsetInfo* cs __attribute__((unused)), const char* b, size_t b_length,
const char* s, size_t s_length, ob_match_t* match, uint nmatch)
{
register const unsigned char *str, *search, *end, *search_end;
if (s_length <= b_length) {
if (!s_length) {
if (nmatch) {
match->beg = 0;
match->end = 0;
match->mb_len = 0;
}
return 1; /* Empty string is always found */
}
str = (const unsigned char*)b;
search = (const unsigned char*)s;
end = (const unsigned char*)b + b_length - s_length + 1;
search_end = (const unsigned char*)s + s_length;
skip:
while (str != end) {
if ((*str++) == (*search)) {
register const unsigned char *i, *j;
i = str;
j = search + 1;
while (j != search_end)
if ((*i++) != (*j++))
goto skip;
if (nmatch > 0) {
match[0].beg = 0;
match[0].end = (size_t)(str - (const unsigned char*)b - 1);
match[0].mb_len = match[0].end;
if (nmatch > 1) {
match[1].beg = match[0].end;
match[1].end = match[0].end + s_length;
match[1].mb_len = match[1].end - match[1].beg;
}
}
return 2;
}
}
}
return 0;
}
ObCollationHandler ob_collation_8bit_simple_ci_handler = {
// NULL /* init */
ob_strnncoll_simple,
ob_strnncollsp_simple,
ob_strnxfrm_simple,
//ob_strnxfrmlen_simple,
ob_like_range_simple,
ob_wildcmp_8bit,
//NULL,//ob_strcasecmp_8bit,
ob_instr_simple,
ob_hash_sort_simple,
ob_propagate_simple};
src/lib/hash/ob_build_in_hashmap.h
浏览文件 @ bacd5fac
......@@ -16,6 +16,7 @@
#include "lib/hash_func/ob_hash_func.h"
#include "lib/hash/ob_hashutils.h"
#include "lib/list/ob_intrusive_list.h"
#include "lib/lock/ob_drw_lock.h"
namespace oceanbase
{
......@@ -240,11 +241,117 @@ public:
return ret;
}
private:
protected:
ObBuildInBucket buckets_[bucket_num];
int64_t count_; // of elements stored in the map.
BucketChain bucket_chain_;
};
template <typename H, int64_t bucket_num = 16>
class ObBuildInHashMapForRefCount : public ObBuildInHashMap <H, bucket_num> {
public:
// Make embedded types easier to use by importing them to the class namespace.
typedef H Hasher; // Rename and promote.
typedef typename Hasher::Key Key; // Key type.
typedef typename Hasher::Value Value; // Stored value (element) type.
typedef typename Hasher::ListHead ListHead; // Anchor for value chain.
ObBuildInHashMapForRefCount() : ObBuildInHashMap<H, bucket_num>() {}
// put a key value pair into HashMap
// @retval OB_SUCCESS for success
// @retval OB_HASH_EXIST when the value's pointer already exist
int set_refactored(Value *value)
{
int ret = OB_SUCCESS;
Key key = Hasher::key(value);
obsys::CWLockGuard wlock(locks_[Hasher::hash(key) % bucket_num]);
ret = ObBuildInHashMap<H, bucket_num>::set_refactored(value);
if (OB_SUCC(ret)) {
Hasher::inc_ref(value);
}
return ret;
}
// put a key value pair into HashMap
// @retval OB_SUCCESS for success
// @retval OB_HASH_EXIST when the value's key already exist
int unique_set(Value *value)
{
int ret = OB_SUCCESS;
Key key = Hasher::key(value);
obsys::CWLockGuard wlock(locks_[Hasher::hash(key) % bucket_num]);
struct ObBuildInHashMap<H, bucket_num>::ObBuildInBucket &bucket = ObBuildInHashMap<H, bucket_num>::buckets_[Hasher::hash(key) % bucket_num];
if (!bucket.chain_.in(value)) {
Value *v = bucket.chain_.head_;
while (NULL != v && ((!Hasher::equal(key, Hasher::key(v)) && (v != value)) || 0 == Hasher::get_ref(v))) {
v = ListHead::next(v);
}
if (NULL == v) {
bucket.chain_.push(value);
++ObBuildInHashMap<H, bucket_num>::count_;
// not empty, put it on the non-empty list.
if (1 == ++(bucket.count_)) {
ObBuildInHashMap<H, bucket_num>::bucket_chain_.push(&bucket);
}
} else {
ret = OB_HASH_EXIST;
}
} else {
ret = OB_HASH_EXIST;
}
if (OB_SUCC(ret)) {
Hasher::inc_ref(value);
}
return ret;
}
// @retval OB_SUCCESS for success
// @retval OB_HASH_NOT_EXIST for key not exist
int get_refactored(Key key, Value *&value)
{
int ret = OB_HASH_NOT_EXIST;
obsys::CRLockGuard rlock(locks_[Hasher::hash(key) % bucket_num]);
const struct ObBuildInHashMap<H, bucket_num>::ObBuildInBucket &bucket = ObBuildInHashMap<H, bucket_num>::buckets_[Hasher::hash(key) % bucket_num];
Value *v = bucket.chain_.head_;
while (NULL != v && (!Hasher::equal(key, Hasher::key(v)) || 0 == Hasher::get_ref(v))) {
v = ListHead::next(v);
}
if (NULL != (value = v)) {
ret = OB_SUCCESS;;
}
if (OB_SUCC(ret)) {
while (true) {
int64_t ref = Hasher::get_ref(v);
if(ref > 0) {
if (!Hasher::bcas_ref(value, ref, ref + 1)) {
PAUSE();
} else {
break;
}
} else {
ret = OB_HASH_NOT_EXIST;
value = NULL;
break;
}
}
}
return ret;
}
void remove(Value* value)
{
if (NULL != value) {
obsys::CWLockGuard wlock(locks_[Hasher::hash(Hasher::key(value)) % bucket_num]);
ObBuildInHashMap<H, bucket_num>::remove(value);
Hasher::destroy(value);
}
}
protected:
mutable obsys::CRWLock locks_[bucket_num];
};
} // namespace hash
} // namespace common
} // namespace oceanbase
......
src/lib/lock/ob_bucket_lock.h
浏览文件 @ bacd5fac
......@@ -23,8 +23,10 @@ namespace common
class ObBucketLock
{
public:
ObBucketLock();
virtual ~ObBucketLock();
/* this function is defined for c driver client compile */
ObBucketLock() {}
/* this function is defined for c driver client compile */
virtual ~ObBucketLock() {}
int init(
const uint64_t bucket_cnt,
const uint32_t latch_id = ObLatchIds::DEFAULT_BUCKET_LOCK,
......
src/lib/lock/ob_latch.cpp
浏览文件 @ bacd5fac
......@@ -279,7 +279,10 @@ int ObLatchWaitQueue::try_lock(
COMMON_LOG(ERROR, "Fail to add proc to wait list, ", K(ret));
}
}
proc.wait_ = 1;
if (OB_EAGAIN == ret) {
proc.wait_ = 1;
}
}
unlock_bucket(bucket);
......
src/lib/net/ob_addr.cpp
浏览文件 @ bacd5fac
......@@ -28,29 +28,49 @@ namespace common
// --------------------------------------------------------
// class ObAddr implements
// --------------------------------------------------------
uint32_t ObAddr::convert_ipv4_addr(const char *ip)
int ObAddr::convert_ipv4_addr(const char *ip)
{
in_addr binary;
int iret = 0;
uint32_t result = 0;
int ret = OB_SUCCESS;
in_addr in;
if (!OB_ISNULL(ip)) {
memset(&binary, 0, sizeof (binary));
iret = inet_pton(AF_INET, ip, &binary);
if (iret == -1) { // no support family
binary.s_addr = 0;
} else if (iret == 0) { // invalid ip string
binary.s_addr = 0;
MEMSET(&in, 0, sizeof (in));
int rt = inet_pton(AF_INET, ip, &in);
if (rt != 1) { // wrong ip or error
in.s_addr = 0;
ret = OB_ERR_UNEXPECTED;
LOG_WARN("convert ipv4 addr failed", K(ip));
} else {
ip_.v4_ = ntohl(in.s_addr);
}
result = ntohl(binary.s_addr);
}
return result;
return ret;
}
int ObAddr::convert_ipv6_addr(const char *ip)
{
int ret = OB_SUCCESS;
in6_addr in6;
if (!OB_ISNULL(ip)) {
memset(&in6, 0, sizeof(in6));
ret = inet_pton(AF_INET6, ip, &in6);
if (ret != 1) {
memset(&in6, 0, sizeof(in6));
ret = OB_ERR_UNEXPECTED;
LOG_WARN("convert ipv6 addr failed", K(ip));
} else {
ret = OB_SUCCESS;
// Stored here in big-endian format
MEMCPY(ip_.v6_, in6.s6_addr, sizeof(ip_.v6_));
}
}
return ret;
}
int ObAddr::parse_from_cstring(const char *ipport)
{
int ret = OB_SUCCESS;
char buf[INET6_ADDRSTRLEN] = "";
char buf[MAX_IP_ADDR_LENGTH] = "";
int port = 0;
if (!OB_ISNULL(ipport)) {
......@@ -73,15 +93,16 @@ int ObAddr::parse_from_cstring(const char *ipport)
if (OB_SUCC(ret)) {
if ('[' != buf[0]) { // IPV4 format
if (false == set_ipv4_addr(buf, port))
{
if (false == set_ipv4_addr(buf, port)) {
ret = OB_INVALID_ARGUMENT;
}
} else { // IPV6 format
const char *ipv6 = buf + 1;
if (']' == buf[strlen(buf) - 1]) {
buf[strlen(buf) - 1] = '\0';
IGNORE_RETURN set_ipv6_addr(ipv6, port);
if (!set_ipv6_addr(ipv6, port)) {
ret = OB_INVALID_ARGUMENT;
}
} else {
ret = OB_INVALID_ARGUMENT;
}
......@@ -109,6 +130,17 @@ int64_t ObAddr::to_string(char *buffer, const int64_t size) const
(ip_.v4_ >> 8) & 0xFF,
(ip_.v4_) & 0xFF);
}
} else if (version_ == IPV6) {
char buf[MAX_IP_ADDR_LENGTH];
struct in6_addr in6;
memset(buf, 0, sizeof(buf));
MEMCPY(in6.s6_addr, ip_.v6_, sizeof(ip_.v6_));
inet_ntop(AF_INET6, &in6, buf, MAX_IP_ADDR_LENGTH);
if (port_ > 0) {
databuff_printf(buffer, size, pos, "\"[%s]:%d\"", buf, port_);
} else {
databuff_printf(buffer, size, pos, "\"%s\"", buf);
}
}
}
return pos;
......@@ -124,6 +156,13 @@ bool ObAddr::ip_to_string(char *buffer, const int32_t size) const
(ip_.v4_ >> 16) & 0xFF,
(ip_.v4_ >> 8) & 0xFF,
(ip_.v4_) & 0xFF);
} else if (version_ == IPV6) {
char buf[MAX_IP_ADDR_LENGTH];
struct in6_addr in6;
memset(buf, 0, sizeof(buf));
MEMCPY(in6.s6_addr, ip_.v6_, sizeof(ip_.v6_));
inet_ntop(AF_INET6, &in6, buf, MAX_IP_ADDR_LENGTH);
snprintf(buffer, size, "%s", buf);
}
res = true;
}
......@@ -133,58 +172,74 @@ bool ObAddr::ip_to_string(char *buffer, const int32_t size) const
int ObAddr::ip_port_to_string(char *buffer, const int32_t size) const
{
int ret = OB_SUCCESS;
int ret_len = 0;
if (NULL == buffer || size <= 0) {
ret = OB_INVALID_ARGUMENT;
} else if (version_ == IPV6) {
ret = OB_NOT_SUPPORTED;
char buf[MAX_IP_ADDR_LENGTH];
struct in6_addr in6;
memset(buf, 0, sizeof(buf));
MEMCPY(in6.s6_addr, ip_.v6_, sizeof(ip_.v6_));
inet_ntop(AF_INET6, &in6, buf, MAX_IP_ADDR_LENGTH);
ret_len = snprintf(buffer, size, "[%s]:%d", buf, port_);
} else {
int ret_len = snprintf(buffer, size, "%d.%d.%d.%d:%d",
ret_len = snprintf(buffer, size, "%d.%d.%d.%d:%d",
(ip_.v4_ >> 24) & 0XFF,
(ip_.v4_ >> 16) & 0xFF,
(ip_.v4_ >> 8) & 0xFF,
(ip_.v4_) & 0xFF,
port_);
if (ret_len < 0) {
ret = OB_ERR_SYS;
} else if (ret_len >= size) {
ret = OB_SIZE_OVERFLOW;
}
}
if (ret_len < 0) {
ret = OB_ERR_SYS;
} else if (ret_len >= size) {
ret = OB_SIZE_OVERFLOW;
}
return ret;
}
bool ObAddr::set_ipv6_addr(const char *ip, const int32_t port)
{
UNUSED(ip);
UNUSED(port);
_OB_LOG(WARN, "set ipv6 address is not complete");
return false;
}
bool bret = true;
int ret = OB_SUCCESS;
if (NULL == ip || port < 0) {
bret = false;
} else if (OB_FAIL(convert_ipv6_addr(ip))) {
bret = false;
} else {
version_ = IPV6;
port_ = port;
}
return bret;
}
bool ObAddr::set_ipv4_addr(const char *ip, const int32_t port)
{
bool ret = true;
if (NULL == ip || port <= 0) {
ret = false;
bool bret = true;
int ret = OB_SUCCESS;
if (NULL == ip || port < 0) {
bret = false;
} else if (OB_FAIL(convert_ipv4_addr(ip))) {
bret = false;
} else {
version_ = IPV4;
ip_.v4_ = convert_ipv4_addr(ip);
port_ = port;
}
return ret;
return bret;
}
bool ObAddr::set_ipv4_addr(const ObString &ip, const int32_t port)
bool ObAddr::set_ip_addr(const ObString &ip, const int32_t port)
{
bool ret = true;
char ip_buf[OB_IP_STR_BUFF] = "";
if (ip.length() >= OB_IP_STR_BUFF) {
char ip_buf[MAX_IP_ADDR_LENGTH] = "";
if (ip.length() >= MAX_IP_ADDR_LENGTH) {
ret = false;
} else {
// ObString may be not terminated by '\0'
MEMCPY(ip_buf, ip.ptr(), ip.length());
ret = set_ipv4_addr(ip_buf, port);
ret = set_ip_addr(ip_buf, port);
}
return ret;
}
......@@ -194,8 +249,12 @@ bool ObAddr::is_valid() const
bool valid = true;
if (port_ <= 0) {
valid = false;
} else if (IPV4 == version_) {
valid = (0 != ip_.v4_);
} else if (IPV6 == version_) {
valid = ((0 != ip_.v6_[0]) || (0 != ip_.v6_[1]) || (0 != ip_.v6_[2]) || (0 != ip_.v6_[3]));
} else {
valid = (IPV4 == version_) && (0 != ip_.v4_);
valid = false;
}
return valid;
}
......@@ -235,10 +294,17 @@ void ObAddr::set_ipv4_server_id(const int64_t ipv4_server_id)
bool ObAddr::operator <(const ObAddr &rv) const
{
if ((version_ != rv.version_) || (IPV4 != version_)) {
int64_t ipcmp = 0;
if (version_ != rv.version_) {
LOG_ERROR("comparision between different IP versions hasn't supported!");
} else if (IPV4 == version_) {
ipcmp = static_cast<int64_t>(ip_.v4_) - static_cast<int64_t>(rv.ip_.v4_);
} else if (IPV6 == version_) {
int pos = 0;
for (; ipcmp == 0 && pos < IPV6_LEN; pos++) {
ipcmp = ip_.v6_[pos] - rv.ip_.v6_[pos];
}
}
int ipcmp = ip_.v4_ - rv.ip_.v4_;
return (ipcmp < 0) || (0 == ipcmp && port_ < rv.port_);
}
......@@ -271,6 +337,68 @@ void ObAddr::set_port(int32_t port)
port_ = port;
}
struct sockaddr_storage ObAddr::get_sockaddr() const
{
struct sockaddr_storage sock_addr;
memset(&sock_addr, 0, sizeof(struct sockaddr_storage));
if (version_ == IPV4) {
struct sockaddr_in in;
memset(&in, 0, sizeof(struct sockaddr_in));
in.sin_family = AF_INET;
in.sin_port = (htons)(static_cast<uint16_t>(port_));
in.sin_addr.s_addr = htonl(ip_.v4_);
MEMCPY(&sock_addr, &in, sizeof(in));
} else if (version_ == IPV6) {
struct sockaddr_in6 in6;
memset(&in6, 0, sizeof(struct sockaddr_in6));
in6.sin6_family = AF_INET6;
in6.sin6_port = (htons)(static_cast<uint16_t>(port_));
MEMCPY(in6.sin6_addr.s6_addr, ip_.v6_, sizeof(ip_.v6_));
MEMCPY(&sock_addr, &in6, sizeof(in6));
}
return sock_addr;
}
void ObAddr::set_sockaddr(const struct sockaddr &sock_addr)
{
if (sock_addr.sa_family == AF_INET) {
version_ = IPV4;
const struct sockaddr_in &in = (struct sockaddr_in&)(sock_addr);
port_ = ntohs(in.sin_port);
ip_.v4_ = ntohl(in.sin_addr.s_addr);
} else if (sock_addr.sa_family == AF_INET6) {
version_ = IPV6;
const struct sockaddr_in6 &in6 = (struct sockaddr_in6&)(sock_addr);
port_ = ntohs(in6.sin6_port);
MEMCPY(ip_.v6_, in6.sin6_addr.s6_addr, sizeof(ip_.v6_));
}
}
bool ObAddr::set_ip_addr(const char *ip, const int32_t port)
{
bool bret = false;
const char *ch = strchr(ip, ':');
if (NULL != ch) {
bret = set_ipv6_addr(ip, port);
} else {
bret = set_ipv4_addr(ip, port);
}
return bret;
}
bool ObAddr::set_ipv6_addr(const uint64_t ipv6_high, const uint64_t ipv6_low, const int32_t port)
{
bool ret = true;
uint64_t *high = reinterpret_cast<uint64_t *>(&ip_.v6_[0]);
uint64_t *low = reinterpret_cast<uint64_t *>(&ip_.v6_[8]);
*high = ipv6_high;
*low = ipv6_low;
version_ = IPV6;
port_= port;
return ret;
}
OB_SERIALIZE_MEMBER(ObAddr, version_, ip_.v6_[0], ip_.v6_[1], ip_.v6_[2], ip_.v6_[3], port_);
} // end namespace common
......
src/lib/net/ob_addr.h
浏览文件 @ bacd5fac
......@@ -19,28 +19,33 @@
#include "lib/container/ob_se_array.h"
#include "lib/json/ob_yson.h"
#include "lib/ob_name_id_def.h"
#include "iocore/net/ob_inet.h"
namespace oceanbase
{
namespace common
{
#define IPV6_LEN 16
class ObAddr
{
OB_UNIS_VERSION(1);
public:
enum VER {
IPV4 = 4, IPV6 = 6
IPV4 = 4,
IPV6 = 6,
IPINVALID,
};
ObAddr()
: version_(IPV4), ip_(), port_(0)
: version_(IPINVALID), ip_(), port_(0)
{
memset(&ip_, 0, sizeof(ip_));
}
ObAddr(VER version, const char *ip, const int32_t port)
: version_(IPV4), ip_(), port_(0)
: version_(IPINVALID), ip_(), port_(0)
{
memset(&ip_, 0, sizeof(ip_));
if (version == IPV4) {
......@@ -50,15 +55,20 @@ public:
}
}
// The server_id format cannot be used in the case of IPV6
// TODO: Consider removing it
explicit ObAddr(const int64_t ipv4_server_id)
: version_(IPV4), ip_(), port_(0)
: version_(IPINVALID), ip_(), port_(0)
{
// server_id only supports IPV4 format
version_ = IPV4;
ip_.v4_ = static_cast<int32_t>(0x00000000ffffffff & (ipv4_server_id >> 32));
port_ = static_cast<int32_t>(0x00000000ffffffff & ipv4_server_id);
}
void reset()
{
version_ = IPINVALID;
port_ = 0;
memset(&ip_, 0, sizeof (ip_));
}
......@@ -68,7 +78,10 @@ public:
return (IPV4 == version_ && INADDR_LOOPBACK == ip_.v4_)
|| (IPV6 == version_ && IN6_IS_ADDR_LOOPBACK(ip_.v6_));
}
static uint32_t convert_ipv4_addr(const char *ip);
int convert_ipv4_addr(const char *ip);
int convert_ipv6_addr(const char *ip);
struct sockaddr_storage get_sockaddr() const;
void set_sockaddr(const struct sockaddr &sock_addr);
int64_t to_string(char *buffer, const int64_t size) const;
bool ip_to_string(char *buffer, const int32_t size) const;
......@@ -76,10 +89,12 @@ public:
TO_YSON_KV(ID(ip), ip_.v4_,
ID(port), port_);
bool set_ip_addr(const char *ip, const int32_t port);
bool set_ip_addr(const ObString &ip, const int32_t port);
bool set_ipv6_addr(const char *ip, const int32_t port);
bool set_ipv6_addr(const uint64_t ipv6_high, const uint64_t ipv6_low, const int32_t port);
bool set_ipv4_addr(const char *ip, const int32_t port);
bool set_ipv4_addr(const uint32_t ip, const int32_t port);
bool set_ipv4_addr(const ObString &ip, const int32_t port);
int parse_from_cstring(const char *ip_str);
int64_t get_ipv4_server_id() const;
......@@ -89,6 +104,7 @@ public:
bool operator !=(const ObAddr &rv) const;
bool operator ==(const ObAddr &rv) const;
bool operator < (const ObAddr &rv) const;
ObAddr& operator = (const ObAddr &rv);
int compare(const ObAddr &rv) const;
bool is_equal_except_port(const ObAddr &rv) const;
inline int32_t get_version() const { return version_; }
......@@ -106,6 +122,8 @@ public:
VER version_;
union
{
// v4 addresses are stored in native byte order,
// v6 addresses are stored in network byte order
uint32_t v4_; //host byte order
uint32_t v6_[4];
} ip_;
......@@ -139,6 +157,16 @@ bool ObAddr::operator ==(const ObAddr &rv) const
&& ip_.v6_[2] == rv.ip_.v6_[2] && ip_.v6_[3] == rv.ip_.v6_[3];
}
ObAddr& ObAddr::operator =(const ObAddr &rv)
{
if (this != &rv) {
version_ = rv.version_;
port_ = rv.port_;
MEMCPY(ip_.v6_, rv.ip_.v6_, sizeof(ip_.v6_));
}
return *this;
}
int ObAddr::compare(const ObAddr &rv) const
{
return memcmp(this, &rv, sizeof(ObAddr));
......
src/lib/ob_define.h
浏览文件 @ bacd5fac
......@@ -53,7 +53,7 @@ const int64_t OB_MAX_CLUSTER_ID = 4294901759;
const int64_t OB_INVALID_CLUSTER_ID = -1;
const int64_t OB_INVALID_ORG_CLUSTER_ID = 0;
const int64_t OB_MAX_ITERATOR = 16;
const int64_t MAX_IP_ADDR_LENGTH = 32;
const int64_t MAX_IP_ADDR_LENGTH = 64;
const int64_t MAX_IP_PORT_LENGTH = MAX_IP_ADDR_LENGTH + 5;
const int64_t MAX_IP_PORT_SQL_LENGTH = MAX_IP_ADDR_LENGTH + 10;
const int64_t OB_MAX_SQL_ID_LENGTH = 32;
......@@ -178,7 +178,6 @@ const int64_t OB_MAX_BATCH_NUMBER = 100;
const int64_t OB_MAX_TABLET_LIST_NUMBER = 64;
const int64_t OB_MAX_DISK_NUMBER = 16; // must no more than ObTimer::MAX_TASK_NUM
const int64_t OB_MAX_TIME_STR_LENGTH = 64;
const int64_t OB_IP_STR_BUFF = 30; //TODO: uniform IP/PORR length
const int64_t OB_IP_PORT_STR_BUFF = 64;
const int64_t OB_RANGE_STR_BUFSIZ = 512;
const int64_t OB_MAX_FETCH_CMD_LENGTH = 2048;
......@@ -573,6 +572,9 @@ static const char *const OB_MYSQL_CLIENT_JDBC_CLIENT_MODE = "__ob_jdbc_client";
// for java client
static const char *const OB_MYSQL_JAVA_CLIENT_MODE_NAME = "__ob_java_client";
// conn attr which transparent transit to observer
static const char *const OB_MYSQL_OB_CLIENT = "__ob_client";
// for obproxy and observer compatibility
enum ObCapabilityFlagShift
{
......@@ -592,6 +594,7 @@ enum ObCapabilityFlagShift
OB_CAP_PROXY_SESSION_SYNC_SHIFT, // 13
OB_CAP_PROXY_FULL_LINK_TRACING_SHIFT, // 14
OB_CAP_PROXY_NEW_EXTRA_INFO_SHIFT, // 15
OB_CAP_PROXY_SESSION_VAR_SYNC_SHIFT, // 16
};
#define OB_TEST_CAPABILITY(cap, tg_cap) (((cap) & (tg_cap)) == (tg_cap))
......@@ -610,6 +613,7 @@ enum ObCapabilityFlagShift
#define OB_CAP_PROXY_SESSION_SYNC OB_CAP_GET_TYPE(common::OB_CAP_PROXY_SESSION_SYNC_SHIFT)
#define OB_CAP_PROXY_FULL_LINK_TRACING OB_CAP_GET_TYPE(common::OB_CAP_PROXY_FULL_LINK_TRACING_SHIFT)
#define OB_CAP_PROXY_NEW_EXTRA_INFO OB_CAP_GET_TYPE(common::OB_CAP_PROXY_NEW_EXTRA_INFO_SHIFT)
#define OB_CAP_PROXY_SESSION_VAR_SYNC OB_CAP_GET_TYPE(common::OB_CAP_PROXY_SESSION_VAR_SYNC_SHIFT)
// for obproxy debug
......
src/lib/ob_errno.cpp
浏览文件 @ bacd5fac
......@@ -8,11 +8,6 @@
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*
* *************************************************************
*
* DO NOT EDIT. This file is automatically generated from `ob_errno.def'.
*
*/
#include "ob_errno.h"
......@@ -2657,6 +2652,14 @@ static struct ObStrErrorInit
SQLSTATE[-OB_TRANS_RPC_TIMEOUT] = "25000";
STR_ERROR[-OB_TRANS_RPC_TIMEOUT] = "transaction rpc timeout";
STR_USER_ERROR[-OB_TRANS_RPC_TIMEOUT] = "transaction rpc timeout";
// in transaction internal routing, if sql not supported to be executed on participant sent to participant
// server return OB_TRANS_FREE_ROUTE_NOT_SUPPORTED
// proxy downgrade transaction internal routing and send the rest of sql to coordinator
MYSQL_ERRNO[-OB_TRANS_FREE_ROUTE_NOT_SUPPORTED] = -1;
SQLSTATE[-OB_TRANS_FREE_ROUTE_NOT_SUPPORTED] = "HY000";
STR_ERROR[-OB_TRANS_FREE_ROUTE_NOT_SUPPORTED] = "Query is not supported to be executed on txn participant node";
STR_USER_ERROR[-OB_TRANS_FREE_ROUTE_NOT_SUPPORTED] = "Query is not supported to be executed on txn participant node";
MYSQL_ERRNO[-OB_LOG_ID_NOT_FOUND] = -1;
SQLSTATE[-OB_LOG_ID_NOT_FOUND] = "HY000";
STR_ERROR[-OB_LOG_ID_NOT_FOUND] = "log id not found";
......@@ -2883,8 +2886,8 @@ static struct ObStrErrorInit
STR_USER_ERROR[-OB_ERR_NULL_DB_VAL_TESTLOAD_TABLE_MAP] = "Null database variable for testload table map to execute testload query";
MYSQL_ERRNO[-OB_ERR_BATCH_INSERT_FOUND] = -1;
SQLSTATE[-OB_ERR_BATCH_INSERT_FOUND] = "HY000";
STR_ERROR[-OB_ERR_BATCH_INSERT_FOUND] = "Batch insert is not supported for sharding table";
STR_USER_ERROR[-OB_ERR_BATCH_INSERT_FOUND] = "Batch insert is not supported for sharding table";
STR_ERROR[-OB_ERR_BATCH_INSERT_FOUND] = "Batch insert is not supported";
STR_USER_ERROR[-OB_ERR_BATCH_INSERT_FOUND] = "Batch insert is not supported";
MYSQL_ERRNO[-OB_ERR_UNSUPPORT_DIFF_TOPOLOGY] = 7202;
SQLSTATE[-OB_ERR_UNSUPPORT_DIFF_TOPOLOGY] = "HY000";
STR_ERROR[-OB_ERR_UNSUPPORT_DIFF_TOPOLOGY] = "Not support different topology table join";
......
src/lib/ob_errno.h
浏览文件 @ bacd5fac
......@@ -8,10 +8,6 @@
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*
* *************************************************************
*
* DO NOT EDIT. This file is automatically generated from `ob_errno.def'.
*/
#ifndef OCEANBASE_LIB_OB_ERRNO_H_
......@@ -23,6 +19,7 @@ namespace oceanbase
{
namespace common
{
static const int OB_INTERNAL_ERROR = -600;
static const int OB_ORA_FATAL_ERROR = -603;
static const int64_t OB_MAX_ERROR_CODE = 10000;
static const int OB_LAST_ERROR_CODE = -8003;
......@@ -704,6 +701,7 @@ namespace common
static const int OB_PARTITION_IS_STOPPED = -6228;
static const int OB_PARTITION_IS_BLOCKED = -6229;
static const int OB_TRANS_RPC_TIMEOUT = -6230;
static const int OB_TRANS_FREE_ROUTE_NOT_SUPPORTED = -6279;
static const int OB_LOG_ID_NOT_FOUND = -6301;
static const int OB_LSR_THREAD_STOPPED = -6302;
static const int OB_NO_LOG = -6303;
......@@ -1425,6 +1423,7 @@ namespace common
#define OB_PARTITION_IS_STOPPED__USER_ERROR_MSG "Partition has been stopped"
#define OB_PARTITION_IS_BLOCKED__USER_ERROR_MSG "Partition has been blocked"
#define OB_TRANS_RPC_TIMEOUT__USER_ERROR_MSG "transaction rpc timeout"
#define OB_TRANS_FREE_ROUTE_NOT_SUPPORTED__USER_ERROR_MSG "Query is not supported to be executed on txn participant node"
#define OB_LOG_ID_NOT_FOUND__USER_ERROR_MSG "log id not found"
#define OB_LSR_THREAD_STOPPED__USER_ERROR_MSG "log scan runnable thread stop"
#define OB_NO_LOG__USER_ERROR_MSG "no log ever scanned"
......
src/lib/ob_name_id_def.h
浏览文件 @ bacd5fac
......@@ -334,7 +334,8 @@ enum ObNameId
};
// get name at runtime
const char* get_name(int32_t id);
/* this function is defined for c driver client compile */
inline const char* get_name(int32_t id) {((void)(id)); return nullptr;};
const char* get_description(int32_t id);
} // end namespace name_id_map
} // end namespace oceanbase
......
src/lib/oblog/ob_async_log_struct.h
浏览文件 @ bacd5fac
......@@ -38,6 +38,7 @@ enum ObLogFDType {
FD_POOL_FILE,
FD_POOL_STAT_FILE,
FD_TRACE_FILE, // ob20 full link trace interface
FD_DRIVER_CLIENT_FILE,
MAX_FD_FILE,
};
......
src/lib/oblog/ob_log.cpp
浏览文件 @ bacd5fac
......@@ -375,7 +375,7 @@ ObLogger::~ObLogger()
(void)pthread_mutex_destroy(&file_size_mutex_);
}
void ObLogger::destory_async_log_thread()
void ObLogger::destroy_async_log_thread()
{
set_enable_async_log(false);
set_stop_flush();
......
src/lib/oblog/ob_log.h
浏览文件 @ bacd5fac
......@@ -283,7 +283,7 @@ private:
public:
~ObLogger();
void destory_async_log_thread();
void destroy_async_log_thread();
int init_async_log_thread(const int64_t stacksize = 0);
void destroy_free_litem_queue();
......
src/lib/oblog/ob_log_module.h
浏览文件 @ bacd5fac
......@@ -261,6 +261,11 @@ LOG_MOD_END(STORAGETEST)
#define _OBPROXY_TRACE_LOG(level, _fmt_, args...) \
(OB_MONITOR_LOG_NEED_TO_PRINT(level) ? _OB_PRINT_TYPE(FD_TRACE_FILE, NULL, level, _fmt_, ##args) : (void) 0)
#define OBPROXY_DRIVER_CLIENT_LOG(level, infoString, args...) \
(OB_LOG_NEED_TO_PRINT(level) ? OB_PRINT_TYPE(FD_DRIVER_CLIENT_FILE, NULL, level, infoString, ##args) : (void) 0)
#define _OBPROXY_DRIVER_CLIENT_LOG(level, _fmt_, args...) \
(OB_LOG_NEED_TO_PRINT(level) ? _OB_PRINT_TYPE(FD_DRIVER_CLIENT_FILE, NULL, level, _fmt_, ##args) : (void) 0)
#define _OB_NUM_LEVEL_LOG(level, _fmt_, args...) \
(OB_LOGGER.need_to_print(level) ? \
......
src/lib/oblog/ob_log_print_kv.h
浏览文件 @ bacd5fac
......@@ -21,6 +21,7 @@
#include "lib/utility/ob_macro_utils.h"
#include "lib/utility/ob_template_utils.h"
#include "lib/alloc/alloc_assist.h"
#include "obproxy/ob_proxy_init.h"
#define LOG_N_TRUE "true"
#define LOG_N_FLASE "false"
......@@ -37,8 +38,8 @@ int logdata_printf(char *buf, const int64_t buf_len, int64_t &pos, const char *f
__attribute__((format(printf, 4, 5)));
int logdata_vprintf(char *buf, const int64_t buf_len, int64_t &pos, const char *fmt, va_list args);
#define LOG_STDERR(...) do {if(isatty(STDERR_FILENO)) {fprintf(stderr, __VA_ARGS__); }} while(0)
#define LOG_STDOUT(...) do {if(isatty(STDOUT_FILENO)) {fprintf(stdout, __VA_ARGS__); }} while(0)
#define LOG_STDERR(...) do { if(oceanbase::obproxy::RUN_MODE_PROXY == oceanbase::obproxy::g_run_mode && isatty(STDERR_FILENO)) {fprintf(stderr, __VA_ARGS__); }} while(0)
#define LOG_STDOUT(...) do {if(oceanbase::obproxy::RUN_MODE_PROXY == oceanbase::obproxy::g_run_mode && isatty(STDOUT_FILENO)) {fprintf(stdout, __VA_ARGS__); }} while(0)
//print errmsg of errno.As strerror not thread-safe, need
//to call ERRMSG, KERRMSG which use this class.
......@@ -370,6 +371,46 @@ int logdata_print_kv(char *buf, const int64_t buf_len, int64_t &pos, const char
return logdata_print_key_obj(buf, buf_len, pos, key, false, obj);
}
class ObILogKV
{
public:
virtual int print(char *buf, int64_t buf_len, int64_t &pos, const bool with_comma) const = 0;
};
template<typename T, const bool is_rvalue>
class ObLogKV;
template<typename T>
class ObLogKV<T, false> : public ObILogKV
{
public:
ObLogKV(const char *key, const T &value) : key_(key), value_(value) {}
int print(char *buf, int64_t buf_len, int64_t &pos, const bool with_comma) const override
{
return logdata_print_key_obj(buf, buf_len, pos, key_,
with_comma, value_);
}
private:
const char *key_;
const T &value_;
};
template<typename T> T& unmove(T&& v) { return v; }
template<typename T>
class ObLogKV<T, true> : public ObILogKV
{
public:
ObLogKV(const char *key, const T &&value) : key_(key), value_(unmove(value)) {}
int print(char *buf, int64_t buf_len, int64_t &pos, const bool with_comma) const override
{
return logdata_print_key_obj(buf, buf_len, pos, key_,
with_comma, value_);
}
private:
const char *key_;
const T &value_;
};
}
}
......@@ -679,4 +720,35 @@ int logdata_print_kv(char *buf, const int64_t buf_len, int64_t &pos, const char
log_buffer.check_and_unlock(); \
}
#define LOG_KV(key, obj) \
(::oceanbase::common::ObILogKV&&)::oceanbase::common::ObLogKV<decltype(obj), \
std::is_rvalue_reference<decltype((obj))&&>::value>(key, obj)
#define LOG_KVS_0()
#define LOG_KVS_1()
#define LOG_KVS_2(key, obj) , LOG_KV(key, obj)
#define LOG_KVS_4(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_2(args)
#define LOG_KVS_6(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_4(args)
#define LOG_KVS_8(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_6(args)
#define LOG_KVS_10(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_8(args)
#define LOG_KVS_12(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_10(args)
#define LOG_KVS_14(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_12(args)
#define LOG_KVS_16(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_14(args)
#define LOG_KVS_18(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_16(args)
#define LOG_KVS_20(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_18(args)
#define LOG_KVS_22(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_20(args)
#define LOG_KVS_24(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_22(args)
#define LOG_KVS_26(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_24(args)
#define LOG_KVS_28(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_26(args)
#define LOG_KVS_30(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_28(args)
#define LOG_KVS_32(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_30(args)
#define LOG_KVS_34(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_32(args)
#define LOG_KVS_36(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_34(args)
#define LOG_KVS_38(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_36(args)
#define LOG_KVS_40(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_38(args)
#define LOG_KVS_42(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_40(args)
#define LOG_KVS_44(key, obj, args...) , LOG_KV(key, obj) LOG_KVS_42(args)
#define LOG_KVS_(N, ...) CONCAT(LOG_KVS_, N)(__VA_ARGS__)
#define LOG_KVS(...) "placeholder" LOG_KVS_(ARGS_NUM(__VA_ARGS__), __VA_ARGS__)
#endif
src/lib/oblog/ob_simple_trace.h 0 → 100644
浏览文件 @ bacd5fac
/**
* Copyright (c) 2021 OceanBase
* OceanBase CE is licensed under Mulan PubL v2.
* You can use this software according to the terms and conditions of the Mulan PubL v2.
* You may obtain a copy of Mulan PubL v2 at:
* http://license.coscl.org.cn/MulanPubL-2.0
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*/
#ifndef _OB_SIMPLE_TRACE_H
#define _OB_SIMPLE_TRACE_H
#include "lib/oblog/ob_log.h"
namespace oceanbase
{
namespace common
{
template<int64_t BUFFER_SIZE>
class ObSimpleTraceBase
{
public:
ObSimpleTraceBase() { reset(); }
~ObSimpleTraceBase() { }
void reset()
{
pos_ = 0;
buf_[0] = '\0';
}
template <typename ... Args>
int log_it(const char *info, Args const & ... args)
{
int ret = common::OB_SUCCESS;
const int64_t saved_pos = pos_;
const char delimiter1[] = " | ";
const char delimiter2[] = "... | ";
if (OB_FAIL(fill_buffer(buf_, BUFFER_SIZE - sizeof(delimiter2), pos_, info, args...))) {
if ((BUFFER_SIZE - saved_pos) >= BUFFER_SIZE / 2) {
// rewrite ret
ret = OB_SUCCESS;
(void)databuff_printf(buf_, BUFFER_SIZE, pos_, delimiter2);
}
} else {
(void)databuff_printf(buf_, BUFFER_SIZE, pos_, delimiter1);
}
if (OB_FAIL(ret)) {
pos_ = saved_pos;
buf_[pos_] = '\0';
}
return ret;
}
int64_t to_string(char *buf, const int64_t buf_len) const
{
int64_t pos = 0;
databuff_printf(buf, buf_len, pos, "%s", buf_);
return pos;
}
private:
int fill_kv(char *buf, const int64_t buf_len, int64_t &pos,
const bool with_comma)
{
UNUSED(buf);
UNUSED(buf_len);
UNUSED(pos);
UNUSED(with_comma);
return common::OB_SUCCESS;
}
int fill_kv(char *buf, const int64_t buf_len, int64_t &pos,
const bool with_comma,
const ObILogKV &kv)
{
return kv.print(buf, buf_len, pos, with_comma);
}
template <typename Key, typename Value, typename ... Args>
int fill_kv(char *buf, const int64_t buf_len, int64_t &pos,
const bool with_comma,
const Key &key,
const Value &value,
Args const & ... args)
{
int ret = OB_SUCCESS;
ret = fill_kv(buf, buf_len, pos, with_comma, LOG_KV(key, value));
if (OB_SUCC(ret)) {
ret = fill_kv(buf, buf_len, pos, 1, args...);
}
return ret;
}
template <typename ... Args>
int fill_buffer(char *data, const int64_t buf_len,
int64_t &pos,
const char *info,
Args const & ... args)
{
int ret = OB_SUCCESS;
int64_t tmp_pos = pos;
ret = databuff_printf(data, buf_len, pos, "%s ", info);
if (OB_SUCC(ret)) {
ret = fill_kv(data, buf_len, pos, 0, args...);
}
if (OB_SUCC(ret) && OB_UNLIKELY(OB_LOGGER.get_log_level() >= OB_LOG_LEVEL_TRACE)) {
char tmp_buf[1024];
memset(tmp_buf, 0, sizeof(tmp_buf));
int64_t copy_len = pos - tmp_pos > 1023 ? 1023 : pos - tmp_pos;
MEMCPY(tmp_buf, data + tmp_pos, copy_len);
OB_LOG(TRACE, "trace_log", K(tmp_buf));
}
return ret;
}
private:
char buf_[BUFFER_SIZE];
int64_t pos_;
};
template<int64_t TRACE_SIZE>
class ObSimpleTrace
{
public:
ObSimpleTrace() { reset(); }
~ObSimpleTrace() { }
void reset()
{
first_idx_ = 0;
cur_idx_ = 0;
need_print_ = false;
for (int64_t i = 0; i < BASE_TRACE_CNT; i++) {
traces_[i].reset();
}
}
template <typename ... Args>
void log_it(const char *info, Args const & ... args)
{
if (need_print_) {
int ret = common::OB_SUCCESS;
for (int64_t i = 0; i < 2; i++) {
cur_idx_ = (cur_idx_ + i) % BASE_TRACE_CNT;
if (1 == i) {
if (cur_idx_ == first_idx_) {
if (OB_UNLIKELY(OB_LOGGER.get_log_level() >= OB_LOG_LEVEL_TRACE)) {
// _LOG(TRACE, "print_trace_log", K(*this));
} else {
first_idx_ = (first_idx_ + 1) % BASE_TRACE_CNT;
}
}
traces_[cur_idx_].reset();
}
if (OB_SUCC(traces_[cur_idx_].log_it(info, args...))) {
break;
}
}
(void)ret;
}
}
int64_t to_string(char *buf, const int64_t buf_len) const
{
int64_t pos = 0;
for (int64_t i = 0; i < BASE_TRACE_CNT; i++) {
if (pos < buf_len) {
const int64_t idx = (first_idx_ + i) % BASE_TRACE_CNT;
const int64_t size = traces_[idx].to_string(buf + pos, buf_len - pos);
pos = pos + size;
if (idx == cur_idx_) {
break;
}
} else {
break;
}
}
return pos;
}
void set_need_print(const bool &need_print) { need_print_ = need_print; }
private:
static const int64_t BASE_TRACE_CNT = 4;
static const int64_t BUFFER_SIZE = TRACE_SIZE / BASE_TRACE_CNT;
private:
ObSimpleTraceBase<BUFFER_SIZE> traces_[BASE_TRACE_CNT];
int64_t first_idx_;
int64_t cur_idx_;
bool need_print_;
};
} // end namespace common
} // end namespace oceanbase
#endif /* _OB_TRACE_EVENT_H */
src/lib/profile/ob_trace_id.h
浏览文件 @ bacd5fac
......@@ -37,6 +37,7 @@ struct ObCurTraceId
inline bool is_invalid() { return id_.seq_ == 0 ? true : false; }
inline void init(const ObAddr &ip_port)
{
// TODO: Consider whether to support IPv6
id_.seq_ = ATOMIC_AAF(&(SeqGenerator::seq_generator_), 1);
id_.ip_ = ip_port.get_ipv4();
id_.reserved_ = 0;
......@@ -114,6 +115,8 @@ struct ObCurTraceId
}
};
uint64_t ObCurTraceId::SeqGenerator::seq_generator_ = 0;
int32_t LogExtraHeaderCallback(char *buf, int32_t buf_size,
int level, const char *file,
int line, const char *function, pthread_t tid);
......
src/lib/rowid/ob_urowid.cpp
浏览文件 @ bacd5fac
......@@ -135,7 +135,8 @@ int ObURowIDData::inner_get_pk_value<ObURowIDType>(const uint8_t *rowid_buf,
*(reinterpret_cast<const ObNumberDesc *>(rowid_buf + pos)); \
pos += sizeof(ObNumberDesc); \
int64_t digits_len = sizeof(uint32_t) * num_desc.len_; \
if (OB_LIKELY(pos + digits_len <= rowid_buf_len)) { \
if (OB_LIKELY(0 <= digits_len && digits_len <= rowid_buf_len \
&& pos + digits_len <= rowid_buf_len)) { \
const uint32_t *digits = reinterpret_cast<const uint32_t *>(rowid_buf + pos); \
pos += digits_len; \
pk_val.set_##type(num_desc, const_cast<uint32_t *>(digits)); \
......@@ -161,7 +162,7 @@ int ObURowIDData::inner_get_pk_value<ObURowIDType>(const uint8_t *rowid_buf,
int32_t char_len = \
*(reinterpret_cast<const int32_t *>(rowid_buf + pos)); \
pos += 4; \
if (OB_LIKELY(pos + char_len <= rowid_buf_len)) { \
if (OB_LIKELY(0 <= char_len && pos + char_len <= rowid_buf_len)) { \
const char *str_val = (const char *)rowid_buf + pos; \
pos += char_len; \
ObString str_value(char_len, str_val); \
......@@ -208,19 +209,19 @@ DEF_GET_POD_PK_VALUE(ObIntType, int, int64_t);
DEF_GET_POD_PK_VALUE(ObFloatType, float, float);
DEF_GET_POD_PK_VALUE(ObDoubleType, double, double);
DEF_GET_POD_PK_VALUE(ObDateTimeType, datetime, int64_t);
// DEF_GET_POD_PK_VALUE(ObIntervalYMType, interval_ym, int64_t);
//DEF_GET_POD_PK_VALUE(ObIntervalYMType, interval_ym, int64_t);
DEF_GET_NUMBER_PK_VALUE(ObNumberType, number);
DEF_GET_NUMBER_PK_VALUE(ObNumberFloatType, number_float);
DEF_GET_CHAR_PK_VALUE(ObVarcharType, varchar);
DEF_GET_CHAR_PK_VALUE(ObCharType, char);
// DEF_GET_CHAR_PK_VALUE(ObNVarchar2Type, nvarchar2);
// DEF_GET_CHAR_PK_VALUE(ObNCharType, nchar);
DEF_GET_CHAR_PK_VALUE(ObNVarchar2Type, nvarchar2);
DEF_GET_CHAR_PK_VALUE(ObNCharType, nchar);
// DEF_GET_OTIME_PK_VALUE(ObTimestampTZType, timestamp_tz, uint32_t);
// DEF_GET_OTIME_PK_VALUE(ObTimestampLTZType, timestamp_ltz, uint16_t);
// DEF_GET_OTIME_PK_VALUE(ObTimestampNanoType, timestamp_nano, uint16_t);
DEF_GET_OTIME_PK_VALUE(ObTimestampTZType, timestamp_tz, uint32_t);
DEF_GET_OTIME_PK_VALUE(ObTimestampLTZType, timestamp_ltz, uint16_t);
DEF_GET_OTIME_PK_VALUE(ObTimestampNanoType, timestamp_nano, uint16_t);
#define ALL_TYPES_USED_IN_INNER_FUNC \
ObNullType, \
......@@ -937,7 +938,8 @@ int ObURowIDData::get_rowkey_for_heap_organized_table(ObIArray<ObObj> &rowkey)
int ObURowIDData::get_obobj_or_partition_id_from_decoded(obproxy::proxy::ObProxyPartInfo &part_info,
obproxy::opsql::ObExprResolverResult &resolve_result,
int64_t &partition_id)
int64_t &partition_id,
common::ObIAllocator &allocator)
{
int ret = OB_SUCCESS;
......@@ -970,7 +972,7 @@ int ObURowIDData::get_obobj_or_partition_id_from_decoded(obproxy::proxy::ObProxy
case NO_PK_ROWID_VERSION:
case PK_ROWID_VERSION: {
// ob2.x 3.x
if (OB_FAIL(get_obobj_from_decoded(part_info, resolve_result))) {
if (OB_FAIL(get_obobj_from_decoded(part_info, resolve_result, allocator))) {
COMMON_LOG(WARN, "fail to get obobj from decoded", K(ret));
}
break;
......@@ -986,7 +988,8 @@ int ObURowIDData::get_obobj_or_partition_id_from_decoded(obproxy::proxy::ObProxy
// used for ob2.x 3.x
int ObURowIDData::get_obobj_from_decoded(obproxy::proxy::ObProxyPartInfo &part_info,
obproxy::opsql::ObExprResolverResult &resolve_result)
obproxy::opsql::ObExprResolverResult &resolve_result,
common::ObIAllocator &allocator)
{
int ret = OB_SUCCESS;
......@@ -995,43 +998,55 @@ int ObURowIDData::get_obobj_from_decoded(obproxy::proxy::ObProxyPartInfo &part_i
COMMON_LOG(WARN, "fail to get pk vals", K(ret));
} else {
ObProxyPartKeyInfo &key_info = part_info.get_part_key_info();
bool set_level_one_obj = false;
bool set_level_two_obj = false;
for (int64_t i = 0; OB_SUCC(ret) && i < key_info.key_num_; ++i) {
ObProxyPartKey &part_key = key_info.part_keys_[i];
int8_t idx = -1;
if ((part_key.level_ == PART_KEY_LEVEL_ONE || part_key.level_ == PART_KEY_LEVEL_BOTH)
&& !set_level_one_obj) {
idx = 0;
} else if (part_key.level_ == PART_KEY_LEVEL_TWO
&& !set_level_two_obj) {
idx = 1;
if (PART_KEY_LEVEL_ONE == part_key.level_ && !resolve_result.ranges_[0].start_key_.is_valid()) {
if (OB_FAIL(resolve_result.ranges_[0].build_row_key(part_info.get_part_columns().count(), allocator))) {
COMMON_LOG(WARN, "fail to build row key", K(ret));
}
} else if (PART_KEY_LEVEL_TWO == part_key.level_ && !resolve_result.ranges_[1].start_key_.is_valid()) {
if (OB_FAIL(resolve_result.ranges_[1].build_row_key(part_info.get_sub_part_columns().count(), allocator))) {
COMMON_LOG(WARN, "fail to build row key", K(ret));
}
}
if (idx != -1) {
if (part_key.idx_in_rowid_ >= 0
&& part_key.idx_in_rowid_ < pk_vals.count()) {
ObObj &obj = pk_vals.at(part_key.idx_in_rowid_);
if (is_valid_obj_type(obj.get_type())) {
resolve_result.ranges_[idx].start_key_.assign(&obj, 1);
resolve_result.ranges_[idx].end_key_.assign(&obj, 1);
resolve_result.ranges_[idx].border_flag_.set_inclusive_start();
resolve_result.ranges_[idx].border_flag_.set_inclusive_end();
if (idx == 0) {
set_level_one_obj = true;
if (part_key.idx_in_rowid_ >= 0
&& part_key.idx_in_rowid_ < pk_vals.count()) {
ObObj &obj = pk_vals.at(part_key.idx_in_rowid_);
ObObjType type = obj.get_type();
if (ObCharType == type || ObNCharType == type) {
int32_t val_len = obj.get_val_len();
const char* obj_str = obj.get_string_ptr();
while (val_len > 1) {
if (OB_PADDING_CHAR == *(obj_str + val_len - 1)) {
--val_len;
} else {
set_level_two_obj = true;
break;
}
COMMON_LOG(DEBUG, "succ to get obobj from rowid", K(idx), K(part_key.idx_in_rowid_), K(obj));
} else {
ret = OB_ERR_UNEXPECTED;
COMMON_LOG(WARN, "invalid obj type resolved from rowid content", K(ret), K(obj.get_type()));
}
} else {
obj.set_string(type, obj.get_string_ptr(), val_len);
}
if (part_key.level_ == 0 || part_key.level_ > 2) {
ret = OB_ERR_UNEXPECTED;
COMMON_LOG(WARN, "calc partition id using rowid failed", K(part_key.idx_in_rowid_), K(pk_vals.count()));
COMMON_LOG(WARN, "part key level unexpected", K(part_key.level_));
} else {
int level = static_cast<int>(part_key.level_ - 1);
ObObj *target_start = const_cast<ObObj*>(resolve_result.ranges_[level].start_key_.get_obj_ptr())
+ part_key.idx_in_part_columns_;
ObObj *target_end = const_cast<ObObj*>(resolve_result.ranges_[level].end_key_.get_obj_ptr())
+ part_key.idx_in_part_columns_;
// no need to deep copy
*target_start = obj;
*target_end = obj;
resolve_result.ranges_[level].border_flag_.set_inclusive_start();
resolve_result.ranges_[level].border_flag_.set_inclusive_end();
COMMON_LOG(DEBUG, "succ to get val from rowid", K(obj), K(level), K(part_key.idx_in_part_columns_));
}
} else {
ret = OB_ERR_UNEXPECTED;
COMMON_LOG(WARN, "calc partition id using rowid failed", K(part_key.idx_in_rowid_), K(pk_vals.count()));
}
} // for
} // else
......
src/lib/rowid/ob_urowid.h
浏览文件 @ bacd5fac
......@@ -154,7 +154,8 @@ public:
int get_rowkey_for_heap_organized_table(ObIArray<ObObj> &rowkey);
int get_obobj_or_partition_id_from_decoded(obproxy::proxy::ObProxyPartInfo &part_info,
obproxy::opsql::ObExprResolverResult &resolve_result,
int64_t &partition_id);
int64_t &partition_id,
common::ObIAllocator &allocator);
// if highest bit is 1, lower 7 bits of version filed indicates how many primar key obj
static bool is_valid_part_gen_col_version(int64_t v)
......@@ -231,7 +232,8 @@ private:
int parse_ext_heap_organized_table_rowid(uint64_t &tablet_id, uint64_t &auto_inc);
int get_obobj_from_decoded(obproxy::proxy::ObProxyPartInfo &part_info,
obproxy::opsql::ObExprResolverResult &resolve_result);
obproxy::opsql::ObExprResolverResult &resolve_result,
common::ObIAllocator &allocator);
};
} // end namespace common
......
src/lib/timezone/ob_time_convert.cpp
浏览文件 @ bacd5fac
......@@ -409,6 +409,32 @@ int ObTimeConverter::str_to_datetime(const ObString &str, const ObTimeZoneInfo *
return ret;
}
/**
* @brief cast str to oracle date with format
*
* @param str input string of date
* @param cvrt_ctx
* @param value out param, oracle DATE result
* @return int
*/
int ObTimeConverter::str_to_date_oracle(const ObString &str,
const ObTimeConvertCtx &cvrt_ctx,
ObDateTime &value)
{
int ret = OB_SUCCESS;
ObTime ob_time;
ObDateTime result_value = 0;
ObScale scale = 0;
if (OB_FAIL(str_to_ob_time_oracle_dfm(str, cvrt_ctx, ObDateTimeType, ob_time, scale))) {
LOG_WARN("failed to convert str to ob_time", K(str), K(cvrt_ctx.oracle_nls_format_));
} else if (OB_FAIL(ob_time_to_datetime(ob_time, cvrt_ctx.tz_info_, result_value))) {
LOG_WARN("convert ob_time to datetime failed", K(ret), K(ob_time));
} else {
value = result_value;
}
return ret;
}
int ObTimeConverter::str_to_datetime_format(const ObString &str, const ObString &fmt,
const ObTimeZoneInfo *tz_info, int64_t &value, int16_t *scale)
{
......@@ -3111,7 +3137,7 @@ int ObTimeConverter::ob_time_to_str(const ObTime &ob_time, ObDTMode mode, int16_
int ObTimeConverter::data_fmt_nd(char *buffer, int64_t buf_len, int64_t &pos, const int64_t n, int64_t target)
{
int ret = OB_SUCCESS;
if (OB_UNLIKELY(n <= 0 || target < 0 || target > 999999)) {
if (OB_UNLIKELY(n <= 0 || target < 0 || target > 999999999)) {
ret = OB_ERR_UNEXPECTED;
LIB_TIME_LOG(ERROR, "invalid argument", K(ret), K(n), K(target));
} else if (OB_UNLIKELY(n > buf_len - pos)) {
......@@ -3698,6 +3724,8 @@ int ObTimeConverter::ob_time_to_str_oracle_dfm(const ObTime &ob_time,
break;
}
case ObDFMFlag::FF: {
// FF default FF9 in oracle
int64_t scale = ObDFMFlag::FF9 - ObDFMFlag::FF1 + 1;
if (OB_UNLIKELY(!HAS_TYPE_ORACLE(ob_time.mode_))) {
ret = OB_INVALID_DATE_FORMAT;
} else if (0 == scale) {
......
src/lib/timezone/ob_time_convert.h
浏览文件 @ bacd5fac
......@@ -251,6 +251,7 @@ public:
static int int_to_time(int64_t int64, int64_t &value);
static int int_to_year(int64_t int64, uint8_t &value);
static int str_to_datetime(const ObString &str, const ObTimeZoneInfo *tz_info, int64_t &value, int16_t *scale = NULL);
static int str_to_date_oracle(const ObString &str, const ObTimeConvertCtx &cvrt_ctx, ObDateTime &value);
static int str_to_datetime_format(const ObString &str, const ObString &fmt,
const ObTimeZoneInfo *tz_info, int64_t &value, int16_t *scale = NULL);
static int str_is_date_format(const ObString &str, bool &date_flag);
......@@ -345,7 +346,7 @@ public:
ObDTMode mode, int32_t &delta);
static int ob_time_to_str_oracle_dfm(const ObTime &ob_time, ObScale scale, const ObString &format,
char *buf, int64_t buf_len, int64_t &pos);
static int ob_time_to_datetime(ObTime &ob_time, const ObTimeZoneInfo *sp, int64_t &value);
static int ob_time_to_datetime(ObTime &ob_time, const ObTimeZoneInfo *tz_info, int64_t &value);
static int32_t ob_time_to_date(ObTime &ob_time);
static int ob_time_to_otimestamp(ObTime &ob_time, ObOTimestampData &value);
static int64_t ob_time_to_time(const ObTime &ob_time);
......
src/lib/trace/ob_trace.cpp
浏览文件 @ bacd5fac
......@@ -439,6 +439,7 @@ ObSpanCtx* ObTrace::begin_span(uint32_t span_type, uint8_t level, bool is_follow
new_span = freed_span_.remove_last();
current_span_.add_first(new_span);
new_span->span_type_ = static_cast<uint16_t>(span_type);
new_span->span_id_.high_ = 0;
new_span->span_id_.low_ = ++seq_;
new_span->source_span_ = last_active_span_;
new_span->is_follow_ = is_follow;
......@@ -565,4 +566,4 @@ void ObTrace::dump_span()
}
}
}
\ No newline at end of file
}
src/obproxy/Makemodule.am
浏览文件 @ bacd5fac
......@@ -10,7 +10,11 @@ AM_CPPFLAGS += -I${top_srcdir}/src/obproxy\
obproxy_libobproxy_la_LIBADD := dbconfig/libdbconfig.la lib/regex/regex/libregex.la prometheus/libprometheus_exporter.la
noinst_LTLIBRARIES += obproxy/libobproxy.la
bin_PROGRAMS += obproxy/obproxy
if HAVE_SO
lib_LTLIBRARIES += obproxy/libobproxy_so.la
else
bin_PROGRAMS += obproxy/obproxy
endif
include obproxy/iocore/eventsystem/Makemodule.am
include obproxy/iocore/net/Makemodule.am
......@@ -30,6 +34,7 @@ include obproxy/opsql/Makemodule.am
include obproxy/packet/Makemodule.am
include obproxy/dbconfig/Makemodule.am
include obproxy/engine/Makemodule.am
include obproxy/driver/c/Makemodule.am
include obproxy/optimizer/Makemodule.am
include obproxy/executor/Makemodule.am
include obproxy/qos/Makemodule.am
......@@ -41,6 +46,9 @@ pub_source = \
obproxy/ob_proxy_main.cpp \
obproxy/ob_proxy.h \
obproxy/ob_proxy.cpp
init_so_source =\
obproxy/ob_proxy_init.h\
obproxy/ob_proxy_init.cpp
obproxy_sources :=\
${iocore_eventsystem_sources}\
......@@ -51,6 +59,7 @@ ${proxy_api_sources}\
${proxy_plugins_sources}\
${proxy_mysqllib_sources}\
${proxy_mysql_client_sources}\
${obproxy_c_driver_sources}\
${proxy_mysql_route_sources}\
${proxy_mysql_shard_sources}\
${obutils_sources}\
......@@ -70,6 +79,8 @@ ${obproxy_executor_sources}\
${obproxy_optimizer_sources}\
${obproxy_engine_sources}\
${qos_sources}\
${init_so_source}\
${pub_source}\
${omt_sources}
obproxy_libobproxy_la_SOURCES :=\
......@@ -80,8 +91,15 @@ ${rpc_obmysql_packet_sources} ${obproxy_rpc_obmysql_sources} ${rpc_common_source
${obproxy_sql_sources}\
${obproxy_sources}
if HAVE_SO
obproxy_libobproxy_so_la_SOURCES :=
obproxy_libobproxy_so_la_LIBADD := obproxy/libobproxy.la lib/compress/libzlib_1.0.la lib/compress/libzstd_1.3.8.la ${BIN_LDFLAGS} ${DEP_DIR}/lib64/libprometheus-cpp-pull.a ${DEP_DIR}/lib64/libprometheus-cpp-core.a ${RUNTIME_DIR}/lib/libob_sql_proxy_parser_static.a ${DEP_DIR}/lib/libssl.a ${DEP_DIR}/lib/libcrypto.a -lpthread
obproxy_libobproxy_so_la_LDFLAGS :=${AM_LDFLAGS} ${DEP_DIR}/lib/libgrpc++.a -lpthread -lc -lm -lrt ${DEP_DIR}/lib/libcurl.a -ldl -L${DEP_DIR}/lib -llber ${DEP_DIR}/lib/sqlite/libsqlite3.a ${DEP_DIR}/lib/libgrpc_cronet.a ${DEP_DIR}/lib/libgrpc++_cronet.a -shared ${DEP_DIR}/lib64/libprotobuf.a -fPIC
endif
nodist_obproxy_libobproxy_la_SOURCES = build_version.c
obproxy_obproxy_SOURCES := obproxy/main.cpp ${pub_source}
obproxy_obproxy_SOURCES := obproxy/main.cpp
obproxy_obproxy_LDADD_BASE := obproxy/libobproxy.la lib/compress/libzlib_1.0.la lib/compress/libzstd_1.3.8.la ${BIN_LDFLAGS} ${DEP_DIR}/lib64/libprotobuf.a ${DEP_DIR}/lib/libgrpc.a ${DEP_DIR}/lib/libgrpc++.a ${DEP_DIR}/lib/libgrpc_unsecure.a ${DEP_DIR}/lib/libgrpc++_unsecure.a ${DEP_DIR}/lib/libgrpc++_reflection.a ${DEP_DIR}/lib/libgrpcpp_channelz.a ${DEP_DIR}/lib/libgrpc++_error_details.a ${DEP_DIR}/lib/libgrpc_cronet.a ${DEP_DIR}/lib/libgrpc++_cronet.a ${DEP_DIR}/lib/libgpr.a ${DEP_DIR}/lib/libaddress_sorting.a ${DEP_DIR}/lib/libcares.a ${DEP_DIR}/lib64/libprometheus-cpp-pull.a ${DEP_DIR}/lib64/libprometheus-cpp-core.a ${RUNTIME_DIR}/lib/libob_sql_proxy_parser_static.a ${DEP_DIR}/lib/libssl.a ${DEP_DIR}/lib/libcrypto.a -lpthread ${DEP_DIR}/lib/libcurl.a ${DEP_DIR}/lib/libcrypto.a ${DEP_DIR}/lib/sqlite/libsqlite3.a
obproxy_obproxy_LDFLAGS :=${AM_LDFLAGS} -lpthread -lc -lm -lrt -ldl -L${DEP_DIR}/lib
......
src/obproxy/cmd/ob_cmd_handler.cpp
浏览文件 @ bacd5fac
......@@ -88,8 +88,7 @@ int ObCmdHandler::reset()
} else {
internal_buf_->reset();
seq_ = original_seq_;
protocol_ = ObProxyProtocol::PROTOCOL_NORMAL;
ob20_param_.reset();
// could not reset protocol and ob20 param, cause we need fill external buf according to protocol
}
return ret;
}
......
src/obproxy/cmd/ob_dds_config_handler.cpp
浏览文件 @ bacd5fac
......@@ -361,7 +361,6 @@ int ObDdsConfigHandler::handle_parse_where_fields(ObArenaAllocator* allocator, O
} else {
ObExprParser expr_parser(*allocator, parse_mode);
expr_result.part_key_info_.key_num_ = 0;
expr_result.target_mask_ = 0;
if (OB_FAIL(expr_parser.parse_reqsql(sql, sql_parse_result.get_parsed_length(),
expr_result, sql_parse_result.get_stmt_type(),
connection_collation))) {
......
src/obproxy/cmd/ob_internal_cmd_handler.cpp
浏览文件 @ bacd5fac
......@@ -111,8 +111,7 @@ int ObInternalCmdHandler::reset()
} else {
internal_buf_->reset();
seq_ = original_seq_;
protocol_ = ObProxyProtocol::PROTOCOL_NORMAL;
ob20_param_.reset();
// could not reset protocol and ob20_param in handler, cause we need filll external buf according to protocol
}
return ret;
}
......@@ -621,7 +620,8 @@ int ObInternalCmdHandler::handle_callback(int event, void *data)
MUTEX_TRY_LOCK(lock, action_.mutex_, submit_thread_);
if (lock.is_locked()) {
if (!action_.cancelled_) {
if (INTERNAL_CMD_EVENTS_SUCCESS == saved_event_ && OB_SUCCESS != fill_external_buf()) {
if (INTERNAL_CMD_EVENTS_SUCCESS == saved_event_
&& OB_SUCCESS != fill_external_buf()) {
saved_event_ = INTERNAL_CMD_EVENTS_FAILED;
}
DEBUG_ICMD("do callback now", K_(saved_event));
......
src/obproxy/cmd/ob_sequence_info_handler.cpp
浏览文件 @ bacd5fac
......@@ -175,7 +175,8 @@ int ObSequenceInfoHandler::handle_shard_sequence_params()
} else if (!param_.tnt_id_.empty() && param_.tnt_col_.empty()) {
snprintf(err_msg_, SEQUENCE_ERR_MSG_SIZE, "tnt_col is empty while tnt_id used");
ret = OB_ERR_UNEXPECTED;
} else if (OB_FAIL(ObProxyShardUtils::get_real_info(*logic_db_info,
} else if (OB_FAIL(ObProxyShardUtils::get_real_info(session_info,
*logic_db_info,
sequence_table_name,
parse_result,
shard_conn,
......@@ -211,6 +212,9 @@ int ObSequenceInfoHandler::handle_shard_sequence_params()
table_id_ = table_id;
group_id_ = group_id;
eid_ = es_id;
session_info.set_group_id(group_id);
session_info.set_table_id(table_id);
session_info.set_es_id(es_id);
snprintf(seq_id_buf_, 2048, "%.*s-%.*s-%.*s-%.*s-%.*s-%.*s-%02ld-%02ld-%02ld",
logic_tenant_name_.config_string_.length(), logic_tenant_name_.config_string_.ptr(),
......@@ -222,6 +226,11 @@ int ObSequenceInfoHandler::handle_shard_sequence_params()
group_id_, table_id_, eid_);
param_.seq_id_ = ObString(seq_id_buf_);
}
if (NULL != logic_db_info) {
logic_db_info->dec_ref();
logic_db_info = NULL;
}
LOG_DEBUG("Left handle_shard_sequence_params", K(param_), K(table_id_), K(group_id_), K(eid_));
return ret;
}
......
src/obproxy/cmd/ob_show_route_handler.cpp
浏览文件 @ bacd5fac
......@@ -463,8 +463,6 @@ int ObShowRouteHandler::dump_table_item(const ObTableEntry &entry)
int ret = OB_SUCCESS;
const int64_t count = entry.get_server_count();
ObSqlString server_addr;
uint32_t ipv4 = 0;
int32_t port = 0;
ObRole role = INVALID_ROLE;
ObReplicaType replica_type = REPLICA_TYPE_MAX;
const ObTenantServer *ts = NULL;
......@@ -479,30 +477,24 @@ int ObShowRouteHandler::dump_table_item(const ObTableEntry &entry)
part_info = entry.get_part_info();
}
for (int64_t i = 0; i < count && OB_SUCC(ret); ++i) {
char ip_port_buf[MAX_IP_ADDR_LENGTH];
memset(ip_port_buf, 0, sizeof(ip_port_buf));
if (NULL != ts) {
ipv4 = ts->get_replica_location(i)->server_.get_ipv4();
port = ts->get_replica_location(i)->server_.get_port();
role = ts->get_replica_location(i)->role_;
replica_type = ts->get_replica_location(i)->replica_type_;
ts->get_replica_location(i)->server_.ip_port_to_string(ip_port_buf, sizeof(ip_port_buf));
} else if (NULL != pl) {
ipv4 = pl->get_replica(i)->server_.get_ipv4();
port = pl->get_replica(i)->server_.get_port();
role = pl->get_replica(i)->role_;
replica_type = pl->get_replica(i)->replica_type_;
pl->get_replica(i)->server_.ip_port_to_string(ip_port_buf, sizeof(ip_port_buf));
} else {
ipv4 = 0;
port = -1;
role = INVALID_ROLE;
replica_type = REPLICA_TYPE_MAX;
}
const ObString &string = ObProxyReplicaLocation::get_replica_type_string(replica_type);
if (OB_FAIL(server_addr.append_fmt("server[%ld]=%d.%d.%d.%d:%d,%s,%.*s; ",
if (OB_FAIL(server_addr.append_fmt("server[%ld]=%s,%s,%.*s; ",
i,
(ipv4 >> 24) & 0XFF,
(ipv4 >> 16) & 0xFF,
(ipv4 >> 8) & 0xFF,
(ipv4) & 0xFF,
port,
ip_port_buf,
role2str(role),
string.length(), string.ptr()))) {
WARN_ICMD("fail to append server_addr", K(i), K(ret));
......@@ -674,24 +666,18 @@ int ObShowRouteHandler::dump_partition_item(const ObPartitionEntry &entry)
int ret = OB_SUCCESS;
const int64_t count = entry.get_server_count();
ObSqlString server_addr;
uint32_t ipv4 = 0;
int32_t port = 0;
ObRole role = INVALID_ROLE;
ObReplicaType replica_type = REPLICA_TYPE_MAX;
const ObProxyPartitionLocation &pl = entry.get_pl();
for (int64_t i = 0; i < count && OB_SUCC(ret); ++i) {
ipv4 = pl.get_replica(i)->server_.get_ipv4();
port = pl.get_replica(i)->server_.get_port();
role = pl.get_replica(i)->role_;
replica_type = pl.get_replica(i)->replica_type_;
const ObString &string = ObProxyReplicaLocation::get_replica_type_string(replica_type);
if (OB_FAIL(server_addr.append_fmt("server[%ld]=%d.%d.%d.%d:%d,%s,%.*s; ",
char ip_port_buf[MAX_IP_ADDR_LENGTH];
pl.get_replica(i)->server_.ip_port_to_string(ip_port_buf, sizeof(ip_port_buf));
if (OB_FAIL(server_addr.append_fmt("server[%ld]=%s,%s,%.*s; ",
i,
(ipv4 >> 24) & 0XFF,
(ipv4 >> 16) & 0xFF,
(ipv4 >> 8) & 0xFF,
(ipv4) & 0xFF,
port,
ip_port_buf,
role2str(role),
string.length(), string.ptr()))) {
WARN_ICMD("fail to append server_addr", K(i), K(ret));
......
src/obproxy/cmd/ob_show_sqlaudit_handler.cpp
浏览文件 @ bacd5fac
......@@ -368,9 +368,10 @@ enum
OB_SSA_SERVER_SEND_SAVED_LOGIN_TIME,
OB_SSA_SERVER_SEND_USE_DATABASE_TIME,
OB_SSA_SERVER_SEND_SESSION_VARIABLE_TIME,
OB_SSA_SERVER_SEND_SESSION_USER_VARIABLE_TIME,
OB_SSA_SERVER_SEND_ALL_SESSION_VARIABLE_TIME,
OB_SSA_SERVER_SEND_LAST_INSERT_ID_TIME,
OB_SSA_SERVER_SEND_START_TRANS_TIME,
OB_SSA_SERVER_SEND_XA_START_TIME,
OB_SSA_BUILD_SERVER_REQUEST_TIME,
OB_SSA_PLUGIN_COMPRESS_REQUEST_TIME,
OB_SSA_PREPARE_SEND_REQUEST_TO_SERVER_TIME,
......@@ -408,9 +409,10 @@ const ObProxyColumnSchema SSA_COLUMN_ARRAY[OB_SSA_MAX_COLUMN_ID] = {
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_SAVED_LOGIN_TIME, "server_send_saved_login_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_USE_DATABASE_TIME, "server_send_use_database_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_SESSION_VARIABLE_TIME, "server_send_session_variable_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_SESSION_USER_VARIABLE_TIME, "server_send_session_user_variable_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_ALL_SESSION_VARIABLE_TIME, "server_send_all_session_variable_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_LAST_INSERT_ID_TIME, "server_send_last_insert_id_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_START_TRANS_TIME, "server_send_start_trans_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_SERVER_SEND_XA_START_TIME, "server_send_xa_start_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_BUILD_SERVER_REQUEST_TIME, "build_server_request_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_PLUGIN_COMPRESS_REQUEST_TIME, "plugin_compress_request_ns", obmysql::OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_SSA_PREPARE_SEND_REQUEST_TO_SERVER_TIME, "prepare_send_request_to_server_ns", obmysql::OB_MYSQL_TYPE_LONG),
......@@ -549,9 +551,10 @@ int ObShowSqlauditHandler::dump_sqlaudit_record(ObSqlauditRecord &record)
cells[OB_SSA_SERVER_SEND_SAVED_LOGIN_TIME].set_int(record.cmd_time_stat_.server_send_saved_login_time_);
cells[OB_SSA_SERVER_SEND_USE_DATABASE_TIME].set_int(record.cmd_time_stat_.server_send_use_database_time_);
cells[OB_SSA_SERVER_SEND_SESSION_VARIABLE_TIME].set_int(record.cmd_time_stat_.server_send_session_variable_time_);
cells[OB_SSA_SERVER_SEND_SESSION_USER_VARIABLE_TIME].set_int(record.cmd_time_stat_.server_send_session_user_variable_time_);
cells[OB_SSA_SERVER_SEND_ALL_SESSION_VARIABLE_TIME].set_int(record.cmd_time_stat_.server_send_all_session_variable_time_);
cells[OB_SSA_SERVER_SEND_LAST_INSERT_ID_TIME].set_int(record.cmd_time_stat_.server_send_last_insert_id_time_);
cells[OB_SSA_SERVER_SEND_START_TRANS_TIME].set_int(record.cmd_time_stat_.server_send_start_trans_time_);
cells[OB_SSA_SERVER_SEND_XA_START_TIME].set_int(record.cmd_time_stat_.server_send_xa_start_time_);
cells[OB_SSA_BUILD_SERVER_REQUEST_TIME].set_int(record.cmd_time_stat_.build_server_request_time_);
cells[OB_SSA_PLUGIN_COMPRESS_REQUEST_TIME].set_int(record.cmd_time_stat_.plugin_compress_request_time_);
cells[OB_SSA_PLUGIN_DECOMPRESS_RESPONSE_TIME].set_int(record.cmd_time_stat_.plugin_decompress_response_time_);
......
src/obproxy/cmd/ob_show_topology_handler.cpp
浏览文件 @ bacd5fac
......@@ -13,8 +13,8 @@
#define USING_LOG_PREFIX PROXY_CMD
#include "cmd/ob_show_topology_handler.h"
#include "iocore/eventsystem/ob_event_processor.h"
#include "dbconfig/ob_proxy_db_config_info.h"
#include "iocore/eventsystem/ob_event_processor.h"
#include "lib/string/ob_sql_string.h"
using namespace oceanbase::common;
......@@ -45,17 +45,38 @@ static const EMySQLFieldType column_type[OB_TC_MAX_COLUMN_ID] = {
OB_MYSQL_TYPE_LONG
};
//TopologyColumnID for sharding
enum
{
OB_TC_ID = 0,
OB_TC_GROUP_NAME,
OB_TC_TABLE_NAME,
OB_TC_SCHEMA_TYPE,
OB_TC_SHARD_KEY,
OB_TC_SHARD_RULE,
OB_TC_SHARD_MAX_COLUMN_ID,
};
const ObProxyColumnSchema SHOW_TOPOLOGY_ARRAY[OB_TC_SHARD_MAX_COLUMN_ID] = {
ObProxyColumnSchema::make_schema(OB_TC_ID, "id", OB_MYSQL_TYPE_LONG),
ObProxyColumnSchema::make_schema(OB_TC_GROUP_NAME, "group_name", OB_MYSQL_TYPE_VARCHAR),
ObProxyColumnSchema::make_schema(OB_TC_TABLE_NAME, "table_name", OB_MYSQL_TYPE_VARCHAR),
ObProxyColumnSchema::make_schema(OB_TC_SCHEMA_TYPE, "schema_type", OB_MYSQL_TYPE_VARCHAR),
ObProxyColumnSchema::make_schema(OB_TC_SHARD_KEY, "shard_key", OB_MYSQL_TYPE_VARCHAR),
ObProxyColumnSchema::make_schema(OB_TC_SHARD_RULE, "shard_rule", OB_MYSQL_TYPE_VARCHAR),
};
ObShowTopologyHandler::ObShowTopologyHandler(ObMIOBuffer *buf, ObCmdInfo &info)
: ObCmdHandler(buf, info)
{
}
int ObShowTopologyHandler::handle_show_topology(const ObString &tenant_name,
const ObString &db_name,
const ObString &group_name)
int ObShowTopologyHandler::handle_show_elastic_id(const ObString &tenant_name,
const ObString &db_name,
const ObString &group_name)
{
int ret = OB_SUCCESS;
if (OB_FAIL(dump_topology_header())) {
if (OB_FAIL(dump_elastic_id_header())) {
WARN_CMD("fail to dump_header", K(ret));
} else {
ObDbConfigCache &dbconfig_cache = get_global_dbconfig_cache();
......@@ -144,7 +165,7 @@ int ObShowTopologyHandler::dump_elastic_id(const int64_t eid)
return ret;
}
int ObShowTopologyHandler::dump_topology_header()
int ObShowTopologyHandler::dump_elastic_id_header()
{
int ret = OB_SUCCESS;
if (OB_FAIL(encode_header(column_name, column_type, OB_TC_MAX_COLUMN_ID))) {
......@@ -153,7 +174,7 @@ int ObShowTopologyHandler::dump_topology_header()
return ret;
}
int ObShowTopologyHandler::show_topology_cmd_callback(ObMIOBuffer *buf,
int ObShowTopologyHandler::show_elastic_id_cmd_callback(ObMIOBuffer *buf,
ObCmdInfo &info,
const ObString &logic_tenant_name,
const ObString &logic_database_name,
......@@ -167,7 +188,7 @@ int ObShowTopologyHandler::show_topology_cmd_callback(ObMIOBuffer *buf,
ERROR_CMD("fail to new ObShowTopologyHandler", K(ret));
} else if (OB_FAIL(handler->init())) {
WARN_CMD("fail to init for ObShowTopologyHandler");
} else if (OB_FAIL(handler->handle_show_topology(logic_tenant_name, logic_database_name, group_name))){
} else if (OB_FAIL(handler->handle_show_elastic_id(logic_tenant_name, logic_database_name, group_name))){
WARN_CMD("fail to handle show topology", K(logic_tenant_name), K(logic_database_name), K(group_name), K(ret));
}
......@@ -178,6 +199,255 @@ int ObShowTopologyHandler::show_topology_cmd_callback(ObMIOBuffer *buf,
return ret;
}
int ObShowTopologyHandler::show_topology_cmd_callback(ObMIOBuffer *buf,
ObCmdInfo &info,
const ObString &logic_tenant_name,
const ObString &logic_database_name,
const ObString &table_name)
{
int ret = OB_SUCCESS;
ObShowTopologyHandler *handler = NULL;
if (OB_ISNULL(handler = new(std::nothrow) ObShowTopologyHandler(buf, info))) {
ret = OB_ALLOCATE_MEMORY_FAILED;
ERROR_CMD("fail to new ObShowTopologyHandler", K(ret));
} else if (OB_FAIL(handler->init())) {
WARN_CMD("fail to init for ObShowTopologyHandler");
} else if (OB_FAIL(handler->handle_show_topology(logic_tenant_name, logic_database_name, table_name))){
WARN_CMD("fail to handle show topology", K(logic_tenant_name), K(logic_database_name), K(ret));
}
if (OB_LIKELY(NULL != handler)) {
delete handler;
handler = NULL;
}
return ret;
}
int ObShowTopologyHandler::handle_show_topology(const ObString &tenant_name,
const ObString &db_name,
const ObString &table_name)
{
int ret = OB_SUCCESS;
if (OB_FAIL(encode_header(SHOW_TOPOLOGY_ARRAY, OB_TC_SHARD_MAX_COLUMN_ID))) {
WARN_CMD("fail to dump_header", K(ret));
} else {
ObDbConfigCache &dbconfig_cache = get_global_dbconfig_cache();
ObDbConfigLogicDb *db_info = NULL;
ObShardRule *shard_rule = NULL;
if (OB_ISNULL(db_info = dbconfig_cache.get_exist_db_info(tenant_name, db_name))) {
ret = OB_ERR_BAD_DATABASE;
WARN_CMD("logic database not exist", K(tenant_name), K(db_name), K(ret));
} else if (OB_UNLIKELY(db_info->is_single_shard_db_table())) {
if (OB_FAIL(dump_shard_topology_for_single_db(table_name))) {
WARN_CMD("fail to encode topology packet for single db", K(tenant_name), K(db_name), K(ret));
}
} else if (OB_FAIL(db_info->get_shard_rule(shard_rule, table_name))) {
// no logic table
ret = OB_TABLE_NOT_EXIST;
} else if (OB_UNLIKELY((1 == shard_rule->db_size_) && (1 == shard_rule->tb_size_))) {
if (OB_FAIL(dump_shard_topology_for_shard_db_single_tb(table_name))) {
WARN_CMD("fail to encode topology packet for shard_db_single_tb", K(tenant_name), K(db_name), K(shard_rule->db_size_), K(shard_rule->tb_size_), K(ret));
}
} else if (OB_UNLIKELY(1 == shard_rule->tb_size_)) {
if (OB_FAIL(dump_shard_topology_for_shard_db_non_shard_tb(table_name, shard_rule))) {
WARN_CMD("fail to encode topology packet for shard_db_non_shard_tb", K(tenant_name), K(db_name), K(shard_rule->db_size_), K(shard_rule->tb_size_), K(ret));
}
} else {
if (OB_FAIL(dump_shard_topology_for_shard_db_shard_tb(shard_rule))) {
WARN_CMD("fail to encode topology packet for shard_db_shard_tb", K(tenant_name), K(db_name), K(shard_rule->db_size_), K(shard_rule->tb_size_), K(ret));
}
}
if (NULL != db_info) {
db_info->dec_ref();
db_info = NULL;
}
}
if (OB_SUCC(ret)) {
if (OB_FAIL(encode_eof_packet())) {
WARN_CMD("fail to encode eof packet", K(ret));
}
}
if (OB_FAIL(ret)) {
if (OB_FAIL(encode_err_packet(ret))) {
WARN_CMD("fail to encode internal err packet, callback", K(ret));
} else {
INFO_CMD("succ to encode internal err packet, callback");
ret = OB_SUCCESS;
}
}
if (OB_SUCC(ret)) {
if (OB_FAIL(fill_external_buf())) {
WARN_CMD("fail to fill_external_buf", K(ret));
}
}
return ret;
}
int ObShowTopologyHandler::dump_shard_topology_for_single_db(ObString table_name)
{
int ret = OB_SUCCESS;
ObNewRow row;
ObObj cells[OB_TC_SHARD_MAX_COLUMN_ID];
cells[OB_TC_ID].set_int(0);
cells[OB_TC_GROUP_NAME].set_varchar("group_00");
cells[OB_TC_TABLE_NAME].set_varchar(table_name);
cells[OB_TC_SCHEMA_TYPE].set_varchar("SINGLE");
cells[OB_TC_SHARD_KEY].set_null();
cells[OB_TC_SHARD_RULE].set_null();
row.cells_ = cells;
row.count_ = OB_TC_SHARD_MAX_COLUMN_ID;
if (OB_FAIL(encode_row_packet(row))) {
WARN_CMD("fail to encode row packet for shard", K(row), K(ret));
}
return ret;
}
int ObShowTopologyHandler::dump_shard_topology_for_shard_db_single_tb(ObString table_name)
{
int ret = OB_SUCCESS;
ObNewRow row;
ObObj cells[OB_TC_SHARD_MAX_COLUMN_ID];
cells[OB_TC_ID].set_int(0);
cells[OB_TC_GROUP_NAME].set_varchar("group_00");
cells[OB_TC_TABLE_NAME].set_varchar(table_name);
cells[OB_TC_SCHEMA_TYPE].set_varchar("SHARD");
cells[OB_TC_SHARD_KEY].set_null();
cells[OB_TC_SHARD_RULE].set_null();
row.cells_ = cells;
row.count_ = OB_TC_SHARD_MAX_COLUMN_ID;
if (OB_FAIL(encode_row_packet(row))) {
WARN_CMD("fail to encode row packet for shard", K(row), K(ret));
}
return ret;
}
int ObShowTopologyHandler::dump_shard_topology_for_shard_db_non_shard_tb(ObString table_name, ObShardRule *shard_rule)
{
int ret = OB_SUCCESS;
ObNewRow row;
ObObj cells[OB_TC_SHARD_MAX_COLUMN_ID];
int size = OB_TC_SHARD_MAX_COLUMN_ID;
ObString real_group_name;
ObString real_table_name = table_name;
char group_name_buf[OB_MAX_TABLEGROUP_NAME_LENGTH];
ObString schema_type("SHARD");
char shard_key_buf[OB_MAX_COLUMN_NAME_LENGTH];
ObString shard_key_str = get_shard_key_str(*shard_rule, shard_key_buf, OB_MAX_COLUMN_NAME_LENGTH);
ObString shard_rule_str = get_shard_rule_str(*shard_rule);
int64_t count = shard_rule->db_size_;
for(int64_t index = 0; index < count; ++index) {
int64_t group_index = index;
shard_rule->get_real_name_by_index(shard_rule->db_size_,
shard_rule->db_suffix_len_, group_index,
shard_rule->db_prefix_.config_string_,
shard_rule->db_tail_.config_string_,
group_name_buf, OB_MAX_TABLEGROUP_NAME_LENGTH);
real_group_name = ObString::make_string(group_name_buf);
cells[OB_TC_ID].set_int(index);
cells[OB_TC_GROUP_NAME].set_varchar(real_group_name);
cells[OB_TC_TABLE_NAME].set_varchar(real_table_name);
cells[OB_TC_SCHEMA_TYPE].set_varchar(schema_type);
cells[OB_TC_SHARD_KEY].set_varchar(shard_key_str);
cells[OB_TC_SHARD_RULE].set_varchar(shard_rule_str);
row.cells_ = cells;
row.count_ = size;
if (OB_FAIL(encode_row_packet(row))) {
WARN_CMD("fail to encode row packet for shard", K(row), K(ret));
}
}
return ret;
}
int ObShowTopologyHandler::dump_shard_topology_for_shard_db_shard_tb(ObShardRule *shard_rule)
{
int ret = OB_SUCCESS;
if (OB_UNLIKELY((0 == shard_rule->db_size_) || (shard_rule->tb_size_ < shard_rule->db_size_))) {
WARN_CMD("wrong shard rule for shard_db_shard_tb", K(shard_rule->db_size_), K(shard_rule->tb_size_), K(ret));
} else {
ObNewRow row;
ObObj cells[OB_TC_SHARD_MAX_COLUMN_ID];
int size = OB_TC_SHARD_MAX_COLUMN_ID;
ObString real_group_name;
ObString real_table_name;
char group_name_buf[OB_MAX_TABLEGROUP_NAME_LENGTH];
char table_name_buf[OB_MAX_USER_TABLE_NAME_LENGTH];
char shard_key_buf[OB_MAX_COLUMN_NAME_LENGTH];
ObString schema_type("SHARD");
ObString shard_key_str = get_shard_key_str(*shard_rule, shard_key_buf, OB_MAX_COLUMN_NAME_LENGTH);
ObString shard_rule_str = get_shard_rule_str(*shard_rule);
int64_t count = shard_rule->tb_size_;
for(int64_t index = 0; index < count; ++index) {
int64_t group_index = index / (shard_rule->tb_size_ / shard_rule->db_size_);
shard_rule->get_real_name_by_index(shard_rule->db_size_,
shard_rule->db_suffix_len_, group_index,
shard_rule->db_prefix_.config_string_,
shard_rule->db_tail_.config_string_,
group_name_buf, OB_MAX_TABLEGROUP_NAME_LENGTH);
shard_rule->get_real_name_by_index(shard_rule->tb_size_,
shard_rule->tb_suffix_len_, index,
shard_rule->tb_prefix_.config_string_,
shard_rule->tb_tail_.config_string_,
table_name_buf, OB_MAX_USER_TABLE_NAME_LENGTH);
real_group_name = ObString::make_string(group_name_buf);
real_table_name = ObString::make_string(table_name_buf);
cells[OB_TC_ID].set_int(index);
cells[OB_TC_GROUP_NAME].set_varchar(real_group_name);
cells[OB_TC_TABLE_NAME].set_varchar(real_table_name);
cells[OB_TC_SCHEMA_TYPE].set_varchar(schema_type);
cells[OB_TC_SHARD_KEY].set_varchar(shard_key_str);
cells[OB_TC_SHARD_RULE].set_varchar(shard_rule_str);
row.cells_ = cells;
row.count_ = size;
if (OB_FAIL(encode_row_packet(row))) {
WARN_CMD("fail to encode row packet for shard", K(row), K(ret));
}
}
}
return ret;
}
ObString ObShowTopologyHandler::get_shard_key_str(dbconfig::ObShardRule& shard_rule, char* const target_buf, const int target_buf_len)
{
ObString ret_str = ObString::make_string("empty shard key");
if (OB_LIKELY(0 != shard_rule.tb_rules_.count())) {
ObString& origin_string = shard_rule.tb_rules_.at(0).shard_rule_str_.config_string_;
for(int i = 0; i < origin_string.length(); ++i) {
if(origin_string[i] != '#') {
continue;
} else {
int j = i + 1;
while(j < origin_string.length() && origin_string[j] != '#') {
++j;
}
if((j == origin_string.length()) || (j - i - 1) >= target_buf_len) {
ret_str = ObString::make_string("error shard key");
} else {
MEMCPY(target_buf, origin_string.ptr() + i + 1, j - i - 1);
ret_str.assign_ptr(target_buf, j - i - 1);
}
break;
}
}
}
return ret_str;
}
ObString ObShowTopologyHandler::get_shard_rule_str(dbconfig::ObShardRule& shard_rule)
{
ObString ret_str = ObString::make_string("empty shard rule");
if (OB_LIKELY(0 != shard_rule.tb_rules_.count())) {
ret_str = shard_rule.tb_rules_.at(0).shard_rule_str_.config_string_;
}
return ret_str;
}
} // end of namespace proxy
} // end of namespace obproxy
} // end of namespace oceanbase
src/obproxy/cmd/ob_show_topology_handler.h
浏览文件 @ bacd5fac
......@@ -22,7 +22,9 @@ namespace obproxy
namespace dbconfig
{
class ObGroupCluster;
class ObShardRule;
}
namespace obutils
{
class ObShowTopologyHandler : public ObCmdHandler
......@@ -31,22 +33,37 @@ public:
ObShowTopologyHandler(event::ObMIOBuffer *buf, ObCmdInfo &info);
virtual ~ObShowTopologyHandler() {}
int handle_show_elastic_id(const common::ObString &tenant_name,
const common::ObString &db_name,
const common::ObString &group_name);
static int show_elastic_id_cmd_callback(event::ObMIOBuffer *buf,
ObCmdInfo &info,
const common::ObString &logic_tenant_name,
const common::ObString &logic_database_name,
const common::ObString &group_name);
int handle_show_topology(const common::ObString &tenant_name,
const common::ObString &db_name,
const common::ObString &group_name);
const common::ObString &table_name);
static int show_topology_cmd_callback(event::ObMIOBuffer *buf,
ObCmdInfo &info,
const common::ObString &logic_tenant_name,
const common::ObString &logic_database_name,
const common::ObString &group_name);
const common::ObString &table_name);
public:
static const int64_t OB_MAX_ELASTIC_ID_COUNT = 256;
private:
int dump_topology_header();
int dump_elastic_id_header();
int dump_elastic_id(const int64_t eid);
int dump_shard_topology_for_single_db(ObString table_name);
int dump_shard_topology_for_shard_db_single_tb(ObString table_name);
int dump_shard_topology_for_shard_db_non_shard_tb(ObString table_name, dbconfig::ObShardRule *shard_rule);
int dump_shard_topology_for_shard_db_shard_tb(dbconfig::ObShardRule *shard_rule);
ObString get_shard_key_str(dbconfig::ObShardRule& shard_rule, char* const target_buf, const int target_buf_len);
ObString get_shard_rule_str(dbconfig::ObShardRule& shard_rule);
private:
DISALLOW_COPY_AND_ASSIGN(ObShowTopologyHandler);
};
......
src/obproxy/cmd/ob_show_vip_handler.cpp
浏览文件 @ bacd5fac
......@@ -16,6 +16,8 @@
#include "utils/ob_proxy_utils.h"
#include "iocore/eventsystem/ob_task.h"
#include "iocore/eventsystem/ob_event_processor.h"
#include "share/config/ob_config.h"
#include "obutils/ob_config_processor.h"
using namespace oceanbase::common;
using namespace oceanbase::share;
......@@ -68,7 +70,6 @@ int ObShowVipHandler::main_handler(int event, void *data)
} else if (OB_FAIL(dump_header())) {
WARN_ICMD("fail to dump header", K(ret));
} else {
ObVipTenantCache &vt_cache = get_global_vip_tenant_processor().get_vt_cache();
ObVipTenant vip_tenant;
if (!like_name_.empty()) {
//dump one vip tenant
......@@ -87,9 +88,9 @@ int ObShowVipHandler::main_handler(int event, void *data)
WARN_ICMD("unexpected argument", K(vip_string), K(match_name), K(ret));
} else if (OB_FAIL(get_int_value(match_name, vport))) {
WARN_ICMD("fail to get_int_value", K(match_name), K(vport), K(ret));
} else if (!vip_addr.addr_.set_ipv4_addr(vip_string, static_cast<int32_t>(vport))) {
} else if (!vip_addr.addr_.set_ip_addr(vip_string, static_cast<int32_t>(vport))) {
ret = OB_INVALID_ARGUMENT;
WARN_ICMD("fail to set_ipv4_addr", K(vip_string), K(vport), K(ret));
WARN_ICMD("fail to set_ip_addr", K(vip_string), K(vport), K(ret));
} else if (!vip_addr.is_valid()) {
ret = OB_INVALID_ARGUMENT;
WARN_ICMD("invalid vip_addr", K(vip_addr), K(match_name), K(ret));
......@@ -99,32 +100,38 @@ int ObShowVipHandler::main_handler(int event, void *data)
}
if (OB_SUCC(ret)) {
if (OB_FAIL(vt_cache.get(vip_addr, vip_tenant))) {
WARN_ICMD("fail to get vip_tenant", K(ret));
if (OB_ENTRY_NOT_EXIST == ret) {
//empty result
ret = OB_SUCCESS;
ObConfigItem tenant_item;
ObConfigItem cluster_item;
bool found = false;
vip_tenant.vip_addr_ = vip_addr;
if (OB_FAIL(get_global_config_processor().get_proxy_config_with_level(
vip_tenant.vip_addr_, "", "", "proxy_tenant_name", tenant_item, "LEVEL_VIP", found))) {
WARN_ICMD("get proxy tenant name config failed", K(vip_tenant.vip_addr_), K(ret));
}
if (OB_SUCC(ret) && found) {
if (OB_FAIL(get_global_config_processor().get_proxy_config_with_level(
vip_tenant.vip_addr_, "", "", "rootservice_cluster_name", cluster_item, "LEVEL_VIP", found))) {
WARN_ICMD("get cluster name config failed", K(vip_tenant.vip_addr_), K(ret));
}
}
if (OB_SUCC(ret) && found) {
if (OB_FAIL(vip_tenant.set_tenant_cluster(tenant_item.str(), cluster_item.str()))) {
WARN_ICMD("set tenant and cluster name failed", K(tenant_item), K(cluster_item), K(ret));
} else if (OB_FAIL(dump_item(vip_tenant))) {
WARN_ICMD("fail to dump item", K(vip_tenant), K(ret));
}
} else if (OB_FAIL(dump_item(vip_tenant))) {
WARN_ICMD("fail to dump item", K(vip_tenant), K(ret));
} else {
WARN_ICMD("fail to get vip_tenant", K(ret));
}
} else {
ret = OB_ERR_OPERATOR_UNKNOWN;//return this errno
}
} else {
//dump all vip tenant
ObVipTenantCache::VTHashMap *cache_map = NULL;
obsys::CRLockGuard guard(vt_cache.rwlock_);
if (OB_ISNULL(cache_map = vt_cache.get_cache_map())) {
ret = OB_ERR_UNEXPECTED;
WARN_ICMD("cache_map is null", K(ret));
} else {
for (ObVipTenantCache::VTHashMap::iterator it = cache_map->begin();
OB_SUCC(ret) && it != cache_map->end(); ++it) {
if (OB_FAIL(dump_item(*it))) {
WARN_ICMD("fail to dump item", K(*it), K(ret));
}
}
const char* select_sql = "SELECT a.vid, a.vip, a.vport, a.value, b.value FROM proxy_config as a, proxy_config as b "
"where a.name = 'proxy_tenant_nane' and b.name = 'rootservice_cluster_name';";
if (get_global_config_processor().execute(select_sql, ObShowVipHandler::sqlite3_callback, this)) {
WARN_ICMD("fail to execute sql", K(ret));
}
}
}
......@@ -180,6 +187,40 @@ int ObShowVipHandler::dump_item(const ObVipTenant &vip_tenant)
return ret;
}
int ObShowVipHandler::sqlite3_callback(void *data, int argc, char **argv, char **column_name)
{
int ret = OB_SUCCESS;
if (OB_UNLIKELY(NULL == data || NULL == argv || NULL == column_name || 5 != argc)) {
ret = OB_ERR_UNEXPECTED;
WARN_ICMD("argument is unexpected", K(argc), K(ret));
} else {
ObShowVipHandler *handler = reinterpret_cast<ObShowVipHandler*>(data);
char* vip;
int64_t vport = 0;
int64_t vid = -1;
ObVipTenant vip_tenant;
ObString tenant;
ObString cluster;
if (OB_UNLIKELY(NULL == argv[0] || NULL == argv[1] || NULL == argv[2] || NULL == argv[3] || NULL == argv[4])) {
ret = OB_ERR_UNEXPECTED;
WARN_ICMD("argument is unexpected", K(ret));
} else {
vid = atoi(argv[0]);
vip = argv[1];
vport = atoi(argv[2]);
tenant = argv[3];
cluster = argv[4];
vip_tenant.vip_addr_.set(vip, static_cast<int32_t>(vport), vid);
if (OB_FAIL(vip_tenant.set_tenant_cluster(tenant, cluster))) {
WARN_ICMD("fail to set tenant cluster", K(vip_tenant), K(ret));
} else if (OB_FAIL(handler->dump_item(vip_tenant))) {
WARN_ICMD("fail to dump item", K(vip_tenant), K(ret));
}
}
}
return ret;
}
static int show_vip_cmd_callback(ObContinuation *cont, ObInternalCmdInfo &info,
ObMIOBuffer *buf, ObAction *&action)
{
......
src/obproxy/cmd/ob_show_vip_handler.h
浏览文件 @ bacd5fac
......@@ -25,7 +25,8 @@ namespace obutils
class ObShowVipHandler : public ObInternalCmdHandler
{
public:
ObShowVipHandler(event::ObContinuation *cont, event::ObMIOBuffer *buf, const ObInternalCmdInfo &info);
ObShowVipHandler(event::ObContinuation *cont, event::ObMIOBuffer *buf,
const ObInternalCmdInfo &info);
virtual ~ObShowVipHandler() {}
int main_handler(int event, void *data);
......@@ -33,6 +34,7 @@ private:
int dump_header();
int dump_body();
int dump_item(const ObVipTenant &vip_tenant);
static int sqlite3_callback(void *data, int argc, char **argv, char **column_name);
DISALLOW_COPY_AND_ASSIGN(ObShowVipHandler);
};
......
src/obproxy/dbconfig/grpc/ob_proxy_grpc_client.cpp
浏览文件 @ bacd5fac
......@@ -156,15 +156,15 @@ int ObGrpcClient::fill_request_node(DiscoveryRequest &request)
const char *is_sidecar = getenv("IS_SIDECAR");
if (OB_NOT_NULL(is_sidecar) && OB_UNLIKELY(strncmp(is_sidecar, "false", strlen(is_sidecar)) == 0)) {// app is not in sidecar
char ipstr[OB_IP_STR_BUFF];
MEMSET(ipstr, 0, OB_IP_STR_BUFF);
char ipstr[MAX_IP_ADDR_LENGTH];
MEMSET(ipstr, 0, MAX_IP_ADDR_LENGTH);
ObSEArray<ObStringKV, LABELS_NUMS> labels;
char json[MAX_REQUEST_LABELS_LENGTH];
MEMSET(json, 0, sizeof(json));
if (OB_FAIL(getip_from_nodeid(ipstr, OB_IP_STR_BUFF))) {
if (OB_FAIL(getip_from_nodeid(ipstr, MAX_IP_ADDR_LENGTH))) {
LOG_WARN("can not get ip from node id", K(ret));
} else if (OB_UNLIKELY(strlen(ipstr) == 0)) {
ret = OB_ERR_UNEXPECTED;
......@@ -227,7 +227,7 @@ int ObGrpcClient::getip_from_nodeid(char *ipstr, int64_t length)
if (OB_SUCC(ret)) {
if (OB_UNLIKELY(length < ip.length())) {
ret = OB_SIZE_OVERFLOW;
LOG_WARN("IP string in node id is longer than OB_IP_STR_BUFF", K(length), K(ip.length()), K(ret));
LOG_WARN("IP string in node id is longer than MAX_IP_ADDR_LENGTH", K(length), K(ip.length()), K(ret));
} else {
memcpy(ipstr, ip.ptr(), ip.length());
ipstr[ip.length()] = '\0';
......@@ -321,7 +321,17 @@ int ObGrpcClient::to_json(ObIArray<obmysql::ObStringKV>& labels, char *json, int
bool ObGrpcClient::sync_read(DiscoveryResponse &response)
{
return stream_->Read(&response);
bool status = false;
int ret = OB_SUCCESS;
status = stream_->Read(&response);
if (!status) {
LOG_INFO("the stream has been closed, will rebuild the grpc client stream");
finish_rpc();
if (OB_FAIL(init_stream())) {
LOG_WARN("fail to init stream", K(ret));
}
}
return status;
}
void ObGrpcClient::finish_rpc()
......
src/obproxy/dbconfig/ob_proxy_db_config_info.cpp
浏览文件 @ bacd5fac
......@@ -1636,10 +1636,9 @@ int ObShardRule::get_physic_index(const SqlFieldResult &sql_result,
const ObProxyShardRuleInfo &rule = rules.at(i);
ObProxyExprCtx expr_ctx(physic_size, type, is_elastic_index, &allocator);
ObProxyExpr *proxy_expr = rule.expr_;
ObObj result_obj;
ObSEArray<ObObj, 4> result_array;
int64_t tmp_index = OBPROXY_MAX_DBMESH_ID;
int64_t tmp_index_for_one_obj = OBPROXY_MAX_DBMESH_ID;
LOG_DEBUG("begin to calc rule", K(rule));
ObProxyExprCalcItem calc_item(const_cast<SqlFieldResult*>(&sql_result));
......@@ -1654,16 +1653,23 @@ int ObShardRule::get_physic_index(const SqlFieldResult &sql_result,
LOG_WARN("calc proxy expr failed", K(ret));
}
} else {
result_obj = result_array.at(0);
ObObj tmp_obj;
if (OB_FAIL(ObProxyFuncExpr::get_int_obj(result_obj, tmp_obj))) {
LOG_WARN("get int obj failed", K(result_obj), K(ret));
} else if (OB_FAIL(tmp_obj.get_int(tmp_index))) {
LOG_WARN("get int failed", K(ret));
} else {
if (tmp_index < 0 || tmp_index >= physic_size) {
ret = OB_EXPR_CALC_ERROR;
LOG_WARN("invalid index", K(tmp_index), K(physic_size), K(ret));
for(int64_t j = 0; OB_SUCC(ret) && j < result_array.count(); j++) {
ObObj& result_obj = result_array.at(j);
ObObj tmp_obj;
if (OB_FAIL(ObProxyFuncExpr::get_int_obj(result_obj, tmp_obj, allocator))) {
LOG_WARN("get int obj failed", K(result_obj), K(ret));
} else if (OB_FAIL(tmp_obj.get_int(tmp_index_for_one_obj))) {
LOG_WARN("get int failed", K(ret));
} else if (tmp_index_for_one_obj < 0 || tmp_index_for_one_obj >= physic_size) {
ret = OB_EXPR_CALC_ERROR;
LOG_WARN("invalid index", K(tmp_index_for_one_obj), K(physic_size), K(ret));
} else if (OBPROXY_MAX_DBMESH_ID == tmp_index) {
tmp_index = tmp_index_for_one_obj;
} else if (tmp_index != tmp_index_for_one_obj) {
ret = OB_ERR_DISTRIBUTED_NOT_SUPPORTED;
LOG_WARN("different physic index for one sharding key is not supported", K(tmp_index), K(last_index), K(ret));
} else {
//nothing
}
}
}
......@@ -1749,7 +1755,7 @@ int ObShardRule::get_physic_index_array(const SqlFieldResult &sql_result,
for (int64_t i = 0; OB_SUCC(ret) && i < result_obj_array.count(); i++) {
ObObj tmp_obj;
int64_t tmp_index;
if (OB_FAIL(ObProxyFuncExpr::get_int_obj(result_obj_array.at(i), tmp_obj))) {
if (OB_FAIL(ObProxyFuncExpr::get_int_obj(result_obj_array.at(i), tmp_obj, allocator))) {
LOG_WARN("get int obj failed", K(ret), K(result_obj_array.at(i)));
} else if (OB_FAIL(tmp_obj.get_int(tmp_index))) {
LOG_WARN("get int failed", K(ret));
......@@ -2009,6 +2015,91 @@ int ObDbConfigLogicDb::get_shard_router(const ObString &tb_name, ObShardRouter *
return ret;
}
int ObDbConfigLogicDb::get_first_group_shard_connector(ObShardConnector *&shard_conn, int64_t es_id,
bool is_read_stmt, ObTestLoadType testload_type)
{
int ret = OB_SUCCESS;
ObShardTpo *shard_tpo = NULL;
ObGroupCluster *gc_info = NULL;
if (OB_FAIL(get_shard_tpo(shard_tpo))) {
LOG_WARN("fail to get shard tpo info", K(ret));
} else if (OB_ISNULL(shard_tpo)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("shard tpo info is null", K(ret));
} else if (is_single_shard_db_table()) {
ObShardTpo::GCHashMap::iterator it = (const_cast<ObShardTpo::GCHashMap &>(shard_tpo->gc_map_)).begin();
gc_info = &(*it);
} else {
ObShardRule *logic_tb_info = NULL;
{
ObDbConfigCache &dbconfig_cache = get_global_dbconfig_cache();
obsys::CRLockGuard guard(dbconfig_cache.rwlock_);
ObDbConfigChildArrayInfo<ObShardRouter>::CCRHashMap &map = const_cast<ObDbConfigChildArrayInfo<ObShardRouter>::CCRHashMap &>(sr_array_.ccr_map_);
for (ObDbConfigChildArrayInfo<ObShardRouter>::CCRHashMap::iterator it = map.begin(); it != map.end(); ++it) {
ObShardRouter::MarkedRuleHashMap &mr_map = it->mr_map_;
for (ObShardRouter::MarkedRuleHashMap::iterator sub_it = mr_map.begin();
sub_it != mr_map.end(); ++sub_it) {
logic_tb_info = &(*sub_it);
break;
}
}
}
if (OB_SUCC(ret)) {
char real_database_name[OB_MAX_DATABASE_NAME_LENGTH];
if (OB_FAIL(logic_tb_info->get_real_name_by_index(logic_tb_info->db_size_, logic_tb_info->db_suffix_len_, 0,
logic_tb_info->db_prefix_.config_string_,
logic_tb_info->db_tail_.config_string_,
real_database_name, OB_MAX_DATABASE_NAME_LENGTH))) {
LOG_WARN("fail to get real group name", KPC(logic_tb_info), K(ret));
} else if (OB_FAIL(shard_tpo->get_group_cluster(ObString::make_string(real_database_name), gc_info))) {
LOG_DEBUG("group does not exist", "phy_db_name", real_database_name, K(ret));
} else if (OB_ISNULL(gc_info)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("group cluster info is null", "phy_db_name", real_database_name, K(ret));
}
}
}
if (OB_SUCC(ret)) {
// Calculate es_id
int64_t es_size = gc_info->get_es_size();
ObString shard_name;
if (-1 == es_id || OBPROXY_MAX_DBMESH_ID == es_id) {
if (OB_FAIL(gc_info->get_elastic_id_by_weight(es_id, is_read_stmt))) {
LOG_WARN("fail to get eid by read weight", KPC(gc_info));
} else {
LOG_DEBUG("succ to get eid by weight", K(es_id));
}
} else if (OB_UNLIKELY(es_id >= es_size)) {
ret = OB_EXPR_CALC_ERROR;
LOG_WARN("es index is larger than elastic array", K(es_id), K(es_size), K(ret));
}
}
if (OB_SUCC(ret)) {
ObString shard_name;
if (FALSE_IT(shard_name = gc_info->get_shard_name_by_eid(es_id))) {
} else if (TESTLOAD_NON != testload_type) {
if (OB_FAIL(get_testload_shard_connector(shard_name, testload_prefix_.config_string_, shard_conn))) {
LOG_WARN("testload shard connector not exist", "testload_type", get_testload_type_str(testload_type),
K(shard_name), "testload_prefix", testload_prefix_, K(ret));
}
} else if (OB_FAIL(get_shard_connector(shard_name, shard_conn))) {
LOG_WARN("shard connector does not exist", K(shard_name), K(ret));
}
}
if (NULL != shard_tpo) {
shard_tpo->dec_ref();
shard_tpo = NULL;
}
return ret;
}
int ObDbConfigLogicDb::get_all_shard_table(ObIArray<ObString> &all_table)
{
int ret = OB_SUCCESS;
......@@ -3729,7 +3820,8 @@ int ObDbConfigLogicDb::get_shard_table_info(const ObString &table_name,
char *real_table_name, int64_t tb_name_len,
int64_t &group_index, int64_t &tb_index, int64_t &es_index,
const ObString &hint_table, ObTestLoadType testload_type,
const bool is_read_stmt)
const bool is_read_stmt,
const int64_t last_es_index /*OBPROXY_MAX_DBMESH_ID*/)
{
int ret = OB_SUCCESS;
......@@ -3766,7 +3858,12 @@ int ObDbConfigLogicDb::get_shard_table_info(const ObString &table_name,
bool is_elastic_index = true;
ObString shard_name;
if (-1 == es_index || (OBPROXY_MAX_DBMESH_ID == es_index && logic_tb_info->es_rules_.empty())) {
if (OB_FAIL(gc_info->get_elastic_id_by_weight(es_index, is_read_stmt))) {
if (is_read_stmt
&& (last_es_index >= 0)
&& (last_es_index < es_size)) {
//In multiple statements, the read request can use the effective es id of the previous statement
es_index = last_es_index;
} else if (OB_FAIL(gc_info->get_elastic_id_by_weight(es_index, is_read_stmt))) {
LOG_WARN("fail to get eid by read weight", KPC(gc_info));
} else {
LOG_DEBUG("succ to get eid by weight", K(es_index));
......@@ -3850,7 +3947,7 @@ int ObDbConfigLogicDb::get_single_table_info(const ObString &table_name,
char *real_table_name, int64_t tb_name_len,
int64_t &group_id, int64_t &table_id, int64_t &es_id,
const ObString &hint_table, ObTestLoadType testload_type,
const bool is_read_stmt)
const bool is_read_stmt, const int64_t last_es_id/*OBPROXY_MAX_DBMESH_ID*/)
{
int ret = OB_SUCCESS;
......@@ -3871,7 +3968,12 @@ int ObDbConfigLogicDb::get_single_table_info(const ObString &table_name,
const ObGroupCluster &gc_info = *it;
int64_t es_size = gc_info.get_es_size();
if (es_id < 0 || OBPROXY_MAX_DBMESH_ID == es_id) {
if (OB_FAIL(gc_info.get_elastic_id_by_weight(es_id, is_read_stmt))) {
if (is_read_stmt
&& (last_es_id >= 0)
&& (last_es_id < es_size)) {
//In multiple statements, the read request can use the effective es id of the previous statement
es_id = last_es_id;
} if (OB_FAIL(gc_info.get_elastic_id_by_weight(es_id, is_read_stmt))) {
LOG_WARN("fail to get random eid", K(gc_info));
}
} else if (OB_UNLIKELY(es_id >= es_size)) {
......@@ -4557,8 +4659,8 @@ int ObDbConfigLogicDb::get_table_name_by_index(ObSqlParseResult &parse_result,
{
int ret = OB_SUCCESS;
ObProxySelectStmt *select_stmt = static_cast<ObProxySelectStmt*>(parse_result.get_proxy_stmt());
ObProxySelectStmt::ExprMap &table_exprs_map = select_stmt->get_table_exprs_map();
ObProxyDMLStmt *dml_stmt = static_cast<ObProxyDMLStmt*>(parse_result.get_proxy_stmt());
ObProxyDMLStmt::ExprMap &table_exprs_map = dml_stmt->get_table_exprs_map();
SqlFieldResult &sql_result = parse_result.get_sql_filed_result();
char real_table_name[OB_MAX_TABLE_NAME_LENGTH];
......@@ -4570,8 +4672,8 @@ int ObDbConfigLogicDb::get_table_name_by_index(ObSqlParseResult &parse_result,
for (int64_t i = 0; OB_SUCC(ret) && i < table_index_array.count(); i++) {
int64_t tb_index = table_index_array.at(i);
ObProxySelectStmt::ExprMap::iterator iter = table_exprs_map.begin();
ObProxySelectStmt::ExprMap::iterator end = table_exprs_map.end();
ObProxyDMLStmt::ExprMap::iterator iter = table_exprs_map.begin();
ObProxyDMLStmt::ExprMap::iterator end = table_exprs_map.end();
table_name_map_wrapper.reuse();
for (; OB_SUCC(ret) && iter != end; iter++) {
......
src/obproxy/dbconfig/ob_proxy_db_config_info.h
浏览文件 @ bacd5fac
......@@ -936,7 +936,8 @@ public:
char *real_table_name, int64_t tb_name_len,
int64_t &group_id, int64_t &table_id, int64_t &es_id,
const common::ObString &hint_table,
ObTestLoadType testload_type, const bool is_read_stmt);
ObTestLoadType testload_type, const bool is_read_stmt,
const int64_t last_es_id = OBPROXY_MAX_DBMESH_ID);
int get_shard_table_info(const common::ObString &table_name,
obutils::SqlFieldResult &sql_result,
ObShardConnector *&shard_conn,
......@@ -944,7 +945,8 @@ public:
char *real_table_name, int64_t tb_name_len,
int64_t &group_index, int64_t &tb_index, int64_t &es_index,
const common::ObString &hint_table,
ObTestLoadType testload_type, const bool is_read_stmt);
ObTestLoadType testload_type, const bool is_read_stmt,
const int64_t last_es_index = OBPROXY_MAX_DBMESH_ID);
int get_shard_prop(const common::ObString & shard_name,
ObShardProp* &shard_prop);
int get_real_table_name(const ObString &table_name, obutils::SqlFieldResult &sql_result,
......@@ -968,6 +970,8 @@ public:
int64_t es_index,
common::ObIArray<int64_t> &group_index_array,
common::ObIArray<ObShardConnector*> &shard_connector_array);
int get_first_group_shard_connector(ObShardConnector *&shard_conn, int64_t es_id,
bool is_read_stmt, ObTestLoadType testload_type);
int get_table_name_by_index(obutils::ObSqlParseResult &parse_result,
ObTestLoadType testload_type,
ObIAllocator &allocator,
......
src/obproxy/driver/c/Makemodule.am 0 → 100644
浏览文件 @ bacd5fac
obproxy_c_driver_sources:=\
obproxy/driver/c/ob_proxy_vc_c.h\
obproxy/driver/c/ob_proxy_vc_c.cpp
\ No newline at end of file
src/obproxy/driver/c/ob_proxy_vc_c.cpp 0 → 100644
浏览文件 @ bacd5fac
/**
* Copyright (c) 2021 OceanBase
* OceanBase Database Proxy(ODP) is licensed under Mulan PubL v2.
* You can use this software according to the terms and conditions of the Mulan PubL v2.
* You may obtain a copy of Mulan PubL v2 at:
* http://license.coscl.org.cn/MulanPubL-2.0
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*/
#include "ob_proxy_vc_c.h"
#include "proxy/client/ob_driver_client.h"
#include "ob_proxy_init.h"
#include "lib/oblog/ob_log.h"
#include "lib/objectpool/ob_concurrency_objpool.h"
#include <errno.h>
using namespace oceanbase::common;
using namespace oceanbase::obproxy;
using namespace oceanbase::obproxy::proxy;
int ob_proxy_vc_c_init(const char *config, void **p_driver_client)
{
int ret = OB_SUCCESS;
ObDriverClient *driver_client = NULL;
if (OB_ISNULL(config) || OB_ISNULL(p_driver_client)) {
ret = OB_INVALID_ARGUMENT;
MPRINT("invalid argument config=%p, p_driver_client=%p, ret=%d", config, p_driver_client, ret);
} else if (OB_FAIL(init_obproxy_client(config))) {
MPRINT("init obproxy client failed, ret=%d", ret);
} else if (OB_ISNULL(driver_client = new(std::nothrow) ObDriverClient())) {
ret = OB_ALLOCATE_MEMORY_FAILED;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "driver client is null unexpected", K(ret));
} else if (OB_FAIL(driver_client->init())) {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "driver client init failed", K(ret));
} else {
*p_driver_client = static_cast<void *>(driver_client);
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "create driver_client success", KP(driver_client));
}
if(OB_FAIL(ret) && OB_NOT_NULL(driver_client)) {
delete driver_client;
}
return ret;
}
int ob_proxy_vc_c_connect(void *driver_client_handle, int64_t conn_timeout, int64_t sock_timeout)
{
int ret = OB_SUCCESS;
ObDriverClient *driver_client;
if (OB_ISNULL(driver_client_handle) || conn_timeout < 0 || sock_timeout < 0) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), K(conn_timeout), K(sock_timeout), K(ret));
} else {
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
driver_client->set_connect_timeout(conn_timeout);
if (conn_timeout == 0 && OB_FAIL(driver_client->sync_connect())) {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "driver client connect failed", K(ret));
} else if (conn_timeout > 0 && OB_FAIL(driver_client->sync_connect_with_timeout())) {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "driver client connect with timeout failed", K(conn_timeout), K(ret));
} else {
driver_client->set_send_timeout(sock_timeout);
driver_client->set_recv_timeout(sock_timeout);
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "driver client connect success", K(driver_client));
}
}
return ret;
}
int ob_proxy_vc_c_recv(void *driver_client_handle, char *buffer, int64_t offset, int64_t mlen, int64_t *recv_len, int64_t flag)
{
int ret = OB_SUCCESS;
int64_t rlen = 0;
ObDriverClient *driver_client;
UNUSED(flag);
if (OB_ISNULL(driver_client_handle) || OB_ISNULL(buffer) || OB_ISNULL(recv_len)) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), K(buffer), K(recv_len), K(ret));
} else {
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
if (OB_FAIL(driver_client->sync_recv_with_timeout(buffer + offset, mlen, rlen))) {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "sync recv failed", K(ret));
} else {
if (rlen < 0) {
switch (errno) {
case ECONNRESET:
case EPIPE:
OBPROXY_DRIVER_CLIENT_LOG(WARN, "Connection reset", "errno", errno);
break;
case EBADF:
OBPROXY_DRIVER_CLIENT_LOG(WARN, "Socket closed", "errno", errno);
break;
case EINTR:
OBPROXY_DRIVER_CLIENT_LOG(WARN, "Operation interrupted", "errno", errno);
break;
default:
OBPROXY_DRIVER_CLIENT_LOG(WARN, "socket read error", "errno", errno);
break;
}
ret = OB_ERR_UNEXPECTED;
} else if (rlen == 0) {
ret = OB_ERR_UNEXPECTED;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "peer close", KP(driver_client), K(ret));
} else {
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "read success", K(rlen));
*recv_len = rlen;
}
}
}
return ret;
}
int ob_proxy_vc_c_send(void *driver_client_handle, const char *buffer, int64_t offset, int64_t mlen, int64_t *send_len, int64_t flag)
{
int ret = OB_SUCCESS;
int64_t wlen = 0;
ObDriverClient *driver_client;
UNUSED(flag);
if (OB_ISNULL(driver_client_handle) || OB_ISNULL(buffer) || OB_ISNULL(send_len)) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), K(buffer), K(send_len), K(ret));
} else {
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
if (OB_FAIL(driver_client->sync_send_with_timeout(buffer + offset, mlen, wlen))) {
ret = OB_ERR_UNEXPECTED; //send error;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "sync_send_with_timeout failed", K(wlen), K(ret));
} else {
if (wlen > 0) {
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "sync write success", K(wlen));
*send_len = wlen;
} else {
ret = ob_get_sys_errno();
if (errno == ECONNRESET) {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "wlen is unexpected, connection reset", K(wlen), K(ret));
} else {
OBPROXY_DRIVER_CLIENT_LOG(WARN, "wlen is unexpected, write failed", K(wlen), K(ret));
}
}
}
}
return ret;
}
int ob_proxy_vc_c_close(void *driver_client_handle)
{
int ret = OB_SUCCESS;
ObDriverClient *driver_client;
if (OB_ISNULL(driver_client_handle)) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), K(ret));
} else {
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "free driver client", KP(driver_client_handle));
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
delete driver_client;
driver_client = NULL;
}
return ret;
}
int ob_proxy_vc_set_timeout(void *driver_client_handle, int type, int timeout)
{
int ret = OB_SUCCESS;
ObDriverClient *driver_client;
const int C_DRIVER_CONNECT_TIMEOUT = 0;
const int C_DRIVER_READ_TIMEOUT = 1;
const int C_DRIVER_WRITE_TIMEOUT = 2;
const int C_DRIVER_TIMEOUT_MAX = 3;
if (OB_ISNULL(driver_client_handle) || type < 0 || C_DRIVER_TIMEOUT_MAX <= type) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), K(type), K(ret));
} else {
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
if (C_DRIVER_CONNECT_TIMEOUT == type) {
driver_client->set_connect_timeout(timeout);
} else if (C_DRIVER_READ_TIMEOUT == type) {
driver_client->set_recv_timeout(timeout);
} else if (C_DRIVER_WRITE_TIMEOUT == type) {
driver_client->set_send_timeout(timeout);
}
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "driver client set timeout succ", K(type), K(timeout));
}
return ret;
}
int ob_proxy_vc_get_timeout(void *driver_client_handle, int type, int *timeout)
{
int ret = OB_SUCCESS;
ObDriverClient *driver_client;
const int C_DRIVER_CONNECT_TIMEOUT = 0;
const int C_DRIVER_READ_TIMEOUT = 1;
const int C_DRIVER_WRITE_TIMEOUT = 2;
const int C_DRIVER_TIMEOUT_MAX = 3;
if (OB_ISNULL(driver_client_handle) || OB_ISNULL(timeout) || type < C_DRIVER_CONNECT_TIMEOUT || C_DRIVER_TIMEOUT_MAX <= type) {
ret = OB_INVALID_ARGUMENT;
OBPROXY_DRIVER_CLIENT_LOG(WARN, "invalid argument", KP(driver_client_handle), KP(timeout), K(type), K(ret));
} else {
driver_client = static_cast<ObDriverClient *>(driver_client_handle);
if (C_DRIVER_CONNECT_TIMEOUT == type) {
*timeout = static_cast<int>(driver_client->get_connect_timeout());
} else if (C_DRIVER_READ_TIMEOUT == type) {
*timeout = static_cast<int>(driver_client->get_recv_timeout());
} else if (C_DRIVER_WRITE_TIMEOUT == type) {
*timeout = static_cast<int>(driver_client->get_send_timeout());
}
OBPROXY_DRIVER_CLIENT_LOG(DEBUG, "driver client get timeout succ", K(type), "timeout", *timeout);
}
return ret;
}
\ No newline at end of file
src/obproxy/driver/c/ob_proxy_vc_c.h 0 → 100644
浏览文件 @ bacd5fac
/**
* Copyright (c) 2021 OceanBase
* OceanBase Database Proxy(ODP) is licensed under Mulan PubL v2.
* You can use this software according to the terms and conditions of the Mulan PubL v2.
* You may obtain a copy of Mulan PubL v2 at:
* http://license.coscl.org.cn/MulanPubL-2.0
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*/
#ifndef _OB_OPROXY_VC_C_H
#define _OB_OPROXY_VC_C_H
#include <stdint.h>
#ifdef __cplusplus
extern "C" {
#endif
/**
* @brief
* libobclient calls this function to init obproxy and get driver_client handle
* @param config obproxy config string
* @param driver_client point to driver_client handle
* @return int OB_SUCCESS means success and other means failure
*/
int ob_proxy_vc_c_init(const char *config, void **driver_client);
/**
* @brief
* libobclient calls this function to connect to obproxy
* @param driver_client driver_client handle
* @param conn_timeout connection timeout
* @param sock_timeout socket timeout
* @return int
*/
int ob_proxy_vc_c_connect(void *driver_client, int64_t conn_timeout, int64_t sock_timeout);
/**
* @brief
* libobclient calls this function to receive data from obproxy
* @param driver_client driver_client handle
* @param buffer receive buffer
* @param offset buffer offset
* @param mlen max receive length
* @param recv_len point to receive length
* @param flag receive flag
* @return int
*/
int ob_proxy_vc_c_recv(void *driver_client, char *buffer, int64_t offset, int64_t mlen, int64_t *recv_len, int64_t flag);
/**
* @brief
* libobclient calls this function to send data to obproxy
* @param driver_client driver_client handle
* @param buffer send buffer
* @param offset buffer offset
* @param mlen send buffer len
* @param send_len point to send length
* @param flag send flag
* @return int
*/
int ob_proxy_vc_c_send(void *driver_client, const char *buffer, int64_t offset, int64_t mlen, int64_t *send_len, int64_t flag);
/**
* @brief
* libobclient calls this function to close driver client, but not stop obproxy service
* @param driver_client driver_client handle
* @return int
*/
int ob_proxy_vc_c_close(void *driver_client);
/**
* @brief
* libobclient calls this function to set timeout
* @param driver_client driver_client handle
* @param type type for timeout
* @param timeout ms
* @return int
*/
int ob_proxy_vc_set_timeout(void *driver_client, int type, int timeout);
/**
* @brief
* libobclient calls this function to get timeout
* @param driver_client driver_client handle
* @param type type for timeout
* @param timeout ms
* @return int
*/
int ob_proxy_vc_get_timeout(void *driver_client_handle, int type, int *timeout);
#ifdef __cplusplus
}
#endif /* __cplusplus */
#endif /*_OB_OPROXY_VC_JNI_H */
\ No newline at end of file
src/obproxy/engine/Makemodule.am
浏览文件 @ bacd5fac
......@@ -11,4 +11,6 @@ obproxy/engine/ob_proxy_operator_agg.cpp\
obproxy/engine/ob_proxy_operator_table_scan.h\
obproxy/engine/ob_proxy_operator_table_scan.cpp\
obproxy/engine/ob_proxy_operator_async_task.h\
obproxy/engine/ob_proxy_operator_async_task.cpp
obproxy/engine/ob_proxy_operator_async_task.cpp\
obproxy/engine/ob_proxy_operator_cont.h\
obproxy/engine/ob_proxy_operator_cont.cpp
src/obproxy/engine/ob_proxy_operator_agg.cpp
浏览文件 @ bacd5fac
......@@ -604,7 +604,7 @@ int ObProxyStreamAggOp::handle_response_result(void *data, bool &is_final, ObPro
ResultRow *row = NULL;
while (OB_SUCC(ret) && OB_SUCC(opres->next(row))) {
ObProxyGroupUnit group_unit(allocator_);
if (OB_FAIL(group_unit.init(row, group_exprs))) {
if (OB_FAIL(group_unit.init(row, result_fields_, group_exprs))) {
LOG_WARN("fail to init group unit", K(ret));
} else if (NULL == current_group_unit_) {
if (OB_FAIL(ObProxyGroupUnit::create_group_unit(allocator_, current_group_unit_, group_unit))) {
......@@ -732,7 +732,7 @@ int ObProxyMemMergeAggOp::handle_response_result(void *data, bool &is_final, ObP
while (OB_SUCC(ret) && OB_SUCC(opres->next(row))) {
ObProxyGroupUnit group_unit(allocator_);
ObProxyGroupUnit *current_group_unit = NULL;
if (OB_FAIL(group_unit.init(row, group_exprs))) {
if (OB_FAIL(group_unit.init(row, result_fields_, group_exprs))) {
LOG_WARN("fail to init group unit", K(ret));
} else if (OB_FAIL(group_unit_map_.get_refactored(group_unit, current_group_unit))) {
if (OB_HASH_NOT_EXIST == ret) {
......@@ -827,7 +827,8 @@ ObProxyGroupUnit::~ObProxyGroupUnit()
}
}
int ObProxyGroupUnit::init(ResultRow *row, const ObIArray<ObProxyGroupItem*>& group_exprs)
int ObProxyGroupUnit::init(ResultRow *row, ResultFields *result_fields,
const ObIArray<ObProxyGroupItem*>& group_exprs)
{
int ret = OB_SUCCESS;
......@@ -837,11 +838,21 @@ int ObProxyGroupUnit::init(ResultRow *row, const ObIArray<ObProxyGroupItem*>& gr
for (int64_t i = 0; OB_SUCC(ret) && i < group_exprs.count(); i++) {
if (OB_FAIL(group_exprs.at(i)->calc(ctx, calc_item, group_values_))) {
LOG_WARN("fail to calc group exprs", K(ret));
} else {
int64_t index = group_exprs.at(i)->get_expr()->get_index();
if (-1 != index) {
ObObj &value = group_values_.at(i);
if (OB_FAIL(change_sql_value(value, result_fields->at(index), &allocator_))) {
LOG_WARN("fail to change sql value", K(value),
"filed", result_fields->at(index), K(ret));
}
}
}
}
if (OB_SUCC(ret)) {
row_ = row;
result_fields_ = result_fields;
}
return ret;
......@@ -880,6 +891,7 @@ int ObProxyGroupUnit::assign(const ObProxyGroupUnit &group_unit)
LOG_WARN("fail to assign group value", K(ret));
} else {
row_ = group_unit.get_row();
result_fields_ = group_unit.get_result_fields();
}
return ret;
......@@ -898,8 +910,19 @@ int ObProxyGroupUnit::do_aggregate(ResultRow *row)
agg_values.reuse();
if (OB_FAIL(agg_unit->get_agg_expr()->calc(ctx, calc_item, agg_values))) {
LOG_WARN("fail to calc agg expr", K(ret));
} else if (OB_FAIL(agg_unit->merge(agg_values))) {
LOG_WARN("fail to merge agg value", K(ret));
} else {
int64_t index = agg_unit->get_agg_expr()->get_index();
if (-1 != index) {
ObObj &value = agg_values.at(0);
if (OB_FAIL(change_sql_value(value, result_fields_->at(index), &allocator_))) {
LOG_WARN("fail to change sql value", K(value),
"filed", result_fields_->at(index), K(ret));
}
}
if (OB_SUCC(ret) && OB_FAIL(agg_unit->merge(agg_values))) {
LOG_WARN("fail to merge agg value", K(ret));
}
}
}
......
src/obproxy/engine/ob_proxy_operator_agg.h
浏览文件 @ bacd5fac
......@@ -350,12 +350,13 @@ class ObProxyGroupUnit
{
public:
ObProxyGroupUnit(common::ObIAllocator &allocator)
: allocator_(allocator), row_(NULL),
: allocator_(allocator), row_(NULL), result_fields_(NULL),
group_values_(ObModIds::OB_SE_ARRAY_ENGINE, ENGINE_ARRAY_NEW_ALLOC_SIZE),
agg_units_(ObModIds::OB_SE_ARRAY_ENGINE, ENGINE_ARRAY_NEW_ALLOC_SIZE) {}
~ObProxyGroupUnit();
int init(ResultRow *row, const common::ObIArray<ObProxyGroupItem*>& group_by_exprs);
int init(ResultRow *row, ResultFields *result_fields,
const common::ObIArray<ObProxyGroupItem*>& group_by_exprs);
uint64_t hash() const;
bool operator==(const ObProxyGroupUnit &group_unit) const;
......@@ -366,6 +367,7 @@ public:
int set_agg_value();
ResultRow *get_row() const { return row_; }
ResultFields *get_result_fields() const { return result_fields_; }
const common::ObIArray<ObObj>& get_group_values() const { return group_values_; }
const common::ObIArray<ObProxyAggUnit*>& get_agg_units() { return agg_units_;; }
......@@ -385,6 +387,7 @@ public:
private:
common::ObIAllocator &allocator_;
ResultRow *row_;
ResultFields *result_fields_;
common::ObSEArray<ObObj, 4> group_values_;
common::ObSEArray<ObProxyAggUnit*, 4> agg_units_;
};
......
src/obproxy/engine/ob_proxy_operator_cont.cpp 0 → 100644
浏览文件 @ bacd5fac
/**
* Copyright (c) 2021 OceanBase
* OceanBase Database Proxy(ODP) is licensed under Mulan PubL v2.
* You can use this software according to the terms and conditions of the Mulan PubL v2.
* You may obtain a copy of Mulan PubL v2 at:
* http://license.coscl.org.cn/MulanPubL-2.0
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*/
#define USING_LOG_PREFIX PROXY
#include "ob_proxy_operator_cont.h"
#include "lib/oblog/ob_log_module.h"
using namespace oceanbase::common;
using namespace oceanbase::obproxy::event;
using namespace oceanbase::obproxy::obutils;
namespace oceanbase
{
namespace obproxy
{
namespace engine
{
int ObProxyOperatorCont::init(ObProxyOperator* operator_root, uint8_t seq, const int64_t timeout_ms)
{
int ret = OB_SUCCESS;
if (OB_UNLIKELY(NULL != buf_)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("unexpected buf is not null", K(ret));
} else if (OB_ISNULL(buf_ = new_empty_miobuffer())) {
ret = OB_ALLOCATE_MEMORY_FAILED;
LOG_WARN("fail to new_empty_miobuffer for buf", K(ret));
} else if (OB_ISNULL(buf_reader_ = buf_->alloc_reader())) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fail to allocate buffer reader", K(ret));
} else {
operator_root_ = operator_root;
timeout_ms_ = timeout_ms;
seq_ = seq;
}
return ret;
}
int ObProxyOperatorCont::main_handler(int event, void *data)
{
int ret = OB_SUCCESS;
if (NULL == execute_thread_) {
execute_thread_ = &self_ethread();
}
switch (event) {
case EVENT_INTERVAL: {
timeout_action_ = NULL;
// If it has been canceled, then the underlying operator has been released,
// and here we need to set pending_action_ to NULL
if (action_.cancelled_) {
pending_action_ = NULL;
}
if (OB_FAIL(handle_timeout())) {
LOG_WARN("fail to handle timeout");
}
break;
}
case EVENT_IMMEDIATE: {
pending_action_ = NULL;
if (OB_FAIL(cancel_timeout_action())) {
LOG_WARN("fail to cancel_timeout_action", K(ret));
} else if (OB_FAIL(handle_event_start())) {
LOG_INFO("fail to handle event start", K(ret));
} else if (OB_FAIL(schedule_timeout())) {
LOG_WARN("fail to schedule timeout action", K(ret));
}
break;
}
case VC_EVENT_READ_COMPLETE: {
pending_action_ = NULL;
if (OB_FAIL(cancel_timeout_action())) {
LOG_WARN("fail to cancel_timeout_action", K(ret));
} else if (OB_FAIL(handle_event_complete(data))) {
LOG_WARN("fail to handle event complete", K(ret));
}
break;
}
case ASYNC_PROCESS_INFORM_OUT_EVENT: {
pending_action_ = NULL;
if (OB_FAIL(cancel_timeout_action())) {
LOG_WARN("fail to cancel_timeout_action", K(ret));
} else if (OB_FAIL(handle_event_inform_out())) {
LOG_WARN("fail to handle inform out event", K(ret));
}
break;
}
case VC_EVENT_ACTIVE_TIMEOUT:
case EVENT_ERROR:
default: {
pending_action_ = NULL;
if (OB_FAIL(cancel_timeout_action())) {
LOG_WARN("fail to cancel_timeout_action", K(ret));
} else if (action_.cancelled_) {
terminate_ = true;
LOG_INFO("async task has been cancelled, will kill itself", K(ret));
} else if (NULL != cb_cont_) {
need_callback_ = true;
} else {
terminate_ = true;
}
if (EVENT_ERROR != event || VC_EVENT_ACTIVE_TIMEOUT != event) {
LOG_WARN("error state, nerver run here", K(event), K(ret));
} else {
LOG_INFO("error state", K(event));
}
break;
}
}
if (!terminate_ && (need_callback_ || OB_FAIL(ret))) {
if (execute_thread_ == &self_ethread()) {
execute_thread_->is_need_thread_pool_event_ = true;
}
if (OB_FAIL(handle_callback())) {
LOG_WARN("fail to handle callback", K(ret));
}
}
if (terminate_) {
if (execute_thread_ == &self_ethread()) {
execute_thread_->is_need_thread_pool_event_ = true;
}
destroy();
}
return EVENT_DONE;
}
int ObProxyOperatorCont::init_task()
{
int ret = OB_SUCCESS;
if (OB_FAIL(operator_root_->open(this, pending_action_, timeout_ms_))) {
LOG_WARN("fail to open operator", K_(timeout_ms), K(ret));
} else if (OB_FAIL(operator_root_->get_next_row())) {
LOG_WARN("fail to get next row", K(ret));
} else if (OB_ISNULL(pending_action_)) {
LOG_WARN("pending action should not be null", K(ret));
}
return ret;
}
int ObProxyOperatorCont::finish_task(void *data)
{
int ret = OB_SUCCESS;
ObProxyResultResp *result_resp = reinterpret_cast<ObProxyResultResp*>(data);
if (result_resp->is_resultset_resp()) {
if (OB_FAIL(build_executor_resp(buf_, seq_, result_resp))) {
LOG_WARN("fail to build shard scan resp", K(ret));
}
} else if (OB_FAIL(ObMysqlPacketUtil::encode_err_packet(*buf_, seq_, result_resp->get_err_code(), result_resp->get_err_msg()))) {
LOG_WARN("fail to encode err pacekt buf", K_(seq), "errmsg", result_resp->get_err_msg(),
"errcode", result_resp->get_err_code(), K(ret));
}
return ret;
}
int ObProxyOperatorCont::schedule_timeout()
{
int ret = OB_SUCCESS;
if (OB_UNLIKELY(NULL != timeout_action_)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("timeout action must be NULL", K_(timeout_action), K(ret));
} else if (OB_ISNULL(timeout_action_ = execute_thread_->schedule_in(this, HRTIME_MSECONDS(timeout_ms_)))) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fail to schedule timeout", K_(timeout_action), K_(timeout_ms), K(ret));
}
return ret;
}
int ObProxyOperatorCont::build_executor_resp(ObMIOBuffer *write_buf, uint8_t &seq, ObProxyResultResp *result_resp)
{
int ret = OB_SUCCESS;
int64_t column_count = result_resp->get_column_count();
ObSEArray<obmysql::ObMySQLField, 1, common::ObIAllocator&> *fields = NULL;
ObSEArray<ObField, 4> ob_fields;
// header , cols , first eof
if (OB_SUCC(ret)) {
if (OB_FAIL(result_resp->get_fields(fields))) {
LOG_WARN("fail to push field", K(fields), K(ret));
} else if (OB_ISNULL(fields)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fields should not be null", K(ret));
} else if (OB_UNLIKELY(fields->count() != column_count)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fields count should equal column_count", K(column_count), "fileds count", fields->count(), K(ret));
} else if (OB_FAIL(ObMysqlPacketUtil::encode_header(*write_buf, seq, *fields))) {
LOG_WARN("fail to encode header", K(fields), K(seq), K(ret));
} else {
for (int64_t i = 0; OB_SUCC(ret) && i < column_count; i++) {
obmysql::ObMySQLField &mysql_field = fields->at(i);
ObField ob_field;
if (OB_FAIL(ObSMUtils::to_ob_field(mysql_field, ob_field))) {
LOG_WARN("fail to covert to ob field", K(mysql_field), K(ret));
} else if (OB_FAIL(ob_fields.push_back(ob_field))) {
LOG_WARN("fail to push ob field", K(ob_field), K(ret));
}
}
}
}
// rows
if (OB_SUCC(ret)) {
ObObj *objs = NULL;
int64_t buf_len = sizeof(ObObj) * column_count;
if (OB_ISNULL(objs = static_cast<ObObj *>(op_fixed_mem_alloc(buf_len)))) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fail to alloc obj array", K(column_count), K(ret));
} else {
ObNewRow row;
//ObSEArray<common::ObObj, 4> *row_array;
ObSEArray<common::ObObj*, 4, common::ObIAllocator&> *row_array;
while ((OB_SUCC(ret)) && (OB_SUCC(result_resp->next(row_array)))) {
if (OB_ISNULL(row_array)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("row_array should not be null", K(ret));
} else if (OB_UNLIKELY(row_array->count() != column_count)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("row_array count should equal column_count", K(column_count), "row_array count", row_array->count(), K(ret));
}
for (int64_t i = 0; OB_SUCC(ret) && i < column_count; i++) {
objs[i] = *(row_array->at(i));
}
if (OB_SUCC(ret)) {
row.cells_ = objs;
row.count_ = column_count;
if (OB_FAIL(ObMysqlPacketUtil::encode_row_packet(*write_buf, seq, row, &ob_fields))) {
LOG_WARN("fail to encode row", K(seq), K(row), K(ret));
} else {
row.reset();
row_array = NULL;
}
}
}
if (OB_ITER_END == ret) {
ret = OB_SUCCESS;
}
}
if (OB_NOT_NULL(objs)) {
op_fixed_mem_free(objs, buf_len);
objs = NULL;
}
}
// second eof
if (OB_SUCC(ret)) {
if (OB_FAIL(ObMysqlPacketUtil::encode_eof_packet(*write_buf, seq))) {
LOG_WARN("fail to encode row", K(seq), K(ret));
}
}
return ret;
}
void ObProxyOperatorCont::destroy()
{
LOG_DEBUG("ObProxyOperatorCont will be destroyed", KP(this));
if (OB_LIKELY(NULL != buf_reader_)) {
buf_reader_->dealloc();
buf_reader_ = NULL;
}
if (OB_LIKELY(NULL != buf_)) {
free_miobuffer(buf_);
buf_ = NULL;
}
ObAsyncCommonTask::destroy();
}
} // end of namespace engine
} // end of namespace obproxy
} // end of namespace oceanbase
src/obproxy/engine/ob_proxy_operator_cont.h 0 → 100644
浏览文件 @ bacd5fac
/**
* Copyright (c) 2021 OceanBase
* OceanBase Database Proxy(ODP) is licensed under Mulan PubL v2.
* You can use this software according to the terms and conditions of the Mulan PubL v2.
* You may obtain a copy of Mulan PubL v2 at:
* http://license.coscl.org.cn/MulanPubL-2.0
* THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
* EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
* MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
* See the Mulan PubL v2 for more details.
*/
#ifndef OBPROXY_OPERATOR_CONT_H
#define OBPROXY_OPERATOR_CONT_H
#include "obutils/ob_async_common_task.h"
#include "ob_proxy_operator.h"
namespace oceanbase
{
namespace obproxy
{
namespace engine
{
class ObProxyOperatorCont : public obutils::ObAsyncCommonTask
{
public:
ObProxyOperatorCont(event::ObContinuation *cb_cont, event::ObEThread *submit_thread)
: ObAsyncCommonTask(cb_cont->mutex_, "operator cont", cb_cont, submit_thread),
seq_(0), timeout_ms_(0), operator_root_(NULL), execute_thread_(NULL), data_(NULL),
buf_(NULL), buf_reader_(NULL)
{
SET_HANDLER(&ObProxyOperatorCont::main_handler);
}
~ObProxyOperatorCont() {}
int main_handler(int event, void *data);
int init(ObProxyOperator* operator_root, uint8_t seq, const int64_t timeout_ms = 0);
virtual int init_task();
virtual int finish_task(void *data);
virtual void *get_callback_data() { return buf_reader_; }
virtual int schedule_timeout();
virtual void destroy();
private:
int build_executor_resp(event::ObMIOBuffer *write_buf, uint8_t &seq, ObProxyResultResp *result_resp);
private:
uint8_t seq_;
int64_t timeout_ms_;
ObProxyOperator* operator_root_;
event::ObEThread *execute_thread_;
void *data_;
event::ObMIOBuffer *buf_;
event::ObIOBufferReader *buf_reader_;
DISALLOW_COPY_AND_ASSIGN(ObProxyOperatorCont);
};
} // end of namespace engine
} // end of namespace obproxy
} // end of namespace oceanbase
#endif // OBPROXY_OPERATOR_CONT_H
src/obproxy/engine/ob_proxy_operator_result.h
浏览文件 @ bacd5fac
......@@ -15,6 +15,7 @@
#include "common/ob_row.h"
//#include "common/ob_field.h"
#include "common/obsm_utils.h"
#include "common/ob_obj_cast.h"
#include "rpc/obmysql/ob_mysql_field.h"
#include "proxy/mysqllib/ob_resultset_fetcher.h"
#include "lib/container/ob_se_array.h"
......@@ -123,6 +124,44 @@ int change_sql_field(const ObMysqlField *src_field, obmysql::ObMySQLField *&dst_
return ret;
}
int change_sql_value(ObObj &value, obmysql::ObMySQLField &field, ObIAllocator *allocator)
{
int ret = OB_SUCCESS;
if (value.is_varchar()) {
ObObjType ob_type;
ObCollationType cs_type = static_cast<ObCollationType>(field.charsetnr_);
// utf8_general_ci => CS_TYPE_UTF8MB4_GENERAL_CI
if (33 == field.charsetnr_) {
cs_type = CS_TYPE_UTF8MB4_GENERAL_CI;
// utf8_bin => CS_TYPE_UTF8MB4_BIN
} else if (83 == field.charsetnr_) {
cs_type = CS_TYPE_UTF8MB4_BIN;
}
value.set_collation_type(cs_type);
if (0 != value.get_string_len()) {
// Convert the column to a specific type, if it cannot be converted, keep varchar
if (OB_FAIL(ObSMUtils::get_ob_type(ob_type, field.type_))) {
COMMON_LOG(INFO, "cast ob type from mysql type failed", K(ob_type), "elem_type", field.type_, K(ret));
ret = OB_SUCCESS;
} else if (ObTimestampType == ob_type || ObTimeType == ob_type
|| ObDateType == ob_type || ObDateTimeType == ob_type) {
//do nothing
} else {
ObCastCtx cast_ctx(allocator, NULL, CM_NULL_ON_WARN, cs_type);
// use src_obj as buf_obj
if (OB_FAIL(ObObjCasterV2::to_type(ob_type, cs_type, cast_ctx, value, value))) {
COMMON_LOG(WARN, "failed to cast obj", "row", value, K(ob_type), K(cs_type), K(ret));
}
}
}
}
return ret;
}
class ObProxyResultResp : public executor::ObProxyParallelResp
{
public:
......
src/obproxy/engine/ob_proxy_operator_sort.cpp
浏览文件 @ bacd5fac
......@@ -405,7 +405,7 @@ int ObProxyMemMergeSortOp::handle_response_result(void *data, bool &is_final, Ob
} else if (OB_ISNULL(sort_unit = new (tmp_buf) ObProxyMemMergeSortUnit(allocator_))) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fail to new merge sort unit", K(ret));
} else if (OB_FAIL(sort_unit->init(row, input->get_order_exprs()))) {
} else if (OB_FAIL(sort_unit->init(row, result_fields_, input->get_order_exprs()))) {
LOG_WARN("fail to init sort unit", K(ret));
} else if (OB_FAIL(sort_units_.push_back(sort_unit))) {
LOG_WARN("fail to push back sort unit", K(ret));
......@@ -460,7 +460,7 @@ int ObProxyMemMergeSortOp::handle_response_result(void *data, bool &is_final, Ob
return ret;
}
int ObProxyMemMergeSortUnit::init(ResultRow *row, ObIArray<ObProxyOrderItem*> &order_exprs)
int ObProxyMemMergeSortUnit::init(ResultRow *row, ResultFields *result_fields, ObIArray<ObProxyOrderItem*> &order_exprs)
{
int ret = OB_SUCCESS;
......@@ -468,6 +468,7 @@ int ObProxyMemMergeSortUnit::init(ResultRow *row, ObIArray<ObProxyOrderItem*> &o
LOG_WARN("fail to assign order exprs", K(ret));
} else {
row_ = row;
result_fields_ = result_fields;
if (OB_FAIL(calc_order_values())) {
LOG_WARN("fail to get order value", K(ret));
}
......@@ -487,6 +488,15 @@ int ObProxyMemMergeSortUnit::calc_order_values()
for (int64_t i = 0; OB_SUCC(ret) && i < order_exprs_.count(); i++) {
if (OB_FAIL(order_exprs_.at(i)->calc(ctx, calc_item, order_values_))) {
LOG_WARN("fail to calc order exprs", K(ret));
} else {
int64_t index = order_exprs_.at(i)->get_expr()->get_index();
if (-1 != index) {
ObObj &value = order_values_.at(i);
if (OB_FAIL(change_sql_value(value, result_fields_->at(index), &allocator_))) {
LOG_WARN("fail to change sql value", K(value),
"filed", result_fields_->at(index), K(ret));
}
}
}
}
......@@ -566,7 +576,7 @@ int ObProxyStreamSortOp::handle_response_result(void *data, bool &is_final, ObPr
} else if (OB_ISNULL(sort_unit = new (tmp_buf) ObProxyStreamSortUnit(allocator_))) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("fail to new merge sort unit", K(ret));
} else if (OB_FAIL(sort_unit->init(opres, input->get_order_exprs()))) {
} else if (OB_FAIL(sort_unit->init(opres, result_fields_, input->get_order_exprs()))) {
LOG_WARN("fail to init sort unit", K(ret));
} else if (OB_FAIL(sort_units_.push_back(sort_unit))) {
LOG_WARN("fail to push back sort unit", K(ret));
......@@ -638,7 +648,8 @@ int ObProxyStreamSortOp::handle_response_result(void *data, bool &is_final, ObPr
return ret;
}
int ObProxyStreamSortUnit::init(ObProxyResultResp* result_set, common::ObIArray<ObProxyOrderItem*> &order_exprs)
int ObProxyStreamSortUnit::init(ObProxyResultResp* result_set, ResultFields *result_fields,
common::ObIArray<ObProxyOrderItem*> &order_exprs)
{
int ret = OB_SUCCESS;
if (OB_FAIL(order_exprs_.assign((order_exprs)))) {
......@@ -646,6 +657,9 @@ int ObProxyStreamSortUnit::init(ObProxyResultResp* result_set, common::ObIArray<
} else if (OB_ISNULL(result_set_ = result_set)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("result set should not NULL", K(ret));
} else if (OB_ISNULL(result_fields_ = result_fields)) {
ret = OB_ERR_UNEXPECTED;
LOG_WARN("result fields should not NULL", K(ret));
} else if (OB_FAIL(next())) {
// The first time, there must be a result, so it must not be OB_ITER_END
LOG_WARN("fail to exec next", K(ret));
......
src/obproxy/engine/ob_proxy_operator_sort.h
浏览文件 @ bacd5fac
......@@ -142,12 +142,13 @@ class ObProxyMemMergeSortUnit
{
public:
ObProxyMemMergeSortUnit(common::ObIAllocator &allocator)
: allocator_(allocator), row_(NULL),
: allocator_(allocator), row_(NULL), result_fields_(NULL),
order_values_(ObModIds::OB_SE_ARRAY_ENGINE, ENGINE_ARRAY_NEW_ALLOC_SIZE),
order_exprs_(ObModIds::OB_SE_ARRAY_ENGINE, ENGINE_ARRAY_NEW_ALLOC_SIZE) {}
~ObProxyMemMergeSortUnit() {}
int init(ResultRow *row, common::ObIArray<ObProxyOrderItem*> &order_exprs);
int init(ResultRow *row, ResultFields *result_fields,
common::ObIArray<ObProxyOrderItem*> &order_exprs);
virtual bool compare(const ObProxyMemMergeSortUnit* sort_unit) const;
int calc_order_values();
......@@ -159,6 +160,7 @@ public:
protected:
common::ObIAllocator &allocator_;
ResultRow *row_;
ResultFields *result_fields_;
common::ObSEArray<ObObj, 4> order_values_;
common::ObSEArray<ObProxyOrderItem*, 4> order_exprs_;
};
......@@ -188,7 +190,8 @@ public:
: ObProxyMemMergeSortUnit(allocator), result_set_(NULL) {}
~ObProxyStreamSortUnit() {}
int init(ObProxyResultResp* result_set, common::ObIArray<ObProxyOrderItem*> &order_exprs);
int init(ObProxyResultResp* result_set, ResultFields *result_fields,
common::ObIArray<ObProxyOrderItem*> &order_exprs);
virtual bool compare(const ObProxyStreamSortUnit* sort_unit) const;
int next();
......
src/obproxy/engine/ob_proxy_operator_table_scan.cpp
浏览文件 @ bacd5fac
......@@ -20,6 +20,7 @@
#include "executor/ob_proxy_parallel_cont.h"
#include "executor/ob_proxy_parallel_execute_cont.h"
#include "proxy/shard/obproxy_shard_utils.h"
#include "proxy/mysqllib/ob_proxy_mysql_request.h"
using namespace oceanbase::obproxy::executor;
using namespace oceanbase::common;
......@@ -28,6 +29,15 @@ namespace oceanbase {
namespace obproxy {
namespace engine {
ObProxyTableScanOp::~ObProxyTableScanOp()
{
for (int64_t i = 0; i < pres_array_.count(); i++) {
executor::ObProxyParallelResp *pres = pres_array_.at(i);
op_free(pres);
pres = NULL;
}
}
int ObProxyTableScanOp::open(event::ObContinuation *cont, event::ObAction *&action, const int64_t timeout_ms)
{
child_cnt_ = 1; //fake child
......@@ -77,7 +87,8 @@ int ObProxyTableScanOp::get_next_row()
"phy table count", table_name_maps.count(),
"shard prop count", shard_props.count(),
"dbkey count", db_key_names.count(), K(ret));
} else if (OB_FAIL(sql_parser.parse_sql_by_obparser(request_sql, NORMAL_PARSE_MODE, parse_result, true))) {
} else if (OB_FAIL(sql_parser.parse_sql_by_obparser(proxy::ObProxyMysqlRequest::get_parse_sql(request_sql),
NORMAL_PARSE_MODE, parse_result, true))) {
LOG_WARN("parse_sql_by_obparser failed", K(request_sql), K(ret));
}
......@@ -88,10 +99,9 @@ int ObProxyTableScanOp::get_next_row()
ObSqlString new_sql;
char *tmp_buf = NULL;
bool is_oracle_mode = db_key_name->server_type_ == common::DB_OB_ORACLE;
if (OB_FAIL(proxy::ObProxyShardUtils::do_rewrite_shard_select_request(request_sql, parse_result, is_oracle_mode,
table_name_map_warraper.get_hash_map(),
db_key_name->database_name_, false,
new_sql))) {
if (OB_FAIL(proxy::ObProxyShardUtils::rewrite_shard_dml_request(request_sql, new_sql, parse_result,
is_oracle_mode, table_name_map_warraper.get_hash_map(),
db_key_name->database_name_, false))) {
LOG_WARN("fail to rewrite shard request", K(request_sql), K(is_oracle_mode), K(ret));
} else if (OB_ISNULL(tmp_buf = (char *)allocator_.alloc(new_sql.length()))) {
ret = common::OB_ALLOCATE_MEMORY_FAILED;
......@@ -135,6 +145,8 @@ int ObProxyTableScanOp::handle_result(void *data, bool &is_final, ObProxyResultR
} else if (OB_ISNULL(pres = reinterpret_cast<executor::ObProxyParallelResp*>(data))) {
ret = OB_INVALID_ARGUMENT;
LOG_WARN("invalid input, pres type is not match", K(ret));
} else if (OB_FAIL(pres_array_.push_back(pres))) {
LOG_WARN("fail to push result resp to array", K(ret));
} else {
if (pres->is_ok_resp()) { // it is the OK packet
// For the OK package, it is only necessary to construct an OK package in the case of
......@@ -156,10 +168,6 @@ int ObProxyTableScanOp::handle_result(void *data, bool &is_final, ObProxyResultR
}
LOG_DEBUG("handle_result success", K(ret), K(pres));
}
if (OB_NOT_NULL(pres)) {
op_free(pres);
pres = NULL;
}
return ret;
}
......
src/obproxy/executor/ob_proxy_parallel_execute_cont.h
浏览文件 @ bacd5fac
......@@ -44,6 +44,7 @@ public:
ObMysqlField *get_field() const { return rs_fetcher_->get_field(); }
int64_t get_column_count() { return column_count_; }
int64_t get_cont_index() { return cont_index_; }
int64_t to_string(char *buf, int64_t buf_len) const;
private:
proxy::ObClientMysqlResp *resp_;
......
src/obproxy/iocore/eventsystem/ob_shard_scan_all_task.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/iocore/net/ob_inet.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/iocore/net/ob_inet.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/ob_proxy.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/ob_proxy_init.cpp 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/ob_proxy_init.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/ob_proxy_main.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/ob_proxy_main.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/obutils/ob_proxy_stmt.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/obutils/ob_state_info.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/omt/ob_vip_tenant_cpu.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/proxy/client/ob_driver_client.cpp 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/proxy/client/ob_driver_client.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/stat/ob_mysql_stats.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/stat/ob_mysql_stats.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/Makemodule.am
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_layout.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_layout.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_proxy_lib.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_proxy_utils.h
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_target_db_server.cpp 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/obproxy/utils/ob_target_db_server.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/Makemodule.am
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/ob_request.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/ob_mysql_field.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/ob_mysql_field.h
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/ob_mysql_global.h
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/packet/ompk_prepare_execute.cpp 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/packet/ompk_prepare_execute.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/packet/ompk_prepare_execute_req.cpp 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/rpc/obmysql/packet/ompk_prepare_execute_req.h 0 → 100644
浏览文件 @ bacd5fac
此差异已折叠。
src/share/config/ob_common_config.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/config/ob_config.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc_hash.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc_key.cpp
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc_key.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc_list.h
浏览文件 @ bacd5fac
此差异已折叠。
src/share/part/ob_part_desc_range.h
浏览文件 @ bacd5fac
此差异已折叠。
Markdown is supported
0% 或 .
You are about to add 0 people to the discussion. Proceed with caution.
先完成此消息的编辑！
想要评论请 注册 或