(cherry picked from commit f88ea9df)

(cherry picked from commit 075f28b7)

### Motivation

Fix #4732

### Modifications

Add options to rewrite the namespace delimiter, disable by default

Enable rewrite namespace delimiter can work well with superset:
<img width="1279" alt="superset" src="https://user-images.githubusercontent.com/12592133/61385412-f0f35700-a8e4-11e9-87b2-a31b62128b58.png">

### Does this pull request potentially affect one of the following parts:

*If `yes` was chosen, please highlight the changes*

  - Dependencies (does it add or upgrade a dependency): (no)
  - The public API: (no)
  - The schema: (no)
  - The default values of configurations: (no)
  - The wire protocol: (no)
  - The rest endpoints: (no)
  - The admin cli options: (no)
  - Anything that affects deployment: (no)

### Documentation

  - Does this pull request introduce a new feature? (no)

(cherry picked from commit 6ddd51ff)

Motivation
In PR #4764, we allow to create partitioned topic with 1 partition, But in Pulsar Client, user still not able to do it.
This fix try to make sure user could create consumer/producer for 1 partitioned topic .

Modifications
- old and new added unit test passed.
(cherry picked from commit 128287e3)

### Motivation
Fix concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.

### Modifications
* Adds test to expose concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.
* Fixes concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.
(cherry picked from commit 5bf319e4)

### Motivation
Incorrect value is being used in Pulsar Helm template `autorecovery-deployment.yaml`

### Modifications
Proper variable name set.

### Verifying this change
Fixed variable name is already set in `values.yaml` and `values-mini.yaml`.
This change is a trivial rework / code cleanup without any test coverage.

### Documentation
None needed.

(cherry picked from commit 2b52ccb2)

Fixes #4852

Bug:

After create an empty ledger by sub/unsub and trim ledgers, the first position in managedLedger is ahead of the last when create a consumer without reopen the managed ledger.

Reason：

Last position will be update when opening managedLedger.
Because when creating an empty Ledger without reopening managedLedger, the last position is not be updated. Since the ledger pointed of the last position has been deleted, the first position point to the new ledger.

Fix:

The first position should fall back to last position.
(cherry picked from commit 46bfb2c2)

### Motivation

jdbc sink treat all record as INSERT before #4358 , now it requires an indispensable action property which seems to be a break change, and we can deal records without any action property as INSERT.

### Modifications

treat action not set as INSERT action like before.
(cherry picked from commit d8356d8c)

(cherry picked from commit 00ce7815)

the 0 size limit can't means "infinite" size quota
(cherry picked from commit 77f63709)

(cherry picked from commit 3c9136b2)

*Motivation*

Currently fetching schema information is done synchronously.
It is called in netty callback threads and will potentially block
async operations.

*Modifications*

Make most of the operations asynchronously in SchemaInfoProvider.
(cherry picked from commit 91c4254c)

fix #4755
Retention policy is used in Ledger GC and Topic GC.
Default retention policy is not uploaded to zookeeper, but it is getted from zookeeper when it is used.
So, if zookeeper doesn't have retention policy, we should load it from default config file.
In Ledger GC configuration, it's OK.
https://github.com/apache/pulsar/blob/075f28b71c8fd9259ce8e136dbb81c0629c3f271/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L739-L742
But in Topic GC, do nothing.
(cherry picked from commit 1dcac8cc)

(cherry picked from commit abe27e63)

### Modifications

When submitting a function via REST interface, if user doesn't provide the function package or function runtime as part of function config, the function will get submitted and deployed to run but the typeClassName for the source will not be set causing a weird error in the function instance code
(cherry picked from commit e1547e12)

### Motivation

A message doesn't get automatically ACKed if there is the output topic is not set
(cherry picked from commit 19b883b7)

(cherry picked from commit 88c85447)

### Motivation
Continue the PR of #4151
(cherry picked from commit 1ab35b01)

* Convert anonymous functions to lambda

* Replacing lambda with anonymous implementation, because lambda cannot be mocked

(cherry picked from commit 84364ddc)

### Motivation

Currently our JDBC Sink not support deletion and update events.
Support for delete and update events.

### Modifications

Support for delete and update events.
Add some document for JDBC Sink. https://github.com/apache/pulsar/issues/4073

### Verifying this change

local Unit Test pass.
Integration test pass

### Motivation

`pulsar-function-go/conf` package apply `instance-conf-path` with default value `HOME_PATH+github.com/apache/pulsar/pulsar-function-go/conf/conf.yaml`, once function deployed, the running node may not have the yaml conf file exist, then go function will panic with `not found conf file` error.

This PR changed the logic of config parsing, parse `confContent` first, then parse `confFilePath` if `confContent` empty.

(cherry picked from commit 8a3b3af6)

### Motivation

After the changes in #3118, there has a been a sharp increase of memory utilization for the UnackedMessageTracker due to the time buckets being created.

This is especially true when the acktimeout is set to a larger value (eg: 1h) where 3600 time-buckets are being created. This lead to use 20MB per partition even when no message is tracked.

Allowing to configure the tick time so that application can tune it based on needs.

Additionally, fixed the logic that keeps creating hash maps and throwing them away at each tick time iteration, since that creates a lot of garbage and doesn't take care of the fact that the hash maps are expanding based on the required capacity (so next time they are already of the "right" size).

On a final note: the current default of 1sec seems very wasteful. Something like 10s should be more appropriate as default.

(cherry picked from commit f13af487)

*Motivation*

When using PulsarService or BrokerService for testing, it might require accessing
the components in PulsarService and BrokerService. This change is adding setters
and getters to access the components in PulsarService & BrokerService
(cherry picked from commit 5cff1691)

### Motivation

Currently, if the kubernetes namespace set to deploy functions in is different than the one in which brokers/workers reside, get status and stats doesn't work because the url for instances does not specify the namespace.
(cherry picked from commit 8c3445ad)

(cherry picked from commit 26fc4efe)

### Motivation

Currently golang function needs to be compiled before deploy to pulsar, so the executable permission is required when function package is downloaded to local node from bookkeeper. This PR is intent to make golang function package executable after download from bookkeeper, to make sure the function is ok to run.
(cherry picked from commit bf58c8eb)

**Motivation**
when create partitioned topic, there is a check that `numPartitions > 1`, if numPartitions==1, it will fail.
Some user may want to create partitioned topic with only 1 topic at the start time, and during using it, could update to more topics later.

**Modification**
change check of `numPartitions > 1` to `numPartitions > 0`

expect all existing ut passed.
(cherry picked from commit 5162393c)

(cherry picked from commit d8f61553)

### Motivation
to fix #4724

### Verifying this change
Add the tests for it

(cherry picked from commit 6ec2e16b)

(cherry picked from commit 9605aede)

### Motivation

If trying to build master C++ code with the `USE_LOG4CXX` flag turned ON, compilation errors occur:
```
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc: In static member function ‘static log4cxx::LevelPtr pulsar::Log4CxxLogger::getLevel(pulsar::Logger::Level)’:
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:53:18: error: ‘DEBUG’ was not declared in this scope
             case DEBUG:
                  ^
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:55:18: error: ‘INFO’ was not declared in this scope
             case INFO:
                  ^
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:57:18: error: ‘WARN’ was not declared in this scope
             case WARN:
                  ^
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:59:18: error: ‘ERROR’ was not declared in this scope
             case ERROR:
                  ^
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:52:16: error: enumeration value ‘LEVEL_DEBUG’ not handled in switch [-Werror=switch]
         switch (level) {
                ^
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:52:16: error: enumeration value ‘LEVEL_INFO’ not handled in switch [-Werror=switch]
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:52:16: error: enumeration value ‘LEVEL_WARN’ not handled in switch [-Werror=switch]
/tmp/pulsar/pulsar-client-cpp/lib/Log4cxxLogger.cc:52:16: error: enumeration value ‘LEVEL_ERROR’ not handled in switch [-Werror=switch]
cc1plus: some warnings being treated as errors
make[2]: *** [lib/CMakeFiles/pulsarStatic.dir/Log4cxxLogger.cc.o] Error 1
make[2]: *** Waiting for unfinished jobs....
```

This is because the enum values renamed in https://github.com/apache/pulsar/pull/4664 are still used in `Log4cxxLogger.cc`.

### Modifications

Fixed the enum values used in `Log4cxxLogger.cc` by adding the prefix `LEVEL_`.

(cherry picked from commit af8ea71a)

Fixes #4589

Motivation
When using Partition-topic, the logic of redeliver messages will not be triggered when the time of ackTimeout arrives.

This is because the unAckedMessageTrackerPtr_->add(msg.getMessageId()) is not call in the listener handling of partitioned topic in cpp code
(cherry picked from commit cc5f25bf)

Fixes #4734

### Motivation

"publish_time" is Pulsar SQL internal column, as Pulsar only stores timestamps, it doesn’t store the timezone information. Use timestamp as "publish_time" type is more correct way in Pulsar SQL.

### Modifications

Change type of publish_time to timestamp.

### Verifying this change

predicate of publish_time is pushdown

Use `__publish_time__` to trim messages:
```
SELECT COUNT(*)
FROM "sql-test-1"
WHERE "__publish_time__" >= TIMESTAMP '2019-07-18 17:26:50.119'
AND  "__publish_time__" < TIMESTAMP '2019-07-18 17:26:51.119';
```
![image](https://user-images.githubusercontent.com/12592133/61447301-43835080-a983-11e9-814b-bc2b378f02b9.png)

Without `__publish_time__` predicate:
```
SELECT COUNT(*)
FROM "sql-test-1";
```
![image](https://user-images.githubusercontent.com/12592133/61447427-82190b00-a983-11e9-8d3f-3bf2a4798047.png)

(cherry picked from commit 6f5416ee)

## Motivation

Currently, In Pulsar Kerberos authentication, The server type part of pulsar node principle is hard coded as "broker" and "proxy". The expected principle for pulsar nodes would be like "broker/brokera.host.name@your.com" or "proxy/proxyb.host.name@your.com".

But some times, user may want to re-use existing principle like "u-service/host.name@some.com", to test and play around, then "u-service" will not match service type of "broker" or "proxy", and the authentication will be rejected.
This change is to remove the check of "broker" or "proxy" service type check.

## Modifaction

Remove the check of "broker" or "proxy" service type check.
(cherry picked from commit f5b20cd3)

* Fix: predicate pushdown for Pulsar SQL NPE

* fix unit test

(cherry picked from commit ac10b006)

Signed-off-by: xiaolong.ran <ranxiaolong716@gmail.com>(cherry picked from commit 23dd38f3)

Upgrading to Mockito 2.28 and PowerMock 2.0. This a pre-step to be able to run CI with Java 11 / 12

(cherry picked from commit 108780c7)

Fixes https://github.com/apache/pulsar/issues/4228

Master Issue: https://github.com/apache/pulsar/issues/4228

### Motivation

Use Pulsar schema in pulsar kafka client.

### Modifications

Support schema of pulsar for pulsar kafka client

### Verifying this change

Add Unit test
(cherry picked from commit 99d3c11e)

*Motivation*

Currently producers uses `System.currentTimeMillis()` as publish timestamp by default.
However at some use cases, producers would like to a different way for generating publish timestamp.
E.g. in a database use case, a producer might be use HLC (Hybrid Logic Clock) as publish timestamp;
in integration tests, it might require the producer to use a deterministic way to generate publish timestamp.

*Changes*

This PR introduces a `clock` in building the client. This allows applications to override the system clock
with its own implementation.

*Verify the change*

Add unit test to test customized clock in both batch and non-batch cases.

(cherry picked from commit 7397b960)

* Add static import statements for Assert to simplify the test in the presto module

* Use the preferred way of the schema's creation. The predicates and functions were converted to lambda

(cherry picked from commit c421ca6e)