---

*Motivation*

Add default handler to handle transaction related commands.

Fixes #4791

**Motivation**

Currently the Pulsar Kafka wrapper is using Kafka 0.10.x version. However, there are users who use legacy-kafka version in their system and willing to move to pulsar. This PR provides pulsar-kafka adapter for kafka-api-version 0.9.X. So, this adapter can help users in their migration process from kafka-0.9 to pulsar.

### Motivation

Currently, when running Pulsar Functions, Sources, and Sinks in Kubernetes. The resources requests and resource limits are set to the same values.  While this is ok and everything will run as it should, actual resource utilization in the cluster might be low.   To increase actual resource utilization, we need to be able to overcommit a certain amount in our clusters

### Modifications

The checksyle plugin was added to the pulsar sql module to enforce the defined style. All violations were fixed:

- Ordering of imports.
- Formatting of the code.
- Absent Javadoc comments.
- Other small issues.

 MacOS clang/llvm compiler imcompatible code in Unit test case BasicEndToEndTest.cc #4871

### Motivation
Fix concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.

### Modifications
* Adds test to expose concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.
* Fixes concurrent access of `uninitializedCursors` in `ManagedLedgerImpl.asyncOpenCursor`.

**Motivation**

Fixes #4698 and #4699 

**List of changes**

1. Two new folders under pulsar/deployment/kubernetes/generic : 
- original : contains the original scripts (pre Kubernetes 1.9)
- k8s-1-9-and-above : contains the new scripts with APIs ipdated (Kubernetes 1.9 and above)

2. bookie.yaml :
- Migrate DaemonSet api version from extensions/v1beta1 to apps/v1
- Declare the bookie service before the DaemonSet object for bookies

3. broker.yaml
- Migrate Deployment api version from apps/v1beta1 to apps/v1
- Declare the bookie service before the Deployment object for brokers

4. monitoring.yaml :
- Migrate all Deployment api version from apps/v1beta1 to apps/v1
- Declare each service before the Deployment object the service is bound to

5. zookeeper.yaml :
- Migrate StatefulSet api version from apps/v1beta1 to apps/v1
- Declare the service before the StatefulSet object for zookeeper

6. proxy.yaml
- Migrate Deployment api version from apps/v1beta1 to apps/v1
- Declare the bookie service before the Deployment object for proxy.

* Add *Understand Schema* Section

* Update

* Update

* Update

* update

* Update

* Update

* Update

* Update

* Update

* update

Merge Request for #4808: TYPO in C++ client producer method for processing failure case, and add corresponding unit test case. (#4873)

Definitely, this is a typo. This method is dealing with the Failed Message with the GIVEN result, but not a CERTAIN result.

Contribution Checklist
#4808 : TYPO in C++ client producer method for processing failure case
Add c++ client producer failure message unit test case.

UT passed:

BatchMessageTest

*Motivation*

Add new commands for the transaction.

*Modifications*

- Add new property for `CommandSend`
- Add new command `CommandEndTxnOnPartition`

### Motivation
Incorrect value is being used in Pulsar Helm template `autorecovery-deployment.yaml`

### Modifications
Proper variable name set.

### Verifying this change
Fixed variable name is already set in `values.yaml` and `values-mini.yaml`.
This change is a trivial rework / code cleanup without any test coverage.

### Documentation
None needed.

* improve doc for kerberos

* add more info

* also update 2.4.0 doc

* updated following comments

* change following comments

Fixes #4852

Bug:

After create an empty ledger by sub/unsub and trim ledgers, the first position in managedLedger is ahead of the last when create a consumer without reopen the managed ledger.

Reason：

Last position will be update when opening managedLedger.
Because when creating an empty Ledger without reopening managedLedger, the last position is not be updated. Since the ledger pointed of the last position has been deleted, the first position point to the new ledger.

Fix:

The first position should fall back to last position.

*Modifications*

Add primary operation of transaction. Keep all actions persistently.

Describe the modifications you've done.

- add commit operation
- add abort operation
- add openreader operation

### Motivation

jdbc sink treat all record as INSERT before #4358 , now it requires an indispensable action property which seems to be a break change, and we can deal records without any action property as INSERT.

### Modifications

treat action not set as INSERT action like before.

Fixes #4554 

### Motivation
Add a new deploy tutorial for new Pulsar Functions structure.

### Modifications
Based on the original functions-deploying file, re-adjust the structure and language of functions-deploy.

Structure of Schema Chapter: https://github.com/apache/pulsar/issues/4789

* add thread/process modes

* update the differences between thread and process modes

* update

* update

* Update examples, thread/process modes

* remove process and thread modes

* update

* Refine functions overview and structure

* keep old files and structure, add new ones

* Update

* add a note at the beginning

* create new files for the new structure

* remove two examples

* will create new PR for each of those files

* Update overview

* remove thread/process modes differences

the 0 size limit can't means "infinite" size quota

### Motivation
Continue the issues of #4782

### Verifying this change
Add the tests for it

* [Transaction][buffer] Add data ledger position in txn commit marker
---

*Motivation*

Add the data ledger position into the commit marker of the transaction log.

*Modifications*

Create a new TxnCommitMarker in the proto.

*Motivation*

Currently KeyValue schema doesn't support using AUTO_CONSUME.

This PR is to add this support.

This PR is based on #4836

*Changes*

- refactor a bit on Schema interface to support fetching schema info for both AutoConsumeSchema and KeyValueSchema before subscribing
- add AUTO_CONSUME support to KeyValueSchema
- add tests

* Add maven plugin for style checking. Fix some style violations.

* Fix issues shown by the style checker in the pulsar-flink module

*Motivation*

Currently fetching schema information is done synchronously.
It is called in netty callback threads and will potentially block
async operations.

*Modifications*

Make most of the operations asynchronously in SchemaInfoProvider.

### Motivation
The original PR https://github.com/apache/pulsar/pull/4770 is merged before I could give some feedback, so I just create a new pull request.

### Modifications
1. Refine the structure, remove some unnecessary info currently. Thus it is displayed to users as a complete guide. 
2. Refine the language part, make it clear and concise.
3. Though "upgrade ZooKeeper" is optional, could we add information for it? So users can decide whether they want to upgrade ZooKeeper. When they upgrade ZooKeeper, they know how to upgrade.

fix #4755 
Retention policy is used in Ledger GC and Topic GC.
Default retention policy is not uploaded to zookeeper, but it is getted from zookeeper when it is used.
So, if zookeeper doesn't have retention policy, we should load it from default config file.
In Ledger GC configuration, it's OK.
https://github.com/apache/pulsar/blob/075f28b71c8fd9259ce8e136dbb81c0629c3f271/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java#L739-L742
But in Topic GC, do nothing.

s/is/it/

* Add develop part

* Add new site2/docs/functions-develop.md for new structure

* Adopt advice from Jerry

* Update Python and Go examples

* remove some tabs for Go

* add Go example in logger section

* remove Encryptiion at the end

* restructure Example in SerDe section

### Modifications

When submitting a function via REST interface, if user doesn't provide the function package or function runtime as part of function config, the function will get submitted and deployed to run but the typeClassName for the source will not be set causing a weird error in the function instance code

### Motivation

A message doesn't get automatically ACKed if there is the output topic is not set