*Motivation*

Currently KeyValue schema doesn't support using AUTO_CONSUME.

This PR is to add this support.

This PR is based on #4836

*Changes*

- refactor a bit on Schema interface to support fetching schema info for both AutoConsumeSchema and KeyValueSchema before subscribing
- add AUTO_CONSUME support to KeyValueSchema
- add tests

### Motivation

Fix #3742

In order to decode the message correctly by AVRO schema, we need to know the schema what the message is.

### Modification

- Introduced Schema Reader and Schema Writer for StructSchema.
   - Reader is used to decode message
   - Writer is used to encode message
- The implementations of StructSchema, provides their schema reader and writer implementations. 
- Introduced a schema reader cache for caching the readers for different schema versions. 

*Motivation*

Fixes #3925

We have 3 places of defining schema type enums. We kept adding
new schema type in pulsar-common. However we don't update the schema type
in wire protocol and schema storage.

This causes `SchemaType.NONE` is stored in SchemaRegistry.
It fails debeizum connector on restarting.

*Modifications*

Make sure all 3 places have consistent schema type definitions.
Record the correct schema type.

*Motivation*

In order to introduce `GenericRecordBuilder`, we need to know the fields in a `GenericSchema`.
Otherwise, there is no way for us to build a GenericRecord.

*Modifications*

This proposal refactors current generic schema by introducing a `GenericSchema`. This generic schema
provides interfaces to retrieve the fields of a `GenericRecordSchema`.

*Additionally*

This proposal adding the primitive schemas into `Schema` class. So people can program primitive schemas
using Schema interface rather than specific implementations.

*Motivation*

Currently AUTO_CONSUME only supports decoding records from latest schema.
All the schema versions are lost. It makes AUTO_CONSUME less useful in some use cases,
such as CDC. Because there is no way for the applications to know which version of schema
that a message is using.

In order to support multi-version schema, we need to propagate schema version from
message header through schema#decode method to the decoded record.

*Modifications*

- Introduce a new decode method `decode(byte[] data, byte[] schemaVersion)`. This allows the implementation
  to leverage the schema version.
- Introduce a method `supportSchemaVersioning` to tell which decode methods to use. Because most of the schema
  implementations such as primitive schemas and POJO based schema doesn't make any sense to use schema version.
- Introduce a SchemaProvider which returns a specific schema instance for a given schema version
- Implement a MultiVersionGenericRecordSchema which decode the messages based on schema version. All the records
  decoded by this schema will have schema version and its corresponding schema definitions.

*NOTES

This implementation only introduce the mechanism. But it doesn't wire the multi-version schema
with auto_consume schema. There will be a subsequent pull request on implementing a schema provider
that fetches and caches schemas from brokers.

* Separate pulsar client API interfaces in different module

* Added provided protobuf dep

* Fixed build and tests

* Fixed import name

* Fixed AuthenticationTls class name

* Fixed AutoProduceSchema name

* remove pulsar-client-schema

* shade avro dependencies in pulsar-client-kafka-shaded

* refactor api to separate module

* adding schema back

* cleaning up

* fix license issue

* fixing licenses

* cleaning up

* adding back lombok

* cleaning up

*Motivation*

We need an upgrade/backward compatibility story for schema enforcement. Especially around:

- `Producers cannot connect without a schema to topics with a schema`

*Changes*

- provide a flag on brokers to enable schema validation (and disabled it by default). this allows a smooth upgrade on brokers,
  otherwise, it will break all non-java producers on topics with schema immediately when upgrade to the new version

*Motivation*

Currently trigger function is broken due to the schema enforcement we added recently:
A producer without schema can't produce messages into a topic with schema.

*Changes*

- Rename `AUTO` to `AUTO_CONSUME`
- Introduce `AUTO_PRODUCE` schema. The schema produces `byte[]`, but it will validate
  the bytes are compatible with the schema associated with the topic before producing.
- Change trigger function to use `AUTO_PRODUCE` schema.

### Motivation

Sometimes consumers and readers don't have the pojo for the message schemas. so we can use `Schema.Auto` to detect
the schema and reflect the messages into generic records.

 ### Changes

- Introduce AutoSchema and `SchemaType.AUTO`
- fetch schema information first when `AutoSchema` is used

* fixing protobuf schema shading issue

* add note on multiple input topics

* fix CSS for highlight blocks

* more java API docs

* fix scrolling issue

* add CLI documentation for pulsar-admin functions

* fix YAML issues

* add PF to features list on front page

* add note about SPEs to feature description

* remove unnecessary include in local config

* add new badges to templates

* add section on core programming modeL

* add section on the SDK

* begin adding section on java sdk

* comparison table for native vs SDK

* add decrementCounter method

* add link

* add counters section

* update example functions

* revert Context object to master

* add user config section for java

* fix misspelling in error output

* add new PF config to YAML descriptor

* add processing guarantees doc to the sidebar

* update download URL and add trigger command to CLI docs

* add logTopic flag to CLI docs

* use native python function in quickstart

* add intro to java section

* update example functions and finish draft of API doc

* fix URL in sidebar config

* finish user config section in API doc

* add example ContextFunction

* finish draft of processing guarantees doc

* add section on triggering

* comment out subscription types section for now

* remove unnecessary console.log statement

* remove <hr> in docs template

* add missing DefaultSerDe class import

* fix error output to match test expectation

* Add missing license header to new example Pulsar Function

* Renamed RequestHandler to PulsarFunction

* Fixed header

* Create pulsar-functions module (#1)

* Create pulsar-functions module

* rename `sdk` package to `api`

* Added the first cut of the Java interface for Pulsar functions (#2)

* Cleaned up the logic of supported types

* Create pulsar-functions module (#1)

* Create pulsar-functions module

* rename `sdk` package to `api`

* Added the first cut of the Java interface for Pulsar functions (#2)

* Don't wait for the result. Instead chain it via CompletedFuture

* Integrate spawner in cli

* add scripts to run pulsar functions

* add example

* Make pulsar-functions run

* Create pulsar-functions module (#1)

* Create pulsar-functions module

* rename `sdk` package to `api`

* Added the first cut of the Java interface for Pulsar functions (#2)

* Added a simple example

* Utilities for arquillian integration tests

A set of utilities for working with clusters set up in arquillian.

- Utilities to work with docker
- Utilities to work with arquillian pulsar clusters
- Await strategies that arquillian uses to know if a node is up
- Stop actions to copy logs at the end of a test suite

* Move to log4j2

* Pull in extra dependencies for log4j yml configuration

* Updates headers with ASF source files header

* change license to additional files