This commit adds a new option optionally disable authentication for the
`/metrics` endpoint in the pulsar-proxy.

Currently, authentication is required for the metrics endpoint when
authentication is enabled, which makes monitoring more difficult.
However, rather than just disable it completely and allow for metrics to
be exposed to any unknown user, this makes it opt in.

It could be argued that it should default to false, but as it is likely
that the proxy is the only component potentially exposed to the public internet, we
default to not exposing data.

Fixes #4920

Before this patch, all keys are read as RSA, which meant that only RSA
compatible JWT signing algorithms could be used, specifically, this
limited the use of ECDSA family of JWT keys.

This changes this by changing the signature we use to parse keys to also
take a SignatureAlgorithm and also adds a new config option
`tokenPublicAlg` which can be used to signify what algorithm the
broker/proxy should use when reading public keys. However, these all
default to RS256, which, should indicate to decode as RSA (even if
another RS/PS algoritm is used).

This also adds some new options to the Token CLI tool for those commands
that weren't respecting the algorithm, but these are defaulted to RS256
as well.

* Add support for AWS instance and role creds

This commit makes changes to the tiered storage support for S3
to allow for support of ec2 metadata instance credentials as well as
additional config options for assuming a role to get credentials.

This works by changing the way we provide credentials to use the
funtional `Supplier` interface and for using the AWS specific
`SessionCredentials` object for when we detect that the
`CredentialProvider` is providing credentials that have a session token.

* [tiered_storage] Tweak s3 credential handling to check on boot

This changes the s3 handling slightly, instead of falling back to static
credentials, we instead now fail if no s3 credentials can be found and
change the unit tests to start a broker with s3 credentials.

With the new Supplier API, we now fetch credentials on every request.
Because of this, the failure and subsequent try/catch is costly and the
integration tests were using this, which caused them to be significantly
slower.

Instead, we just check to see if we can fetch creds, and if we can't
consider it an error condition to exit the app as it is unlikely in a
production scenario to not have some credentials.

* fix s3 test for missing creds

* [pulsar-broker] add support of secondary bookie-isolation-group

* fix docs

* add negative test

* Fix tests

* By default, auto configure the size of Bookie read/write cache

* Updated ref docs

* Allow to configure the managed ledger cache eviction frequency

* Fixed test

* Simplified the cache eviction to make it predictable at the configured frequency

* Address comments

* Apply eviction on slowest active reader by preference

* Re-introduced backlogged subscriptions test

* Addressed comments

* Use config option

* Fixed active/inactive logic and read position

* Use dedicated thread for cache evictions

* Added config options in docs

* Fixed tests

* Added time triggered eviction test

* Fixed flaky test

* Fixed tests

This reverts commit 77d4c28d.

* Remove broker mode to handle persistent/non-persistent topics separately

* fix unused imports

[pulsar-broker] provide option to disable redelivery-tracker to reduce in memory positionImpl footprint and gc improvement (#3584)

* [pulsar-broker] provide option to disable redelivery-tracker to reduce in memory positionImpl footprint and gc improvement

* change config name

### Motivation

Replace all ```global zookeeper``` with  ```configuration store``` in all documents.

Relates to #3338
Closes #3305 

## Motivation
Fix #3119. This allows to configure region-aware policy and read-reordering so that brokers first read on bookies of their own region.

## Modifications
1. Added parameters:
```
// Enable region-aware bookie selection policy. BK will chose bookies from
// different regions and racks when forming a new bookie ensemble
// If enabled, the value of bookkeeperClientRackawarePolicyEnabled is ignored
bookkeeperClientRegionawarePolicyEnabled=false

// Enable/disable reordering read sequence on reading entries.
bookkeeperClientReorderReadSequenceEnabled=false
```

2. Fixed bug in ZkBookieRackAffinityMapping: the value set to racksWithHost by deserialize() was overriden by the affectation in setConf(). The fix just moves the hostname workaround in setConf().

## Result
Users can enable bookkeeperClientRegionawarePolicyEnabled and bookkeeperClientReorderReadSequenceEnabled to make brokers read on bookies of their own region

* PIP-25: Token based authentication

* Addressed comments

* Use Authorization header

* Update to support env: data: and file: as sources for keys and tokens

* Fixed cli description

* Updated broker.conf

* Improved consistency in reading keys and CLI tools

* Fixed check for http headers

* Accept rel time with no specified unit

* Fixed reading data: URL

* Addressed comments

* Added integration tests

* Addressed comments

* Added CLI command to validate token against key

* Fixed integration tests

* Removed env:

* Fixed rel time parsing

PR #1225 added the ability to configure the TLS protocol version and
cipher on the server side, but this was never added to the
documentation. This patch fixes that omission.

Issue: #2402

* Removed mentions to incubator-pulsar

* Removed DISCLAIMER link from offloaders package

* Trigger build

### Motivation

Increasing the default value of `maxConcurrentLookupRequests` from 10K to 50K to increase limit of sustainable lookups before rejecting requests.

### Motivation

The default timeout for broker graceful shutdown is set to 3 seconds. This can give little room to do graceful shutdown when the broker is serving a lot of topics.

There is no big downside in increasing the timeout to a much bigger value.

### Changes

Break down the `Concepts and Architecture` page into multiple pages and move it as a separate top-level category.

### Motivation

Improve the documentation and usability of the pulsar website. This moves the website and documentation to a new framework (https://docusaurus.io/)  which will make it easier to maintain going forward.

### Modifications

A new version of the website in site2 directory. Also updates the pulsar build docker to add the new website build dependencies.

### Result

A more usable website and documentation.

A preview of the site can be seen here: https://cckellogg.github.io/incubator-pulsar
*All the links and images might not work on this site since it's a test only site*