- 30 3月, 2019 2 次提交
-
-
由 Boyang Jerry Peng 提交于
* adding memory padding to memory limits set in kubernetes * cleaning up
-
由 Boyang Jerry Peng 提交于
* fix: add retry and wait for kubernetes creation/deletion * cleaning up * adding retry logic for cleaning up subscriptions * adjust number of retries * correcting namespace
-
- 15 2月, 2019 4 次提交
-
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
### Motivation Fixes the Apache RAT check to exclude the auto-generated files from Avro compiler and Avro JSON schema definitions.
-
由 Matteo Merli 提交于
### Motivation Building Python wheels is failing in the updated Docker image for Python 3.4, 3.5 and 3.6. Fixing it here by explicitly including the library defined variables.
-
由 Matteo Merli 提交于
-
- 14 2月, 2019 8 次提交
-
-
由 Fangbin Sun 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
### Motivation Fixed flaky test in `AuthPluginTest.testTlsDetectPulsarSslWithHostNameValidation`. The failure was due to the fact that test is creating a producer asynchronously but not waiting for the future to be completed.
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
* If no clusters are specified, the namespace should by default use current cluster * Don't apply to v1 local namespaces * It's better to use the isV2() check instead of isGlobal()
-
由 Rajan Dhabalia 提交于
[pulsar-broker] provide option to disable redelivery-tracker to reduce in memory positionImpl footprint and gc improvement (#3584) * [pulsar-broker] provide option to disable redelivery-tracker to reduce in memory positionImpl footprint and gc improvement * change config name
-
由 Fangbin Sun 提交于
-
- 13 2月, 2019 10 次提交
-
-
由 Enrico Olivelli 提交于
[Issue 3458: Tag Pulsar ledgers in order to distinguish from other leggers in the same Bookkeeper cluster (#3525) Fixes #3458 ### Motivation See #3458 ### Modifications Add a new LedgerMetadataUtils class which holds the logic for building "metadata" to be attached to ### Verifying this change This change is a trivial rework / code cleanup without any test coverage.
-
由 冉小龙 提交于
* [go client] add SubscriptionInitPos option in ConsumerOptions Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com> * add comment for SubscriptionInitPos option Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
-
由 Vincent Xie 提交于
Signed-off-by: NVinceShieh <vinceshieh@gmail.com>
-
由 冉小龙 提交于
* [go client] add validation hostName for ClientOptions Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com> * [go client] add validation hostName for ClientOptions Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com> * rename ValidationHostName to TLSValidateHostname Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
-
由 Matteo Merli 提交于
-
由 Bruno Bonnin 提交于
### Motivation Provides a builtin MongoDB Connector, in order to ease the storage of JSON formated message in MongoDB. It's a sink connector. ### Modifications Add a new sub-module in the `pulsar-io`module. ### Verifying this change This change added tests and can be verified as follows: * deploy the connector with configuration file containing the following fields: ``` configs: mongoUri: mongodb://hostname:port database: pulsar collection: messages ``` * start a mongodb instance * send messages in the topic declared when deploying the connector * check in MongoDB if the messages have been stored in the collection `messages`
-
由 Sanjeev Kulkarni 提交于
* moved all windowing related code into its own submodule. * added pom
-
由 Matteo Merli 提交于
-
由 Fangbin Sun 提交于
-
由 冉小龙 提交于
* [cpp-client] add multi topic for consumer #3494 Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com> * use initial_position instead to multi-topics Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
-
- 12 2月, 2019 4 次提交
-
-
由 Rajan Dhabalia 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
* Added docs for using schema in Python client lib * Added info on required fields
-
- 11 2月, 2019 4 次提交
-
-
由 Fangbin Sun 提交于
### Motivation Add a unit test for KafkaAbstractSink to check valid configs. ### Modifications Add KafkaAbstractSinkTest.
-
由 Sijie Guo 提交于
*Motivation* Add state support in python functions *Changes* - Bump bookkeeper version, so the table service has the changes to support python functions - Add state to python function
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
- 10 2月, 2019 4 次提交
-
-
由 Sijie Guo 提交于
*Motivation* This change was part of #2714. Moving it out as a separate change to make reviews easier. *Changes* Bump bookkeeper version to 4.9.0
-
由 Rajan Dhabalia 提交于
* [pulsar-broker][replicator] add remote cluster name in producer name * add padding between remote and local cluster-names * broker-producer can parse remote-cluster name in producer-name
-
由 Sijie Guo 提交于
*Motivation* There are a few shaded modules introduced when functions was introduced. They are not needed anymore. We should remove them. *Modifications* Remote those unused modules.
-
由 massakam 提交于
* Create SSL context in constructor of ChannelInitializer * Set TLS ports only in tests that need to enable TLS
-
- 09 2月, 2019 4 次提交
-
-
由 Matteo Merli 提交于
### Motivation Remove compile and link dependency on JsonCPP. Instead use boost::property_tree with JSON parser. This will not require a link time dependency.
-
由 Sijie Guo 提交于
*Motivation* Sometimes users would like to only execute a set of test suite in command line. *Modifications* Make the suite file name configurable in the pom file. So people can run a specific test suite using ``` mvn -pl tests/integration -DredirectTestOutputToFile=false -DintegrationTestSuiteFile=pulsar-thread-suite.xml clean test -PintegrationTests ``` Also categorize the tests info a few test suites. This would allow us break down the integration test job into multiple smaller test jobs.
-
由 Boyang Jerry Peng 提交于
### Motivation Some runtime might require a minimum about of resources to run a function instance. Currently functions don't support the check to determine if the resources requested is greater the the minimum required. ### Modification Add a worker config to set the minimum amount of resource a function must request
-
由 Rajan Dhabalia 提交于
### Motivation Sometimes while shutting down the broker, unknown uncompleted callback tries to update `managed-cursor` even if managed-cursor is closed and because of that topic which is already loaded by new broker sees below exception while updating managed-cursor-metadata and cursor is not able to update mark-delete position until it unloaded again. ``` 09:22:02.641 [BookKeeperClientWorker-OrderedExecutor-8-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper - [sample/global/ns1/persistent/TopicBadVersion] [pulsar.repl.us-east] Updating cursor info ledgerId=234567890 mark-delete=12345678900:5306 00:00:05.642 [bookkeeper-ml-workers-OrderedExecutor-14-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion] Failed to update consumer pulsar.repl.us-east org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion Caused by: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[pulsar-broker-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper.lambda$null$103(MetaStoreImplZookeeper.java:287) ~[managed-ledger-original-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [managed-ledger-original-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.7.2.jar:4.7.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.22.Final.jar:4.1.22.Final] ``` It seems it always happens for replicator-cursor and not sure exact step to reproduce but it might happen when bookies might be in unstable state (not sure about the root cause). one of the example: Behavior: broker-1 unloads the topic by `09:21:59.451` and broker-2 loads the same topic at `09:22:00.135`. But broker-1 has log of updating metadata-znode at `09:22:05.207`. So, broker-2 fails to update cursor-metadata with exception `BadVersionException` ``` Previous-broker: broker-1 09:21:58.855 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Disabling ownership: sample/global/ns1/0x2aaaaaa8_0x35555552 : : 09:21:59.668 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading sample/global/ns1/0x2aaaaaa8_0x35555552 namespace-bundle with 345 topics completed in 812.0 ms : 09:21:58.930 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.service.BrokerService - [persistent://sample/global/ns1/TopicBadVersion] Unloading topic : 09:21:59.451 [bookkeeper-ml-workers-OrderedExecutor-14-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion][pulsar.repl.us-east] Closed cursor at md-position=12345678900:5306 09:21:59.451 [bookkeeper-ml-workers-OrderedExecutor-14-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://sample/global/ns1/TopicBadVersion] Topic closed : 09:22:05.207 [BookKeeperClientWorker-OrderedExecutor-2-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion] Error updating cursor pulsar.repl.us-east position 12345678900:5306 in meta-ledger 14451380450: BookKeeper client is closed 09:22:05.207 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper - [sample/global/ns1/persistent/TopicBadVersion] [pulsar.repl.us-east] Updating cursor info ledgerId=-1 mark-delete=12345678900:5306 ``` ``` Current-broker: broker-2 09:22:00.135 [pulsar-ordered-OrderedExecutor-7-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Opening managed ledger sample/global/ns1/persistent/TopicBadVersion ``` ### Modification It happens when someone tries to update cursor metadata by calling `persistPositionMetaStore(..)` so, it should have safe guard and should not update metadata if cursor is already closed. ### Result It avoids any unexpected behavior while updating cursor-metadata in closed cursor state.
-