- 13 2月, 2019 1 次提交
-
-
由 冉小龙 提交于
* [cpp-client] add multi topic for consumer #3494 Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com> * use initial_position instead to multi-topics Signed-off-by: Nxiaolong.ran <ranxiaolong716@gmail.com>
-
- 12 2月, 2019 4 次提交
-
-
由 Rajan Dhabalia 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
* Added docs for using schema in Python client lib * Added info on required fields
-
- 11 2月, 2019 4 次提交
-
-
由 Fangbin Sun 提交于
### Motivation Add a unit test for KafkaAbstractSink to check valid configs. ### Modifications Add KafkaAbstractSinkTest.
-
由 Sijie Guo 提交于
*Motivation* Add state support in python functions *Changes* - Bump bookkeeper version, so the table service has the changes to support python functions - Add state to python function
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
- 10 2月, 2019 4 次提交
-
-
由 Sijie Guo 提交于
*Motivation* This change was part of #2714. Moving it out as a separate change to make reviews easier. *Changes* Bump bookkeeper version to 4.9.0
-
由 Rajan Dhabalia 提交于
* [pulsar-broker][replicator] add remote cluster name in producer name * add padding between remote and local cluster-names * broker-producer can parse remote-cluster name in producer-name
-
由 Sijie Guo 提交于
*Motivation* There are a few shaded modules introduced when functions was introduced. They are not needed anymore. We should remove them. *Modifications* Remote those unused modules.
-
由 massakam 提交于
* Create SSL context in constructor of ChannelInitializer * Set TLS ports only in tests that need to enable TLS
-
- 09 2月, 2019 10 次提交
-
-
由 Matteo Merli 提交于
### Motivation Remove compile and link dependency on JsonCPP. Instead use boost::property_tree with JSON parser. This will not require a link time dependency.
-
由 Sijie Guo 提交于
*Motivation* Sometimes users would like to only execute a set of test suite in command line. *Modifications* Make the suite file name configurable in the pom file. So people can run a specific test suite using ``` mvn -pl tests/integration -DredirectTestOutputToFile=false -DintegrationTestSuiteFile=pulsar-thread-suite.xml clean test -PintegrationTests ``` Also categorize the tests info a few test suites. This would allow us break down the integration test job into multiple smaller test jobs.
-
由 Boyang Jerry Peng 提交于
### Motivation Some runtime might require a minimum about of resources to run a function instance. Currently functions don't support the check to determine if the resources requested is greater the the minimum required. ### Modification Add a worker config to set the minimum amount of resource a function must request
-
由 Rajan Dhabalia 提交于
### Motivation Sometimes while shutting down the broker, unknown uncompleted callback tries to update `managed-cursor` even if managed-cursor is closed and because of that topic which is already loaded by new broker sees below exception while updating managed-cursor-metadata and cursor is not able to update mark-delete position until it unloaded again. ``` 09:22:02.641 [BookKeeperClientWorker-OrderedExecutor-8-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper - [sample/global/ns1/persistent/TopicBadVersion] [pulsar.repl.us-east] Updating cursor info ledgerId=234567890 mark-delete=12345678900:5306 00:00:05.642 [bookkeeper-ml-workers-OrderedExecutor-14-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion] Failed to update consumer pulsar.repl.us-east org.apache.bookkeeper.mledger.ManagedLedgerException$BadVersionException: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion Caused by: org.apache.zookeeper.KeeperException$BadVersionException: KeeperErrorCode = BadVersion at org.apache.zookeeper.KeeperException.create(KeeperException.java:118) ~[pulsar-broker-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper.lambda$null$103(MetaStoreImplZookeeper.java:287) ~[managed-ledger-original-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.mledger.util.SafeRun$1.safeRun(SafeRun.java:32) [managed-ledger-original-2.2.5-.jar:2.2.5-] at org.apache.bookkeeper.common.util.SafeRunnable.run(SafeRunnable.java:36) [bookkeeper-common-4.7.2.jar:4.7.2] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [?:1.8.0_131] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [?:1.8.0_131] at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [netty-all-4.1.22.Final.jar:4.1.22.Final] ``` It seems it always happens for replicator-cursor and not sure exact step to reproduce but it might happen when bookies might be in unstable state (not sure about the root cause). one of the example: Behavior: broker-1 unloads the topic by `09:21:59.451` and broker-2 loads the same topic at `09:22:00.135`. But broker-1 has log of updating metadata-znode at `09:22:05.207`. So, broker-2 fails to update cursor-metadata with exception `BadVersionException` ``` Previous-broker: broker-1 09:21:58.855 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Disabling ownership: sample/global/ns1/0x2aaaaaa8_0x35555552 : : 09:21:59.668 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.namespace.OwnedBundle - Unloading sample/global/ns1/0x2aaaaaa8_0x35555552 namespace-bundle with 345 topics completed in 812.0 ms : 09:21:58.930 [shutdown-thread-49-1] INFO org.apache.pulsar.broker.service.BrokerService - [persistent://sample/global/ns1/TopicBadVersion] Unloading topic : 09:21:59.451 [bookkeeper-ml-workers-OrderedExecutor-14-0] INFO org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion][pulsar.repl.us-east] Closed cursor at md-position=12345678900:5306 09:21:59.451 [bookkeeper-ml-workers-OrderedExecutor-14-0] INFO org.apache.pulsar.broker.service.persistent.PersistentTopic - [persistent://sample/global/ns1/TopicBadVersion] Topic closed : 09:22:05.207 [BookKeeperClientWorker-OrderedExecutor-2-0] WARN org.apache.bookkeeper.mledger.impl.ManagedCursorImpl - [sample/global/ns1/persistent/TopicBadVersion] Error updating cursor pulsar.repl.us-east position 12345678900:5306 in meta-ledger 14451380450: BookKeeper client is closed 09:22:05.207 [BookKeeperClientWorker-OrderedExecutor-2-0] INFO org.apache.bookkeeper.mledger.impl.MetaStoreImplZookeeper - [sample/global/ns1/persistent/TopicBadVersion] [pulsar.repl.us-east] Updating cursor info ledgerId=-1 mark-delete=12345678900:5306 ``` ``` Current-broker: broker-2 09:22:00.135 [pulsar-ordered-OrderedExecutor-7-0] INFO org.apache.bookkeeper.mledger.impl.ManagedLedgerImpl - Opening managed ledger sample/global/ns1/persistent/TopicBadVersion ``` ### Modification It happens when someone tries to update cursor metadata by calling `persistPositionMetaStore(..)` so, it should have safe guard and should not update metadata if cursor is already closed. ### Result It avoids any unexpected behavior while updating cursor-metadata in closed cursor state.
-
由 Matteo Merli 提交于
### Motivation Added integration test to validate that a Java client with schema can interoperate with a Python client with equivalent schema declaration.
-
由 Matteo Merli 提交于
### Motivation Fixes #3538. The second consumer was being created with `Exclusive` type and was actually failing to connect.
-
由 Matteo Merli 提交于
* Improved SimpleProducerConsumerTest to avoid receive timeout * Use testDupConsumersOnSharedModeNotThrowsExcOnUnsubscribe to avoid interference with other tests running in parallel
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
由 Matteo Merli 提交于
-
- 08 2月, 2019 14 次提交
-
-
由 Sijie Guo 提交于
*Motivation* Currently bookie http server is disabled by default. Setting the http server port to 8000 and include a vertx http server implementation to prepare enabling http server on bookie. *Modifications* Currently bookkeeper only provides two http server bindings, one is vertx, the other one is finagle server binding. both are netty based http server. so technically it doesn't introduce additional dependencies. However if people prefers a jetty implementation, I can add a jetty binding in bookkeeper.
-
由 Boyang Jerry Peng 提交于
-
由 Fangbin Sun 提交于
-
由 Matteo Merli 提交于
* Added suppport for Kafka partitioner and explicit setting of partition on record * Addressed comments * Use StandardCharsets and fixed alignment * Fixed initialization of custom partitioner * Feed realistic cluster metadata for Kafka partitioner implementation
-
由 Boyang Jerry Peng 提交于
-
由 Boyang Jerry Peng 提交于
For integration tests, copy function logs out of container on shutdown for debugging purposes (#3529)
-
由 Matteo Merli 提交于
* Build static lib with bundled dependencies * Added gitignore entry * Fixed target_link_libraries when not linking statically * Fixed linking on python wrapper
-
由 Matteo Merli 提交于
* Upgrade AsyncHTTP client to 2.7.0 * Fixed license files
-
由 Matteo Merli 提交于
-
由 Jia Zhai 提交于
* consumer auto subscribe * producer update for partiton extends * change following Matteo's comments * enable by default
-
由 Sanjeev Kulkarni 提交于
-
由 Matteo Merli 提交于
-
由 Boyang Jerry Peng 提交于
fix: RuntimeInfo doesn't get set when transferring ownership of function from another worker to this worker (#3523) * fix: RuntimeInfo doesn't get set when transferring ownership of function from another worker to this worker * fix and add test * fix bug and adding tests * fixing kubernetes runtime * add debugging message
-
由 Matteo Merli 提交于
Use testDupConsumersOnSharedModeNotThrowsExcOnUnsubscribe to avoid interference with other tests running in parallel (#3521) * Use testDupConsumersOnSharedModeNotThrowsExcOnUnsubscribe to avoid interference with other tests running in parallel * Fixed formatting
-
- 07 2月, 2019 2 次提交
-
-
由 k2la 提交于
-
由 Matteo Merli 提交于
This reverts commit 8fcc7ca5.
-
- 05 2月, 2019 1 次提交
-
-
由 Fangbin Sun 提交于
* Support Kafka-IO to pass extra configs. * Add a setting in the connector config file and a unit test for this part of changes * Remove unused import * A minor change and a unit test for this part of changes * add missing io-kafka.md
-