- 28 8月, 2019 1 次提交
- 23 5月, 2019 1 次提交
-
-
由 Sijie Guo 提交于
[schema] AutoConsume should use the schema associated with messages as both writer and reader schema (#4325) * [schema] AutoConsume should use the schema associated with messages for both writer and reader schema *Motivation* AutoConsume should use the schema associated with the messages for decoding the schemas. *Modifications* - provide a flag enable or disable using the provided schema as the reader schema - for AUTO_CONSUME schema, disable usnig the provided schema as the reader schema. so it can use the right schema version for decoding messages into right generic records - provide a few util methods for displaying schema data * Handle 64 bytes schema version * Addressed review comments
-
- 28 4月, 2019 1 次提交
-
-
由 congbo 提交于
### Motivation Fix #3742 In order to decode the message correctly by AVRO schema, we need to know the schema what the message is. ### Modification - Introduced Schema Reader and Schema Writer for StructSchema. - Reader is used to decode message - Writer is used to encode message - The implementations of StructSchema, provides their schema reader and writer implementations. - Introduced a schema reader cache for caching the readers for different schema versions.
-
- 08 3月, 2019 1 次提交
-
-
由 Sijie Guo 提交于
*Motivation* In order to introduce `GenericRecordBuilder`, we need to know the fields in a `GenericSchema`. Otherwise, there is no way for us to build a GenericRecord. *Modifications* This proposal refactors current generic schema by introducing a `GenericSchema`. This generic schema provides interfaces to retrieve the fields of a `GenericRecordSchema`. *Additionally* This proposal adding the primitive schemas into `Schema` class. So people can program primitive schemas using Schema interface rather than specific implementations.
-
- 28 2月, 2019 1 次提交
-
-
由 Sijie Guo 提交于
*Motivation* In order to introduce `GenericRecordBuilder`, we need to know the fields in a `GenericSchema`. Otherwise, there is no way for us to build a GenericRecord. *Modifications* This proposal refactors current generic schema by introducing a `GenericSchema`. This generic schema provides interfaces to retrieve the fields of a `GenericRecordSchema`. *Additionally* This proposal adding the primitive schemas into `Schema` class. So people can program primitive schemas using Schema interface rather than specific implementations.
-
- 26 2月, 2019 1 次提交
-
-
由 Sijie Guo 提交于
*Motivation* Currently AUTO_CONSUME only supports decoding records from latest schema. All the schema versions are lost. It makes AUTO_CONSUME less useful in some use cases, such as CDC. Because there is no way for the applications to know which version of schema that a message is using. In order to support multi-version schema, we need to propagate schema version from message header through schema#decode method to the decoded record. *Modifications* - Introduce a new decode method `decode(byte[] data, byte[] schemaVersion)`. This allows the implementation to leverage the schema version. - Introduce a method `supportSchemaVersioning` to tell which decode methods to use. Because most of the schema implementations such as primitive schemas and POJO based schema doesn't make any sense to use schema version. - Introduce a SchemaProvider which returns a specific schema instance for a given schema version - Implement a MultiVersionGenericRecordSchema which decode the messages based on schema version. All the records decoded by this schema will have schema version and its corresponding schema definitions. *NOTES This implementation only introduce the mechanism. But it doesn't wire the multi-version schema with auto_consume schema. There will be a subsequent pull request on implementing a schema provider that fetches and caches schemas from brokers.
-
- 07 1月, 2019 1 次提交
-
-
由 Eren Avsarogullari 提交于
### Motivation This PR aims to cover some of minor `Schema` fixes. ### Modifications 1- `SchemaSerializationException` already extends `RuntimeException` so `RuntimeException` wraps look redundant. 2- Fixes variable name 3- Fixes redundant line breaks ### Test 1- Adding a new UT for `SchemaSerializationException` case on `JSONSchema.decode()` function
-
- 15 12月, 2018 1 次提交
-
-
由 Boyang Jerry Peng 提交于
* remove pulsar-client-schema * shade avro dependencies in pulsar-client-kafka-shaded * refactor api to separate module * adding schema back * cleaning up * fix license issue * fixing licenses * cleaning up * adding back lombok * cleaning up
-
- 05 9月, 2018 1 次提交
-
-
由 Sijie Guo 提交于
### Motivation The `GenericSchema` and `GenericRecord` are used for schema auto detection. Currently it only supports Avro. This PR is to support json. ### Changes Introduce generic schema and generic record for json schema
-