kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hans Jespersen <h...@confluent.io>
Subject Re: Data in kafka topic in Json format
Date Sat, 03 Jun 2017 00:59:06 GMT
You have shared the Kafka connect properties but not the source connector config.
Which source connector are you using? Does it override the default settings you provided?
Are you running the connector in standalone mode or distributed mode?
Also what are you using to consume the messages and see the message format?

-hans



> On Jun 2, 2017, at 9:10 AM, Mina Aslani <aslanimina@gmail.com> wrote:
> 
> Hi Hans,
> 
> Thank you for your quick response, appreciate it.
> 
> In *kafka-connect* docker, I see below settings in
> *kafka-connect.properties* file in *kafka-connect *directory:
> 
> key.converter.schemas.enable=false
> key.converter.schema.registry.url=http://kafka-schema-registry:<PORT>
> value.converter.schema.registry.url=http://kafka-schema-registry:<PORT>
> 
> key.converter=org.apache.kafka.connect.json.JsonConverter
> value.converter=org.apache.kafka.connect.json.JsonConverter
> 
> internal.value.converter.schemas.enable=false
> rest.advertised.host.name=kafka-connect
> internal.value.converter=org.apache.kafka.connect.json.JsonConverter
> internal.key.converter=org.apache.kafka.connect.json.JsonConverter
> And the settings in *schema-registry *directory of *kafka-connect *docker
> are as
> 
> https://github.com/confluentinc/schema-registry/tree/master/config
> 
> Should I consider any other settings for *kafka-connect* or
> *schema-registry* to get the real json object NOT string
> formatted/stringified json which has extra "\"  and is not json any more?
> 
> Best regards,
> Mina
> 
> On Fri, Jun 2, 2017 at 11:18 AM, Hans Jespersen <hans@confluent.io> wrote:
> 
>> 
>> My earlier comment still applies but in Kafka Connect the equivalent of a
>> serializer/deserializer (serdes) is called a “converter”.
>> Check which converter you have configured for your source connector and if
>> it is overriding whatever the default converter is configured for the
>> connect worker it is running in.
>> 
>> -hans
>> 
>> 
>> 
>> 
>>> On Jun 2, 2017, at 8:12 AM, Mina Aslani <aslanimina@gmail.com> wrote:
>>> 
>>> Hi,
>>> 
>>> I would like to add that I use kafka-connect and schema-registery
>> version `
>>> 3.2.1-6`.
>>> 
>>> Best regards,
>>> Mina
>>> 
>>> On Fri, Jun 2, 2017 at 10:59 AM, Mina Aslani <aslanimina@gmail.com>
>> wrote:
>>> 
>>>> Hi.
>>>> 
>>>> Is there any way that I get the data into a Kafka topic in Json format?
>>>> The source that I ingest the data from have the data in Json format,
>>>> however when I look that data in the kafka topic, schema and payload
>> fields
>>>> are added and data is not in json format.
>>>> 
>>>> I want to avoid implementing a transformer to get the data from the
>> topic
>>>> and publishes Json in another one.
>>>> 
>>>> Your input is appreciated.
>>>> 
>>>> Best regards,
>>>> Mina
>>>> 
>> 
>> 


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message