beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Kirpichov (JIRA)" <>
Subject [jira] [Created] (BEAM-2221) Make KafkaIO coder specification less awkward
Date Mon, 08 May 2017 17:59:04 GMT
Eugene Kirpichov created BEAM-2221:

             Summary: Make KafkaIO coder specification less awkward
                 Key: BEAM-2221
             Project: Beam
          Issue Type: Improvement
          Components: sdk-java-extensions
            Reporter: Eugene Kirpichov
            Assignee: Raghu Angadi

readWithCoders and writeWithCoders functions are awkward because they don't emphasize enough
that coders are a poor choice for interpreting wire format.

The only reason to specify coders in KafkaIO is when coder inference from Deserializer fails.
To emphasize that, let's change the API to be withKeyDeserializer(Deserializer) as the default
choice and withKeyDeserializerAndCoder(Deserializer,Coder) if inference fails; likewise for

Remove functions using coders to interpret wire format from the API. A common case of that
is Avro and Proto - for that, introduce special helper functions, I guess like with{Avro,Proto}{Key,Value}(...),
which under the hood may be allowed to reuse Avro/ProtoCoder as a utility, but do not expose
this fact.

This message was sent by Atlassian JIRA

View raw message