flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3617) NPE from CaseClassSerializer when dealing with null Option field
Date Fri, 04 Nov 2016 09:26:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15635776#comment-15635776
] 

Fabian Hueske commented on FLINK-3617:
--------------------------------------

I agree, {{Option}}'s purpose is to avoid {{null}} values and usually, the composite serializer
(in this case CaseClassSerializer) should handle {{null}} fields.
However, there are also atomic serializers that encode null themselves (like StringSerializer).

So the question is: do we actually want to fix this issue by adapting the OptionSerializer
(as proposed in PR #2723) or close it as Won't Fix (because we do not want to change CaseClassSerializer)
or Not an Issue (because Option should not be null anyways)?

IMO, we could fix it with the changed serialization format of the OptionSerializer (see PR
#2723).

Opinions [~StephanEwen] [~jgrier] [~chermenin]?


> NPE from CaseClassSerializer when dealing with null Option field
> ----------------------------------------------------------------
>
>                 Key: FLINK-3617
>                 URL: https://issues.apache.org/jira/browse/FLINK-3617
>             Project: Flink
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.0.0
>            Reporter: Jamie Grier
>
> This error occurs when serializing a Scala case class with an field of Option[] type
where the value is not Some or None, but null.
> If this is not supported we should have a good error message.
> java.lang.RuntimeException: ConsumerThread threw an exception: null
> 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09.run(FlinkKafkaConsumer09.java:336)
> 	at org.apache.flink.streaming.api.operators.StreamSource.run(StreamSource.java:78)
> 	at org.apache.flink.streaming.runtime.tasks.SourceStreamTask.run(SourceStreamTask.java:56)
> 	at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:224)
> 	at org.apache.flink.runtime.taskmanager.Task.run(Task.java:559)
> 	at java.lang.Thread.run(Thread.java:745)
> Caused by: java.lang.RuntimeException
> 	at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:81)
> 	at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:39)
> 	at org.apache.flink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:158)
> 	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer09$ConsumerThread.run(FlinkKafkaConsumer09.java:473)
> Caused by: java.lang.NullPointerException
> 	at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:100)
> 	at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:30)
> 	at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:100)
> 	at org.apache.flink.api.scala.typeutils.CaseClassSerializer.serialize(CaseClassSerializer.scala:30)
> 	at org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.serialize(StreamRecordSerializer.java:107)
> 	at org.apache.flink.streaming.runtime.streamrecord.StreamRecordSerializer.serialize(StreamRecordSerializer.java:42)
> 	at org.apache.flink.runtime.plugable.SerializationDelegate.write(SerializationDelegate.java:56)
> 	at org.apache.flink.runtime.io.network.api.serialization.SpanningRecordSerializer.addRecord(SpanningRecordSerializer.java:79)
> 	at org.apache.flink.runtime.io.network.api.writer.RecordWriter.emit(RecordWriter.java:84)
> 	at org.apache.flink.streaming.runtime.io.StreamRecordWriter.emit(StreamRecordWriter.java:86)
> 	at org.apache.flink.streaming.runtime.io.RecordWriterOutput.collect(RecordWriterOutput.java:78)
> 	... 3 more



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message