spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Somogyi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-26783) Kafka parameter documentation doesn't match with the reality (upper/lowercase)
Date Fri, 01 Feb 2019 14:22:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-26783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16758348#comment-16758348
] 

Gabor Somogyi commented on SPARK-26783:
---------------------------------------

[~zsxwing] [~kabhwan] The more I'm playing with the things the more I think there could be
different issues involved (not sure have effect on each other).

1. "failOnDataLoss": I'll ask the reporter on SPARK-23685 because not yet able to repro. Let's
see whether the code or the doc has to be updated.
2. Generic data source implementation issue. Namely API doesn't guarantee lowercase params
but the user code is depending on that. For example [this|https://github.com/apache/spark/blob/aea5f506463c19fac97547ba7a28f9dd491e3a6a/external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaSourceProvider.scala#L66]
but there are other places.
Not sure it has anything to do with the first but could cause potentially such issues.


> Kafka parameter documentation doesn't match with the reality (upper/lowercase)
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-26783
>                 URL: https://issues.apache.org/jira/browse/SPARK-26783
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 3.0.0
>            Reporter: Gabor Somogyi
>            Priority: Minor
>
> A good example for this is "failOnDataLoss" which is reported in SPARK-23685. I've just
checked and there are several other parameters which suffer from the same issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message