spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jungtaek Lim (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (SPARK-26783) Kafka parameter documentation doesn't match with the reality (upper/lowercase)
Date Fri, 01 Feb 2019 01:00:00 GMT

    [ https://issues.apache.org/jira/browse/SPARK-26783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16757842#comment-16757842
] 

Jungtaek Lim commented on SPARK-26783:
--------------------------------------

I'm not sure about what [~sindiri] left a comment on the PR:

{quote}
Originally this pr was created as "failOnDataLoss" doesn't have any impact when set in structured
streaming. But found out that ,the variable that needs to be used is "failondataloss" (all
in lower case).
{quote}

I just played with test "failOnDataLoss=false should not return duplicated records: v1" in
KafkaDontFailOnDataLossSuite, and looks like it works as intended (as case-insensitive manner).

"failOnDataLoss" -> "false" // passed
"failondataloss" -> "false" // passed
"FAILONDATALOSS" -> "false" // passed
<default value> // failed
"failOnDataLoss" -> "true" // failed

> Kafka parameter documentation doesn't match with the reality (upper/lowercase)
> ------------------------------------------------------------------------------
>
>                 Key: SPARK-26783
>                 URL: https://issues.apache.org/jira/browse/SPARK-26783
>             Project: Spark
>          Issue Type: Bug
>          Components: Structured Streaming
>    Affects Versions: 3.0.0
>            Reporter: Gabor Somogyi
>            Priority: Minor
>
> A good example for this is "failOnDataLoss" which is reported in SPARK-23685. I've just
checked and there are several other parameters which suffer from the same issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message