spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gabor Somogyi (JIRA)" <j...@apache.org>
Subject [jira] [Created] (SPARK-28367) Kafka connector infinite wait because metadata never updated
Date Fri, 12 Jul 2019 12:03:00 GMT
Gabor Somogyi created SPARK-28367:
-------------------------------------

             Summary: Kafka connector infinite wait because metadata never updated
                 Key: SPARK-28367
                 URL: https://issues.apache.org/jira/browse/SPARK-28367
             Project: Spark
          Issue Type: Bug
          Components: Structured Streaming
    Affects Versions: 3.0.0
            Reporter: Gabor Somogyi


Spark uses an old and deprecated API named poll(long) which never returns and stays in live
lock if metadata is not updated (for instance when broker disappears at consumer creation).

I've created a small standalone application to test it and the alternatives: https://github.com/gaborgsomogyi/kafka-get-assignment




--
This message was sent by Atlassian JIRA
(v7.6.14#76016)

---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscribe@spark.apache.org
For additional commands, e-mail: issues-help@spark.apache.org


Mime
View raw message