spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Wendell (JIRA)" <>
Subject [jira] [Commented] (SPARK-4939) Python updateStateByKey example hang in local mode
Date Wed, 21 Jan 2015 20:49:35 GMT


Patrick Wendell commented on SPARK-4939:

[~tdas] [~davies] [~kayousterhout] Because there is still discussion about this, and this
is modifying a very complex component in Spark, I'm not going to block on this for 1.2.1.
Once we merge a patch we can decide whether to put it into 1.2 based on what the final patch
looks like. It is definitely inconvenient that this doesn't work in local mode, but much less
of a problem than introducing a bug in the scheduler for production cluster workloads.

As a workaround we could suggest running this example with local-cluster.

> Python updateStateByKey example hang in local mode
> --------------------------------------------------
>                 Key: SPARK-4939
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: PySpark, Spark Core, Streaming
>    Affects Versions: 1.2.0, 1.3.0
>            Reporter: Davies Liu
>            Assignee: Davies Liu
>            Priority: Blocker

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message